Thanks all for your suggestions.
Actually I have noticed that with 90k establish TCP sessions, I have around
amount of entries in the routing cache. For each TCP session there is an
inbound and outbond
cache entry like this:
22.214.171.124 126.96.36.199 188.8.131.52 1500 0 0
184.108.40.206 220.127.116.11 18.104.22.168 l 0 0 0 lo
This system has accepted that many sessions before when running 2.4 and this
with 2.6. Now, I can't be sure that traffic pattern are exactly the same so
Im not drawing conclusions
I will try to raise gc_tresh and keep you informed.
----- Original Message -----
From: "David S. Miller" <davem@xxxxxxxxxx>
To: "Robert Olsson" <Robert.Olsson@xxxxxxxxxxx>
Cc: <francois@xxxxxxxxxxxx>; <netdev@xxxxxxxxxxx>
Sent: Tuesday, December 02, 2003 12:26 PM
Subject: Re: 2.6.0-test11: dst_cache_overflow causing unresponsive box
> On Tue, 2 Dec 2003 11:44:31 +0100
> Robert Olsson <Robert.Olsson@xxxxxxxxxxx> wrote:
> > No experience with 90k TCP-flows but it seems GC is not able to free
> > the dst-entries for some reason. This will slowly kill your box with
> > symptoms you describe. We have ask TCP-experts for timer settings to
> > pending sessions etc. Also check slab for any other objects growing as
> > dst cache overflow is most likely secondary effect in your case. rtstat
> > looks sane expect for the high number of dst-entries. Tuning is another
> > story.
> Let us assume, for the sake of back of the envelope calculations, that
> all 90k TCP connections speak to unique destinations. Let us further
> assume that all of them have at least one packet in flight.
> This means the routing cache must be able to hold at least 90k entries.
> All of these routing cache entires will be referenced by the packets
> in the TCP retransmission queues of all the sockets, and thus the
> entries are unreclaimable.
> You are setting net.ipv4.route.max_size to 655360 which should be more
> than enough. But you also have to make the net.ipv4.route.gc_thresh
> more reasonable as well, perhaps 90K as a test.
> If net.ipv4.route.gc_thresh is lower than 90K and my assertions above
> hold, then the kernel will try to garbage collect too early, all the
> routing cache entries will be in use and therefore uncollectable,
> and you'll get the message you're seeing.
> Try to pump up gc_thresh and see if that helps.