On Sunday 16 January 2005 06:32 am, Robert Olsson wrote:
> Jeremy M. Guthrie writes:
> > I actually upped the buffer count to 8192 buffers instead of 10k.
> > Of the 74 samples I have thus far, 57 have been clean of errors.
> > Most of the sample errors appear to be shortly after the cache flush.
>
> I don't really believe in increasing RX buffers to this extent. We
> verified that you have CPU available and the drops occur when the timer
> based GC happens. Increasing buffers decreases overall performance and adds
> jitter.
I just took a look at my logs, the increase to the max on the card of 4K RX
buffers has stopped packet drops except during GC. I agree, it isn't pretty
and it is not the solution I would like. In the mean time, it has at least
stopped the 0.3% round-the-clock packet loss which I was seeing even at rates
as low as 25K pps.
> We saw also the timed based GC were taking the dst-entries from about
> 600k to 40k in one shot. I think this what we should look into. Just
> GC is "work" also after GC a lot flows has to be recreated doing fib
> lookup and creating new entries. We want to smoothen the GC process so
> happen more frequent and does less work.
Agreed. I went to the extreme because I can really see the % idle CPU. If I
am constantly setting up new flows then the % of free CPU shoots way down.
Minus the effects of GC, a larger flow table equates into free CPU.
> Some time ago an "in-flow" GC (as opposed to timer based) was added to
> the routing code look for cand in route.c. In setup like yours (and ours)
> it would be better to relay on this process to a higher extent. Anyway
> in /proc/sys/net/ipv4/route/ you have the files.
> gc_elasticity, gc_interval, gc_thresh etc I would avoid gc_min_interval.
> And you can play with your running system and for drops without causing
> your users to much pain.
> We save the patch for routing without route hash and GC until later,
Okay. I will bring my interval down from an hour down to ten minutes and do
further tuning.
--
--------------------------------------------------
Jeremy M. Guthrie jeremy.guthrie@xxxxxxxxxx
Senior Network Engineer Phone: 608-298-1061
Berbee Fax: 608-288-3007
5520 Research Park Drive NOC: 608-298-1102
Madison, WI 53711
pgpC0U5AtdfKl.pgp
Description: PGP signature
|