netdev
[Top] [All Lists]

Re: [PATCH + RFC] neighbour/ARP cache scalability

To: "David S. Miller" <davem@xxxxxxxxxxxxx>
Subject: Re: [PATCH + RFC] neighbour/ARP cache scalability
From: Harald Welte <laforge@xxxxxxxxxxxx>
Date: Wed, 22 Sep 2004 13:14:47 +0200
Cc: netdev@xxxxxxxxxxx
In-reply-to: <20040921203118.734a0a7e.davem@xxxxxxxxxxxxx>
References: <20040920225140.GH1307@xxxxxxxxxxxxxxxxxxxxxxx> <20040921203118.734a0a7e.davem@xxxxxxxxxxxxx>
Sender: netdev-bounce@xxxxxxxxxxx
User-agent: Mutt/1.5.6+20040818i
On Tue, Sep 21, 2004 at 08:31:18PM -0700, David S. Miller wrote:
 
> Ok Harald, I did some snooping around and here is what
> I've come up with.

Thanks for digging into this issue.

> 1) We have 5 or 6 implementations of "walk neighbour hash
>    table in sequence", that's rediculious and is the only
>    reason that hashtable layout or number of entries is
>    even known by code outside of net/core/neighbour.c
> [...]
> 1.5) tbl->ops->hash() should return the raw hash, the caller
>      will do the necessary masking.
> [...]
> 
> 2) We should size these hash table dynamically, growing them
>    at neigh_create() time.

great, that sounds like a scalable approach.  I'm looking forward to
reading through the code :)

> 3) If we have a sysctl for this stuff at all, it will be for
>    the limit of the hash table growth, but I don't think that
>    is necessary given #2

But we will still have the existing sysctls for the garbage collector, I
guess (like gc_thresh*).  

And I still want to address the "all complete entries flushed due to
lots of bogus incomplete entires" issue somehow.  As stated before, I
have seen this on happen on live systems.  

Do you agree that this is an existing problem?

I think just having a certain 'reserve' for complete entries (or a let's
say 80% limit for incomplete ones) should address this issue.

btw: I guess you're implementing this as 2.6.x only patch.  Once you're
finished, I'll have to do a 2.4.x backport anyway [for business reasons
*g*].  So don't bother doing that, I'll post the patch here (which
doesn't imply that I want to have it merged mainline).

-- 
- Harald Welte <laforge@xxxxxxxxxxxx>               http://www.gnumonks.org/
============================================================================
Programming is like sex: One mistake and you have to support it your lifetime

Attachment: signature.asc
Description: Digital signature

<Prev in Thread] Current Thread [Next in Thread>