On Sun, 17 Mar 2002, Andi Kleen wrote:
> On Fri, Mar 15, 2002 at 10:43:12PM -0500, Donald Becker wrote:
> > A final related note: drivers should always allocate consistently
> > sized skbuffs. Specifically, Ethernet drivers should always allocate
> > 1536 byte buffers. Using a single buffer size gives the skbuff
> > allocator the opportunity to efficiently recycle buffers rather than
> > having to allocate and free a mix of sizes.
> The current skbuff allocator uses kmalloc for the data part. kmalloc
> currently always rounds to a power of two. So for any
> 2K>=value+sizeof(struct skb_shared_info)>1K
> you will get a 2K buffer. The skbuff head recycling works independently
> of the data size.
I sure Andi noticed the wording, but for others that didn't:
I wrote: "the opportunity", which the current implementation doesn't
take advantage of. There have been prototype allocator implementations
that recycled skbuff data areas, and keeping the drivers implementations
consistent will make it easier for this type optimization to be done.
Memory allocators are important for performance, and must be frequently
re-tuned as systems evolved with new cache structures and memory use.
Keeping the job of the memory allocator simple is critical, as
allocators can have a surprisingly bad effect on performance. Not only
can they to have much worse performance than their memory access
count indicates because of cache effects, the resulting cache flushing
slows down unrelated code in ways that are difficult to measure.
A bit of history on packet size:
The selection of 1500 (1518) byte packets for Ethernet is somewhat
awkward for allocators. The size was selected back in the days when
3Mbps Ethernet was being designed. Memory was handled in 256 byte
units, and the primary considerations were access latency (helped by a
small packet size), network physical span and having a packet size large
enough to be handled efficiently.
The selection of 9KB for jumbo packets was set by the size of
inexpensive static RAM chips in the mid-80's. Seriously. That was
hardware constraint that caused the early Sun 68K machines to be
designed with a 8KB page size instead of a VAX-like 4KB page size. That
caused the de facto standard NFS block size to be 8KB, which was later
the motivation for selecting 9KB for jumbo packets over the other
Donald Becker becker@xxxxxxxxx
Scyld Computing Corporation http://www.scyld.com
410 Severn Ave. Suite 210 Second Generation Beowulf Clusters
Annapolis MD 21403 410-990-9993