|Subject:||Re: TOE brain dump|
|From:||Jeff Garzik <jgarzik@xxxxxxxxx>|
|Date:||Sun, 03 Aug 2003 02:40:33 -0400|
|Cc:||Werner Almesberger <werner@xxxxxxxxxxxxxxx>, Nivedita Singhvi <niv@xxxxxxxxxx>|
|References:||<20030802140444.E5798@almesberger.net> <3F2BF5C7.firstname.lastname@example.org> <3F2C0C44.email@example.com> <20030802184901.G5798@almesberger.net>|
|User-agent:||Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021213 Debian/1.2.1-2.bunk|
Werner Almesberger wrote:
Jeff Garzik wrote:
Really fast, really long pipes in practice don't exist for 99.9% of all Internet users.
When you approach traffic levels that push you want to offload most of the TCP net stack, then TCP isn't the right solution for you anymore, all things considered.
The Linux net stack just isn't built to be offloaded. TOE engines will either need to (1) fall back to Linux software for all-but-the-common case (otherwise netfilter, etc. break), or, (2) will need to be hideously complex beasts themselves. And I can't see ASIC and firmware designers being excited about implementing netfilter on a PCI card :)
Unfortunately some vendors seem to choosing TOE option #3: TCP offload which introduces many limitations (connection limits, netfilter not supported, etc.) which Linux never had before. Vendors don't seem to realize TOE has real potential to damage the "good network neighbor" image the net stack has. The Linux net stack's behavior is known, documented, predictable. TOE changes all that.
There is one interesting TOE solution, that I have yet to see created: run Linux on an embedded processor, on the NIC. This stripped-down Linux kernel would perform all the header parsing, checksumming, etc. into the NIC's local RAM. The Linux OS driver interface becomes a virtual interface with a large MTU, that communicates from host CPU to NIC across the PCI bus using jumbo-ethernet-like data frames. Management frames would control the ethernet interface on the other side of the PCI bus "tunnel".
So, fix the other end of the pipeline too, otherwise this fast network stuff is flashly but pointless. If you want to serve up data from disk, then start creating PCI cards that have both Serial ATA and ethernet connectors on them :) Cut out the middleman of the host CPU and host memory bus instead of offloading portions of TCP that do not need to be offloaded.
|<Prev in Thread]||Current Thread||[Next in Thread>|