> The trick when you start crawling up to a significant portion of your
> memory bandwidth with I/O traffic is to remember that application code
> which used to take 5% CPU to execute can start taking a LOT MORE CPU
> due to memory contention. 80 MB/s of I/O traffic is going to have
> substantial additional main memory consumption overhead. Whether
> it'll really interfere depends upon the design of your box.
This is from the box in the middle doing reads and writes of
256k. I hacked up the test setup from stuff I found lying around
(including an old Tigon-I, since I didn't have enough Tigon-II's)
With jumbo frames
# description host sample_KB total_MB sample_KB/s avge_KB/s
cpu_sec user_sec sys_sec sec/MB cpu_pct
1 sink toy3 209715.203 2306.867 45367.223 45554.204
0.130 0.010 0.120 0.001 3
1 sink toy3 209715.203 2516.583 45540.930 45551.712
0.160 0.000 0.160 0.001 3
1 sink toy3 209715.203 2726.298 45582.977 45545.238
0.120 0.000 0.120 0.001 3
1 sink toy3 209715.203 2936.013 45600.402 45551.686
0.200 0.010 0.190 0.001 4
1 sink toy3 209715.203 3145.728 45577.043 45562.543
0.190 0.010 0.180 0.001 4
Without them around 28M/s.
The machine does have two PCI buses (L440GX), but I have no idea how
they're actually wired. Also used a non-SMP kernel on the machines by
accident as I just noticed trying to figure out if binding the NIC's
to different CPU's would make any difference.
>
> > We are looking at using the Netgear GA620 network card - is this the
> > best?
>
> You must use a card which has ST acceleration. Currently the Alteon,
> is the only GigE card that has it. Ask Pekka for the particulars.
Anything based on the Alteon should be fine. The Netgear only
has 512k of memory, which seems to be a bit tight with the STP
acceleration included, so cards with 1MB like 3c985B might be a
better choice. I'll definately try to get it working well with 512k boards
too, especially since the Netgears seem to be quite a bit cheaper...
> with 1500 byte ethernet frames. Somebody in England (I seem to have
> lost the reference) did custom Alteon firmware for a similar project
> and got wire speed performance. There was some mention of trying to
> adapt that firmware to accelerating ST. Again, Pekka can probably
> tell you where the bodies are buried with that.
http://www.cl.cam.ac.uk/Research/SRG/netos/arsenic/
|