Probably page size. 4k is one page so those are probably the most efficient IOs.
There must be some additional handling required to squeeze multiple pages into
an MTU. Have you profiled things at all to see what additional code has to run
in order to handle multiple pages?
Abhijit Karmarkar wrote:
i have observed that using jumbo frames (mtu=9000) decreases the thruput
(i am timing one-way ttcp). trying w/ different mtu's i see 4096 give
me the best numbers:
1500 (default) ~846Mbps
4096 ~930Mbps <== highest
my setup is:
- 2 nodes connected directly (cross-over cable)
- each node: 2-way, 2.4G Xeon. 4G RAM., running RHEL3 (2.4.21-4.ELsmp)
- intel gige (82543GC), e1000 ver. (5.1.11-k1)
i think the cards are: 64bit/66Mhz PCI.
- ipv4.tcp_r/wmem and core.r/wmem_max set sufficiently high (512KB)
- using ttcp to xfer ~8GB one-way.
why doesn't my thruput increase with increase in MTU? is it because of
small number of rx/txdescriptors on 82543GC (max=256?) or something
are there any driver parameters that i can tune to get better numbers
with larger MTUs?
"Give a man a fish, and he will eat for a day, hit him with a fish and
he leaves you alone" - me