Hi all,
What are the best values for RxIntDelay, RxDescriptors, TxIntDelay and
TxDescriptors for e1000 driver to get maximum connection rate. My tests on a
dual CPU box running RHEL 3.0 SMP kernel (2.4.21-4ELsmp) are causing the
network interface to lock down (with irqbalance process killed). With
irqbalance process running the system locks down.
I have set txqueuelen to 20000, killed irqbalance process and setup
smp_affinity so that CPU1 (on dual PIII box) handles all interrupts
generated by this device. When pounding the box with about 10000 HTTP
conn/sec I see CPU1 utilized about 60% whereas CPU0 about 10% (BTW I was
using a trivial filesize of 32 bytes ... don't ask why :-) After about 5
minutes I see the connection rate drop to 0 and ifconfig shows that the NIC
is dropping packets. The following is the output of ifconfig
RX packets:13468106 errors:2206 dropped:2206 overruns:1558 frame:0
TX packets:13466855 errors:0 dropped:0 overruns:0 carrier:4
collisions:0 txqueuelen:20000
RX bytes:1277599772 (1218.4 Mb) TX bytes:1277420502 (1218.2 Mb)
Interrupt:24 Base address:0xc8e0 Memory:fe920000-fe940000
I set RxDescriptors to 4096 and RxIntDelay to 0 (BTW I see this even with
RxIntDelay of 64). I also set the following:
echo 300000 > /proc/sys/net/core/hot_list_length
echo 300000 > /proc/sys/net/core/netdev_max_backlog
The rest of the TCP tunables were obtained from the SpecWeb results pages.
http://www.spec.org/osg/web99/results/res2003q3/web99-20030818-00245.html
Also, what is the "idle=poll" setting suggested on the Specweb results page
mean? I have not set this in my tests. Will it make any difference? If so
what and why?
I'll really appreciate it if someone can help me choose the correct values
to tune the kernel, driver and card to stop the interface/system from
locking down. Another thing, when the interface locks down (i.e., no network
activity on it) I see the following CPU utilization:
CPU states: cpu user nice system irq softirq iowait
idle
total 0.0% 0.0% 0.5% 0.0% 50.0% 0.0% 49.5%
cpu00 0.0% 0.0% 1.0% 0.0% 0.0% 0.0% 99.0%
cpu01 0.0% 0.0% 0.0% 0.0% 100.0% 0.0% 0.0%
It appears as if the CPU is busy processing the ksoftirqd/1 process. Here is
the output of stat for this process:
% cat /proc/6/stat
6 (ksoftirqd/1) S 1 1 1 0 -1 64 0 0 0 0 0 0 0 0 34 19 0 0 125 0 0 4294967295
0 0 0 0 0 0 2147483647 0 0 3222466767 0 0 17 1 0 0 0 0 0 0
And if it is of any helf the output of softnet_stat
% cat /proc/net/softnet_stat
000c9732 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000003
00cd9256 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000004
Any help will be greatly appreciated.
tx
Yedok
_________________________________________________________________
Find and compare great deals on Broadband access at the MSN High-Speed
Marketplace. http://click.atdmt.com/AVE/go/onm00200360ave/direct/01/
|