Several tg3 patches have been recently posted, and I've
collected a bit of performance data with them. Mostly
I'm concerned about reducing per-interrupt overhead
(due to PIOs), and CPU Utilization, so the following data
shows how these patches change the number of received
packets/interrupt and CPU utilization.
(Let me know if I've missed any relevant patches.)
The data is labelled as follows:
-------------------------------
base : "vanilla" 2.6.12-rc2 kernel
base+mchan1-3 : 2.6.12-rc2 kernel + 3 patches
from mchan [1]
base+mchan1-3+tagged: 2.6.12-rc2 + 3 patches
from mchan [1] + tagged status
patch from davem [2]
base+mchan1-3+coal : 2.6.12-rc2 + 3 patches
from mchan [1] + tg3 interrupt
coalescence patch [3]
[1] http://marc.theaimsgroup.com/?l=linux-netdev&m=111446723510962&w=2
(This is one of a series of 3 patches - the others can't be
found in the archive. But they're all in 2.6.12-rc4.)
[2] http://marc.theaimsgroup.com/?l=linux-netdev&m=111567944730302&w=2
[3] http://marc.theaimsgroup.com/?l=linux-netdev&m=111586526522981&w=2
The system I used was an Altix with 1300MHz CPUs, and
a 5704 Broadcom NIC. The workload was bulk data receive
via TCP, with 1500 a byte MTU.
The following tables summarize the data in the attached
graphs. I had to (grossly) interpolate in some cases,
see the graphs for the real data.
CPU Util[%]
-------------------------------------
base base+ base+ base+
mchan1-3 mchan1-3+ mchan1-3+
Link Util[%] tagged coal
=====================================================
40 36 34 41 27
60 48 45 50 35
80 58 56 58 40
90 59 58 63 42
95 - 57 - 42
Packets/Intr
-------------------------------------
base base+ base+ base+
mchan1-3 mchan1-3+ mchan1-3+
Link Util[%] tagged coal
=====================================================
40 2.2 2.3 1.4 3.4
60 2.7 2.9 1.8 4.1
80 3.0 3.2 2.3 5.2
90 3.1 3.4 2.4 6.2
95 - 3.5 - 6.6
"mchan1-3" gets us up to ~.5 more packets/interrupt,
and adding the "coal" patch ~3.5 more. The "tagged"
patch made things a bit worse, though I haven't pinned
down exactly why that is. Processing more packets per
interrupt results in lower CPU utilization, largely
because we spend less time waiting for PIOs to flush.
--
Arthur
cpu_vs_link.2.png
Description: CPU utilization
packets_vs_link.2.png
Description: Received Packets/Interrupt
|
