Hi,
Welcome to the new e1000 guys. Here is our current ppc64+2.6+e1000 hitlist
(apologies to those who have received this before):
- We can wrap around the tx ring and start overwriting valid entries. On
ppc64 that means we leak IOMMU entries and eventually run out of them.
Olof Johansson debugged this problem and provided the attached fix
(e1000_tx_wraparound).
- TSO HW workaround doesnt seem to be working. In TSO mode, some e1000s
report DMA complete too early. There is a workaround in the current
driver:
#ifdef NETIF_F_TSO
/* Workaround for premature desc write-backs
* in TSO mode. Append 4-byte sentinel desc */
if(mss && f == (nr_frags-1) && size == len &&
size > 8)
size -= 4;
#endif
However we were still getting card lockups so Jeremy put together a
patch based on our original workaround when we first found the bug
(e1000_tso_hack). Basically it keeps a one entry cache of skbs and pci
mappings and always frees the second to last one. Yes its a hack and
it will leak skbs and IOMMU entries but with it applied we no longer
see any problems.
BTW I wonder if this is the problem Scott mentioned in his checkin
comment to disable TSO by default on e1000 2.6.
- e1000 tx spinlock is extremely painful.
Heres a web benchmark run with 2.6 BK (numbers are in %):
9.7149 e1000_xmit_frame
4.2336 tcp_v4_rcv
3.9495 skb_release_data
3.8244 qdisc_restart
3.5437 iommu_free
Heres the profile after I backed the tx spinlock patch out (attached):
6.4075 pci_iommu_map_single
4.3752 tcp_v4_rcv
4.1046 skb_release_data
3.8190 iommu_free
3.7119 default_idle
3.6749 qdisc_restart
3.4538 iommu_alloc
2.0153 do_tcp_sendpages
1.8828 __d_lookup
1.7772 e1000_clean_tx_irq
1.7654 atomic_dec_and_lock
1.7354 e1000_xmit_frame
And performance improved. Olof had suggestions for how to work around
this problem without incurring the cost of a spinlock.
Anton
e1000_tx_wraparound
Description: Text document
e1000_tso_hack
Description: Text document
e1000_tx_spinlock
Description: Text document
|