netdev
[Top] [All Lists]

e1000 issues

To: netdev@xxxxxxxxxxx
Subject: e1000 issues
From: Anton Blanchard <anton@xxxxxxxxx>
Date: Sun, 28 Mar 2004 11:10:52 +1000
Cc: cramerj@xxxxxxxxx, john.ronciak@xxxxxxxxx, Ganesh.Venkatesan@xxxxxxxxx, jk@xxxxxxxxxx, olof@xxxxxxxxxxxxxx
Sender: netdev-bounce@xxxxxxxxxxx
User-agent: Mutt/1.5.5.1+cvs20040105i
Hi,

Welcome to the new e1000 guys. Here is our current ppc64+2.6+e1000 hitlist
(apologies to those who have received this before):

- We can wrap around the tx ring and start overwriting valid entries. On
  ppc64 that means we leak IOMMU entries and eventually run out of them.
  Olof Johansson debugged this problem and provided the attached fix
  (e1000_tx_wraparound).

- TSO HW workaround doesnt seem to be working. In TSO mode, some e1000s
  report DMA complete too early. There is a workaround in the current
  driver:

#ifdef NETIF_F_TSO
                        /* Workaround for premature desc write-backs
                         * in TSO mode.  Append 4-byte sentinel desc */
                        if(mss && f == (nr_frags-1) && size == len &&
size > 8)
                                size -= 4;
#endif

  However we were still getting card lockups so Jeremy put together a
  patch based on our original workaround when we first found the bug
  (e1000_tso_hack). Basically it keeps a one entry cache of skbs and pci
  mappings and always frees the second to last one. Yes its a hack and
  it will leak skbs and IOMMU entries but with it applied we no longer
  see any problems.

  BTW I wonder if this is the problem Scott mentioned in his checkin
  comment to disable TSO by default on e1000 2.6.

- e1000 tx spinlock is extremely painful.

Heres a web benchmark run with 2.6 BK (numbers are in %):

9.7149  e1000_xmit_frame
4.2336  tcp_v4_rcv
3.9495  skb_release_data
3.8244  qdisc_restart
3.5437  iommu_free

Heres the profile after I backed the tx spinlock patch out (attached):

6.4075  pci_iommu_map_single
4.3752  tcp_v4_rcv
4.1046  skb_release_data
3.8190  iommu_free
3.7119  default_idle
3.6749  qdisc_restart
3.4538  iommu_alloc
2.0153  do_tcp_sendpages
1.8828  __d_lookup
1.7772  e1000_clean_tx_irq
1.7654  atomic_dec_and_lock
1.7354  e1000_xmit_frame

And performance improved. Olof had suggestions for how to work around
this problem without incurring the cost of a spinlock.

Anton

Attachment: e1000_tx_wraparound
Description: Text document

Attachment: e1000_tso_hack
Description: Text document

Attachment: e1000_tx_spinlock
Description: Text document

<Prev in Thread] Current Thread [Next in Thread>
  • e1000 issues, Anton Blanchard <=