David S. Miller a écrit :
Ok, now that we have the tagged status stuff sorted I began
to work on putting the hw mitigation bits back into the
driver. The discussion on the DMA rw-ctrl settings is still
ongoing, but I will get back to it shortly.
This is the first step, we cache the settings in the tg3
struct and put those values into the chip via tg3_set_coalesce().
ETHTOOL_GCOALESCE is supported, setting is not.
Hi David
I am using 2.6.12 now, but still experiment a high number of interrupts per
second on
my tg3 NIC, on an dual Opteron based machine.
(about 7300 interrupts per second generated by one interface eth0, 100Mbit/s
link)
Is there anything I can try to tune the coalescing ?
Being able to handle 100 packets each interrupt instead of one or two would
certainly help.
I dont mind about latency. But of course I would like not to drop packets :)
But maybe the BCM5702 is not able to delay an interrupt ?
Thank you
Eric Dumazet
----------------------------------------------------------------------------------------
# lspci -v
02:03.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5702 Gigabit
Ethernet (rev 02)
Subsystem: Broadcom Corporation BCM5702 1000Base-T
Flags: bus master, 66Mhz, medium devsel, latency 64, IRQ 27
Memory at 00000000fe000000 (64-bit, non-prefetchable) [size=64K]
Expansion ROM at <unassigned> [disabled] [size=64K]
Capabilities: [40] PCI-X non-bridge device.
Capabilities: [48] Power Management version 2
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3
Enable-
# ethtool -c eth0
Coalesce parameters for eth0:
Adaptive RX: off TX: off
stats-block-usecs: 1000000
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
rx-usecs: 20
rx-frames: 5
rx-usecs-irq: 20
rx-frames-irq: 5
tx-usecs: 72
tx-frames: 53
tx-usecs-irq: 20
tx-frames-irq: 5
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
# ethtool -S eth0
NIC statistics:
rx_octets: 104634072366
rx_fragments: 0
rx_ucast_packets: 852685070
rx_mcast_packets: 0
rx_bcast_packets: 20429
rx_fcs_errors: 0
rx_align_errors: 0
rx_xon_pause_rcvd: 0
rx_xoff_pause_rcvd: 0
rx_mac_ctrl_rcvd: 0
rx_xoff_entered: 0
rx_frame_too_long_errors: 0
rx_jabbers: 0
rx_undersize_packets: 0
rx_in_length_errors: 0
rx_out_length_errors: 0
rx_64_or_less_octet_packets: 451956709
rx_65_to_127_octet_packets: 272058231
rx_128_to_255_octet_packets: 63364655
rx_256_to_511_octet_packets: 35814973
rx_512_to_1023_octet_packets: 11867701
rx_1024_to_1522_octet_packets: 17643210
rx_1523_to_2047_octet_packets: 0
rx_2048_to_4095_octet_packets: 0
rx_4096_to_8191_octet_packets: 0
rx_8192_to_9022_octet_packets: 0
tx_octets: 134640205605
tx_collisions: 0
tx_xon_sent: 0
tx_xoff_sent: 0
tx_flow_control: 0
tx_mac_errors: 0
tx_single_collisions: 0
tx_mult_collisions: 0
tx_deferred: 0
tx_excessive_collisions: 0
tx_late_collisions: 0
tx_collide_2times: 0
tx_collide_3times: 0
tx_collide_4times: 0
tx_collide_5times: 0
tx_collide_6times: 0
tx_collide_7times: 0
tx_collide_8times: 0
tx_collide_9times: 0
tx_collide_10times: 0
tx_collide_11times: 0
tx_collide_12times: 0
tx_collide_13times: 0
tx_collide_14times: 0
tx_collide_15times: 0
tx_ucast_packets: 774312055
tx_mcast_packets: 13
tx_bcast_packets: 246
tx_carrier_sense_errors: 0
tx_discards: 0
tx_errors: 0
dma_writeq_full: 21375
dma_write_prioq_full: 0
rxbds_empty: 0
rx_discards: 2644
rx_errors: 0
rx_threshold_hit: 57384403
dma_readq_full: 27100189
dma_read_prioq_full: 1557267
tx_comp_queue_full: 35712755
ring_set_send_prod_index: 747986769
ring_status_update: 502110997
nic_irqs: 446148615
nic_avoided_irqs: 55962382
nic_tx_threshold_hit: 37282069
# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX: 511
RX Mini: 0
RX Jumbo: 255
TX: 0
Current hardware settings:
RX: 200
RX Mini: 0
RX Jumbo: 100
TX: 511
# ethtool eth0
Settings for eth0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
Link detected: yes
# cat /proc/interrupts (HZ=200)
CPU0 CPU1
0: 164055 14038348 IO-APIC-edge timer
2: 0 0 XT-PIC cascade
8: 0 0 IO-APIC-edge rtc
14: 4073 368224 IO-APIC-edge ide0
15: 0 20 IO-APIC-edge ide1
27: 35985951 421578656 IO-APIC-level eth0, eth1
NMI: 874625217 905019517 (oprofile running)
LOC: 14201857 14201976
ERR: 0
MIS: 0
oprofile data :
# more /tmp/vmlinux.oprofile
CPU: Hammer, speed 2205.08 MHz (estimated)
Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask
of 0x00 (No unit mask) count 100000
samples cum. samples % cum. % symbol name
20208503 20208503 7.7982 7.7982 ipt_do_table
8336463 28544966 3.2169 11.0151 tcp_v4_rcv
7746814 36291780 2.9894 14.0045 handle_IRQ_event
7117968 43409748 2.7467 16.7512 tg3_poll
6585377 49995125 2.5412 19.2924 memcpy
5184695 55179820 2.0007 21.2931 ip_route_input
4346890 59526710 1.6774 22.9705 kfree
4214007 63740717 1.6261 24.5967 copy_user_generic_c
4093885 67834602 1.5798 26.1764 tcp_ack
4006753 71841355 1.5462 27.7226 tg3_interrupt_tagged
3778976 75620331 1.4583 29.1809 tcp_rcv_established
3756498 79376829 1.4496 30.6304 ip_queue_xmit
3418999 82795828 1.3193 31.9498 schedule
3274459 86070287 1.2636 33.2134 try_to_wake_up
3034809 89105096 1.1711 34.3844 tcp_sendmsg
2846436 91951532 1.0984 35.4828 kmem_cache_alloc
2745147 94696679 1.0593 36.5422 free_block
2679056 97375735 1.0338 37.5760 kmem_cache_free
2595289 99971024 1.0015 38.5775 fn_hash_lookup
2582072 102553096 0.9964 39.5738 __memset
2576462 105129558 0.9942 40.5681 tcp_transmit_skb
2528313 107657871 0.9756 41.5437 tcp_recvmsg
2392370 110050241 0.9232 42.4669 timer_interrupt
2365615 112415856 0.9129 43.3797 system_call
2358666 114774522 0.9102 44.2899 sockfd_lookup
2357192 117131714 0.9096 45.1995 tcp_poll
2340568 119472282 0.9032 46.1027 ip_rcv
2315805 121788087 0.8936 46.9964 tcp_match
2276212 124064299 0.8784 47.8747 sys_epoll_wait
2260913 126325212 0.8725 48.7472 __mod_timer
2173905 128499117 0.8389 49.5861 tg3_start_xmit
2057738 130556855 0.7941 50.3801 __switch_to
2022435 132579290 0.7804 51.1605 ep_poll_callback
2020449 134599739 0.7797 51.9402 sock_wfree
1913008 136512747 0.7382 52.6784 find_busiest_group
1891578 138404325 0.7299 53.4083 local_bh_enable
1860130 140264455 0.7178 54.1261 ip_local_deliver
1793639 142058094 0.6921 54.8183 __ip_route_output_key
1789287 143847381 0.6905 55.5087 alloc_skb
1770972 145618353 0.6834 56.1921 tcp_write_timer
1727286 147345639 0.6665 56.8587 __wake_up
1634111 148979750 0.6306 57.4893 skb_release_data
1625157 150604907 0.6271 58.1164 __kmalloc
1567198 152172105 0.6048 58.7211 tcp_v4_do_rcv
1562495 153734600 0.6029 59.3241 __kfree_skb
|