If you don't trust us, please, ignore this email.
Sorry.
That's the number we have. And are actually very similar from what other
colleagues of us got.
The point is:
while a PCI-X linux or (or click) box can receive (receive just up to
the netif_receive_skb() level and then discard the skb) up to more than
wire speed using off-the-shelf gigabit ethernet hardware, there is no
way to transmit more than about half that speed. This is true
considering minimum sized ethernet frames.
This holds true with
- linux 2.4.x and 2.6.x and click-linux 2.4.x
- intel e1000 or broadcom drivers (modified to drop packets after the
netif_receive_skb())
- whichever driver version you like (with minor modifications).
The only modification to the driver we did consists in carefully
prefecting the data in the CPU internal cache.
Some details and results can be retreived from
http://www.tlc-networks.polito.it/~mellia/euroTLC.pdf
Part of this results are presented in this paper
A. Bianco, J.M. Finochietto, G. Galante, M. Mellia, F. Neri
Open-Source PC-Based Software Routers: a Viable Approach to High-Performance
Packet Switching
Third Internation Workshop on QoS in Multiservice IP Networks
Catania, Feb 2005
http://www.tlc-networks.polito.it/mellia/papers/Euro_qos_ip.pdf
Hope this helps.
> I'm forwarding this to netdev, as these are very interesting
> results (even if I don't beleive them).
>
> If you point us at the code/versions we will be better able to answer.
>
> Marco Mellia wrote:
> > We are trying to stress the e1000 hardware/driver under linux and Click
> > to see what is the maximum number of packets per second that can be
> > received/transmitted by a single NIC.
> >
> > We found something which is counterintuitive:
> >
> > - in reception, we can receive ALL the traffic, regardeless of the
> > packet size (or if you prefer, we can receive ALL the minimum sized
> > packet at gigabit speed)
>
> I questioned whether you actually did receive at that rate to
> which you responded:
>
> > - using Click, we can receive 100% of (small) packets at gigabit
> > speed with TWO cards (2gigabit/s ~ 2.8Mpps)
> > - using linux and standard e1000 driver, we can receive up to about
> > 80% of traffic from a single nic (~1.1Mpps)
> > - using linux and a modified (simplified) version of the driver, we
> > can receive 100% on a single nic, but not 100% using two nics (up
> > to ~1.5Mpps).
> >
> > Reception means: receiving the packet up to the rx ring at the
> > kernel level, and then IMMEDIATELY drop it (no packet processing,
> > no forwarding, nothing more...)
> >
> > Using NAPI or IRQ has littel impact (as we are not processing the
> > packets, the livelock due to the hardIRQ preemption versus the
> > softIRQ managers is not entered...)
> >
> > But the limit in TRANSMISSION seems to be 700Kpps. Regardless of
> > - the traffic generator,
> > - the driver version,
> > - the O.S. (linux/click),
> > - the hardware (broadcom card have the same limit).
>
> >
> > - in transmission we CAN ONLY trasmit about 700.000 pkt/s when the
> > minimum sized packets are considered (64bytes long ethernet minumum
> > frame size). That is about HALF the maximum number of pkt/s considering
> > a gigabit link.
> >
> > What is weird, is that if we artificially "preload" the NIC tx-fifo with
> > packets, and then instruct it to start sending them, those are actually
> > transmitted AT WIRE SPEED!!
> >
> > These results have been obtained considering different software
> > generators (namely, UDPGEN, PACKETGEN, Application level generators)
> > under LINUX (2.4.x, 2.6.x), and under CLICK (using a modified version of
> > UDPGEN).
> >
> > The hardware setup considers
> > - a 2.8GHz Xeon hardware
> > - PCI-X bus (133MHz/64bit)
> > - 1G of Ram
> > - Intel PRO 1000 MT single, double, and quad cards, integrated or on a
> > PCI slot.
> >
> > Different driver versions have been used, and while there are (small)
> > differencies when receiving packets, ALL of them present the same
> > trasmission limits.
> >
> > Moreover, the same happen considering other vendors cards (broadcom
> > based chipset).
> >
> > Is there any limit on the PCI-X (or PCI) that can be the bottleneck?
> > Or Limit on the number of packets per second that can be stored in the
> > NIC tx-fifo?
> > May the lenght of the tx-fifo impact on this?
> >
> > Any hints will be really appreciated.
> > Thanks in advance
>
> cheers,
> Pádraig.
--
Ciao, /\/\/\rco
+-----------------------------------+
| Marco Mellia - Assistant Professor|
| Tel: 39-011-2276-608 |
| Tel: 39-011-564-4173 |
| Cel: 39-340-9674888 | /"\ .. . . . . . . . . . . . .
| Politecnico di Torino | \ / . ASCII Ribbon Campaign .
| Corso Duca degli Abruzzi 24 | X .- NO HTML/RTF in e-mail .
| Torino - 10129 - Italy | / \ .- NO Word docs in e-mail.
| http://www1.tlc.polito.it/mellia | .. . . . . . . . . . . . .
+-----------------------------------+
The box said "Requires Windows 95 or Better." So I installed Linux.
|