netdev
[Top] [All Lists]

Re: Tx queueing

To: Andrew Morton <andrewm@xxxxxxxxxx>
Subject: Re: Tx queueing
From: Donald Becker <becker@xxxxxxxxx>
Date: Thu, 18 May 2000 12:37:53 -0400 (EDT)
Cc: "netdev@xxxxxxxxxxx" <netdev@xxxxxxxxxxx>
In-reply-to: <392407D4.BE586507@xxxxxxxxxx>
Sender: owner-netdev@xxxxxxxxxxx
On Fri, 19 May 2000, Andrew Morton wrote:

> A number of drivers do this:
> 
> start_xmit()
> {
>       netif_stop_queue()
>       ...
>       if (room for another packet)
>               netif_wake_queue()
>       ...
> }
> 
> I suspect this is a simple port from the dev->tbusy days.

Yes, it's likely from a blind search-and-substitute.
Using stop_queue() and wake_queue() in the transmit path is part of the
reason why the 2.3 changes are bogus.
It is not possible, with these semantics, to write a driver that works well
in both pre-2.3 and 2.3.

> It would seem to be more sensible to do
> 
> start_xmit()
> {
>       ...
>       if (!room for another packet)
>               netif_stop_queue()
> }

This doesn't give us a way to set dev->tbusy, which is required for all
pre-2.3 kernels.

> Also, I'm still attracted to the idea of dequeueing packets within the
> driver (the 'pull' model) rather than stuffing them in via
> qdisc_restart() and the BH callback.  A while back Don said:
> 
> > The BSD stack uses the scheme of dequeuing packets in the ISR.  This was a
> > good design in the VAX days, and with primative hardware that handled only
> > single packets.  But it has horrible cache behavior, needs an extra lock,
> > and can result the interrupt service routine running a very long time,
> > blocking interrupts.
> 
> I never understood the point about cache behaviour.  Perhaps he was
> referring to the benefit which a sequence of short loops has over a
> single, long loop?

There are several issues.  Here are a few.
   A 'pull' semantic has the code tracing everywhere, with the obvious bad
   cache impact.
   You need both a 'pull' transmit routine and a 'push' transmit routine.

>  And nowadays we only block interrupts for this
> device (or things on this device's IRQ?).

Perhaps all on that IRQ.  But you didn't really need to service the SCSI
controller or the other network interfaces, did you?

> Has anyone done any serious work with NIC/CPU bonding?

The Mindcraft "benchmark" is superficially obvious, but the big network
difference was that they were apparently using the TCP/IP checksum hardware
on the i82559.  This has far more effect on SMP performance than anything
else that was done.  We didn't even find out that the chip had the feature
until months later, and still don't have the documentation on how to use
it.

Donald Becker                           becker@xxxxxxxxx
Scyld Computing Corporation
410 Severn Ave. Suite 210
Annapolis MD 21403



<Prev in Thread] Current Thread [Next in Thread>