Hi Jamal,
Thanks for the info and the paper pointer. This looks very interesting.
I've added a couple of comments below...
On Sun, Dec 24, 2000 at 12:03:27PM -0500, jamal wrote:
[...]
>
> I think some load balancing heuristic is needed. I think the heurusitic
> should _not_ be based on counting packets only but rather on CPU load.
> For example, if your IDE is trashing a lot of interupts then you want to
> take this into account as well. As well if a CPU is running a lot of user
> processes you need to take into account those issues.
>
That's the rationale/approach we took in DYNIX/ptx and it certainly works
well with large numbers of processors and high loads.
> But definitely some form of dynamic IRQ routing is a good idea. Sounds
> like a very exciting project (and a conference paper). From my
> conversation with Gerrit they seem to have solved this. My knowledge of the
> APIC is very sparse and i dont have time at the moment.
> Dynamic IO-APIC reprogramming, if it can be done very efficiently is a
> definete win. But like i said i dont have the knowledge there and i
> believe in numbers.
> Seems Gerrit and co had some scheme of figuring which CPU is least loaded
> and handing the interupt to it.
The scheme is that you tell the APIC what your current priority is. The APIC
has a task priority register, but Linux doesn't use it. We just set it to
accept-all at boot time and leave it alone. If you use it to indicate your
current priority, the APIC bus will deliver the interrupt to the least-loaded
CPU. The RR behaviour (yes I'm a Brit :-) happens if there's a choice of
"least loaded".
>
> > Last time I looked, Alphas didn't have APICs. We need to design a
> > sensible architecture-neutral interrupt bonding API (or at least a
> > queryable one) before we run off making x86-specific changes.
>
> How is IRQ affinity achieved on Alphas then? ;-> I thought this was a
> common thing that comes in the PCI package. And if they cant do IRQ
> affinity, i would say they deserve to miss this boat as well.
>
I don't know what the interrupt controller architecure is on the Alpha, but
I suspect it isn't primitive.
> >
> > As a footnote, and I know this won't be a popular view on lse-tech -
>
> BTW, what is this LSE list? I just joined it.
>
It's the (a?) "Linux Scalability Effort". In fact, I'm about to post a
roadmap document which tries to give a better feel for what we wanted to
achieve here. In a nutshell, it's "what can we do to make Linux run better
on big iron, *without* breaking it on uni-proc/embedded systems".
> > philosophically speaking I believe that 2.4 has given enough to the
> > big-end guys. I hope that in 2.5, more emphasis and kernel developer
> > talent will be devoted to the other 99.99% of Linux users. Better
> > device support, plug-n-play, manageability, upgradability, etc. Linux
> > seems to be becoming more and more a server OS lately and I'd like to
> > see that turned around.
> >
> > Of course the three-letter corps need the scalability. Good luck to
> > them and thanks for supporting Linux.
>
There are people in the Linux Technology Center at IBM working on a number
of the above. It just so happens that people like Gerrit and myself work for
the company formally known as Sequent (we were bought by IBM), and hence
our area of expertise is large-scale SMP and NUMA systems.
> > For the privateers, yes, it's
> > *fun* to make Linux faster and it is gratifying, but we need to be
> > aware that it is also *easy*. Solving the problems which are faced by
> > the wider community of Linux users is going to be dull, and hard.
> >
>
> I am not sure if *easy* is correct description here. Fun, yes. That is
> why i participate (and maybe you as well). And if it is fun by definition
> it means i do what i like. I think a combination of people like us results
> in an overall improved Linux as long as there are not too many overlaps.
> And even overlaps might not be a bad idea if you have plenty of time.
> I find Gnome development boring, dull, and hard (not that i cant do it if
> you pointed a gun at me). I am sure the Gnome people think the same about
> what i do.
>
Indeed. I'm unconvinced that working on the kernel is easy, at least not
in comparison to programing in userland. There are people who are scared
witless a the thought of kernel work, but are totally at home (and highly)
skilled working on applications. I do think there is a great deal more work
to be done in userland then on the kernel, and I sincerely hope that enough
people can be found to do it. It's just not my area of expertise.
Regards,
Tim
--
Tim Wright - timw@xxxxxxxxx or timw@xxxxxxxxxxx or twright@xxxxxxxxxx
IBM Linux Technology Center, Beaverton, Oregon
"Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI
|