On Mon, 2004-11-29 at 08:09, Robert Olsson wrote:
> jamal writes:
>
> > Have to read the paper - When Robert was last visiting here; we did some
> > tests and packet recycling is not very valuable as far as SMP is
> > concerned (given that packets can be alloced on one CPU and freed on
> > another). There a clear win on single CPU machines.
>
>
> Correct yes at you lab about 2 1/2 years ago.
How time flies when you are having fun ;->
> I see those experiments in a
> different light today as we never got any packet budget contribution
> from SMP with shared mem arch whatsoever. Spent a week w. Alexey in the lab
> to understand whats going on. Two flows with total affinity (for each CPU)
> even removed all locks and part of the IP stack. We were still confused...
>
> When Opteron/NUMA gave good contribution in those setups. We start thinking
> it must be latency and memory controllers that makes the difference. As w.
> each CPU has it's own memory and memory controller in Opteron case.
>
> So from that aspect we expecting the impossible from recycling patch
> maybe it will do better on boxes w. local memory.
>
Interesting thought. Not using a lot of my brain cells to compute i
would say that it would get worse. But i suppose the real reason this
gets nasty on x86 style SMP is because cache misses are more expensive
there, maybe?
> But I think we should give it up in current form skb recycling. If extend
> it to deal cache bouncing etc. We end up having something like slab in
> every driver. slab has improved is not so dominant in profiles now.
>
nod.
> Also from what I understand new HW and MSI can help in the case where
> pass objects between CPU. Did I dream or did someone tell me that S2IO
> could have several TX ring that could via MSI be routed to proper cpu?
I am wondering if the per CPU tx/rx irqs are valuable at all. They sound
like more hell to maintain.
> slab packet-objects have been discussed. It would do some contribution
> but is the complexity worth it?
May not be worth it.
>
> Also I think it could possible to do more lightweight variant of skb
> recycling in case we need to recycle PCI-mapping etc.
>
I think its valuable to have it for people with UP; its not worth the
complexity for SMP IMO.
cheers,
jamal
|