Re: Linux support for RDMA (was: [Ksummit-2005-discuss] Summary of 2005 Kernel Summit Proposed Topics)

[Grant Grundler <grundler@xxxxxxxxxxxxxxxx>]

On Mon, Apr 04, 2005 at 09:54:10AM -0700, Dmitry Yusupov wrote:
> >     3) store back into RAM
> 
> no. we are talking about receive side optimization only.
> why do you think store back into RAM comes to the picture?

Application eventually wants to read the data.

> also keep in mind that we have huge L2 & L3 caches today and write
> operation is usually very well buffered.

Agreed. But how effective the cache is will depend on if the CPU
(application) can process the data as fast as it arrives (and still
be in the cache). Otherwise the data will get pushed out in (3)
and recalled later when the app can consume it (4th time across).

It also assumes the application is running on a CPU core that shares
the cache with the CPU that did the copy. If the CPU is saturated
with the copy (ok, assume we've got 2 Cores per socket), then the
other CPU has to be *assigned* manually to make sure it does the
other part.

Jamal learned all this when he moved to a dual core PPC for
his fast routing work. Jamal, did that ever make it into
a paper?

grant