Re: [Myricom help #36078] application sleeps indefinitely trying to read gm_dma_malloc'ed buffer through O_DIRECT

[Myricom Technical Support <help@xxxxxxxx>]

Hi, Ashutosh,

> Disabling the O_DIRECT flag makes things work fine. Using standard memory
> allocated (i.e. memory not registered with GM using gm_dma_malloc()) using
> a malloc() or a posix_memalign() makes sure it works fine too.
>
> I am guessing this is an issue between gm_dma_malloc() pinning and locking
> the memory for DMA transfers; and XFS not being able to handle memory page
> locking/pinning when initiation a DIRECT I/O from that buffer, but only
> when reading.

Yes, this is a fundamental problem of conflictual use between O_DIRECT 
and memory registration in GM.

(We are surprised to hear that this ever worked with some Linux kernels.)

I have forwarded your question to our GM software developers, and they 
are pondering ways of allocating the initial memory differently that may 
allow this to work.

Susan

--
Susan Blackford
Member of Technical Staff
Myricom Inc.