Re: xfs_iread_extents high order allocation failure (with fix idea)

[Neil Bortnak <nbortnak@xxxxxxxxx>]

Neil Bortnak wrote:
> Hi everyone,
>
> First off, thanks for a great filesystem. I've been using it for years 
> without a
> hiccup.
>
> Currently, I have a program which does a lot of simultaneous random 
> accesses to
> alot of big files on a fairly large LVM volume (1.1T). I think I have 
> traced
> the problem to a number of high order (5) allocations by xfs_iread_extents.
> Eventually (due to memory fragmentation I guess) the system is unable to
> process the request. After several failed attempt the OOM killer steps 
> in and
> kills the process.
>
> This appears to be the same bug as the one in June at:
> http://oss.sgi.com/archives/linux-xfs/2004-06/msg00079.html
> There is a patch included, but Nathan Scott said that Christoph wasn't 
> entirely
> happy with it. I had a possible idea for a fix...
>
> Since physically contiguous pages are only strictly needed by the 
> hardware (I
> understand it is a performance issue otherwise), and xfs is up a little 
> higher
> than that, there shouldn't technically be a reason I couldn't use vmalloc
> instead of kmalloc in xfs's kmem_alloc routine, is there?
>
> More appropriately I could add a flag to the kmem_alloc routine so that 
> it will
> use either one based on the preferences of the caller. Then I could use 
> vmalloc
> from xfs_iread_extents, or other places where I expect a large alloc. Also,
> whatever is being allocated here seems to get allocated and then sticks 
> around,
> so if vmalloc doesn't use the slab, the alloc/dealloc overhead still 
> might not
> be too large.
>
> I am happy to write up a patch to do this and test it, but before I do 
> anything
> really stupid, I thought I'd check in with you guys.
>
> Good idea? Bad idea?
>
> Neil
Ok, I guess I was paying attention to the wrong thing when I was going through the code. I now know 
that I can use vmalloc in kmem_alloc because... <shock> it's already there, which is triggered by a 
request that is too large. Just like I wanted really. Doh! Sorry for my list-posted blunder.
In any case, what I don't get is that my error is due to a failure allocating a 128k block (order 
5), when the MAX_SLAB_SIZE is set to 64k (order 4). I don't understand why kmalloc would try to 
allocate 128k of pages for what should be a 64k slab (as that's out max size). There doesn't seem to 
be a mechanism for allocating extra memory just in case you need another object. It allocates just 
enough pages to fulfill the request. Odd...
In any case I am going to apply the patch offered in the previous discussion and toy with the idea 
of setting MAX_SLAB_SIZE to 32k. Will there be/is there an approved fix for this coming soon?
Thanks,

Neil