On Wed, Sep 08, 2004 at 10:21:17PM +0200, Jakob Oestergaard wrote:
> Would *you* base a production environment on that ? ;)
I might if I used ext3, they didn't look all the bad but I only
glanced over them.
> 2.5G RAM IIRC - but slab is in lowmem so the machine will barf when
> it reaches ~850MB slab.
Ick. This is a 'known problem' that some people hit. Basically you
get bazillions of dentries as the backup runs (or whatever, find /fs
-noleaf, works pretty well too) which consume slab. What's worse is
that they pin other slab objects used by XFS. Checking 'bloatmeter'
on my desktop I see:
xfs_inode: 90327KB 115723KB 78.5
linvfs_icache: 84445KB 106528KB 79.27
dentry_cache: 24118KB 62147KB 38.80
This shows for me the largest slab users are xfs_inode and
linvfs_icache, these can't be freed until the dentry_cache objects
that pin them are also freed (the slab-freeing logic isn't aware of
this and I'm not sure that even if it were what would help much even
if it were but I always meant to test if it would help).
The problem is basically we don't reclaim slab/lowmem hard enough at
times and even when we do slab fragmentation means we can still have
lots of memory tied up until we really aggressively prune the slab.
> There seems to be a common trend that NFS+XFS+SMP is causing this. I
> suppose that could explain some of that.
It it's OOM death due to lack of lowmem then no, it's not XFS to blame
but XFS doesn't help as it uses a bit more slab that other fs' for the
same use (I don't have numbers on this, but I'm pretty sure it's a
fair bit more last I checked). I looked at the in-core structures to
see if they could be shrunk I while ago and nothing jumped out as
being an obvious candidate for assassination.
Ideally the slab blanacing logic needs more work. I really don't know
this personally at all but akpm and nick did some changes in the 2.6.5
time-frame which helped and I wonder if more work can be done here?
It was suggested to me:
"you can try changing include/linux/mm.h DEFAULT_SEEKS from 2 to 1.
that will make slab be reclaimed more aggressively."
does that help at all?
I also wonder if -mm kernels might behave differently?
... not that this helps you, but because slab/lowmem balancing is such
a pain on i386, I'll personally never get another i386 platform again
almost entirely because of this when amd64 platforms are so cheap and
don't have this problem (you don't have this special case were a
large chunk of memory is only usable for certain things).
Sorry I can't think of anything more productive right this second (and
I know saying get an opteron isn't what yoiu want to head).
 Well, arguably slab can still get out of hand but it usually
doesn't consume all of a special class of memory and cause the
machine to die...