On Tuesday March 6, ak@xxxxxxx wrote:
>
> I think it would be best to fix nfsd to cache file handles instead of dentries
> and only do release when the nfsfh leaves the nfsfh cache.
> This way could also help ext2, which caches readahead context in struct file.
>
> Put Neil Brown into cc. Neil, do you see any obstacles with that?
>
>
> -Andi
Firstly, lets check that I understand the situation.
You find that under high memory pressure dentries for files that are
currently being used by an NFS client get pruned from the dcache (and
so the associated inodes leave the icache) resulting in slower
filehandle-to-dentry lookup times when the next access is made.
You are suggesting that some sort of filehandle cache go into nfsd to
hold onto these denties a bit longer so that on the next lookup they
are more likely to still be there.
Is that right?
I had hoped that the normal dcache mechanisms would have held onto
them long enough, but I can see that under high memory pressure this
may not be long enough.
It might be enough just to add:
if (!IS_ERR(result) result->d_flags |= DCACHE_REFERENCED;
after the first call to nfsd_iget in find_fh_dentry.
This would give the dentry an extra turn around the lru list.
If that isn't enough, I think that the best fix would be to modify
prune_dcache to be a bit more generous.
Something like:
add
#define DCACHE_DELAY 0x0010 /* recently used, don't discard for 20ms */
to dache.h, then change prune_dcache to add
if ((dentry->d_flags & DCACHE_DELAY) &&
time_before(jiffies, dentry->d_reftime+(HZ-1)/50+1)) {
list_add(&dentry->d_lru, &dentry_unused);
goto next;
}
Then whenever nfsd lets go of a dentry after an operation that would
likely be followed by some other operations on the same dentry, the
DCACHE_DELAY bit would get set.
Ofcourse you would have to try to get that change passed Linux and
AlViro, but they might go for it, especially if you can point to
performance numbers. The "20ms" is a bit icky but I don't know what
else to do about that.
You could ofcourse do all this inside knfsd, or even inside the
individual filesystem, but it makes the code a lot more complex. You
need to make sure you flush the cache in a timely fashion, and let go
of denties that have been unlinked and all that stuff.
If we get the dcache to do the delay, then it is easy as it already
copes with those other things.
NeilBrown
|