xfs
[Top] [All Lists]

Re: 2.6.xx: NFS: directory motion/cam2 contains a readdir loop

To: Christoph Hellwig <hch@xxxxxxxxxxxxx>
Subject: Re: 2.6.xx: NFS: directory motion/cam2 contains a readdir loop
From: Justin Piszcz <jpiszcz@xxxxxxxxxxxxxxx>
Date: Wed, 27 Jul 2011 15:44:20 -0400 (EDT)
Cc: "J. Bruce Fields" <bfields@xxxxxxxxxxxx>, linux-nfs@xxxxxxxxxxxxxxx, linux-kernel@xxxxxxxxxxxxxxx, xfs@xxxxxxxxxxx
In-reply-to: <20110727193937.GA5354@xxxxxxxxxxxxx>
References: <alpine.DEB.2.02.1107270952270.1451@xxxxxxxxxxxxxxxx> <20110727160752.GC974@xxxxxxxxxxxx> <alpine.DEB.2.02.1107271227340.1451@xxxxxxxxxxxxxxxx> <20110727181111.GA23009@xxxxxxxxxxxxx> <alpine.DEB.2.02.1107271530340.1451@xxxxxxxxxxxxxxxx> <20110727193937.GA5354@xxxxxxxxxxxxx>
User-agent: Alpine 2.02 (DEB 1266 2009-07-14)


On Wed, 27 Jul 2011, Christoph Hellwig wrote:

On Wed, Jul 27, 2011 at 03:35:01PM -0400, Justin Piszcz wrote:
Currently I do not see any dupes, however I have a script that moves
images out of the directory once an hour:
0 * * * * /usr/local/bin/move_to_old2.sh > /dev/null 2>&1

Do you keep adding files to the directory while you move files out?
Yes, otherwise there are too many files in the directory and viewers, e.g.,
each geeqie (picture viewer) will use > 4-6GB of memory, so I try to keep
it around 5,000 pictures or less.

What's the rate of additions/removals to the directory?
Additions it depends, around 5,000 over a 12hr period, 416/hr, current:

atom:/d1/motion# find cam1|wc
   5215    5215  166853
atom:/d1/motion# find cam2|wc
   5069    5069  162181
atom:/d1/motion# find cam3|wc
   5594    5594  178981
atom:/d1/motion#


If we add files to the directory while removing others we could easily
re-use the same offset for a different file.


Justin.

<Prev in Thread] Current Thread [Next in Thread>