[Top] [All Lists]

Re: understanding speculative preallocation

To: Jason Rosenberg <jbr@xxxxxxxxxxxx>
Subject: Re: understanding speculative preallocation
From: Ben Myers <bpm@xxxxxxx>
Date: Fri, 26 Jul 2013 16:42:38 -0500
Cc: stan@xxxxxxxxxxxxxxxxx, xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <CAA+BczTt-ZN7DU8gpWAY0SdL0KWL2Z_y8qKvmxMn7b+T9frK4Q@xxxxxxxxxxxxxx>
References: <1374823420041-35002.post@xxxxxxxxxxxxx> <20130726115021.GO13468@dastard> <CAA+BczQesNL2VmFmrcBNKXcM-Sfx0bXkXPRP5xMx6=Bv+NWrUA@xxxxxxxxxxxxxx> <51F2CD8B.8080207@xxxxxxxxxxxxxxxxx> <CAA+BczQGYoJVL0twvz2GRhH30teFPSJOsKWtofbXBrom4_Q6hg@xxxxxxxxxxxxxx> <20130726205018.GM3111@xxxxxxx> <CAA+BczSpn7Cu2rZWJ9Q2b1-Xh0365R9ykGxKHHymtRm2Q7xCuA@xxxxxxxxxxxxxx> <CAA+BczTt-ZN7DU8gpWAY0SdL0KWL2Z_y8qKvmxMn7b+T9frK4Q@xxxxxxxxxxxxxx>
User-agent: Mutt/1.5.20 (2009-06-14)
Hi Jason,

On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:
> Is it safe to say that speculative preallocation will not be used if a file
> is opened read-only?

The blocks will only be reserved on an appending write.

> It turns out that the kafka server does indeed write lots of log files, and
> rotate them after they reach a max size, but never closes the files until
> the app exits, or until it deletes the files.  This is because it needs to
> make them available for reading, etc.   So, an obvious change for kafka
> might be to close each log file after rotating, and then re-open it
> read-only for consumers of the data.  Does that sound like a solution that
> would pro-actively release pre-allocated storage?

An interesting idea, and I'm not quite sure.  The blocks past EOF are freed in
xfs_release on close in some circumstances, and it looks like you have a chance
to call xfs_free_eofblocks (at least in the most uptodate codebase) if you did
not use explicit preallocation (e.g. fallocate or an xfs ioctl) and did not
open it append-only.  You could reopen with read-write flags and it wouldn't
make a difference vs read-only, so long as you don't do an appending write.

Seems like it's worth a try.  Another possibility is to look into what would
happen if you do a truncate up to i_size when you're ready to stop appending to
the file.  I haven't checked that out though.


<Prev in Thread] Current Thread [Next in Thread>