Re: understanding speculative preallocation

[Jason Rosenberg <jbr@xxxxxxxxxxxx>]

Thanks Dave,

The automatic prealloc removal, if no new writes after 5 minutes, sounds perfect for my use case.  But realistically, I'm not likely to get our org to push/find an os update just for this purpose too easily.

So, in the meantime, the question remains, assuming I have the version I have currently (dynamic preallocation, persists indefinitely until the file is closed/app quits, etc.), will this idea work (e.g. close the file after writing, then re-open read-only?).  Currently, the app does keep the files open indefinitely long after writing has stopped, and this is of course resulting in the preallocation persisting indefinitely.

Jason

On Fri, Jul 26, 2013 at 9:30 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:

On Fri, Jul 26, 2013 at 05:11:55PM -0400, Jason Rosenberg wrote:

> Is it safe to say that speculative preallocation will not be used if a file
> is opened read-only?
>
> It turns out that the kafka server does indeed write lots of log files, and
> rotate them after they reach a max size, but never closes the files until
> the app exits, or until it deletes the files.  This is because it needs to
> make them available for reading, etc.   So, an obvious change for kafka
> might be to close each log file after rotating, and then re-open it
> read-only for consumers of the data.  Does that sound like a solution that
> would pro-actively release pre-allocated storage?

No need - the mainline code that has a periodic background scan that
stops buildup of unused prealocation. i.e. if the file is clean for
5 minutes, then the prealloc will be removed. Hence it doesn't
matter what the application does with it - if it holds it open and
doesn't write to the file, then the prealloc will get removed. More
will be added the next time the file is written, but until then it
won't use excessive space.

Cheers,

Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx