understanding speculative preallocation

[Jason Rosenberg]

Is it safe to say that speculative preallocation will not be used if a file
is opened read-only?

It turns out that the kafka server does indeed write lots of log files, and
rotate them after they reach a max size, but never closes the files until
the app exits, or until it deletes the files.  This is because it needs to
make them available for reading, etc.   So, an obvious change for kafka
might be to close each log file after rotating, and then re-open it
read-only for consumers of the data.  Does that sound like a solution that
would pro-actively release pre-allocated storage?

Thanks,

Jason


On Fri, Jul 26, 2013 at 5:04 PM, Jason Rosenberg <jbr at squareup.com> wrote:

> Thanks Ben,
>
> This is helpful.
>
>
> On Fri, Jul 26, 2013 at 4:50 PM, Ben Myers <bpm at sgi.com> wrote:
>
>> Hi Jason,
>>
>> On Fri, Jul 26, 2013 at 04:38:21PM -0400, Jason Rosenberg wrote:
>> > Thanks for the info (most of it was, in fact, news to me).  I'm an
>> > application developer trying to debug a disk space problem, that's all.
>>  So
>> > far, I've tracked it down to being an XFS issue.
>>
>> The speculative block reservations have been an issue for awhile.  You
>> are not
>> the first person to take issue with it.
>>
>> > Regardless, is there a version history for XFS vis-a-via mainline Linux?
>>
>> You can find a full version history for XFS back to 2.6.12 or so here:
>> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=xfs/xfs.git;a=summary
>>
>> If you're interested in going older than that look here:
>> http://oss.sgi.com/cgi-bin/gitweb.cgi?p=archive/xfs-import.git;a=summary
>>
>> The function you'll most likely want to track is xfs_iomap_write_delay,
>> which
>> calls xfs_iomap_eof_want_preallocate, both of which are in
>> fs/xfs/xfs_iomap.c.
>>
>> Recently Brian Foster added a scanner to remove the speculative block
>> reservations on a timer which may give you some relief.  See
>> xfs_queue_eofblocks in fs/xfs/xfs_icache.c
>>
>> Regards,
>>         Ben
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20130726/9714c5b9/attachment-0001.html>