<div class="gmail_quote">On Tue, May 8, 2012 at 1:47 PM, Dave Chinner <span dir="ltr"><<a href="mailto:david@fromorbit.com" target="_blank">david@fromorbit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On Tue, May 08, 2012 at 01:10:55PM +0800, Zhu Han wrote:<br>
> On Tue, May 8, 2012 at 12:40 PM, Dave Chinner <<a href="mailto:david@fromorbit.com">david@fromorbit.com</a>> wrote:<br>
> > On Tue, May 08, 2012 at 11:24:52AM +0800, Zhu Han wrote:<br>
> > > On Tue, May 8, 2012 at 7:59 AM, Dave Chinner <<a href="mailto:david@fromorbit.com">david@fromorbit.com</a>><br>
> > wrote:<br>
</div><div class="im">> > > > And so now you've triggered the speculative delayed allocation<br>
> > > > beyond EOF, which is normal behaviour. Hence there are currently<br>
> > > > unused blocks beyond EOF which will get removed either when the next<br>
> > > > close(fd) occurs on the file or the inode is removed from the cache.<br>
> > > ><br>
> > ><br>
> > > Close(fd) should be invoked before dd quits. But why the extra blocks<br>
> > > beyond EOF are not freed?<br>
> ><br>
> > The removal is conditional on how many times the fd has been closed<br>
> > with dirty data on the inode.<br>
> ><br>
> > > The only way I found to remove the extra blocks is truncate the file to<br>
> > its<br>
> > > real size.<br>
> ><br>
> > If the close() didn't remove them, they will be removed when the<br>
> > inode ages out of the cache. Why do you even care about them?<br>
><br>
> Our distributed system depends on the real length of files to account the<br>
> space usage.<br>
<br>
</div>That's ..... naive. It's never been valid to assume that the file<br>
size is an accurate reflection of space usage, especially as it will<br>
*always* be wrong for sparse files. In the same light, you also<br>
cannot assume that it is an accurate reflection for non-sparse files<br>
because we can do both explicit and speculative allocation beyond<br>
EOF which only du will show. Not to mention that metadata is not<br>
accounted in the file length, and that can consume a significant<br>
amount of space, too.<br>
<div class="im"><br>
> This behavior make the account inaccurate.<br>
<br>
</div>The block usage reported by XFS is both accurate and correct. The<br>
file size reported by XFS is both accurate and correct. You're<br>
"account inaccuracy" is assuming that they are the same. Perhaps you<br>
should be using quotas for accurate space usage accounting?<br>
<br>
Anyway, if you really want to stop speculative delayed allocation<br>
beyond EOF, then use the allocsize mount option to control it.<br></blockquote><div><br><br>Thanks for help. <br><br>I can control the size of pre-allocation, so no data are written beyond the pre-allocated block range, so no speculative allocation is triggered. Besides it, our system can sync the accurate space usage of mount point periodically. <br>
<br>Can you give any hints about the most lightweight approach to get the accurate block usage of whole file system?<br> <br></div><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
Cheers,<br>
<br>
Dave.<br>
--<br>
Dave Chinner<br>
<a href="mailto:david@fromorbit.com">david@fromorbit.com</a><br>
</div></div></blockquote></div><br>