xfs
[Top] [All Lists]

Re: df bigger than ls?

To: Brian Candler <B.Candler@xxxxxxxxx>
Subject: Re: df bigger than ls?
From: Eric Sandeen <sandeen@xxxxxxxxxxx>
Date: Wed, 07 Mar 2012 12:04:26 -0600
Cc: xfs@xxxxxxxxxxx
In-reply-to: <20120307171619.GA23557@xxxxxxxx>
References: <20120307155439.GA23360@xxxxxxxx> <20120307171619.GA23557@xxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:10.0.2) Gecko/20120216 Thunderbird/10.0.2
On 3/7/12 11:16 AM, Brian Candler wrote:
> On Wed, Mar 07, 2012 at 03:54:39PM +0000, Brian Candler wrote:
>> core.size = 1085407232
>> core.nblocks = 262370
> 
> core.nblocks is correct here: space used = 262370 * 4 = 1049480 KB
> 
> (If I add up all the non-hole extents I get 2098944 blocks = 1049472 KB
> so there are two extra blocks of something)
> 
> This begs the question of where stat() is getting its info from?
> 
> Ah... but I've found that after unmounting and remounting the filesystem
> (which I had to do for xfs_db), du and stat report the correct info.
> 
> In fact, dropping the inode caches is sufficient to fix the problem:

Yep.

XFS speculatively preallocates space off the end of a file.  The amount of
space allocated depends on the present size of the file, and the amount of
available free space.  This can be overridden
with mount -o allocsize=64k (or other size for example)

$ git log --pretty=oneline fs/xfs | grep specul
b8fc82630ae289bb4e661567808afc59e3298dce xfs: speculative delayed allocation 
uses rounddown_power_of_2 badly
055388a3188f56676c21e92962fc366ac8b5cb72 xfs: dynamic speculative EOF 
preallocation

so:

# dd if=/dev/zero of=bigfile bs=1M count=1100 &>/dev/null
# ls -lh bigfile
-rw-r--r--. 1 root root 1.1G Mar  7 11:47 bigfile
# du -h bigfile
1.1G    bigfile

but:

# rm -f bigfile
# for I in `seq 1 1100`; do dd if=/dev/zero of=bigfile conv=notrunc bs=1M 
seek=$I count=1 &>/dev/null; done
# ls -lh bigfile
-rw-r--r--. 1 root root 1.1G Mar  7 11:49 bigfile
# du -h bigfile
2.0G    bigfile

This should get freed when the inode is dropped from the cache; hence your 
cache drop bringing it back to size.

But there does seem to be an issue here; if I make a 4G filesystem and repeat 
the above test 3 times, the 3rd run gets ENOSPC, and the last file written 
comes up short, while the first one retains all it's extra preallocated space:

# du -hc bigfile*
2.0G    bigfile1
1.1G    bigfile2
907M    bigfile3

Dave, is this working as intended?  I know the speculative preallocation amount 
for new files is supposed to go down as the fs fills, but is there no way to 
discard prealloc space to avoid ENOSPC on other files?

-Eric

> root@storage1:~# du -h /disk*/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk10/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk11/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk12/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk1/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk2/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk3/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk4/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk5/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk6/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk7/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk8/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 2.0G  /disk9/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> root@storage1:~# echo 3 >/proc/sys/vm/drop_caches 
> root@storage1:~# du -h /disk*/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk10/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk11/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk12/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk1/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk2/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk3/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk4/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk5/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk6/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk7/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk8/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> 1.1G  /disk9/scratch2/work/PRSRA1/PRSRA1.1.0.bff
> root@storage1:~# 
> 
> Very odd, but not really a major problem other than the confusion it causes.
> 
> Regards,
> 
> Brian.
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs
> 

<Prev in Thread] Current Thread [Next in Thread>