xfs
[Top] [All Lists]

Re: XFS on 2.6.26: reading the first 4K of a large file takes ages

To: Florian Weimer <fweimer@xxxxxx>
Subject: Re: XFS on 2.6.26: reading the first 4K of a large file takes ages
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Fri, 21 May 2010 18:26:19 +1000
Cc: Stewart Smith <stewart@xxxxxxxxxxxxxxxx>, Christoph Hellwig <hch@xxxxxxxxxxxxx>, xfs@xxxxxxxxxxx
In-reply-to: <828w7d69h8.fsf@xxxxxxxxxx>
References: <8239xojfco.fsf@xxxxxxxxxx> <20100519114826.GA18224@xxxxxxxxxxxxx> <82sk5m7oyz.fsf@xxxxxxxxxx> <87zkztojwh.fsf@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx> <828w7d69h8.fsf@xxxxxxxxxx>
User-agent: Mutt/1.5.20 (2009-06-14)
On Fri, May 21, 2010 at 06:43:15AM +0000, Florian Weimer wrote:
> * Stewart Smith:
> 
> > On Thu, 20 May 2010 12:11:00 +0000, Florian Weimer <fweimer@xxxxxx> wrote:
> >> Thanks for confirming my hunch.  I don't think it's worth fixing this
> >> in XFS.  The database should call posix_fallocate() before flushing
> >> its internal cache to the file in essentially random order, but it's
> >> difficult to get upstream to implement this (the source code is a bit
> >> hard to follow, unfortunately).
> >
> > Which database?
> 
> Oracle Berkeley DB.
> 
> > You could always mount with allocsize
> 
> This happens with "allocsize=4194304".

Because allocsize only works for allocations extending the file.

> > or use other tools to do the preallocation before things got too
> > bad.
> 
> Is there a way to transparently preallocate a few GB after the current
> end of the file?  That would be helpful because Berkeley DB wouldn't
> have to know about it.

Yes. the fallocate() syscall has a mode that allows allocation
beyond the current end of file, as does the XFS_IOC_RESVSP ioctl.

Or, even easier, with xfs_io:

$ stat /mnt/test/foo
  File: `/mnt/test/foo'
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
....
$ xfs_io -f -c "resvsp 0 1048576" /mnt/test/foo
$ stat /mnt/test/foo
  File: `/mnt/test/foo'
  Size: 0               Blocks: 2048       IO Block: 4096   regular empty file
....
$ xfs_bmap -vp /mnt/test/foo
/mnt/test/foo:
 EXT: FILE-OFFSET      BLOCK-RANGE      AG AG-OFFSET        TOTAL FLAGS
   0: [0..2047]:       171912..173959    0 (171912..173959)  2048 10000
$

/mnt/test/foo still a zero length file but has 1MB of extents allocated.

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>