xfs
[Top] [All Lists]

Re: [RFC, PATCH 0/2] xfs: dynamic speculative preallocation for delalloc

To: Michael Monnerie <michael.monnerie@xxxxxxxxxxxxxxxxxxx>
Subject: Re: [RFC, PATCH 0/2] xfs: dynamic speculative preallocation for delalloc
From: Dave Chinner <david@xxxxxxxxxxxxx>
Date: Mon, 18 Oct 2010 10:49:06 +1100
Cc: xfs@xxxxxxxxxxx
In-reply-to: <201010171631.53183@xxxxxx>
References: <C8DCC927.48D3%ivan.novick@xxxxxxx> <201010150914.55604@xxxxxx> <20101015114508.GI4681@dastard> <201010171631.53183@xxxxxx>
User-agent: Mutt/1.5.20 (2009-06-14)
On Sun, Oct 17, 2010 at 04:31:48PM +0200, Michael Monnerie wrote:
> On Freitag, 15. Oktober 2010 Dave Chinner wrote:
> > On Fri, Oct 15, 2010 at 09:14:54AM +0200, Michael Monnerie wrote:
> > > The question is: what is a good value for preallocation?
> > 
> > If you can't answer that, use the defaults.
> 
> Who can really without testing? For a webserver, hosting hundreds of 
> different websites, with many ftp uploads and where the rest of the 
> writes are webservers temp and log files, do you know the best value 
> without investigation? That's what I meant.

Yes, I do know that - the best value is the default value.

> > > I'd guess for a database 1GB seems reasonable,
> > 
> > depends on  your database. If it does direct IO, then allocsize is
> > completely meaningless....
> 
> True, I was thinking of postgresql and mysql (innodb, 1 file per table). 
> Both increment file sizes when needed.

MySQL can do direct IO, so you can't just assume that the database
app will tell you what type of IO it is doing.

> > IIRC, that is the default behaviour for _physical_ extent allocation
> > on files larger than one stripe unit (i.e stripe aligned and sized
> > allocation). That is very different from speculative allocation done
> > at delayed allocation time, which is purely in-memory until physical
> > allocation occurs.
> 
> Together with your other answer (swalloc...) I get the picture now. So 
> delalloc, speculative prealloc, and physical allocation are done, where 
> the first two are in-memory. And the mount option allocsize sets the 
> delalloc size.
>  
> Now something comes to my mind: When I make an ftp upload, it's very 
> slow in terms of disk speed. Say 2MB/s. How long would delalloc wait 
> before flushing buffers on disk? 

delalloc does not do any waiting. The system decides when to flush
data to disk. We even do delalloc for sync writes - the system
flushes the data to disk immediately instead of via memory pressure
or background writeback.

> Isn't that controlled by the values of 
> vm.dirty_background_bytes and vm.dirty_expire_centisecs and 
> vm.dirty_writeback_centisecs?

Exactly.

> Then dirty_background_bytes would be the 
> maximum possible delalloc size?

No. Those numbers control the amount of dirty pages in memory, not
the amount of space we've allocated speculatively. Indeed, by
definition speculative allocation past EOF means it is space that
currently has no dirty pages.... ;)

Cheers,

Dave.
-- 
Dave Chinner
david@xxxxxxxxxxxxx

<Prev in Thread] Current Thread [Next in Thread>