xfs
[Top] [All Lists]

Re: Fragmentation Issue We Are Having

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: Fragmentation Issue We Are Having
From: Brian Candler <B.Candler@xxxxxxxxx>
Date: Wed, 18 Apr 2012 10:00:51 +0100
Cc: David Fuller <dfuller@xxxxxxxxx>, xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=date:from:to :cc:subject:message-id:references:mime-version:content-type :in-reply-to; s=sasl; bh=uZsRX87I/JxBjvXgz1Rz3mZHOKo=; b=NNzMaFR chrKEb6TTR99VeArG7zgPmCrgqgy0Ot5NpM3HN2eqMFfarAEaBtmJO7D8k0uUGvH pDlOIlH2gDqT1h/6mPfVZJBXKizeVFsZkMsUllzxSFJ6oEQ5Lkhy2J61fgF4bdp5 /u+r9qlYTrlhVSz+FBzG9Md0mKZ7LbIB8Eu8=
Domainkey-signature: a=rsa-sha1; c=nofws; d=pobox.com; h=date:from:to:cc :subject:message-id:references:mime-version:content-type :in-reply-to; q=dns; s=sasl; b=nXBaqe+3OMmSziNQrvd/Gx7K90vzerxyZ dwymh27888i4Sn4kVGtX2DxMhBRsHt9UNJ32DouwA4w+Bs06TKtTCj7EG5/7vW1y pwBRVfu5YGN71WuXOhBnVMuQ0jDo7+Q2Kkiua+cIpPw0IBecZaLG/JS/4cXL9yax qpJsUSvv7A=
In-reply-to: <20120418013607.GK6734@dastard>
References: <CADrkzimg891ZBGK7-UzhGeey16KwH-ZXpEqFr=O3KwD3qA9LwQ@xxxxxxxxxxxxxx> <20120412075747.GB30891@xxxxxxxx> <CADrkzi=JNsbXJHkcb=oOZHLEYMBDUkNHu9O8JFT9h+kSArL47A@xxxxxxxxxxxxxx> <20120413071905.GA823@xxxxxxxx> <20120413075634.GD6734@dastard> <20120413081725.GA3640@xxxxxxxx> <20120417002610.GC6734@dastard> <20120417085828.GA13168@xxxxxxxx> <20120418013607.GK6734@dastard>
User-agent: Mutt/1.5.21 (2010-09-15)
On Wed, Apr 18, 2012 at 11:36:07AM +1000, Dave Chinner wrote:
> And it assumes that inode32 cannot do locality of
> files at all, when in fact it has tunable locality through a syctl.
> 
> Indeed, here's some the performance enhancing games SGI play that
> can only be achieved by using the inode32 allocator:
> 
> http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=linux&db=bks&fname=/SGI_Admin/LX_XFS_AG/ch07.html

Ah, that's new to me. So with inode32 and
  sysctl fs.xfs.rotorstep=255
you can get roughly the same locality benefit for sequentially-written files
as inode64?  (Aside: if you have two processes writing files to two different
directories, will they end up mixing their files in the same AG? That
could hurt performance at readback time if reading them sequentially)

I'm not really complaining about anything here except the dearth of
readily-accessible information. If I download the whole admin book:
http://techpubs.sgi.com/library/manuals/4000/007-4273-004/pdf/007-4273-004.pdf
I see the option inode64 only mentioned once in passing (as being
incompatible with ibound). So if there's detailled information on
what exactly inode64 does and when to use it, it must be somewhere else.

Here's my user story. As a newbie, my first test was to make a 3TB
filesystem on a single drive, and I was doing a simple workload consisting
of writing 1000 files per directory sequentially.  I could achieve a
sequential write speed of 75MB/s but only a sequential read speed of 25MB/s. 
After questioning this on the list and eventually finding that files were
scattered around the disk (thanks to xfs_bmap).  I was pointed to the
inode64 option, which I had seen in the FAQ but hadn't realised how big a
performance difference it would make.

This wasn't just an idle benchmark: my main application is creating a corpus
of files and then processing that corpus of files (either sequentially, or
with multiple processes each working sequentially through subsections of the
corpus)

Regards,

Brian.

<Prev in Thread] Current Thread [Next in Thread>