xfs
[Top] [All Lists]

Re: allocsize mount option

To: Eric Sandeen <sandeen@xxxxxxxxxxx>
Subject: Re: allocsize mount option
From: Gim Leong Chin <chingimleong@xxxxxxxxxxxx>
Date: Wed, 13 Jan 2010 17:42:16 +0800 (SGT)
Cc: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.sg; s=s1024; t=1263375736; bh=QpyizeS+eAvbutLkdGdBtYEaXbV9pnEDoOEivwY+19E=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=O/qtF46wh/8zynwAQkYfvnl4eFucRPWKVJgSF+0BqIgaBOxwWW2Fjo2Pdj3T0MJ337QGyGwe9dAzhvCsLVs2KW2AmgxmeUjbBdYeaHsdCsnQr2pVRboXQ1IMV27P0W6MK9i4fBvmqlZtkSX1eYlLgXvrgpPRTy0ZkdHCh5nqppM=
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.sg; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=zGnZgC3I4BIjIci/fni5n7CY4OQ7U1X/wtNx8wSf8JBIiF6uSJawhaUJ7EcBKiIvFT3/Mu4WuCJthr+l1asle3rx1uztP0HPZaLjmyXyUO2sRRu6ODNl/aAgEPK7ElScL71Y8lKs+r+H3SitPjPJeXJnmqhuhaOM0D35kURl3KM=;
Hi,


The application is ANSYS, which writes 128 GB files.  The existing computer 
with SUSE Linux Enterprise Desktop 11 which is used for running ANSYS, has two 
software RAID 0 devices made up of five 1 TB drives.  The /home partition is 
4.5 T, and it is now 4 TB full.  I see a fragmentation > 19%.


I have just set up a new computer with 16 WD Cavair Black 1 TB drives connected 
to an Areca 1680ix-16 RAID with 4 GB cache.  14 of these drives are in RAID 6 
with 128 kB stripes.  The OS is also SLED 11.  The system has 16 GB memory, and 
AMD Phenom II X4 965 CPU.

I have done tests writing 100 30 MB files and 1 GB, 10 GB and 20 GB files, with 
single instance and multiple instances.

There is a big difference in writing speed when writing 20 GB files when using 
allocsize=1g and not using the option.  That is without the inode64 option, 
which gives further speed gains.

I use dd for writing the 1 GB, 10 GB and 20 GB files.

mkfs.xfs -f -b size=4k -d agcount=32,su=128k,sw=12 -i size=256,align=1,attr=2 
-l version=2,su=128k,lazy-count=1 -n version=2 -s size=512 -L /data /dev/sdb1


defaults,nobarrier,usrquota,grpquota,noatime,nodiratime,allocsize=1g,logbufs=8,logbsize=256k,largeio,swalloc

The start of the partition has been set to LBA 3072 using GPT Fdisk to align 
the stripes.

The dd command is:

chingl@tsunami:/data/test/t2> dd if=/dev/zero of=bigfile20GB bs=1073741824 
count=20

Single instance of 20 GB dd repeats were 214, 221, 123 MB/s with allocsize=1g, 
compared to 94, 126 MB/s without.

Two instances of 20 GB dd repeats were aggregate 331, 372 MB/s with 
allocsize=1g, compared to 336, 296 MB/s without.

Three instances of 20 GB dd was aggregate 400 MB/s with, 326 MB/s without.

Six instances of 20 GB dd was 606 MB/s with, 473 MB/s without.


My production configuration is

defaults,nobarrier,usrquota,grpquota,noatime,nodiratime,allocsize=1g,logbufs=8,logbsize=256k,largeio,swalloc,inode64

for which I got up to 297 MB/s for single instance 20 GB dd.



Chin Gim Leong


--- On Tue, 12/1/10, Eric Sandeen <sandeen@xxxxxxxxxxx> wrote:

> From: Eric Sandeen <sandeen@xxxxxxxxxxx>
> Subject: Re: allocsize mount option
> To: "Gim Leong Chin" <chingimleong@xxxxxxxxxxxx>
> Cc: xfs@xxxxxxxxxxx
> Date: Tuesday, 12 January, 2010, 2:16 AM
> Gim Leong Chin wrote:
> > Hi,
> > 
> > Mount options for xfs allocsize=size Sets  the
> buffered I/O
> > end-of-file preallocation size when doing delayed
> allocation writeout
> > (default size is 64KiB).
> > 
> > 
> > I read that setting allocsize to a big value can be
> used to combat
> > filesystem fragmentation when writing big files.
> 
> That's not universally necessary though, depending on how
> you are
> writing them.  I've only used it in the very specific
> case of mythtv
> calling "sync" every couple seconds, and defeating
> delalloc.
> 
> > I do not understand how allocsize works.  Say I
> set allocsize=1g, but
> > my file size is only 1 MB or even smaller.  Will
> the rest of the 1 GB
> > file extent be allocated, resulting in wasted space
> and even file
> > fragmentation problem?
> 
> possibly :)  It's only speculatively allocated,
> though, so you won't
> have 1g for every file; when it's closed the preallocation
> goes
> away, IIRC.
> 
> > Does setting allocsize to a big value result in
> performance gain when
> > writing big files?  Is performance hurt by a big
> value setting when
> > writing files smaller than the allocsize value?
> > 
> > I am setting up a system for HPC, where two different
> applications
> > have different file size characteristics, one writes
> files of GBs and
> > even 128 GB, the other is in MBs to tens of MBs.
> 
> We should probably back up and say:  are you seeing
> fragmentation
> problems -without- the mount option, and if so, what is
> your write pattern?
> 
> -Eric
> 
> > I am not able to find documentation on the behaviour
> of allocsize
> > mount option.
> > 
> > Thank you.
> > 
> > 
> > Chin Gim Leong
> > 
> > 
> > New Email names for you! Get the Email name you've
> always wanted
> > on the new @ymail and @rocketmail. Hurry before
> someone else does! 
> > http://mail.promotions.yahoo.com/newdomains/sg/
> > 
> > _______________________________________________ xfs
> mailing list 
> > xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
> > 
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs
>



      New Email addresses available on Yahoo!
Get the Email name you&#39;ve always wanted on the new @ymail and @rocketmail. 
Hurry before someone else does!
http://mail.promotions.yahoo.com/newdomains/sg/

<Prev in Thread] Current Thread [Next in Thread>