xfs
[Top] [All Lists]

Re: allocsize mount option

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: allocsize mount option
From: Gim Leong Chin <chingimleong@xxxxxxxxxxxx>
Date: Sun, 24 Jan 2010 14:44:08 +0800 (SGT)
Cc: Eric Sandeen <sandeen@xxxxxxxxxxx>, xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com.sg; s=s1024; t=1264315449; bh=oVDtQm3U6/mXoCb4WQAYS1NMEKcmiC6V+upWrOW+tmc=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=QHwUpWjx5SLVVe5kxsvi9xdmp+ansUhyIOBrRghtGNTUv1DcIX+XxE9v4CXThgoMNw9wYeUfiT8B5zS++eQI6Q01VnXwf7VqmxdbT0FWHCy6QJT9SDTFybtDzwnwSXRUfyUHOxoeN1+77SdBfQy0dB7zc/ciBpToaz3mYCj4Hk4=
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com.sg; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding; b=6QIvWc3TM9CZRLUQ3gXYrev4/xjyHVUETFIpS+lQ3BtHAa27wVvG5tVlVA9Nn3zh9nEG9HTq7sxlc98WY7pCxkh8KWtXT2/5W5XGhShv9/0SUJsLUX50/XTnWzPxfRokvetOWPWphVsMUDBVcnvqv8jy3XRvfWE6W/8YVZrRJRc=;
Hi Guys,


An update:

SUSE Linux Enterprise 11 SP1 will be based on kernel 2.6.32, according to 
Novell support, when I asked for backporting.  That will solve my problems.

I do confirm that all the drives (no file system testing) were tested with NCQ 
enabled on the Tyan Opteron motherboard.  What I did learn was that the note 
book drives nowadays come with NCQ, and that all the SATA desktop and note book 
drives showed very poor sequential write performance with cache disabled.

On another note, there was a post by Michael Monnerie on 20 NOV 2009 about 
kernel 2.6.27, XFS, inode64 and NFS.
I have the 10.9 TB inode64 XFS file system exported via NFS 4 by SLED 11, and 
mounted by SLES 10 SP2, and I have no problems with it.

Finally, A big Thank You to Dave Chinner and Eric Sandeen for your kind 
assistance!


GL

--- On Fri, 15/1/10, Gim Leong Chin <chingimleong@xxxxxxxxxxxx> wrote:

> From: Gim Leong Chin <chingimleong@xxxxxxxxxxxx>
> Subject: Re: allocsize mount option
> To: "Dave Chinner" <david@xxxxxxxxxxxxx>
> Cc: "Eric Sandeen" <sandeen@xxxxxxxxxxx>, xfs@xxxxxxxxxxx
> Date: Friday, 15 January, 2010, 11:08 AM
> Hi Dave,
> 
> Thank you for the advice!
> 
> I have done Direct IO dd tests writing the same 20 GB
> files.  The results are an eye opener!  bs=1GB, count=2
> 
> Single instance repeats of 830, 800 MB/s, compared to
> >100 to under 300 MB/s for buffered.
> 
> Two instances aggregate of 304 MB/s, six instances
> aggregate of 587 MB/s.
> 
> System drive /home RAID 1 of 130 MB/s compared to 51 MB/s
> buffered.
> 
> So the problem is with the buffered writes.
> 
> 
> > Youἀd have to get all the fixes from 2.6.30 to
> 2.6.32,
> > and the
> > backport would be very difficult to get right. Better
> > would
> > be طust to upgrade the kernel to 2.6.32 ;)
> 
> 
> If I change the kernel, I would have no support from
> Novell.  I would try my luck and convince them.
> 
> > > > I'd suggest that you might need to look at
> > increasing the
> > > > maximum IO
> > > > size for the block device
> > > > (/sys/block/sdb/queue/max_sectors_kb),
> > > > maybe the request queue depth as well to
> get
> > larger IOs to
> > > > be pushed
> > > > to the raid controller. if you can, at least
> get
> > it to the
> > > > stripe
> > > > width of 1536k....
> > > 
> > > Could you give a good reference for performance
> tuning
> > of these
> > > parameters?  I am at a total loss here.
> > 
> > Welcome to the black art of storage subsystem tuning
> ;)
> > 
> > I'm not sure there is a good reference for tuning the
> block
> > device
> > parameters - most of what I know was handed down by
> word of
> > mouth
> > from gurus on high mountains.
> > 
> > The overriding principle, though, is to try to ensure
> that
> > the
> > stripe width sized IOs can be issued right through the
> IO
> > stack to
> > the hardware, and that those IOs are correctly aligned
> to
> > the
> > stripes. You've got the filesystem configuration and
> layout
> > part
> > correct, now it's just tuning the block layer to pass
> the
> > IO's
> > through.
> 
> Can I confirm that
> (/sys/block/sdb/queue/max_sectors_kb)=stripe width 1536 kB
> 
> Which parameter is "request queue depth"?  What should be
> the value?
> 
> 
>  
> > FWIW, your tests are not timing how longit takes for
> all
> > the
> > data to hit the disk, only how long it takes to get
> into
> > cache.
> 
> 
> Thank you!  I do know that XFS buffers writes
> extensively.  The drive LEDs remain lighted long after
> the OS says the writes are completed.  Plus some
> timings are physically impossible.
> 
>  
> > That sounds wrong - it sounds like NCQ is not
> functioning
> > properly
> > as with NCQ enabled, disabling the drive cache should
> not
> > impact
> > throughput at all....
> 
> I do not remember clearly if NCQ is available for that
> motherboard, it is an Ubuntu 32-bit, but I do remember
> seeing queue depth in the kernel.  I will check it out
> next week.
> 
> But what I read is that NCQ hurts single write
> performance.  That is also what I found with another
> Areca SATA RAID in Windows XP.
> 
> What I found with all the drives we tested was that
> disabling the cache badly hurt sequential write performance
> (no file system, write data directly to designated LBA).
> 
> 
> 
> > I'd suggest trying to find another distributor that
> will
> > bring them
> > in for you. Putting that many drives in a single
> chassis is
> > almost
> > certainly going to cause vibration problems,
> especially if
> > you get
> > all the disk heads moving in close synchronisation
> (which
> > is what
> > happens when you get all your IO sizing and alignment
> > right).
> 
> I am working on changing to the WD Caviar RE4 drives. 
> Not sure if I can pull it off.
> 
> 
> 
> Chin Gim Leong
> 
> 
>       New Email names for you! 
> Get the Email name you've always wanted on the new
> @ymail and @rocketmail. 
> Hurry before someone else does!

> http://mail.promotions.yahoo.com/newdomains/sg/
> 


      ______________________________________________________________________
Search, browse and book your hotels and flights through Yahoo! Travel.
http://sg.travel.yahoo.com

<Prev in Thread] Current Thread [Next in Thread>