xfs
[Top] [All Lists]

Re: Warning: AG size is a multiple of stripe width?

To: Gim Leong Chin <chingimleong@xxxxxxxxxxxx>
Subject: Re: Warning: AG size is a multiple of stripe width?
From: Lachlan McIlroy <lmcilroy@xxxxxxxxxx>
Date: Sun, 19 Jun 2011 21:21:37 -0400 (EDT)
Cc: xfs@xxxxxxxxxxx
In-reply-to: <385532.69322.qm@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
Reply-to: Lachlan McIlroy <lmcilroy@xxxxxxxxxx>
----- Original Message -----
> Hi,
> 
> 
> I have a Sun workstation with eight Cheetah 15K.5 SAS 300 GB on RAID
> 1E (RAID 10) on LSI SAS3081E-R.
> 
> I am installing SLED 11 SP1 on it and I thought I will do a thorough
> optimization right down to the partition boundaries.
> 
> Since the default for XFS is to create four aggregation groups, and
> with the reasoning that Cheetah can do double the seeks of normal 7200
> RPM drives, I have four aggregation groups per drive for a total of 16
> for 70 GB /dev/sda2 partition, and eight per drive for total of 32 for
> /dev/sda3 partition (1011 GB).
> 
> I have aligned the partition start and end with the stripe width
> boundaries.
> The stripe size is 64 kB, stripe width is 4*64 kB = 256 kB, in terms
> of 512 byte sectors:
> 
> 70 GB /
> 
> No Start End Number
> 1 512 67109375 32 GB = 67108864 sectors = 131072 stripe sets
> 
> 2 67109376 213910015 70 GB = 146800640 sectors = 286720 stripe sets
> 
> 3 213910016 2335932415 Left = 2335932416 - 213910016 = 2122022400
> sectors = 4144575 stripe sets
> 
> 
> When I do the following:
> 
> mkfs.xfs -f -b size=4k -d agcount=16,su=64k,sw=4 -i
> size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2
> -s size=512 -L / /dev/sda2
> 
> Warning: AG size is a multiple of stripe width. This can cause
> performance problems by aligning all AGs on the same disk. To avoid
> this, run mkfs with an AG size that is one stripe unit smaller, for
> example 1146864
> 
> agcount=16 agsize=1146880 blks
> bsize=4096
> sunit=16 swidth=64 blks
> 
> mkfs.xfs -f -b size=4k -d agcount=32,su=64k,sw=4 -i
> size=256,align=1,attr=2 -l version=2,su=64k,lazy-count=1 -n version=2
> -s size=512 -L /home /dev/sda3
> 
> Warning: AG size is a multiple of stripe width. This can cause
> performance problems by aligning all AGs on the same disk. To avoid
> this, run mkfs with an AG size that is one stripe unit smaller, for
> example 8289136

Each allocation group has some headers that are updated frequently and if
the allocation group size is a multiple of the AG size then all of the AG
headers will land on the same disk.  This may cause congestion on this
disk and reduce the performance of the rest of the stripe.

> 
> agcount=32 agsize=8289152 blks
> bsize=4096
> sunit=16 swidth=64 blks
> 
> I am really puzzled since I thought all I am doing is distributing 4
> aggregation groups per drive for sda2 and 8 per drive for sda3.
> 
> What have I done wrong and what is the flaw with my understanding?

If you had concatenated the disks then your logic would make sense but
since you've striped the disks then all of the allocations groups will be
represented in part on each of the disks.  So if you have 32 AGs then each
disk will need to accommodate (part of) each of the 32 AGs instead of
the recommended 4 AGs which will probably make the disks seek a lot more.

> 
> Thank you!
> 
> 
> Chin Gim Leong
> 
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs

<Prev in Thread] Current Thread [Next in Thread>