xfs hardware RAID alignment over linear lvm

[Stewart Webb]

Thanks for all this info Stan and Dave,

> "Stripe size" is a synonym of XFS sw, which is su * #disks.  This is the
> amount of data written across the full RAID stripe (excluding parity).

The reason I stated Stripe size is because in this instance, I have 3ware
RAID controllers, which refer to
this value as "Stripe" in their tw_cli software (god bless manufacturers
renaming everything)

I do, however, have a follow-on question:
On other systems, I have similar hardware:
3x Raid Controllers
1 of them has 10 disks as RAID 6 that I would like to add to a logical
volume
2 of them have 12 disks as a RAID 6 that I would like to add to the same
logical volume

All have the same "Stripe" or "Strip Size" of 512 KB

So if I where going to make 3 seperate xfs volumes, I would do the
following:
mkfs.xfs -d su=512k sw=8 /dev/sda
mkfs.xfs -d su=512k sw=10 /dev/sdb
mkfs.xfs -d su=512k sw=10 /dev/sdc

I assume, If I where going to bring them all into 1 logical volume, it
would be best placed to have the sw value set
to a value that is divisible by both 8 and 10 - in this case 2?

Obviously, this is not an ideal situation, and I will most likely modify
the hardware to better suite.
But I'd really like to fully understand this.

Thanks for any insight you are able to give

Regards


On 25 September 2013 22:57, Dave Chinner <david at fromorbit.com> wrote:

> On Wed, Sep 25, 2013 at 03:34:01PM -0600, Chris Murphy wrote:
> >
> > On Sep 25, 2013, at 3:18 PM, Stan Hoeppner <stan at hardwarefreak.com>
> wrote:
> >
> > > On 9/25/2013 7:56 AM, Stewart Webb wrote:
> > >> Hi All,
> > >
> > > Hi Stewart,
> > >
> > >> I am trying to do the following:
> > >> 3 x Hardware RAID Cards each with a raid 6 volume of 12 disks
> presented to
> > >> the OS
> > >> all raid units have a "stripe size" of 512 KB
> > >
> > > Just for future reference so you're using correct terminology, a value
> > > of 512KB is surely your XFS su value, also called a "strip" in LSI
> > > terminology, or a "chunk" in Linux software md/RAID terminology.  This
> > > is the amount of data written to each data spindle (excluding parity)
> in
> > > the array.
> > >
> > > "Stripe size" is a synonym of XFS sw, which is su * #disks.  This is
> the
> > > amount of data written across the full RAID stripe (excluding parity).
> > >
> > >> so given the info on the xfs.org wiki - I sould give each filesystem
> a
> > >> sunit of 512 KB and a swidth of 10 (because RAID 6 has 2 parity disks)
> > >
> > > Partially correct.  If you format each /dev/[device] presented by the
> > > RAID controller with an XFS filesystem, 3 filesystems total, then your
> > > values above are correct.  EXCEPT you must use the su/sw parameters in
> > > mkfs.xfs if using BYTE values.  See mkfs.xfs(8)
> > >
> > >> all well and good
> > >>
> > >> But - I would like to use Linear LVM to bring all 3 cards into 1
> logical
> > >> volume -
> > >> here is where my question crops up:
> > >> Does this effect how I need to align the filesystem?
> > >
> > > In the case of a concatenation, which is what LVM linear is, you should
> > > use an XFS alignment identical to that for a single array as above.
>                                                  ^^^^^^
> > So keeping the example, 3 arrays x 10 data disks, would this be su=512k
> and sw=30?
>
> No, the alignment should match that of a *single* 10 disk array,
> so su=512k,sw=10.
>
> Linear concatentation looks like this:
>
> offset          volume                          array
> 0               +-D1-+-D2-+.....+-Dn-+          0       # first sw
> .....
> X-sw            +-D1-+-D2-+.....+-Dn-+          0
> X               +-E1-+-E2-+.....+-En-+          1       # first sw
> .....
> 2X-sw           +-E1-+-E2-+.....+-En-+          1
> 2X              +-F1-+-F2-+.....+-Fn-+          2       # first sw
> .....
> 3X-sw           +-F1-+-F2-+.....+-Fn-+          2
>
> Where:
>         D1...Dn are the disks in the first array
>         E1...En are the disks in the second array
>         F1...Fn are the disks in the third array
>         X is the size of the each array
>         sw = su * number of data disks in array
>
> As you can see, all the volumes are arranged in a single column -
> identical to a larger single array of the same size.  Hence the
> exposed alignment of a single array is what the filesystem should be
> aligned to, as that is how the linear concat behaves.
>
> You also might note here that if you want the second and subsequent
> arrays to be correctly aligned to the initial array in the linear
> concat (and you do want that), the arrays must be sized to be an
> exact multiple of the stripe width.
>
> Cheers,
>
> Dave.
> --
> Dave Chinner
> david at fromorbit.com
>
> _______________________________________________
> xfs mailing list
> xfs at oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>



-- 
Stewart Webb
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20130926/169c475e/attachment-0001.html>