Re: xfs hardware RAID alignment over linear lvm

[Stewart Webb <stew@xxxxxxxxxxxxxxxxxx>]

Thanks for all this info Stan and Dave,

> "Stripe size" is a synonym of XFS sw, which is su * #disks.  This is the
> amount of data written across the full RAID stripe (excluding parity).

The reason I stated Stripe size is because in this instance, I have 3ware RAID controllers, which refer to

this value as "Stripe" in their tw_cli software (god bless manufacturers renaming everything)

I do, however, have a follow-on question:

On other systems, I have similar hardware:

3x Raid Controllers

1 of them has 10 disks as RAID 6 that I would like to add to a logical volume

2 of them have 12 disks as a RAID 6 that I would like to add to the same logical volume

All have the same "Stripe" or "Strip Size" of 512 KB

So if I where going to make 3 seperate xfs volumes, I would do the following:

mkfs.xfs -d su=512k sw=8 /dev/sda

mkfs.xfs -d su=512k sw=10 /dev/sdb

mkfs.xfs -d su=512k sw=10 /dev/sdc

I assume, If I where going to bring them all into 1 logical volume, it would be best placed to have the sw value set

to a value that is divisible by both 8 and 10 - in this case 2?

Obviously, this is not an ideal situation, and I will most likely modify the hardware to better suite.

But I'd really like to fully understand this.

Thanks for any insight you are able to give

Regards

On 25 September 2013 22:57, Dave Chinner <david@xxxxxxxxxxxxx> wrote:

On Wed, Sep 25, 2013 at 03:34:01PM -0600, Chris Murphy wrote:
>
> On Sep 25, 2013, at 3:18 PM, Stan Hoeppner <stan@xxxxxxxxxxxxxxxxx> wrote:
>
> > On 9/25/2013 7:56 AM, Stewart Webb wrote:
> >> Hi All,
> >
> > Hi Stewart,
> >
> >> I am trying to do the following:
> >> 3 x Hardware RAID Cards each with a raid 6 volume of 12 disks presented to
> >> the OS
> >> all raid units have a "stripe size" of 512 KB
> >
> > Just for future reference so you're using correct terminology, a value
> > of 512KB is surely your XFS su value, also called a "strip" in LSI
> > terminology, or a "chunk" in Linux software md/RAID terminology.  This
> > is the amount of data written to each data spindle (excluding parity) in
> > the array.
> >
> > "Stripe size" is a synonym of XFS sw, which is su * #disks.  This is the
> > amount of data written across the full RAID stripe (excluding parity).
> >
> >> so given the info on the xfs.org wiki - I sould give each filesystem a
> >> sunit of 512 KB and a swidth of 10 (because RAID 6 has 2 parity disks)
> >
> > Partially correct.  If you format each /dev/[device] presented by the
> > RAID controller with an XFS filesystem, 3 filesystems total, then your
> > values above are correct.  EXCEPT you must use the su/sw parameters in
> > mkfs.xfs if using BYTE values.  See mkfs.xfs(8)
> >
> >> all well and good
> >>
> >> But - I would like to use Linear LVM to bring all 3 cards into 1 logical
> >> volume -
> >> here is where my question crops up:
> >> Does this effect how I need to align the filesystem?
> >
> > In the case of a concatenation, which is what LVM linear is, you should
> > use an XFS alignment identical to that for a single array as above.

                                                 ^^^^^^

> So keeping the example, 3 arrays x 10 data disks, would this be su=512k and sw=30?

No, the alignment should match that of a *single* 10 disk array,
so su=512k,sw=10.

Linear concatentation looks like this:

offset          volume                          array
0               +-D1-+-D2-+.....+-Dn-+          0       # first sw
.....
X-sw            +-D1-+-D2-+.....+-Dn-+          0
X               +-E1-+-E2-+.....+-En-+          1       # first sw
.....
2X-sw           +-E1-+-E2-+.....+-En-+          1
2X              +-F1-+-F2-+.....+-Fn-+          2       # first sw
.....
3X-sw           +-F1-+-F2-+.....+-Fn-+          2

Where:
        D1...Dn are the disks in the first array
        E1...En are the disks in the second array
        F1...Fn are the disks in the third array
        X is the size of the each array
        sw = su * number of data disks in array

As you can see, all the volumes are arranged in a single column -
identical to a larger single array of the same size.  Hence the
exposed alignment of a single array is what the filesystem should be
aligned to, as that is how the linear concat behaves.

You also might note here that if you want the second and subsequent
arrays to be correctly aligned to the initial array in the linear
concat (and you do want that), the arrays must be sized to be an
exact multiple of the stripe width.

Cheers,

Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx

_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs

--

Stewart Webb