128TB filesystem limit?

[Steve Costaras]

On 03/25/2010 23:56, david at lang.hm wrote:
> On Thu, 25 Mar 2010, Eric Sandeen wrote:
>
>> david at lang.hm wrote:
>>> On Fri, 26 Mar 2010, Dave Chinner wrote:
>>
>> ...
>>
>>>> Is there any reason for putting partitions on these block devices?
>>>> You could just use the block devices without partitions, and that
>>>> will avoid alignment potential problems....
>>>
>>> I would like to raid to auto-assemble and I can't do that without
>>> partitions, can I
>>
>> I think you can.... it's not like MD is putting anything in the 
>> partition
>> table; you just give it block devices, I doubt it cares if it's a whole
>> disk or some partition.
>>
>> Worth a check anyway ;)
>
> I know that md will work on raw devices, but the auto-assembly stuff 
> looks for the right partition type, I would have to maintain a conf 
> file across potential system rebuilds if I used the raw partitions.
>
>> ...
>>
>>
>>> the next fun thing is figuring out what sort of stride, etc 
>>> parameters I
>>> should have used for this filesystem.
>>
>> mkfs.xfs should suss that out for you automatically based on talking 
>> to md;
>> of course you'd want to configure md to line up well with the hardware
>> alignment.
>
> in this case md thinks it's working with 10 12.8TB drives, I really 
> doubt that it's going to do the right thing.
>
> I'm not exactly sure what the right thing is in this case. the 
> hardware raid is useing 64K chunks across 16 drives (so 14 * 64K worth 
> of data per stripe), but there are 10 of these stripes before you get 
> back to hitting the same drive again.
>
> David Lang
>

It does here at least, I never use partition tables on any of the arrays 
here just use LVM against what it sees as the 'raw' disk.    I haven't 
tried it w/ a 128TB array but with smaller ones that's what I've used in 
the past (hw raid, md raid-0, file system).   Recently for systems now I 
just use HW raid; LVM; and then filesystem (lvm does the striping/raid-0 
function).    when you create the physical volume w/ lvm just make sure 
you allign it (older versions use --metadatasize to 'pad' the start 
offset), newer versions have the dataalignment option.


Steve