By "<span style="color:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">According to your advice...</span>", I mean what your demonstrated.<div><br></div><div>I mount with inode64, and everything is working perfectly.</div>
<div>Many thanks, really appreciate!<br><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Nov 9, 2012 at 11:08 AM, Dave Chinner <span dir="ltr"><<a href="mailto:david@fromorbit.com" target="_blank">david@fromorbit.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=""><div class="h5">On Fri, Nov 09, 2012 at 10:04:57AM +0800, huubby zhou wrote:<br>
> Hi, Dave,<br>
><br>
> Thanks for the answer, it's great, and I apologize for the terrible format.<br>
><br>
> >You can't, directly. If you have enough contiguous free space in the<br>
> >AG that you are allocating in, then you will get contiguous files if<br>
> >the allocation size lines up with the filesystem geometry:<br>
> ><br>
> >$ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0<br>
> 512m" foo.$i ; done<br>
> >$ sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"<br>
> > EXT: FILE-OFFSET BLOCK-RANGE AG AG-OFFSET TOTAL FLAGS<br>
> > sudo xfs_bmap -vp foo.[1-9] foo.10 |grep " 0:"<br>
> > 0: [0..1048575]: 8096..1056671 0 (8096..1056671) 1048576 10000<br>
> > 0: [0..1048575]: 1056672..2105247 0 (1056672..2105247) 1048576 10000<br>
> > 0: [0..1048575]: 2105248..3153823 0 (2105248..3153823) 1048576 10000<br>
> > 0: [0..1048575]: 3153824..4202399 0 (3153824..4202399) 1048576 10000<br>
> > 0: [0..1048575]: 4202400..5250975 0 (4202400..5250975) 1048576 10000<br>
> > 0: [0..1048575]: 5250976..6299551 0 (5250976..6299551) 1048576 10000<br>
> > 0: [0..1048575]: 6299552..7348127 0 (6299552..7348127) 1048576 10000<br>
> > 0: [0..1048575]: 7348128..8396703 0 (7348128..8396703) 1048576 10000<br>
> > 0: [0..1048575]: 8396704..9445279 0 (8396704..9445279) 1048576 10000<br>
> > 0: [0..1048575]: 9445280..10493855 0 (9445280..10493855) 1048576<br>
> 10000<br>
> ><br>
> >So all those files are contiguous both internally and externally. If<br>
> >there isn't sufficient contiguous freespace, or there is allocator<br>
> >contention, this won't happen - it's best effort behaviour....<br>
><br>
> I believe you got these in a single AG, but I do the allocation in<br>
> filesystem<br>
> with multi-AGs, specifically, it is a 6T storage space, and I run the<br>
> mkfs.xfs<br>
> without setting the AG number/size, it ends up with 32 AGs.<br>
> My files layout:<br>
> - 0 - dir<br>
> | - 0 - dir<br>
> | | - 1 - file<br>
> | | - 2 - file<br>
> | | - 3 - file<br>
> | | - 4 - file<br>
> | | - 5 - file<br>
> | | - ... - file<br>
> | | - 128 - file<br>
> | - 1 - dir<br>
> | | - 1 - file<br>
> | | - 2 - file<br>
> | | - 3 - file<br>
> | | - 4 - file<br>
> | | - 5 - file<br>
> | | - ... - file<br>
> | | - 128 - file<br>
> | - ... - dir<br>
> Every file is 512MB, every directory holds 512MB*128=64GB.<br>
<br>
</div></div>Yup, that's exactly by design. That's how the inode64 allocation<br>
policy is supposed to work.<br>
<div class="im"><br>
> According to your advice and XFS document, I tried to set the AG size to<br>
> 64GB,<br>
<br>
</div>What advice might that be? I don't thikn I've ever recommended<br>
anyone use 96*64GB AGs. Unless you have 96 allocations all occurring<br>
at the same time (very rare, in my experience), there is no need for<br>
some many AGs.<br>
<div><div class="h5"><br>
<br>
> for avoiding the allocator contention and keeping all files in single<br>
> directory<br>
> fall in the same AG, but it didn't work. The files are still in different<br>
> AGs.<br>
> My xfs_info:<br>
> meta-data=/dev/sdc2 isize=256 agcount=96, agsize=16777216<br>
> blks<br>
> = sectsz=512 attr=0<br>
> data = bsize=4096 blocks=1610116329, imaxpct=25<br>
> = sunit=0 swidth=0 blks, unwritten=1<br>
> naming =version 2 bsize=4096<br>
> log =internal log bsize=4096 blocks=32768, version=1<br>
> = sectsz=512 sunit=0 blks, lazy-count=0<br>
> realtime =none extsz=4096 blocks=0, rtextents=0<br>
><br>
> The files:<br>
> $ for i in `seq 1 10` ; do sudo xfs_io -f -c "truncate 512m" -c "resvsp 0<br>
> 512m" foo.$i ; done<br>
> $ sudo xfs_bmap -vp *| grep " 0:"<br>
> 0: [0..1048575]: 2147483712..2148532287 16 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 3355443264..3356491839 25 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2281701440..2282750015 17 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2415919168..2416967743 18 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2550136896..2551185471 19 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2684354624..2685403199 20 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2818572352..2819620927 21 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 2952790080..2953838655 22 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 3087007808..3088056383 23 (64..1048639) 1048576<br>
> 10000<br>
> 0: [0..1048575]: 3221225536..3222274111 24 (64..1048639) 1048576<br>
> 10000<br>
<br>
</div></div>That's inode32 allocator behaviour (rotoring each new allocation<br>
across a different AG). Mount with inode64 - it's the default in the<br>
latest kernels - and it will behave as I demonstrated.<br>
<div class=""><div class="h5"><br>
Cheers,<br>
<br>
Dave.<br>
<br>
--<br>
Dave Chinner<br>
<a href="mailto:david@fromorbit.com">david@fromorbit.com</a><br>
</div></div></blockquote></div><br></div></div>