xfs
[Top] [All Lists]

Re: specify agsize?

To: Eric Sandeen <sandeen@xxxxxxxxxxx>
Subject: Re: specify agsize?
From: aurfalien <aurfalien@xxxxxxxxx>
Date: Sun, 14 Jul 2013 09:46:58 -0700
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=agTU8o85V8Jlx5eV1qb68C7mdKt2zULhcPpcwEwFK7Q=; b=OLI0f6eOAuRAwIatGjOzPbd4i9v+k29dLAsvfZuelITZZ2EhgPaoqn700WqMXYJITC 8/QFEiMC+1Sg087dlA42n94fW88+hgBsYDOImVk3C0UVZM8s3DXISfBY9rGGp6XabLF2 ZVEjLomb7dZDDJfikvmDlMpXedACaBCTRQXrHvmemet7ZoU/pRV2DhTHVZ0O6BfIx2Dr hWm4jiBjM5Q4d6S6/7HMWGigrr+mgjYk7BJlJMNH4qdKIS63xgE/CUsvWLw6WV1ojkjY sGpNCAGdwb5GHvKEDCxVatQTAJ0snkNFu7HTEBMd+rxCuGoZCnrFIy+LOZqZX4s5ooPb Hgsw==
In-reply-to: <51E2CE83.9080003@xxxxxxxxxxx>
References: <6A14EB72-A699-47AF-937D-D6DA1CF12ACB@xxxxxxxxx> <51E2092D.7090409@xxxxxxxxxxx> <9AB8D1D3-29D7-4C43-A624-37024CA4EFD9@xxxxxxxxx> <51E2CE83.9080003@xxxxxxxxxxx>
Sorry to top post.

But this was exactly the kind of info I was hoping for.

Thanks Eric.

- aurf

On Jul 14, 2013, at 9:14 AM, Eric Sandeen wrote:

> On 7/13/13 11:20 PM, aurfalien wrote:
>> 
>> On Jul 13, 2013, at 7:13 PM, Eric Sandeen wrote:
>> 
>>> On 7/13/13 7:11 PM, aurfalien wrote:
>>>> Hello again,
>>>> 
>>>> I have a Raid 6 x16 disk array with 128k stripe size and a 512 byte block 
>>>> size.
>>>> 
>>>> So I do;
>>>> 
>>>> mkfs.xfs -f -l size=512m -d su=128k,sw=14 
>>>> /dev/mapper/vg_doofus_data-lv_data
>>>> 
>>>> And I get;
>>>> 
>>>> meta-data=/dev/mapper/vg_doofus_data-lv_data isize=256    agcount=32, 
>>>> agsize=209428640 blks
>>>>        =                       sectsz=512   attr=2, projid32bit=0
>>>> data     =                       bsize=4096   blocks=6701716480, imaxpct=5
>>>>        =                       sunit=32     swidth=448 blks
>>>> naming   =version 2              bsize=4096   ascii-ci=0
>>>> log      =internal log           bsize=4096   blocks=131072, version=2
>>>>        =                       sectsz=512   sunit=32 blks, lazy-count=1
>>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>>>> 
>>>> 
>>>> All is fine but I was recently made aware of tweaking agsize.
>>> 
>>> Made aware by what?  For what reason?
>> 
>> Autodesk has this software called Flame which requires very very fast
>> local storage using XFS. They have an entire write up on how to calc
>> proper agsize for optimal performance.
> 
> http://wikihelp.autodesk.com/Creative_Finishing/enu/2012/Help/05_Installation_Guides/Installation_and_Configuration_Guide_for_Linux_Workstations/0118-Advanced118/0194-Manually194/0199-Creating199
> 
> I guess?
> 
> That's quite a procedure!  And I have to say, a slightly strange one at first 
> glance.
> 
> It'd be nice if they said what they were trying to accomplish rather than 
> just giving you a long recipe.
> 
> In the end, I think they are trying to create 128AGs and maybe work around 
> some mkfs corner case or other.
> 
>> I never mess with agsize but it is require  when creating the XFS
>> file system for use with Flame.  I realize its tailored for there
>> apps particular IO characteristics, so I'm curious about it.
> 
> In general more AGs allow more concurrency for some operations;
> it also will generally change how/where files in multiple directories get
> allocated.
> 
>>>> So I would like to mess around and iozone any diffs between the above
>>>> agcount of 32 and whatever agcount changes I may do.
>>> 
>>> Unless iozone is your machine's normal workload, that will probably prove 
>>> to be uninteresting.
>> 
>> Well, it will give me a base line comparison of non tweaked agsize vs 
>> tweaked agsize.
> 
> Not necessarily, see above; I'm not sure what iozone invocation would
> show any effects from more or fewer AGs.  Anyway, iozone != flame, not
> by a long shot! :)
> 
>>>> I didn't see any mention of agsize/agcount on the XFS FAQ and would
>>>> like to know, based on the above, why does XFS think I have 32
>>>> allocation groups with the corresponding size?
>>> 
>>> It doesn't think so, it _knows_ so, because it made them itself.  ;)
>> 
>> Yea but based on what?
>> 
>> Why 32 at there current size?
> 
> see calc_default_ag_geometry()
> 
> Since you are in multidisk mode (you have stripe geometry) it uses more AGs 
> for more AGs since it knows you have more spindles:
> 
>        } else if (dblocks > GIGABYTES(512, blocklog))
>                shift = 5;
> 
> 2^5 = 32
> 
> If you hadn't been in multidisk mode you would have gotten 25 AGs due to the 
> max AG size of 1T.
> 
>>>> And are these optimal
>>>> numbers?
>>> 
>>> How high is up?
>>> 
>>> Here's the appropriate faq entry:
>>> 
>>> http://xfs.org/index.php/XFS_FAQ#Q:_I_want_to_tune_my_XFS_filesystems_for_.3Csomething.3E
>> 
>> Problem is I run Centos so the line;
>> 
>> "As of kernel 3.2.12, the default i/o scheduler, CFQ, will defeat much of 
>> the parallelization in XFS. "
>> 
>> ... doesn't really apply.
> 
> Well, my point was that your original question, "are these optimal numbers?" 
> included absolutely no context of your workload, so the best answer is yes - 
> the default mkfs behavior is optimal for a generic, unspecified workload.
> 
> I don't have access to Autodesk Flame so I really don't know how it behaves 
> or what an optimal tuning might be.
> 
> Anyway, I think the calc_default_ag_geometry() info above answered your 
> original question of "why does XFS think I have 32 allocation groups with the 
> corresponding size?" - that's simply the default mkfs algorithm when in 
> multidisk mode, for a disk of this size.
> 
> -Eric
> 

<Prev in Thread] Current Thread [Next in Thread>