Received: with ECARTIS (v1.0.0; list xfs); Mon, 24 Sep 2007 13:43:43 -0700 (PDT) X-Spam-Checker-Version: SpamAssassin 3.2.0-pre1-r499012 (2007-01-23) on oss.sgi.com X-Spam-Level: X-Spam-Status: No, score=0.1 required=5.0 tests=AWL,BAYES_00,J_CHICKENPOX_32, J_CHICKENPOX_43,SPF_HELO_PASS autolearn=no version=3.2.0-pre1-r499012 Received: from lucidpixels.com (lucidpixels.com [75.144.35.66]) by oss.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id l8OKhaQ3014419 for ; Mon, 24 Sep 2007 13:43:39 -0700 Received: by lucidpixels.com (Postfix, from userid 1001) id 695471C000260; Mon, 24 Sep 2007 16:43:39 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by lucidpixels.com (Postfix) with ESMTP id 64B4B4019B1E; Mon, 24 Sep 2007 16:43:39 -0400 (EDT) Date: Mon, 24 Sep 2007 16:43:39 -0400 (EDT) From: Justin Piszcz X-X-Sender: jpiszcz@p34.internal.lan To: Ralf Gross cc: linux-xfs@oss.sgi.com Subject: Re: mkfs options for a 16x hw raid5 and xfs (mostly large files) In-Reply-To: <20070924203958.GA4082@p15145560.pureserver.info> Message-ID: References: <20070923093841.GH19983@p15145560.pureserver.info> <20070924173155.GI19983@p15145560.pureserver.info> <20070924203958.GA4082@p15145560.pureserver.info> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: ClamAV version 0.90, clamav-milter version devel-120207 on oss.sgi.com X-Virus-Status: Clean X-archive-position: 13072 X-ecartis-version: Ecartis v1.0.0 Sender: xfs-bounce@oss.sgi.com Errors-to: xfs-bounce@oss.sgi.com X-original-sender: jpiszcz@lucidpixels.com Precedence: bulk X-list: xfs On Mon, 24 Sep 2007, Ralf Gross wrote: > Justin Piszcz schrieb: >>> A bit ot: will I waste space on the RAID device with a 256K chunk size >>> and small files? Or does this only depend on the block size of the fs >>> (4KB at the moment). >> >> That's a good question, I believe its only respective of the filesystem >> size, but will wait for someone to confirm, nice benchmarks! >> >> I use a 1 MiB stripe myself as I found that to give the best performance. > > 256KB is the largest chunk size I can choose for a raid set. BTW: the HW-RAID > is an Overland Ultamus 4800. > > The funny thing is, that performance (256KB chunks) is even better without > adding any sw/su option to the mkfs command. > > mkfs.xfs /dev/sdd1 -f > > Sequential Reads > File Blk Num Avg Maximum Lat% Lat% CPU > Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff > ----- ----- --- ------ ------ --------- ----------- -------- -------- ----- > 20000 4096 1 208.33 23.81% 0.055 49.55 0.00000 0.00000 875 > 20000 4096 2 199.48 43.72% 0.116 376.85 0.00000 0.00000 456 > > Random Reads > File Blk Num Avg Maximum Lat% Lat% CPU > Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff > ----- ----- --- ------ ------ --------- ----------- -------- -------- ----- > 20000 4096 1 2.83 0.604% 4.131 38.81 0.00000 0.00000 469 > 20000 4096 2 4.53 1.700% 4.995 67.15 0.00000 0.00000 266 > > Sequential Writes > File Blk Num Avg Maximum Lat% Lat% CPU > Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff > ----- ----- --- ------ ------ --------- ----------- -------- -------- ----- > 20000 4096 1 188.15 42.98% 0.047 7547.93 0.00027 0.00000 438 > 20000 4096 2 167.76 76.89% 0.100 7521.34 0.00078 0.00000 218 > > Random Writes > File Blk Num Avg Maximum Lat% Lat% CPU > Size Size Thr Rate (CPU%) Latency Latency >2s >10s Eff > ----- ----- --- ------ ------ --------- ----------- -------- -------- ----- > 20000 4096 1 2.08 0.869% 0.016 0.13 0.00000 0.00000 239 > 20000 4096 2 1.80 1.501% 0.020 6.28 0.00000 0.00000 12 > > > Ralf > > I find that to be the case with SW RAID (defaults are best) Although with 16 drives(?) that is awfully slow. I have 6 SATA's I get 160-180 MiB/s raid5 and 250-280 MiB/s raid 0 (sw raid). With 10 raptors I get ~450 MiB/s write and ~550-600 MiB/s read, again XFS+SW raid. Justin.