xfs
[Top] [All Lists]

Tuning for tons of small files?

To: linux-xfs@xxxxxxxxxxx
Subject: Tuning for tons of small files?
From: Dave Lloyd <dlloyd@xxxxxxxxx>
Date: Wed, 19 Apr 2006 11:25:08 -0500
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: Thunderbird 1.5 (X11/20051201)
I'm trying to tune XFS for faster read and write times when accessing tons of small files. System is 2.6.16 on a 2x2 core opteron. Options that the fs was made with:
-f -d sunit=128,swidth=512,unwritten=1 -l version=2,sunit=128

Mount options:
noatime,largeio,ihashsize=131072,attr2,barrier,allocsize=1073741824

This is across four hardware raid0 filesystems. Right now the read and write performance for larger files (~8gb) just kills ext3, perhaps between 2-4x faster in some instances, but ext3 kills xfs for accessing lots of small files.

I've seen that tuning the logbufs up may help out, so that's my next step. I was also thinking that configuring the hardware raids as a 16 disk JBOD and making it a concat md volume may also distribute out the I/O as what I believe what is killing us here is seek time (of course, this is if md is smart and tells xfs how the dev is laid out so it can put the ags at the correct place to round-robin I/O to the different phys devs and I'm not so sure that md is that smart).

Any other ideas to tune XFS for better performance when accessing tons of small files?

--
Dave Lloyd
Test Engineer, Exegy, Inc.
314.450.5342
dlloyd@xxxxxxxxx

Attachment: dlloyd.vcf
Description: Vcard

<Prev in Thread] Current Thread [Next in Thread>
  • Tuning for tons of small files?, Dave Lloyd <=