xfs
[Top] [All Lists]

Re: xfs for free hosting on linux ? performance questions

To: Linux XFS <xfs@xxxxxxxxxxx>
Subject: Re: xfs for free hosting on linux ? performance questions
From: pg_lxra@xxxxxxxxxxxxxxxxxxx (Peter Grandi)
Date: Mon, 10 Mar 2008 00:02:15 +0000
In-reply-to: <47D45926.5070806@xxxxxxxxx>
References: <47D45926.5070806@xxxxxxxxx>
Sender: xfs-bounce@xxxxxxxxxxx
[ ... 

> data will be in many directories, users files are in
> /var/www/users/username and i have more than 100 000
> accounts. also a policy to make max 3 MB files, so most of my
> files are around 50 KB i think.

This is a much better situation than the frequent inane one
where someone asks about using a filesystem as a database for
small records.

But a ''source files'' filesystem is still a bit of challenge.

> [ ... ] mount -o nobarrier,noatime,logbufs=8,logbsize=256k [
> ... ]

Even with backups, 'nobarrier' means begging for trouble.

> [ ... ] number of files 10k, transactions 10k, subdirs 20k,
> was giving very very slow reads, around 200KB/s, while
> reiser4/btrfs were like 2MB/s reads, but even ext3 was faster
> - any idea why? i tested on 2.6.24.3 and 2.6.25-rc4 ) [ ... ] 
> that 3 disks RAID-0 array with tweaked xfs is as fast as just
> one disk with reiser4 (my free hosting with millions of files)
> [ ... ]

Just use the by now very well tested ReiserFS version 3 for
that. Just as you would use XFS for bulk streaming of large
files.

Recently on the same 4x(1+1) RAID10 f2 I did some (single
threaded, single file reading and writing) tests involving
various file systems with both large files (2-12 2GB ones) and a
directory tree contain lots of small Java source files (6.5GB in
50k directories containing 150k files).

While most file systems could reach something like 250MB/s
writing and 430MB/s reading with the small set of large files
(with XFS a few dozen MB/s ahead of most of the others and would
have been more so on multithreaded), on the directory tree
ReiserFS did around 57MB/s, JFS around 36MB/s, and XFS around
20-25MB/s.

There are several reasons why ReiserFS is better suited for
large directory tree with lots of small files; one of them is
that it was designed for that :-).


<Prev in Thread] Current Thread [Next in Thread>