Hi Michal!
Michal Szymanski wrote:
On Tue, Oct 26, 2004 at 12:05:30PM +0200, Frank Hellmann wrote:
Similar effects here. Basically we have the same setup (Hardware RAID5
with Software RAID0 striped together). I ran into some problems during a
xfs_repair. See these articles:
http://marc.theaimsgroup.com/?l=linux-xfs&m=109714077512194&w=2
http://marc.theaimsgroup.com/?l=linux-xfs&m=109719162327429&w=2
Not to good news, I am afraid. We are currently working on a scheme to
cache our files into multiple directories instead of the filesystem
root. That as far as I can tell helps to avoid this issue. So you
shouldn't run into these during xfs_repair.
Concering xfs_check... anyone?
Well, I am even more confused now. I got "attracted" to XFS by reading a
comparison between XFS and EXT3 which stated that EXT3 on a huge
partition is a memory and time hog.
Don't be confused. XFS is a giant leap forward, at least from my view.
Remember the Large Block Device (2+ TB devices) support is working
"correctly" since Kernel 2.6.6. So there might be a few issues that have
not showed up until recently.
Indeed, I have several hardware RAID
arrays sliced into sub-2TB partitions with EXT3 FS, and it takes a few
hours to complete a filesystem check after a crash. Well, but it
completes. Now the XFS, advertized as a true 64-bit FS, capable of
dealing with Petabytes, seems to be unable to check an almost empty
partition and/or repair a partition with several hundred thousand files.
We have two production servers running a similar setup. We work with
film images that are usually 10MB-60MB in size and about 130.000 files
minimum. Except for that one issue (putting everything into the
filesystem root) this works really well, has good performance and is
stable. We have way more issues with the nvidia drivers, than anything else.
It seems to me, that xfs_check is running into some wrap-around bug at
the 2TB limit and just spits out the "out of memory" error. I don't have
sufficent knowledge to fix this (IMHO there is a bigger chance of me
breaking it completly).
On the other hand, I've read a page "Who's using XFS" with some
important projects (like Sloan Digital Sky Survey) generating huge
amounts of data. Do they also have partitions that they cannot check/repair?
I am guessing that they use IRIX and that is known to support larger
devices since quiet some time. I guess the user-tools will just catch up
with the new kernel features soon. As I said LBD is known to be working
since only a _few_ month.
The oss.sgi.com/projects/xfs pages content suggests that actually 'fcsk'
is not needed anymore on a journalling FS like XFS. So maybe we can just
live without it?
No. There will be times, you'll need it. Powerloss is never going to
give you predictable results.
To resume, could someone tell me whether I can safely put my data on a
3.6TB XFS filesystem or not? The machine it is currently attached to
has 4GB RAM + 2GB Swap. If this is not enough for XFS to do a
check/repair, I would say it is not a solution for me.
As I said, in a very unlikely event (about 500.000 files in the fs-root,
+ Nvidia driver crashing the machine) there is an issue repairing it.
I all other cases we never had an issue with xfs and the xfs_repair. And
we have to run xfs_repair at least once a week due to the nvidia crashes
and hard resets.
regards, Michal.
PS. I've just found, on http://oss.sgi.com/projects/xfs/irix-linux.html
Linux XFS filesystems are limited to 2 Terabytes in size due to
limitations in the Linux block device I/O layers.
Is it just an out-of-date page? Or, maybe, it is just the true reason of
our problems?
A bit outdated I would say. But maybe someone else would like to comment
on the LBD stability and sanity?
Cheers,
Frank...
--
--------------------------------------------------------------------------
Frank Hellmann Optical Art GmbH Waterloohain 7a
DI Supervisor http://www.opticalart.de 22769 Hamburg
frank@xxxxxxxxxxxxx Tel: ++49 40 5111051 Fax: ++49 40 43169199
|