xfs
[Top] [All Lists]

Re: Defrag pitfals (was Re: Support for XFS File systems)

To: Bogdan Costescu <bogdan.costescu@xxxxxxxxxxxxxxxxxxxxx>
Subject: Re: Defrag pitfals (was Re: Support for XFS File systems)
From: Chris Wedgwood <cw@xxxxxxxx>
Date: Wed, 12 Feb 2003 10:25:00 -0800
Cc: linux-xfs@xxxxxxxxxxx
In-reply-to: <Pine.LNX.4.44.0302121146510.4728-100000@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
References: <20030211201735.GA6124@xxxxxxxx> <Pine.LNX.4.44.0302121146510.4728-100000@xxxxxxxxxxxxxxxxxxxxxxxxxxx>
Sender: linux-xfs-bounce@xxxxxxxxxxx
User-agent: Mutt/1.3.28i
On Wed, Feb 12, 2003 at 11:54:12AM +0100, Bogdan Costescu wrote:

> Could you (or somebody else) please elaborate on this ?

Short answer:

   xfs_fsr can (and usually does) destroy locality of files.


Longer answer:

xfs_fsr, when run without a filename (or list of filenames), does a
bulkstat of the filesystem and finds candidate inodes to defragment.
bulkstat avoid having to do a tree-traversal and it's pretty fast for
the most part.

xfs_fsr defragments the files by creating a new (temporary) file,
copying the data from the fragmented file to the new file swapping the
extents before releasing the temporary file.

The problem with this is that it can destroy locality of files:
consider a directory of source code for example, most of the files are
*near* each other (xfs_bmap -v will show this) as XFS tries to group
files in same directory into the same AG as the parent directory[1].

Now, because xfs_fsr doesn't actually create the new file in the
directory of the file it's going to replace, the filesystem can't try
to place the new (temporary) file near the others and it may end up
being located somewhere very different.

Since XFS does a pretty good job at preventing fragmentation, for
things like large trees of source code it appears better to have the
odd file (slightly) fragmented that the odd file scattered elsewhere
on disk.

Now, if you have a small number of directories with fragmented files
and want to defragment the fragmented files and *NOT* ruin file
locality, you can try "xfs_fsr /path/to/dir/*" and see how that fares.

> I've switched several file systems to XFS exactly because of the
> feature of defragmenting while the FS was still mounted. I've tried
> to find this kind of information prior to switching to XFS, but I
> failed... empirical evidence suggests that defragmentation helps a
> lot in my case, but I would really like to be aware of situations
> when this feature would become detrimental.

I'm curious as to why it helps so much in your case... what are you
doing that causes lots of fragmentation?  Does your disk ever get
really full?  Do you do lots of synchronous/or direct writes?  Do lots
of NFS clients access your filesystem?

As a general rule, XFS doesn't fragment all that badly.  On my desktop
for /home, which is pretty active (thousand of files created and
deleted every day), I have about 240 fragmented files from over almost
800,000 files.



Thanks,

 --cw

[1] Actually, I'm a bit vague on this and too lazy to check the source
    right now, so maybe I'm wrong in which case someone can
    correct me...


<Prev in Thread] Current Thread [Next in Thread>