On Wed, 2003-02-12 at 14:16, Chris Wedgwood wrote:
> On Wed, Feb 12, 2003 at 09:07:21PM +0100, Bogdan Costescu wrote:
>
> > [Maybe OT] Inbetween these messages, I talked to a friend about a
> > networking problem that he has as part of a P2P network.
>
> Applications which write to multiple 'parts' of the file
> simultaneously (ie. reading different parts of the file from different
> foreign hosts) tend to cause bad fragmentation. I'm not sure if this
> is better or worse with other filesystems.
>
> > Then made the connection that the way files are stored when
> > downloaded from such network is very similar to our situation
> > (talking here only about large stuff like ISO images, movies and
> > whatnot) - files are written to disk over several hours, days or
> > weeks, depending on Internet link speed and availability.
>
> Slowly writing to disk over a long period of time whilst there is
> other disk activity increases the likely hood of fragmentation. A
> good example here is log files.
>
> It's even worse if you write pattern means you open and close the file
> between writes. If this isn't the case you could in theory tune the
> preallocation on the filesystem to make it greater and see if that
> helps (I tried it here with positive results).
>
> > The difference might be in appended vs. random writting mode; for
> > the networks that support chunked file transfer they are probably
> > written as sparse files.
>
> Writing to sparse files randomly bites. Without application level
> changes this is pretty hard to do much about.
>
> > Then the files have to be read for writting to CD (ISO images),
> > played (movies) etc., so they have the same problem.
>
> A small mount of fragmentation isn't really that bad... I guess it
> depends on the average extent size and their physical layout on disk
> and to how hard the drive works to access this data.
>
A thought occurrs to me here, xfs will not drop space out beyond eof
if XFS_DIFLAG_PREALLOC is set in the inode flags. Right now the only
way to get this is to use the preallocate call.
If you look in xfs_setattr you can see some code like this:
if (mask & XFS_AT_XFLAGS) {
ip->i_d.di_flags = 0;
if (vap->va_xflags & XFS_XFLAG_REALTIME) {
ip->i_d.di_flags |= XFS_DIFLAG_REALTIME;
/* This is replicated in the io core for
* CXFS use
*/
ip->i_iocore.io_flags |= XFS_IOCORE_RT;
}
/* can't set PREALLOC this way, just ignore it
*/
}
It would not be too hard to extend this to allow setting the prealloc
flag. There is an XFS_IOC_FSSETXATTR which would allow setting it.
We could extend the ioctl interface for xfs to take the
EXT2_IOC_SETFLAGS ioctl which is used by ext2,ext3 and reiserfs,
and use one of its bits.
We could then have files which would have less tendency to get
fragmented as you extended them over time. They still would get
chopped up, just less so than before.
Just a thought.
Steve
--
Steve Lord voice: +1-651-683-3511
Principal Engineer, Filesystem Software email: lord@xxxxxxx
|