> We are looking to migrate some existing advfs partitions on Compaq Tru64
> Unix 4.0e to xfs partitions on linux and have observed extremely weird
> problems in copying the files. The different scenarios and outcomes are
> outlined below.
>
> Test environment
>
> Linux Server: Dell PowerEdge 4400 with hardware RAID on aacraid driver. OS
> RedHat71. Tried with both SGI supplied kernels and 2.4.9 kernels.
>
> Alpha Server: AlphaStation 200, Compaq Tru64 4.0e
>
> Tar: Gnu tar on both systems (sym link on linux to /bin/tar to provide same
> calling convention).
>
> Test data: ex-users data - contains mix of usual stuff
>
> TEST 1 - XFS
>
> The xfs file system is NFS mounted on Alpha (push from the Alpha)
>
> # cd /Ufs/servbg/users
> # /bin/gnutar cfS - test | (cd /mnt/users; /bin/gnutar xpfS - )
>
> Result: test directory expands from 21987 on advfs to 86224 on xfs (also,
> the size of the data is initially higher and drops to this value a short
> time after the copy is finished).
This is caused by something called the NFS reference cache within XFS,
basically XFS allocated more space than you ask for in anticipation of
future writes. The excess space is cleaned up at close time in the
normal case. NFS effectively does an open/close around each write and
this leads to truncation of the excess space at the end of each write.
Because of this XFS keeps a cache of files on which NFS writes have been
performed, the reference count on these is kept high. Entries are purged
from the cache as new files are written and also over time. I suspect
that the latter half of the cache flushing is not working on Linux,
especially after recent kernel changes. If you unmounted and remounted
the filesystem the space would be recovered. We will look into the
cache code, it does sound broken.
The du space reporting is being influenced by the same cache.
Steve
p.s. trust me you do want the cache, adding it increased XFS performance
under NFS on linux by about an order of magnitude.
>
> TEST 2 - XFS
>
> The advfs file system on the Alpha is NFS mounted on the Linux machine (pull
> from the Linux box).
>
> # cd /mnt/users
> # /bin/gnutar cfS - test | (cd /Ufs/servb00/users; /bin/gnutar xpfS -)
>
> Result: test directory expands from 21987 on advfs to 23860 on local xfs
> partition /Ufs/servb00 - this is probably a result of the differing block
> sizes so OK.
>
> TEST 3 - EXT2
>
> The ext2 file system is NFS mounted on Alpha (push from the Alpha)
>
> # cd /Ufs/servbg/users
> # /bin/gnutar cfs - test | (cd /mnt; /bin/gnutar xpfS - )
>
> Result : test directory contracts from 21987 on advfs to 21714 on ext2
>
> TEST 4 - EXT2
>
> The advfs file system on the Alpha is NFS mounted on the Linux machine (pull
> from the Linux box).
>
> # cd /mnt/users
> # /bin/gnutar cfS - test | (cd /tmp; /bin/gnutar xpfS -)
>
> Result: test directory contracts from 21987 on advfs to 21313 on ext2
>
> PROBLEMS:
>
> The most worrying problem for us is that remotely mounting an XFS directory
> to a non-linux machines appears to result in quite excessive use of
> diskspace. Looking at the test data, we found that (for example), a
> .netscape/cache directory would grow from 9M to 33M. Is this type of
> behaviour a known problem and is it likely to bite during normal operation
> of the filesystem when nfs-mounted.
>
> The usage reported by du on the linux server for the test directory shows
> erratic value. During the copy, it rises to a higher value and then
> decreases a bit - this seems counter intuitive. Indeed, different runs with
> the same test data would show slight variations in resultant disk space used
> for TEST 1.
>
> thanks in advance,
>
> Bret
> --
> Bret Giddings, Systems Manager, Computing Service, University of Essex
> Tel: (01206) 872577 Email: bret@xxxxxxxxxxx Fax: (01206) 860585
|