Corruption on ext3 with XFS kernel

[Simon Matter <simon.matter@ch.sauter-bc.com>]

Hi,

I have seven IBM deathstar 60G disks here which have been replaced. I
want to use four of them in my home server so I wanted to do some tests
to make sure they don't break in the first week. I have connected four
drives to a Promise Ultra100TX2 IDE controller and put it into my test
server. The box is running kernel-2.4.18-18SGI_XFS_1.2pre3. I have
created a softraid5 on the disks creating a 180G device. I then created
an ext3 fs with default settings. Mounted the fs on /mnt/md9, mounted my
real servers data on /mnt/nfs. Then I used cp -a /mnt/nfs
/mnt/md9/nfs[n] five times creating five identical copies of the nfs
mounted data (/mnt/md9/nfs1, /mnt/md9/nfs2...) with a size of 28G in
16500 file in each copy (143G used / 81%).
Then I did two times five diff's running at the same time and I was very
surprised what came out:

Test with ext3, first run:
[root@crash root]# diff -r /mnt/md9/nfs4 /mnt/md9/nfs5
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs3
[root@crash root]# diff -r /mnt/md9/nfs3 /mnt/md9/nfs4
[root@crash root]# diff -r /mnt/md9/nfs3 /mnt/md9/nfs5
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs3
Binary files
/mnt/md9/nfs2/Linux/StarOffice/soa-5_2-ga-bin-windows-de.exe and
/mnt/md9/nfs3/Linux/StarOffice/soa-5_2-ga-bin-windows-de.exe differ

Test with ext3, second run:
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs3
[root@crash root]# diff -r /mnt/md9/nfs1 /mnt/md9/nfs2
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs4
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs1
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs2
Binary files
/mnt/nfs/mp3/classic/joseph_haydn-symphonies_101_103.track05.mp3 and
/mnt/md9/nfs2/mp3/classic/joseph_haydn-symphonies_101_103.track05.mp3
differ

Huh! The first 'diff -r /mnt/md9/nfs2 /mnt/md9/nfs3' showed a difference
in a file while the second run didn't. The second run showed another
difference!!

Now, I tried using XFS to see whether it happens there too. I created an
XFS filesystem on the same raid5 volume and did exactly the same steps.

Test with XFS, first run:
[root@crash root]# diff -r /mnt/md9/nfs4 /mnt/md9/nfs5
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs3
[root@crash root]# diff -r /mnt/md9/nfs3 /mnt/md9/nfs4
[root@crash root]# diff -r /mnt/md9/nfs3 /mnt/md9/nfs5
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs3

Test with XFS, second run:
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs3
[root@crash root]# diff -r /mnt/md9/nfs1 /mnt/md9/nfs2
[root@crash root]# diff -r /mnt/md9/nfs2 /mnt/md9/nfs4
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs1
[root@crash root]# diff -r /mnt/nfs /mnt/md9/nfs2

Now, it looks like a problem with ext3. My question is, could it have
_something_ to do with the XFS patch in this kernel? Did anybody do
similar tests? Unfortunately this box has XFS root so I can't just
switch to vanilla or original RedHat kernel and every test takes me
~1day. Is there a way to find out what's going wrong here?

Simon