Data corruption, md5 changes on every mount

[Dmitry Panov]

Hi guys,

I have a 2TiB XFS which is about 60% full. Recently I've noticed that 
the daily inc. backup reports file contents change for files that are 
not supposed to change.

I've created an LVM snapshot and ran xfs_check/xfs_repair. xfs_check did 
report a few problems (unknown node type). After that I ran a simple 
test: mount, calculate md5 of the problematic files, report if it 
changed, umount, sleep 10 sec. That script reported that md5 sum of at 
least one file was changing on every cycle.

Analyzing the differences I found that a 4k block that should contain 
all zeros sometimes contains random garbage (luckily most of the files 
are pcm wavs, so it's easy to verify). However I did not analyze every 
occurrence so this may be not 100% true. The files do not look as they 
are sparse according to du. Interestingly one of them appears to occupy 
one block more than necessary.

Then I did cp -a file newfile, mv newfile file and re-ran the test. No 
problems reported since.

As there were a few unclean umounts I think most likely it is a 
filesystem corruption that went unspotted by xfs_repair. It would not 
surprise me too much because xfs_repair took just 3.5 min.

Any ideas? I could just copy the files and pretend noting happened but 
is there a guarantee that doing so won't corrupt other data?


Best regards,

-- 
Dmitry Panov