I have a 2TiB XFS which is about 60% full. Recently I've noticed that
the daily inc. backup reports file contents change for files that are
not supposed to change.
I've created an LVM snapshot and ran xfs_check/xfs_repair. xfs_check did
report a few problems (unknown node type). After that I ran a simple
test: mount, calculate md5 of the problematic files, report if it
changed, umount, sleep 10 sec. That script reported that md5 sum of at
least one file was changing on every cycle.
Analyzing the differences I found that a 4k block that should contain
all zeros sometimes contains random garbage (luckily most of the files
are pcm wavs, so it's easy to verify). However I did not analyze every
occurrence so this may be not 100% true. The files do not look as they
are sparse according to du. Interestingly one of them appears to occupy
one block more than necessary.
Then I did cp -a file newfile, mv newfile file and re-ran the test. No
problems reported since.
As there were a few unclean umounts I think most likely it is a
filesystem corruption that went unspotted by xfs_repair. It would not
surprise me too much because xfs_repair took just 3.5 min.
Any ideas? I could just copy the files and pretend noting happened but
is there a guarantee that doing so won't corrupt other data?