Data corruption, md5 changes on every mount

[Dmitry Panov]

Hi Dave,

On 11/12/2011 23:53, Dave Chinner wrote:
> On Sun, Dec 11, 2011 at 01:21:37PM +0000, Dmitry Panov wrote:
>> Hi guys,
>>
>> I have a 2TiB XFS which is about 60% full. Recently I've noticed
>> that the daily inc. backup reports file contents change for files
>> that are not supposed to change.
> What kernel/platform? What version of xfsprogs? What kind of
> storage?
It's linux kernel 3.0.0 at the moment, however it used to run different 
versions and I can't tell for sure when the problem started. xfsprogs 
version is 3.1.2.

The storage is a 2 node cluster with hardware RAID1+0 and drbd.
>> I've created an LVM snapshot and ran xfs_check/xfs_repair. xfs_check
>> did report a few problems (unknown node type). After that I ran a
>> simple test: mount, calculate md5 of the problematic files, report
>> if it changed, umount, sleep 10 sec. That script reported that md5
>> sum of at least one file was changing on every cycle.
> That sounds like you've got a dodgy drive.

That would be my guess too, however the problem occurs on both nodes 
(i.e. it doesn't go away when the other node becomes active) and the 
same files affected which makes hard drives or RAID controller or RAM 
failure very unlikely.


Is there any way to perform a more thorough check, than xfs_check does?


Best regards,

-- 
Dmitry Panov