xfs
[Top] [All Lists]

Re: Data corruption, md5 changes on every mount

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: Data corruption, md5 changes on every mount
From: Dmitry Panov <dmitry.panov@xxxxxxxxxxx>
Date: Mon, 12 Dec 2011 00:13:40 +0000
Cc: xfs@xxxxxxxxxxx
In-reply-to: <20111211235334.GJ14273@dastard>
References: <4EE4AE61.6000306@xxxxxxxxxxx> <20111211235334.GJ14273@dastard>
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:8.0) Gecko/20111105 Thunderbird/8.0
Hi Dave,

On 11/12/2011 23:53, Dave Chinner wrote:
On Sun, Dec 11, 2011 at 01:21:37PM +0000, Dmitry Panov wrote:
Hi guys,

I have a 2TiB XFS which is about 60% full. Recently I've noticed
that the daily inc. backup reports file contents change for files
that are not supposed to change.
What kernel/platform? What version of xfsprogs? What kind of
storage?
It's linux kernel 3.0.0 at the moment, however it used to run different versions and I can't tell for sure when the problem started. xfsprogs version is 3.1.2.

The storage is a 2 node cluster with hardware RAID1+0 and drbd.
I've created an LVM snapshot and ran xfs_check/xfs_repair. xfs_check
did report a few problems (unknown node type). After that I ran a
simple test: mount, calculate md5 of the problematic files, report
if it changed, umount, sleep 10 sec. That script reported that md5
sum of at least one file was changing on every cycle.
That sounds like you've got a dodgy drive.

That would be my guess too, however the problem occurs on both nodes (i.e. it doesn't go away when the other node becomes active) and the same files affected which makes hard drives or RAID controller or RAM failure very unlikely.


Is there any way to perform a more thorough check, than xfs_check does?


Best regards,

--
Dmitry Panov

<Prev in Thread] Current Thread [Next in Thread>