[Top] [All Lists]

Re: XFS filesystem on EC2 instance corrupts and shuts down

To: Shrinath M <shrinath.m@xxxxxxxxxx>
Subject: Re: XFS filesystem on EC2 instance corrupts and shuts down
From: Ric Wheeler <rwheeler@xxxxxxxxxx>
Date: Wed, 06 Mar 2013 07:59:36 -0500
Cc: xfs@xxxxxxxxxxx, Sabyasachi Ruj <sabyasachi.ruj@xxxxxxxxxx>, Vivek Goel <vivek.goel@xxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <CAOdS1h=7X4O1O7X8YOwxtLm7G=fc+J+6hJxJ1RKbDmfTZXTpeg@xxxxxxxxxxxxxx>
References: <CAOdS1h=7X4O1O7X8YOwxtLm7G=fc+J+6hJxJ1RKbDmfTZXTpeg@xxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130219 Thunderbird/17.0.3
On 03/06/2013 03:07 AM, Shrinath M wrote:
We are experiencing a strange XFS corruption issue. If we look in /var/log/messages, it simply says -

Mar 5 01:14:33 ip-100-0-100-1 kernel: [14139930.248619] XFS (md0): Corruption detected. Unmount and run xfs_repair

It shuts down the filesystem after this. On rebooting, it calls xfs_repair automatically and everything comes back to normal. We have had 2 such occurrences till now, I am attaching the relevant parts of /var/log/messages here, assuming someone can enlighten me on whats going wrong.

Machine details are as follows -

We are using Amazon AMI version: Amazon Linux AMI release 2012.09 We are running 8 EBS volumes of 512 MB each, in RAID 0 Array. $~: uname -a Linux ip-100-0-100-1 3.2.34-55.46.amzn1.x86_64 #1 SMP Tue Nov 20 10:06:15 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Ask me if anyone wants any more details

I think that you would need to verify that the Amazon storage is not throwing errors - do your logs show IO errors or issues before XFS hits an issue?



<Prev in Thread] Current Thread [Next in Thread>