xfs
[Top] [All Lists]

Re: XFS: I/O Error Detected / 2.6.27.39

To: Piotr Kandziora <piotr.kandziora@xxxxxxxxxx>
Subject: Re: XFS: I/O Error Detected / 2.6.27.39
From: Eric Sandeen <sandeen@xxxxxxxxxxx>
Date: Tue, 16 Nov 2010 15:54:06 -0600
Cc: xfs@xxxxxxxxxxx, Artur Piechocki <artur.piechocki@xxxxxxxxxx>, Janusz Bak <jb@xxxxxxxxxx>, lukasz.wittig@xxxxxxxxxx
In-reply-to: <4CE282DB.8060200@xxxxxxxxxx>
References: <4CE282DB.8060200@xxxxxxxxxx>
User-agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6
On 11/16/10 7:10 AM, Piotr Kandziora wrote:
> Hi,
> 
> Our environment is following:
> - we have 24GB RAM,
> - we are using 3ware controller (and it does not report any errors),
> - we have one big logical volume (20TB) exported via NFS with large amount of 
> small files (about 150k),
> - we are doing periodically backup of this logical volume using rsync to 
> another server.
> - we have kernel 2.6.27.39,
> 
> Unfortunately our system is freezing unexpectedly without reason. We started 
> investigating this problem and noticed that cache memory is slowly increasing.
> 
> We tried to dump this cache memory using:
> /bin/echo "3" > /proc/sys/vm/drop_caches
> 
> In a result, cache was dumped, but in logs we noticed a lot of errors with 
> XFS:
> 
> [kern.warning] kernel: xfs_iunlink_remove: xfs_inotobp()  returned an error 
> 22 on dm-16.  Returning error.

so this got EINVAL:

#define EINVAL          22      /* Invalid argument */

> [kern.notice] kernel: xfs_inactive:\011xfs_ifree() returned an error = 22 on 
> dm-16
> [kern.notice] kernel: xfs_force_shutdown(dm-16,0x1) called from line 1406 of 
> file fs/xfs/xfs_vnodeops.c.  Return address = 0x
> [kern.alert] kernel: Filesystem \"dm-16\": I/O Error Detected.  Shutting down 
> filesystem: dm-16

which shut down the filesystem:

> [kern.alert] kernel: Please umount the filesystem, and rectify the problem(s)

leading to IO errors after that:

#define EIO              5      /* I/O error */

> [kern.warning] kernel: xfs_imap_to_bp: xfs_trans_read_buf()returned an error 
> 5 on dm-16.  Returning error.
> [kern.warning] kernel: xfs_imap_to_bp: xfs_trans_read_buf()returned an error 
> 5 on dm-16.  Returning error.
> [kern.warning] kernel: xfs_imap_to_bp: xfs_trans_read_buf()returned an error 
> 5 on dm-16.  Returning error.
> 
> We are wondering if this is problem connected to hardware or rather this is 
> XFS problem (if yes, was it fixed?).

I'd probably start with xfs_repair (with -n if you want a dry run)
and see if it finds any metadata corruption, first.

-Eric
 
> Best regards
> Piotr K
> 
> _______________________________________________
> xfs mailing list
> xfs@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/xfs
> 

<Prev in Thread] Current Thread [Next in Thread>