XFS corruption - Ubuntu 14.04 VM with RDM

[Dave Chinner]

On Mon, Nov 30, 2015 at 10:48:45PM +0200, Amir Soroka wrote:
> Hello,
> We have a corruption issue. Would be happy to get a solution.
> 
> Configuration is Ubuntu 14.04 (generic) as a VM over ESXi 5.5, with 1.6 TB
> RDM - storage has a Cache battery, Raid 6, SSD disks.

Please reproduce on bare metal, and with a recent kernel.

> During some heavy indexing activity we get a corruption as you can see
> below.

Doesn't tell me anything about the workload. Please include:

http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F

> Some more information:
> * Mount options: defaults,noatime,nodiratime,nobarrier,inode64,logbufs=8 0 0
> * XFS was created like this: mkfs.xfs -f -d su=131072,sw=8 -i size=1024
> /dev/sdb1
> * Parted command was: /sbin/parted -s $i mklabel gpt mkpart /dev/sdb1 xfs
> 2048s 100%
> 
> 
> [   10.395575] XFS (sdb1): Mounting V4 Filesystem
> [   10.430107] XFS (sdb1): Ending clean mount
> [  675.504000] XFS (sdb1): Metadata corruption detected at
> xfs_agf_read_verify+0x61/0x100 [xfs], block 0x1

There should be lots more error logging than that. Where are the
stack traces, the hex dumps, etc that go along with normal
corruption reports?

> [84997.869987] XFS (sdb1): Metadata corruption detected at
> xfs_inode_buf_verify+0x7d/0xe0 [xfs], block 0x110

There's more than one type of corruption - I'd suggest that you have
a vmware/esx level problem....

> [84997.871163] XFS (sdb1): Unmount and run xfs_repair
> [84997.871760] XFS (sdb1): First 64 bytes of corrupted metadata buffer:
> [84997.874798] XFS (sdb1): metadata I/O error: block 0x110

Yup, you've removed all the detailed corruption information in the
output. i.e. all the bits that might tell us what went wrong.
Please post the logs *in full*.

Cheers,

Dave.
-- 
Dave Chinner
david at fromorbit.com