On Sat, Mar 26, 2016 at 03:09:27PM -0400, Andrew Ryder wrote:
Hello,
I have an mdadm array with a xfs v5 filesystem on it and its begun to give
me issues when trying to mount it as well as complete xfs_repair. Not sure
if anyone might be able to shed some light on what is going on or how to
correct the issue?
When I try and mount the fs, it complains with:
[ 388.479847] XFS (md2): Mounting V5 Filesystem
[ 388.494686] XFS (md2): metadata I/O error: block 0x15d6d39c0
("xlog_bread_noalign") error 5 numblks 8192
[ 388.495013] XFS (md2): failed to find log head
[ 388.495018] XFS (md2): log mount/recovery failed: error -5
[ 388.495090] XFS (md2): log mount failed
So a read I/O error from the kernel...
This is where its not making any sense for me, If I try and run "xfs_repair
/dev/md2" it fails with:
Phase 1 - find and verify superblock...
- reporting progress in intervals of 15 minutes
Phase 2 - using internal log
- zero log...
xfs_repair: read failed: Input/output error
failed to find log head
zero_log: cannot find log head/tail (xlog_find_tail=-5)
fatal error -- ERROR: The log head and/or tail cannot be discovered. Attempt
to mount the
filesystem to replay the log or use the -L option to destroy the log and
attempt a repair.
... similar read error from xfsprogs...
But if I run "xfs_repair -L /dev/md2" which gives:
Phase 1 - find and verify superblock...
- reporting progress in intervals of 15 minutes
Phase 2 - using internal log
- zero log...
xfs_repair: read failed: Input/output error
failed to find log head
zero_log: cannot find log head/tail (xlog_find_tail=-5)
xfs_repair: libxfs_device_zero write failed: Input/output error
... and it looks like it fails to write as well when trying to zero the
log...
then try and re-run "xfs_repair /dev/md2" it starts traversing the
filesystem all the way to "Phase 7" then errors with:
Phase 7 - verify and correct link counts...
- 14:36:55: verify and correct link counts - 33 of 33 allocation
groups done
Maximum metadata LSN (64:2230592) is ahead of log (0:0).
Format log to cycle 67.
xfs_repair: libxfs_device_zero write failed: Input/output error
Yet at this point I can now mount the filesystem..
... and this is effectively a repeat of the write error as we try to
format the log with a correct LSN based on the metadata LSN tracked by
the repair process. Your kernel is old enough that runtime probably
won't complain either way (note that 3.19 might be considered a fairly
early kernel for using CRC support). Perhaps the first write attempt
zeroed enough of the log before it failed that log recovery wasn't
required, and thus these problematic I/Os were avoided.
What's the history of this fs? Has it been working for some time, just
recently formatted? What lead to the need for log recovery? What's the
mdadm --detail info, member device size, total array size, xfs_info of
the filesystem, etc..?
Does xfs_repair run clean at this point? If so, does 'xfs_repair -L'
still reproduce the write error (note that I'm assuming you have a clean
log such that this command will not cause data loss). If so, an strace
of the repair process might be interesting...
Brian
Checking the drives with smartctl shows no errors nor does 'dmesg' show any
hardware i/o or controller related errors...
I've tried scrubbing the array and no bad sectors are found either..
I'm running kernel 3.19.8 with xfsprogs 4.5.
Thanks,
Andrew
_______________________________________________
xfs mailing list
xfs@xxxxxxxxxxx
http://oss.sgi.com/mailman/listinfo/xfs
