xfs_force_shutdown

[Eric Sandeen]

Hieu Le Trung wrote:
> Eric Sandeen wrote:
>> Hieu Le Trung wrote:
>>> Hi,
>>>
>>> What may cause metadata becomes bad? I got xfs_force_shutdown with
> 0x2
>>> parameter.
>> Software bugs or hardware problems.  If you provide the actual kernel
>> message we can offer more info on what xfs saw and why it shut down.
> 
> I'm not sure which one is it but the issue is hard to reproduce. 
> I have following in the dmesg but I'm not sure it's the right one
>   <1>I/O error in filesystem ("sda2") meta-data dev sda2 block 0xf054f4
> ("xlog_iodone") error 5 buf count 32768

Were there IO errors from the storage before this?  i.e. did some lower
layer go bad.

>   <5>xfs_force_shutdown(sda2,0x2) called from line 956 of file
> fs/xfs/xfs_log.c.  Return address = 0x801288d8
> 
> Furthermore, the driver's write cache is
>   <5>SCSI device sda: drive cache: write back

That's fine...

> The xfs_logprint shows 'Bad log record header'
> xfs_logprint: /dev/sda2 contains a mounted and writable filesystem
>     data device: 0x802
>     log device: 0x802 daddr: 15735648 length: 20480
> 
> Header 0xa4 wanted 0xfeedbabe
> **********************************************************************
> * ERROR: header cycle=164         block=14634                        *
> **********************************************************************
> Bad log record header
> 
> So I wonder what may cause bad record header?

Probably the IO errors when attempting to write to the log ...

>>> How can I analyze the metadata dump file?
>> the metadump file is just the metadata skeleton of the filesystem; you
>> can mount it, repair it, point xfs_db at it to debug it, etc.
> 
> Is there any tutorials or guideline in using xfs_db to debug the issue?

xfs_db has a manpage, but I'm not sure the answer will be found by using
it.  It will only look at what data made it to the disk, and you had an
IO error.

-Eric