xfs
[Top] [All Lists]

XFS umount with IO errors seems to lead to memory corruption

To: xfs@xxxxxxxxxxx
Subject: XFS umount with IO errors seems to lead to memory corruption
From: Chris Holcombe <xfactor973@xxxxxxxxx>
Date: Mon, 9 Feb 2015 13:24:15 -0800
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=6WXeroEdqperZBoI4uOFtLnGHj0f0DdE/wyEnyxD/fc=; b=fDRzz4aJipfVki1et3V6DF0SR1ce0orzTqgHdRxBtKin9CGLqw8lu6QU2kge/SILar 0y59GwOJ4wM5FcNGcHH9W62ZZfk+0R5VZ87hIi1iYlagVNi2duaFwXgj0S71Z5A/rlZ7 9krZtVAhGc2vrLHu9w8xO0DWwCh/Fd9zgyZTpHIfdPZ0z3PCEzMNUkbH0+bIIROjwjAV 2lCUxJ/v3Q7PjG7MsqiFsFjlzw2hcDARNzMagOrHXz/fBexqG7lEl3gKIaydhRur/uKS THUIxSkxmT/0nt09LBcr4TjM4zWQOozS3bFNbAIAESPfsBBuhlgAG0X4K70H0a5kM94f WGdQ==
Hi Dave,

http://www.spinics.net/lists/linux-xfs/msg00061.html
Back in Dec 2013 you responded to this message saying that you would
take a look at it.  Was a fix for this ever issued?  I'm seeing very
similar stacktraces:

Feb  7 00:27:32 node008-cont001 kernel: [83405.490909] INFO: task
umount:29224 blocked for more than 120 seconds.
Feb  7 00:27:32 node008-cont001 kernel: [83405.498645]       Tainted:
G        W     3.13.0-39-generic #66-Ubuntu
Feb  7 00:27:32 node008-cont001 kernel: [83405.506273] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb  7 00:27:32 node008-cont001 kernel: [83405.515244] umount
D ffff880c4fc34480     0 29224  29221 0x00000082
Feb  7 00:27:32 node008-cont001 kernel: [83405.515253]
ffff880201211db0 0000000000000086 ffff880c39cb1800 ffff880201211fd8
Feb  7 00:27:32 node008-cont001 kernel: [83405.515260]
0000000000014480 0000000000014480 ffff880c39cb1800 ffff880c33386480
Feb  7 00:27:32 node008-cont001 kernel: [83405.515267]
ffff880c395e4bc8 ffff880c333864c0 ffff880c333864e8 ffff880c33386490
Feb  7 00:27:32 node008-cont001 kernel: [83405.515274] Call Trace:
Feb  7 00:27:32 node008-cont001 kernel: [83405.515298]
[<ffffffff81723109>] schedule+0x29/0x70
Feb  7 00:27:32 node008-cont001 kernel: [83405.515384]
[<ffffffffa023b0c9>] xfs_ail_push_all_sync+0xa9/0xe0 [xfs]
Feb  7 00:27:32 node008-cont001 kernel: [83405.515396]
[<ffffffff810aafd0>] ? prepare_to_wait_event+0x100/0x100
Feb  7 00:27:32 node008-cont001 kernel: [83405.515438]
[<ffffffffa0236f13>] xfs_log_quiesce+0x33/0x70 [xfs]
Feb  7 00:27:32 node008-cont001 kernel: [83405.515479]
[<ffffffffa0236f62>] xfs_log_unmount+0x12/0x30 [xfs]
Feb  7 00:27:32 node008-cont001 kernel: [83405.515510]
[<ffffffffa01ed846>] xfs_unmountfs+0xc6/0x150 [xfs]
Feb  7 00:27:32 node008-cont001 kernel: [83405.515541]
[<ffffffffa01ef211>] xfs_fs_put_super+0x21/0x60 [xfs]
Feb  7 00:27:32 node008-cont001 kernel: [83405.515550]
[<ffffffff811bf452>] generic_shutdown_super+0x72/0xf0
Feb  7 00:27:32 node008-cont001 kernel: [83405.515557]
[<ffffffff811bf707>] kill_block_super+0x27/0x70
Feb  7 00:27:32 node008-cont001 kernel: [83405.515565]
[<ffffffff811bf9ed>] deactivate_locked_super+0x3d/0x60
Feb  7 00:27:32 node008-cont001 kernel: [83405.515572]
[<ffffffff811bffa6>] deactivate_super+0x46/0x60
Feb  7 00:27:32 node008-cont001 kernel: [83405.515578]
[<ffffffff811dcd96>] mntput_no_expire+0xd6/0x170
Feb  7 00:27:32 node008-cont001 kernel: [83405.515584]
[<ffffffff811de31e>] SyS_umount+0x8e/0x100
Feb  7 00:27:32 node008-cont001 kernel: [83405.515591]
[<ffffffff8172f7ed>] system_call_fastpath+0x1a/0x1f


These type of errors are showing up in the logs:

Feb  7 00:27:34 node008-cont001 kernel: [83407.466853] XFS (dm-8):
metadata I/O error: block 0x0 ("xfs_buf_iodone_callbacks") error 19
numblks 1
...
Feb  7 00:27:39 node008-cont001 kernel: [83412.510982] XFS (dm-8):
metadata I/O error: block 0x0 ("xfs_buf_iodone_callbacks") error 19
numblks 1
Feb  7 00:27:44 node008-cont001 kernel: [83417.555152] XFS (dm-8):
metadata I/O error: block 0x0 ("xfs_buf_iodone_callbacks") error 19
numblks 1
...
Feb  7 00:27:54 node008-cont001 kernel: [83427.643428] XFS (dm-8):
metadata I/O error: block 0x0 ("xfs_buf_iodone_callbacks") error 19
numblks 1
Feb  7 00:27:57 node008-cont001 kernel: [83429.879442] XFS:: 568
callbacks suppressed
Feb  7 00:27:57 node008-cont001 kernel: [83429.879450] XFS (dm-8):
Detected failing async write on buffer block 0x0. Retrying async
write.
Feb  7 00:27:57 node008-cont001 kernel: [83429.879450]
Feb  7 00:27:57 node008-cont001 kernel: [83429.931438] XFS (dm-8):
Detected failing async write on buffer block 0x0. Retrying async
write.
Feb  7 00:27:57 node008-cont001 kernel: [83429.931438]
Feb  7 00:27:57 node008-cont001 kernel: [83429.983444] XFS (dm-8):
Detected failing async write on buffer block 0x0. Retrying async
write.
Feb  7 00:27:57 node008-cont001 kernel: [83429.983444]



Thanks for the help!
Chris

<Prev in Thread] Current Thread [Next in Thread>