[BUG] XFS: Assertion failed: atomic_read(&pag->pag_ref) == 0, file: fs/xfs/xfs_mount.c, line: 272

[Arkadiusz Bubała]

On 21.05.2013 14:23, Arkadiusz Bubała wrote:
> Hello,
> I've got a call trace which should be fixed by "drop buffer io 
> reference when a bad bio is built" patch 
> (http://patchwork.xfs.org/patch/3956/). Error occured on already 
> patched Linux kernel 3.2.42.
>
> Test environment consist two machines target and initiator.
> First machine works as target with QLogic Corp. ISP2432-based 4Gb 
> Fibre Channel device. Storage is placed on two KINGSTON SNV425S SSD 
> working as RAID0 array. RAID is managed by LSI MegaRAID SAS 1068 
> controller.
> Second machine works as initiator with the same QLogic card.
>
> After few days of running test script I got following call trace and 
> XFS stopped working.
>
Sorry I provided incomplete dmesg logs. These should provide more 
information:

[90011.884812] XFS (dm-46): metadata I/O error: block 0x1 ("xfs_trans_read_buf") error 5 buf count 512
[90011.941376] XFS (dm-46): xlog_recover_check_summary agf read failed agno 0 error 5
[90011.987890] XFS (dm-46): metadata I/O error: block 0x2 ("xfs_trans_read_buf") error 5 buf count 512
[90012.044179] XFS (dm-46): xlog_recover_check_summary agi read failed agno 0 error 5
[90012.092176] XFS (dm-46): metadata I/O error: block 0x7d0001 ("xfs_trans_read_buf") error 5 buf count 512
[90012.150379] XFS (dm-46): xlog_recover_check_summary agf read failed agno 1 error 5
[90012.196776] XFS (dm-46): metadata I/O error: block 0x7d0002 ("xfs_trans_read_buf") error 5 buf count 512
[90012.196780] XFS (dm-46): xlog_recover_check_summary agi read failed agno 1 error 5
[90012.196791] XFS (dm-46): metadata I/O error: block 0xfa0001 ("xfs_trans_read_buf") error 5 buf count 512
[90012.196795] XFS (dm-46): xlog_recover_check_summary agf read failed agno 2 error 5
[90012.196802] XFS (dm-46): metadata I/O error: block 0xfa0002 ("xfs_trans_read_buf") error 5 buf count 512
[90012.196806] XFS (dm-46): xlog_recover_check_summary agi read failed agno 2 error 5
[90012.196813] XFS (dm-46): metadata I/O error: block 0x1770001 ("xfs_trans_read_buf") error 5 buf count 512
[90012.196817] XFS (dm-46): xlog_recover_check_summary agf read failed agno 3 error 5
[90012.196823] XFS (dm-46): metadata I/O error: block 0x1770002 ("xfs_trans_read_buf") error 5 buf count 512
[90012.196827] XFS (dm-46): xlog_recover_check_summary agi read failed agno 3 error 5
[90012.196843] XFS (dm-46): metadata I/O error: block 0x40 ("xfs_trans_read_buf") error 5 buf count 8192
[90012.196847] XFS (dm-46): xfs_imap_to_bp: xfs_trans_read_buf() returned error 5.
[90012.196852] XFS (dm-46): failed to read root inode
[90012.196963] XFS: Assertion failed: atomic_read(&pag->pag_ref) == 0, file: fs/xfs/xfs_mount.c, line: 272
[90012.196982] ------------[ cut here ]------------
[90012.196984] kernel BUG at fs/xfs/xfs_message.c:101!
[90012.196987] invalid opcode: 0000 [#1] SMP
[90012.196990] CPU 2
[90012.196992] Modules linked in: iscsi_scst(O) scst_vdisk(O) libcrc32c qla2x00tgt(O) scst(O) ext2 drbd(O) iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi bonding qla2xxx(O) sg scsi_transport_fc megaraid_sas bnx2 acpi_power_meter usbserial uhci_hcd ohci_hcd ehci_hcd aufs [last unloaded: megaraid_sas]
[90012.197013]
[90012.197016] Pid: 10262, comm: mount Tainted: G           O 3.2.42-oe64-00000-g12db8b5 #14 Dell Inc. PowerEdge R510/0DPRKF
[90012.197022] RIP: 0010:[<ffffffff812ff8ed>]  [<ffffffff812ff8ed>] assfail+0x1d/0x30
[90012.197031] RSP: 0000:ffff8800be43fc68  EFLAGS: 00010296
[90012.197034] RAX: 0000000000000071 RBX: ffff8800512e8cc0 RCX: 0000000000000046
[90012.197037] RDX: 0000000000000000 RSI: 0000000000000046 RDI: ffffffff81c1c380
[90012.197039] RBP: ffff8800be43fc68 R08: 0000000000000006 R09: 000000000000ffff
[90012.197042] R10: 0000000000000006 R11: 000000000000000a R12: ffff8800a41f4800
[90012.197045] R13: 0000000000000000 R14: ffff8800a41f49e8 R15: ffff8800a41f49f8
[90012.197049] FS:  0000000000000000(0000) GS:ffff88012b240000(0063) knlGS:00000000f75456c0
[90012.197052] CS:  0010 DS: 002b ES: 002b CR0: 000000008005003b
[90012.197055] CR2: 000000000818e508 CR3: 000000001d9d7000 CR4: 00000000000006e0
[90012.197058] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[90012.197061] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[90012.197064] Process mount (pid: 10262, threadinfo ffff8800be43e000, task ffff88007fe64a60)
[90012.197066] Stack:
[90012.197068]  ffff8800be43fca8 ffffffff8134eb7a 0000000000000005 ffff8800a41f4800
[90012.197072]  ffff8800a41f4818 0000000000000005 ffff8800a41f4800 0000000000000000
[90012.197076]  ffff8800be43fcf8 ffffffff8135054d 0000000000000000 0000000000000000
[90012.197080] Call Trace:
[90012.197088]  [<ffffffff8134eb7a>] xfs_free_perag+0x8a/0xc0
[90012.197092]  [<ffffffff8135054d>] xfs_mountfs+0x31d/0x700
[90012.197097]  [<ffffffff81301fab>] xfs_fs_fill_super+0x1cb/0x270
[90012.197103]  [<ffffffff811476da>] mount_bdev+0x19a/0x1d0
[90012.197107]  [<ffffffff81301de0>] ? xfs_fs_write_inode+0x180/0x180
[90012.197114]  [<ffffffff8138b016>] ? selinux_sb_copy_data+0x156/0x1d0
[90012.197118]  [<ffffffff81300200>] xfs_fs_mount+0x10/0x20
[90012.197123]  [<ffffffff81146df1>] mount_fs+0x41/0x180
[90012.197129]  [<ffffffff8115f7ae>] vfs_kern_mount+0x5e/0xc0
[90012.197133]  [<ffffffff8116075e>] do_kern_mount+0x4e/0x100
[90012.197138]  [<ffffffff81161f26>] do_mount+0x516/0x740
[90012.197144]  [<ffffffff811064e9>] ? __get_free_pages+0x9/0x40
[90012.197150]  [<ffffffff81187cc2>] compat_sys_mount+0xa2/0x220
[90012.197156]  [<ffffffff8178be83>] ia32_do_call+0x13/0x13
[90012.197158] Code: 66 66 90 66 66 66 90 66 66 66 90 66 66 90 55 41 89 d0 48 89 f1 48 89 fa 48 c7 c6 b8 bf 9b 81 31 ff 48 89 e5 31 c0 e8 53 ff ff ff<0f>  0b eb fe 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 55 31
[90012.197189] RIP  [<ffffffff812ff8ed>] assfail+0x1d/0x30
[90012.197193]  RSP<ffff8800be43fc68>
[90012.197196] ---[ end trace ed2e349225f77763 ]---
[90015.960377] XFS (dm-44): xfs_log_force: error 5 returned.



-- 
Best regards
Arkadiusz Bubała