Hello!
We"ve got two different systems running with XFS, one of them is
publishing shares via NFS, the other via Samba. We've recently
encountered filesystem issues on both servers, and received the
following call traces. They look very much alike.
Are these something you are familiar with? We'd like to at least
establish that the problem really is with XFS... We don't really have
the possibility to update the kernels (2.6.27.10) on those machines, but
we can possibly apply patches etc. These systems are usually under a
fairly high load and host a large number of files.
Filesystem "dm-16": Disabling barriers, trial barrier write failed
XFS mounting filesystem dm-16
Starting XFS recovery on filesystem: dm-16 (logdev: internal)
XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1590 of file
fs/xfs/xfs_alloc.c. Caller 0xffffffff80398ca7
Pid: 15745, comm: mount Not tainted 2.6.27.10
#24
Call Trace:
[<ffffffff80396fb8>] xfs_free_ag_extent+0x378/0x740
[<ffffffff80398ca7>] xfs_free_extent+0xc7/0x110
[<ffffffff803d7642>] xlog_recover_process_efi+0x122/0x1a0
[<ffffffff803d7727>] xlog_recover_process_efis+0x67/0x90
[<ffffffff803d87dc>] xlog_recover_finish+0x1c/0xd0
[<ffffffff803d0790>] xfs_log_mount_finish+0x20/0x30
[<ffffffff803dac3d>] xfs_mountfs+0x2bd/0x640
[<ffffffff803c0440>] xfs_fstrm_free_func+0x0/0x90
[<ffffffff803dbaca>] xfs_mru_cache_create+0x15a/0x1c0
[<ffffffff803f2c04>] xfs_fs_fill_super+0x214/0x450
[<ffffffff80299100>] set_bdev_super+0x0/0x10
[<ffffffff8029925f>] get_sb_bdev+0x13f/0x180
[<ffffffff803f29f0>] xfs_fs_fill_super+0x0/0x450
[<ffffffff802994f1>] vfs_kern_mount+0x81/0x160
[<ffffffff8029961d>] do_kern_mount+0x4d/0x110
[<ffffffff802b0fbb>] do_new_mount+0x9b/0xe0
[<ffffffff802b1849>] do_mount+0x209/0x220
[<ffffffff80274ea2>] __alloc_pages_internal+0x92/0x430
[<ffffffff802752d7>] __get_free_pages+0x17/0x80
[<ffffffff802cbdf4>] compat_sys_mount+0xa4/0x270
[<ffffffff80228282>] ia32_sysret+0x0/0xa
Failed to recover EFIs on filesystem: dm-16
XFS: log mount finish failed
NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
NFSD: starting 90-second grace period
bootsplash: status on console 0 changed to off
usb 3-2: USB disconnect, address 2
00000000: 00 00 00 00 00 28 00 e0 00 00 00 00 00 00 00 00 .....(..........
Filesystem "dm-20": XFS internal error xfs_da_do_buf(2) at line 2112 of
file fs/xfs/xfs_da_btree.c. Caller 0xffffffff803b34b4
Pid: 23023, comm: smbd Not tainted 2.6.27.10
#24
Call Trace:
[<ffffffff803b33b6>] xfs_da_do_buf+0x626/0x6b0
[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
[<ffffffff8023ee0f>] try_to_del_timer_sync+0x4f/0x60
[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
[<ffffffff803b77e5>] xfs_dir2_block_lookup_int+0x45/0x1a0
[<ffffffff803b77e5>] xfs_dir2_block_lookup_int+0x45/0x1a0
[<ffffffff803b7958>] xfs_dir2_block_lookup+0x18/0xc0
[<ffffffff803b624f>] xfs_dir2_isblock+0x1f/0x60
[<ffffffff803b696d>] xfs_dir_lookup+0x19d/0x1c0
[<ffffffff803e3267>] xfs_lookup+0x57/0xd0
[<ffffffff8069dfd9>] _spin_lock_bh+0x9/0x20
[<ffffffff803ee6e4>] xfs_vn_lookup+0x64/0xc0
[<ffffffff802a9f95>] d_alloc+0x125/0x1b0
[<ffffffff8029f155>] do_lookup+0x175/0x220
[<ffffffff8029ea79>] generic_permission+0x69/0x130
[<ffffffff8029fa01>] __link_path_walk+0x801/0xdb0
[<ffffffff805de663>] sock_aio_read+0x163/0x170
[<ffffffff802a0007>] path_walk+0x57/0xb0
[<ffffffff802a0183>] do_path_lookup+0x123/0x1b0
[<ffffffff802a06f4>] user_path_at+0x44/0x80
[<ffffffff8024a450>] autoremove_wake_function+0x0/0x30
[<ffffffff802af5c1>] mntput_no_expire+0x21/0x120
[<ffffffff8029a1f3>] vfs_stat_fd+0x23/0x80
[<ffffffff8022879f>] sys32_stat64+0x1f/0x70
[<ffffffff80228282>] ia32_sysret+0x0/0xa
00000000: 00 00 00 00 00 28 00 e0 00 00 00 00 00 00 00 00 .....(..........
Filesystem "dm-20": XFS internal error xfs_da_do_buf(2) at line 2112 of
file fs/xfs/xfs_da_btree.c. Caller 0xffffffff803b34b4
Pid: 23023, comm: smbd Not tainted 2.6.27.10
#24
Call Trace:
[<ffffffff803b33b6>] xfs_da_do_buf+0x626/0x6b0
[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
[<ffffffff8023ee0f>] try_to_del_timer_sync+0x4f/0x60
[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
[<ffffffff803b77e5>] xfs_dir2_block_lookup_int+0x45/0x1a0
[<ffffffff803b77e5>] xfs_dir2_block_lookup_int+0x45/0x1a0
[<ffffffff803b7958>] xfs_dir2_block_lookup+0x18/0xc0
[<ffffffff803b624f>] xfs_dir2_isblock+0x1f/0x60
[<ffffffff803b696d>] xfs_dir_lookup+0x19d/0x1c0
[<ffffffff803e3267>] xfs_lookup+0x57/0xd0
[<ffffffff8069dfd9>] _spin_lock_bh+0x9/0x20
[<ffffffff803ee6e4>] xfs_vn_lookup+0x64/0xc0
[<ffffffff802a9f95>] d_alloc+0x125/0x1b0
[<ffffffff8029f155>] do_lookup+0x175/0x220
[<ffffffff8029ea79>] generic_permission+0x69/0x130
[<ffffffff8029fa01>] __link_path_walk+0x801/0xdb0
[<ffffffff805de663>] sock_aio_read+0x163/0x170
[<ffffffff802a0007>] path_walk+0x57/0xb0
[<ffffffff802a0183>] do_path_lookup+0x123/0x1b0
[<ffffffff802a06f4>] user_path_at+0x44/0x80
[<ffffffff8024a450>] autoremove_wake_function+0x0/0x30
[<ffffffff802af5c1>] mntput_no_expire+0x21/0x120
[<ffffffff8029a1f3>] vfs_stat_fd+0x23/0x80
[<ffffffff8022879f>] sys32_stat64+0x1f/0x70
[<ffffffff80228282>] ia32_sysret+0x0/0xa
>From the other system we only have these logs:
2010/01/27 00:22:07|Pid: 17209, comm: nfsd Not tainted 2.6.27.10 #12
2010/01/27 00:22:07|
2010/01/27 00:22:07|Call Trace:
2010/01/27 00:22:07|[<ffffffff803b33b6>] xfs_da_do_buf+0x626/0x6b0
2010/01/27 00:22:07|[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
2010/01/27 00:22:07|[<ffffffff803ead07>] xfs_buf_read_flags+0x67/0x90
2010/01/27 00:22:07|[<ffffffff803df247>] xfs_trans_read_buf+0x147/0x310
2010/01/27 00:22:07|[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
2010/01/27 00:22:07|[<ffffffff803b3748>] xfs_da_node_lookup_int+0x88/0x2a0
2010/01/27 00:22:07|[<ffffffff803b3748>] xfs_da_node_lookup_int+0x88/0x2a0
2010/01/27 00:22:07|[<ffffffff803bd2f8>] xfs_dir2_node_lookup+0x48/0x120
2010/01/27 00:22:07|[<ffffffff803b697a>] xfs_dir_lookup+0x1aa/0x1c0
2010/01/27 00:22:07|[<ffffffff803e3267>] xfs_lookup+0x57/0xd0
2010/01/27 00:22:07|[<ffffffff8035279e>] nfsd_permission+0x7e/0x130
2010/01/27 00:22:07|[<ffffffff803ee6e4>] xfs_vn_lookup+0x64/0xc0
2010/01/27 00:22:07|[<ffffffff802a9f95>] d_alloc+0x125/0x1b0
2010/01/27 00:22:07|[<ffffffff802a048c>] __lookup_hash+0xec/0x180
2010/01/27 00:22:07|[<ffffffff802a0629>] lookup_one_len+0x59/0x60
2010/01/27 00:22:07|[<ffffffff8035023c>] nfsd_lookup_dentry+0x12c/0x4d0
2010/01/27 00:22:07|[<ffffffff80350610>] nfsd_lookup+0x30/0x100
2010/01/27 00:22:07|[<ffffffff803599a7>] nfsd3_proc_lookup+0xa7/0x100
2010/01/27 00:22:07|[<ffffffff8034ca6e>] nfsd_dispatch+0xae/0x260
2010/01/27 00:22:07|[<ffffffff80669a87>] svc_process+0x307/0x740
2010/01/27 00:22:07|[<ffffffff8034c892>] nfsd+0x172/0x2a0
2010/01/27 00:22:07|[<ffffffff8034c720>] nfsd+0x0/0x2a0
2010/01/27 00:22:07|[<ffffffff80249d3c>] kthread+0x6c/0xa0
2010/01/27 00:22:07|[<ffffffff8020d1c9>] child_rip+0xa/0x11
2010/01/27 00:22:07|[<ffffffff80249cd0>] kthread+0x0/0xa0
2010/01/27 00:22:07|[<ffffffff8020d1bf>] child_rip+0x0/0x11
2010/01/27 00:22:07|
2010/01/27 00:22:09|Pid: 17189, comm: nfsd Not tainted 2.6.27.10 #12
2010/01/27 00:22:09|
2010/01/27 00:22:09|Call Trace:
2010/01/27 00:22:09|[<ffffffff803b33b6>] xfs_da_do_buf+0x626/0x6b0
2010/01/27 00:22:09|[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
2010/01/27 00:22:09|[<ffffffff803b2bed>] xfs_da_buf_make+0x13d/0x150
2010/01/27 00:22:09|[<ffffffff803df247>] xfs_trans_read_buf+0x147/0x310
2010/01/27 00:22:09|[<ffffffff803b34b4>] xfs_da_read_buf+0x24/0x30
2010/01/27 00:22:09|[<ffffffff803bbd5b>] xfs_dir2_leafn_lookup_for_entry+0x16b/0x350
2010/01/27 00:22:09|[<ffffffff803bbd5b>] xfs_dir2_leafn_lookup_for_entry+0x16b/0x350
2010/01/27 00:22:09|[<ffffffff803b38f7>] xfs_da_node_lookup_int+0x237/0x2a0
2010/01/27 00:22:09|[<ffffffff803bd2f8>] xfs_dir2_node_lookup+0x48/0x120
2010/01/27 00:22:09|[<ffffffff803b697a>] xfs_dir_lookup+0x1aa/0x1c0
2010/01/27 00:22:09|[<ffffffff803e3267>] xfs_lookup+0x57/0xd0
2010/01/27 00:22:09|[<ffffffff8035279e>] nfsd_permission+0x7e/0x130
2010/01/27 00:22:09|[<ffffffff803ee6e4>] xfs_vn_lookup+0x64/0xc0
2010/01/27 00:22:09|[<ffffffff802a9f95>] d_alloc+0x125/0x1b0
2010/01/27 00:22:09|[<ffffffff802a048c>] __lookup_hash+0xec/0x180
2010/01/27 00:22:09|[<ffffffff802a0629>] lookup_one_len+0x59/0x60
2010/01/27 00:22:09|[<ffffffff8035023c>] nfsd_lookup_dentry+0x12c/0x4d0
2010/01/27 00:22:09|[<ffffffff80350610>] nfsd_lookup+0x30/0x100
2010/01/27 00:22:09|[<ffffffff803599a7>] nfsd3_proc_lookup+0xa7/0x100
2010/01/27 00:22:09|[<ffffffff8034ca6e>] nfsd_dispatch+0xae/0x260
2010/01/27 00:22:09|[<ffffffff80669a87>] svc_process+0x307/0x740
2010/01/27 00:22:09|[<ffffffff8034c892>] nfsd+0x172/0x2a0
2010/01/27 00:22:09|[<ffffffff8034c720>] nfsd+0x0/0x2a0
2010/01/27 00:22:09|[<ffffffff80249d3c>] kthread+0x6c/0xa0
2010/01/27 00:22:09|[<ffffffff8020d1c9>] child_rip+0xa/0x11
2010/01/27 00:22:09|[<ffffffff80249cd0>] kthread+0x0/0xa0
2010/01/27 00:22:09|[<ffffffff8020d1bf>] child_rip+0x0/0x11