xfs
[Top] [All Lists]

Vanilla 3.0.78

To: "xfs@xxxxxxxxxxx" <xfs@xxxxxxxxxxx>
Subject: Vanilla 3.0.78
From: Stefan Priebe - Profihost AG <s.priebe@xxxxxxxxxxxx>
Date: Mon, 29 Jul 2013 09:39:37 +0200
Cc: "xfs-masters@xxxxxxxxxxx" <xfs-masters@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130623 Thunderbird/17.0.7
Hi,

while running 3.0.78 and doing heavy rsync tasks on a raid 50 i'm gettig
these call traces:

[396113.710927] INFO: task rsync:25944 blocked for more than 120 seconds.
[396113.760608] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[396113.811356] rsync           D ffff88042eb70560     0 25944  25943
0x00000000
[396113.861996]  ffff88000ef4b218 0000000000000082 ffff88000ef4b1f8
0000000000000000
[396113.912069]  0000000000010800 ffff88000ef4bfd8 ffff88000ef4a010
0000000000010800
[396113.962197]  ffff88000ef4bfd8 0000000000010800 ffffffff81a0b020
ffff88042eb70000
[396114.013067] Call Trace:
[396114.063597]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[396114.114651]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[396114.165719]  [<ffffffff811f9ec0>] ? xfs_btree_log_keys+0x50/0x90
[396114.216982]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[396114.268645]  [<ffffffff8147bc59>] __down+0x69/0xb0
[396114.320322]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[396114.371695]  [<ffffffff81078c9c>] down+0x3c/0x50
[396114.422894]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[396114.474072]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[396114.524772]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[396114.575620]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[396114.625803]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[396114.676022]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[396114.725920]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[396114.776482]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[396114.826416]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[396114.876304]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[396114.924793]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[396114.972421]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[396115.019255]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[396115.065919]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[396115.112754]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[396115.158908]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[396115.204162]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[396115.247605]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[396115.291805]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[396115.335010]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[396115.378068]  [<ffffffff810cd832>] __writepage+0x12/0x40
[396115.420772]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[396115.466613]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[396115.512538]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[396115.555134]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[396115.597647]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[396115.640635]  [<ffffffff810c4cab>] __filemap_fdatawrite_range+0x5b/0x60
[396115.682990]  [<ffffffff810c4f4e>] filemap_fdatawrite_range+0xe/0x10
[396115.724450]  [<ffffffff812399c7>] xfs_flush_pages+0x67/0xb0
[396115.765531]  [<ffffffff8122fc7a>] xfs_setattr+0x9da/0xa80
[396115.806444]  [<ffffffff8123c8a6>] xfs_vn_setattr+0x16/0x20
[396115.846806]  [<ffffffff8112d9bd>] notify_change+0x11d/0x300
[396115.887020]  [<ffffffff8111328c>] do_truncate+0x5c/0x90
[396115.926632]  [<ffffffff81113387>] sys_ftruncate+0xc7/0xd0
[396115.965535]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[396116.003705] INFO: task rsync:16891 blocked for more than 120 seconds.
[396116.041978] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[396116.080957] rsync           D ffff880044d9b5a0     0 16891  16890
0x00000000
[396116.120493]  ffff88062b129b68 0000000000000086 0000000000000000
ffff880400000001
[396116.160730]  0000000000010800 ffff88062b129fd8 ffff88062b128010
0000000000010800
[396116.200394]  ffff88062b129fd8 0000000000010800 ffffffff81a0b020
ffff880044d9b040
[396116.240183] Call Trace:
[396116.279644]  [<ffffffff81108e3b>] ? kmem_cache_free+0x1b/0xf0
[396116.320904]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[396116.360893]  [<ffffffff8147c6bd>] rwsem_down_failed_common+0xbd/0x150
[396116.401765]  [<ffffffff8147c763>] rwsem_down_write_failed+0x13/0x20
[396116.442829]  [<ffffffff812861c3>] call_rwsem_down_write_failed+0x13/0x20
[396116.483521]  [<ffffffff8147bac0>] ? down_write+0x20/0x30
[396116.524244]  [<ffffffff8121244e>] xfs_ilock+0x7e/0xd0
[396116.564991]  [<ffffffff81239107>] xfs_file_aio_write_checks+0x47/0x150
[396116.606056]  [<ffffffff8123951d>] xfs_file_buffered_aio_write+0x6d/0x180
[396116.647785]  [<ffffffff812397d3>] xfs_file_aio_write+0x1a3/0x2b0
[396116.689301]  [<ffffffff811142a2>] do_sync_write+0xd2/0x110
[396116.730611]  [<ffffffff8104d6ce>] ? perf_event_task_sched_out+0x2e/0x90
[396116.772345]  [<ffffffff8147cc8e>] ? common_interrupt+0xe/0x13
[396116.814509]  [<ffffffff8107dbc8>] ? ktime_get_ts+0xa8/0xe0
[396116.855862]  [<ffffffff811147b8>] vfs_write+0xc8/0x190
[396116.896935]  [<ffffffff8111496c>] sys_write+0x4c/0x90
[396116.937889]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[412651.656492] INFO: task flush-8:16:5619 blocked for more than 120
seconds.
[412651.698333] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[412651.741269] flush-8:16      D ffff88042e6635a0     0  5619      2
0x00000000
[412651.785328]  ffff88042da27120 0000000000000046 ffff88042e3c86a8
0000000000000024
[412651.829433]  0000000000010800 ffff88042da27fd8 ffff88042da26010
0000000000010800
[412651.873521]  ffff88042da27fd8 0000000000010800 ffff88042f89c860
ffff88042e663040
[412651.917893] Call Trace:
[412651.961539]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[412652.005977]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[412652.050757]  [<ffffffff8122cd90>] ? xfs_trans_log_buf+0x90/0xd0
[412652.095552]  [<ffffffff811f9ec0>] ? xfs_btree_log_keys+0x50/0x90
[412652.139883]  [<ffffffff8147bc59>] __down+0x69/0xb0
[412652.184525]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[412652.228969]  [<ffffffff81078c9c>] down+0x3c/0x50
[412652.273330]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[412652.317308]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[412652.361185]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[412652.404909]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[412652.448291]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[412652.492152]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[412652.535849]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[412652.579896]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[412652.624049]  [<ffffffff8127a109>] ? cpumask_next_and+0x29/0x50
[412652.667864]  [<ffffffff810fe9de>] ? alloc_pages_current+0x7e/0xe0
[412652.711426]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[412652.754460]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[412652.796510]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[412652.838066]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[412652.879153]  [<ffffffff81212c73>] ? xfs_iext_get_ext+0x33/0x80
[412652.920043]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[412652.960023]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[412652.999673]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[412653.039712]  [<ffffffff81212c73>] ? xfs_iext_get_ext+0x33/0x80
[412653.079928]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[412653.120173]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[412653.160213]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[412653.202678]  [<ffffffff810cd832>] __writepage+0x12/0x40
[412653.245561]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[412653.285796]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[412653.325281]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[412653.364922]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[412653.404711]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[412653.443941]  [<ffffffff81138454>] writeback_single_inode+0xf4/0x260
[412653.483453]  [<ffffffff81138955>] writeback_sb_inodes+0xe5/0x1b0
[412653.523014]  [<ffffffff81138db9>] writeback_inodes_wb+0x99/0x160
[412653.561999]  [<ffffffff811396f3>] wb_writeback+0x2f3/0x450
[412653.601805]  [<ffffffff8147cc8e>] ? common_interrupt+0xe/0x13
[412653.641066]  [<ffffffff81139a6f>] wb_do_writeback+0x21f/0x270
[412653.679713]  [<ffffffff81139b5a>] bdi_writeback_thread+0x9a/0x230
[412653.717842]  [<ffffffff81139ac0>] ? wb_do_writeback+0x270/0x270
[412653.755257]  [<ffffffff81139ac0>] ? wb_do_writeback+0x270/0x270
[412653.791866]  [<ffffffff81073286>] kthread+0x96/0xa0
[412653.827748]  [<ffffffff8147e314>] kernel_thread_helper+0x4/0x10
[412653.863566]  [<ffffffff810731f0>] ? kthread_worker_fn+0x130/0x130
[412653.898946]  [<ffffffff8147e310>] ? gs_change+0xb/0xb
[412653.934129] INFO: task rsync:16475 blocked for more than 120 seconds.
[412653.969849] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[412654.006947] rsync           D ffff88042e660560     0 16475  16474
0x00000000
[412654.044772]  ffff880487331218 0000000000000082 ffff88042e3c86a8
0000000000000003
[412654.083421]  0000000000010800 ffff880487331fd8 ffff880487330010
0000000000010800
[412654.122261]  ffff880487331fd8 0000000000010800 ffff8801fefcc860
ffff88042e660000
[412654.161041] Call Trace:
[412654.199520]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[412654.238551]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[412654.277727]  [<ffffffff811f9ec0>] ? xfs_btree_log_keys+0x50/0x90
[412654.319874]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[412654.361564]  [<ffffffff8147bc59>] __down+0x69/0xb0
[412654.400663]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[412654.440047]  [<ffffffff81078c9c>] down+0x3c/0x50
[412654.479105]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[412654.518100]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[412654.557207]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[412654.596206]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[412654.634624]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[412654.673479]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[412654.712002]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[412654.750449]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[412654.789052]  [<ffffffff811f59ef>] ?
xfs_bmap_add_extent_delay_real+0x64f/0x1700
[412654.828682]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[412654.867729]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[412654.906586]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[412654.945568]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[412654.984313]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[412655.023299]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[412655.061549]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[412655.101113]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[412655.140458]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[412655.179687]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[412655.219207]  [<ffffffff810cd832>] __writepage+0x12/0x40
[412655.258002]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[412655.296797]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[412655.336008]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[412655.375370]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[412655.414534]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[412655.454009]  [<ffffffff810c4cab>] __filemap_fdatawrite_range+0x5b/0x60
[412655.493627]  [<ffffffff810c4f4e>] filemap_fdatawrite_range+0xe/0x10
[412655.533156]  [<ffffffff812399c7>] xfs_flush_pages+0x67/0xb0
[412655.572856]  [<ffffffff8122fc7a>] xfs_setattr+0x9da/0xa80
[412655.613257]  [<ffffffff8123c8a6>] xfs_vn_setattr+0x16/0x20
[412655.653033]  [<ffffffff8112d9bd>] notify_change+0x11d/0x300
[412655.692725]  [<ffffffff8111328c>] do_truncate+0x5c/0x90
[412655.731966]  [<ffffffff81113387>] sys_ftruncate+0xc7/0xd0
[412655.771199]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[412655.810243] INFO: task rsync:30660 blocked for more than 120 seconds.
[412655.848982] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[412655.888250] rsync           D ffff88042d871d80     0 30660  30659
0x00000000
[412655.927393]  ffff880415137218 0000000000000086 ffff8804151371a8
ffffffff81078be2
[412655.967391]  0000000000010800 ffff880415137fd8 ffff880415136010
0000000000010800
[412656.007316]  ffff880415137fd8 0000000000010800 ffffffff81a0b020
ffff88042d871820
[412656.046983] Call Trace:
[412656.085817]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[412656.126302]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[412656.166035]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[412656.206776]  [<ffffffff812321d5>] ? kmem_zone_zalloc+0x35/0x50
[412656.250152]  [<ffffffff8122cd90>] ? xfs_trans_log_buf+0x90/0xd0
[412656.291970]  [<ffffffff8147bc59>] __down+0x69/0xb0
[412656.331764]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[412656.371898]  [<ffffffff81078c9c>] down+0x3c/0x50
[412656.411888]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[412656.452059]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[412656.492297]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[412656.532531]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[412656.572399]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[412656.612389]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[412656.652851]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[412656.692462]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[412656.731749]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[412656.771739]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[412656.811371]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[412656.850939]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[412656.890922]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[412656.930893]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[412656.970295]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[412657.009689]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[412657.049157]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[412657.088642]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[412657.128997]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[412657.169076]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[412657.209303]  [<ffffffff810cd832>] __writepage+0x12/0x40
[412657.249098]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[412657.288823]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[412657.328563]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[412657.368193]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[412657.407870]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[412657.447095]  [<ffffffff810c4cab>] __filemap_fdatawrite_range+0x5b/0x60
[412657.487191]  [<ffffffff810c4f4e>] filemap_fdatawrite_range+0xe/0x10
[412657.527229]  [<ffffffff812399c7>] xfs_flush_pages+0x67/0xb0
[412657.567153]  [<ffffffff8122fc7a>] xfs_setattr+0x9da/0xa80
[412657.607216]  [<ffffffff8123c8a6>] xfs_vn_setattr+0x16/0x20
[412657.647641]  [<ffffffff8112d9bd>] notify_change+0x11d/0x300
[412657.687480]  [<ffffffff8111328c>] do_truncate+0x5c/0x90
[412657.726860]  [<ffffffff81113387>] sys_ftruncate+0xc7/0xd0
[412657.765326]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[412657.803043] INFO: task rsync:16891 blocked for more than 120 seconds.
[412657.840819] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[412657.879354] rsync           D ffff880044d9b5a0     0 16891  16890
0x00000000
[412657.917922]  ffff88062b129b68 0000000000000086 0000000000000000
ffff880200000001
[412657.956861]  0000000000010800 ffff88062b129fd8 ffff88062b128010
0000000000010800
[412657.996037]  ffff88062b129fd8 0000000000010800 ffffffff81a0b020
ffff880044d9b040
[412658.035286] Call Trace:
[412658.074171]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[412658.113709]  [<ffffffff8147c6bd>] rwsem_down_failed_common+0xbd/0x150
[412658.154355]  [<ffffffff8147c763>] rwsem_down_write_failed+0x13/0x20
[412658.194313]  [<ffffffff812861c3>] call_rwsem_down_write_failed+0x13/0x20
[412658.234820]  [<ffffffff8147bac0>] ? down_write+0x20/0x30
[412658.275104]  [<ffffffff8121244e>] xfs_ilock+0x7e/0xd0
[412658.315310]  [<ffffffff81239107>] xfs_file_aio_write_checks+0x47/0x150
[412658.355943]  [<ffffffff8123951d>] xfs_file_buffered_aio_write+0x6d/0x180
[412658.396638]  [<ffffffff812397d3>] xfs_file_aio_write+0x1a3/0x2b0
[412658.437447]  [<ffffffff811142a2>] do_sync_write+0xd2/0x110
[412658.478284]  [<ffffffff8107dbc8>] ? ktime_get_ts+0xa8/0xe0
[412658.519038]  [<ffffffff811147b8>] vfs_write+0xc8/0x190
[412658.559661]  [<ffffffff8111496c>] sys_write+0x4c/0x90
[412658.599950]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[412658.640329] INFO: task rsync:27527 blocked for more than 120 seconds.
[412658.681340] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[412658.723153] rsync           D ffff88042dde4dc0     0 27527   5627
0x00000000
[412658.765631]  ffff88000c999808 0000000000000086 ffff88042dd4f2e8
ffffc90000000000
[412658.808827]  0000000000010800 ffff88000c999fd8 ffff88000c998010
0000000000010800
[412658.852414]  ffff88000c999fd8 0000000000010800 ffff88042f991820
ffff88042dde4860
[412658.896628] Call Trace:
[412658.939953]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[412658.984168]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[412659.027943]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[412659.071098]  [<ffffffff81235b7a>] ? xfs_buf_cond_lock+0x2a/0xc0
[412659.114604]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[412659.157907]  [<ffffffff8147bc59>] __down+0x69/0xb0
[412659.201369]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[412659.246860]  [<ffffffff81078c9c>] down+0x3c/0x50
[412659.293899]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[412659.337497]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[412659.380474]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[412659.423531]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[412659.466079]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[412659.508637]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[412659.551046]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[412659.593820]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[412659.636680]  [<ffffffff81108e3b>] ? kmem_cache_free+0x1b/0xf0
[412659.679799]  [<ffffffff8122a0ae>] ? xfs_trans_free_item_desc+0x2e/0x30
[412659.723313]  [<ffffffff8122a1e7>] ? xfs_trans_free_items+0x87/0xb0
[412659.766472]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[412659.808765]  [<ffffffff811e6d17>] ? xfs_free_extent+0x97/0x120
[412659.850247]  [<ffffffff811e6d28>] xfs_free_extent+0xa8/0x120
[412659.891039]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[412659.931242]  [<ffffffff811f01cd>] xfs_bmap_finish+0x15d/0x1a0
[412659.970975]  [<ffffffff8121717f>] xfs_itruncate_finish+0x15f/0x330
[412660.010652]  [<ffffffff8122ebe3>] xfs_inactive+0x313/0x470
[412660.049573]  [<ffffffff8123f2ba>] xfs_fs_evict_inode+0xba/0x100
[412660.087971]  [<ffffffff8112c99b>] evict+0x7b/0x140
[412660.126324]  [<ffffffff8112cdd8>] iput_final+0xc8/0x180
[412660.164635]  [<ffffffff8112cec3>] iput+0x33/0x40
[412660.202813]  [<ffffffff811230c4>] do_unlinkat+0x164/0x1c0
[412660.240689]  [<ffffffff81130ed8>] ? mntput+0x18/0x30
[412660.278332]  [<ffffffff811162bd>] ? fput+0x1d/0x30
[412660.315609]  [<ffffffff811126d0>] ? filp_close+0x60/0x90
[412660.352846]  [<ffffffff81123131>] sys_unlink+0x11/0x20
[412660.389835]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[417932.351148] INFO: task flush-8:16:5619 blocked for more than 120
seconds.
[417932.389328] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[417932.428412] flush-8:16      D ffff88042e6635a0     0  5619      2
0x00000000
[417932.468679]  ffff88042da27120 0000000000000046 ffff88042e3c86a8
000000000000001e
[417932.509997]  0000000000010800 ffff88042da27fd8 ffff88042da26010
0000000000010800
[417932.550591]  ffff88042da27fd8 0000000000010800 ffff8801fefcc860
ffff88042e663040
[417932.590999] Call Trace:
[417932.629814]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[417932.668838]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[417932.707796]  [<ffffffff811fa733>] ? xfs_btree_del_cursor+0x63/0x70
[417932.746792]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[417932.784584]  [<ffffffff8147bc59>] __down+0x69/0xb0
[417932.822045]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[417932.860343]  [<ffffffff81078c9c>] down+0x3c/0x50
[417932.897945]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[417932.935951]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[417932.973550]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[417933.011390]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[417933.048936]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[417933.086642]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[417933.124124]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[417933.161818]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[417933.199627]  [<ffffffff810fe9de>] ? alloc_pages_current+0x7e/0xe0
[417933.237788]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[417933.276072]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[417933.314175]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[417933.352335]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[417933.390764]  [<ffffffff81212c73>] ? xfs_iext_get_ext+0x33/0x80
[417933.428967]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[417933.466814]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[417933.504962]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[417933.543502]  [<ffffffff81212c73>] ? xfs_iext_get_ext+0x33/0x80
[417933.582007]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[417933.620741]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[417933.659551]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[417933.698203]  [<ffffffff810cd832>] __writepage+0x12/0x40
[417933.736676]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[417933.775417]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[417933.814118]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[417933.852705]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[417933.891799]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[417933.930897]  [<ffffffff81138454>] writeback_single_inode+0xf4/0x260
[417933.970365]  [<ffffffff81138955>] writeback_sb_inodes+0xe5/0x1b0
[417934.009561]  [<ffffffff81138db9>] writeback_inodes_wb+0x99/0x160
[417934.048630]  [<ffffffff811396f3>] wb_writeback+0x2f3/0x450
[417934.087944]  [<ffffffff8147cc8e>] ? common_interrupt+0xe/0x13
[417934.127261]  [<ffffffff81139a6f>] wb_do_writeback+0x21f/0x270
[417934.166603]  [<ffffffff81139b5a>] bdi_writeback_thread+0x9a/0x230
[417934.205473]  [<ffffffff81139ac0>] ? wb_do_writeback+0x270/0x270
[417934.243674]  [<ffffffff81139ac0>] ? wb_do_writeback+0x270/0x270
[417934.280260]  [<ffffffff81073286>] kthread+0x96/0xa0
[417934.316477]  [<ffffffff8147e314>] kernel_thread_helper+0x4/0x10
[417934.352432]  [<ffffffff810731f0>] ? kthread_worker_fn+0x130/0x130
[417934.388926]  [<ffffffff8147e310>] ? gs_change+0xb/0xb
[417934.424356] INFO: task rsync:16475 blocked for more than 120 seconds.
[417934.459973] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[417934.497620] rsync           D ffff88042e660560     0 16475  16474
0x00000000
[417934.538513]  ffff880487331218 0000000000000082 ffff8804873311a8
ffffffff00000000
[417934.579249]  0000000000010800 ffff880487331fd8 ffff880487330010
0000000000010800
[417934.617923]  ffff880487331fd8 0000000000010800 ffff88042fa1e080
ffff88042e660000
[417934.656752] Call Trace:
[417934.695112]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[417934.734191]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[417934.773444]  [<ffffffff812321d5>] ? kmem_zone_zalloc+0x35/0x50
[417934.812649]  [<ffffffff81078be2>] ? down_trylock+0x32/0x50
[417934.852084]  [<ffffffff8147bc59>] __down+0x69/0xb0
[417934.891346]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[417934.931227]  [<ffffffff81078c9c>] down+0x3c/0x50
[417934.970405]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[417935.009641]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[417935.048842]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[417935.087990]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[417935.126664]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[417935.165700]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[417935.204566]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[417935.243406]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[417935.282639]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[417935.322257]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[417935.361292]  [<ffffffff811e6f73>] ? xfs_alloc_vextent+0x1d3/0x580
[417935.400613]  [<ffffffff811e6f0b>] xfs_alloc_vextent+0x16b/0x580
[417935.439495]  [<ffffffff811f0e77>] xfs_bmap_btalloc+0x257/0x6e0
[417935.478274]  [<ffffffff811f1309>] xfs_bmap_alloc+0x9/0x10
[417935.517264]  [<ffffffff811f77fe>] xfs_bmapi+0x99e/0x1090
[417935.555550]  [<ffffffff811ef165>] ?
xfs_bmap_search_multi_extents+0xa5/0x110
[417935.594479]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[417935.633438]  [<ffffffff81218bb6>] xfs_iomap_write_allocate+0x176/0x330
[417935.672722]  [<ffffffff81234051>] xfs_map_blocks+0x1d1/0x260
[417935.712129]  [<ffffffff812342a2>] xfs_vm_writepage+0x1c2/0x500
[417935.751264]  [<ffffffff810cd832>] __writepage+0x12/0x40
[417935.789867]  [<ffffffff810cea59>] write_cache_pages+0x239/0x4d0
[417935.828959]  [<ffffffff810cd820>] ? set_page_dirty+0x70/0x70
[417935.868322]  [<ffffffff81134688>] ? generic_getxattr+0x88/0x90
[417935.907789]  [<ffffffff810ced3c>] generic_writepages+0x4c/0x70
[417935.947262]  [<ffffffff81232dff>] xfs_vm_writepages+0x4f/0x60
[417935.986511]  [<ffffffff810ced7c>] do_writepages+0x1c/0x40
[417936.025537]  [<ffffffff810c4cab>] __filemap_fdatawrite_range+0x5b/0x60
[417936.065140]  [<ffffffff810c4f4e>] filemap_fdatawrite_range+0xe/0x10
[417936.105320]  [<ffffffff812399c7>] xfs_flush_pages+0x67/0xb0
[417936.145237]  [<ffffffff8122fc7a>] xfs_setattr+0x9da/0xa80
[417936.185206]  [<ffffffff8123c8a6>] xfs_vn_setattr+0x16/0x20
[417936.224767]  [<ffffffff8112d9bd>] notify_change+0x11d/0x300
[417936.264369]  [<ffffffff8111328c>] do_truncate+0x5c/0x90
[417936.303371]  [<ffffffff81113387>] sys_ftruncate+0xc7/0xd0
[417936.341555]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b
[417936.379030] INFO: task rsync:25549 blocked for more than 120 seconds.
[417936.417012] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
[417936.455324] rsync           D ffff88042daa35a0     0 25549   5627
0x00000000
[417936.493927]  ffff8801fd84b808 0000000000000082 ffff8801fd84b7b8
0000000000000000
[417936.532884]  0000000000010800 ffff8801fd84bfd8 ffff8801fd84a010
0000000000010800
[417936.571574]  ffff8801fd84bfd8 0000000000010800 ffffffff81a0b020
ffff88042daa3040
[417936.611035] Call Trace:
[417936.649563]  [<ffffffff8147aa2a>] schedule+0x3a/0x60
[417936.688577]  [<ffffffff8147ae65>] schedule_timeout+0x1d5/0x2b0
[417936.728017]  [<ffffffff81201098>] ? xfs_da_do_buf+0x5f8/0x760
[417936.767408]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[417936.806501]  [<ffffffff8147bc59>] __down+0x69/0xb0
[417936.845640]  [<ffffffff812366d6>] ? _xfs_buf_find+0xf6/0x270
[417936.885062]  [<ffffffff81078c9c>] down+0x3c/0x50
[417936.925012]  [<ffffffff81235ab0>] xfs_buf_lock+0x40/0xe0
[417936.964598]  [<ffffffff812366d6>] _xfs_buf_find+0xf6/0x270
[417937.004007]  [<ffffffff812368a9>] xfs_buf_get+0x59/0x190
[417937.043544]  [<ffffffff81236f07>] xfs_buf_read+0x27/0x100
[417937.082900]  [<ffffffff8122d118>] xfs_trans_read_buf+0x1e8/0x410
[417937.122465]  [<ffffffff811e3741>] xfs_read_agf+0x61/0x1a0
[417937.161379]  [<ffffffff811e38b4>] xfs_alloc_read_agf+0x34/0xd0
[417937.200439]  [<ffffffff811e6c11>] xfs_alloc_fix_freelist+0x401/0x470
[417937.239532]  [<ffffffff81108e3b>] ? kmem_cache_free+0x1b/0xf0
[417937.278505]  [<ffffffff8122a0ae>] ? xfs_trans_free_item_desc+0x2e/0x30
[417937.317752]  [<ffffffff8122a1e7>] ? xfs_trans_free_items+0x87/0xb0
[417937.356843]  [<ffffffff81226c73>] ? xfs_perag_get+0x33/0xb0
[417937.396008]  [<ffffffff811e6d17>] ? xfs_free_extent+0x97/0x120
[417937.435181]  [<ffffffff811e6d28>] xfs_free_extent+0xa8/0x120
[417937.473635]  [<ffffffff81232127>] ? kmem_zone_alloc+0x77/0xf0
[417937.511878]  [<ffffffff811f01cd>] xfs_bmap_finish+0x15d/0x1a0
[417937.553173]  [<ffffffff8121717f>] xfs_itruncate_finish+0x15f/0x330
[417937.595039]  [<ffffffff8122ebe3>] xfs_inactive+0x313/0x470
[417937.633483]  [<ffffffff8123f2ba>] xfs_fs_evict_inode+0xba/0x100
[417937.671809]  [<ffffffff8112c99b>] evict+0x7b/0x140
[417937.710082]  [<ffffffff8112cdd8>] iput_final+0xc8/0x180
[417937.748325]  [<ffffffff8112cec3>] iput+0x33/0x40
[417937.786032]  [<ffffffff811230c4>] do_unlinkat+0x164/0x1c0
[417937.823972]  [<ffffffff81114815>] ? vfs_write+0x125/0x190
[417937.861674]  [<ffffffff81002dcb>] ? math_state_restore+0x3b/0x70
[417937.899449]  [<ffffffff81123131>] sys_unlink+0x11/0x20
[417937.936815]  [<ffffffff8147d2bb>] system_call_fastpath+0x16/0x1b

Any ideas what this is?

mount options are:rw,noatime,nodiratime,inode64,logbufs=8,logbsize=256k

Greets,
Stefan

<Prev in Thread] Current Thread [Next in Thread>