xfs
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: Failing XFS filesystem underlying Ceph OSDs

from [Alex Gorbachev] [Permanent Link][Original]
To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: Failing XFS filesystem underlying Ceph OSDs
From: Alex Gorbachev <ag@xxxxxxxxxxxxxxxxxxx>
Date: Sun, 5 Jul 2015 00:25:47 -0400
Cc: xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
In-reply-to: <20150704233802.GS7943@dastard>
References: <CADb9451tB71D3XCqcOkDxzpzbdEHqwj7XCZUpL8yg1DzYbpwBw@xxxxxxxxxxxxxx> <20150703235141.GQ7943@dastard> <CADb9451j0WEeL4m-fEHzrOf3ys+eSHJKNzcN9oHAOs2kDLL_kQ@xxxxxxxxxxxxxx> <20150704233802.GS7943@dastard>
Hi Dave,
Â
> sysctl vm.swappiness=20 (can probably be 1 as per article)
>
> sysctl vm.min_free_kbytes=262144

That's not an explanation for what looks to be page cache radix
tree coruption. Memory reclaim still occurs with the settings you
have now and, well, those changes occurred back in 3.5 - some
3 years ago - so it's not really an explanation for a problem with a
recent 4.1 kernel...

> So far no issues, but I need to wait a week to see if anything shows up.
> Thank you for reviewing the error codes.

I expect that you'll see the problems again...

We have experienced the problem in various guises with kernels 3.14, 3.19, 4.1-rc2 and now 4.1, so it's not new to us, just different error stack. Below are some other stack dumps of what manifested as the same error. Â

Do you think we need to look at RAM handling by this Supermicro machine type?Â

Best regards,
Alex
Â

Cheers,

Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx

ÂApr 30 15:26:04 roc-4r-sca025 kernel: [1070485.999993] Â[<ffffffff817cf4b9>] schedule+0x29/0x70
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000010] Â[<ffffffffc07caee7>] _xfs_log_force+0x187/0x280 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000014] Â[<ffffffff810a4150>] ? try_to_wake_up+0x2a0/0x2a0
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000030] Â[<ffffffffc07cb019>] xfs_log_force+0x39/0xc0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000047] Â[<ffffffffc07d6542>] xfsaild_push+0x552/0x5a0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000050] Â[<ffffffff817d2264>] ? schedule_timeout+0x124/0x210
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000067] Â[<ffffffffc07d662f>] xfsaild+0x9f/0x140 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000083] Â[<ffffffffc07d6590>] ? xfsaild_push+0x5a0/0x5a0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000086] Â[<ffffffff81095e29>] kthread+0xc9/0xe0
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000089] Â[<ffffffff81095d60>] ? flush_kthread_worker+0x90/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000093] Â[<ffffffff817d3718>] ret_from_fork+0x58/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000096] Â[<ffffffff81095d60>] ? flush_kthread_worker+0x90/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000100] INFO: task xfsaild/sdg1:2606 blocked for more than 120 seconds.
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000113] Â Â Â Not tainted 3.19.4-031904-generic #201504131440
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000125] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000140] xfsaild/sdg1 Â ÂD ffff8810102dfcd8 Â Â 0 Â2606 Â Â Â2 0x00000000
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000143] Âffff8810102dfcd8 ffff8810344c2d80 ffff8810102dffd8 0000000000013e40
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000146] Âffff881033383b80 ffff8810389589d0 ffff881033491d70 ffff8810102dfcf8
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000149] Âffff8810344c3100 ffff881033491d70 ffff881032acd128 ffff88107fc75100
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000152] Call Trace:
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000155] Â[<ffffffff817cf4b9>] schedule+0x29/0x70
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000172] Â[<ffffffffc07caee7>] _xfs_log_force+0x187/0x280 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000175] Â[<ffffffff810a4150>] ? try_to_wake_up+0x2a0/0x2a0
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000192] Â[<ffffffffc07cb019>] xfs_log_force+0x39/0xc0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000208] Â[<ffffffffc07d6542>] xfsaild_push+0x552/0x5a0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000211] Â[<ffffffff817d2264>] ? schedule_timeout+0x124/0x210
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000228] Â[<ffffffffc07d662f>] xfsaild+0x9f/0x140 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000244] Â[<ffffffffc07d6590>] ? xfsaild_push+0x5a0/0x5a0 [xfs]
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000247] Â[<ffffffff81095e29>] kthread+0xc9/0xe0
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000251] Â[<ffffffff81095d60>] ? flush_kthread_worker+0x90/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000254] Â[<ffffffff817d3718>] ret_from_fork+0x58/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000257] Â[<ffffffff81095d60>] ? flush_kthread_worker+0x90/0x90
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000261] INFO: task xfsaild/sdd1:2707 blocked for more than 120 seconds.
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000273] Â Â Â Not tainted 3.19.4-031904-generic #201504131440
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000285] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000300] xfsaild/sdd1 Â ÂD ffff88100f813cd8 Â Â 0 Â2707 Â Â Â2 0x00000000
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000303] Âffff88100f813cd8 ffff88102b1d9480 ffff88100f813fd8 0000000000013e40
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000306] Âffff881033387700 ffff881038959d70 ffff8810351a09d0 ffff88100f813cf8
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000310] Âffff88102b1d8800 ffff8810351a09d0 ffff8810329acd28 ffff88107fcb5100
Apr 30 15:26:04 roc-4r-sca025 kernel: [1070486.000313] Call Trace:



Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.414753] BUG: unable to handle kernel NULL pointer dereference at      (null)
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.414779] IP: [<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.414823] PGD 1013574067 PUD 100ffcb067 PMD 0Â
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.414842] Oops: 0000 [#1] SMPÂ
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.414856] Modules linked in: xfs libcrc32c intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp ipmi_ssif kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd sb_edac edac_core joydev mei_me mei lpc_ich ioatdma ipmi_si 8250_fintek ipmi_msghandler 8021q garp mrp stp llc wmi bonding mac_hid lp parport mlx4_en vxlan ip6_udp_tunnel udp_tunnel hid_generic usbhid mpt2sas ahci igb hid libahci i2c_algo_bit mlx4_core dca raid_class ptp scsi_transport_sas pps_core arcmsr
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415068] CPU: 0 PID: 687310 Comm: ceph-osd Not tainted 4.1.0-040100rc2-generic #201505032335
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415084] Hardware name: Supermicro X9DRD-7LN4F(-JBOD)/X9DRD-EF/X9DRD-7LN4F, BIOS 3.0a 12/05/2013
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415102] task: ffff88100f33eeb0 ti: ffff880065cb0000 task.ti: ffff880065cb0000
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415116] RIP: 0010:[<ffffffffc04be80f>] Â[<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415155] RSP: 0018:ffff880065cb32c8 ÂEFLAGS: 00010286
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415165] RAX: 0000000000000000 RBX: ffff880065cb37b0 RCX: 0000000000000000
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415210] RDX: ffff880065cb32f4 RSI: ffff880065cb32f0 RDI: ffff880004000000
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415254] RBP: ffff880065cb32c8 R08: ffff880b687bf478 R09: ffff880065cb3270
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415297] R10: dead000000100100 R11: dead000000200200 R12: ffffea000ccf9840
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415341] R13: ffffea000ccf9860 R14: ffffea000ccf9840 R15: ffff880ab6ce39d0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415386] FS: Â00007ff4be95c700(0000) GS:ffff88103f200000(0000) knlGS:0000000000000000
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415432] CS: Â0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415459] CR2: 0000000000000000 CR3: 000000007942d000 CR4: 00000000000407f0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415502] Stack:
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415523] Âffff880065cb3328 ffffffffc04be880 0000000000000000 ffffea000ccf9840
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415574] Âffff880065cb3498 0000000000000000 ffffea000ccf9840 ffff880065cb37b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415626] Âffff880065cb3498 ffffea000ccf9860 ffffea000ccf9840 ffff880ab6ce3b28
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415677] Call Trace:
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415718] Â[<ffffffffc04be880>] xfs_vm_releasepage+0x40/0x120 [xfs]
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415751] Â[<ffffffff8118a7d2>] try_to_release_page+0x32/0x50
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415782] Â[<ffffffff8119fe6d>] shrink_page_list+0x69d/0x720
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415812] Â[<ffffffff811a058d>] shrink_inactive_list+0x1dd/0x5d0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415843] Â[<ffffffff8119bf2d>] ? get_lru_size+0x3d/0x40
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415872] Â[<ffffffff811a0fb4>] shrink_lruvec+0x234/0x370
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415901] Â[<ffffffff81198e25>] ? pagevec_lru_move_fn+0xf5/0x110
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415931] Â[<ffffffff811a11db>] shrink_zone+0xeb/0x300
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415961] Â[<ffffffff811b38fd>] ? compact_zone+0x7d/0x4f0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.415990] Â[<ffffffff811a1864>] shrink_zones+0x104/0x2a0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416019] Â[<ffffffff811b3f6d>] ? compact_zone_order+0x5d/0x70
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416051] Â[<ffffffff810f0666>] ? ktime_get+0x46/0xb0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416079] Â[<ffffffff811a1ad7>] do_try_to_free_pages+0xd7/0x160
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416110] Â[<ffffffff811a1e17>] try_to_free_pages+0xb7/0x170
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416139] Â[<ffffffff8119450a>] __alloc_pages_nodemask+0x5ba/0x9c0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416172] Â[<ffffffff811db51c>] alloc_pages_current+0x9c/0x110
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416203] Â[<ffffffff811e3a8a>] allocate_slab+0x20a/0x2e0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416232] Â[<ffffffff811e3b91>] new_slab+0x31/0x1f0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416261] Â[<ffffffff817ef6a4>] __slab_alloc+0x18e/0x2a3
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416290] Â[<ffffffff816cea97>] ? __alloc_skb+0x87/0x2b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416318] Â[<ffffffff816cea67>] ? __alloc_skb+0x57/0x2b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416348] Â[<ffffffff811e8687>] __kmalloc_node_track_caller+0xb7/0x290
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416379] Â[<ffffffff816cea97>] ? __alloc_skb+0x87/0x2b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416407] Â[<ffffffff816ce5fc>] __kmalloc_reserve.isra.57+0x3c/0xa0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416437] Â[<ffffffff816cea97>] __alloc_skb+0x87/0x2b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416467] Â[<ffffffff8172ee91>] sk_stream_alloc_skb+0x41/0x130
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416496] Â[<ffffffff8172f963>] tcp_sendmsg+0x2d3/0xa90
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416526] Â[<ffffffff8175b467>] inet_sendmsg+0x67/0xa0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416555] Â[<ffffffff816c5cb4>] ? copy_msghdr_from_user+0x154/0x1b0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416585] Â[<ffffffff816c4f5d>] sock_sendmsg+0x4d/0x60
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416613] Â[<ffffffff816c61f3>] ___sys_sendmsg+0x2b3/0x2c0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416644] Â[<ffffffff810fcf67>] ? futex_wait+0x1c7/0x2c0
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416673] Â[<ffffffff810fd122>] ? futex_wake+0x72/0x140
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416703] Â[<ffffffff81221185>] ? __fget_light+0x25/0x70
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416731] Â[<ffffffff816c6c19>] __sys_sendmsg+0x49/0x90
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416760] Â[<ffffffff816c6c79>] SyS_sendmsg+0x19/0x20
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416788] Â[<ffffffff81803bb2>] system_call_fastpath+0x16/0x75
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.416816] Code: 07 48 89 e5 f6 c4 08 74 43 48 8b 7f 30 48 89 f8 eb 19 66 2e 0f 1f 84 00 00 00 00 00 c7 02 01 00 00 00 48 8b 40 08 48 39 c7 74 1f <48> 8b 08 80 e5 10 75 e9 48 8b 08 80 e5 02 74 e7 c7 06 01 00 00Â
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.417040] RIP Â[<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.417094] ÂRSP <ffff880065cb32c8>
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.417118] CR2: 0000000000000000
Jun 26 07:02:56 roc-4r-sca015 kernel: [462437.417402] ---[ end trace 71135a48b0dfd313 ]---
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759401] BUG: unable to handle kernel NULL pointer dereference at      (null)
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759457] IP: [<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759519] PGD 0Â
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759543] Oops: 0000 [#2] SMPÂ
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759571] Modules linked in: xfs libcrc32c intel_rapl iosf_mbi x86_pkg_temp_thermal intel_powerclamp coretemp ipmi_ssif kvm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper cryptd sb_edac edac_core joydev mei_me mei lpc_ich ioatdma ipmi_si 8250_fintek ipmi_msghandler 8021q garp mrp stp llc wmi bonding mac_hid lp parport mlx4_en vxlan ip6_udp_tunnel udp_tunnel hid_generic usbhid mpt2sas ahci igb hid libahci i2c_algo_bit mlx4_core dca raid_class ptp scsi_transport_sas pps_core arcmsr
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759889] CPU: 7 PID: 110 Comm: kswapd0 Tainted: G Â Â ÂD Â Â Â Â 4.1.0-040100rc2-generic #201505032335
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759939] Hardware name: Supermicro X9DRD-7LN4F(-JBOD)/X9DRD-EF/X9DRD-7LN4F, BIOS 3.0a 12/05/2013
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.759987] task: ffff88103712bc60 ti: ffff8810331bc000 task.ti: ffff8810331bc000
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760032] RIP: 0010:[<ffffffffc04be80f>] Â[<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760100] RSP: 0018:ffff8810331bf818 ÂEFLAGS: 00010282
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760127] RAX: 0000000000000000 RBX: ffffea000cd08d80 RCX: 0000000000000000
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760171] RDX: ffff8810331bf844 RSI: ffff8810331bf840 RDI: ffff880004000068
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760216] RBP: ffff8810331bf818 R08: ffff880b687bf478 R09: ffff8810331bf850
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760260] R10: ffff880b687bf518 R11: 0000000000000220 R12: ffffea000cd08d80
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760304] R13: 0000000000000000 R14: ffff880ab6ce3b28 R15: ffff880ab6ce39d0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760348] FS: Â0000000000000000(0000) GS:ffff88103f3c0000(0000) knlGS:0000000000000000
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760394] CS: Â0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760422] CR2: 0000000000000000 CR3: 0000000001e0f000 CR4: 00000000000407e0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760466] Stack:
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760487] Âffff8810331bf878 ffffffffc04be880 ffff880ab6ce3b40 ffff880ab6ce3b28
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760539] Âffff8810331bf888 0000000000000000 ffff880b687bf478 ffffea000cd08d80
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760591] Âffff880ab6ce3b28 0000000000000000 ffff880ab6ce3b28 000000000000000b
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760643] Call Trace:
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760686] Â[<ffffffffc04be880>] xfs_vm_releasepage+0x40/0x120 [xfs]
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760720] Â[<ffffffff8118a7d2>] try_to_release_page+0x32/0x50
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760752] Â[<ffffffff8119b431>] invalidate_inode_page+0x71/0x90
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760783] Â[<ffffffff8119b528>] invalidate_mapping_pages+0xd8/0x1e0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760815] Â[<ffffffff8121ff22>] inode_lru_isolate+0x122/0x1b0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760846] Â[<ffffffff811b54b1>] __list_lru_walk_one.isra.6+0x81/0x150
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760877] Â[<ffffffff8121fe00>] ? insert_inode_locked+0x190/0x190
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760907] Â[<ffffffff8121fe00>] ? insert_inode_locked+0x190/0x190
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760938] Â[<ffffffff811b566a>] list_lru_walk_one+0x4a/0x60
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760966] Â[<ffffffff81220274>] prune_icache_sb+0x44/0x60
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.760996] Â[<ffffffff81207505>] super_cache_scan+0x155/0x1a0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761026] Â[<ffffffff8119c06f>] do_shrink_slab+0x13f/0x2c0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761056] Â[<ffffffff811a10b0>] ? shrink_lruvec+0x330/0x370
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761086] Â[<ffffffff810aeb95>] ? sched_clock_cpu+0x85/0xc0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761115] Â[<ffffffff8119c358>] shrink_slab+0xd8/0x110
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761143] Â[<ffffffff811a13bf>] shrink_zone+0x2cf/0x300
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761173] Â[<ffffffff811aa500>] ? zone_statistics+0xa0/0xc0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761203] Â[<ffffffff811a16e0>] kswapd_shrink_zone+0xe0/0x160
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761232] Â[<ffffffff811a2323>] balance_pgdat+0x2c3/0x430
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761261] Â[<ffffffff811a25c3>] kswapd+0x133/0x250
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761289] Â[<ffffffff811a2490>] ? balance_pgdat+0x430/0x430
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761322] Â[<ffffffff8109cdc9>] kthread+0xc9/0xe0
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761350] Â[<ffffffff8109cd00>] ? flush_kthread_worker+0x90/0x90
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761381] Â[<ffffffff81803fe2>] ret_from_fork+0x42/0x70
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761410] Â[<ffffffff8109cd00>] ? flush_kthread_worker+0x90/0x90
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761439] Code: 07 48 89 e5 f6 c4 08 74 43 48 8b 7f 30 48 89 f8 eb 19 66 2e 0f 1f 84 00 00 00 00 00 c7 02 01 00 00 00 48 8b 40 08 48 39 c7 74 1f <48> 8b 08 80 e5 10 75 e9 48 8b 08 80 e5 02 74 e7 c7 06 01 00 00Â
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761669] RIP Â[<ffffffffc04be80f>] xfs_count_page_state+0x3f/0x70 [xfs]
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761725] ÂRSP <ffff8810331bf818>
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.761749] CR2: 0000000000000000
Jun 26 07:40:55 roc-4r-sca015 kernel: [464718.762039] ---[ end trace 71135a48b0dfd314 ]---



[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  Re: Failing XFS filesystem underlying Ceph OSDs, Dave Chinner
Next by Date:  Re: Failing XFS filesystem underlying Ceph OSDs, Dave Chinner
Previous by Thread:  Re: Failing XFS filesystem underlying Ceph OSDs, Dave Chinner
Next by Thread:  Re: Failing XFS filesystem underlying Ceph OSDs, Dave Chinner
Indexes:  [Date] [Thread] [Top] [All Lists]