xfs
[Top] [All Lists]

Error during bulk removal of files

To: xfs@xxxxxxxxxxx
Subject: Error during bulk removal of files
From: Subranshu Patel <spatel.ml@xxxxxxxxx>
Date: Fri, 10 May 2013 21:47:51 +0530
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:date:message-id:subject:from:to :content-type; bh=eq0hK896ltbzkFk/9JeiwftGN79R9V9hQ+nNggIIY4g=; b=hu8iobWsRm0p1fvFTDEpGAQIs+i/JRo9COixi5ouXvoNfx6bzGKVRT5qFZ3iBzBkmk jIWhJ28e1iJM/pMY2Gp/60BvX1lzQqynY9phSTPm3Ha2AXx1viA03D0uzOn0qMBPabCo zpT8A6Uf9RM5SgsbnktzaWfZMX7jG/CLTo9gd02lUnifo+E+UZnG62/sYqKtKJ/NYsXX ylV7/UZWBuEP4RvV+MWkSzPFAtc7UZOU2NeXdb42GjwnZRojV9znR4+/AA+1g2lkNgBA FrSdi3SieXFyvLDpf3rLGr3hd/vdmKsB34M2Ewewfd9ouGQwOi4ygUSeYZ/VS99TlAgb LBAw==
I am using mdtest benchmarking tool to remove 7 million files of 1KB.
After some time, the mdtest process gets blocked.

Dmesg shows the following

--------------------->8------------

May 10 18:49:00 machine1 kernel: INFO: task xfssyncd/dm-3:8156 blocked
for more than 120 seconds.

May 10 18:49:00 machine1 kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.

May 10 18:49:00 machine1 kernel: xfssyncd/dm-3 D 0000000000000002 0
8156 2 0x00000080

May 10 18:49:00 machine1 kernel: ffff88185932bc70 0000000000000046
ffff88185e58a400 ffff8800282566e8

May 10 18:49:00 machine1 kernel: 00000000000176f6 ffff88185fb44080
ffff880c6100aaa0 ffffffff8160b400

May 10 18:49:00 machine1 kernel: ffff88185fb44638 ffff88185932bfd8
000000000000fb88 ffff88185fb44638

May 10 18:49:00 machine1 kernel: Call Trace:

May 10 18:49:00 machine1 kernel: [<ffffffffa056c492>] xlog_wait+0x72/0x90 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffff81060250>] ?
default_wake_function+0x0/0x20

May 10 18:49:00 machine1 kernel: [<ffffffffa056e08b>]
xlog_grant_log_space+0x3ab/0x520 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa0582d1a>] ?
kmem_zone_zalloc+0x3a/0x50 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffff8127f3ac>] ? random32+0x1c/0x20

May 10 18:49:00 machine1 kernel: [<ffffffffa057d201>] ?
xfs_trans_ail_push+0x21/0x80 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa056e2e6>]
xfs_log_reserve+0xe6/0x140 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa057bad0>]
xfs_trans_reserve+0xa0/0x210 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa055f0a3>]
xfs_fs_log_dummy+0x43/0x90 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa056c9c4>] ?
xfs_log_need_covered+0x94/0xd0 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa0591831>]
xfs_sync_worker+0x81/0x90 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa059171e>] xfssyncd+0x17e/0x210 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffffa05915a0>] ? xfssyncd+0x0/0x210 [xfs]

May 10 18:49:00 machine1 kernel: [<ffffffff81091d66>] kthread+0x96/0xa0

May 10 18:49:00 machine1 kernel: [<ffffffff8100c14a>] child_rip+0xa/0x20

May 10 18:49:00 machine1 kernel: [<ffffffff81091cd0>] ? kthread+0x0/0xa0

May 10 18:49:00 machine1 kernel: [<ffffffff8100c140>] ? child_rip+0x0/0x20

--------------------->8------------

The XFS filesystem size is 14TB (LVM), kernel version 2.6.32

Machine has a memory of 96GB

Please let me know if this is a known issue and have been already
fixed in higher kernel version.

--
Subranshu

<Prev in Thread] Current Thread [Next in Thread>