We run the systemtap script in the background. The kmem_alloc message appeared
in the /var/log/messages. The call stack is captured by the systemtap is
follow.
0xffffffffa02a0030 : xfs_err+0x0/0x90 [xfs]
0xffffffffa02a853d : kmem_alloc+0xbd/0xf0 [xfs]
0xffffffffa02eeb6e : xfs_log_commit_cil+0x39e/0x4c0 [xfs]
0xffffffffa02a7d3c : xfs_trans_commit+0x11c/0x240 [xfs]
0xffffffffa02dd353 : xfs_remove+0x343/0x380 [xfs]
0xffffffffa029d93a : xfs_vn_unlink+0x5a/0xc0 [xfs]
0xffffffff811bde51
0xffffffff811be056 (inexact)
0xffffffff811b166e (inexact)
0xffffffff810823cc (inexact)
0xffffffff811c11b6 (inexact)
0xffffffff815f3219 (inexact)
Nov 5 18:13:21 csx-ceph1-003 kernel: XFS: possible memory allocation deadlock
in kmem_alloc (mode:0x8250)
#! /usr/bin/env stap
probe module("xfs").function("xfs_err").call
{
print_backtrace();
}
# nohup stap -o /var/systemtap/kmem_alloc_bt.out backtrace.stp &
Redhat support case is https://access.redhat.com/support/cases/#/case/01500239
Thanks,
-Al
-----Original Message-----
From: Dave Chinner [mailto:david@xxxxxxxxxxxxx]
Sent: Tuesday, October 13, 2015 1:26 AM
To: Al Lau (alau2)
Cc: xfs@xxxxxxxxxxx
Subject: Re: mkfs.xfs -n size=65536
On Tue, Oct 13, 2015 at 07:28:48AM +0000, Al Lau (alau2) wrote:
> Are the xfs_db and filefrag the utilities to use to determine file
> fragmentation?
>
> # df -k /var/kmem_alloc
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/sdf1 3905109820 3359385616 545724204 87% /var/kmem_alloc
> # xfs_db -r -c frag /dev/sdf1
> actual 438970, ideal 388168, fragmentation factor 11.57%
http://xfs.org/index.php/XFS_FAQ#Q:_The_xfs_db_.22frag.22_command_says_I.27m_over_50.25._Is_that_bad.3F
> # ls -l fragmented
> -rw-r--r--. 1 root root 3360239878657 Oct 13 07:25 fragmented #
> filefrag fragmented
> fragmented: 385533 extents found
That's a lot of extents, but for a 3TB sparse file that is being written in
random 4k blocks, that's expected and there's little you can do about it.
Preallocation of the file or use of extent size hints will reduce physical
fragmentation, but you only want to use those if the file will eventually
become non-sparse and sequential read IO performance is required...
i.e. the definition of "fragmented" really depends on the application, IO
patterns and whether the current physical layout is acheiving the desired
performance attributes of the file in question....
Cheers,
Dave.
--
Dave Chinner
david@xxxxxxxxxxxxx
|