xfs
[Top] [All Lists]

Re: Still seeing hangs in xlog_grant_log_space

To: xfs@xxxxxxxxxxx
Subject: Re: Still seeing hangs in xlog_grant_log_space
From: Chris J Arges <chris.j.arges@xxxxxxxxxxxxx>
Date: Wed, 6 Jun 2012 08:00:00 -0700 (PDT)
In-reply-to: <20120518101010.GW25351@dastard>
References: <CADLDEKsP4DsXf_G07ub+a-ODbrJbsiprRJUX1fJdaQ41TB7+Xg@xxxxxxxxxxxxxx> <20120423143843.GN9541@dastard> <CADLDEKvFF3FvEHVtmwdWhbM58_jrCRX+Uk9vLBg1hA8sizh5BQ@xxxxxxxxxxxxxx> <20120423235840.GQ9541@dastard> <CADLDEKsfckBw2oVYFfaaTbpe8Ri+rYJr2e5SB7-pM0BU9nRUeA@xxxxxxxxxxxxxx> <20120424120731.GT9541@dastard> <CADLDEKs01GnxgYh2UTt1waVDUXHbB_RcBcUTBr5REFg5aD5jHA@xxxxxxxxxxxxxx> <20120425223845.GX9541@dastard> <CADLDEKvYkpUnMrqdMyqCmsYrZcUtiJ6ZRhrRu_ERTjn=r7M3Pg@xxxxxxxxxxxxxx> <20120426224412.GA9541@dastard> <CADLDEKs6oMDA-6OhmcFxyRoBVpduKtSput=53TQGn9NCAOXC1Q@xxxxxxxxxxxxxx> <20120426230738.GB9541@dastard> <CADLDEKuKLeYiqhQW0E9g_bS0VXoxPGPOck3N004Pxg4_Opbzow@xxxxxxxxxxxxxx> <20120427110922.GF9541@dastard> <CADLDEKtUHAGcOPT1jtcvyJVk+zsoL5_thYFtHJYs+w=6EGuVSA@xxxxxxxxxxxxxx> <CADLDEKs4YbNzj2c0HKHwSdUfKy0efdQRe1rOsWDkWUgd+BOGHw@xxxxxxxxxxxxxx> <20120507171908.GA16881@xxxxxxx> <CADLDEKvgT_FcGhJKoPaQv0mh_Jqdaqu8SYatc9xxU7vOY217YQ@xxxxxxxxxxxxxx> <loom.20120510T180646-433@post. gmane.org> <20120516184231.GK16099@xxxxxxx> <4FB3FA1D.6050102@xxxxxxxxxxxxx> <4FB41C1D.8000808@xxxxxxx> <20120518101010.GW25351@dastard>

Dave Chinner wrote:
> 
> It seems unlikely, but if you turn on kmemleak it might find a
> memory leak or overwrite that is causing this.
> 

Running kmemleak on a 3.0.0 kernel results in the following:

dmesg:
[ 3855.751393] XFS (sda2): xlog_verify_grant_tail: space >
BBTOB(tail_blocks)
[22987.932317] kmemleak: 1 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)

kmemleak:
unreferenced object 0xffff88015612a340 (size 208):
 comm "copy-files", pid 1483, jiffies 4310560285 (age 16571.656s)
 hex dump (first 32 bytes):
    00 f3 1b 57 01 88 ff ff 00 08 00 58 01 88 ff ff  ...W.......X....
    a0 a7 32 a0 ff ff ff ff 04 00 00 00 00 00 00 00  ..2.............
 backtrace:
    [<ffffffff815c6196>] kmemleak_alloc+0x26/0x50
    [<ffffffff811542e3>] kmem_cache_alloc+0x123/0x190
    [<ffffffffa0313967>] kmem_zone_alloc+0x67/0xe0 [xfs]
    [<ffffffffa03139fd>] kmem_zone_zalloc+0x1d/0x50 [xfs]
    [<ffffffffa02b033f>] xfs_allocbt_init_cursor+0xdf/0x130 [xfs]
    [<ffffffffa02adb4c>] xfs_alloc_ag_vextent_near+0x6c/0xd80 [xfs]
    [<ffffffffa02aea88>] xfs_alloc_ag_vextent+0x228/0x290 [xfs]
    [<ffffffffa02af7d9>] xfs_alloc_vextent+0x649/0x8c0 [xfs]
    [<ffffffffa02bcfc6>] xfs_bmap_btalloc+0x286/0x7c0 [xfs]
    [<ffffffffa02bd521>] xfs_bmap_alloc+0x21/0x40 [xfs]
    [<ffffffffa02c6ba3>] xfs_bmapi+0xdc3/0x1950 [xfs]
    [<ffffffffa02f5059>] xfs_iomap_write_allocate+0x179/0x340 [xfs]
    [<ffffffffa03147d5>] xfs_map_blocks+0x215/0x380 [xfs]
    [<ffffffffa0315792>] xfs_vm_writepage+0x1b2/0x510 [xfs]
    [<ffffffff811142e7>] __writepage+0x17/0x40
    [<ffffffff8111485d>] write_cache_pages+0x20d/0x460

analysis:
    I turned on kmemleak, function tracing, and xfs debugging in the build
in which I ran this. So far I’ve been able to run the copy-files script for
about 24 hrs  without failure. I’m not sure if this is because all these
features are turned on and it has slowed something down (so it takes longer
to reproduce), or if the debugging code is changing the behavior. I’m not
sure if this backtrace is valid, so I’m attaching an annotated objdump of my
xfs module.

--chris j arges

http://old.nabble.com/file/p33970511/xfs_objdump_3.0.txt.tar.bz2
xfs_objdump_3.0.txt.tar.bz2 
-- 
View this message in context: 
http://old.nabble.com/Still-seeing-hangs-in-xlog_grant_log_space-tp33732886p33970511.html
Sent from the Xfs - General mailing list archive at Nabble.com.

<Prev in Thread] Current Thread [Next in Thread>