xfs
[Top] [All Lists]

Re: 3.14-rc2 XFS backtrace because irqs_disabled.

To: Dave Chinner <david@xxxxxxxxxxxxx>, Jens Axboe <axboe@xxxxxxxxx>, Tejun Heo <tj@xxxxxxxxxx>, Steven Rostedt <rostedt@xxxxxxxxxxx>
Subject: Re: 3.14-rc2 XFS backtrace because irqs_disabled.
From: Linus Torvalds <torvalds@xxxxxxxxxxxxxxxxxxxx>
Date: Tue, 11 Feb 2014 22:59:58 -0800
Cc: Dave Jones <davej@xxxxxxxxxx>, Al Viro <viro@xxxxxxxxxxxxxxxxxx>, Eric Sandeen <sandeen@xxxxxxxxxxx>, Linux Kernel <linux-kernel@xxxxxxxxxxxxxxx>, xfs@xxxxxxxxxxx
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=N8DzgKkLS2JUW73TMf0eO3np+svGw9GHpkoch5M59s0=; b=GACQjuQdlHrpBv70QKnjiuGFNRlliW89KZsWHDNw4009N00Girq4UjoAnOr5a6FVRD upOxPOUZdfPZgnR9qctuZ9DWeD8736f2iynkD+OMfqRi8pRbRZgrLZtHcmmbPsfIvAjD Ve76iNt3koEstJlAZZ+iZGNItFk49sqL7BYjNKYnJ+WwFS2uhU5No+2pC+awiHwYq5EV jink4+MnHJk9ZWQ9gq5fIW18Z1E4Ui8uhh266ETVA55eDCCzHrfHkyisQc3xlnodweRL kSNrJVQ05rLBvI17R7bmMisoMa/ZnzUB6MyisYIu7tmxZVq6dl68w05fL3RGgu1sFhjz zzkA==
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=N8DzgKkLS2JUW73TMf0eO3np+svGw9GHpkoch5M59s0=; b=Fh1PvkT28RVfVQP+nqqApQwfhz6R2r1qnp0u6UK+EfV/BE21nK1vyr2SS8fHBFTg+O rrNfY+ZdrOZDBEWiLr++QigXIwnzC82211cfokH0j+FmwcEUy6StAxMdhb4Yv908C/Xe tlTkiXsVME8vyTrJj1ie5u/FpTFSlxJF3D5tU=
In-reply-to: <20140212063150.GD13997@dastard>
References: <20140211210841.GM13647@dastard> <52FA9ADA.9040803@xxxxxxxxxxx> <20140212004403.GA17129@xxxxxxxxxx> <20140212010941.GM18016@xxxxxxxxxxxxxxxxxx> <CA+55aFwoWT-0A_KTkXMkNqOy8hc=YmouTMBgWUD_z+8qYPphjA@xxxxxxxxxxxxxx> <20140212040358.GA25327@xxxxxxxxxx> <20140212042215.GN18016@xxxxxxxxxxxxxxxxxx> <20140212054043.GB13997@dastard> <20140212055027.GA28502@xxxxxxxxxx> <20140212061038.GC13997@dastard> <20140212063150.GD13997@dastard>
Sender: linus971@xxxxxxxxx
On Tue, Feb 11, 2014 at 10:31 PM, Dave Chinner <david@xxxxxxxxxxxxx> wrote:
>
> FYI, just creating lots of files with open(O_CREAT):
>
> [  348.718357] fs_mark (4828) used greatest stack depth: 2968 bytes left
> [  348.769846] fs_mark (4814) used greatest stack depth: 2312 bytes left
> [  349.777717] fs_mark (4826) used greatest stack depth: 2280 bytes left
> [  418.139415] fs_mark (4928) used greatest stack depth: 1936 bytes left
> [  460.492282] fs_mark (4993) used greatest stack depth: 1336 bytes left
> [  544.825418] fs_mark (5104) used greatest stack depth: 1112 bytes left
> [  689.503970] fs_mark (5265) used greatest stack depth: 1000 bytes left
>
> We've got absolutely no spare stack space anymore in the IO path.
> And the IO path can't get much simpler than filesystem -> virtio
> block device.

Ugh, that's bad. A thousand bytes of stack space is much too close to
any limits.

Do you have the stack traces for these things so that we can look at
worst offenders?

If the new block-mq code is to blame, it needs to be fixed.
__virtblk_add_req() has a 300-byte stack frame, it seems. Looking
elsewhere, blkdev_issue_discard() has 350 bytes of stack frame, but is
hopefully not in any normal path - online discard is moronic, and I'm
assuming XFS doesn't do that.

There's a lot of 200+ byte stack frames in block/blk-core.s, and they
all seem to be of the type perf_trace_block_buffer() - things created
with DECLARE_EVENT_CLASS(), afaik. Why they all have 200+ bytes of
frame, I have no idea. That sounds like a potential disaster too,
although hopefully it's mostly leaf functions - but leaf functions
*deep* in the callchain. Tejun? Steven, why _do_ they end up with such
huge frames?

And if the stack use comes from the VFS layer, we can probably work on
that too. But I don't think that has really changed much lately..

                Linus

<Prev in Thread] Current Thread [Next in Thread>