xfs
[Top] [All Lists]

Re: [PATCH 2/2] xfs: mark the xfs-alloc workqueue as high priority

To: Eric Sandeen <sandeen@xxxxxxxxxxx>
Subject: Re: [PATCH 2/2] xfs: mark the xfs-alloc workqueue as high priority
From: Tejun Heo <tj@xxxxxxxxxx>
Date: Fri, 9 Jan 2015 13:23:10 -0500
Cc: Eric Sandeen <sandeen@xxxxxxxxxx>, xfs-oss <xfs@xxxxxxxxxxx>
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=YT2JWmsDfryxb/yA/O1apptzlUFVfaxebQ14NJX9QjA=; b=beroDwtjyFoT9NsESyEle1Iybw8PLdLBY9ZAVy7zMnlxvIwE0LlWrWWdn6d33iZ6Rp 0VHqcgcKoSod6hEgyZd5CCFlriLdIsAZkL/wyuYB+4DJ5yiyJMJB9X/c84kJkT9CyI6E C1h1qvzyhf70HbCuO3vnWSAL1VRrhXg0roVs0heoseMDwb8iLIYsCEXKJvQGRi9h55wC Hf+ugm3s0inQSDUb4QXjPZY4s0wbB8owLeGrWZzUOCkHZUz3vkVNfJ6nJTN4277BUp5e RN5NofQXKG7418YizCReXbY5tbFjnXPL2ty7HXNwVazbxpvh+5zdqf8bn5ha7OLyLPSk s8IQ==
In-reply-to: <54B019F4.8030009@xxxxxxxxxxx>
References: <54B01927.2010506@xxxxxxxxxx> <54B019F4.8030009@xxxxxxxxxxx>
Sender: Tejun Heo <htejun@xxxxxxxxx>
User-agent: Mutt/1.5.23 (2014-03-12)
Hello, Eric.

On Fri, Jan 09, 2015 at 12:12:04PM -0600, Eric Sandeen wrote:
> I had a case reported where a system under high stress
> got deadlocked.  A btree split was handed off to the xfs
> allocation workqueue, and it is holding the xfs_ilock
> exclusively.  However, other xfs_end_io workers are
> not running, because they are waiting for that lock.
> As a result, the xfs allocation workqueue never gets
> run, and everything grinds to a halt.

I'm having a difficult time following the exact deadlock.  Can you
please elaborate in more detail?

> To be honest, it's not clear to me how the workqueue
> subsystem manages this sort of thing.  But in testing,
> making the allocation workqueue high priority so that
> it gets added to the front of the pending work list,
> resolves the problem.  We did similar things for
> the xfs-log workqueues, for similar reasons.

Ummm, this feel pretty voodoo.  In practice, it'd change the order of
things being executed and may make certain deadlocks unlikely enough,
but I don't think this can be a proper fix.

> diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c
> index e5bdca9..9c549e1 100644
> --- a/fs/xfs/xfs_super.c
> +++ b/fs/xfs/xfs_super.c
> @@ -874,7 +874,7 @@ xfs_init_mount_workqueues(
>               goto out_destroy_log;
>  
>       mp->m_alloc_workqueue = alloc_workqueue("xfs-alloc/%s",
> -                     WQ_MEM_RECLAIM|WQ_FREEZABLE, 0, mp->m_fsname);
> +                    WQ_MEM_RECLAIM|WQ_FREEZABLE|WQ_HIGHPRI, 0, mp->m_fsname);

And this at least deserves way more explanation.

>       if (!mp->m_alloc_workqueue)
>               goto out_destroy_eofblocks;
>  
> 

Thanks.

-- 
tejun

<Prev in Thread] Current Thread [Next in Thread>