xfs
[Top] [All Lists]

Re: long hangs when deleting large directories (3.0-rc3)

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: long hangs when deleting large directories (3.0-rc3)
From: Markus Trippelsdorf <markus@xxxxxxxxxxxxxxx>
Date: Tue, 21 Jun 2011 20:57:01 +0200
Cc: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=simple; d=mail.ud10.udmedia.de; h= date:from:to:cc:subject:message-id:references:mime-version: content-type:in-reply-to; q=dns/txt; s=beta; bh=l1zN2FO6logWq66Q ix93thD/zW8+cDuUIpmdE0RjKOs=; b=BTteXcBTRXrx5v16ON2AC81vIVsCamFu WOAcsYgxGKtARLlPpHQLgg947PBiy3pz7A5gd5f4f7T7bcVxePX3l9xlY48kRt3P ZhCwOmzTaVMZxcqJTxKtU6cF8t4Qso4rviQUodA1xpwMRt9VSNWPJdFh1Ngy50z2 5fvkTpwtDSE=
In-reply-to: <20110621182414.GA1723@xxxxxxxxxxxxxx>
References: <20110619222447.GI561@dastard> <20110620005415.GA1730@xxxxxxxxxxxxxx> <20110620013449.GO561@dastard> <20110620020236.GB1730@xxxxxxxxxxxxxx> <20110620023625.GP561@dastard> <20110620060351.GC1730@xxxxxxxxxxxxxx> <20110620111359.GA12632@xxxxxxxxxxxxxx> <20110621042513.GN32466@dastard> <20110621080219.GA1684@xxxxxxxxxxxxxx> <20110621182414.GA1723@xxxxxxxxxxxxxx>
On 2011.06.21 at 20:24 +0200, Markus Trippelsdorf wrote:
> On 2011.06.21 at 10:02 +0200, Markus Trippelsdorf wrote:
> > On 2011.06.21 at 14:25 +1000, Dave Chinner wrote:
> > > That is, you really need to get a profile of the rm -rf process - or
> > > whatever is consuming the CPU time - (e.g. via perf or ftrace)
> > > across the hang to so we can narrow down the potential cause of the
> > > latency. Speaking of which, latencytop might be helpful in
> > > identifying where input is getting held up....
> > 
> > I've recorded a profile with "perf record -g /home/markus/rm_sync"
> > ~ % cat rm_sync
> > rm -fr /mnt/tmp/tmp/linux && sync
> 
> FWIW here are two links to svg time-charts produced by:
> 
> perf timechart record /home/markus/rm_sync
> 
> http://trippelsdorf.de/timechart1.svg
> http://trippelsdorf.de/timechart2.svg
> 

And this is what the mysterious kworker is doing during the sync.
It's the one consuming most of the CPU time.

    39.96%      kworker/3:0  [kernel.kallsyms]                            
0xffffffff811da9da k [k] xfs_trans_ail_update_bulk                         
                |                                                               
                                                                     
                --- xfs_trans_ail_update_bulk                                   
                                                                     
                    xfs_trans_committed_bulk                                    
                                                                     
                    xlog_cil_committed                                          
                                                                     
                    xlog_state_do_callback                                      
                                                                     
                    xlog_state_done_syncing                                     
                                                                     
                    xlog_iodone                                                 
                                                                     
                    xfs_buf_iodone_work                                         
                                                                     
                    process_one_work                                            
                                                                     
                    worker_thread                                               
                                                                     
                    kthread                                                     
                                                                     
                    kernel_thread_helper                                        
                                                                     
           
-- 
Markus

<Prev in Thread] Current Thread [Next in Thread>