Hung in D state during fclose

[Cheung, Norman]

-----Original Message-----
From: xfs-bounces at oss.sgi.com [mailto:xfs-bounces at oss.sgi.com] On Behalf Of Dave Chinner
Sent: Tuesday, February 12, 2013 9:16 PM
To: Cheung, Norman
Cc: 'linux-xfs at oss.sgi.com'
Subject: Re: Hung in D state during fclose

On Wed, Feb 13, 2013 at 12:12:47AM +0000, Cheung, Norman wrote:
> Dave,
> 
> One other point I have forgotten to mention is that the parent thread 
> will wait for 5 minutes and then lower the thread priority (from -2 
> back to 20) and set a global variable to signal the threads to exit.  
> The blocked thread responded well and exit from D state and fclose 
> completed with no error.

So it's not hung - it's just very slow?

[NLC] It will pause forever.  I tried replaced the timeout with a sleep loop, and it will pause forever.    That is how I got the stack trace.

You have 256GB of memory. It's entirely possible that you've dirtied a large amount of memory and everything is simply stuck waiting for writeback to occur. Perhaps you should have a look at the utilisation of your disks when this still occurs. 'iostat -x -d -m 5' will give you some insight into utilsation when a hang occurs...

[NLC] I  have set the dirty_background_bytes to 40M and dirty_writeback_centisecs to 400; and watched the Meminfo.  I don't get a  lot of dirtied memory accumulation -- 7 to 10 M average.  The disk usage from the sar log was steady at 150M/sec for each of the disk; but a few seconds after the fclose all disk activities stopped. Also the CPU % was quiet as well.     

> This cause me to wonder if it  is possible that  some XFS threads and 
> my application thread might be in a deadlock.

Deadlocks are permanent, so what you are seeing is not a deadlock....

[NLC] Would it be possible that there is priority inversion between my  disk writing threads and the XFS threads & flush threads?  My application thread runs at -2 priority.

Many thanks,
Norman


Cheers,

Dave.
--
Dave Chinner
david at fromorbit.com

_______________________________________________
xfs mailing list
xfs at oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs