On Thu, Mar 06, 2008 at 04:31:27PM -0500, Kris Kersey wrote:
> Hello,
>
> I'm working on a NAS product and we're currently having lock-ups that
> seem to be hanging in XFS code. We're running a NAS that has 1024 NFSD
> threads accessing three RAID mounts. All three mounts are running XFS
> file systems. Lately we've had random lockups on these boxes and I am
> now running a kernel with KDB built-in.
>
> The lock-up takes the form of all NFSD threads in D state with one out
> of three pdflush threads in D state. The assumption can be made that
> all NFSD threads are waiting on the one pdflush thread to complete. So
> two times now when an NAS has gotten in this state I have accessed KDB
> and ran a stack trace on the pdflush thread. Both times the thread was
> stuck on xlog_grant_log_space+0xdb.
Try bumping XFS_TRANS_PUSH_AIL_RESTARTS to a much larger number and
seeing if the problem goes away....
Alternatively, that restart hack is backed by a "watchdog" timeout
in 2.6.25-rc1, so if that is the cause of the problem perhaps the
latest -rcX kernel will prevent the hang?
BTW, you can get all the traces of D state threads through the sysrq
interface, so you don't need to drop into kdb to get this.....
Cheers,
Dave.
--
Dave Chinner
Principal Engineer
SGI Australian Software Group
|