[Top] [All Lists]

Re: XFS hang during xfs_fsr run

To: Dave Chinner <david@xxxxxxxxxxxxx>
Subject: Re: XFS hang during xfs_fsr run
From: Michael Weissenbacher <mw@xxxxxxxxxxxx>
Date: Fri, 12 Mar 2010 10:45:19 +0100
Cc: Christoph Hellwig <hch@xxxxxxxxxxxxx>, Eric Sandeen <sandeen@xxxxxxxxxxx>, xfs@xxxxxxxxxxx
In-reply-to: <20100311233934.GB4732@dastard>
References: <4B8FA2CD.6010904@xxxxxxxxxxxx> <20100304131511.GH14317@xxxxxxxxxxxxxxxx> <20100304134641.GA26871@xxxxxxxxxxxxx> <4B8FC1B7.3070505@xxxxxxxxxxxx> <20100304222611.GK14317@xxxxxxxxxxxxxxxx> <4B92C71C.5010003@xxxxxxxxxxxx> <20100308000601.GF28189@xxxxxxxxxxxxxxxx> <4B94EADD.2080108@xxxxxxxxxxxx> <4B953D3F.3090002@xxxxxxxxxxx> <4B975C5C.5090806@xxxxxxxxxxxx> <20100311233934.GB4732@dastard>
User-agent: Thunderbird (X11/20090817)
Hi Dave!
Hi Michael - have you got any idea what the files are that are
hitting this? This failure is implying that the inode is still dirty
after syncing all the data. Is something trying to modify it while
XFS is trying to map it?
Yes, as far as i can tell it's always a file that some process is currently modifying. It happens ofter with some file unter /var/log which syslog is currently modifying. I tried setting the "no-defrag" flag via xfs_io's chattr on all log files but that didn't seem to help. It seems that cyrus imapd is triggering this problem far more likely than any other program. Some examples of files where it usually hangs:
/var/spool/imap/x/user/xxxx/cyrus.cache (lsof -> cyrus)
/var/imap/db/log.xxxxxxx (lsof -> cyrus)
/var/log/xxx.log (lsof -> syslog)

We've seen this occasionally in the past, but we've never been able
to reproduce it with any reliability.  Hence any information you can
extract would certainly help us here.
I've also seen this happen occasionally before, but recently it started to happen on about every second run of xfs_fsr. Because this is a production system i've ceased using xfs_fsr for the moment. I've not been able to reproduce it on others systems yet.


<Prev in Thread] Current Thread [Next in Thread>