On Tue, Sep 18, 2012 at 08:49:57AM +1000, Dave Chinner wrote:
> On Mon, Sep 17, 2012 at 11:21:20AM -0500, Ben Myers wrote:
> > Hi Chris,
> > On Mon, Sep 17, 2012 at 09:51:37AM -0600, Chris Friesen wrote:
> > > We're running 2.6.27 (upgrading not currently possible, embedded product).
> > >
> > > We had a situation arise where we could see in stack traces that a
> > > number of tasks were stuck in vn_iowait(). That function takes a
> > > pointer to xfs_inode_t. Given that, would it be possible to work
> > > backwards to determine a filesystem path corresponding to that
> > > inode? I realize it would likely only go back to the head of the
> > > filesystem, but that would be fine.
> > Yep, it's possible.
> > echo t > /proc/sysrq-trigger won't give you the full stack, but you may be
> > able
> > to get an idea of what is going on, well enough to make some guesses as to
> > the
> > culprit. That's probably your best first step.
> Doesn't give you the path of the inode, or anything that can tell
> you what the inode number is so you could run "find <dir> -inum
> <num>" after the system has recovered.
> > If you have crash installed:
> > 1) disassemble vn_iowait and find out which register the xfs_inode_t
> > pointer is in.
> > 2) print out the full stacks of all the processes in the system. 'bt -f', I
> > think.
> 'bt -F <pid>' tells you what entries on the stack are of a
> particular symbol. i.e. they'll show up as [xfs_inode] instead of a
> number for all addresses from inside the xfs inode slab cache.
> comapring the output of 'bt -F' and 'bt -f' will get you the inode
> address without needing to be able to read x86-64 assembly
> > 3) grep for the xfs_inode_t pointer on those stacks.
> Which doesn't get you the path, either. From there, follow to the
> struct inode, and from there to the struct dentry and grab the file
> name, then walk the d_parent dentry links back up to the root and
> construct the path that way.
> Alternatively, I think you can walk from the top down by looking
> that the open files that a given process has and working down to the
> xfs_inode, but it's been a long time since I've had to come from
> that direction. Indeed, you might even be able to use lsof to do
> this and get an idea from userspace what files are currently
> open on the hung processes.
Yep, I misunderstood the question. I didn't mean to mislead you Chris!