On Thu, Jul 07, 2011 at 02:34:12PM -0500, Eric Sandeen wrote:
> It seems like you need an "exit strategy" - you probably cannot leave
> your fs mounted -forever- ...
Yes, of course. I didn't mean to imply that I'd leave it like this
I will be on vacation next week, and it's not really reschedulable. So
my plan was to ride the filesystem through next week, hope for the best,
then fix it when I return, rather than attempt to start a fix now and
risk ending up with a broken filesystem. Ideally I would preemptively
switch to a warm backup before I leave, but that won't be ready in time.
(I currently do have all the important data backed up, but it is to
various different spaces where I had free disk space. The warm backup
should be ready early next week if the filesystem does go belly-up
before I return.)
> If it were me, if possible, I'd make backups of the fs as it's mounted
> now, then umount it and make an xfs_metadump of it, restore that metadump
> to a new sparse file, and point xfs_repair at that metadata image file,
> to see what repair might do with it.
> If repair eats it alive, then we can look into more xfs_db type surgery
> to fix things up more nicely...
This sounds like a reasonable plan. It looks like xfs_metadump needs a
umounted or readonly filesystem in order to work properly; is there any
way to estimate how long such a dump would take, and how large it would
be from an almost-full 11TB fs with however many inodes it has (~19 million
IIRC)? I want to plan downtime and usable disk space accordingly.
Would xfs_metadump create the same dump from a filesystem remounted ro
as from a filesystem not mounted? I think you suggested this idea in
an earlier post. In a very optimistic scenario, I could imagine
remounting the original filesystem ro, taking the metadump, then being
able to remount rw so that I could put it back into service while I
work with the metadump. Then, once I knew more about the metadump, I
could do an actual umount and fix the filesystem using the information
gathered from the metadump testing. If they will produce the same
metadump, then it could be a win-win if it's able to successfully
remount rw afterwards; and if it can't, it wasn't any additional effort
or risk to try.
Will xfsprogs 3.1.5 work with the older kernel, and will it make a
better dump than 2.9.4? I have built xfsprogs from source, but if it
might have problems working with the kmod-xfs kernel module I can use
the 2.9.4 tools instead. (Again, in keeping with the hope-for-the-best
scenario above, if avoiding a reboot won't be too harmful it'd be
I think you also mentioned that xfs_metadump can not dump frozen
filesystems, but the man page for 3.1.5 says it can. FWIW xfs_metadump
refused to work on a frozen filesystem on my test machine, which is
version 2.9.4 (though from an older CentOS base). (xfs_freeze does look
like a nice tool though!)