XFS corruption with failover

[Lachlan McIlroy]

----- "Eric Sandeen" <sandeen at sandeen.net> wrote:

> Felix Blyakher wrote:
> > On Aug 13, 2009, at 3:17 PM, John Quigley wrote:
> > 
> >> Folks:
> >>
> >> We're deploying XFS in a configuration where the file system is  
> >> being exported with NFS.  XFS is being mounted on Linux, with  
> >> default options; an iSCSI volume is the formatted media.  We're  
> >> working out a failover solution for this deployment utilizing Linux
>  
> >> HA.  Things appear to work correctly in the general case, but in  
> >> continuous testing we're getting XFS superblock corruption on a
> very  
> >> reproducible basis.
> >> The sequence of events in our test scenario:
> >>
> >> 1. NFS server #1 online
> >> 2. Run IO to NFS server #1 from NFS client
> >> 3. NFS server #1 offline, (via passing 'b' to /proc/sysrq-trigger)
> >> 4. NFS server #2 online
> >> 5. XFS mounted as part of failover mechanism, mount fails
> >>
> >> The mount fails with the following:
> >>
> >> <snip>
> >> kernel: XFS mounting filesystem sde
> >> kernel: Starting XFS recovery on filesystem: sde (logdev:
> internal)
> >> kernel: XFS: xlog_recover_process_data: bad clientid
> >> kernel: XFS: log mount/recovery failed: error 5
> > 
> > This is an IO error. Is the block device (/dev/sde) accessible
> > from the server #2 OK? Can you dd from that device?
> 
> Are you sure?
> 
>                 if (ohead->oh_clientid != XFS_TRANSACTION &&
>                     ohead->oh_clientid != XFS_LOG) {
>                         xlog_warn(
>                 "XFS: xlog_recover_process_data: bad clientid");
>                         ASSERT(0);
>                         return (XFS_ERROR(EIO));
>                 }
> 
> so it does say EIO but that seems to me to be the wrong error; loks
> more
> like a bad log to me.
> 
> It does make me wonder if there's any sort of per-initiator caching
> on
> the iscsi target or something.  </handwave>
Should barriers be enabled in XFS then?

> 
> -Eric
> 
> _______________________________________________
> xfs mailing list
> xfs at oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs