<div dir="ltr"><span style="color:rgb(80,0,80)">> Running xfs_repair /dev/md1 the first time resulted in suggestion to</span><br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">
> mount/unmount to replay log, but mounting would not work. After running<br>
> xfs_repair -v -L -P /dev/md1 this happens:<br>
> (lots of output on stderr, moving to Phase 3, then more output - not sure<br>
> if it is relevant, the log file is ~170Mb in size), then stops and prints<br>
> the only line on stdout:<br>
<br>
</div>Oh dear. A log file that big indicates that something *bad* has<br>
happened to the array. i.e that it has most likely been put back<br>
together wrong.<br>
<br>
Before going any further with xfs_repair, please verify that the<br>
array has been put back together correctly....<br>
<div class="im"><br></div></blockquote><div><br></div><div style>The raid array did not suffer, at least, not according to mdadm; it is now happily recovering the one disk that officially failed, but the whole thing assembled without a problem</div>
<div style>There was a similar crash several weeks ago on this same array, but had ext4 system back then.</div><div style>I was able to save some of the latest stuff, and decided to move to xfs as something more reliable.</div>
<div style>I suspect now I should also had replaced the disk controller then.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
> xfs_repair: dinode.c:768: process_bmbt_reclist_int: Assertion `i <<br>
> *numrecs' failed.<br>
> Aborted<br>
><br>
> After inserting a printf before the assert, I get the following:<br>
><br>
> i = 0, *numrecs = -570425343 for printf( "%d, %d")<br>
> or<br>
> i= 0, *numrecs = 3724541953 for printf("%ld, %ld) - makes me wonder if<br>
> it's signed/unsigned int related<br>
<br>
</div>numrecs is way out of the normal range, so that's probably what is<br>
triggering it.<br>
<br>
i.e this in process_exinode():<br>
<br>
numrecs = XFS_DFORK_NEXTENTS(dip, whichfork);<br>
<br>
is where the bad number is coming from, and that implies a corrupted<br>
inode. it's a __be32 on disk, the kernel considers it a xfs_extnum_t<br>
in memory which is a int32_t because:<br>
<br>
#define NULLEXTNUM ((xfs_extnum_t)-1)<br>
<br>
So, negative numbers on disk are invalid.<br>
....<br>
<br>
The patch below should fix the assert failure.<br>
<div class="im"><br></div></blockquote><div><br></div><div style>I'll try it - don't really have other options at the moment</div><div style> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">
> If I try now (after running xfs_repair -L) to mount the fs read-only, it<br>
> mounts but says some directories have structures that need cleaning, so the<br>
> dirs are inaccessible.<br>
><br>
> Any suggestion on how to possibly fix this?<br>
<br>
</div>I suspect you've damaged it beyond repair now.<br>
<br>
If the array was put back together incorrectly in the first place<br>
(which is likely given the damage being reported), then<br>
you've made the problem a whole lot worse by writing to it in an<br>
attempt to repair it.<br>
<br>
I'd suggest that you make sure the array is correctly<br>
repaired/ordered/reocvered before doing anything else, then<br>
running xfs_repair on what is left and hoping for the best. Even after<br>
repair is finished, you'll need to go through all the data with a<br>
fine toothed comb to work out what has been lost, corrupted or<br>
overwritten with zeros or other stuff.<br>
<br>
I suspect you'll be reaching for the backup tapes long before you<br>
get that far, though...<br></blockquote><div><br></div><div><br></div><div style>Well, we'll see how it goes.</div><div style><br></div><div style>Thanks for the patch and the quick reply!</div><div style><br></div><div style>
Sincerely,</div><div style>Victor</div><div style> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Cheers,<br>
<br>
Dave.<br>
--<br>
Dave Chinner<br>
<a href="mailto:david@fromorbit.com">david@fromorbit.com</a><br>
<br>
<br>
xfs_repair: validate on-disk extent count better<br>
<br>
From: Dave Chinner <<a href="mailto:dchinner@redhat.com">dchinner@redhat.com</a>><br>
<br>
When scanning a btree format inode, we trust the extent count to be<br>
in range. However, values of the range 2^31 <= cnt < 2^32 are<br>
invalid and can cause problems with signed range checks. This<br>
results in assert failures which validating the extent count such<br>
as:<br>
<div class="im"><br>
xfs_repair: dinode.c:768: process_bmbt_reclist_int: Assertion `i < *numrecs' failed.<br>
<br>
</div>Validate the extent count is at least within the positive range of a<br>
singed 32 bit integer before using it.<br>
<br>
Signed-off-by: Dave Chinner <<a href="mailto:dchinner@redhat.com">dchinner@redhat.com</a>><br>
---<br>
repair/dinode.c | 25 +++++++++++++++++++++++--<br>
1 file changed, 23 insertions(+), 2 deletions(-)<br>
<br>
diff --git a/repair/dinode.c b/repair/dinode.c<br>
index 5a2da39..239bb7b 100644<br>
--- a/repair/dinode.c<br>
+++ b/repair/dinode.c<br>
@@ -1293,7 +1293,7 @@ process_exinode(<br>
xfs_bmbt_rec_t *rp;<br>
xfs_dfiloff_t first_key;<br>
xfs_dfiloff_t last_key;<br>
- int numrecs;<br>
+ int32_t numrecs;<br>
int ret;<br>
<br>
lino = XFS_AGINO_TO_INO(mp, agno, ino);<br>
@@ -1302,6 +1302,15 @@ process_exinode(<br>
numrecs = XFS_DFORK_NEXTENTS(dip, whichfork);<br>
<br>
/*<br>
+ * We've already decided on the maximum number of extents on the inode,<br>
+ * and numrecs may be corrupt. Hence make sure we only allow numrecs to<br>
+ * be in the range of valid on-disk numbers, which is:<br>
+ * 0 < numrecs < 2^31 - 1<br>
+ */<br>
+ if (numrecs < 0)<br>
+ numrecs = *nex;<br>
+<br>
+ /*<br>
* XXX - if we were going to fix up the btree record,<br>
* we'd do it right here. For now, if there's a problem,<br>
* we'll bail out and presumably clear the inode.<br>
@@ -2038,11 +2047,23 @@ process_inode_data_fork(<br>
{<br>
xfs_ino_t lino = XFS_AGINO_TO_INO(mp, agno, ino);<br>
int err = 0;<br>
+ int nex;<br>
+<br>
+ /*<br>
+ * extent count on disk is only valid for positive values. The kernel<br>
+ * uses negative values in memory. hence if we see negative numbers<br>
+ * here, trash it!<br>
+ */<br>
+ nex = be32_to_cpu(dino->di_nextents);<br>
+ if (nex < 0)<br>
+ *nextents = 1;<br>
+ else<br>
+ *nextents = nex;<br>
<br>
- *nextents = be32_to_cpu(dino->di_nextents);<br>
if (*nextents > be64_to_cpu(dino->di_nblocks))<br>
*nextents = 1;<br>
<br>
+<br>
if (dino->di_format != XFS_DINODE_FMT_LOCAL && type != XR_INO_RTDATA)<br>
*dblkmap = blkmap_alloc(*nextents, XFS_DATA_FORK);<br>
*nextents = 0;<br>
</blockquote></div><br></div></div>