<html><head></head><body bgcolor="#FFFFFF"><div>On Jun 29, 2012, at 12:46 AM, Changliang Chen <<a href="mailto:hqucocl@gmail.com">hqucocl@gmail.com</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div>Hi Eric,<div><br></div><div> Is this issue <font face="Verdana, Arial, Helvetica, sans-serif">resolved? We have </font>
<span style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)">been getting the same problem,</span> though we had upgrated the kernel to 2.6.18-308.8.2.el5.<font face="Verdana, Arial, Helvetica, sans-serif"><br>
</font><br></div></div></blockquote>I do not know; if it were rhel I'd suggest logging a support ticket. I've not seen anything similar on rhel.<div><br></div><div>Did you make sure there is no xfs kmod rpm installed? What does modinfo xfs say?</div><div><br><blockquote type="cite"><div><div><div class="gmail_quote">On Tue, Apr 3, 2012 at 2:03 AM, Eric Sandeen <span dir="ltr"><<a href="mailto:sandeen@sandeen.net" target="_blank">sandeen@sandeen.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 4/2/12 8:09 AM, Mark Rechler wrote:<br>
> Hi Eric,<br>
><br>
> Thank you for the reply. We are running CentOS 5.8, with the<br>
> 2.6.18-164.10.1.el5.centos.plus kernel as it was mentioned in a bug<br>
> report that has similar behavior, but ultimately a different kernel<br>
> panic (<a href="http://bugs.centos.org/view.php?id=4089" target="_blank">http://bugs.centos.org/view.php?id=4089</a>). We have tried<br>
> running xfs_repair in the past and it has not proved useful. The odd<br>
> part is that these are fresh systems (just installed). If it helps,<br>
> we are also running glusterfs on these boxes though load does not<br>
> always correlate to a kernel panic.<br>
<br>
</div>I can't say for sure what's in that respun "extra" centos kernel,<br>
but I can say this: the error you hit indicates that xfs read a<br>
buffer, and wound up with a metadata buffer which had unrecognized<br>
magic - i.e. it did not look like metadata as expected. Seeing what<br>
looks like corruption, it shut down.<br>
<br>
This reminds me a little of<br>
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=512552" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=512552</a><br>
which I fixed for RHEL customers a while back, where cancelled<br>
readahead in MD was resulting in xfs thinking a buffer was<br>
uptodate, but in fact it was uninitialized, hence it found<br>
garbage and shut down in this way.<br>
<br>
Something similar seems to be happening in your case, if xfs_repair<br>
comes up clean; somehow xfs is getting hold of a buffer which<br>
apparently doesn't match what xfs_repair found to be a consistent<br>
filesystem.<br>
<br>
So I might suspect something in the storage stack?<br>
<br>
Also please be sure you don't have kmod-xfs or xfs-kmod installed<br>
on your centos box, which is a truly ancient and completely unsupported<br>
backport of xfs from long, long ago.<br>
<br>
-Eric<br>
<br>
> Thanks,<br>
> Mark<br>
<div><div class="h5">><br>
> On Fri, Mar 30, 2012 at 6:44 PM, Eric Sandeen <<a href="mailto:sandeen@sandeen.net">sandeen@sandeen.net</a> <mailto:<a href="mailto:sandeen@sandeen.net">sandeen@sandeen.net</a>>> wrote:<br>
><br>
> On 3/30/12 5:02 PM, Mark Rechler wrote:<br>
> > Hi Everyone,<br>
> ><br>
> > We've been getting a lot of errors (across several kernels) and eventually a kernel panic. Any insight into these errors would be much appreciated.<br>
> ><br>
> > Errors:<br>
> > Filesystem "dm-3": XFS internal error xfs_da_do_buf(2) at line 2112 of file fs/xfs/xfs_da_btree.c. Caller 0xffffffff883c1826<br>
><br>
> Saying which CentOS it is would help ;) And, standard disclaimers about how CentOS doesn't come with upstream _or_ distro support, etc etc...<br>
><br>
> But xfs_da_do_buf(2) indicates on-disk corruption, having encountered a bad magic number when reading from the disk. Have you tried xfs_repair?<br>
><br>
> -Eric<br>
><br>
> > Call Trace:<br>
> > [<ffffffff883c1725>] :xfs:xfs_da_do_buf+0x503/0x5b1<br>
> > [<ffffffff883c1826>] :xfs:xfs_da_read_buf+0x16/0x1b<br>
> > [<ffffffff883c1826>] :xfs:xfs_da_read_buf+0x16/0x1b<br>
> > [<ffffffff883aeb71>] :xfs:xfs_attr_leaf_get+0x2e/0x99<br>
> > [<ffffffff883aeb71>] :xfs:xfs_attr_leaf_get+0x2e/0x99<br>
> > [<ffffffff883aec7f>] :xfs:xfs_attr_fetch+0xa3/0xd5<br>
> > [<ffffffff883a7aa8>] :xfs:xfs_acl_iaccess+0x64/0xd4<br>
> > [<ffffffff883f264a>] :xfs:xfs_check_acl+0x1b/0x2b<br>
> > [<ffffffff8000f550>] generic_permission+0x40/0xca<br>
> > [<ffffffff8000d902>] permission+0x81/0xc8<br>
> > [<ffffffff8000999d>] __link_path_walk+0x173/0xf42<br>
> > [<ffffffff8000e9cc>] link_path_walk+0x42/0xb2<br>
> > [<ffffffff8000cc9c>] do_path_lookup+0x275/0x2f1<br>
> > [<ffffffff8001278e>] getname+0x15b/0x1c2<br>
> > [<ffffffff800236f6>] __user_walk_fd+0x37/0x4c<br>
> > [<ffffffff8003f1f6>] vfs_lstat_fd+0x18/0x47<br>
> > [<ffffffff8008c46e>] default_wake_function+0x0/0xe<br>
> > [<ffffffff800efddf>] sys_lgetxattr+0x4e/0x5f<br>
> > [<ffffffff8002a996>] sys_newlstat+0x19/0x31<br>
> > [<ffffffff8005d229>] tracesys+0x71/0xe0<br>
> > [<ffffffff8005d28d>] tracesys+0xd5/0xe0<br>
> ><br>
> > Code: 0f b6 40 02 89 44 24 04 e9 95 00 00 00 44 0f b6 Z3 44 3b 65<br>
> > RIP [<ffffffffff8841bfaf>] :xfs:xfs_attr_shortform_getvalue+0x24/0xe2<br>
> > RSP <ffff81020752dbc8><br>
> > CR2: 00000000000002<br>
> > <0>Kernel panic - not syncing: Fatal exception<br>
> ><br>
> > Thanks,<br>
> > Mark<br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > xfs mailing list<br>
</div></div>> > <a href="mailto:xfs@oss.sgi.com">xfs@oss.sgi.com</a> <mailto:<a href="mailto:xfs@oss.sgi.com">xfs@oss.sgi.com</a>><br>
> > <a href="http://oss.sgi.com/mailman/listinfo/xfs" target="_blank">http://oss.sgi.com/mailman/listinfo/xfs</a><br>
<div class="HOEnZb"><div class="h5">><br>
><br>
<br>
_______________________________________________<br>
xfs mailing list<br>
<a href="mailto:xfs@oss.sgi.com">xfs@oss.sgi.com</a><br>
<a href="http://oss.sgi.com/mailman/listinfo/xfs" target="_blank">http://oss.sgi.com/mailman/listinfo/xfs</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><br>Regards,<br><br>Cocl<br>ops manager<br>19lou Operation & Maintenance Dept<br>
</div>
</div></blockquote></div></body></html>