<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Corbel;
panose-1:2 11 5 3 2 2 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Dear Sirs <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I wonder if you can help with an issue we see re-occuring on a regular basis with one of our HP systems which uses a HP 420 Raid controller
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Action taken <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We first saw the following: <br>
[root@ content]# ls <br>
ls: cannot open directory .: Input/output error <br>
<br>
<br>
[root@ /]# ls -ltr <br>
ls: cannot access content <br>
total 358 <br>
d?????????? ? ? ? ? ? content <br>
drwxr-xr-x. 2 root root 4096 Jun 28 2011 srv <br>
drwxr-xr-x. 2 root root 4096 Jun 28 2011 media <br>
drwxr-xr-x. 2 root root 4096 Feb 22 2012 cgroup <br>
drwx------. 2 root root 16384 Jul 21 2012 lost+found <br>
drwxr-xr-x. 2 root root 4096 Jul 21 2012 selinux <br>
<br>
We try to run: <br>
[root@ /]# xfs_check /dev/md0 <br>
xfs_check: /dev/md0 contains a mounted and writable filesystem <br>
fatal error -- couldn't initialize XFS library <br>
We also tried to umount the /dev/md0 before runniing xfs_check but no luck. We received the error: device is in use<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We use xfs for one of our large raid file systems and we are seeing the xfs filesystem go offline with the following messages in dmesg
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">messages-20140921:Sep 18 23:01: kernel: XFS (md0): Device md0: metadata write error block 0x5e28623d8
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): I/O error occurred: meta-data dev md0 block 0x445cccc40 ("xlog_iodone") error 5 buf count 32768
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_do_force_shutdown(0x2) called from line 891 of file fs/xfs/xfs_log.c. Return address = 0xffffffffa2c428dc
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): Log I/O Error Detected. Shutting down filesystem
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): Please umount the filesystem and rectify the problem(s)
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_imap_to_bp: xfs_trans_read_buf() returned error 5.
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_iunlink_remove: xfs_itobp() returned error 5.
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): I/O error occurred: meta-data dev md0 block 0x445cccc80 ("xlog_iodone") error 5 buf count 32768
<br>
messages-20140921:Sep 18 23:01:04 kernel: XFS (md0): xfs_do_force_shutdown(0x2) called from line 891 of file fs/xfs/xfs_log.c. Return address = 0xffffffffa2c428dc<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">XFS (md0): xfs_log_force: error 5 returned. <br>
XFS (md0): xfs_log_force: error 5 returned. <br>
XFS (md0): xfs_log_force: error 5 returned.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In all occurrences the only way to recover from this is to reboot the system and allow xfs_repair to run during boot this clears the issue until next time
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We have checked the RAID health and nothing seems to be amiss, if you could help with this it would be much appreciated
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards Simon <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b><span lang="EN-CA" style="font-family:"Arial","sans-serif""><o:p> </o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif"">Simon Dray
</span><span lang="EN-CA" style="font-size:10.0pt;font-family:Wingdings">s</span><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif"">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif"">p: +44.1223 716.400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif"">p: +44.1223 716.476
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-CA" style="font-size:10.0pt;font-family:"Arial","sans-serif"">e: sdray@espial.com
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:8.0pt;font-family:"Corbel","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"">1st Floor, 335 Cambridge Science Park, Milton Road, Cambridge, Cambridgeshire, CB4 0WN, United Kingdom.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif""><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:8.0pt;font-family:"Corbel","sans-serif"">Understanding is a three-edged sword</span><span style="font-size:8.0pt;font-family:"Corbel","sans-serif";color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>