-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
Just wanted to make sure since I didn't catch any mention of these
checks. And based on your thoroughness I assume you ran memtest after
the ram replacement. What I'd try next in your situation is to boot a
different version of the kernel (possibly a different distro) and see
if the errors are the same, I'd try something bootable from a DVD or a
USB stick. What do you think?
On 07/20/2015 11:35 AM, Leslie Rhorer wrote:
> On 7/20/2015 3:05 AM, Martin Papik wrote:
>
> Since you've already found one HW related fault, would you consider
> booting into memtest for a couple of passes just to be on the safe
> side.
>
>> I did that after confirming the one stick of memory was bad.
>> Twice. I got over 20,000 errors on the bad stick, and 0 on the
>> good one. I also swapped the locations on the motherboard, and
>> the bad stick still failed while the good one passed 100%.
>
> And did you by any chance look at SMART if applicable and possibly
> running a test on the drives.
>
>> Yes. SMART found no errors, but think about it. Every time tar
>> tries to create a directory when untarring that file in that
>> location, the file system croaks when it tries to create a
>> directory. Not when reading and not when writing other than when
>> it creates a directory. When I create the directory manualy, the
>> process quits failing at that point and fails later on during a
>> different directory create. The array remains intact when
>> reading, and dmesg shows no drive errors. I've re-synced the
>> array, which reads every byte on all 8 drives without a single
>> mismatch - several times. To my knowledge, no read has ever
>> failed except after the filesystem goes offline. I thought
>> reads were failing during the CRC checks, but that was a red
>> herring.
>
> Another test I sometimes do when I'm unsure about disks is "cat
> /dev/sda > /dev/null" (i.e. a whole disk read test)
>
>> echo repair > /sys/block/md0/md/sync_action reads not one drive,
>> but every byte on all 8 drives.
>
> and see (dmesg) if any errors show up, unless
>
>> 'Nary one, and no mismatches.
>
> you're willing to run badblocks in a read-write nondestructive
> mode. In my experience the read test or badblocks can be run
> simultaneously with smartctl -t long. But as a start I'd look at
> smartctl --all /dev/sd? and see if there are any bad signs. I hope
> this helps. Good luck
>
>
> On 07/20/2015 10:41 AM, Leslie Rhorer wrote:
>>>> On 7/19/2015 6:27 PM, Dave Chinner wrote:
>>>>> On Sat, Jul 18, 2015 at 08:02:50PM -0500, Leslie Rhorer
>>>>> wrote:
>>>>>>
>>>>>> I found the problem with md5sum (and probably nfs, as
>>>>>> well). One of the memory modules in the server was bad.
>>>>>> The problem with XFS persists. Every time tar tried to
>>>>>> create the directory:
>>>>>
>>>>> Now you need to run xfs_repair.
>>>>
>>>> I do that every time the array implodes. It makes no
>>>> difference. It never mentions cleaning the structure tar
>>>> says needs cleaning, and the next time I run tar on that
>>>> file, the filesystem craters.
>>>>
>>>> _______________________________________________ xfs mailing
>>>> list xfs@xxxxxxxxxxx http://oss.sgi.com/mailman/listinfo/xfs
>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCgAGBQJVrLbVAAoJELsEaSRwbVYr/JoQAKGcNBTtswnSJ9SYpBQMc8aO
m2WQaHzLDPkSPLWYeWSGc3clPuf4FdP3A9bDcclCnVV/Ex0WJiCalYfa1Zqpnq5P
BinRp1w/cbfTTazLspFT9ySuoloOqNXTPz0MB4uxRTnIDb3Hcahw0O6HhOuZixW3
ocaEOXqVs1cc4YzPwT4Z9aWBEX3ZutMvxNKM4VWT1m8aoRZ3eJMPUKHN04PDUKyT
4Mwilypg9R6r6iberZ9zVwFy0LerElg9Cb90AGLNpyGCutGbOZH7VsoBUTnAmh2E
dz4uruFU0x8n87MQccXfSvZQIWG16UDxwjQjEiD4EHtRhYYTNVgq2V8ak94u8w99
0p5WG5+dEnVV0Qgjk2DaZy305LP+5oc2D9GkXJgGTFjMPVV3+9Tnq/XDlm2Hgxn8
hq2q0DoPDQVFMzNLxpGCJfuIdAO3o7z/1rjHpeP2Ol6pPw+hT8SQMehTBU4vMlcp
SeZzg485rVtQrWtXVJaRhITAQWSvQxjm9QqLAMdon0oxdKAPZIOtQgr8oEGKgfr7
mknqFPon7sa0c4nAZT7DtTOS+OATbTnYAoUqIuxRf4NCD7dbFUQrccU4/peEE4/H
SPzOfgOiAArOVZwWEc7JvydpcKqaEUzYb2KyzsGJFuJHZodrSTzXmUMg/Muc+iQ5
Ao/NeFe/1flevZ060ZEX
=1/q4
-----END PGP SIGNATURE-----
|