ARC-1120 and MD very sloooow

[Jimmy Thrasibule]

> Right.  It's unusual to see this many mount options.  FYI, the XFS
> default is relatime, which is nearly identical to noatime.  Specifying
> noatime won't gain you anything.  Do you really need nosuid, nodev, noexec?

Well better say what I don't want on the filesystem no?

 >Do you also see the low write speed and slow ls on md0, any/all of your
> md/RAID10 arrays?

Yes, all drive operations are slow, unfortunately, I have no drives in
the machine
that are not managed by the controller to push tests further.

> The usual: "iostat -x -d -m 5" output while the test is running.
> Also, you are using buffered IO, so changing it to use direct IO
> will tell us exactly what the disks are doing when Io is issued.
> blktrace is your friend here....

I've ran the following:


    # dd if=/dev/zero of=/srv/store/video/test.zero bs=512K count=6000
oflag=direct
    6000+0 records in
    6000+0 records out
    3145728000 bytes (3.1 GB) copied, 179.945 s, 17.5 MB/s

    # dd if=/srv/store/video/test.zero of=/dev/null iflag=direct
    6144000+0 records in
    6144000+0 records out
    3145728000 bytes (3.1 GB) copied, 984.317 s, 3.2 MB/s


Traces are huge for the read test so I put them on Google Drive + SHA1 sums:
https://drive.google.com/folderview?id=0BxJZG8aWsaMaVWkyQk1ELU5yX2c

Drives `sdc` to `sdf` are part of the RAID10 array. Only drives `sdc` and `sde`
are used when reading.

> That makes me wonder if the controller and drive write caches have been disabled.
> That could explain this.

Caching is enabled for the controller but not much information.


    > sys info
    The System Information
    ===========================================
    Main Processor     : 500MHz
    CPU ICache Size    : 32KB
    CPU DCache Size    : 32KB
    CPU SCache Size    : 0KB
    System Memory      : 128MB/333MHz/ECC
    Firmware Version   : V1.49 2010-12-02
    BOOT ROM Version   : V1.49 2010-12-02
    Serial Number      : Y611CAABAR200126
    Controller Name    : ARC-1120
    ===========================================


By the way is enabling the controller cache a good idea? I would disable
it and let the kernel manage.

--
Jimmy