>> I am using xfs on a raid 5 (~100TB) and put log on external
>> ssd device, the mount information is: /dev/sdc on
>> /data/fhgfs/fhgfs_storage type xfs
>> when doing only reading / only writing , the speed is very
>> fast(~1.5G), but when do both the speed is very slow (100M),
>> and high r_await(160) and w_await(200000).
> What are your kernel version, mount options and xfs_info output ?
Those are usually important details, but in this case the
information that matters is already present.
There is a ratio of 31 (thirty one) between 'swidth' and 'sunit'
and assuming that this reflects the geometry of the RAID5 set
and given commonly available disk sizes it can be guessed that
with amazing "bravery" someone has configured a RAID5 out of 32
(thirty two) high capacity/low IOPS 3TB drives, or something
It is even "braver" than that: if the device name
"/data/fhgfs/fhgfs_storage" is dedscriptive, this "brave"
RAID5 set is supposed to hold the object storage layer of a
BeeFS highly parallel filesystem, and therefore will likely
have mostly-random accesses.
This issue should be moved to the 'linux-raid' mailing list as
from the reported information it has nothing to do with XFS.
BTW the 100MB/s aggregate over 31 drives means around 3MB/s per
drive, which seems pretty good for a RW workload with
mostly-random accesses with high RMW correlation.
It is notable but not surprising that XFS works well even with
such a "brave" choice of block storage layer, untainted by any
"cowardly" consideration of the effects of RMW and using drives
designed for capacity rather than IOPS.