need troubleshooting help to track down a xfsdatad or hardware problem

[david at lang.hm]

I have 20 identical boxes that have a 16 drive raid 6 array (3ware 
controller), on 17 of the boxes everything seems fine, but on 3 of the 
boxes the xfsdatad process appears to be having problems writing to the 
drive (iostat shows 5MB throughput), and a huge amount of data ends up 
buffered in memory, but cannot get written out to the disk.

what can I do to figure out if the problem is somewhere in software or in 
hardware (and if hardware, any ideas how to figure out what is happening 
on these 3 boxes)

David Lang