fragmentation question

[Brady Chang <bchang@xxxxxxxxxxxxx>]

Hello All,
I have an issue with fragmentation on a particular device
thanks for any advice.

-Brady

I have a Dell r510 with 12 disks 
2xraid 5 (6 disks each) 
raid group1:
48 GB   carved out for os mounted as / 
remaining space  2.7 TB for xfs mounted as /data1
raid group2:
48 GB  for swap
remaining space 2.7 TB for xfs mounted as /data2

The strange thing is that /data1 never gets fragmented where as /data2 is badly fragmented.
I believe increase allocsize would help, but not sure how to explain why /data2(/dev/sdd) always gets fragmented and not /data1(/dev/sdb)

It's a data warehouse application.  the I/O is balanced between /data1 and /data2:
output of xfs_db
[root@sdw4 data1]# xfs_db -c frag -r /dev/sdb
actual 14353, ideal 13702, fragmentation factor 4.54%
[root@sdw4 data1]# xfs_db -c frag -r /dev/sdd
actual 408674, ideal 13719, fragmentation factor 96.64%
df output
/dev/sdb              2.7T  967G  1.8T  36% /data1
/dev/sdd              2.7T  1.1T  1.7T  39% /data2

LABEL=/data1        /data1     xfs     allocsize=1048576,logbufs=8,noatime,nodiratime 0 0
LABEL=/data2        /data2     xfs     allocsize=1048576,logbufs=8,noatime,nodiratime 0 0


raid config output:
[root@sdw4 data1]# omreport storage vdisk 
List of Virtual Disks in the System

Controller PERC H700 Integrated (Slot 4)
ID                        : 0
Status                    : Ok
Name                      : boot
State                     : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured                   : Not Applicable
Progress                  : Not Applicable
Layout                    : RAID-5
Size                      : 48.99 GB (52602470400 bytes)
Device Name               : /dev/sda
Bus Protocol              : SAS
Media                     : HDD
Read Policy               : No Read Ahead
Write Policy              : Force Write Back
Cache Policy              : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disabled

ID                        : 1
Status                    : Ok
Name                      : data1
State                     : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured                   : Not Applicable
Progress                  : Not Applicable
Layout                    : RAID-5
Size                      : 2,742.89 GB (2945150484480 bytes)
Device Name               : /dev/sdb
Bus Protocol              : SAS
Media                     : HDD
Read Policy               : No Read Ahead
Write Policy              : Force Write Back
Cache Policy              : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disabled

ID                        : 2
Status                    : Ok
Name                      : swap
State                     : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured                   : Not Applicable
Progress                  : Not Applicable
Layout                    : RAID-5
Size                      : 48.99 GB (52602470400 bytes)
Device Name               : /dev/sdc
Bus Protocol              : SAS
Media                     : HDD
Read Policy               : No Read Ahead
Write Policy              : Force Write Back
Cache Policy              : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disabled

ID                        : 3
Status                    : Ok
Name                      : data2
State                     : Ready
Hot Spare Policy violated : Not Assigned
Virtual Disk Bad Blocks   : No
Secured                   : Not Applicable
Progress                  : Not Applicable
Layout                    : RAID-5
Size                      : 2,742.89 GB (2945150484480 bytes)
Device Name               : /dev/sdd
Bus Protocol              : SAS
Media                     : HDD
Read Policy               : No Read Ahead
Write Policy              : Force Write Back
Cache Policy              : Not Applicable
Stripe Element Size       : 128 KB
Disk Cache Policy         : Disabled