XFS Test Results Interpretation
Andrew Debenham
adebenham at sotech1.com
Sat Oct 23 21:59:58 CDT 2010
Hello -
I am in the process of trying to determine the best parameters to use when creating and mounting an XFS file system on some new hardware we are going to be using. My company has two separate applications that will be running on the same hardware (but on separate systems). One application is the PostgreSQL database and the other is a custom application that does writes to many (~1,000) relatively small (~672MB) files concurrently.
I decided to use Bonnie++ to test a 17TB XFS file system, varying the mkfs.xfs and the mount parameters, in order to determine the optimal settings for our new systems. I originally tried attaching my spreadsheet to this email, but I'm guessing your server doesn't accept attachments. So, I've inserted my results as a table below; hopefully it will come through in a somewhat readable format. If not, please send me your email and I'll forward you the original spreadsheet. For each set of parameters, I ran Bonnie++ 3 separate times and took the average of the results.
Also, a couple of things to note. First, we are already locked into this version of CentOS and the current kernel, so even though they are a bit dated, we will not be able to upgrade to the latest kernel. Also, here are the release and version numbers of the XFS packages currently installed on these systems:
kmod-xfs-0.4-2
xfsprogs-devel-2.9.4-1.el5.centos
When I started specifying the sunit and swidth parameters, I expected to see the sequential block input and output to improve, but my final results didn't really show this (maybe my expectations were just wrong?). I was hoping that someone with a lot more XFS expertise could take a look at my results and interpret them for me? Do my test results look accurate or is it possible that I setup my test incorrectly? Based on the two applications that will be using an XFS file system, is there an optimal set of parameters that I can use for one or both applications? Any suggestions or recommendations would be greatly appreciated. Thanks in advance for your help in this matter.
- Andrew
Data
Sequential Output
Sequential Input
Random Seeks
Sequential Create
Random Create
Number of Files
mkfs.xfs Options
Mount Options
Chunk Size
Per Char
Block
Rewrite
Per Char
Block
Create
Read
Delete
Create
Read
Delete
K/sec
%CP
K/sec
%CP
K/sec
%CP
K/sec
%CP
K/sec
%CP
/sec
%CP
/sec
%CP
/sec
%CP
/sec
%CP
/sec
%CP
/sec
%CP
/sec
%CP
64352M
94013
99
472221
50
85811
11
80600
92
221248
14
377.6
0
2814
70
8036
92.7
12635
93.3
2924
71.7
7927
92.3
9819
80.7
10:688128:5120:1
-l size=16384b
rw
64352M
95284
99
450426
47.7
78142
10
74617
86.7
187983
12.3
341.7
0
2488
63
8672
99
11862
93
2669
67.3
7251
83
11165
87
10:688128:5120:1
-l size=16384b
rw,noatime,nodiratime,logbufs=8
64352M
90512
99
494516
51.3
84420
10.7
78464
89.7
207651
12
379.5
0.33
2694
64
9000
99
12772
97.3
3059
71
8480
93.3
10178
81.3
10:688128:5120:1
-l size=16384b
rw,noatime,nodiratime,logbufs=8,nobarrier
64352M
88295
99
501854
53
89128
11
80788
92.3
230101
16.7
368.4
0
3196
80.3
8552
98.3
12818
94.7
2693
64.3
7189
83.7
11396
90
10:688128:5120:1
-l size=128m
rw
64352M
91501
99
487799
51.3
91343
11.3
83336
95
236927
17
379.7
0
2852
72
8394
96.7
12879
95.7
2865
68
7957
91.3
11540
88
10:688128:5120:1
-l size=128m
rw,noatime,nodiratime,logbufs=8
64352M
96319
99
453202
47
82270
10.3
78626
89.3
204320
12.3
377.1
0.33
2948
70.7
8905
97.7
13072
97.7
3004
69.7
8324
92.3
12029
97.3
10:688128:5120:1
-l size=128m
rw,noatime,nodiratime,logbufs=8,nobarrier
64352M
92101
99
477882
50
90622
11
82742
94.7
239307
14.7
377.0
0
2800
69.3
8954
99
11597
85.3
2699
64.3
8836
100
10299
83.7
10:688128:5120:1
-l version=2,size=16384b
rw
64352M
98567
99
448664
46.7
81706
10
78214
90
217844
16
362.0
0
2529
64
8899
98
12754
95.7
2988
69
8419
93.3
9267
72.7
10:688128:5120:1
-l version=2,size=16384b
rw,noatime,nodiratime,logbufs=8
64352M
92854
99
482572
50.7
79940
9.67
76456
88.3
193196
14.3
344.5
0
2782
68
8925
99.3
12953
98.3
2904
67.7
8272
93.3
10709
86.7
10:688128:5120:1
-l version=2,size=16384b
rw,noatime,nodiratime,logbufs=8,nobarrier
64352M
96836
99
482944
51
92771
12
82157
94
240734
14
383.0
0.67
3176
79.3
8419
97
12932
96
2925
70.3
8572
99
11595
88.3
10:688128:5120:1
-l version=2,size=128m
rw
64352M
96595
99
493446
51.3
82685
10.3
78582
89.7
214803
12.7
366.2
0.33
3644
86.7
8805
99
13503
98.3
3316
77.3
7564
87
9005
69.3
10:688128:5120:1
-l version=2,size=128m
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
94856
99
516368
54
79747
10
75690
87.3
198909
13.7
352.6
0
3680
86.3
8800
99
13825
98.7
3652
85.3
7952
90.3
10891
85
10:688128:5120:1
-l version=2,size=128m
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
90933
99
463918
48.7
81266
10.3
76380
88.3
229959
14
372.3
0.33
2704
67.7
8670
99.3
12473
96.3
2878
70
8572
97.7
9065
70
10:688128:5120:1
-l version=2,size=16384b -i attr=2
rw
64352M
89084
99
459402
49
80776
10.3
75495
86.7
230625
12.7
371.6
0.33
3173
74.7
7410
83.3
13247
96.7
4017
93.3
7778
87.3
10813
82.3
10:688128:5120:1
-l version=2,size=16384b -i attr=2
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
92994
99
486561
51.7
78986
9.67
76097
87.7
230976
12
379.1
0.33
3652
85.3
7255
81.7
11593
83.3
3941
90.7
8964
98.7
9354
71.7
10:688128:5120:1
-l version=2,size=16384b -i attr=2
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
89910
99
508021
53.3
81036
10
78428
90
214695
16.3
362.2
0
2891
72.3
8658
99.3
12888
95
2833
69
7999
93
11380
84.7
10:688128:5120:1
-l version=2,size=128m -i attr=2
rw
64352M
96944
99
487031
51
91412
11.3
82028
93.3
235545
14.3
374.9
0
3589
84.7
8579
99
13744
99
3622
84.3
7983
91
10703
82
10:688128:5120:1
-l version=2,size=128m -i attr=2
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
92401
99
490172
51.7
78263
9.67
77018
88.7
235954
14.3
382.6
0.33
3612
84.3
8754
99
12172
89
3414
82.7
6170
71.3
9253
80.3
10:688128:5120:1
-l version=2,size=128m -i attr=2
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
91268
99
464753
49.3
66828
8
67592
81
125015
10
362.6
0
2745
66.3
8755
99
9344
74.3
3198
73.7
7310
85
7252
60
10:688128:5120:1
-l version=2,size=16384b -d su=262144,sw=9
rw
64352M
95877
99
435503
46
60831
7
67252
80.3
125840
10
361.2
0
3044
75
8688
99.7
5479
53.3
3013
70.7
7009
81.3
11294
86.7
10:688128:5120:1
-l version=2,size=16384b -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
95787
99
481562
51
90843
11.3
82987
94.7
240849
13
378.4
0
3249
75.3
8124
89.7
9918
83.7
3582
85.3
7283
82.7
6402
50.3
10:688128:5120:1
-l version=2,size=16384b -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
88591
99
475663
50.7
83011
10.3
77000
89
220378
14
362.7
0
4197
97
8659
99.3
6089
47.3
3121
76
6257
71.3
10488
79.7
10:688128:5120:1
-l version=2,size=128m -d su=262144,sw=9
rw
64352M
91727
99
465144
49
80260
10
77585
89.7
233708
14.7
368.1
0
3739
87
8507
99.7
5478
42.7
3396
78.7
8602
94
8534
64.7
10:688128:5120:1
-l version=2,size=128m -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
95867
99
464367
49.3
80796
10
75411
87.7
218984
17
370.1
0
3671
87
8658
99
5500
50.7
3530
82.7
7536
85
8368
66.7
10:688128:5120:1
-l version=2,size=128m -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
98446
99
459868
48.3
78259
10
76096
88
226517
16
365.3
0
3281
76.7
6440
73
6267
47.3
2992
69.7
7664
89.3
7732
60.7
10:688128:5120:1
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
rw
64352M
91304
99
442525
47.3
82323
10.3
76163
88
229872
15.7
380.9
0
3015
71
7284
80.7
11545
89.3
3737
88
4466
50
9422
71
10:688128:5120:1
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
92167
99
396332
42
81941
10.3
76401
88
233618
15.7
382.5
0
3626
86.7
6662
75.3
11444
84
4003
91.7
8074
92.3
4023
31.7
10:688128:5120:1
-l version=2,size=16384b -i attr=2 -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
64352M
92294
99
479220
50.3
81340
10.3
77236
89
235145
14.7
372.3
0.33
3854
89.7
8452
99
5212
43
3518
87
5118
60.7
12679
95
10:688128:5120:1
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
rw
64352M
95706
99
471999
49.7
82204
10.3
77009
88.7
225017
16.3
368.1
0
3740
88
8535
99.3
6571
49.7
3826
88.3
4362
49.7
11466
86.7
10:688128:5120:1
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k
64352M
96086
99
407269
43
82465
10
75962
88
236212
16.7
372.6
0.33
3793
88.3
8828
99.3
5416
45.3
4103
94.3
7496
85
8085
65
10:688128:5120:1
-l version=2,size=128m -i attr=2 -d su=262144,sw=9
rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier
MAX:
98567
99
516368
54
92771
12
83336
95
240849
17
383.0
0.67
4197
97
9000
99.7
13825
99
4103
94.3
8964
100
12679
97.3
MIN:
88295
99
396332
42
60831
7
67252
80.3
125015
10
341.7
0
2488
63
6440
73
5212
42.7
2669
64.3
4362
49.7
4023
31.7
AVG:
93504
99
470040
49.5
82048
10.2
77414
89.2
217398
14.2
369.9
0.12
3239
77.7
8376
95
10599
80.5
3281
77.4
7513
85.5
9807
76.6
MED:
92924
99
473942
50
81824
10.3
77014
88.8
228195
14.3
372.0
0
3186
76
8658
99
12017
91.2
3160
74.8
7853
89.8
10239
81
Controller
RAID
# Disks
Stripe Size
Type
Disk Size
FS Size
OS
Kernel
Motherboard
Chipset
RAM
Processors
3ware 9650SE-12M
6
11
256 KB
SATA
1.82 TB
17 TB
CentOS 5.3
2.6.18-128.el5
Supermicro X8DTU
Intel 5520 (Tylersburg)
32GB PC3-8500 1066MHz
Dual Intel Xeon E5540 @2.53GHz
________________________________
This email and any attachments may be confidential and are intended solely for the use of the individual to whom it is addressed.
If you are not the intended recipient of this email, the following stipulations govern the use of this information: You may not take any action based upon its contents. You may not copy or show this message or attachments to anyone. You should contact the sender and subsequently delete this message and all attachments.
Any views or opinions expressed are solely those of the author and do not necessarily represent those of Special Operations Technology, Incorporated.
While antivirus software has been applied, you should perform due diligence to check this email and attachments for the presence of viruses. No warranties or assurances are made in relation to the safety and content of this email and attachments. Special Operations Technology, Incorporated accepts no liability for any damages caused by any virus transmitted by or contained in this email and attachments.
No liability is accepted for any consequences arising from this email transmission whatsoever.
Special Operations Technology, Incorporated is a premier IT professional services firm focused in the government and law enforcement space.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20101023/8e548172/attachment.htm>
More information about the xfs
mailing list