xfs
[Top] [All Lists]

XFS Test Results Interpretation

To: "xfs@xxxxxxxxxxx" <xfs@xxxxxxxxxxx>
Subject: XFS Test Results Interpretation
From: Andrew Debenham <adebenham@xxxxxxxxxxx>
Date: Sat, 23 Oct 2010 22:59:58 -0400
Accept-language: en-US
Acceptlanguage: en-US
Thread-index: ActwMZzEeRaM9q+4QAmH40wTMsHiyACk6JiwABejsSA=
Thread-topic: XFS Test Results Interpretation

Hello -

 

I am in the process of trying to determine the best parameters to use when creating and mounting an XFS file system on some new hardware we are going to be using.  My company has two separate applications that will be running on the same hardware (but on separate systems).  One application is the PostgreSQL database and the other is a custom application that does writes to many (~1,000) relatively small (~672MB) files concurrently.

 

I decided to use Bonnie++ to test a 17TB XFS file system, varying the mkfs.xfs and the mount parameters, in order to determine the optimal settings for our new systems.  I originally tried attaching my spreadsheet to this email, but I’m guessing your server doesn’t accept attachments.  So, I’ve inserted my results as a table below; hopefully it will come through in a somewhat readable formatIf not, please send me your email and I’ll forward you the original spreadsheet.  For each set of parameters, I ran Bonnie++ 3 separate times and took the average of the results.

 

Also, a couple of things to note.  First, we are already locked into this version of CentOS and the current kernel, so even though they are a bit dated, we will not be able to upgrade to the latest kernel.  Also, here are the release and version numbers of the XFS packages currently installed on these systems:

kmod-xfs-0.4-2

xfsprogs-devel-2.9.4-1.el5.centos

 

When I started specifying the sunit and swidth parameters, I expected to see the sequential block input and output to improve, but my final results didn’t really show this (maybe my expectations were just wrong?).  I was hoping that someone with a lot more XFS expertise could take a look at my results and interpret them for me?  Do my test results look accurate or is it possible that I setup my test incorrectly?  Based on the two applications that will be using an XFS file system, is there an optimal set of parameters that I can use for one or both applications?  Any suggestions or recommendations would be greatly appreciated.  Thanks in advance for your help in this matter.

 

- Andrew

 

 

  

Data

Sequential Output

Sequential Input

Random Seeks

Sequential Create

Random Create

Number of Files

mkfs.xfs Options

Mount Options

Chunk Size

Per Char

Block

Rewrite

Per Char

Block

Create

Read

Delete

Create

Read

Delete

K/sec

%CP

K/sec

%CP

K/sec

%CP

K/sec

%CP

K/sec

%CP

/sec

%CP

/sec

%CP

/sec

%CP

/sec

%CP

/sec

%CP

/sec

%CP

/sec

%CP

64352M

94013

99

472221

50

85811

11

80600

92

221248

14

377.6

0

2814

70

8036

92.7

12635

93.3

2924

71.7

7927

92.3

9819

80.7

10:688128:5120:1

-l size=16384b

rw

64352M

95284

99

450426

47.7

78142

10

74617

86.7

187983

12.3

341.7

0

2488

63

8672

99

11862

93

2669

67.3

7251

83

11165

87

10:688128:5120:1

-l size=16384b

rw,noatime,nodiratime,logbufs=8

64352M

90512

99

494516

51.3

84420

10.7

78464

89.7

207651

12

379.5

0.33

2694

64

9000

99

12772

97.3

3059

71

8480

93.3

10178

81.3

10:688128:5120:1

-l size=16384b

rw,noatime,nodiratime,logbufs=8,nobarrier

64352M

88295

99

501854

53

89128

11

80788

92.3

230101

16.7

368.4

0

3196

80.3

8552

98.3

12818

94.7

2693

64.3

7189

83.7

11396

90

10:688128:5120:1

-l size=128m

rw

64352M

91501

99

487799

51.3

91343

11.3

83336

95

236927

17

379.7

0

2852

72

8394

96.7

12879

95.7

2865

68

7957

91.3

11540

88

10:688128:5120:1

-l size=128m

rw,noatime,nodiratime,logbufs=8

64352M

96319

99

453202

47

82270

10.3

78626

89.3

204320

12.3

377.1

0.33

2948

70.7

8905

97.7

13072

97.7

3004

69.7

8324

92.3

12029

97.3

10:688128:5120:1

-l size=128m

rw,noatime,nodiratime,logbufs=8,nobarrier

64352M

92101

99

477882

50

90622

11

82742

94.7

239307

14.7

377.0

0

2800

69.3

8954

99

11597

85.3

2699

64.3

8836

100

10299

83.7

10:688128:5120:1

-l version=2,size=16384b

rw

64352M

98567

99

448664

46.7

81706

10

78214

90

217844

16

362.0

0

2529

64

8899

98

12754

95.7

2988

69

8419

93.3

9267

72.7

10:688128:5120:1

-l version=2,size=16384b

rw,noatime,nodiratime,logbufs=8

64352M

92854

99

482572

50.7

79940

9.67

76456

88.3

193196

14.3

344.5

0

2782

68

8925

99.3

12953

98.3

2904

67.7

8272

93.3

10709

86.7

10:688128:5120:1

-l version=2,size=16384b

rw,noatime,nodiratime,logbufs=8,nobarrier

64352M

96836

99

482944

51

92771

12

82157

94

240734

14

383.0

0.67

3176

79.3

8419

97

12932

96

2925

70.3

8572

99

11595

88.3

10:688128:5120:1

-l version=2,size=128m

rw

64352M

96595

99

493446

51.3

82685

10.3

78582

89.7

214803

12.7

366.2

0.33

3644

86.7

8805

99

13503

98.3

3316

77.3

7564

87

9005

69.3

10:688128:5120:1

-l version=2,size=128m

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

94856

99

516368

54

79747

10

75690

87.3

198909

13.7

352.6

0

3680

86.3

8800

99

13825

98.7

3652

85.3

7952

90.3

10891

85

10:688128:5120:1

-l version=2,size=128m

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

90933

99

463918

48.7

81266

10.3

76380

88.3

229959

14

372.3

0.33

2704

67.7

8670

99.3

12473

96.3

2878

70

8572

97.7

9065

70

10:688128:5120:1

-l version=2,size=16384b -i attr=2

rw

64352M

89084

99

459402

49

80776

10.3

75495

86.7

230625

12.7

371.6

0.33

3173

74.7

7410

83.3

13247

96.7

4017

93.3

7778

87.3

10813

82.3

10:688128:5120:1

-l version=2,size=16384b -i attr=2

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

92994

99

486561

51.7

78986

9.67

76097

87.7

230976

12

379.1

0.33

3652

85.3

7255

81.7

11593

83.3

3941

90.7

8964

98.7

9354

71.7

10:688128:5120:1

-l version=2,size=16384b -i attr=2

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

89910

99

508021

53.3

81036

10

78428

90

214695

16.3

362.2

0

2891

72.3

8658

99.3

12888

95

2833

69

7999

93

11380

84.7

10:688128:5120:1

-l version=2,size=128m -i attr=2

rw

64352M

96944

99

487031

51

91412

11.3

82028

93.3

235545

14.3

374.9

0

3589

84.7

8579

99

13744

99

3622

84.3

7983

91

10703

82

10:688128:5120:1

-l version=2,size=128m -i attr=2

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

92401

99

490172

51.7

78263

9.67

77018

88.7

235954

14.3

382.6

0.33

3612

84.3

8754

99

12172

89

3414

82.7

6170

71.3

9253

80.3

10:688128:5120:1

-l version=2,size=128m -i attr=2

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

91268

99

464753

49.3

66828

8

67592

81

125015

10

362.6

0

2745

66.3

8755

99

9344

74.3

3198

73.7

7310

85

7252

60

10:688128:5120:1

-l version=2,size=16384b -d su=262144,sw=9

rw

64352M

95877

99

435503

46

60831

7

67252

80.3

125840

10

361.2

0

3044

75

8688

99.7

5479

53.3

3013

70.7

7009

81.3

11294

86.7

10:688128:5120:1

-l version=2,size=16384b -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

95787

99

481562

51

90843

11.3

82987

94.7

240849

13

378.4

0

3249

75.3

8124

89.7

9918

83.7

3582

85.3

7283

82.7

6402

50.3

10:688128:5120:1

-l version=2,size=16384b -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

88591

99

475663

50.7

83011

10.3

77000

89

220378

14

362.7

0

4197

97

8659

99.3

6089

47.3

3121

76

6257

71.3

10488

79.7

10:688128:5120:1

-l version=2,size=128m -d su=262144,sw=9

rw

64352M

91727

99

465144

49

80260

10

77585

89.7

233708

14.7

368.1

0

3739

87

8507

99.7

5478

42.7

3396

78.7

8602

94

8534

64.7

10:688128:5120:1

-l version=2,size=128m -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

95867

99

464367

49.3

80796

10

75411

87.7

218984

17

370.1

0

3671

87

8658

99

5500

50.7

3530

82.7

7536

85

8368

66.7

10:688128:5120:1

-l version=2,size=128m -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

98446

99

459868

48.3

78259

10

76096

88

226517

16

365.3

0

3281

76.7

6440

73

6267

47.3

2992

69.7

7664

89.3

7732

60.7

10:688128:5120:1

-l version=2,size=16384b -i attr=2 -d su=262144,sw=9

rw

64352M

91304

99

442525

47.3

82323

10.3

76163

88

229872

15.7

380.9

0

3015

71

7284

80.7

11545

89.3

3737

88

4466

50

9422

71

10:688128:5120:1

-l version=2,size=16384b -i attr=2 -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

92167

99

396332

42

81941

10.3

76401

88

233618

15.7

382.5

0

3626

86.7

6662

75.3

11444

84

4003

91.7

8074

92.3

4023

31.7

10:688128:5120:1

-l version=2,size=16384b -i attr=2 -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

64352M

92294

99

479220

50.3

81340

10.3

77236

89

235145

14.7

372.3

0.33

3854

89.7

8452

99

5212

43

3518

87

5118

60.7

12679

95

10:688128:5120:1

-l version=2,size=128m -i attr=2 -d su=262144,sw=9

rw

64352M

95706

99

471999

49.7

82204

10.3

77009

88.7

225017

16.3

368.1

0

3740

88

8535

99.3

6571

49.7

3826

88.3

4362

49.7

11466

86.7

10:688128:5120:1

-l version=2,size=128m -i attr=2 -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k

64352M

96086

99

407269

43

82465

10

75962

88

236212

16.7

372.6

0.33

3793

88.3

8828

99.3

5416

45.3

4103

94.3

7496

85

8085

65

10:688128:5120:1

-l version=2,size=128m -i attr=2 -d su=262144,sw=9

rw,noatime,nodiratime,logbufs=8,logbsize=256k,nobarrier

MAX:

98567

99

516368

54

92771

12

83336

95

240849

17

383.0

0.67

4197

97

9000

99.7

13825

99

4103

94.3

8964

100

12679

97.3

MIN:

88295

99

396332

42

60831

7

67252

80.3

125015

10

341.7

0

2488

63

6440

73

5212

42.7

2669

64.3

4362

49.7

4023

31.7

AVG:

93504

99

470040

49.5

82048

10.2

77414

89.2

217398

14.2

369.9

0.12

3239

77.7

8376

95

10599

80.5

3281

77.4

7513

85.5

9807

76.6

MED:

92924

99

473942

50

81824

10.3

77014

88.8

228195

14.3

372.0

0

3186

76

8658

99

12017

91.2

3160

74.8

7853

89.8

10239

81

Controller

RAID

# Disks

Stripe Size

Type

Disk Size

FS Size

OS

Kernel

Motherboard

Chipset

RAM

Processors

3ware 9650SE-12M 

6

11

256 KB

SATA

1.82 TB

17 TB

CentOS 5.3

2.6.18-128.el5

Supermicro X8DTU

Intel 5520 (Tylersburg)

32GB PC3-8500 1066MHz

Dual Intel Xeon E5540 @2.53GHz

 



This email and any attachments may be confidential and are intended solely for the use of the individual to whom it is addressed.

If you are not the intended recipient of this email, the following stipulations govern the use of this information: You may not take any action based upon its contents. You may not copy or show this message or attachments to anyone. You should contact the sender and subsequently delete this message and all attachments.

Any views or opinions expressed are solely those of the author and do not necessarily represent those of Special Operations Technology, Incorporated.

While antivirus software has been applied, you should perform due diligence to check this email and attachments for the presence of viruses. No warranties or assurances are made in relation to the safety and content of this email and attachments. Special Operations Technology, Incorporated accepts no liability for any damages caused by any virus transmitted by or contained in this email and attachments.

No liability is accepted for any consequences arising from this email transmission whatsoever.

Special Operations Technology, Incorporated is a premier IT professional services firm focused in the government and law enforcement space.
<Prev in Thread] Current Thread [Next in Thread>