Poor performance using discard
Thomas Lynema
lyz27 at yahoo.com
Fri Mar 2 08:57:00 CST 2012
> > Also, I think you need to provide a block trace (output of
> > blktrace/blkparse for the rm -rf workloads) for both the XFS and
> > ext4 cases so we can see what discards are actually being issued and
> > how long they take to complete....
>
Partial xfs results attached this was the overall summary with xfs.
The summary looks like this.
CPU0 (xfs.trace):
Reads Queued: 1,274, 9,216KiB Writes Queued: 9,981,
156,710KiB
Read Dispatches: 1,738, 8,656KiB Write Dispatches: 9,855,
157,818KiB
Reads Requeued: 495 Writes Requeued: 72
Reads Completed: 1,844, 12,916KiB Writes Completed: 15,486,
8,360KiB
Read Merges: 0, 0KiB Write Merges: 42,
364KiB
Read depth: 10 Write depth: 11
IO unplugs: 95 Timer unplugs: 3
CPU1 (xfs.trace):
Reads Queued: 153, 1,372KiB Writes Queued: 14,521,
183,608KiB
Read Dispatches: 235, 2,348KiB Write Dispatches: 15,246,
183,627KiB
Reads Requeued: 24 Writes Requeued: 748
Reads Completed: 197, 1,868KiB Writes Completed: 14,885,
11,825KiB
Read Merges: 0, 0KiB Write Merges: 19,
60KiB
Read depth: 10 Write depth: 11
IO unplugs: 39 Timer unplugs: 7
CPU2 (xfs.trace):
Reads Queued: 1,156, 7,520KiB Writes Queued: 10,873,
160,987KiB
Read Dispatches: 1,586, 7,416KiB Write Dispatches: 10,724,
160,687KiB
Reads Requeued: 446 Writes Requeued: 69
Reads Completed: 586, 3,820KiB Writes Completed: 5,310,
4,103KiB
Read Merges: 0, 0KiB Write Merges: 57,
664KiB
Read depth: 10 Write depth: 11
IO unplugs: 54 Timer unplugs: 8
CPU3 (xfs.trace):
Reads Queued: 126, 1,436KiB Writes Queued: 5,197,
47,004KiB
Read Dispatches: 135, 1,124KiB Write Dispatches: 5,388,
46,187KiB
Reads Requeued: 20 Writes Requeued: 333
Reads Completed: 82, 940KiB Writes Completed: 5,037,
1,551KiB
Read Merges: 0, 0KiB Write Merges: 91,
708KiB
Read depth: 10 Write depth: 11
IO unplugs: 50 Timer unplugs: 8
Total (xfs.trace):
Reads Queued: 2,709, 19,544KiB Writes Queued: 40,572,
548,309KiB
Read Dispatches: 3,694, 19,544KiB Write Dispatches: 41,213,
548,319KiB
Reads Requeued: 985 Writes Requeued: 1,222
Reads Completed: 2,709, 19,544KiB Writes Completed: 40,718,
25,839KiB
Read Merges: 0, 0KiB Write Merges: 209,
1,796KiB
IO unplugs: 238 Timer unplugs: 26
Throughput (R/W): 45KiB/s / 60KiB/s
Events (xfs.trace): 309,109 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (xfs.trace):
Reads Queued: 1,274, 9,216KiB Writes Queued: 9,981,
156,710KiB
Read Dispatches: 1,738, 8,656KiB Write Dispatches: 9,855,
157,818KiB
Reads Requeued: 495 Writes Requeued: 72
Reads Completed: 1,844, 12,916KiB Writes Completed: 15,486,
8,360KiB
Read Merges: 0, 0KiB Write Merges: 42,
364KiB
Read depth: 10 Write depth: 11
IO unplugs: 95 Timer unplugs: 3
CPU1 (xfs.trace):
Reads Queued: 153, 1,372KiB Writes Queued: 14,521,
183,608KiB
Read Dispatches: 235, 2,348KiB Write Dispatches: 15,246,
183,627KiB
Reads Requeued: 24 Writes Requeued: 748
Reads Completed: 197, 1,868KiB Writes Completed: 14,885,
11,825KiB
Read Merges: 0, 0KiB Write Merges: 19,
60KiB
Read depth: 10 Write depth: 11
IO unplugs: 39 Timer unplugs: 7
CPU2 (xfs.trace):
Reads Queued: 1,156, 7,520KiB Writes Queued: 10,873,
160,987KiB
Read Dispatches: 1,586, 7,416KiB Write Dispatches: 10,724,
160,687KiB
Reads Requeued: 446 Writes Requeued: 69
Reads Completed: 586, 3,820KiB Writes Completed: 5,310,
4,103KiB
Read Merges: 0, 0KiB Write Merges: 57,
664KiB
Read depth: 10 Write depth: 11
IO unplugs: 54 Timer unplugs: 8
CPU3 (xfs.trace):
Reads Queued: 126, 1,436KiB Writes Queued: 5,197,
47,004KiB
Read Dispatches: 135, 1,124KiB Write Dispatches: 5,388,
46,187KiB
Reads Requeued: 20 Writes Requeued: 333
Reads Completed: 82, 940KiB Writes Completed: 5,037,
1,551KiB
Read Merges: 0, 0KiB Write Merges: 91,
708KiB
Read depth: 10 Write depth: 11
IO unplugs: 50 Timer unplugs: 8
Total (xfs.trace):
Reads Queued: 2,709, 19,544KiB Writes Queued: 40,572,
548,309KiB
Read Dispatches: 3,694, 19,544KiB Write Dispatches: 41,213,
548,319KiB
Reads Requeued: 985 Writes Requeued: 1,222
Reads Completed: 2,709, 19,544KiB Writes Completed: 40,718,
25,839KiB
Read Merges: 0, 0KiB Write Merges: 209,
1,796KiB
IO unplugs: 238 Timer unplugs: 26
Throughput (R/W): 45KiB/s / 60KiB/s
Events (xfs.trace): 309,109 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (xfs.trace):
Reads Queued: 1,274, 9,216KiB Writes Queued: 9,981,
156,710KiB
Read Dispatches: 1,738, 8,656KiB Write Dispatches: 9,855,
157,818KiB
Reads Requeued: 495 Writes Requeued: 72
Reads Completed: 1,844, 12,916KiB Writes Completed: 15,486,
8,360KiB
Read Merges: 0, 0KiB Write Merges: 42,
364KiB
Read depth: 10 Write depth: 11
IO unplugs: 95 Timer unplugs: 3
CPU1 (xfs.trace):
Reads Queued: 153, 1,372KiB Writes Queued: 14,521,
183,608KiB
Read Dispatches: 235, 2,348KiB Write Dispatches: 15,246,
183,627KiB
Reads Requeued: 24 Writes Requeued: 748
Reads Completed: 197, 1,868KiB Writes Completed: 14,885,
11,825KiB
Read Merges: 0, 0KiB Write Merges: 19,
60KiB
Read depth: 10 Write depth: 11
IO unplugs: 39 Timer unplugs: 7
CPU2 (xfs.trace):
Reads Queued: 1,156, 7,520KiB Writes Queued: 10,873,
160,987KiB
Read Dispatches: 1,586, 7,416KiB Write Dispatches: 10,724,
160,687KiB
Reads Requeued: 446 Writes Requeued: 69
Reads Completed: 586, 3,820KiB Writes Completed: 5,310,
4,103KiB
Read Merges: 0, 0KiB Write Merges: 57,
664KiB
Read depth: 10 Write depth: 11
IO unplugs: 54 Timer unplugs: 8
CPU3 (xfs.trace):
Reads Queued: 126, 1,436KiB Writes Queued: 5,197,
47,004KiB
Read Dispatches: 135, 1,124KiB Write Dispatches: 5,388,
46,187KiB
Reads Requeued: 20 Writes Requeued: 333
Reads Completed: 82, 940KiB Writes Completed: 5,037,
1,551KiB
Read Merges: 0, 0KiB Write Merges: 91,
708KiB
Read depth: 10 Write depth: 11
IO unplugs: 50 Timer unplugs: 8
Total (xfs.trace):
Reads Queued: 2,709, 19,544KiB Writes Queued: 40,572,
548,309KiB
Read Dispatches: 3,694, 19,544KiB Write Dispatches: 41,213,
548,319KiB
Reads Requeued: 985 Writes Requeued: 1,222
Reads Completed: 2,709, 19,544KiB Writes Completed: 40,718,
25,839KiB
Read Merges: 0, 0KiB Write Merges: 209,
1,796KiB
IO unplugs: 238 Timer unplugs: 26
Throughput (R/W): 45KiB/s / 60KiB/s
Events (xfs.trace): 309,109 entries
Skips: 0 forward (0 - 0.0%)
CPU0 (xfs.trace):
Reads Queued: 1,274, 9,216KiB Writes Queued: 9,981,
156,710KiB
Read Dispatches: 1,738, 8,656KiB Write Dispatches: 9,855,
157,818KiB
Reads Requeued: 495 Writes Requeued: 72
Reads Completed: 1,844, 12,916KiB Writes Completed: 15,486,
8,360KiB
Read Merges: 0, 0KiB Write Merges: 42,
364KiB
Read depth: 10 Write depth: 11
IO unplugs: 95 Timer unplugs: 3
CPU1 (xfs.trace):
Reads Queued: 153, 1,372KiB Writes Queued: 14,521,
183,608KiB
Read Dispatches: 235, 2,348KiB Write Dispatches: 15,246,
183,627KiB
Reads Requeued: 24 Writes Requeued: 748
Reads Completed: 197, 1,868KiB Writes Completed: 14,885,
11,825KiB
Read Merges: 0, 0KiB Write Merges: 19,
60KiB
Read depth: 10 Write depth: 11
IO unplugs: 39 Timer unplugs: 7
CPU2 (xfs.trace):
Reads Queued: 1,156, 7,520KiB Writes Queued: 10,873,
160,987KiB
Read Dispatches: 1,586, 7,416KiB Write Dispatches: 10,724,
160,687KiB
Reads Requeued: 446 Writes Requeued: 69
Reads Completed: 586, 3,820KiB Writes Completed: 5,310,
4,103KiB
Read Merges: 0, 0KiB Write Merges: 57,
664KiB
Read depth: 10 Write depth: 11
IO unplugs: 54 Timer unplugs: 8
CPU3 (xfs.trace):
Reads Queued: 126, 1,436KiB Writes Queued: 5,197,
47,004KiB
Read Dispatches: 135, 1,124KiB Write Dispatches: 5,388,
46,187KiB
Reads Requeued: 20 Writes Requeued: 333
Reads Completed: 82, 940KiB Writes Completed: 5,037,
1,551KiB
Read Merges: 0, 0KiB Write Merges: 91,
708KiB
Read depth: 10 Write depth: 11
IO unplugs: 50 Timer unplugs: 8
Total (xfs.trace):
Reads Queued: 2,709, 19,544KiB Writes Queued: 40,572,
548,309KiB
Read Dispatches: 3,694, 19,544KiB Write Dispatches: 41,213,
548,319KiB
Reads Requeued: 985 Writes Requeued: 1,222
Reads Completed: 2,709, 19,544KiB Writes Completed: 40,718,
25,839KiB
Read Merges: 0, 0KiB Write Merges: 209,
1,796KiB
IO unplugs: 238 Timer unplugs: 26
Throughput (R/W): 45KiB/s / 60KiB/s
Events (xfs.trace): 309,109 entries
Skips: 0 forward (0 - 0.0%)
>
>
> Thanks for the input and patience.
>
> ~tom
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xfs.trace.blktrace.3.bz2
Type: application/x-bzip
Size: 358698 bytes
Desc: not available
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20120302/76837e22/attachment-0001.bz2>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://oss.sgi.com/pipermail/xfs/attachments/20120302/76837e22/attachment-0001.sig>
More information about the xfs
mailing list