xfs
[Top] [All Lists]

about the xfs performance

To: xfs@xxxxxxxxxxx
Subject: about the xfs performance
From: Songbo Wang <hack.coo@xxxxxxxxx>
Date: Mon, 11 Apr 2016 22:14:06 +0800
Delivered-to: xfs@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=c1MQRj4Ssnq0D693XT83n7SCQuuCrP6oZT0SUyDvRgI=; b=neLQhvoOGPB2Yr+8ezvUDGlIQSZUi18iODbp1y4p037CSYcGhlpON57a1eauwNh5Bp c0kgaV2xeJq44hCnNH0ULtjNDDytHrSp+JgrwGLdNIHg3Aoap3YFFchJfwERYnTUXQFm M7SkQICS9DFUcg9XFu2j8nd7o8hoWX9heifmREP44ag1lk1pqgX7bRFiysosWQhXnYP5 wp2Ud7unqMsoP85CWfYj1bJECp9KKHgerbMUhxo1smT59STHnJzsnmzZpAZ6f9p0Qg81 YJ4C6myz/odHCo6jfHUCfqbnVVWjWvWt7f9GdGBAqwCnfFvJtRFku9QjDM9e27J0aW8O aylg==
Hi xfsers:

I got some troubles on the performance of Âxfs.Â
The environment is ,Â
  Âxfs version is 3.2.1,Â
  Âcentos 7.1,Â
  Âkernel version:3.10.0-229.el7.x86_64.
  Âpcie-ssd card,Â
  Âmkfs: mkfs.xfs /dev/hioa2 -f -n size=64k -i size=512 -d agcount=40 -l size=1024m.
  Âmount:Âmount /dev/hioa2 /mnt/ Â-t xfs -o rw,noexec,nodev,noatime,nodiratime,nobarrier,discard,inode64,logbsize=256k,delaylog
I use the following command to test iops:Âfio -ioengine=libaio -bs=4k -direct=1 -thread -rw=randwrite -size=50G -filename=/mnt/test -name="EBS 4KB randwrite test" -iodepth=64 -runtime=60
The results is normal at the beginning which is about 210kÂïbut some seconds later, the results down to 19kÂ.

I did a senond test ,Â
  Âumount the /dev/hioa2,Â
  Âfio -ioengine=libaio -bs=4k -direct=1 Â-thread -rw=randwrite -filename=/dev/hioa2 Â-name="EBS 8KB randwrite test" -iodepth=64 -runtime=60
The results was normal, the iops is about 210k all the time.

Then I use perf-tools to trace xfs:
  Â# ./funccount -i 1 'xfs*'

In the first test case , when the iops seems normal , the results is as follows:
FUNC Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂCOUNT
xfs_free_eofblocks          Â1
xfs_attr_get             Â5
xfs_attr_get_int           Â5
xfs_attr_name_to_xname        Â5
xfs_attr_shortform_getvalue      5
xfs_da_hashname            5
xfs_ilock_attr_map_shared       5
xfs_xattr_get             5
xfs_fs_statfs             7
xfs_icsb_count            Â7
xfs_icsb_sync_counters        Â7
xfs_icsb_sync_counters_locked     7
xfs_can_free_eofblocks        Â9
xfs_file_release           Â9
xfs_release              9
xfs_icsb_counter_disabled      Â21
xfs_file_open            Â30
xfs_file_aio_read          Â33
xfs_readlink             37
xfs_vn_follow_link          37
xfs_vn_put_link           Â37
xfs_buf_delwri_submit_nowait     41
xfs_trans_ail_cursor_first      41
xfs_trans_ail_cursor_init      Â41
xfs_vn_getattr            44
xfs_file_mmap            Â48
xfs_cil_prepare_item.isra.1 Â Â Â Â1004
xfs_extent_busy_clear       Â1004
xfs_inode_item_committing     Â1004
xfs_inode_item_data_fork_size. Â Â 1004
xfs_inode_item_format       Â1004
xfs_inode_item_format_data_for   1004
xfs_inode_item_size        Â1004
xfs_inode_item_unlock       Â1004
xfs_log_calc_unit_res       Â1004
xfs_log_commit_cil         1004
xfs_log_done            1004
xfs_log_reserve          Â1004
xfs_log_space_wake         1004
xfs_log_ticket_put         1004
xfs_trans_add_item         1004
xfs_trans_alloc          Â1004
xfs_trans_apply_dquot_deltas    1004
xfs_trans_commit          1004
xfs_trans_free           1004
xfs_trans_free_dqinfo       Â1004
xfs_trans_free_item_desc      1004
xfs_trans_free_items        1004
xfs_trans_ijoin          Â1004
xfs_trans_log_inode        Â1004
xfs_trans_reserve         Â1004
xfs_trans_unreserve_and_mod_sb   1004
xfs_vn_update_time         1004
xfs_destroy_ioend         84845
xfs_end_io_direct_write      84845
xfs_finish_ioend         Â84845
xfs_file_aio_write        Â84880
xfs_file_dio_aio_write      Â84882
xfs_vm_direct_IO Â Â Â Â Â Â Â Â Â84882
xfs_get_blocks_direct       84883
xfs_bmapi_read          Â84885
xfs_bmap_search_extents      84886
xfs_bmap_search_multi_extents   84886
xfs_iext_bno_to_ext        84887
xfs_iext_bno_to_irec       Â84887
xfs_iext_get_ext         Â84890
xfs_iext_idx_to_irec       Â84890
xfs_ilock_data_map_shared     84891
xfs_map_buffer.isra.9 Â Â Â Â Â Â 84892
xfs_file_aio_write_checks     84893
xfs_alloc_ioend          84895
xfs_bmapi_trim_map.isra.11 Â Â Â Â84900
xfs_fsb_to_db           84902
xfs_bmapi_update_map       Â84903
xfs_bmbt_get_all         169793
xfs_find_bdev_for_inode     Â169794
xfs_ilock            Â170868
xfs_iunlock           Â170872
xfs_bmbt_get_blockcount     Â555016
xfs_bmbt_get_startoff      2277490

when the results down to 19kÂï the trace results are:
FUNC Â Â Â Â Â Â Â Â Â Â Â Â Â Â ÂCOUNT
xfs_bitmap_empty           Â1
xfs_free_eofblocks          Â1
xfs_attr_get             Â5
xfs_attr_get_int           Â5
xfs_attr_name_to_xname        Â5
xfs_attr_shortform_getvalue      5
xfs_da_hashname            5
xfs_ilock_attr_map_shared       5
xfs_xattr_get             5
xfs_fs_statfs             7
xfs_icsb_count            Â7
xfs_icsb_sync_counters        Â7
xfs_icsb_sync_counters_locked     7
xfs_can_free_eofblocks        Â9
xfs_file_release           Â9
xfs_release              9
xfs_file_open            Â30
xfs_file_aio_read          Â33
xfs_readlink             37
xfs_vn_follow_link          37
xfs_vn_put_link           Â37
xfs_buf_delwri_submit_nowait     41
xfs_trans_ail_cursor_first      41
xfs_trans_ail_cursor_init      Â41
xfs_alloc_ag_vextent         43
xfs_alloc_ag_vextent_near      Â43
xfs_alloc_find_best_extent      43
xfs_alloc_fix_freelist        43
xfs_alloc_fixup_trees        Â43
xfs_alloc_read_agf          43
xfs_alloc_read_agfl         Â43
xfs_alloc_update           43
xfs_alloc_update_counters.isra    43
xfs_alloc_vextent          Â43
xfs_bmbt_alloc_block         43
xfs_btree_get_buf_block.constp    43
xfs_btree_reada_bufs         43
xfs_btree_readahead_sblock.isr    43
xfs_btree_split           Â43
xfs_btree_split_worker        43
xfs_read_agf             43
xfs_trans_get_buf_map        Â43
xfs_allocbt_dup_cursor        44
xfs_allocbt_get_minrecs       Â44
xfs_allocbt_init_rec_from_cur    Â44
xfs_allocbt_update_lastrec      44
xfs_alloc_fix_minleft        Â44
xfs_alloc_log_agf          Â44
xfs_alloc_longest_free_extent    Â44
xfs_bmbt_init_rec_from_key      44
xfs_bmbt_update_cursor        44
xfs_btree_buf_to_ptr.isra.17 Â Â Â Â 44
xfs_btree_check_sblock        44
xfs_btree_init_block_cur.isra. Â Â Â 44
xfs_btree_init_block_int       44
xfs_btree_islastblock        Â44
xfs_trans_mod_dquot_byino      Â44
xfs_trans_mod_sb           44
xfs_vn_getattr            44
xfs_file_mmap            Â48
xfs_iext_irec_new          Â57
xfs_iext_realloc_indirect      Â57
xfs_btree_copy_ptrs         Â66
xfs_btree_shift_keys.isra.23 Â Â Â Â 66
xfs_btree_shift_ptrs         66
xfs_btree_log_ptrs          72
xfs_alloc_get_rec          Â86
xfs_alloc_lookup_eq         Â86
xfs_alloc_compute_aligned      Â88
xfs_btree_get_rec          Â88
xfs_extent_busy_trim         88
xfs_iext_add_indirect_multi     108
xfs_alloc_compute_diff       Â110
xfs_alloc_fix_len          110
xfs_allocbt_init_cursor       132
xfs_iext_irec_compact        163
xfs_iext_remove           163
xfs_iext_remove_indirect      Â163
xfs_buf_item_relse         Â173
xfs_allocbt_init_ptr_from_cur    176
xfs_btree_set_sibling.isra.12 Â Â Â 176
xfs_buf_item_free          177
xfs_buf_item_free_format      Â177
xfs_buf_hold            Â248
xfs_buf_item_get_format       248
xfs_log_item_init          248
xfs_allocbt_get_maxrecs       264
xfs_bmbt_get_minrecs        Â326
xfs_btree_dec_cursor        Â369
xfs_btree_delete          Â369
xfs_btree_delrec          Â369
xfs_bmbt_set_allf          633
xfs_bmbt_set_startblock      Â1007
xfs_bmbt_set_startoff       Â1007
xfs_trans_alloc          Â1029
xfs_vn_update_time         1029
xfs_btree_lshift          1048
xfs_btree_decrement        Â1332
xfs_allocbt_init_key_from_rec   Â1481
xfs_allocbt_key_diff        2427
xfs_btree_check_lblock       3653
xfs_btree_lastrec         Â3653
xfs_buf_free            3660
xfs_bmbt_dup_cursor        Â3696
xfs_btree_check_block       Â3696
xfs_buf_allocate_memory      Â3703
xfs_btree_dup_cursor        3738
xfs_btree_increment        Â4324
xfs_btree_make_block_unfull    Â4699
xfs_btree_rshift          4699
xfs_btree_updkey          4738
xfs_btree_copy_keys.isra.22 Â Â Â Â4824
xfs_btree_log_keys         4832
xfs_iext_add            4876
xfs_iext_insert          Â4876
xfs_btree_reada_bufl        5465
xfs_btree_readahead_lblock.isr   5465
xfs_buf_readahead_map       Â5508
xfs_btree_readahead        Â5670
xfs_inode_item_format       Â6071
xfs_inode_item_format_data_for   6072
xfs_inode_item_size        Â6072
xfs_inode_item_data_fork_size. Â Â 6073
xfs_trans_log_inode        Â6115
xfs_buf_item_pin          6315
xfs_bmapi_write          Â6674
xfs_end_io             6674
xfs_iomap_write_unwritten     Â6674
xfs_bmap_add_extent_unwritten_ Â Â 6675
xfs_bmapi_convert_unwritten    Â6675
xfs_destroy_ioend         Â6675
xfs_end_io_direct_write      Â6676
xfs_finish_ioend          6677
xfs_bmap_finish          Â6683
xfs_file_aio_write         6784
xfs_bmapi_read           6785
xfs_file_aio_write_checks     Â6785
xfs_file_dio_aio_write       6785
xfs_get_blocks_direct       Â6785
xfs_ilock_data_map_shared     Â6785
xfs_map_buffer.isra.9 Â Â Â Â Â Â Â6785
xfs_vm_direct_IO Â Â Â Â Â Â Â Â Â 6785
xfs_alloc_ioend          Â6786
xfs_fsb_to_db           Â6786
xfs_bmbt_set_blockcount      Â7691
xfs_log_commit_cil         7704
xfs_trans_commit          7704
xfs_trans_free_items        7704
xfs_trans_reserve         Â7705
xfs_trans_unreserve_and_mod_sb   7705
xfs_log_done            7706
xfs_trans_free           7706
xfs_log_reserve          Â7708
xfs_trans_ijoin          Â7708
xfs_extent_busy_clear       Â7712
xfs_inode_item_unlock       Â7712
xfs_log_space_wake         7712
xfs_log_ticket_put         7712
xfs_trans_apply_dquot_deltas    7712
xfs_inode_item_committing     Â7713
xfs_log_calc_unit_res       Â7713
xfs_trans_free_dqinfo       Â7713
xfs_bmbt_update          Â8311
xfs_btree_update          8354
xfs_bmbt_init_rec_from_cur     9758
xfs_bmbt_disk_set_all       Â9760
xfs_bmbt_set_all          9760
xfs_btree_insert          9795
xfs_btree_insrec          9838
xfs_bmbt_init_cursor       Â10379
xfs_btree_del_cursor       Â10498
xfs_trans_del_item        Â11101
xfs_buf_item_dirty        Â11109
xfs_bmbt_lookup_eq        Â11551
xfs_bmbt_init_ptr_from_cur    Â11563
xfs_btree_get_iroot.isra.8 Â Â Â Â11563
xfs_buf_item_format        11690
xfs_buf_item_size         11690
xfs_buf_item_size_segment.isra  Â11690
xfs_btree_lookup         Â11723
xfs_btree_get_sibling.isra.11 Â Â 11974
xfs_buf_item_unlock        12229
xfs_buf_item_committing      12237
xfs_icsb_modify_counters     Â13351
xfs_icsb_lock_cntr        Â13365
xfs_icsb_unlock_cntr       Â13365
xfs_iext_bno_to_irec       Â13458
xfs_bmap_search_extents      13459
xfs_bmap_search_multi_extents   13459
xfs_iext_bno_to_ext        13459
xfs_bmapi_trim_map.isra.11 Â Â Â Â13469
xfs_bmapi_update_map       Â13469
xfs_find_bdev_for_inode      13573
xfs_btree_shift_recs.isra.24 Â Â Â14798
xfs_cil_prepare_item.isra.1 Â Â Â 17769
xfs_bmbt_disk_set_allf      Â18126
xfs_btree_is_lastrec       Â18581
xfs_btree_log_block        19648
xfs_btree_set_ptr_null.isra.10 Â Â19649
xfs_btree_offsets         19703
xfs_iunlock            21348
xfs_ilock             21349
xfs_btree_copy_recs.isra.21 Â Â Â 22849
xfs_buf_item_init         23331
xfs_buf_unlock          Â23345
xfs_btree_setbuf         Â23948
xfs_btree_log_recs        Â24190
xfs_icsb_counter_disabled     26751
xfs_buf_read_map         Â28786
xfs_buf_get_map          28829
xfs_btree_ptr_is_null.isra.9 Â Â Â29274
xfs_btree_read_buf_block.isra. Â Â29684
xfs_btree_set_refs.isra.13 Â Â Â Â29712
xfs_trans_add_item        Â31037
xfs_trans_free_item_desc     Â31052
xfs_buf_trylock          32463
xfs_perag_put           32606
xfs_perag_get           32633
xfs_buf_rele           Â32693
xfs_trans_brelse         Â35148
xfs_btree_ptr_addr        Â38591
xfs_btree_ptr_offset       Â38738
xfs_trans_read_buf_map      Â40938
xfs_trans_buf_item_match     Â41000
xfs_btree_ptr_to_daddr      Â44562
xfs_btree_lookup_get_block    Â46548
xfs_trans_log_buf         48783
xfs_buf_item_log         Â48806
xfs_trans_buf_set_type      Â48815
xfs_iext_get_ext         Â51712
xfs_bmbt_get_all         Â53669
xfs_btree_get_block        54424
xfs_iext_idx_to_irec       Â56768
xfs_bmbt_get_maxrecs       Â57774
xfs_bmbt_get_blockcount      89650
xfs_bmbt_disk_get_startoff    Â98285
xfs_bmbt_init_key_from_rec    Â98285
xfs_btree_rec_addr        115199
xfs_btree_rec_offset       163635
xfs_btree_key_addr        176837
xfs_btree_key_offset       186525
xfs_bmbt_key_diff        Â254763
xfs_lookup_get_search_key    Â257190
xfs_bmbt_get_startoff      Â363158
xfs_next_bit           631334
xfs_buf_offset         Â1166477


Compare with the two results above, I found Âwhen the iops down to 19kÂ, lots of functions were traced, such asÂxfs_buf_trylock,Âxfs_iext_insert,Âxfs_btree_insert, etc.

I cannot find what cause the iops down to 19kÂ, any suggestion ?


<Prev in Thread] Current Thread [Next in Thread>