xfs
[Top] [All Lists]

TAKE - xfsdump/xfsrestore

To: linux-xfs@xxxxxxxxxxx
Subject: TAKE - xfsdump/xfsrestore
From: Timothy Shimmin <tes@xxxxxxxxxxxxxxxxxxxxxxx>
Date: Thu, 13 Dec 2001 20:30:39 +1100 (EST)
Sender: owner-linux-xfs@xxxxxxxxxxx
This puts back some changes from IRIX (pv#837505) into Linux 
primarily for xfsrestore.
One main change was to speed up xfsrestore during directory
processing when the number of directory entries is greater
than around a million.
Normal verbose msg now outputs what drive dump strategy is being 
used since this has been a source of confusion in the past.
A few other diagnostics have been added.

--Tim

Date:  Thu Dec 13 01:23:07 PST 2001
Workarea:  snort.melbourne.sgi.com:/home/diskb/build4/tes/slinx-xfs
desc:  xfsrestore performance improvements when have over 1 million files

The following file(s) were checked into:
  bonnie.engr.sgi.com:/isms/slinx/2.4.x-xfs


Modid:  xfs-cmds:slinx:108365a
cmd/xfsdump/VERSION - 1.24
        - bump version for xfsrestore perf changes

cmd/xfsdump/doc/CHANGES - 1.28
        - Brief description of changes for xfsrestore perf. improvement
          for large fs.

cmd/xfsdump/man/man8/xfsrestore.8 - 1.7
        - add info about -w option

cmd/xfsdump/dump/content.c - 1.16
        - Print out more useful dump size estimates.

cmd/xfsdump/restore/node.c - 1.2
        - - don't assert mmap64() success - TEST it - admittedly only for hdr
          - do nodesperseg calculation for large window support
          - do a loop calling mmap64() on smaller size requests to
          try to get the largest map to hold the dirent tree [optimisation]
          - in node_alloc()
          -> test win_map() failure instead of assert it
          -> add some diagnostics
          - in node_map()
          -> test win_map() failure instead of assert it
          - in node_free()
          -> test node_map() failure instead of assert it

cmd/xfsdump/restore/node.h - 1.2
        - - add large-window param
          - rename mincnt to dir_nondir - to better describe what it is

cmd/xfsdump/restore/tree.c - 1.10
        - - add n_sibprevh - previous sibling ptr - for fast deletes 
[optimisation]
          - update nodesz for prev ptr field
          - add padding as struct will be implictly rounded to
          8 byte boundary - this makes it clearer
          - in tree_init(), test for Node allocation failing
          - in tree_begindir(), test for Node allocation failing
          - in tree_addent(), test for Node allocation failing
          - in tree_cb_links(),
          -> if callback function fails then return RV_NOTOK
          -> if Node allocation fails then return RV_ERROR
          - in Node_alloc
          -> test for node_alloc() failure instead of asserting it
          -> init our new previous sibling ptr
          - in Node_free
          -> reset our new previous sibling ptr
          - in adopt()
          -> need to update the previous sibling ptr
          when child added to front of list
          (assuming a child previously exists)
          - in disown() - the child deletion function
          -> can use previous sibling pointer instead of
          requiring a while loop to find the previous
          sibling
          - add some diagnostics to link_in
          - add some diagnostics to hash_find

cmd/xfsdump/restore/tree.h - 1.2
        - - change prototypes which now have return values
          to return errors
          - add largewindow param

cmd/xfsdump/restore/getopt.h - 1.2
        - Add -w option for restore.

cmd/xfsdump/restore/content.c - 1.18
        - - add dirent counter - so can see how many dir entries processed
          - add large window option
          - add conditional compiled "still waiting for dirs" msgs
          - turn window locks off around applydirdump()
          as must be single threaded [optimisation]
          - turn window locks off around treepost()
          as must be single threaded [optimisation]
          - handle case where tree_begindir() fails in applydirdump()
          return the rv
          - handle case where tree_addent() fails in applydirdump()
          return the rv
          - add summary msg for dirs and dirents processed from dump
          - set rval for Media_mfile_next()
          - handle case where tree_cb_links() fails in restore_file()
          return the rv
          - handle case where tree_cb_links() fails in restore_extattr()
          return the rvrestore/getopt.h
          - have multiple extended attribute buffers

cmd/xfsdump/restore/win.c - 1.3
        - - get rid of w_lasttouched field and the setting of it
          as it is never used and it calls time()
          many times over [ optimisations ]
          - add t_winmmaps and win_getnum_mmaps()
          to see how many mmap calls for tree windows
          are made - so can check on performance
          - add win_locks_on() and win_locks_off() functions
          - add more diagnostics
          - in win_map(), test mmap64() success instead of asserting it

cmd/xfsdump/restore/win.h - 1.2
        - add fuction protos: win_locks_off, win_locks_on and win_getnum_mmaps

cmd/xfsdump/common/openutil.c - 1.2
        - Fix up the allocation of a name which includes pid.

cmd/xfsdump/common/drive.c - 1.2
        - Make the chosen strategy be output without needing debug level.

cmd/xfsdump/common/mlog.h - 1.4
        - Add mlog_override_level prototype.

cmd/xfsdump/common/mlog.c - 1.8
        - Add mlog_override_level function useful for debugging
          problems where need msgs unconditionally.

cmd/xfsdump/common/drive_scsitape.c - 1.5
        - Make the strategy name more human readable.

cmd/xfsdump/common/drive_minrmt.c - 1.5
        - Make the strategy name more human readable.

cmd/xfsdump/common/main.c - 1.16
        - Add usage info for -w option.

cmd/xfsdump/common/drive_simple.c - 1.3
        - Make the strategy name more human readable.

cmd/xfstests/common.dump - 1.27
        - Update the filtering with quotas on the message about the
          number of entries processed since a quota file will add
          to the number.

cmd/xfstests/064.out - 1.2
        - Update for message about the number of entries processed.

cmd/xfstests/061.out - 1.3
        - Update for message about the number of entries processed.

cmd/xfstests/063.out - 1.2
        - Update for message about the number of entries processed.

cmd/xfstests/047.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/046.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/043.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/055.out - 1.6
        - Update for message about the number of entries processed.

cmd/xfstests/056.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/025.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/024.out - 1.6
        - Update for message about the number of entries processed.

cmd/xfstests/027.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/026.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/023.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/022.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/028.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/035.out - 1.6
        - Update for message about the number of entries processed.

cmd/xfstests/037.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/036.out - 1.5
        - Update for message about the number of entries processed.

cmd/xfstests/039.out - 1.6
        - Update for message about the number of entries processed.

cmd/xfstests/038.out - 1.6
        - Update for message about the number of entries processed.



<Prev in Thread] Current Thread [Next in Thread>
  • TAKE - xfsdump/xfsrestore, Timothy Shimmin <=