On Mon, Feb 07, 2005 at 11:45:30PM +0100, Anders Saaby wrote:
> Hi List,
>
> At the moment we are testing an NFS server based on a Xeon Nocona (em64t) and
> 2.6.11-rc3 compiled for X86_64/em64t as the machine has 8GB memory.
>
> Filesystem is based on XFS with external journal.
.....
> As you can see, the systime is ~90%.
>
> I saw an earlier mail from Nathan regarding xfs_iget_core optimizations in
> december, but as the following kernel profile shows, this procedure is
> responsible for most of the load:
>
> This is a kernel profile at the same time:
>
> <PROFILE SNIP>
> 99621 total 0,0387
> 56322 xfs_iget_core 45,1298
.....
How many inodes are cached? Can you dump out the xfs slabs from
/proc/slabinfo when this is occurring? i.e.
# egrep "^(xfs|linvfs|dentry)" /proc/slabinfo
xfs_acl 212 212 304 4 4 1 : 3844 961
xfs_chashlist 9944 9944 32 22 22 1 : 7812 1953
xfs_ili 5976 5976 192 72 72 1 : 7812 1953
xfs_ifork 0 0 64 0 0 1 : 7812 1953
xfs_efi_item 5760 5760 352 128 128 1 : 3844 961
xfs_efd_item 5764 5764 360 131 131 1 : 3844 961
xfs_buf_item 7998 7998 184 93 93 1 : 7812 1953
xfs_dabuf 2324 2324 24 4 4 1 : 7812 1953
xfs_da_state 132 132 488 4 4 1 : 3844 961
xfs_trans 7600 9522 872 460 529 1 : 3844 961
xfs_inode 160341 201579 544 6858 6951 1 : 3844 961
xfs_btree_cur 332 332 192 4 4 1 : 7812 1953
xfs_bmap_free_item 2324 2324 24 4 4 1 : 7812 1953
xfs_buf_t 8988 8988 384 214 214 1 : 3844 961
linvfs_icache 166115 199500 640 7919 7980 1 : 3844 961
dentry_cache 80693 155682 256 2397 2511 1 : 7812 1953
#
If you have a significant number of cached inodes, and you are
repeatedly missing the cache, then we will spend a significant
amount of time searching the cache...
Judging by the amount of memory used and not in the page cache
(~700MiB from your top output), I'd say there are quite a few
cached inodes.
> NOTE: This behavior does not show unless most of the memory is used for cache
> (Right after reboot, systime is ~5%).
Probably because there aren't many cached inodes. Can you dump the
slabinfo at this time as well?
Cheers,
Dave.
--
Dave Chinner
R&D Software Engineer
SGI Australian Software Group
|