On Mon, Nov 12, 2012 at 03:58:15PM +0200, Linas Jankauskas wrote:
> /dev is automaticaly mounted by automount and it is always equal to
> half physical memory.
Is it? It's not on my RHEL6.3 boxes - it's got a 10MB udev
filesystem mounted on /dev. Anyway, irrelelvant.
> RAID geometry:
> Logical Drive: 1
> Size: 20.0 TB
> Fault Tolerance: RAID 5
> Strip Size: 64 KB
Ok, so su=16, sw=11.
> Here is output of perf top:
> 18.42% [kernel] [k] _spin_lock
> 16.07% [xfs] [k] xfs_alloc_busy_trim
> 10.46% [xfs] [k] xfs_alloc_get_rec
> 9.46% [xfs] [k] xfs_btree_get_rec
> 8.38% [xfs] [k] xfs_btree_increment
> 8.31% [xfs] [k] xfs_alloc_ag_vextent_near
> 6.82% [xfs] [k] xfs_btree_get_block
> 5.72% [xfs] [k] xfs_alloc_compute_aligned
> 4.01% [xfs] [k] xfs_btree_readahead
> 3.53% [xfs] [k] xfs_btree_rec_offset
> 2.92% [xfs] [k] xfs_btree_rec_addr
> 1.30% [xfs] [k] _xfs_buf_find
This looks like allocation of extents, not freeing of extents. Can
you attach the entire output?
> strace of rsync process:
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 99.99 18.362863 165431 111 ftruncate
Which means this must be *extending* files. But an strace is not
what I need right now - the event trace via trace-cmd is what I need
to determine exactly what is happening. Five seconds of tracing
output while this problem is happening is probably going to be
What I'd really like to know is what type of files you are rsyncing
to this box. can you post your typical rsync command? Are you
rsyncing over the top of existing files, or new copies? How big are
the files? are tehy sparse? what's a typical xfs_bmap -vp output on
one of these files that has taken this long to ftruncate?
Further, I need to know what free space looks like in your
filesystem. the output of something like:
for i in `seq 0 1 19`; do
echo -s "\nagno: $i\n"
xfs_db -r -c "freesp -s -a $i" /dev/sda5
xfs_db -r -c "frag" /dev/sda5
While give an indication of the state of freespace in the