on 20 Mar 2002 10:23:49 -0600: Steve Lord wrote:
>
> On Wed, 2002-03-20 at 10:11, Ian D. Hardy wrote:
> > On Wed, 20 Mar 2002 09:06:55 -0600: Steve Lord wrote:
> > >
> >
> > I suspect that this file grow gradually over a reasonably long computational
> > run, hense the OS would tend to deallocate much of the pre-allocated space
> > in
> > an extent before it is required hence the high level of fragmentation (if my
> > understanding of XFS and the pre-allocation of extents is correct).
>
> Is this created locally, or via NFS? It is only a 12M file, it should be
> a lot less fragmented than that.
>
All user access to this filesystem is via NFS (this server is the master
node to a 170+ node computational cluster). Agreed - 889 extents on a
12M file is a bit excessive! Only myself and a couple of other admins
have direct login access to this server/master node.
>
> OK, once the machine has been up for a while - a few hours use should be
> enough, can you send me the output of cat /proc/slabinfo and the free
> command. I do have one bugfix which would result in an oops in kfree
> which was your original problem. It would take heavy memory pressure to
> trigger it - and, I suspect fragmented files.
The servers been up for ~5.5hrs now.
# free
total used free shared buffers cached
Mem: 1028664 1018060 10604 0 16612 927256
-/+ buffers/cache: 74192 954472
Swap: 1028120 11396 1016724
(I've attached the slabinfo output - should be easier to read if the
mail system doesn't reformat it!).
Note, this server is essentially an NFS fileserver (it also runs dhcpd
and rsync services to support the booting of the computional nodes in the
cluster but this is minimal and is not occuring at the time of the
crashes). Therefore the only pressure on memory is from the NFS daemons
and the filesystem/buffer caches + std daemons/backup software - Legato
Networker).
>
> Steve (who wishes he had never got out of bed this morning!)
............ know the feeling, won't help you but I'm 6 hours
ahead of you so ~ 6hours closer to going back to bed!
Thanks
Ian
--
/////////////Technical Coordination, Research Services////////////////////
Ian Hardy Tel: 023 80 593577
Computing Services
Southampton University email: idh@xxxxxxxxxxx
Southampton S017 1BJ, UK. i.d.hardy@xxxxxxxxxxx
\\'BUGS: The notion of errors is ill-defined' (IRIX man page for netstat)\slabinfo - version: 1.1 (statistics) (SMP)
kmem_cache 98 98 284 7 7 1 : 98 98 7
0 0 : 124 62 : 28 7 0 0
nfs_write_data 0 0 380 0 0 1 : 10 10 1
1 0 : 124 62 : 1 2 2 0
nfs_read_data 0 0 364 0 0 1 : 100 165 16
16 0 : 124 62 : 125 33 142 0
nfs_page 0 0 96 0 0 1 : 160 432 10
10 0 : 252 126 : 169 21 180 0
xfs_dqtrx 171 171 200 9 9 1 : 285 85521 1202
1193 0 : 252 126 : 159169 4526 162399 92
xfs_dquots 11 22 356 2 2 1 : 22 61 2
0 0 : 124 62 : 7 7 1 0
tcp_tw_bucket 39 40 96 1 1 1 : 120 5060 36
35 0 : 252 126 : 363 168 494 0
tcp_bind_bucket 159 290 24 2 2 1 : 287 16903 2
0 0 : 252 126 : 1261 137 1362 0
tcp_open_request 0 0 64 0 0 1 : 177 2150 35
35 0 : 252 126 : 1358 73 1396 0
inet_peer_cache 53 234 48 3 3 1 : 312 5344 4
1 0 : 252 126 : 281 61 285 0
ip_fib_hash 11 145 24 1 1 1 : 145 145 1
0 0 : 252 126 : 22 3 13 0
ip_dst_cache 400 552 160 23 23 1 : 576 28384 25
2 0 : 252 126 : 2398 277 2373 1
arp_cache 262 288 120 9 9 1 : 288 8252 13
4 0 : 252 126 : 629 107 593 0
blkdev_requests 640 660 88 15 15 1 : 792 792 18
3 0 : 252 126 : 750 36 128 0
xfs_chashlist 1188 1305 24 9 9 1 : 1576 64268 11
2 0 : 252 126 : 3244 530 2699 1
xfs_ili 4066 4077 144 151 151 1 : 4455 124835 291
140 0 : 252 126 : 10940 1848 8522 35
xfs_ifork 0 0 64 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
xfs_efi_item 56 56 268 4 4 1 : 140 28659 617
613 0 : 124 62 : 35869 2534 37780 5
xfs_efd_item 56 56 268 4 4 1 : 140 32079 554
550 0 : 124 62 : 35883 2457 37779 6
xfs_buf_item 175 175 156 7 7 1 : 350 193760 355
348 0 : 252 126 : 168345 3082 171059 3
xfs_dabuf 145 145 24 1 1 1 : 145 188712 420
419 0 : 252 126 : 411426 2857 413863 0
xfs_da_state 11 22 348 1 2 1 : 44 14312 810
808 0 : 124 62 : 37951 2083 39224 0
xfs_trans 133 133 584 19 19 1 : 168 105512 2451
2432 0 : 124 62 : 301704 7958 306498 710
xfs_inode 10692 10692 444 1188 1188 1 : 16245 76912 3890
2702 0 : 124 62 : 34678 9029 28750 394
xfs_btree_cur 52 52 148 2 2 1 : 104 62935 1364
1362 0 : 252 126 : 182332 3690 184658 0
xfs_bmap_free_item 145 145 24 1 1 1 : 145 169652 356
355 0 : 252 126 : 36622 2369 38635 0
page_buf_t 210 210 188 10 10 1 : 399 326361 329
319 0 : 252 126 : 962344 4262 966213 29
page_buf_reg_t 1 113 32 1 1 1 : 113 113 1
0 0 : 252 126 : 0 2 0 0
avl_object_t 1 113 32 1 1 1 : 113 113 1
0 0 : 252 126 : 0 2 0 0
avl_entry_t 302 378 28 3 3 1 : 504 432732 6
3 0 : 252 126 : 949692 3511 953132 33
dnotify cache 0 0 28 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
file lock cache 156 156 100 4 4 1 : 156 48521 34
30 0 : 252 126 : 64843 528 65323 0
fasync cache 0 0 24 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
uid_cache 6 226 32 2 2 1 : 226 982 2
0 0 : 252 126 : 6 10 8 0
skbuff_head_cache 722 792 164 33 33 1 : 888 1486979 51
18 0 : 252 126 : 2186779 12117 2190604 7823
sock 180 180 832 20 20 2 : 198 2488 98
78 0 : 124 62 : 16435 384 16557 0
sigqueue 21 56 140 2 2 1 : 168 15648 115
113 0 : 252 126 : 9159 746 9788 0
cdev_cache 15 234 48 3 3 1 : 1950 2434 25
22 0 : 252 126 : 1831 55 1835 11
bdev_cache 11 118 64 2 2 1 : 118 118 2
0 0 : 252 126 : 18 4 9 0
mnt_cache 33 112 68 2 2 1 : 112 112 2
0 0 : 252 126 : 27 3 0 0
inode_cache 11784 11784 492 1473 1473 1 : 43288 188656 7258
307 0 : 124 62 : 126056 16912 123171 777
dentry_cache 8745 8745 116 265 265 1 : 45705 475422 1720
24 0 : 252 126 : 144528 6893 140749 437
dquot 0 0 128 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
filp 746 782 112 23 23 1 : 782 879 23
0 0 : 252 126 : 721 48 0 0
names_cache 2 2 4096 2 2 1 : 12 922 609
607 0 : 60 30 : 1948860 1389 1949640 0
buffer_head 239415 240093 104 6485 6489 1 : 973951 6712159 41536
35047 0 : 252 126 : 8699722 124284 8499867 43524
mm_struct 181 189 144 7 7 1 : 189 31135 34
27 0 : 252 126 : 3910 384 4204 0
vm_area_struct 1300 1300 76 26 26 1 : 1500 72363 57
31 0 : 252 126 : 182876 797 182461 37
fs_cache 181 252 44 3 3 1 : 252 38257 3
0 0 : 252 126 : 3958 357 4258 0
files_cache 99 99 424 11 11 1 : 162 9748 127
116 0 : 124 62 : 3957 482 4258 0
signal_act 117 117 1312 39 39 1 : 159 2241 177
138 0 : 60 30 : 3904 587 4204 2
size-131072(DMA) 0 0 131072 0 0 32 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-131072 0 0 131072 0 0 32 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-65536(DMA) 0 0 65536 0 0 16 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-65536 3 3 65536 3 3 16 : 3 3 3
0 0 : 0 0 : 0 0 0 0
size-32768(DMA) 0 0 32768 0 0 8 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-32768 0 0 32768 0 0 8 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-16384(DMA) 0 0 16384 0 0 4 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-16384 50 51 16384 50 51 4 : 58 1894255 2533
2482 0 : 0 0 : 0 0 0 0
size-8192(DMA) 0 0 8192 0 0 2 : 0 0 0
0 0 : 0 0 : 0 0 0 0
size-8192 17 18 8192 17 18 2 : 45 39499 999
981 0 : 0 0 : 0 0 0 0
size-4096(DMA) 0 0 4096 0 0 1 : 0 0 0
0 0 : 60 30 : 0 0 0 0
size-4096 438 498 4096 438 498 1 : 683 7150152 77280
76782 0 : 60 30 : 12340730 391471 12419436 235115
size-2048(DMA) 2 2 2048 1 1 1 : 2 2 1
0 0 : 60 30 : 1 2 0 0
size-2048 760 760 2048 380 380 1 : 804 772396 7195
6815 0 : 60 30 : 2677371 40959 2685838 24607
size-1024(DMA) 0 0 1024 0 0 1 : 0 0 0
0 0 : 124 62 : 0 0 0 0
size-1024 480 480 1024 120 120 1 : 524 43435 616
496 0 : 124 62 : 54523 2676 56215 0
size-512(DMA) 0 0 512 0 0 1 : 0 0 0
0 0 : 124 62 : 0 0 0 0
size-512 656 656 512 82 82 1 : 680 295376 696
614 0 : 124 62 : 2197760 7471 2203070 960
size-256(DMA) 0 0 264 0 0 1 : 0 0 0
0 0 : 124 62 : 0 0 0 0
size-256 210 210 264 14 14 1 : 345 257909 55
41 0 : 124 62 : 323397 4409 327510 164
size-128(DMA) 4 28 136 1 1 1 : 28 28 1
0 0 : 252 126 : 3 2 0 0
size-128 2078 2100 136 75 75 1 : 2100 711569 75
0 0 : 252 126 : 4024428 5833 4028521 150
size-64(DMA) 0 0 72 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
size-64 832 1166 72 22 22 1 : 1643 156683 35
13 0 : 252 126 : 21280 1444 22116 1
size-32(DMA) 0 0 40 0 0 1 : 0 0 0
0 0 : 252 126 : 0 0 0 0
size-32 4934 5612 40 61 61 1 : 7360 554828 127
66 0 : 252 126 : 451465 4627 451193 152
|