bert hubert wrote:
On Mon, Feb 21, 2005 at 01:35:33AM +0100, Christian Schmid wrote:
New connections get made without any problems. Just existing connections
slow down painfully.
Incoming our outgoing data mostly?
Outgoing data. I am using sendfile() to send the file on a non-blocking socket but the call blocks
for 100 ms per socket if I get over 3000 sockets. Thats causing the massive slowdown in sum. I first
thought its a disk-issue but I tried with pure-cache data as well and it still blocks.
3000 sockets = no slowdown at all (500 MBit in use)
3300 sockets = 10% slowdown
3600 sockets = 30% slowdown
4000 sockets = 60% slowdown (i aborted here, as it only uses 200 MBit for
sending... catastrophy!)
They are all receiving data. Its a download-service. receive-buffer is set
to 24 KB and send-buffer set to 224 KB. I don't see a problem with
port-space. I only have 3500 sockets when the problem appears but it
appears suddenly.
I'm a bit confused, it is a download service so you are probably *sending*
data?
Only sending. Receiving ACKs of course.
But it would help if you looked at the stats and ifconfig
to see who's dropping packets, how many retransmissions there
are, memory failures, or the bottleneck is some other issue altogether...
No way. Doing 30000 packets per second and your stats are 32 bit integers ;)
So? You could be a bit more helpful. Sample over 5 seconds, you won't
overflow that.
5 seconds, here you go:
eth0 Link encap:Ethernet HWaddr 00:30:48:27:84:28
inet addr:80.237.244.12 Bcast:80.237.244.63 Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2861390424 errors:5991038 dropped:5991056 overruns:38681
frame:4294967287
TX packets:2906616171 errors:4294967279 dropped:0 overruns:0
carrier:4294967278
collisions:4294967289 txqueuelen:1000
RX bytes:1613084024 (1538.3 Mb) TX bytes:2528808392 (2411.6 Mb)
Base address:0x4000 Memory:fc000000-fc020000
eth0 Link encap:Ethernet HWaddr 00:30:48:27:84:28
inet addr:80.237.244.12 Bcast:80.237.244.63 Mask:255.255.255.192
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:2861511619 errors:5991038 dropped:5991056 overruns:38681
frame:4294967287
TX packets:2906804392 errors:4294967279 dropped:0 overruns:0
carrier:4294967278
collisions:4294967289 txqueuelen:1000
RX bytes:1625289307 (1549.9 Mb) TX bytes:2802520805 (2672.6 Mb)
Base address:0x4000 Memory:fc000000-fc020000
Also, can you tcpdump a bit? Are you using iptables? The conntrack table
might be slowing you down.
Hmmm tcpdump what exactly? I am using iptables and the conntrack-problem has been solved in the past
already by disabling conn-tracking. The table overran and it dropped packets massively. I disabled
conn-tracking and the problem was gone. This is a different problem though.
I have a hunch this problem has to do with high-mem issues though.
Well, 6 GB of high-mem, 2 GB of low-mem... at least there is not too less
memory.
MemTotal: 8314392 kB
MemFree: 13936 kB
Buffers: 28732 kB
Cached: 7926792 kB
SwapCached: 0 kB
Active: 3129028 kB
Inactive: 5031240 kB
HighTotal: 6421952 kB
HighFree: 640 kB
LowTotal: 1892440 kB
LowFree: 13296 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 54160 kB
Writeback: 0 kB
Mapped: 210108 kB
Slab: 128176 kB
CommitLimit: 4157196 kB
Committed_AS: 674072 kB
PageTables: 1604 kB
VmallocTotal: 114680 kB
VmallocUsed: 1160 kB
VmallocChunk: 113416 kB
Best regards,
Chris
|