On Tue, 5 Feb 2002, Andi Kleen wrote:
> When you claim something that fails the sanity checks of several other
> people and you blame Linux it is your duty to provide evidence for it.
> In TCP world the standard evidence is a tcpdump. tcpdumps don't lie.
> Without a tcpdump we can as well assume that the problem doesn't exist.
oh, all right :-)
attached.
sample:
blue is sending a request to xed for a packet. Supermon (on xed) in turn
sends status requests to several other nodes. Xed formats each "blob" of
data from the other nodes and sends it to blue. The 'S' to supermon
results in one byte of 'S' to the other nodes, and the data comes back
from the other nodes as a large S-expression (in other words the delays
you see in this tcpdump are NOT from the other nodes).
Here's one cycle. Blue sends 'S' (one byte) to 'xed', and 'xed' is sending
data back. As you can see, in the middle of xed sending lots of data back,
there is this fine ~40 ms. delay. This delay occurs for each transaction
at least once, and limits us to 25 hz. request processing. We also had
this problem before with the 'mon' programs on the nodes but have
developed a workaround, which we will also be applying to supermon.
Please note that this is a problem that seems to have come along with 2.4.
We are fixing user code that did not have this problem on 2.2. I have no
argument with your effort to improve ack behavior in 2.4. I just wish
you'd let me turn it off.
The only option we have turned on is TCP_NODELAY, but that has proven to
be pretty important for this application. However if you want me to use
different options I'm willing to try it.
BTW, I've done NFS over TCP once or twice (once in the SunOS kernel), and
this kind of behavior leaves me worried about what your TCP will do to NFS
if we ever try NFS over TCP on Linux.
1012942315.229586 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
P 2:3(1) ack 1534 win 8931 <nop,nop,timestamp 85699266 877445890> (DF)
1012942315.230563 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 1534:1618(84) ack 3 win 5792 <nop,nop,timestamp 877445891 85699266> (DF)
1012942315.268651 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 3:3(0) ack 1618 win 8931 <nop,nop,timestamp 85699270 877445891> (DF)
1012942315.268651 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 1618:2295(677) ack 3 win 5792 <nop,nop,timestamp 877445930 85699270>
(DF)
1012942315.268651 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 3:3(0) ack 2295 win 10305 <nop,nop,timestamp 85699270 877445930> (DF)
1012942315.268651 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 2295:3067(772) ack 3 win 5792 <nop,nop,timestamp 877445930 85699270>
(DF)
1012942315.268651 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 3:3(0) ack 3067 win 11580 <nop,nop,timestamp 85699270 877445930> (DF)
1012942315.268651 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
P 3:4(1) ack 3067 win 11580 <nop,nop,timestamp 85699270 877445930> (DF)
1012942315.269628 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 3067:3151(84) ack 4 win 5792 <nop,nop,timestamp 877445931 85699270> (DF)
1012942315.308693 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 4:4(0) ack 3151 win 11580 <nop,nop,timestamp 85699274 877445931> (DF)
1012942315.308693 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 3151:3828(677) ack 4 win 5792 <nop,nop,timestamp 877445971 85699274>
(DF)
1012942315.308693 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 4:4(0) ack 3828 win 13124 <nop,nop,timestamp 85699274 877445971> (DF)
1012942315.308693 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 3828:4139(311) ack 4 win 5792 <nop,nop,timestamp 877445971 85699274>
(DF)
1012942315.308693 eth0 < blue.acl.lanl.gov.32790 > xed.acl.lanl.gov.2709:
. 4:4(0) ack 4139 win 13124 <nop,nop,timestamp 85699274 877445971> (DF)
1012942315.308693 eth0 > xed.acl.lanl.gov.2709 > blue.acl.lanl.gov.32790:
P 4139:4600(461) ack 4 win 5792 <nop,nop,timestamp 877445971 85699274>
(DF)
ron
acack
Description: Text document
|