netdev
[Top] [All Lists]

Re: select says I can read, but recvfrom hangs

To: Andi Kleen <ak@xxxxxx>
Subject: Re: select says I can read, but recvfrom hangs
From: "D. Hugh Redelmeier" <hugh@xxxxxxxxxx>
Date: Thu, 21 Jun 2001 15:42:24 -0400 (EDT)
Cc: <netdev@xxxxxxxxxxx>
In-reply-to: <20010621182650.50332@colin.muc.de>
Reply-to: <hugh@xxxxxxxxxx>
Sender: owner-netdev@xxxxxxxxxxx
| From: Andi Kleen <ak@xxxxxx>

| It is a hint when multiple processes access the same socket.

Yes.  In this case there is only one process.

| When that's
| not the case it would be a kernel bug. Because no such bugs are known
| (and such things tend to get noticed) I would suspect the freeswan kernel
| patches.

Fair enough.  That must remain as a possibility, but I don't think
that it is the case.

Another experiment has shed some more light.  I had the user eliminate
the Pluto code to exploit IP_RECVERR / MSG_ERRQUEUE.  No more hangs.

This means that the Pluto support for IP_RECVERR / MSG_ERRQUEUE
provokes the problem.  Either the Pluto code is wrong or there is a
bug in the feature.  From past experience, it is most likely that I've
misunderstood how this is to be done.

- the select says that there is something to be read from the file
  descriptor (socket).  Remember that the socket has the option
  IP_RECVERR set:
        setsockopt(fd, SOL_IP, IP_RECVERR, (const void *)&on, sizeof(on))

- the recvfrom that hangs looks like:
        packet_len = recvfrom(ifp->fd, bigbuffer, sizeof(bigbuffer), 0
            , &from.sa, &from_len);
  Note the "0" for flags.

- if the packet_len is set to -1, an attempt is made to read from the
  MSG_ERRQUEUE:

            /* we are going to be daring: we'll try to use information
             * passed on because of IP_RECVERR.
             * The API is sparsely documented, and may be LINUX-only.
             *
             * - ip(7) describes IP_RECVERR
             * - recvmsg(2) describes MSG_ERRQUEUE
             * - readv(2) describes iovec
             * - cmsg(3) describes how to process auxilliary messages
             *
             * ??? we should link this message with one we've sent
             * so that the diagnostic can refer to that negotiation.
             */
#if defined(IP_RECVERR) && defined(MSG_ERRQUEUE)
            {
                struct msghdr emh;
                struct iovec eiov;
                union {
                    /* force alignment (not documented as necessary) */
                    struct cmsghdr ecms;

                    /* how much space is enough? */
                    unsigned char space[256];
                } ecms_buf;
                struct cmsghdr *cm;
                char fromstr[INET6_ADDRSTRLEN + sizeof(" port 65536")];

                zero(&from.sa);
                from_len = sizeof(from);

                emh.msg_name = &from.sa;        /* ??? filled in? */
                emh.msg_namelen = sizeof(from);
                emh.msg_iov = &eiov;
                emh.msg_iovlen = 1;
                emh.msg_control = &ecms_buf;
                emh.msg_controllen = sizeof(ecms_buf);
                emh.msg_flags = 0;

                eiov.iov_base = bigbuffer;      /* see readv(2) */
                eiov.iov_len = sizeof(bigbuffer);

                packet_len = recvmsg(ifp->fd, &emh, MSG_ERRQUEUE);
        ...

This code assumes that for every error return from the recvfrom either
there is a MSG_ERRQUEUE message, or at least it is safe to try to read
one -- it won't block.  As far as I can tell, this has never blocked.
But the number of errors isn't high, so testing hasn't been intense.

Is it the case that a queued MSG_ERRQUEUE message will cause select to
say that there is something to read?  I'd expect so.

This code assumes that if there is a queued MSG_ERRQUEUE message, an
attempt to recvfrom with flags = 0 will not hang, but instead produce
an error return.  Is this wrong?  If it is wrong, it contradicts my
understanding of an answer that Andi gave me last fall.  This could
explain the hang that we are observing.

I don't see where this last question is clearly answered in the
documentation.

Can you see anything else suspicious in this code?

Hugh Redelmeier
hugh@xxxxxxxxxx  voice: +1 416 482-8253


<Prev in Thread] Current Thread [Next in Thread>