Hello,
Recently I noticed that when I simultaneously do 'up' to many network
interfaces
(many is ~15) netlink drops part of the messages about interface state change
and thus
my userspace tools don't know that some interfaces are in up state now. The
error that
I get from netlink socket is "No buffer space available".
After looking at the code I saw that the only way I can get such error from
netlink
is if sk->rmem_allock is bigger than sk->rcvbuf. I can enlarge sk->rcvbuf, but
for each
interface I receive six messages and each of this messages is smaller then 200
bytes.
the default size of sk->rcvbuf is 65535 bytes, so why messages about 15
interfaces can't
fit in default buffer size?
So I looked at rtmsg_ifinfo function. We allocate skb of size NLMSG_GOODSIZE
there
(NLMSG_GOODSIZE appears to be one page size), fill only ~200 bytes and broadcast
the message to all netlink sockets that should receive it. When we actually
deliver
skb to the socket we add skb->truesize (4096 bytes) to sk->rmem_allock and not
the
size of the actual message (200 byte). So the number of messages that can be in
sk->receive_queue simultaneously is about 16 only!
Now, I understand that we have to use skb->truesize for accounting and not
skb->len, but
waste 4000 bytes for each NEWLINK message is to much IMO. I see two solutions
to the problem:
First is to define NLMSG_GOODSIZE to something more reasonable (small) and
second to aggregate
many small messages to one big multipart message and store it in one skb. Only
when the skb is
full we will add another skb to the receive_queue.
I've implemented second approach for netlink_broadcast_deliver just to explain
what I mean (see
attached patch against 2.4.0). The same thing can be done in netlink_unicast
too.
Is there other way to avoid such waste of space in netlink socket's rcvbuf?
Comments are welcom!
--
Gleb.
patch
Description: Text document
|