pcp
[Top] [All Lists]

Re: [pcp] Secure sockets builds have high daemon memory utilisation

To: Dave Brolley <brolley@xxxxxxxxxx>
Subject: Re: [pcp] Secure sockets builds have high daemon memory utilisation
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Thu, 14 Aug 2014 03:37:13 -0400 (EDT)
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <53EBCCEB.2030604@xxxxxxxxxx>
References: <1896810420.23212457.1402370819966.JavaMail.zimbra@xxxxxxxxxx> <5397573A.90102@xxxxxxxxxx> <53EBCCEB.2030604@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: NphA5Ca54+WqdNg8Oxyrr/jzrUNxeQ==
Thread-topic: Secure sockets builds have high daemon memory utilisation
Hi Dave,

----- Original Message -----
> This commit (brolley/dev in pcpfans) delays the initialization of the
> NSPR, NSS and SASL libraries until their respective functionalities are
> needed. As noted in the commit message, a significant amount of
> pre-connect and pre-accept NSPR socket code became obsolete and has been
> removed. Also, as I had hoped, the NSPR/native division in the IPC table
> is also gone.

Awesome, awesome, awesome - thanks Dave, and thanks for not accepting
mediocrity!  You mentioned elsewhere you were getting zeroes from your
pmap measures, I suspect that'd be because pmcd is running as pcp user
- you may need to run pmap as root?  I see zeroes if I don't do that,
so possibly that's the issue you're hitting there.

I've attached some before and after photos from my desktop for your
perusal anyway - we're back pretty close to where we were.  There's
another ~300K that could be gained (--without-secure-sockets builds
show this) - a dlopen on NSS/SSL would probably knock that off if we
were keen for more, and then the next biggest hitter is dbus (which
I'm guessing we pull in via Avahi?) at ~200K.  Worth keeping the
latter in mind if we start adding additional advertising/discovery
methods via exotic shlibs down the track - we could rethink that idea
of demand-loadable advertising/discovery mechanisms.

But, we're in a much happier place now I think (we should still reduce
that SSL cache size also I guess?, for those who do run with NSS/SSL -
100000s of connection cache entries just seems excessive for pmcd).

> There is one qa issue which affects tests 712 and 966. In the old
> implementation, servers would initialize NSS right away and, should that
> fail, were able to report that secure connections are not supported
> during the connection handshake with the first client to connect. With
> the new implementation, NSS initialization is not attempted until the
> first secure connection is actually requested (after the first
> connection handshake). Thus, when the first client connects, a server
> built --with-secure-sockets will report that secure connections are
> supported, even though that may ultimately turn out not to be true (e.g.
> NSS init fails because there is no server certificate). Once NSS init
> has failed, the server is then able to report that secure connections
> are not supported during subsequent connection handshakes.
> 
> The visible result is that the first client to connect may be told that
> secure connections are supported, only to get "connection reset by peer"
> when a secure connection is actually requested. With the old
> implementation, the first client to connect would be told during the
> handshake that secure connections are not supported and would be able to
> correctly report that to the user.
> 
> I'm not quite sure how to handle this. Suggestions please! One possible
> solution is to have clients try to connect a second time when they
> receive "connection reset by peer" the first time. During the second
> handshake, the server will correctly report that secure sockets are not
> supported. Another related solution might be to have the server report
> that the status of secure connection support is unknown. This could then
> be used as the trigger for a second connection attempt by the client
> should an initial secure connection attempt fail.
> 

Perhaps a new error code (as in "pmerr -l") could be introduced to give a
more meaningful message than ECONNRESET on that first failure?  ISTR we
end up with ECONNRESET in alot of cases via NSS/SSL which made debugging
quite tricky ... so distinguishing more cases would be good.  Relatively
simple then - no need for retries, protocol tweaking, etc.

> One other thing to note is that, while the entire socket I/O abstraction
> layer API remains intact, much of it (the pre-connect and pre-accept
> functions) now have only one implementation (the native one). Similarly,
> the types __pmFdSet, __pmSockAddr, __pmAddrInfo and __pmHostEnt now also
> have only the native implementation. It is tempting to consider removing
> the abstraction layer from around these functions and types, however,
> I'm leaning toward leaving them as they are:
> 
>   * they could become necessary again in the future should a different
>     implementation of secure sockets be needed
>   * they present a level of consistency to users of the socket I/O API
>     in that all aspects are presented as __pm* functions and types. This
>     eliminates the need for callers to know where the underlying
>     abstraction is actually applied, and where it is not.

*nod* ... leaving 'em there gets my vote.

cheers.

--
Nathan

Attachment: mem
Description: Binary data

<Prev in Thread] Current Thread [Next in Thread>