pcp
[Top] [All Lists]

Re: [pcp] Patch for Python's pmExtractValue Function

To: Nathan Scott <nathans@xxxxxxxxxx>, Stan Cox <scox@xxxxxxxxxx>
Subject: Re: [pcp] Patch for Python's pmExtractValue Function
From: Tom Yearke <tyearke@xxxxxxxxxxx>
Date: Mon, 14 Apr 2014 00:29:28 -0400
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <235383721.4734118.1397439752269.JavaMail.zimbra@xxxxxxxxxx>
References: <53342449.7070602@xxxxxxxxxxx> <53446B81.4020602@xxxxxxxxxx> <235383721.4734118.1397439752269.JavaMail.zimbra@xxxxxxxxxx>
User-agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0
Sorry about the delay in getting back to you all. I've had a number of other issues come up over the last few days that have prevented me from more fully investigating the issue, which I had hoped to do before replying but have not yet.

Pointing to a Python string (instead of a C string instantiated by Python) works for my purposes. The only side effect this introduces, I believe, is that the string buffer pointed to is now an immutable one, but I can't imagine anyone's Python scripts involve modifying the string buffer that a pmAtomValue points to. Unless someone's doing some low-level manipulation with ctypes, I like this solution better than my own, since it avoids some expensive string copying.

The only issue that I'm concerned with in this new solution is that, as in my first patch, the value of outAtom.vp is being passed to LIBC.free. In later testing of my first solution, I found that this method of freeing the C string led to seg faults when multiple processes using large amounts of memory were run, suggesting something isn't translated properly when 64-bit memory addresses are in use. The issue was resolved by using a ctypes char pointer (obtained by doing c_char_p.from_buffer_copy(outAtom, pmAtomValue.cp.offset)) instead of a Python integer (obtained by calling outAtom.vp). This is where the compatibility issue mentioned comes in - from_buffer_copy is only available in Python 2.6+. From what I recall in investigating this fix, I believe from_buffer_copy may be a convenience function for a series of functions available in 2.5 to accomplish the same task. I have not yet found what that series of functions was, but perhaps the ctypes source code contains it.

I am hoping to finally get back to this issue in the morning, my timezone, but if you'd like to take a look in the meantime, I would welcome it. I'm afraid I don't have a script that's not tightly coupled to other resources to share, but calling pmExtractValue with a pmResult for a string metric, followed by losing the reference to the returned pmAtomValue, will create the leak. The example code at the top of pmapi.py should be a good starting point.

Tom

On 4/13/2014 9:42 PM, Nathan Scott wrote:
Hi guys,

----- Original Message -----
I am not absolutely certain about the Python 2.6+ availability
requirement so I think this will do things in a similar 2.5 supported
way.  Does this work for you?

We're a day or two out from a release, any help I can offer here?
Do we have a reproducible test case I can try out, demonstrating
the problem?  (perhaps a little py script I can use, and then add
a bit of valgrind sauce via /usr/bin/python to observe the leak?)

thanks!

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>