pcp
[Top] [All Lists]

Re: [pcp] PCP question ... leading to pmimport discussion

To: "kenj@xxxxxxxxxxxxxxxx" <kenj@xxxxxxxxxxxxxxxx>
Subject: Re: [pcp] PCP question ... leading to pmimport discussion
From: Greg Banks <gnb@xxxxxxxxxxx>
Date: Tue, 6 Jul 2010 12:10:37 +1000
Cc: "nathans@xxxxxxxxxx" <nathans@xxxxxxxxxx>, "pmatousu@xxxxxxxxxxxxx" <pmatousu@xxxxxxxxxxxxx>, "pcp@xxxxxxxxxxx" <pcp@xxxxxxxxxxx>
In-reply-to: <1278317711.21023.72.camel@xxxxxxxxxxxxxxxx>
References: <1619637932.410411277773130222.JavaMail.root@xxxxxxxxxxxxxxxxxx> <1278024574.2198.1.camel@xxxxxxxxxxxxxxxx> <4C2D53C7.6070300@xxxxxxxxxxx> <1278102403.10129.12.camel@xxxxxxxxxxxxxxxx> <4C3135E2.4040900@xxxxxxxxxxx> <1278317711.21023.72.camel@xxxxxxxxxxxxxxxx>
User-agent: Thunderbird 2.0.0.23 (X11/20090817)
Ken McDonell wrote:

OK.  pmiSetArchive() has been replaced by pmiStart(char *archive) since
we now have a mandatory call to pmiStart() for each context (formerly
called streams) and this needs an archive basename that cannot be
defaulted.

The hostname and timezone will default to the local host, but can be
explicitly set after pmiStart() by calling pmiSetHostname() or
pmiSetTimezone().

Sounds good.
I think it would be a good idea to carefully define and document the semantics of calling pmiChangeStream() while partway through a sequence of pmiPutValue() calls, i.e. what happens to the partly constructed pmResult.

It is preserved ... the partial pmResult state is maintained per context
and kept until pmiWrite() is called.

Which leads me to another thought where we may have a disconnect ... I
was assuming the set of metrics and instance domains and instances was
private per context.  I realize that this may not be what you're
expecting.
[...]
Thoughts?

I think it's quite likely that the set of instances will be different between different contexts, even if the metrics remain the same; imagine a metric whose instance domain is the disks on a system. As for metrics and indoms, the most general assumption would be to make them per-context, but reduces convenience in more common cases. Tough call. Perhaps you can compromise and allow the metadata to be either global or per-context at the application writer's choice?

One way might be to have pmiStart() inherit any metadata from the context which is current when it's called. This is slightly more useful if it's possible to create a context which has no associated file but just holds metadata, e.g. by passing a NULL archive name to pmiStart(); and if pmiUse() can be made to deliberately invalidate the current context.

With that arrangement, the app writer can choose to have identical metadata by doing:

metadata = pmiStart(NULL) // create a context whose only purpose is to hold
pmiAddMetric("foo.bar", ...)            // metrics and instances
pmiAddInstance(123, "mumble", 2)
pmiAddInstance(123, "fratz", 3)
one = pmiStart("one") // `one' copies metrics and instances from `metadata'
pmiUse(metadata)
two = pmiStart("two")                   // `two' ditto
pmiUse(one)
pmiAddValue("foo.bar", "mumble", ...)
pmiUse(two)
pmiAddValue("foo.bar", "mumble", ...)

or she may instead choose to have completely diffrerent metadata by doing:

pmiUse(-1) // ensure there is no current context one = pmiStart("one") // no current context, so `one' starts with
                                       // no metrics and no instances
pmiAddMetric("foo.bar", ...)
pmiAddInstance(123, "mumble", 2)
pmiAddInstance(123, "fratz", 3) // these metrics and instances are for `one' only
pmiUse(-1)
two = pmiStart("two") // `two' also starts with no metrics or instances
pmiAddMetric("baz.quux", ...)
pmiAddInstance(123, "fnarp", 2) // these metrics and instances are for `two' only
pmiUse(one)
pmiAddValue("foo.bar", "mumble", ...)
pmiUse(two)
pmiAddValue("baz.quux", "fnarp", ...)

or she may instead choose to have identical metrics but different instances by doing:

metadata = pmiStart(NULL)
pmiAddMetric("foo.bar", ...) // we want to use this metric for all contexts one = pmiStart("one") // `one' copies metrics and (no) instances from `metadata'
pmiAddInstance(123, "mumble", 2)
pmiAddInstance(123, "fratz", 3) // these instances go into `one' only
pmiUse(metadata)
two = pmiStart("two") // `two' copies metrics and (no) instances from `metadata' pmiAddInstance(123, "mumble", 2) // this instance goes into `two' only
pmiUse(one)
pmiAddValue("foo.bar", "mumble", ...)
pmiUse(two)
pmiAddValue("foo.bar", "mumble", ...)

For complete generality you might also need a call to forget all the metadata from the current context.

Keeping the partial pmResults per-context makes sense to me. That way an app can build a single time slice across multiple archives breadth-first, which may be the most convenient way if the input data is organised like that.
...
    $import = PCP::IMPORT->new('');
I don't see any point to having this object; if I understand the API above it will have no state and just serves as a subroutine scoping mechanism.
There is a heap of state that has to be held across calls to
libpcp_import routines (metadata, partial pmResult, ...).
Sure, but all that state is in the C library right?

Correct, so I'll just lose the object abstraction.  I assume trimming
the pmi prefix off the routine names for the Perl wrapper routines is a
bad idea, as this leads to potential name clashes with names that are
not prefixed PCP::IMPORT::, e.g. Start(), End(), Write() etc.  Is there
a convention to follow here?  My initial though was pmiFooBar() ->
pmi_foo_bar() for the Perl wrapper, but I really don't care so if there
some preferred naming regime point me in the right direction and I'll
follow that.

I'm not really a Perl expert, but if I understand the Exporter behaviour correctly then your concern about name clashes is well founded. A brief perusal of the Perl modules on this system shows examples with names like fooBar() and even FooBar(), sometimes in order to reflect the underlying C names. So you could use the C names in Perl without mapping, and just throw them all into a module PCP::Import, so app code looks like


use PCP:Import;
my $metadata = pmiStart();              # archive name = undef
pmiAddMetric("foo.bar", ...);
pmiAddInstance(123, "mumble", 2);
pmiAddInstance(123, "fratz", 3);
my $one = pmiStart("one");
pmiUse($metadata);
my $two = pmiStart("two");
pmiUse($one);
pmiAddValue("foo.bar", "mumble", ...);
pmiUse($two);
pmiAddValue("foo.bar", "mumble", ...) ;

--
Greg.

<Prev in Thread] Current Thread [Next in Thread>