On Thu, 2011-03-10 at 08:57 +1100, Ken McDonell wrote:
> On Wed, 2011-03-09 at 14:51 -0600, David Smith wrote:
>
> > Thanks. What's checked in now handles a single logfile and multiple
> > consumers. Now I'm expanding it to handle multiple logfiles (via
> > instances) and multiple clients.
> >
>
> OK, with instances we may be heading for a problem in paradise ...
> implicit in everything I've done so far is an assumption that event
> record metrics are singular (only one instance, namely PM_IN_NULL) for
> each metric.
>
> I ripped all of the instance stuff out of pmval in making pmevent. This
> was when I realized that numval == 1 is required for each of
> __pmDumpEventRecords, __pmCheckEventRecords, pmUnpackEventRecords in
> libpcp.
>
> I had been assuming that rather than an instance domain, dynamic metrics
> could be used to export multiple sets of event records from a single
> PMDA, but for your logfile dredging PMDA this probably won't work well.
>
> If there is a genuine use case where event records exported for a single
> metric with multiple instances is needed, we need to circle back and
> reconsider the assumptions in the libpcp code.
>
> I've expanded this mail to include the broader list to solicit feedback.
Further to my earlier mail, I think we should consider event records to
belong to a "stream", where a stream may be a logfile (in David's case),
or from a process/thread, or a session, or an event trace for an
operation, or ...
The options we have for a PMDA exporting multiple streams appear to be
as follows.
1. Export one metric per stream, using the dynamic metric
services to augment the namespace as needed. For example
logger.var.log.messages
[record]
logger.param_string "some messages text"
logger.var.log.auth.log
[record]
logger.param_string "some auth text"
2. Export one metric, with the stream id as the first parameter
of each event record. For example
logger.events
[record]
logger.stream "/var/log/messages"
logger.param_string "some messages text"
[record]
logger.stream "/var/log/auth.log"
logger.param_string "some auth text"
3. Export one metric with one instance per stream. For example
logger.event["/var/log/messages"]
[record]
logger.param_string "some messages text"
logger.event["/var/log/auth.log"]
[record]
logger.param_string "some auth text"
I suspect 2. may be the most commonly used encoding for a PMDA exporting
multiple event stream.
1. and 2. work today. But 3. is not supported at the moment.
So the discussion needs to be "should we support 3. as well?"
|