nathans wrote:
> [...]
> this server process would not need to run pmlogconf/pmieconf, I think.
Considering kenj's problems, I believe that running pm*conf from the
cron FOO_check.sh is not a good idea after all (and have a patch in
pcpfans.git fche/dev to take that part back out). So that suggests
that the new server should run pm*conf and hand off finished files to
whoever the agent whereby our erring fellow-creatures are deprived of
that liberty which is so dear to us all ... oops, rewind ... the agent
that watches over pmlogger processes.
> It'd just update the control file(s) and the crontab-driven existing
> pm{ie,logger}_check functionality takes it from there. With that
> control.d addition, it'd just be creating a one-line file for each
> new host found, in the /etc/pcp/{pmie,pmlogger}/control.d directory.
There are at least two problems with this scheme.
First, the _check* scripts run too infrequently. For a machine that
comes up, we'd like to start logging it within (say) seconds, rather
than up to 30 minutes. (This could be worked around by hand-invoking
the _check* routine upon the arrival of new hosts, though then we have
a lot more cpu consumption, and a lot more busy-work checking on other
pmloggers.)
Second, there is nothing that handles the disappearance of remote
nodes, or equivalently, a sysadmin commenting out lines in
pm{logger|ie}/config.default. The _check* scripts may notice them but
don't consider it their problem to kill them.
Overall, the _check scripts don't seem to be a very good fit for
robust and rapid management of pmlogger/pmie lifecycles. It seems
like we'd need a baby init(8) or systemd(8).
(While I was learning about this part, I was also struck by the
perhaps heavy-handed way in which pmlogger_daily* / pmlognew work.
Every day, they track, snare, and shoot down pmloggers, move/compress
files, create some new configuration for them, then restart pmloggers.
In the mean time, event or performance data can get lost... Please
let's fix this, e.g., to be based on signals to long-lived pmloggers.)
- FChE
|