Hi,
FWIW, FYI, I wrote a custom PMDA once which had to use a shell script to
collect the results and the shell script took something between 1-5
seconds to finish being too much for pmcd, below was my threading
solution - as you can see, it's extremely simple. Basically create a
thread to the background to collect updates, the main thread provides
results instantly to pmcd (from a previous update). No locking (but
would be simple to add) as it wouldn't really matter if the result is
from the previous or current update and if someone fetches for metrics
more often than every 30 seconds (we didn't need new metrics more
often), the previous results would be returned without creating a new
thread at all.
--- a/custom/pmdacustom.pl
+++ b/custom/pmdacustom.pl
@@ -17,6 +17,8 @@ use warnings;
use PCP::PMDA;
use Scalar::Util qw(looks_like_number);
use POSIX;
+use threads;
+use threads::shared;
my $pmda_id = 499;
my $pmda_name = 'custom';
@@ -25,7 +27,7 @@ my $query = '/var/lib/pcp/pmdas/custom/custom-pcp.sh';
my $query_ival = 30; # seconds
our $pmda;
-our %metrics;
+our %metrics :shared;
# Configuration files for overriding the above settings
for my $file (pmda_config('PCP_PMDAS_DIR') .
"/$pmda_name/$pmda_name.conf", "./$pmda_name.conf") {
@@ -155,15 +157,21 @@ sub custom_process_results {
}
}
+sub custom_read_data {
+ my $q_res = `$query`; # ignore stderr
+ my %f_res = custom_format_results($q_res);
+ sleep 10; # allow current fetches to finish
+ custom_process_results(%f_res);
+}
+
sub custom_fetch {
if ((strftime("%s", localtime()) - strftime("%s", @ts)) <
$query_ival) {
return;
}
@ts = localtime();
- my $q_res = `$query`; # ignore stderr
- my %f_res = custom_format_results($q_res);
- custom_process_results(%f_res);
+ my $thr = threads->create(\&custom_read_data);
+ $thr->detach();
}
sub custom_fetch_callback {
Cheers,
On 2015-09-11 17:16, Martins Innus wrote:
> Hi,
>
> Here is a proposed pmda for the Slurm scheduler
> (http://slurm.schedmd.com/). Its gone through several iterations,
> mostly because the Slurm api can take a long time to return information
> on a heavily loaded system and this was causing the pmda to be killed by
> pmcd. I'm new to perl threading, but this seems to be working fairly
> well so far. I implemented a very basic stub of the Slurm api to allow
> QA to run anywhere.
>
> For now it just shows information on jobs that are running on the node,
> my plan is to add more metrics in the future. There are a few stubs and
> comments to that affect in the code, mostly as reminders to myself. Let
> me know if you want those removed.
>
> Only tested live on a Centos 7 system since thats the only thing we have
> Slurm running on.
>
> https://github.com/ubccr/pcp/tree/pmdaslurm
>
> Thanks
>
> Martins
>
>
> commit 129bfc5fd788d747c5ec2acfa70cd51900a55a60
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Fri Sep 11 13:59:05 2015 +0000
>
> pmdaslurm : add QA out file
>
> qa/1031.out | 34 ++++++++++++++++++++++++++++++++++
> 1 file changed, 34 insertions(+)
>
> commit baf0ffa73afab365cbb7872554a5958cca88cc3c
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Fri Sep 11 13:37:58 2015 +0000
>
> pmdaslurm : Add a missing Remove file
>
> src/pmdas/slurm/Remove | 24 ++++++++++++++++++++++++
> 1 file changed, 24 insertions(+)
>
> commit 628605e8a179a2c2339c59f58a4a8cfe71ae278d
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Fri Sep 11 13:26:31 2015 +0000
>
> QA for pmda slurm
>
> qa/1031 | 62 ++++++++++++++++
> qa/GNUmakefile | 2 +-
> qa/GNUmakefile.install | 2 +-
> qa/group | 2 +
> qa/slurm/GNUmakefile | 21 ++++++
> qa/slurm/GNUmakefile.install | 21 ++++++
> qa/slurm/Slurm.pm | 141
> +++++++++++++++++++++++++++++++++++++
> qa/slurm/Slurm/GNUmakefile | 17 +++++
> qa/slurm/Slurm/GNUmakefile.install | 19 +++++
> qa/slurm/Slurm/Hostlist.pm | 21 ++++++
> src/pmdas/slurm/pmdaslurm.pl | 3 -
> 11 files changed, 306 insertions(+), 5 deletions(-)
>
> commit 2d5a8c47c0ce2cfe693d2868315734a9c6a47b25
> Merge: ebd77c6 cd69480
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Tue Sep 8 14:24:54 2015 -0400
>
> Merge branch 'master' into pmdaslurm
>
> commit ebd77c6baa0b18f2fd28731177728de425f0a661
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Wed Aug 12 13:16:38 2015 -0400
>
> pmdaslurm : Reduce memory use
>
> Only keep track of the jobs that are running in the current host
>
> src/pmdas/slurm/pmdaslurm.pl | 190
> ++++++++++---------------------------------
> 1 file changed, 44 insertions(+), 146 deletions(-)
>
> commit a20cb883f0ad60f4dc86e42dd71042b86c937613
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Fri Jul 17 18:47:42 2015 +0000
>
> pmdaslurm : add build to spec file
>
> build/rpm/pcp.spec.in | 25 +++++++++++++++++++++++--
> 1 file changed, 23 insertions(+), 2 deletions(-)
>
> commit bd480889adf256798437ae48ad42b2691e349ea0
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Thu Jul 9 16:06:14 2015 -0400
>
> Use timestamp in slurm_load_jobs to reduce the number of queries
>
> src/pmdas/slurm/pmdaslurm.pl | 20 +++++++++++++-------
> 1 file changed, 13 insertions(+), 7 deletions(-)
>
> commit 20361d45d97d30c8b822d1a8de0e8eb948a3b549
> Merge: 8a64111 fec6f94
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Mon Jul 6 16:11:40 2015 -0400
>
> Merge branch 'master' into pmdaslurm
>
> commit 8a64111ed602c8b36eb3502dce9f16c4b59b2f34
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Wed Jun 10 20:10:08 2015 +0000
>
> pmdaslurm : remove a debug config
>
> src/pmdas/slurm/pmdaslurm.pl | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> commit cf37c2a8c094ac3dfccdb76e8ae7994e4b3fc378
> Merge: b1eceeb b493149
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Wed Jun 10 15:15:55 2015 -0400
>
> Merge branch 'master' into pmdaslurm
>
> commit b1eceeb742cb4d0ba55e0694e25507626b0ce7a3
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Wed Jun 3 13:55:50 2015 -0400
>
> pmda slurm : don't die if slurm controller is not up
>
> src/pmdas/slurm/pmdaslurm.pl | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> commit ce90ee5ca0933744df5c77594559b35c955fbe7c
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Thu May 21 15:39:42 2015 -0400
>
> First cut at multi-threading the slurm pmda
>
> src/pmdas/slurm/pmdaslurm.pl | 185
> ++++++++++++++++++++++++++++++-------------
> 1 file changed, 132 insertions(+), 53 deletions(-)
>
> commit 38c6ea99e773edbf51b52f92e474873eddd4b8f8
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Tue Apr 21 15:57:03 2015 -0400
>
> pmdaslurm - more hostname fixes
>
> src/pmdas/slurm/pmdaslurm.pl | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> commit 494534adb9a3fa8b6f1ae68890b9efb9af6894b0
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Tue Apr 21 19:53:06 2015 +0000
>
> pmdaslurm - fix hostname lookup
>
> src/pmdas/slurm/pmdaslurm.pl | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> commit 9aae2d6b2d41fc4fbbb172e778038fb99597cc25
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Tue Apr 21 19:00:15 2015 +0000
>
> pmdaslurm - fix Install options
>
> src/pmdas/slurm/Install | 1 +
> 1 file changed, 1 insertion(+)
>
> commit 1dff4dcd3cc118063bba3a76121a9044b658c9e6
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Thu Apr 16 14:08:46 2015 -0400
>
> pmdaslurm add build config
>
> src/pmdas/GNUmakefile | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> commit da096205daa024cb9f00207884673e2fb4f24d8e
> Author: Martins Innus <minnus@xxxxxxxxxxx>
> Date: Thu Apr 16 14:04:24 2015 -0400
>
> Initial pmdaslurm checkin
>
> Uses the slurm perlapi component to provide information
> on jobs, nodes, and users in a slurm cluster
>
> src/pmdas/slurm/GNUmakefile | 45 +++++
> src/pmdas/slurm/Install | 27 +++
> src/pmdas/slurm/pmdaslurm.1 | 57 +++++++
> src/pmdas/slurm/pmdaslurm.pl | 398
> +++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 527 insertions(+)
>
> _______________________________________________
> pcp mailing list
> pcp@xxxxxxxxxxx
> http://oss.sgi.com/mailman/listinfo/pcp
--
Marko Myllynen
|