pmdaslurm

[Martins Innus <minnus@xxxxxxxxxxx>]

Hi,

    Here is a proposed pmda for the Slurm scheduler 
(http://slurm.schedmd.com/).  Its gone through several iterations, 
mostly because the Slurm api can take a long time to return information 
on a heavily loaded system and this was causing the pmda to be killed by 
pmcd.  I'm new to perl threading, but this seems to be working fairly 
well so far.  I implemented a very basic stub of the Slurm api to allow 
QA to run anywhere.
For now it just shows information on jobs that are running on the node, 
my plan is to add more metrics in the future.  There are a few stubs and 
comments to that affect in the code, mostly as reminders to myself.  Let 
me know if you want those removed.
Only tested live on a Centos 7 system since thats the only thing we have 
Slurm running on.
https://github.com/ubccr/pcp/tree/pmdaslurm

Thanks

Martins


commit 129bfc5fd788d747c5ec2acfa70cd51900a55a60
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Fri Sep 11 13:59:05 2015 +0000

    pmdaslurm : add QA out file

 qa/1031.out | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

commit baf0ffa73afab365cbb7872554a5958cca88cc3c
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Fri Sep 11 13:37:58 2015 +0000

    pmdaslurm : Add a missing Remove file

 src/pmdas/slurm/Remove | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

commit 628605e8a179a2c2339c59f58a4a8cfe71ae278d
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Fri Sep 11 13:26:31 2015 +0000

    QA for pmda slurm

 qa/1031                            |  62 ++++++++++++++++
 qa/GNUmakefile                     |   2 +-
 qa/GNUmakefile.install             |   2 +-
 qa/group                           |   2 +
 qa/slurm/GNUmakefile               |  21 ++++++
 qa/slurm/GNUmakefile.install       |  21 ++++++
 qa/slurm/Slurm.pm                  | 141 
+++++++++++++++++++++++++++++++++++++
 qa/slurm/Slurm/GNUmakefile         |  17 +++++
 qa/slurm/Slurm/GNUmakefile.install |  19 +++++
 qa/slurm/Slurm/Hostlist.pm         |  21 ++++++
 src/pmdas/slurm/pmdaslurm.pl       |   3 -
 11 files changed, 306 insertions(+), 5 deletions(-)

commit 2d5a8c47c0ce2cfe693d2868315734a9c6a47b25
Merge: ebd77c6 cd69480
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Tue Sep 8 14:24:54 2015 -0400

    Merge branch 'master' into pmdaslurm

commit ebd77c6baa0b18f2fd28731177728de425f0a661
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Wed Aug 12 13:16:38 2015 -0400

    pmdaslurm : Reduce memory use

    Only keep track of the jobs that are running in the current host

 src/pmdas/slurm/pmdaslurm.pl | 190 
++++++++++---------------------------------
 1 file changed, 44 insertions(+), 146 deletions(-)

commit a20cb883f0ad60f4dc86e42dd71042b86c937613
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Fri Jul 17 18:47:42 2015 +0000

    pmdaslurm : add build to spec file

 build/rpm/pcp.spec.in | 25 +++++++++++++++++++++++--
 1 file changed, 23 insertions(+), 2 deletions(-)

commit bd480889adf256798437ae48ad42b2691e349ea0
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Thu Jul 9 16:06:14 2015 -0400

    Use timestamp in slurm_load_jobs to reduce the number of queries

 src/pmdas/slurm/pmdaslurm.pl | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

commit 20361d45d97d30c8b822d1a8de0e8eb948a3b549
Merge: 8a64111 fec6f94
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Mon Jul 6 16:11:40 2015 -0400

    Merge branch 'master' into pmdaslurm

commit 8a64111ed602c8b36eb3502dce9f16c4b59b2f34
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Wed Jun 10 20:10:08 2015 +0000

    pmdaslurm : remove a debug config

 src/pmdas/slurm/pmdaslurm.pl | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

commit cf37c2a8c094ac3dfccdb76e8ae7994e4b3fc378
Merge: b1eceeb b493149
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Wed Jun 10 15:15:55 2015 -0400

    Merge branch 'master' into pmdaslurm

commit b1eceeb742cb4d0ba55e0694e25507626b0ce7a3
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Wed Jun 3 13:55:50 2015 -0400

    pmda slurm : don't die if slurm controller is not up

 src/pmdas/slurm/pmdaslurm.pl | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

commit ce90ee5ca0933744df5c77594559b35c955fbe7c
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Thu May 21 15:39:42 2015 -0400

    First cut at multi-threading the slurm pmda

 src/pmdas/slurm/pmdaslurm.pl | 185 
++++++++++++++++++++++++++++++-------------
 1 file changed, 132 insertions(+), 53 deletions(-)

commit 38c6ea99e773edbf51b52f92e474873eddd4b8f8
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Tue Apr 21 15:57:03 2015 -0400

    pmdaslurm - more hostname fixes

 src/pmdas/slurm/pmdaslurm.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit 494534adb9a3fa8b6f1ae68890b9efb9af6894b0
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Tue Apr 21 19:53:06 2015 +0000

    pmdaslurm - fix hostname lookup

 src/pmdas/slurm/pmdaslurm.pl | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

commit 9aae2d6b2d41fc4fbbb172e778038fb99597cc25
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Tue Apr 21 19:00:15 2015 +0000

    pmdaslurm - fix Install options

 src/pmdas/slurm/Install | 1 +
 1 file changed, 1 insertion(+)

commit 1dff4dcd3cc118063bba3a76121a9044b658c9e6
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Thu Apr 16 14:08:46 2015 -0400

    pmdaslurm add build config

 src/pmdas/GNUmakefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

commit da096205daa024cb9f00207884673e2fb4f24d8e
Author: Martins Innus <minnus@xxxxxxxxxxx>
Date:   Thu Apr 16 14:04:24 2015 -0400

    Initial pmdaslurm checkin

    Uses the slurm perlapi component to provide information
    on jobs, nodes, and users in a slurm cluster

 src/pmdas/slurm/GNUmakefile  |  45 +++++
 src/pmdas/slurm/Install      |  27 +++
 src/pmdas/slurm/pmdaslurm.1  |  57 +++++++
 src/pmdas/slurm/pmdaslurm.pl | 398 
+++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 527 insertions(+)