pcp
[Top] [All Lists]

pcp updates: pmdagfs2 updates, libpcp user/group acls, qa

To: pcp@xxxxxxxxxxx
Subject: pcp updates: pmdagfs2 updates, libpcp user/group acls, qa
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue, 4 Jun 2013 19:57:40 -0400 (EDT)
Cc: Paul Evans <pevans@xxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <170200043.18792001.1370390208377.JavaMail.root@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: 2iB/svS5Nj0uVr7tLZIMBjGNht0rhA==
Thread-topic: pcp updates: pmdagfs2 updates, libpcp user/group acls, qa
Changes committed to git://oss.sgi.com/pcp/pcp.git dev

 configure                            |   23 
 configure.in                         |    5 
 qa/051                               |   12 
 qa/051.out.1                         | 1015 -----------------------------------
 qa/051.out.4                         | 1005 ++++++++++++++++++++++++++++++++++
 qa/051.out.5                         | 1006 ++++++++++++++++++++++++++++++++++
 qa/449                               |    9 
 qa/451                               |   12 
 qa/470                               |   12 
 qa/472                               |   12 
 qa/473                               |   12 
 qa/474                               |   12 
 qa/475                               |   12 
 qa/654                               |  159 ++---
 qa/654.out                           |   66 ++
 qa/655                               |   80 ++
 qa/655.out                           |   70 ++
 qa/GNUmakefile                       |    2 
 qa/common.filter                     |    3 
 qa/common.gfs2                       |  134 ++++
 qa/group                             |    1 
 qa/src/chkacc1.c                     |   16 
 qa/src/chkacc2.c                     |   16 
 qa/src/chkacc3.c                     |    8 
 qa/src/multithread2.c                |    7 
 src/include/pcp/impl.h               |   21 
 src/include/pcp/platform_header.h.in |   11 
 src/libpcp/src/GNUmakefile           |    7 
 src/libpcp/src/access.c              |  727 ++++++++++++++++++++++---
 src/libpcp/src/accounts.c            |  445 +++++++++++++++
 src/libpcp/src/check-statics         |   14 
 src/libpcp/src/internal.h            |    6 
 src/libpcp/src/util.c                |  107 ---
 src/pmcd/src/client.c                |    2 
 src/pmcd/src/config.c                |  205 +++++--
 src/pmcd/src/pmcd.c                  |    5 
 src/pmdas/gfs2/GNUmakefile           |   11 
 src/pmdas/gfs2/Install               |   14 
 src/pmdas/gfs2/control.c             |  121 +++-
 src/pmdas/gfs2/control.h             |   31 +
 src/pmdas/gfs2/glocks.c              |    1 
 src/pmdas/gfs2/glocks.h              |    2 
 src/pmdas/gfs2/glstats.c             |  119 +++-
 src/pmdas/gfs2/glstats.h             |   57 +
 src/pmdas/gfs2/help                  |  271 ++++++---
 src/pmdas/gfs2/lock_time.c           |  940 +++++++++++++++++++++-----------
 src/pmdas/gfs2/lock_time.h           |  117 +++-
 src/pmdas/gfs2/pmda.c                |  270 +++++++--
 src/pmdas/gfs2/pmdagfs2.h            |   19 
 src/pmdas/gfs2/pmns                  |   36 +
 src/pmdas/gfs2/root                  |    9 
 src/pmdas/gfs2/sbstats.c             |   11 
 src/pmdas/gfs2/sbstats.h             |    2 
 src/pmlogger/ports.c                 |    4 
 54 files changed, 5327 insertions(+), 1967 deletions(-)

commit c515049b166e5dfb3d43967a835fa3dcfb7f75f7
Merge: c607456 71399c3
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Wed Jun 5 09:16:47 2013 +1000

    Merge branch 'dev' into gfs2

commit 71399c38a111a17368fe4bd7bcb0703f41085e12
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Wed Jun 5 09:07:08 2013 +1000

    Convert the multi-threaded tests to use runtime feature detection
    
    The multi-threaded tests were all using the presence of a macro in
    the installed pcp/impl.h to determine whether they should run.
    This is problematic in that the header may be out of date, or may
    not be present at all if the devel package not installed.
    
    Now that we have a libpcp runtime feature test mechanism, use it.

commit 744ecdac511f4905e8e8dd47f71c13af36208434
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Wed Jun 5 09:03:18 2013 +1000

    Laying the groundwork for user and group access control lists
    
    Core functionality for user and group additions to the existing host
    based access control mechanism.
    
    Adds configure checks for reentrant/otherwise pwent and grent APIs.
    Support in libpcp for adding users and groups to access lists, for
    dumping these lists (diagnostics), and for saving/restoring lists.
    All user/group functionality in libpcp shifted into new accounts.c
    source file which provides platform-specific implementations making
    use of the appropriate pwent/grent APIs for each platform.
    
    Extensions to the pmcd access control parser to add the new keyworks
    for host/user/group differentiation on each access line (defaults to
    host, for backward-compatibility).  Abstract the name parsing into a
    shared routine (ParseNames) used by all three of the ACL types.  And
    switch pmcd diagnostic interfaces to make it report on all types.
    
    QA test modifications allow for the additional diagnostics with user
    and group ACLs alongside hosts.

commit c6074566db75e6149eab7d7b57b1923cbaa126b8
Author: Paul Evans <pevans@xxxxxxxxxx>
Date:   Tue Jun 4 14:00:54 2013 +0100

    pmdagfs2: gfs2_store() related cleanup and fixes to QA tests qa/654 qa/654
    
    Removed the needless on-stack "value" local variable in gfs2_store() and
    all other related code belonging to it.
    
    Fixed both QA tests to exercise the pmda 654 runs and can now be passed on
    any machine that has the gfs2 modules through the use of loop-back mounted
    filesystems (expects no mounted gfs2 filesystem). 654.out has been updated
    in order to match the expected outcome of running this test.
    
    655 has been updated to be able to exercise the gfs2_glock_lock_time 
metrics.
    In order to simulate a correct run, the test machine requires gfs2 modules 
to
    be loaded, an install with support for the additional gfs2 tracepoints and
    additionally a filesystem must be loaded in a shared cluster environment
    configuration. Because a locally mounted filesystem will not work, 
additional
    checks for cman are made otherwise the test will not run. An expected 
655.out
    has been produced for this test.
    
    common.gfs2 has been updated to reflect the new changes made to both of the
    654 and 655 QA tests.

commit e7311d3443e3e5c4c5bf50751c4dfe950bb56f40
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Sat May 25 11:28:34 2013 +1000

    Minor qa test tidying - update comments, dup localconfig use

commit 9b97f90e2431607f0115b98e7920ac6c4ce35b2d
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri May 24 17:12:49 2013 +1000

    Bug fixes and refactoring in gfs2_refresh_lock_time
    
    The cache cull call on the lock_time_indom was being
    done inside the loop which accumulates glock stats;
    therefore whenever it takes multiple reads to consume
    the available glock stats, those from the previous
    reads would have been accidentally culled.
    
    One reason this was not immediately obvious, perhaps,
    was that the code to extract a single glock stat was
    interwoven within gfs2_refresh_lock_time - I've moved
    this out into its own function which clarifies things
    (to my eyes at least).
    
    It then also became clear that the "hash==NULL" check
    here can never fail, since hash is an on-stack buffer,
    so I simply removed that (dead) error-checking branch.
    
    Finally, this refactoring also makes the last buglet a
    bit clearer - we fopen a file (allocates and returns a
    FILE*), then extract the fd via fileno(3), but when we
    are cleaning up at the end we call close on the fd and
    not fclose on the fp - which leaks the FILE* structure.

commit b031f3488296305c55ab1f1090f7408541a1b9c7
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri May 24 17:01:33 2013 +1000

    Simplify pmdagfs2 gfs.control metric implementation a bit
    
    Removes an unreachable return case from gfs2_control_fetch,
    and makes the code a little more generalised (no switch).
    
    Simpligies gfs_control_check_value in two ways - removes a
    branch which (re)sets local "value" variable to zero, which
    it already was guaranteed to be (dead code removal).  Also,
    make the onstack char array ("buffer") word aligned to make
    life easier for the compiler.

commit 1c32e9cb98100bcfffd434826be1f31545aea7ae
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri May 24 16:56:25 2013 +1000

    Fix shell syntax error in qa/654 - misspelt function name
    
    Resolves "654: line 54: _filler_pminfo: command not found".

commit bf8928be2f1bdd5dbc359ab320321ffbb9d8ffe1
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Fri May 24 16:52:27 2013 +1000

    Small code consistency cleanups in pmdagfs2 code
    
    Resolve a handful of remaining funciton open-brace code
    style differences in qa scripts and C code.
    
    Annotated a couple of lock_time.c routines as static.
    
    Major/minor device number handling code is consistently
    making use of unsigned integers now, and when we sscanf
    'em from a string, add a check to make doubly sure that
    values were returned.
    
    Removed a no-longer-used struct declaration in a header
    file (lock_time.h - struct node / linkedList_t).
    
    Fixed some gsf2 -> gfs2 typos in comments.

commit 6354c3540e8ac8b81d77abd2fad50a674c15a01a
Author: Paul Evans <pevans@xxxxxxxxxx>
Date:   Thu May 23 16:51:30 2013 +0100

    pmdagfs2: changes based on review feedback on pcp mailing list
    
    Made general improvements with code for the pmda over last time whilst
    taking on suggestions made by Nathan Scott. Have improved collection
    code for the glock_lock_time trace-point metrics as well as switching
    to the provided pmdaCache(3) data structure instead of a home made
    linked-list datastructure.
    
    Introduced a new gfs2.control.glock_lock_time metric which can be used
    to enable/disable the glock_lock_time statistic collection based on
    its value, this can be set using pmStore(3)on this metric. On installs
    without the support for trace-point based statistics, the metrics are
    disabled by default because the necessary config files will not be found.
    
    Cleaned up the collection loop for the trace-point data, have moved to a
    single loop that removes the dependency on allocation of dynamic memory
    to achieve collection of the data. These changes will reduce the memory
    requirements of the pmda and speeds up the pmda with less iteration of
    the data.
    
    Changed the data structure to hold our collected data, we have gone
    from home made linked-list to the available pmdaCache(3) this reduces
    the amount of code, simplifies reading and reduces again the amount
    of iteration of the locks we have (no more remove duplicate and easier
    addition through a well defined hash).
    
    Changed the collection rate by changing the calling
    gfs2_refresh_lock_time() from once-per each file-system to once for
    all file-systems, this reduces the complexity of data storage. Reducing
    potential of lost lock data in-between refreshes. Changed the number of
    calls to gfs2_refresh_lock_time() from once per each filesystem to now
    being called once for all file-systems also removed forgotten testing
    code that was left within the gfs2_fetch_refresh function.
    
    Separated instance refresh so that the collection of dev_id is handled
    within another function in order to increase code readability. Introduced
    the necessary pmStore(3) hooks so that we can use the control based
    metrics to handle whether we are collecting statistics from the
    trace-point based metrics borrowing from the techniques used in the
    linux pmda.
    
    Split the QA files into two halves, the first 654 handles testing of
    the base part of the pmda (glstats, sbstats, glocks metrics). 655 then
    handles testing for the trace-point based metrics with not_run escapes
    in an event that trace-point based metrics are not available for the
    install (i.e. Debian unstable?). A new common.gfs2 script handles common
    functions used in both of the tests.

commit 656068f562c5eed45bb84690e8648ff546e001ad
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Wed May 1 11:44:00 2013 +1000

    Rework the gfs2 qa test handling of trace file enablement
    
    Needs to take care that superuser privilege has been aquired,
    and if the file doesn't exist, we need to use _notrun rather
    than failing the test with an echo message.

commit 3b11a4637465e9b0a84ca4ab05b311fd6ad5de7c
Author: Nathan Scott <nathans@xxxxxxxxxx>
Date:   Wed May 1 11:38:04 2013 +1000

    Fix typo in a pmdagfs2 comment

commit 449c286a9f92968fd6fcef3e6729ab35ac4e311f
Author: Paul Evans <pevans@xxxxxxxxxx>
Date:   Tue Apr 30 13:04:52 2013 +0100

    Changes to pmdagfs2 to add addtional filesystem metrics and their
    corresponding help text.
    
    Changed the location on where the indom cache fetched filesystem
    names to /sys/fs/gfs2/ to allow the additional query for the device
    id of the filesystem which is needed for the gfs2_glock_lock_time
    tracepoint metrics. Also added the required code to support the
    additional metrics.
    
    Added metrics for glstats "/sys/kernel/debug/gfs2/<fs>/glstats" these
    metrics can count the number of glocks that exists currently for the
    filesystem and separate how many per type of glock.
    
    Added metrics for the gfs2_glock_lock_time tracepoint. These metrics
    determine the current *worst* glock for the filesystem at the time of
    query. The data is fetched from "/sys/kernel/debug/tracing/trace_pipe"
    which lists the values of glocks for all filesystems as they are updated
    in (real-time). We read through the pipe taking all of the buffered values
    and calculate which is our worst glock, the device id for each filesystem
    is used to separate the locks for each filesystem.
    
    Created some additional qa tests in "qa/654".

<Prev in Thread] Current Thread [Next in Thread>
  • pcp updates: pmdagfs2 updates, libpcp user/group acls, qa, Nathan Scott <=