NUMA stands for Non-Uniform Memory Access, in other words a system whose memory is not all in one place. The system consists of multiple "nodes" of cpu's (processors) and/or memory which are linked together by a special high-speed network. All cpu's have access to all of memory, but a cpu's access to memory on the local or a nearby node is faster than to distant nodes.
For more explanation see the whitepaper A NUMA API for Linux and also the original An NUMA API for Linux.

numactl

The numactl program allows you to run your application program on specific cpu's and memory nodes. It does this by supplying a NUMA memory policy to the operating system before running your program.

For example,

   numactl  --cpubind=0  --membind=0,1  myprog
runs program "myprog" on cpu 0, using memory on nodes 0 and 1.

The package contains other commands, such as numademo, numastat and memhog. The numademo command provides a quick overview of NUMA performance on your system.

libnuma

The libnuma library provides convenient ways for you to add NUMA memory policies into your own program. It is a shared object (.so) library.

For example, you can call numa_bind() to cause your program to be bound to (in other words, run on) specific nodes.

Obtaining numactl and libnuma

You can probably obtain the numactl command and libnuma library as installable packages from your Linux distribution.

Download of Source Code

This link -- download -- will open a window with a list of downloadable source code.

Choose a compressed "tarball" (in gzip format). We recommend the latest release. (The suffix -rc1 means "release candidate". In other words, it is in development.)

If you would like to verify that the download worked correctly, use the md5sum command and the downloaded MD5SUMS file.

For example,

   md5sum numactl-1.0.2.tar.gz
   cat MD5SUMS
Compare the checksums.

Compile and Install from Source

Using release 1.0.2 as an example:

Expand the tarball:

  tar xfzv numactl-1.0.2.tar.gz

cd numactl-1.0.2

make

make install
(you must be root to install the library and header file)

Testing Compiled Commands and Library

In the test/ directory there is a small regression test suite.

Note that these tests assume an unloaded machine with memory free on every node. Otherwise you will get spurious failures in the non strict memory policies (preferred, interleave)

To run the tests:

make test

Contributing

Send comments, discussion and patches to linux-numa@vger.kernel.org

To subscribe to the list, place "subscribe linux-numa" in the body of an email and send it to majordomo@vger.kernel.org. Majordomo will return a confirmation message that you will have to return.

The numactl/libnuma maintainer is Cliff Wickman. Andi Kleen handed off ownership in March, 2008.

Copyright

numactl and the demo programs are under the GNU General Public License, v.2
libnuma is under the GNU Lesser General Public License, v2.1.

The manpages are under the same license as the Linux manpages (see the files)

numademo links with a library (stream_lib) derived from the C version of STREAM by John D. McCalpin and Joe R. Zagar for one sub benchmark. See stream_lib.c for the license. In particular when you publish numademo output you might need to pay attention there or filter out the STREAM results.

It also uses a public domain Mersenne Twister implementation from Michael Brundage.

The bitmask declarations and bitmask_*() routines are: Copyright (c) 2004_2007 Silicon Graphics, Inc. (SGI) All rights reserved. SGI publishes it under the terms of the GNU General Public License, v2, as published by the Free Software Foundation.

Author:
Andi Kleen, SUSE Labs
Version 1.0.3-rc1 by Cliff Wickman and Christoph Lameter, SGI