Permissions

Certain operations are subject to permission checks by the Kernel. Often there are resource limits below which no additional permissions are required (getrlimit(2)), exceeding those limits typically requires the appropriate capabilities(7) or to run as root/have SUID bit set.

Capabilities

Notable capabilities:

CAP_SYS_NICE allows process to lower nice level (numapp::DynamicScheduler), set a real-time scheduler (numapp::StaticScheduler), set arbitrary CPU affinity (numapp::CpuAffinity) and use numapp::MemPolicyFlag::MoveAll when applying memory policies.
CAP_IPC_LOCK allows locking/unlocking memory (numapp::MemLock, numapp::MemLockAll et.c.).

See capabilities(7) for details.

Resource Limits

Notable limits:

RLIMIT_AS limits maximum virtual address space.
RLIMIT_CPU limits amount of CPU time a process can consume.
RLIMIT_MEMLOCK limits amount of memory that can be locked.
RLIMIT_NICE specifies ceiling for nice level.
RLIMIT_RTPRIO specifies ceiling of real-time scheduler priority.
RLIMIT_RTTIME effectively limits how long a real-time process may run without yielding to kernel.

See getrlimit(2) for details.

Dependencies

Library dependencies:

pthread
numa (a.k.a. libnuma or numactl)

General Utilities

Before use the availability of NUMA on the host system should be ensured with numapp::NumaAvailable().

numapp::NumaAvailable() in <numapp/numa.hpp>.
numapp::NumaPolicies containing a set of optional policies, can be used as a general store of: all possible policies. numapp::thisThread::Apply() also provide an overload to apply all the policies to the current thread.
numapp::MakeThread() to create NUMA-aware thread, see example below.
numapp::ForEach() which is useful to e.g. perform an action for every CPU in a CPU mask.

NUMA++ provides a utility to create threads with NUMA policies in <numapp/thread.hpp> with numapp::MakeThread().

An example of creating a thread that only runs on CPUs 0-3:

#include <chrono>
#include <iostream>
#include <numapp/numa.hpp>
#include <numapp/thread.hpp>
 
void ThreadFunc(std::chrono::milliseconds duration) {
    std::cout << "Thread affinity: " << numapp::CpuAffinity::MakeFromActive() << std::endl;
 
    std::this_thread::sleep_for(duration);
}
 
int main() {
    using namespace numapp;
    using namespace std::chrono_literals;
    if (!NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    NumaPolicies policies;
    policies.SetCpuAffinity(CpuAffinity::MakeFromCpuStringAll("0-3"));
    auto thread = MakeThread("myThread", policies, &ThreadFunc, 500ms);
 
    thread.join();
}

Memory APIs

Attention: If you take the step of managing memory with NUMA in mind, it is important to understand how the primitives provided by NUMA++ works and the context they are used. It may be helpful to think of NUMA++ as block allocators with the minimum block size being the system page size for normal allocations and the specified huge page size for huge page allocations. It is not recommended to use this as a general purpose allocator. As an example, foonathan-memory has a compatible concept BlockAllocator.

Manual Allocation

Allocate pages of memory with specified memory policy (see function group).

To e.g. allocate one page from NUMA node 1

// Creates a bind policy for NUMA node 1 and allocates with that policy.
std::size_t size = numapp:GetPageSize();
void* buffer = numapp::Allocate(size, numapp::MemPolicy::MakeBindNode(1));
/* ... */
numapp::Free(buffer, size);

numapp::Allocate()
numapp::Free()

Hardware Queries

Provides the following functions:

Memory Locking

Prevents all or parts of process memory from being paged to the swap area.

numapp::MemLock() / numapp::MemUnlock() to lock/unlock an address range.
numapp::MemLockAll() / numapp::MemUnlockAll() to lock/unlock all process memory.

This is an example of how to effectively disable swapping completely for the process:

Note: This example will cause all allocated memory to be locked, not only the resident memory which will increase memory pressure and possibly cause out-of-memory situations.

#include <iostream>
#include <numapp/memory.hpp>
 
int main() {
    using numapp::operator|;
    using numapp::LockAllFlag;
    if (!numapp::NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    // Lock current and future mappings
    if (auto ec = numapp::MemLockAll(LockAllFlag::Current | LockAllFlag::Future); ec) {
        std::cerr << "Failed to lock memory!" << std::endl;
        return -1;
    }
}

Memory Policy

Controls how physical memory is by default allocated by a thread or the memory placement for an already mapped virtual memory range.

Note: Important to note is that the memory policy is used by the Kernel when allocating physical memory, not when application allocates virtual memory. It is also the policy of the thread that trigger the page fault which will be used, which is not necessarily the thread that allocated the virtual memory.

A memory policy is described using numapp::MemPolicy in <numapp/mempolicy.hpp>. And is applied to the current thread with e.g. the function numapp::thisThread::Apply(numapp::MemPolicy const&) or a memory range with numapp::Apply(void*,std::size_t,numapp::MemPolicy const&,MemPolicyFlag).

To apply a policy temporarily to the current thread, e.g. to have the policy used when mmap'ing some memory, the numapp::ScopedMemPolicy can be used to automatically apply new policy and restore the old, within a scope.

A helper function to apply a memory policy to the current thread stack memory is also provided with numapp::thisThread::ApplyStack().

Memory policies are inherited at fork(), clone() (without CLONE_VM flag) and exec*.

The following example applies a bind policy for NUMA nodes 0-1 to the current thread:

#include <iostream>
#include <numapp/numa.hpp>
#include <numapp/mempolicy.hpp>
 
int main() {
    using numapp::MemPolicy;
    using numapp::Nodemask;
    if (!numapp::NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    auto policy = MemPolicy(MemPolicy::Mode::Bind, Nodemask::MakeFromNodestring("0-1"));
    // Set default policy for main thread
    if (auto ec = numapp::thisThread::Apply(policy); ec) {
        std::cout << "Failed to apply policy: " << ec.message() << std::endl;
        return 1;
    }
    // And apply policy to stack
    if (auto ec = numapp::thisThread::ApplyStack(policy); ec) {
        std::cout << "Failed to apply policy: " << ec.message() << std::endl;
        return 1;
    }
}

The following example applies policy to a memory range, rather than setting the policy to the current thread:

#include <iostream>
#include <numapp/memory.hpp>
#include <numapp/mempolicy.hpp>
 
void BindMemoryRange(int node, void* address, std::size_t size) {
    using numapp::MemPolicy;
    using numapp::operator|;
    using numapp::LockFlag;
    using numapp::MemPolicyFlag;
 
    auto policy = MemPolicy::MakeBindNode(node);
    auto flags = MemPolicyFlag::Move | MemPolicyFlag::Strict;
 
    if (auto ec = numapp::Apply(address, size, policy, flags); ec) {
        std::cerr << "Failed to apply policy: " << ec.message() << std::endl;
        throw std::system_error(ec);
    }
 
    if (auto ec = numapp::MemLock(address, size, LockFlag::PreFault); ec) {
        std::cerr << "Failed to apply policy: " << ec.message() << std::endl;
        throw std::system_error(ec);
    }
}

Polymorphic Memory Resource

NUMA++ provide implementation of std::pmr::memory_resource with numapp::PageResource that can be used to allocate memory with specified memory policy using the STL allocator std::pmr::polymorphic_allocator.

An example of this is shown below where the pool allocator std::pmr::unsynchronized_pool_resource use memory from numapp::PageResource.

#include <string>
#include <vector>
 
#include <numapp/memory.hpp>
 
void Example() {
    // Create a pool resource that allocates memory from NUMA node 1 with using
    // PageResource.
    auto resource = numapp::PageResource(numapp::MemPolicy::MakeBindNode(1),
                                         numapp::MemPolicyFlag::Strict);
 
    // Important Note
    // --------------
    //
    // Pool resource likely requires memory for its implementation, which will
    // use the same upstream memory resource `resource`. This is not necessarily
    // optimal as the pool implementation may perform multiple allocations, each
    // smaller than the page size, which will each take at minimum 1 page. This
    // overhead may be discounted by example sharing the pool resource for the
    // full application, or per NUMA node.
    auto pool = std::pmr::unsynchronized_pool_resource(&resource);
 
    // Now we can use the `pool` with e.g. STL containers that use
    // `std::pmr::polymorphic_allocator`
    [[maybe_unused]] auto vector = std::pmr::vector<std::uint8_t>(1024, &pool);
    [[maybe_unused]] auto string = std::pmr::string(1024, 'a', &pool);
}

Note: Boost.SmartPtr provides allocator-aware utilities for constructing std::unique_ptr with boost::allocate_unique().

NUMA++ also provide a std::pmr::memory_resource implementation that locks memory allocated from upstream memory resource in numapp::LockResource.

#include <vector>
 
#include <numapp/memory.hpp>
 
void Example() {
    // Create a pool resource that allocates memory from NUMA node 1 with strict
    // policy, using PageResource.
    auto resource = numapp::PageResource(numapp::MemPolicy::MakeBindNode(1),
                                         numapp::MemPolicyFlag::Strict);
 
    auto locked = numapp::LockResource(numapp::LockFlag::PreFault, &resource);
 
    // Allocate 4 MiB block memory from NUMA node 1, pre-fault it and lock it
    // into memory.
    [[maybe_unused]] auto vector =
        std::pmr::vector<std::uint8_t>(4 * 1024 * 1024, &locked);
}

Huge Pages

NUMA++ provides primitives for allocating and freeing huge pages with numapp::AllocateHuge() and numapp::FreeHuge() as well as std::pmr::memory_resource implementation numapp::HugePageResource.

An example usage is shown below:

#include <numapp/memory.hpp>
 
void Example() {
    // Allocate 16MiB from NUMA node 1 in 2MiB pages.
    auto const size = 16 * 1024 * 1024;
    auto const page_size = numapp::HugePagePreset::Huge2M;
 
    // 1) AllocateHuge will allocate size, rounding up to nearest page_size
    // multiple
    void* ptr = numapp::AllocateHuge(size,
                                     page_size,
                                     numapp::MemPolicy::MakeBindNode(1),
                                     numapp::MemPolicyFlag::Strict);
 
    // 2) It is particularly important to pre-fault huge-pages as the likelihood
    // for failure is higher. If page-fault would fail when memory is accessed
    // the result is a segmentation fault.
    if (auto ec = numapp::MemLock(ptr, size, numapp::LockFlag::PreFault); ec) {
        // Page-fault failed; this would have caused a segmentation fault if not
        // for `numapp::MemLock()`.
        throw std::system_error(ec, "Page fault failed");
    }
 
    // 3) FreeHuge will free memory, rounding up to nearest page_size multiple
    // as required by mmap.
    numapp::FreeHuge(ptr, size, page_size);
}

CPU Affinity API

Control where a thread is executed.

The CPU affinity of the current thread can be controlled using numapp::CpuAffinity in <numapp/cpuaffinity.hpp>.

To e.g. set CPU affinity of current thread to run on cores 0-3, disregarding whether they are isolated or not:

#include <iostream>
#include <numapp/numa.hpp>
#include <numapp/cpuaffinity.hpp>
 
int main() {
    using numapp::CpuAffinity;
    using numapp::Cpumask;
    if (!numapp::NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    auto policy = CpuAffinity(Cpumask::MakeFromCpuStringAll("0-3"));
    // Set CPU affinity for main thread
    if (auto ec = numapp::thisThread::Apply(policy); ec) {
        std::cout << "Failed to apply policy: " << ec.message() << std::endl;
        return 1;
    }
}

Scheduler API

Control how a thread is scheduled for execution by the kernel.

The scheduler and priority of threads can be controlled using the following primitives from <numapp/scheduler.hpp>.

Type	Represents
`numapp::IdleScheduler`	`SCHED_IDLE`.
`numapp::DynamicScheduler`	The dynamic priority schedulers `SCHED_OTHER` and `SCHED_BATCH`.
`numapp::StaticScheduler`	The static priority schedulers `SCHED_RR` and `SCHED_FIFO`.
`numapp::Scheduler`	Represents any scheduler and internally holds one of the previous.

These APIs are used to create and apply policies, either from currently active policy or explicitly created manually.

To e.g. apply the static FIFO scheduling policy with highest priority:

#include <iostream>
#include <numapp/numa.hpp>
#include <numapp/scheduler.hpp>
 
int main() {
    using numapp::StaticScheduler;
    if (!numapp::NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    auto policy = StaticScheduler(StaticScheduler::Policy::Fifo, 99);
    // Set CPU affinity for main thread
    if (auto ec = numapp::thisThread::Apply(policy); ec) {
        std::cout << "Failed to apply policy: " << ec.message() << std::endl;
        return 1;
    }
}

If the type of scheduler is not statically known the sum-type numapp::Scheduler can be used. In this example the current policy is queried:

#include <iostream>
#include <numapp/numa.hpp>
#include <numapp/scheduler.hpp>
 
int main() {
    using namespace numapp;
    if (!NumaAvailable()) {
        std::cout << "NUMA not available";
        return -1;
    }
 
    Scheduler current = Scheduler::MakeFromActive();
 
    if (current.HoldsScheduler<numapp::DynamicScheduler>()) {
        auto sched = current.GetScheduler<numapp::DynamicScheduler>();
        std::cout << "Nice: " << sched.GetNice() << std::endl;
    }
 
    // Or get underlying variant
    std::variant<DynamicScheduler, StaticScheduler, IdleScheduler> variant = current.Get(); 
}

Table of Contents