.\" Copyright (c) 2008 Christian Brueffer
.\" Copyright (c) 2008 Jeffrey Roberson
.\" All rights reserved.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
.\" ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
.\" SUCH DAMAGE.
.\"
.Dd January 29, 2023
.Dt CPUSET 2
.Os
.Sh NAME
.Nm cpuset ,
.Nm cpuset_getid ,
.Nm cpuset_setid
.Nd manage CPU affinity sets
.Sh LIBRARY
.Lb libc
.Sh SYNOPSIS
.In sys/param.h
.In sys/cpuset.h
.Ft int
.Fn cpuset "cpusetid_t *setid"
.Ft int
.Fn cpuset_setid "cpuwhich_t which" "id_t id" "cpusetid_t setid"
.Ft int
.Fn cpuset_getid "cpulevel_t level" "cpuwhich_t which" "id_t id" "cpusetid_t *setid"
.Sh DESCRIPTION
The
.Nm
family of system calls allow applications to control sets of processors and
memory domains and assign processes and threads to these sets.
Processor sets contain lists of CPUs and domains that members may run on
and exist only as long as some process is a member of the set.
All processes in the system have an assigned set.
The default set for all processes in the system is the set numbered 1.
Threads belong to the same set as the process which contains them,
however, they may further restrict their set with the anonymous
per-thread mask to bind to a specific CPU or subset of CPUs and memory domains.
.Pp
Sets are referenced by a number of type
.Ft cpuset_id_t .
Each thread has a root set, an assigned set, and an anonymous mask.
Only the root and assigned sets are numbered.
The root set is the set of all CPUs and memory domains available in the system
or in the system partition the thread is running in.
The assigned set is a subset of the root set and is administratively
assignable on a per-process basis.
Many processes and threads may be members of a numbered set.
.Pp
The anonymous set is a further thread-specific refinement on the assigned
set.
It is intended that administrators will manipulate numbered sets using
.Xr cpuset 1
while application developers will manipulate anonymous sets using
.Xr cpuset_setaffinity 2 and
.Xr cpuset_setdomain 2 .
.Pp
To select the correct set a value of type
.Ft cpulevel_t
is used.
The following values for
.Fa level
are supported:
.Bl -column CPU_LEVEL_CPUSET -offset indent
.It Dv CPU_LEVEL_ROOT Ta "Root set"
.It Dv CPU_LEVEL_CPUSET Ta "Assigned set"
.It Dv CPU_LEVEL_WHICH Ta "Set specified by which argument"
.El
.Pp
The
.Fa which
argument determines how the value of
.Fa id
is interpreted and is of type
.Ft cpuwhich_t .
The
.Fa which
argument may have the following values:
.Bl -column CPU_WHICH_INTRHANDLER -offset indent
.It Dv CPU_WHICH_TID Ta "id is lwpid_t (thread id)"
.It Dv CPU_WHICH_PID Ta "id is pid_t (process id)"
.It Dv CPU_WHICH_TIDPID Ta "id is either a thread or process id"
.It Dv CPU_WHICH_JAIL Ta "id is jid (jail id)"
.It Dv CPU_WHICH_CPUSET Ta "id is a cpusetid_t (cpuset id)"
.It Dv CPU_WHICH_IRQ Ta "id is an irq number"
.It Dv CPU_WHICH_INTRHANDLER Ta "id is an irq number for an interrupt handler"
.It Dv CPU_WHICH_ITHREAD Ta "id is an irq number for an ithread"
.It Dv CPU_WHICH_DOMAIN Ta "id is a NUMA domain"
.El
.Pp
An
.Fa id
of '-1' may be used with a
.Fa which
of
.Dv CPU_WHICH_TID ,
.Dv CPU_WHICH_PID ,
.Dv CPU_WHICH_TIDPID ,
or
.Dv CPU_WHICH_CPUSET
to mean the current thread, process, or current thread's
cpuset.
All cpuset syscalls allow this usage.
.Pp
A
.Fa level
argument of
.Dv CPU_LEVEL_WHICH
combined with a
.Fa which
argument other than
.Dv CPU_WHICH_CPUSET
refers to the anonymous mask of the object.
This mask does not have an id and may only be manipulated with
.Xr cpuset_setaffinity 2 .
.Pp
.Fn cpuset
creates a new set containing the same CPUs as the root set of the current
process and stores its id in the space provided by
.Fa setid .
On successful completion the calling process joins the set and is the
only member.
Children inherit this set after a call to
.Xr fork 2 .
.Pp
.Fn cpuset_setid
attempts to set the id of the object specified by the
.Fa which
argument.
Currently
.Dv CPU_WHICH_PID
is the only acceptable value for which as
threads do not have an id distinct from their process and the API does
not permit changing the id of an existing set.
Upon successful completion all of the threads in the target process will
be running on CPUs permitted by the set.
.Pp
.Fn cpuset_getid
retrieves a set id from the object indicated by
.Fa which
and stores it in the space pointed to by
.Fa setid .
The retrieved id may be that of either the root or assigned set
depending on the value of
.Fa level .
.Fa level
should be
.Dv CPU_LEVEL_CPUSET
or
.Dv CPU_LEVEL_ROOT
to get the set id from
the process or thread specified by the
.Fa id
argument.
Specifying
.Dv CPU_LEVEL_WHICH
with a process or thread is unsupported since
this references the unnumbered anonymous mask.
.Pp
The actual contents of the sets may be retrieved or manipulated using
.Xr cpuset_getaffinity 2 ,
.Xr cpuset_setaffinity 2 ,
.Xr cpuset_getdomain 2 , and
.Xr cpuset_setdomain 2 .
See those manual pages for more detail.
.Sh RETURN VALUES
.Rv -std
.Sh ERRORS
The following error codes may be set in
.Va errno :
.Bl -tag -width Er
.It Bq Er EINVAL
The
.Fa which
or
.Fa level
argument was not a valid value.
.It Bq Er EDEADLK
The
.Fn cpuset_setid
call would leave a thread without a valid CPU to run on because the set
does not overlap with the thread's anonymous mask.
.It Bq Er EFAULT
The setid pointer passed to
.Fn cpuset_getid
or
.Fn cpuset
was invalid.
.It Bq Er ESRCH
The object specified by the
.Fa id
and
.Fa which
arguments could not be found.
.It Bq Er EPERM
The calling process did not have the credentials required to complete the
operation.
.It Bq Er ENFILE
There was no free
.Ft cpusetid_t
for allocation.
.El
.Sh SEE ALSO
.Xr cpuset 1 ,
.Xr cpuset_getaffinity 2 ,
.Xr cpuset_getdomain 2 ,
.Xr cpuset_setaffinity 2 ,
.Xr cpuset_setdomain 2 ,
.Xr pthread_affinity_np 3 ,
.Xr pthread_attr_affinity_np 3 ,
.Xr cpuset 9
.Sh HISTORY
The
.Nm
family of system calls first appeared in
.Fx 7.1 .
.Sh AUTHORS
.An Jeffrey Roberson Aq Mt jeff@FreeBSD.org