CPUSETS for Linux
What are CPUSETS ?
CPUSETs are lightweight objects in the linux kernel that enable users to
partition their multiprocessor machine by creating execution areas. A
virtualization layer has been added so it becomes possible to split a
machine in terms of CPUs.
The main motivation of this patch is to give the linux kernel full
administration capabilities concerning CPUs.
CPUSETs are strong jails, and a process running inside this predefined area won't be able
to run on other processors than those given to him.
Some domains in which it can be useful :
- Web Servers running multiple instances of the same web application.
- Servers running different applications (for instance, a web server and a database).
- HPC applications, especially in NUMA machines.
CPUSETS allow to:
These features have been implemented as a kernel patch for Linux 2.6 and a
suite of userland tools.
create sets of CPUs on the system, and bind applications to them
provide a way to create sets of cpus *inside* a set of cpus : hence a
system administrator can partition a system among users, and users can
partition their partition among their applications.
- The same applies to memory : the memory used by a cpuset can be restricted to some of the nodes of a NUMA system
- CPU-bound applications : Many applications (as it is often the case
for HPC apps) use to have a "one process on one processor" policy. They
can use sched_setaffinity() to do so, but what if we have to run several
such apps at the same time ? One can do this by creating a cpuset for each
- Web Serving : A server containing a web server (apache for instance)
and a database (MySQL, Oracle) may want to have one part of the machine
dedicated for each task. A cpuset can be created for each task. If the
server is used to run two instances of this web-system, cpusets can be
used to split the system in two, and then in each partition create one
area for the web server and one area for the database.
- User administration : an administrator may give fixed numbers of
CPUs to some classes of users and leave the rest to other users, for
instance. He can do so by creating cpusets and assigning them to users.
- Critical applications : processors inside strict areas may not be
used by other areas. Thus, a critical application (real time ...) may be
run inside an area and be sure that other processes won't use its CPU.
This implies that others applications won't be able to lower its
reactivity. This can be done by creating a cpuset for the critical
application, and another for all the other tasks.
CPUSETs are now integrated in the -mm series of Linux kernels.
However these kernels lack some interesting cpuset features that can be found here as additional patches:
See the available manpages or a more detailed explanation.
Change on the fly the execution area of a whole set of processes (to
give more resources to a critical application, for example). When you change the CPU list of a cpusets, all processes that belong to this cpuset will be migrated, if necessary, to stay inside the CPU list.
translate the masks of CPUs given to sched_setaffinity() so they stay
inside the set of CPUs. With this mechanism, processors are virtualized,
for the use of sched_setaffinity() and /proc information. Thus, any former
application using this syscall to bind processes to processors will
work with virtual CPUs without any change. A new file is added in each cpuset, in the cpuset filesystem, to choose whether a cpuset is virtualized, or not.
Source can be found here.
Any feedback, comment or opinion is welcome.
Mon Oct 11 17:23:40 CEST 2004