When creating new group for cgroups v2 the we cannot check
cgroups.controllers for that cgroup because the directory is created
later. In that case we should check cgroups.subtree_control of parent
group to get list of controllers enabled for child cgroups.
In order to achieve that we will prefer the parent group if it exists,
the current group will be used only for root group.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
This reverts commit 7bca1c9bdc.
As it turns out it's not a good idea on systemd hosts. The root
cgroup can have all controllers enabled but they don't have to be
enabled for sub-cgroups.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When detecting available controllers on host we can be limited by list
of controllers from qemu.conf file.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Currently CPU controller cannot be enabled if there is any real-time
task running and is assigned to non-root cgroup which is the case on
several distributions with graphical environment.
Instead of erroring out treat it as the controller is not available.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
In order to skip controllers that we are not able to activate we need
to return different return value so the caller can decide what to do.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
It might happen that we are not able to enable CPU controller so we
can enable it for thread sub-cgroups only if it's available in parent
cgroup.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The assumption that CPU controller would be always enabled is wrong, we
should use any available controller to create a new sub-cgroup.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This affects only cgroups v2 where enabled controllers are not based on
available mount points but on the list provided in cgroup.controllers
file. However, moving it will fill in placement as well, so it needs
to be freed together with mount point if we don't need that controller.
Before this patch we were assuming that all controllers available in
root cgroup where available in all other sub-cgroups which was wrong.
In order to fix it we need to move the cgroup controllers detection
after cgroup placement was prepared in order to build correct path for
cgroup.controllers file.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
In cgroups v2 we don't have to detect available controllers every single
time if we are creating a new cgroup based on parent cgroup.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In kernel 4.12 there was introduced new BFQ scheduler and in kernel
5.0 the old CFQ scheduler was removed. This has an implication on
the cgroups file names.
If the CFQ controller is enabled we use one file:
io.weight
The new BFQ controller expose one file with different name:
io.bfq.weight
Except for different name they have different syntax.
io.weight:
default $val
major:minor $val
io.bfq.weight:
$val
The difference is that BFQ doesn't support per-device weight.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In kernel 4.12 there was introduced new BFQ scheduler and in kernel
5.0 the old CFQ scheduler was removed. This has an implication on
the cgroups file names.
If the CFQ controller is enabled we use these two files:
blkio.weight
blkio.weight_device
The new BFQ controller expose only one file with different name:
blkio.bfq.weight
The reason is that BFQ controller doesn't support per-device weight.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If we need to get a path of specific file and we need to check its
existence before we use it then we can reuse that path to get value
for specific device. This way we will not build the path again in
virCgroupGetValueForBlkDev.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If we need to get a path of specific file and we need to check its
existence before we use it then we can reuse that path to get/set
values instead of calling the existing get/set value functions which
would be building the path again.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The function was deprecated in udev 219 and all the supported OSes
don't have older version of udev or systemd.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
virCgroup struct is always defined and the free function is not calling
anything that would require OS supporting cgroups.
This fixes an issue if we try to start a VM with QEMU binary that
doesn't support QXL. The start operation will fail in
qemuProcessStartValidateVideo() which will set correct error message,
but later in one of the cleanup paths we will call
qemuDomainObjPrivateDataClear() which always calls virCgroupFree()
and that will fail on OS that doesn't support cgroups and it will
set a new error which will be eventually reported to user.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
The 'bandwidths' variable is allocated using VIR_RESIZE_N so it has to
be freed as well.
==118315== 8 bytes in 1 blocks are definitely lost in loss record 299 of 2,401
==118315== at 0x4C29DAD: malloc (vg_replace_malloc.c:308)
==118315== by 0x4C2C100: realloc (vg_replace_malloc.c:836)
==118315== by 0x52C3FAF: virReallocN (viralloc.c:245)
==118315== by 0x52C4079: virExpandN (viralloc.c:294)
==118315== by 0x532BBA8: virResctrlAllocParseProcessMemoryBandwidth (virresctrl.c:1156)
==118315== by 0x532BBA8: virResctrlAllocParseMemoryBandwidthLine (virresctrl.c:1211)
==118315== by 0x532BBA8: virResctrlAllocParse (virresctrl.c:1414)
==118315== by 0x532BBA8: virResctrlAllocGetGroup (virresctrl.c:1446)
==118315== by 0x532C11D: virResctrlAllocGetDefault (virresctrl.c:1464)
==118315== by 0x532D15E: virResctrlAllocAssign (virresctrl.c:1923)
==118315== by 0x532D15E: virResctrlAllocCreate (virresctrl.c:2042)
==118315== by 0x31E1ABEE: qemuProcessResctrlCreate (qemu_process.c:2596)
==118315== by 0x31E1ABEE: qemuProcessLaunch (qemu_process.c:6444)
==118315== by 0x31E1E341: qemuProcessStart (qemu_process.c:6721)
==118315== by 0x31E81315: qemuDomainObjStart.constprop.50 (qemu_driver.c:7288)
==118315== by 0x31E81A65: qemuDomainCreateWithFlags (qemu_driver.c:7341)
==118315== by 0x54DDB4B: virDomainCreate (libvirt-domain.c:6534)
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Only PCI devices have '/sys/class/net/<ifname>/device/resource' so we
need to skip this check for all other network devices.
Without this patch and RDMA enabled libvirt will not detect any network
device that doesn't have the path above which includes 'lo', 'virbr',
'tun', etc.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1639258
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
If we pass XML to virDomainDefineXML API with these two elements:
...
<title></title>
<description></description>
...
libvirt correctly ignores these two elements and they will not appear
in the parsed XML.
However, if we use virDomainSetMetadata API and with "" as value for
title or description we will end up with the parsed XML that contains
these empty elements.
Let's fix the behavior of this API to behave the same as
virDomainDefineXML.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1518042
Reviewed-by: John Ferlan <jferlan@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Capabilities should not duplicate data that are obvious from our
documentation and will not change with different QEMU binaries
or the way how we compile libvirt.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
When libvirt is reconnecting to running domain that uses cgroup v2
the QEMU process reports cgroup for the emulator directory because the
main thread is in that cgroup. We need to remove the "/emulator" part
in order to match with the root cgroup directory name for that domain.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
The rewrite to support cgroup v2 missed this function. In cgroup v2
we have different files to track tasks.
We would fail to remove cgroup on non-systemd OSes if there is any
extra process assigned to guest cgroup because we would not kill any
process form the guest cgroup.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
This removes code duplication and simplifies cgroup detection.
As a drawback we will not have separate messages to enable cgroup
controller in kernel or to mount it. On the other side the rewrite
adds support for cgroup v2.
The kernel config support was wrong because it was parsing
'/proc/self/cgroup' instead of '/proc/cgroups/' file.
The mount suggestion is removed as well because it will not work
with cgroup v2.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
We need to configure multiple env variables for each set of tests so
create helper functions to do that.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
We need to create the cgroup v2 sysfs the same way as we do for
cgroup v1.
This introduces new VIR_CGROUP_MOCK_MODE env variable which will
configure which cgroup mode each test requires. There are three
different modes:
- legacy: only cgroup v1 is available and it's the default mode
- hybrid: both cgroup v1 and cgroup v2 are available and have some
controllers
- unified: only cgroup v2 is available
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Remove the trailing '/' from prefix. This change is required in order
to introduce tests for unified cgroups. They are usually mounted in
'/sys/fs/cgroup'.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
This enables to use both cgroup v1 and v2 at the same time together
with libvirt. It is supported by kernel and there is valid use-case,
not all controllers are implemented in cgroup v2 so there might be
configurations where administrator would enable these missing
controllers in cgroup v1.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
In order to set CPU cfs period using cgroup v2 'cpu.max' interface
we need to load the current value of CPU cfs quota first because
format of 'cpu.max' interface is '$quota $period' and in order to
change 'period' we need to write 'quota' as well. Writing only one
number changes only 'quota'.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
In cgroups v2 we need to handle threads and processes differently.
If you need to move a process you need to write its pid into
cgrou.procs file and it will move the process with all its threads
as well. The whole process will be moved if you use tid of any thread.
In order to move only threads at first we need to create threaded group
and after that we can write the relevant thread tids into cgroup.threads
file. Threads can be moved only into cgroups that are children of
cgroup of its process.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
When creating cgroup hierarchy we need to enable controllers in the
parent cgroup in order to be usable. That means writing "+{controller}"
into cgroup.subtree_control file. We can enable only controllers that
are enabled for parent cgroup, that means we need to do that for the
whole cgroup tree.
Cgroups for threads needs to be handled differently in cgroup v2. There
are two types of controllers:
- domain controllers: these cannot be enabled for threads
- threaded controllers: these can be enabled for threads
In addition there are multiple types of cgroups:
- domain: normal cgroup
- domain threaded: a domain cgroup that serves as root for threaded
cgroups
- domain invalid: invalid cgroup, can be changed into threaded, this
is the default state if you create subgroup inside
domain threaded group or threaded group
- threaded: threaded cgroup which can have domain threaded or
threaded as parent group
In order to create threaded cgroup it's sufficient to write "threaded"
into cgroup.type file, it will automatically make parent cgroup
"domain threaded" if it was only "domain". In case the parent cgroup
is already "domain threaded" or "threaded" it will modify only the type
of current cgroup. After that we can enable threaded controllers.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Cgroup v2 has only single mount point for all controllers. The list
of controllers is stored in cgroup.controllers file, name of controllers
are separated by space.
In cgroup v2 there is no cpuacct controller, the cpu.stat file always
exists with usage stats.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
If the placement was copied from parent or set to absolute path
there is nothing to do, otherwise set the placement based on
process placement from /proc/self/cgroup or /proc/{pid}/cgroup.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
When reconnecting to a domain we are validating the cgroup name.
In case of cgroup v2 we need to validate only the new format for host
without systemd '{machinename}.libvirt-{drivername}' or scope name
generated by systemd.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
We cannot detect only mount points to figure out whether cgroup v2
is available because systemd uses cgroup v2 for process tracking and
all controllers are mounted as cgroup v1 controllers.
To make sure that this is no the situation we need to check
'cgroup.controllers' file if it's not empty to make sure that cgroup
v2 is not mounted only for process tracking.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Place cgroup v2 backend type before cgroup v1 to make it obvious
that cgroup v2 is preferred implementation.
Following patches will introduce support for hybrid configuration
which will allow us to use both at the same time, but we should
prefer cgroup v2 regardless.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
This reverts commit 1602aa28f8.
There is no need to call virCgroupRemove() nor virCgroupFree() if
virCgroupEnableMissingControllers() fails because it will not modify
'group' at all.
The cleanup of directories is done in virCgroupMakeGroup().
Reviewed-by: John Ferlan <jferlan@redhat.com>
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Reviewed-by: Marc Hartmayer <mhartmay@linux.ibm.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Cgroups are linux specific and we need to make sure that the code is
compiled only on linux. On different OSes it fails the compilation:
../../src/util/vircgroupv1.c:65:19: error: variable has incomplete type 'struct mntent'
struct mntent entry;
^
../../src/util/vircgroupv1.c:65:12: note: forward declaration of 'struct mntent'
struct mntent entry;
^
../../src/util/vircgroupv1.c:74:12: error: implicit declaration of function 'getmntent_r' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
while (getmntent_r(mounts, &entry, buf, sizeof(buf)) != NULL) {
^
../../src/util/vircgroupv1.c:814:39: error: use of undeclared identifier 'MS_NOSUID'
if (mount("tmpfs", root, "tmpfs", MS_NOSUID|MS_NODEV|MS_NOEXEC, opts) < 0) {
^
../../src/util/vircgroupv1.c:814:49: error: use of undeclared identifier 'MS_NODEV'
if (mount("tmpfs", root, "tmpfs", MS_NOSUID|MS_NODEV|MS_NOEXEC, opts) < 0) {
^
../../src/util/vircgroupv1.c:814:58: error: use of undeclared identifier 'MS_NOEXEC'
if (mount("tmpfs", root, "tmpfs", MS_NOSUID|MS_NODEV|MS_NOEXEC, opts) < 0) {
^
../../src/util/vircgroupv1.c:841:65: error: use of undeclared identifier 'MS_BIND'
if (mount(src, group->legacy[i].mountPoint, "none", MS_BIND,
^
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
All the system headers are used only if we are compiling on linux
and they all are present otherwise we would have seen build errors
because in our tests/vircgrouptest.c we use only __linux__ to check
whether to skip the cgroup tests or not.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
tests/vircgrouptest.c uses #ifdef __linux__ for a long time and no
failure was reported so far so it's safe to assume that __linux__ is
good enough to guard cgroup code.
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
With the introduction of cgroup v2 there are new names used with
cgroups based on which version is used:
- legacy: cgroup v1
- unified: cgroup v2
- hybrid: cgroup v1 and cgroup v2
Let's use 'legacy' instead of 'cgroupv1' or 'controllers' in our code.
Reviewed-by: Fabiano Fidêncio <fidencio@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Pavel Hrdina