Consider bitmaps for incoming migration regardless of non-shared storage
flag.
When bitmaps are offered from the source, consult the local image if the
bitmap is present and if not accept migration. Migration of bitmaps
which exist in the qcow2 metadata is skipped because qemu rejects such
setup (although handles it correctly in case of shared storage setup;
see below).
This allows bitmap propagation for cases when the qcow2 image is not
actually shared between destinations but the data is (using the
data_file feature).
At the same time this preserves existing bitmap handling semantics for
other cases. Specifically qemu, in case of shared storage properly
propagates the bitmap which was already recorded in the qcow2 metadata
on disk even if libvirt doesn't instruct migration, yet tolerates
migration instruction if the file is not yet recorded in the on-disk
metadata. In both cases the contents are preserved correctly.
When storage is not shared (which includes even cases when we migrate
it via NBD) it's expected that the bitmaps don't exist on the
destination and thus all will be picked for migration. We can also
infer that this wasn't ever a problem by the fact that the code skipping
migration of existing bitmaps was broken until recently, and qemu
would refuse such config.
I've tested all the above scenarios including verifying that the
resulting bitmaps capture dirtied regions before and after migration.
For testing this the following command is useful:
virsh qemu-monitor-command --domain DOMNAME --hmp 'qemu-io -d /machine/peripheral/virtio-disk0/virtio-backend "write -P 0xcc 4M 1M"'
Which simulates a write from the guest side without the need to interact
with the guest OS.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Until now block dirty bitmaps were offered to destination only if
non-shared storage migration was enabled.
Upcoming patches will want to support it also in cases when storage is
shared but the destination has a qcow2 overlay using the 'data_file'
feature where the qcow2 overlay is not actually shared.
To support that we'll now always offer bitmaps for migration. The
destination can then decide (using existing logic) to pick only the
ones that are not present in the image on destination, which is how
it was supposed to work even now.
The patch removes all the flag checks and simply offers bitmaps in any
case. The overhead incurred by this is one 'query-named-block-nodes'
call to qemu.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
On incoming migration qemu doesn't load bitmaps into memory (which makes
them available under the 'dirty-bitmaps' field which we parse as the
'bitmaps' array in 'qemuBlockNamedNodeData') until after actually
resuming CPUs, thus the check for existing bitmaps never actually
worked.
We need to check the 'qcow2bitmaps' field instead which is populated
from the qcow2 headers prior to activating the image.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We currently probe dirty block tracking bitmaps by looking at the loaded
ones ('dirty-bitmaps'). Unfortunately those may not yet be populated on
incoming migration when the image was not yet activated, but we need to
know which ones are stored in the image so that we don't migrate those
explicitly, which would fail.
Load the list of bitmaps in a qcow2 image from the format specific data,
which is already loaded at that point.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Rather than looking for 30 specific nodenames (via a loop) iterate
everything in the hash table (in a sorted order). This simplifies the
code and provides more test outputs on previously-ignored nodenames.
The listing of internal snapshots in the output was also missing a
newline, which would now cause problems with multiple images reproted.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In previous commit of v12.0.0-85-g2c66b6d72c we've tried to solve
a problem where xdrproc_t is a prototype of a function which
takes only two arguments instead of three. See original commit
for more info. The fix consists of a config time check and
setting XDRPROC_T_3ARGS accordingly (in meson-config.h). This
works for nearly all of our code, except rpcgen which is
intentionally independent of the rest of the code. Therefore, the
macro has to be set extra - by specifying it on the compiler cmd
line.
Fixes: 2c66b6d72c
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
According to 9fa3a8ab6f,
macOS insists on passing 3 arguments for xdrproc_t.
Passing 3 arguments was a good common ground, but since
recently[1] FreeBSD only accepts 2 arguments.
Add a meson.build check whether 3 arguments are accepted,
and add macros which passes either 2 or 3 arguments to
xdrproc_t based on the result of this check.
1: https://cgit.freebsd.org/src/commit/?id=ac5a19ec6989675c8ec6c3ca245dba243d1a6416
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Provide sample XML and CLI args for the iommufd XML schema
for pc, q35, and virt machine types.
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
When launching a qemu VM with the iommufd feature enabled for VFIO
hostdevs:
- Do not allow cgroup, namespace, and seclabel access to VFIO
paths (/dev/vfio/vfio and /dev/vfio/<iommugroup>)
- Allow access to iommufd paths (/dev/iommu and
/dev/vfio/devices/vfio*) for AppArmor, SELinux, and DAC
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Open iommufd FD from libvirt backend without exposing
these FDs to XML users, i.e. one per domain for
/dev/iommu, and pass the FD to qemu command line. Set
per-process memory accounting for iommufd instead of
the default per-user memory accounting.
Suggested-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Open VFIO FDs from libvirt backend without exposing
these FDs to XML users, i.e. one per iommufd hostdev
for /dev/vfio/devices/vfioX, and pass the FD to qemu
command line.
Suggested-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Implement the IOMMU_OPTION_RLIMIT_MODE
ioctl to set per-process memory accounting for
iommufd. This prevents ENOMEM errors from the
default per-user memory accounting when multiple
VMs under the libvirt-qemu user have their pinned
memory summed and checked against a per-process
RLIMIT_MEMLOCK limit.
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Introduce private data for hostdevs and allocate hostdev
private data by default.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Implement a new iommufd attribute under hostdevs' PCI
subsystem driver that can be used to specify associated
iommufd object when launching a qemu VM.
Signed-off-by: Ján Tomko <jtomko@redhat.com>
Signed-off-by: Nathan Chen <nathanc@nvidia.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The "virt" board in QEMU has a "virtualization" option
that is documented like this:
virtualization
Set ``on``/``off`` to enable/disable emulating a guest CPU which implements the
Arm Virtualization Extensions. The default is ``off``.
(from system/arm/virt.rst)
According to the documentation, the "virtualiaztion" option
is related to the "gic-version" option. Specifically, gic version=4
requires virtualization to be enabled. And gic version=max will use
version=4 when virtualization is enabled, and 3 when not.
Libvirt does not currently model neither gic version "4" nor "max"
though.
It is also documented for the "vexpress-a(9|15)" boards, where it is
also disabled by default:
- QEMU defaults to providing a CPU which does not provide either
TrustZone or the Virtualization Extensions: if you want these you
must enable them with ``-machine secure=on`` and ``-machine
virtualization=on``
(system/arm/vexpress.rst).
On the command line it looks like:
qemu-system-aarch64 -machine type=virt,virtualization=on ..
Model it using the "virtualization" element in the "features" section:
<features>
<virtualization/>
</features>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Our network has multiple means of forwarding the traffic and
'hostdev' is one of them. This mode means that the network is
configured to use a set of PCI devices which are then assigned to
individual domains to use (PCI device assignment). Now, as of
v12.0.0-61-gecb2e06bdf our test runners
(testCompareXMLToXMLFiles() and testCompareXMLToConfFiles()) call
networkValidateTests(). For aforementioned type of network this
means checking that the specified set of devices contains only
VFs (see v3.2.0-rc1~24 for more info). It is true that our
virpcimock is preloaded which mimics VFs, but our utils module
(virpci.c specifically) talks to sysfs to check various PCI
device attributes, including whether it's a VF.
This obviously works on Linux and doesn't work anywhere else.
Therefore, until our utils module is taught how to check PCI
attribs on other systems, make the "hostdev" test case expect
validation failure on non-Linux systems.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Now that networkxml2xmltest does both XML -> XML and XML -> conf
tests its name became misleading. Rename it to networkxmlconftest
and move its data into networkxmlconfdata/ dir.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Now that networkxml2xmltest does XML->conf tests the
networkxml2conftest is redundant. Drop it.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The networkxml2xmltest does basic parse -> format tests.
The networkxml2conftest does parse -> conf tests.
Now, majority of XMLs are the same. That is, output XMLs of
networkxml2xmltest and input XMls of networkxml2conftest. There
are only a few differences. This is actually great, because it
will allow either tests to do both test cases.
There are some (subtle) differences in individual test cases
though:
1) some test cases exist only in networkxml2conftest and not
networkxml2xmltest, or
2) some test cases in networkxml2conftest have more values, i.e.
extra elements, extra attributes. or
3) some test cases in networkxml2conftest have less values.
For cases from 1) they were just copied over. For cases from 2)
those extra elements/attributes were added, and for cases from 3)
those extra attributes were removed (to minimize changes to .conf
files in near future).
One caveat though: networkxml2xmlupdatetest uses input XMLs of
networkxml2xmltest too (hence changes under
networkxml2xmlupdateout/ dir). This means that the
"delete-srv-record-protocol" test started failing, because the
input network XML now has more <srv/> records than the test case
anticipated. But this is easy to fix - hence seemingly unrelated
change under networkxml2xmlupdatein/ dir.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Soon, the testRun() will run more than one test case. The input
network XML, however, stays the same. Instead of parsing it and
throwing away immediately, store it temporarily.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
So far, the testInfo struct contained immutable data (from its
lifetime point of view). But that is about to change. For
instance, it will hold parsed network definition (virNetworkDef)
and in order to avoid leaking dynamically allocated data
corresponding free function must be introduced (or clear
function, doesn't really matter). At this point, the structure
might as well be dynamically allocated entirely.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This effectively dissolves testCompareXMLToXMLHelper() into
testRun(). Motivation is that parts of data generated inside of
testCompareXMLToXMLHelper() is going to be reused from the caller
(testRun()).
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This is a beginning of something bigger. The idea is that one
DO_TEST_FULL() macro (and its friends) will run multiple test
cases (just like qemuxmlconftest does). But in order to do that
in a readable fashion, the macro should merely just expand to a
function call. The function will then call virTestRunLog(),
multiple times possibly.
This is the first step in that direction.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The aim of virNetworkXMLOption object is to provide some
immutable data to XML parser (e.g. various callbacks). Since the
object is immutable, it can be created once and then reused by
all test cases.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Currently, there are two calls to virTestCompareToFile() inside
of testCompareXMLToConfFiles(). If the first one fails the
control jumps directly onto the fail label and skips the second
one. This means that When regenerating test case output
(VIR_TEST_REGENERATE_OUTPUT) the test binary has to be called
twice to regenerate all the files. Suboptimal. Try harder to call
both compare helpers.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Inside of testCompareXMLToConfFiles() the
networkDnsmasqConfContents() is called. This may also produce
contents of corresponding hosts file. This is then compared to
expected contents stored on disk as ${testname}.hostsfile. But
due to additional checks virTestCompareToFile() might not even be
called. Problem with that is when there's actual content but the
file doesn't exist the compare helper is not called and thus
VIR_TEST_REGENERATE_OUTPUT trick doesn't work. Let's call the
helper more often as it is perfectly capable of handling this
edge case. What it is not capable of handling is when the file
shouldn't exist at all. So handling of that case is kept.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Inside of testCompareXMLToConfFiles() the network definition is
parsed and if that succeeds a virNetworkObj is created by calling
virNetworkObjNew(). But if the latter fails, the control jumps
onto the fail label where only the object is freed but not
already parsed definition leading to a leak.
Swapping these two steps ensures that if either of them fails no
memleak occurs.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Our network driver calls networkValidate() right after a network
XML is parsed. This is similar to domain validation step when
parsing domain XML. But it's not that convoluted in network
driver. Regardless, any network related test should mimic real
life scenario as close as possible and thus
networkValidateTests() should be called right after domain XML is
parsed.
Now, networkValidate() might query sysfs wrt to PCI devices and
thus tests must start using virpcimock. The function will also
generate random MAC addresses, if needed, hence virrandommock.
With this change, passthrough-pf and passthrough-address-crash
test cases of networkxml2xmltest started failing but looking at
corresponding XMLs those test cases were designed to test just
XML parsing. They were never designed to showcase a "real"
network XML. So mark them as expected fail.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The aim of this internal API is to wrap networkValidate() and it
is meant to be called from our tests.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Soon, individual test cases of networkxml2xmltest will be subject
to networkValidate() call. This means, that input XMLs must be
valid (or marked as expected fail). Anyway, there are couple of
offenders:
1) 8021Qbh-net.xml setting vlan for <forward mode='private'/> is
unsupported,
2) hostdev.xml networkValidate() will check if hostdevs specified
for <forward mode='hostdev'/> are VFs. Use PCI addresses from
virpcimock.
3) openvswitch-net.xml for <forward mode='bridge'/> only
openvswitch type of virtualports is allowed.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
If network config does not require dnsmasq then none is spawned.
Having a test case that would still require generating dnsmasq
config is weird and can lead to spurious results. Just fail such
test case.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
This test case is spurious. If this was real life scenario then
no dnsmasq would be spawned and yet, the test tries to generate
dnsmasq config for it. Just drop the test case and move on.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Whether a network needs dnsmasq or not is decided at the
beginning of networkStartDhcpDaemon(). Move that code into a
separate function so it can be reused later.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
There are two functions implemented in bridge_driver.c that are
used in tests (networkDnsmasqCreateXMLConf() and
networkDnsmasqConfContents()) but are declared in
bridge_driver.h. This goes against our current practice where
such APIs are declared in $name_priv.h.
Therefore, move those APIs to bridge_driver_priv.h
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Instead of including network driver header files as:
#include "network/header.h"
pass network_inc_dir to the compiler and include files directly.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Abstract the places where we create URLs into one place. This is just
refactoring and should not change the behaviour.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
The code calls 'qemu-img rebase' to fix the backing store references.
The 'qemu-img' process here is run as the 'qemu' user or whatever the
defaults and domain XML resolve to. Since this, in certain cases, works
also on images which are not part of the backing chain and in privileged
deployments thus can be owned by 'root:root' the update may fail
(silently).
To preserver root-squash deployments but fix also the above case, retry
the operation on failure as current user.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Mention the 'path' where the detection failed as well as include the
possibility that the 'path' doesn't exist in the message itself.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The 'max_depth' argument of 'virStorageSourceGetMetadata' doesn't just
limit how far the function goes but also fails completely if the chain
is deeper than the passed value.
In 'qemuSnapshotDiskHasBackingDisk' we only care about finding the
backing image, so just one level below, the passed path, but due to the
above setting '1' as max_depth will make the function simply fail every
time.
Extract and reuse QEMU_DOMAIN_STORAGE_SOURCE_CHAIN_MAX_DEPTH as the
detection depth. While '200' layers is overkill for this code, we also
start a full qemu instance just to delete an snapshot so this doens't
matter and still protects from self-referential images.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
The code does a 'qemu-img rebase' rather than a 'qemu-img create' what
the commit suggests. Since we enumerate all arguments right below,
there's no need for a comment.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
When the 'backingStore' pointer is not populated the function calls
'virStorageSourceGetMetadata' to try to populate it but if the on-disk
metadata doesn't have a backing image (e.g. if it's the 'base' image of
the chain) the 'backingStore' or the metadata fetcher fails the pointer
will still be NULL.
The function then calls 'virStorageSourceIsSameLocation' but the
internal functions for dealing with storage sources don't handle NULL
gracefully.
Since the code calling 'qemu-img' based on the data detected here
doesn't actually raise errors if the operations fail there's no point
in raising errors here either.
Closes: https://gitlab.com/libvirt/libvirt/-/issues/844
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
NVMe has PCIe transport, so connect NVMe devices to PCIe bus instead of
classic PCI bus if available.
Signed-off-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
Reviewed-by: Laine Stump <laine@redhat.com>
The aw_bits member of the virDomainIOMMUDef is of type unsigned
int. However, in a few places
(virDomainIOMMUDefCheckABIStability(), virDomainIOMMUDefFormat(),
qemuBuildIOMMUCommandLine()) incorrect printf modifier is used.
Fix those places.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
The virDomainInterfaceAddresses() API returns an array of
_virDomainInterface structs which then describe IP addresses
associated with given domain. The struct contains 'name' member
which is documented deliberately vaguely: "interface name". This
is because depending on the source of truth used (controlled by
'source' argument) the name can be wildly different from the one
in domain XML. Now, in case of source =
VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_ARP, the host's ARP table is
parsed and matching interfaces are found by comparing MAC
addresses. If it's a match then the 'name' is set to net->ifname
(corresponds to /interface/target/@dev). But that is not always
set and sometimes may be NULL (e.g. for hostdevs, usernet). We
can't change the API (like we did for hwaddr in v1.2.14-rc1~105)
because this is already released. So the next best thing to do is
to put the interface alias in there.
To be on a safe side, do the same change to the
VIR_DOMAIN_INTERFACE_ADDRESSES_SRC_LEASE case.
Resolves: https://issues.redhat.com/browse/RHEL-141496
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Ever since of v1.2.14-rc1~105 the hwaddr member of
_virDomainInterface struct can be NULL. And this is documented
inside the struct. But then the public API documents it can never
be NULL which is obviously wrong. Fix the public API
documentation.
Fixes: 3640245db7
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Peter Krempa
Michal Privoznik
Roman Bogorodskiy
Nathan Chen
Pavel Borecki
Richard W.M. Jones
Akihiko Odaki