The goal is twofold: firstly, we want to extend the script so
that it can deal with more than a single git submodule, and
secondly we'd like to reduce the amount of duplicated code.
Moreover, since we're making heavy changes to the code anyway,
we might as well make sure it follows a somewhat consistent
coding style too.
To reduce code duplication, we introduce a new --dry-run
option, which can be used by third parties to figure out
whether calling autogen.sh is necessary or not: this allows
us to get rid of the reimplementation of part of the logic
in cfg.mk and guarantee they'll never get out of sync.
Other changes include: making dirty submodules checking and
cleaning entirely independent of other operations; removing
the use of 'set -e' and handling errors explicitly instead;
better parsing of command line arguments.
The Test-AutoBuild project, that this script is supposed to
be used with, hasn't seen any activity in ~6 years; libvirt's
own CI is running on Jenkins with a completely independent
setup that doesn't use the script at all.
This header file has been created so that we can expose
internal functions to the test suite without making them
public: those in qemu_capabilities.h bearing the comment
/* Only for use by test suite */
are obvious candidates for being moved over.
Depending on the architecture, requirements for ACPI and UEFI can
be different; more specifically, while on x86 UEFI requires ACPI,
on aarch64 it's the other way around.
Enforce these requirements when validating the domain, and make
the error message more accurate by mentioning that they're not
necessarily applicable to all architectures.
Several aarch64 test cases had to be tweaked because they would
have failed the validation step otherwise.
Now that the NO_ACPI and NO_HPET capabilities are set
automatically by virQEMUCapsInitQMPBasicArch() if
appropriate for the architecture, they shouldn't be
used manually to avoid masking bugs.
The capabilities used in test cases should match those used
during normal operation for the tests to make any sense.
This results in the generated command line for a few test
cases (most notably non-x86 test cases that were wrongly
assuming they could use -no-acpi) changing.
Instead of having a single function that probes the
architecture from the monitor and then sets a bunch of
basic capabilities based on it, have a separate function
for each part: virQEMUCapsInitQMPArch() only sets the
architecture, and virQEMUCapsInitQMPBasicArch() only sets
the capabilities.
This split will be useful later on, when we will want to
set basic capabilities from the test suite without having
to go through the pain of mocking the monitor.
Creating dummy <change> elements was a workaround for the
HTML DTD not allowing empty <ul> elements, but we can do
better by tweaking the the XSLT stylesheet.
Now that the source file is validated against a schema that
only allows the <code> HTML tag to be used, we can rely on
that assumption to simplify our XSLT stylesheet.
The value we use internally to represent the lack of a memory
locking limit, VIR_DOMAIN_MEMORY_PARAM_UNLIMITED, doesn't
match the value setrlimit() and prlimit() use for the same
purpose, RLIM_INFINITY, so we have to handle the translation
ourselves.
Partially-resolves: https://bugzilla.redhat.com/1431793
For guests that use <memoryBacking><locked>, our only option
is to remove the memory locking limit altogether.
Partially-resolves: https://bugzilla.redhat.com/1431793
Instead of having a separate function, we can simply return
zero from the existing qemuDomainGetMemLockLimitBytes() to
signal the caller that the memory locking limit doesn't need
to be set for the guest.
Having a single function instead of two makes it less likely
that we will use the wrong value, which is exactly what
happened when we started applying the limit that was meant
for VFIO-using guests to <memoryBacking><locked>-using
guests.
This reverts commit c2e60ad0e5.
Turns out this check is excessively strict: there are ways
other than <memtune><hard_limit> to raise the memory locking
limit for QEMU processes, one prominent example being
tweaking /etc/security/limits.conf.
Partially-resolves: https://bugzilla.redhat.com/1431793
We want pcie-root-ports to be used when available in QEMU,
but at the same time we need to ensure that hosts running
older QEMU releases keep working and that the user can
override the default at any time.
Add a comment for the original pcie-root-port test cases
to make it clear how these new test cases are different.
ioh3420 is emulated Intel hardware, so it always looked
quite out of place in aarch64/virt guests. Even for x86/q35
guests, the recently-introduced pcie-root-port is a better
choice because, unlike ioh3420, it doesn't require IO space
(a fairly constrained resource) to work.
If pcie-root-port is available in QEMU, use it; ioh3420 is
still used as fallback for when pcie-root-port is not
available.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1408808
QEMU 2.9 introduces the pcie-root-port device, which is
a generic version of the existing ioh3420 device.
Make the new device available to libvirt users.
The generated HTML will contain <ul></ul> otherwise, which
triggers an error during 'make check'.
The proper fix would be not to generate the problematic
HTML in the first place but, while I'm working on it, this
workaround will do.
virQEMUCapsHasPCIMultiBus() performs a version check on
the QEMU binary to figure out whether multiple buses are
supported, so to get the correct aliases assigned when
dealing with pSeries guests we need to spoof the version
accordingly in the test suite.
Due to the extra architecture-specific logic, it's already
necessary for users to call virQEMUCapsHasPCIMultiBus(),
so the capability itself is just a pointless distraction.
Now that QEMU_CAPS_DEVICE_PCI_BRIDGE is no longer checked
unless a pci-bridge is really part of the configuration,
and most uses of the legacy PCI controller combo have been
dropped from tests that use PCIe machine types, we can
drop the corresponding capabilities from a lot of test
cases.
qemuDomainAssignPCIAddresses() hardcoded the assumption
that the only way to support devices on a non-zero bus is
to add one or more pci-bridges; however, since we now
support a large selection of PCI controllers that can be
used instead, the assumption is no longer true.
Moreover, this check was always redundant, because the
only sensible time to check for the availability of
pci-bridge is when building the QEMU command line, and
such a check is of course already in place.
In fact, there were *two* such checks, but since one of
the two was relying on the incorrect assumption explained
above, and it was redundant anyway, it has been dropped.
Up until a while ago, libvirt would automatically add a legacy
PCI controllers combo (dmi-to-pci-bridge + pci-bridge) to any
PCIe machine type (x86_64/q35 and aarch64/virt).
As a result, a number of input and output files in the test
suite ended up containing the legacy PCI controllers, even
though they are not needed or in any way relevant to the
feature being tested.
Get rid of most of the occurrences. Most of the time, this
just means removing the controllers from the input file and
regenerating the output files; in a few instances, some
minor tweaking is performed on the input file, most notably
removing the memory balloon: as memory balloon support was
not the scope of the test being changed, there is no loss
of test coverage from doing so.
Several occurrences of the legacy PCI controllers remain in
the test suite, both because removing their usage would have
required even more tweaking, and because we still want to
have coverage of this perfectly valid combination.
When switching over the values in the virDomainControllerModelPCI
enumeration, make sure the proper cast is in place so that the
compiler can warn us when the coverage is not exaustive.
For the same reason, remove the 'default' case from one of the
existing switch statements.
When switching over the values in the virDomainControllerModelPCI
enumeration, make sure the proper cast is in place so that the
compiler can warn us when the coverage is not exaustive.
For the same reason, fold some unstructured checks (performed by
comparing directly against some values in the enumeration) inside
an existing switch statement.
The switch in virDomainPCIControllerModelToConnectType()
had some code that, while techically part of the
_PCIE_SWITCH_DOWNSTREAM_PORT case, was in fact dead due
to the early return.
Get rid of the dead code, and fix the inaccurate function
description while at it.
mknod() is affected my the current umask, so we're not
guaranteed the newly-created device node will have the
right permissions.
Call chmod(), which is not affected by the current umask,
immediately afterwards to solve the issue.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1421036
For the namespaces feature to work properly we need to be able
to make a perfect copy of the original /dev, including ACLs.
By adding a BuildRequires on libacl-devel we ensure that ACL
support will be enabled at configure time and made available
to the QEMU driver.
In order for memory locking to work, the hard limit on memory
locking (and usage) has to be set appropriately by the user.
The documentation mentions the requirement already: with this
patch, it's going to be enforced by runtime checks as well,
by forbidding a non-compliant guest from being defined as well
as edited and started.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1316774
Updating docs/news.xml in the same commit that performs the
documented change makes backports needlessly complicated,
both for mainteinance branches and downstream distributions,
because it introduces additional potential for merge
conflicts.
Document in the contributor guidelines that the release notes
should be updated in a separate commit instead, so that it's
easy to backport just the code change.
The recent deprecation in glibc (commit b76e065991ec) means the
module will fail to build entirely:
nss/libvirt_nss.c: In function '_nss_libvirt_gethostbyname_r':
nss/libvirt_nss.c:363:13: error: RES_USE_INET6 is deprecated [-Werror]
int af = ((_res.options & RES_USE_INET6) ? AF_INET6 : AF_INET);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This resolver option was removed shortly after being introduced,
and application using it are already broken anyway.
Now that we have built a fairly solid process for dealing with
release notes, we should start pushing for contributors to
provide the relevant information along with their code:
documenting the process is clearly a requirement for this to
happen.
Instead of encoding formatting information inside the
corresponding XSLT stylesheet, use a Python script to reformat
the text appropriately based on a few simple markers.
Splitting the task between the XSLT stylesheet and the Python
script allows us to keep both parts very simple.
Currently, building the NEWS file involves using a XSLT stylesheet
to extract information from the same HTML file that's used on the
libvirt website.
The process works, but it's quite fiddly in that it requires the
source HTML to be formatted in a very precise way, and a single
missing newline can mess up the resulting plain text considerably.
Moreover, the XSLT stylesheet itself encodes a lot of the details
of converting to plain text in a way that's not necessarily easy
to understand, tweak or fix.
To improve the process, move all existing entries to a new XML
file that contains exactly the information we care about in a
simple structured format, and start generating both the HTML and
plain text versions of the release notes using XSLT stylesheets
that can now afford to be almost trivial.
virtio-pci is the way forward for aarch64 guests: it's faster
and less alien to people coming from other architectures.
Now that guest support is finally getting there (Fedora 24,
CentOS 7.3, Ubuntu 16.04 and Debian testing all support
virtio-pci out of the box), we'd like to start using it by
default instead of virtio-mmio.
Users and applications can already opt-in by explicitly using
<address type='pci'/>
inside the relevant elements, but that's kind of cumbersome and
requires all users and management applications to adapt, which
we'd really like to avoid.
What we can do instead is use virtio-mmio only if the guest
already has at least one virtio-mmio device, and use virtio-pci
in all other situations.
That means existing virtio-mmio guests will keep using the old
addressing scheme, and new guests will automatically be created
using virtio-pci instead. Users can still override the default
in either direction.
Existing tests such as aarch64-aavmf-virtio-mmio and
aarch64-virtio-pci-default already cover all possible
scenarios, so no additions to the test suites are necessary.
When generating the plain text version of the contributor
guidelines we add a ludicrous amount of vertical whitespace
in some spots. Tweak the XSLT stylesheet and regenerate the
now much better looking file.
When updating the source file in commit bd4f4d1686, I forgot
that we also store the generated plain text version in git and
didn't regenerate it.
I also missed one spot that required an additional <p> tag, so
fix both mistakes in one go.
Some of the <li> elements in the "General tips for contributing
patches" section were missing the corresponding inner <p> element,
so they ended up all lumped together.
Clang 3.9 refuses to compile the existing code with the
following error:
util/virfirewall.c:425:20: error: passing an object that undergoes
default argument promotion to 'va_start'
has undefined behavior [-Werror,-Wvarargs]
va_start(args, layer);
^
util/virfirewall.c:420:37: note: parameter of type 'virFirewallLayer'
is declared here
virFirewallLayer layer,
^
This happens because 'layer' is of type virFirewallLayer, which
is an enum type and not a standard type such as eg. void* or int.
To solve the issue, turn virFirewallAddRule() from a very thin
wrapper around virFirewallAddRuleFullV() to a macro that expands
to a call to virFirewallAddRuleFull() - itself a very thin wrapper
around the aforementioned virFirewallAddRuleFullV() - with no loss
of functionality or type safety.
LIBVIRT_ARG_WITH_ALT is more generic than LIBVIRT_ARG_WITH, which
is tailored at switching features on and off.
Rename the macros according to their intended purpose, and add
some documentation to help developers pick between the two.
We're going to change the format of the source HTML quite
a bit, and we're going to have to update the XSLT stylesheet
accordingly.
Move 2016 releases out of the way before doing so.
Add QMP schema data query for aarch64. The gic capabilities are
unfortunately queried after the QMP schema and thus this patch needs to
undo the temporary removal of the declared support for query-qmp-schema.
Note that as a gicv3 machine was not available the schema data is taken
from the gicv2 case. It should be identical since qemu would be built
from the same source.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Add QMP schema data query for aarch64. The gic capabilities are
unfortunately queried after the QMP schema and thus this patch needs to
undo the temporary removal of the declared support for query-qmp-schema.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Commit e8861f6971 changed our spec file to compile and run
tests in parallel. That's a very good step forward, but why
stop there? Let's run *all* make jobs in parallel and really
put those expensive cores to use!
On my laptop, this shaves ~10s off 'make rpm'.
${exec_prefix} and ${prefix} point to the same directory in
most setups, but when that's not the case the former should
be used for architecture-dependent data such as shared objects,
which makes it the best fit for our Wireshark dissector.
While at it, change all uses of $(var) to ${var}: they are
absolutely identicaly as far as make's concerned, but autoconf
itself seems to prefer the latter form so we might as well
follow suit.
We only need to strip $ws_prefix from $ws_plugindir if we've
retrieved it from pkg-config: if we're building it ourselves
from $libdir, we can just use it without further processing.
Since we're using autoconf to substitute the right value in
Makefile.am now, we can use a less generic name without running
into circular dependencies.
Adding $(prefix) in Makefile.am, as we were doing, means that
it would be prepended even when using --with-ws-plugindir,
which is something we don't want to happen.
Instead, we add it beforehand but take care that it doesn't
get expanded until make is called.
Even when we're building $plugindir ourselves because we can't
retrieve it using pkg-config, we still want to strip the prefix,
except in that case it would be the same prefix we're using for
building libvirt.
The fact that $plugindir is missing also doesn't tell us
anything about $ws_prefix, so we have to handle the two variables
separately.
We only claim support for OSs that are still supported by the
respective vendors, which means anything older than Fedora 23
is out. Reword the comment a bit to highlight the criteria.
The code is entirely correct, but it still managed to trip me
up when I first ran into it because I did not realize right away
that VIR_PCI_CONNECT_TYPES_ENDPOINT was not a single flag, but
rather a mask including both VIR_PCI_CONNECT_TYPE_PCI_DEVICE and
VIR_PCI_CONNECT_TYPE_PCIE_DEVICE.
In order to save the next distracted traveler in PCI Address Land
some time, document this fact with a comment. Add a test case for
the behavior as well.
The Requires relationship is very strong, in that it prevents
a unit from running unless all the units it Requires are
running as well.
This turns out to be a problem because we want to be able to
restart libvirtd at any time without having libvirt-guests
suspend or shutdown running guests.
Turn the Requires relationship into a Wants relationship:
this way starting libvirt-guests will cause systemd to (attempt
to) start libvirtd as well, but stopping or restarting libvirtd
will not alter libvirt-guests' running state.
Commit 839a060 tied the lifecycle of virtlogd more
closely to that of libvirtd. Unfortunately, while starting
virtlogd when libvirtd is started is definitely a good idea,
restarting virtlogd or shutting it down at any time outside
of system poweroff is not.
Revert part of that commit by removing the PartOf= lines,
meaning that only startup requests will be propagated from
libvirtd to virtlogd.
Resolves: https://bugzilla.redhat.com/1372576
Having After=libvirtd.service merely ensures that, if both
services are asked to start, libvirtd.service will start
first.
What we really want is for libvirtd.service to be started
whenever libvirt-guests.service is asked to start. Adding a
Requires= relationship guarantees that will happen.
We use a separate line for each After= relationship in other
unit files: do the same here for consistency's sake, and also
to make future changes nicer to diff
libvirt-guests.service does both suspend *and* resume guests,
depending on whether it's being started or stopped: the
description should reflect this, to avoid confusing messages
during startup.
Replace "active" with "running" (to match virsh list's output)
and don't capitalize libvirt.
We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.
Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.
Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.
Resolves: https://bugzilla.redhat.com/1372576
The first argument should be const char ** instead of
char **, because this is a search function and as such it
doesn't, and shouldn't, alter the haystack in any way.
This change means we no longer have to cast arrays of
immutable strings to arrays of mutable strings; we still
have to do the opposite, though, but that's reasonable.
Since we now pick the default USB controller model when parsing
the guest XML, we can get rid of some duplicated code so that
the default model selection happens in one place only.
Add some comments as well.
Now that the default USB controller model is explicit rather
than implicit for i440fx machines, we have to tweak the
conditions for dropping it in order to keep migration towards
libvirt <= 0.9.4 working.
When the user doesn't specify any model for a USB controller,
we use an architecture-dependent default, but we don't reflect
it in the guest XML.
Pick the default USB controller model when parsing the guest
XML instead of when creating the QEMU command line, so that
our choice is saved back to disk.
Andrea Bolognani