Docutils don't generate <code> for inline literals (``blah``) in rst
but rather put them in the '.literal' class. Add a selector for making
them bold when used in definition list headers.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
'docutils' add a stylesheet to the output html file for direct
consumption. Since we use the html files just as an intermediate step
which is post-processed to add our own stylesheet and drop the docutils
one in the process we can ask 'rst2html' to not add any for an
intermediate file with less garbage.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Show various usage of filters including some useful examples.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
Promote the 'What to attach?' section to a first level heading and
request also the XML config of a VM, coredump backtrace if something
crashed and ask to not tear down the environment for the possibility to
ask for additional data.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
NEWS.rst is based in the root of the repository and 'hvsupport.html'
doesn't have a backing file which can be edited since it's fully
generated. Our 'contribute -> edit this page' link on the bottom of the
page is wrong in those cases.
Fix it by adding the contribute section only when there's a source and
base the 'source' of a html file in the root of the repository.
Along with that we need to modify the scripts/meson-html-gen.py script
to accept optional 'pagesrc' and the XSL template to skip the
'contribute' section when we don't have a source.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Just convert 'docs_html_in_files' into 'docs_html_in_gen'. The target
definitions for those were almost the same.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Provide debug log configuration insight in our kbase.
There are two modifications of the document compared to the wiki
version:
1) The link for reporting a bug agains libvirt was modified to use the
gitlab issue tracker.
2) The link to URI specification details is changed to 'https' protocol.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Set the width to our default value of 70em (max 95%) to prevent
unnecessary line breaks if we have just one panel as it's in the kbase
directory.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When doing an external snapshot we migrate memory to a file as a form of
taking the memory state. This creates a problem as qemu deactivates all
active bitmaps after a successful migration. This means that calling
'query-named-block-nodes' will return an empty list of bitmaps for
devices. We use the bitmap list to propagate the active bitmaps into the
overlay files being created which is required for backups to work after
a snapshot. Since we wouldn't propagate anything a subsequent backup
will fail with:
invalid argument: missing or broken bitmap 'testchck' for disk 'vda'
To fix this, we can simply collect the bitmap list prior to the
migration.
https://bugzilla.redhat.com/show_bug.cgi?id=1862472
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Note that 'numa-mem-supported' turned off for certain machine types
which in turn forced us to generate a newer command line in certain
tests.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
QEMU is going to drop 'vxhs' in the upcoming release so we'll need to
track these separately to prevent test suite breakage.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We already test with real caps so there's no real need for this special
case. While it technically tested the state without TLS encryption key
secrets, it doesn't really matter that much.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The 'commandhelper' checks effectively whether the parent process is
still around to report whether it was daemonized or not.
This creates a unlikely race condition in cases when we do actually
daemonize the process as the intermediate process used for the
daemonization might not have terminated yet which would report wrong
result leading to test failure.
For now there's just 'test4' which actually daemonizes the process.
Add an argument '--check-daemonize' which asks for retries of the
daemonization check in cases where we expect that the commandhelper is
going to be daemonized and use it in 'test4' to make the test more
reliable.
I've observed the test failure sporadically when my box is under load
e.g. while building two trees at once.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Ignore errors from creating "libvirt-tmp-activewrite" bitmap. This
prevents failures of finishing blockjobs if the bitmap already exists.
Note that if the bitmap exists, the worst case that can happen is that
more bits are marked as dirty in the resulting merge.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
There are two possible 'transaction' command arguments in the function.
Rename 'actions' as they deal with creating bitmaps only.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The 'libvirt-tmp-activewrite' bitmap is added during the 'pivot'
operation of block copy and active layer block commit operations
regardless of whether there are any bitmaps to merge, but was not
removed unless a bitmap was merged. This meant that subsequent attempts
to merge into the same image would fail.
Fix it by checking whether the 'libvirt-tmp-activewrite' would be used
by the code and don't skip the code which would delete it.
This is a regression introduced when we switched to the new code for
block commit in <20a7abc2d2d> and for block copy in <7bfff40fdfe5>. The
actual bug originates from <4fa8654ece>.
https://bugzilla.redhat.com/show_bug.cgi?id=1857735
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Commit 20a7abc2d2 tried to delete the possibly leftover bitmap but
neglected to call the actual monitor to do so.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The handler finalizing the active layer block commit doesn't actually
reopen the file for active layer block commit, so the comment and check
are invalid.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
When doing a block copy, there is another chain of images attached to a
disk. Consider them as well when looking up a disk using nodename.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Top level image may get two events, one with the disk target (vda) and
one with disk target with index (vda[3]) if the top level image has an
index.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The index returned by qemuDomainDiskLookupByNodename is the position in
the backing chain rather than the index we report in the XML.
Since with -blockdev they differ now and additionally the disk source
also has an index we need to fix the 'threshold' events we report:
1) If it's the top level image we must always trigger the event without
any suffix as we did until now
2) We must report the correct index
3) We must report the correct index also for the top level image, when
blockdev is used.
This means that we need to potentially emit 2 events, one for the device
without the index and then when blockdev is used and the top level image
has an index we must do it also with the index.
This will fix it for blockdev cases, while also not removing previous
semantics.
https://bugzilla.redhat.com/show_bug.cgi?id=1857204
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Conver the code to the new approach which uses XPath to fetch known
elements rather than looping through all XML children.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Use a switch statement which will not be omitted when adding potential
new types.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to previous commit split out formatting of the mdev subsystem
related stuff.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to previous commit split out formatting of the vHBA subsystem
related stuff.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Split up formatting of the '<address>' element rather that trying to
optimize it with formatting string hacks.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to previous commit split out formatting of the SCSI subsystem
related stuff.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to previous commit split out formatting of the PCI subsystem
related stuff.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Separate out bits related to USB so that the logic isn't entangled in
multiple conditional statements.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Refactor the formatter to the new multiple buffer based approach so that
we can easily separate it into formatters per subsys type.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to previous commits, modify the hostdev detach code to use
blockdev infrastructure to detach (i)SCSI hostdevs.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Similarly to command line creation, use the blockdev helpers when
hotplugging an (i)SCSI hostdev.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In preparation for instantiating (i)SCSI hostdevs via -blockdev,
refactor qemuBuildHostdevSCSICommandLine to use the new infrastructure
which will do it automatically.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Add convertor for creating qemuBlockStorageSourceAttachData which will
allow reusing the infrastructure which we have for attaching disks also
for hostdevs.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We want to instantiate hostdevs via -blockdev too. Add a separate
capability for them for a clean transition. The new capability will be
enabled when QEMU_CAPS_BLOCKDEV is present once all code is prepared.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Don't (re)generate the backend alias (alias of the -drive backend for
now) internally but rather pass it in. Later on it will be replaced by
the nodename when blockdev is used depending on the capabilities.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Move all (i)SCSI related code into a new function named
'qemuBuildHostdevSCSICommandLine'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We now store the alias of the secrets in the status XML so there's no
need to generate it again.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
When upgrading from a libvirt which didn't format private data of a
virStorageSource representing an iSCSI hostdev source, we might need to
generate some internal data so that the code still works as if it was
present in the status XML.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We need the alias to deal with hot-unplug of the hostdev. Use
qemuDomainSecretInfoDestroy which clears only the secrets and not the
alias. The same function is used also for handling disk secrets.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
We store the config of an iSCSI hostdev in a virStorageSource structure.
Parse the private data portion.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
iSCSI subsystem hostdevs store the data as a virStorageSource. Format
the private data part of the virStorageSource in the appropriate place.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
SCSI hostdevs don't have a virStorageSource associated with the backend
in certain cases. Adding a separate field to hold memory for a copy of
the nodename of the storage backend will allow reusing the blockdev
machinery also for SCSI hostdevs.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
It doesn't make sense to format "discard" when doing a -blockdev backend
of scsi-generic used with SCSI hostdevs. Add a way to skip it.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Upcoming commit will need to add another flag for the function so
convert it to a bitwise-or'd array of flags to prevent having 4
booleans.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Commit 82576d8f35 used a string "on" to enable the 'pmem' property.
This is okay for the command line visitor, but the property is declared
as boolean in qemu and thus it will not work when using QMP.
Modify the type to boolean. This changes the command line, but
fortunately the command line visitor in qemu parses both 'yes' and 'on'
as true for the property.
https://bugzilla.redhat.com/show_bug.cgi?id=1854684
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
The semantics of the backup operation don't strictly require that all
disks being backed up are part of the same incremental part (when a disk
was checkpointed/backed up separately or in a different VM), or even
they may not have a previous checkpoint at all (e.g. when the disk
was freshly hotplugged to the vm).
In such cases we can still create a common checkpoint for all of them
and backup differences according to configuration.
This patch adds a per-disk configuration of the checkpoint to do the
incremental backup from via the 'incremental' attribute and allows
perform full backups via the 'backupmode' attribute.
Note that no changes to the qemu driver are necessary to take advantage
of this as we already obey the per-disk 'incremental' field.
https://bugzilla.redhat.com/show_bug.cgi?id=1829829
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Call the post-processing function so that we can validate that it does
the correct thing.
virDomainBackupAlignDisks requires disk definitions to be present so
let's fake them by copying disks from the backup definition and add one
extra disk 'vdextradisk'.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Replace the output by a copy of the input file for further changes once
we start testing virDomainBackupAlignDisks.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Allow enabling TLS for the NBD server used to do pull-mode backups. Note
that documentation already mentions 'tls', so this just implements the
schema and XML bits.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
TLS is required to transport backed-up data securely when using
pull-mode backups.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add fields for storing the aliases necessary to clean up the TLS env for
a backup job after it finishes.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
There are few internal fields of the backup XML. Propagate the
'internal' flag so that the test can verify the XML infrastructure.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Data is valid only when queried as guest writes may increase the backup
size.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Introduce code which merges the appropriate bitmaps and queries the
final size of the backup, so that we can print the XML with size
information.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Avoid printing '0' size in case when we weren't able to determine the
backup size by adding a flag whether the size is valid and interlock
printing of the field according to the flag.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add a dummy secret so that we see what command line is generated.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Until now libvirt didn't allow using encrypted TLS key for disk clients.
Add fields for configuring the secret and propagate defaults.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add infrastructure for hot- and cold-plug of the secret object holding
decryption key for the TLS key.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Store the required data in the private data of a storage source and
ensure that the 'alias' of the secret is formatted in the status XML.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The '*_tls_x509_verify' options are relevant only when we are going to
expose a server socket as client sockets always enable verification.
Split up the macro to separate the common bits from the server bits so
that when we'll later extend support of 'nbd' and 'vxhs' disks which are
client only we can reuse the existing macros.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Move the extraction of the config value so that it makes more sense
after upcoming refactors.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
There's no such parameter. Reword the sentence to account for enabling
TLS-encrypted migration using API flags.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add a helper which will always return the storage source private data
even if it was not allocated before.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
There were two upstream issues filed for the problem so it's worth
mentioning.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Andrea Bolognani <abologna@redhat.com>
oVirt does merge images out of libvirt in some cases. Add docs outlining
how it's done from a high level.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Simplify the docs and reduce maintenance burden by just describing the
algorithm by a pseudo-language. Users are encouraged to use libvirt
anyways and projects such as oVirt which do some management of storage
themselves are unlikely to use bash anyways.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Define what users should look for when wanting to manipulate bitmaps
themselves.
Later on a patch will turn the bash algorithms into pseudocode for
simplicity.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add a section that outlines usage of tools to handle bitmaps and
introduce terms corresponding to the output of qemu-img to be used in
further sections.
With this we can simplify the section about checking bitmap health as we
don't have to explain the qemu-img output but can refer to the newly
defined terms.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Emphasize what needs to happen and also that creating a snapshot doesn't
create the appropriate bitmaps. Also mention that granularity is kept.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Make it obvious what's meant by 'overlay' and 'backing image' for sake
of extension of the document.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Integrate both 'disk-hostdev-scsi-virtio-iscsi-auth-AES' and
'hostdev-scsi-virtio-iscsi-auth' as the new test infrastructure tests
both legacy and 'secret' object cases.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We can add the iSCSI hostdevs to the same test file.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
qemu-2.8 didn't yet support QEMU_CAPS_ISCSI_PASSWORD_SECRET. This
version will allow integrating multiple test cases into one.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Modernize the current state to the pre-blockdev version of qemu to
minimize changes. Later patch will add a 'latest' case too.
Additionally this removes duplicated call of the same test.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
We can add the authenticated iSCSI hostdevs to the same test file.
Additionally this now covers passing secret via the 'secret' object
rather than on the command line.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Peter Krempa