Commit Graph
100 Commits
Author SHA1 Message Date
Peter Krempa d6db013c6e qemu: Pass through arguments of 'ssh' block driver used by libguestfs
We currently don't model the 'ssh' protocol properties properly and
since it seems impossible for now (agent path passed via environment
variable). To allow libguestfs to work as it used in pre-blockdev era we
must carry the properties over to the command line. For this instance we
just store it internally and format it back.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa d089234110 qemublocktest: Add JSON->JSON test cases for block device backends
Add testing of the interpretation of the JSON pseudo-protocol backing
store into JSON structs for blockdev. This will be used to test JSON
pseudo-URIs used by libguestfs while actually also validating the output
against the QMP schema. Since libguestfs uses obsolete/undocumented
values the outputs will differ and a benefit is that modern output is
used now.

The example test case covers the fields and values used by libguestfs
when using the https driver.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 7f8d0ca56a qemublocktest: XMLjsonXML: Test formatting/parsing of modern JSON
The test was invoking the JSON formatter with the 'legacy' flag thus
formatting bunch of obsolete JSON blockdev definitions. We also should
test the modern ones. Add a boolean and re-run all the tests in both
cases.

Additionally for any modern invocation we should also validate that the
output conforms to the QAPI schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 218ce53069 qemublocktest: Extract schema root for blockdev-add validation
Move lookup of the schema root earlier so that multiple functions
can use it for validation.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 1262cdede4 qemublocktest: Load QMP schema earlier
Multiple tests require the schema. Extract the loading into a separate
variable to avoid issues with ownership of the pointer.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 5179cc6b08 virStorageSourceParseBackingJSONUri: Handle undocumented value 'off' for sslverify
libguestfs abuses a quirk of qemu's parser to accept also other variants
of the 'sslverify' field which would be valid on the command line but
are not documented in the QMP schema.

If we encounter the 'off' string instead of an boolean handle it rather
than erroring out to continue support of pre-blockdev configurations.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 77194db01c virstoragefile: Add JSON parser for 'sslverify', 'readahead', 'cookies' and 'timeout'
Add support for parsing the recently added fields from backing file
pseudo-protocol strings.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 9cac141cd6 qemu: block: Implement readahead and timeout properties for 'curl' driver
Pass in the correct fields.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 06d3e8d539 qemu: block: Add support for HTTP cookies
Pass the alias of the secret object holding the cookie data as
'cookie-secret' to qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa b512935b17 qemu: Handle hotplug and commandline for secret objects for http cookies
Implement both commandline support and hotplug by adding the http cookie
handling to 'qemuBlockStorageSourceAttachData' handling functions for
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 877cd35887 qemuDomainSecretStorageSourcePrepare: Setup secret for http cookies
QEMU's curl driver requires the cookies concatenated and allows themi to
be passed in via a secret. Prepare the value for the secret and encrypt
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 4e8faa5cdc qemu: domain: Store data for 'secret' object representing http cookies
The http cookies can have potentially sensitive values and thus should
not be leaked into the command line. This means that we'll need to
instantiate a 'secret' object in qemu to pass the value encrypted.

This patch adds infrastructure for storing of the alias in the status
XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 304da9376c qemu: block: Implement ssl verification configuration
Allow disabling of SSL certificate validation for HTTPS and FTPS drives
in qemu.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 3dd7952f6f qemuxml2argvtest: Add test case for disks with http(s) source
Upcoming patches will implement the support for sslverify, cookies,
readahead, and timeout properties. Add a test file which will collect
the cases.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa c1409e308f qemuDomainValidateStorageSource: Validate new network storage parameters
Ensure that the new fields are allowed only when -blockdev is used or
when they are in the detected part of the backing chain where qemu will
handle them internally.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 63fd461773 conf: Add support for setting timeout and readahead size for network disks
Some disk backends support configuring the readahead buffer or timeout
for requests. Add the knobs to the XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 3b076391be conf: Add support for cookies for HTTP based disks
Add possibility to specify one or more cookies for http based disks.
This patch adds the config parser, storage and validation of the
cookies.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 25481e25b1 conf: Add support for modifying ssl validation for https/ftps disks
To allow turning off verification of SSL cerificates add a new element
<ssl> to the disk source XML which will allow configuring the validation
process using the 'verify' attribute.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 43a3d2e02e qemuDomainGetSecretAESAlias: Replace outstanding uses with qemuAliasForSecret
There are two last callers of this function. Replace them by
qemuAliasForSecret and delete qemuDomainGetSecretAESAlias.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 70d2758a9c qemuDomainSecretStorageSourcePrepare: Change aliases for disk secrets
Originally there was only the secret for authentication so we didn't use
any suffix to tell it apart. With the introduction of encryption we
added a 'luks' suffix for the encryption secrets. Since encryption is
really generic and authentication is not the only secret modify the
aliases for the secrets to better describe what they are used for.

This is possible as we store the disk secrets in the status XML thus
only new machines will use the new secrets.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 86fecaedf5 qemuDomainSecretAESSetupFromSecret: Use 'qemuAliasForSecret'
Replace qemuDomainGetSecretAESAlias by the new function so that we can
reuse qemuDomainSecretAESSetupFromSecret also for setting up other kinds
of objects.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 740dd1a4e5 qemu: Split out initialization of secrets for 'iscsi' hostdevs
Currently we don't have infrastructure to remember the secret aliases
for hostdevs. Since an upcoming patch is going to change aliases for
the disks, initialize the iscsi hostdevs separately so that we can keep
the alias. At the same time let's use qemuAliasForSecret instead of
qemuDomainGetSecretAESAlias when unplugging the iscsi hostdev.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa 95a8c4332d qemuDomainDeviceDiskDefPostParseRestoreSecAlias: Hardcode restored aliases
In order to be able to change the function generating the alias and thus
also the aliases itself, we must hardcode the old format for the case of
upgrading form libvirt which didn't record them in the status XML yet.

Note that this code path is tested by
'tests/qemustatusxml2xmldata/disk-secinfo-upgrade-in.xml'

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa b05322fc03 qemuDomainSecretStorageSourcePrepare: Fix naming of alias variables
The naming of the variables was tied to what they are used for not what
the alias represents. Since we'll need to use some of the aliases for
another type of secrets fix the name so that it makes sense.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa bb2a81df21 qemu: Introduce another helper for creating alias for a 'secret' object
qemuAliasForSecret is meant as a replacement qemuDomainGetSecretAESAlias
with saner API. The sub-type we are creating the alias for is passed in
as a string rather than the unflexible 'isLuks' boolean.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 15:51:44 +01:00
Peter Krempa f742461389 Remove qemuDomainSecretInfoNew
Replace it by a direct call to qemuDomainSecretAESSetupFromSecret.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 13:04:17 +01:00
Peter Krempa 88663e59ef qemuDomainSecretAESSetup: Split out lookup of secret data
Split out the lookup of the secret from the secret driver into
qemuDomainSecretAESSetupFromSecret so that we can also instantiate
secret objects in qemu with data from other sources.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 13:04:17 +01:00
Peter Krempa bad8637892 qemuDomainSecretAESSetup: Allocate and return 'secinfo' here
Rather than passing in an empty qemuDomainSecretInfoPtr allocate it
in this function and return it. This is done by absorbing the check from
qemuDomainSecretInfoNew and removing the internals of
qemuDomainSecretInfoNew.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 13:04:17 +01:00
Peter Krempa 88126d5f0e qemuDomainSecretAESSetup: Automatically free non-secret locals
Use g_autofree for the ciphertext and init vector as they are not
secret and thus don't have to be cleared and use g_new0 to allocate the
iv for parity.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 13:04:17 +01:00
Peter Krempa b544481a91 qemuDomainSecretInfo: Register autoptr cleanup function
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 13:04:17 +01:00
Peter Krempa 7a34e04d82 qemuMigrationParamsResetTLS: Fix comment
The comment mentioned that the function resets migration params, but
that is not true as of commit eb54cb473a

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 11:01:08 +01:00
Peter Krempa 3b61106550 qemuMigrationParamsResetTLS: Adapt to modern memory management
Use g_autofree instead of VIR_FREE and delete the comment mentioning
possible failure to allocate memory.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 11:01:08 +01:00
Peter Krempa 38bc76bcc1 qemu: Don't take double pointer in qemuDomainSecretInfoFree
Using a double pointer prevents the function from being used as the
automatic cleanup function for the given type.

Remove the double pointer use by replacing the calls with
g_clear_pointer which ensures that the pointer is cleared.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 11:01:08 +01:00
Peter Krempa 4415b11d6b qemuBlockStorageSourceDetachPrepare: Get rid of cleanup section
Use g_new0 to completely avoid the 'cleanup' label.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-16 11:01:08 +01:00
Peter Krempa 1cf2f412ed news: Mention regression in virDomainBlockCopy with shallow+reuse flags
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com>
2020-03-16 08:21:30 +01:00
Peter Krempa cc7868a8b3 qemu: blockcopy: Allow late opening of the backing chain of a shallow copy
oVirt used a quirk in the pre-blockdev semantics of drive-mirror which
opened the backing chain of the mirror destination only once
'block-job-complete' was called.

Our introduction of blockdev made qemu open the backing chain images
right at the start of the job. This broke oVirt's usage of this API
because they copy the data into the backing chain during the time the
block copy job is running.

Re-introduce late open of the backing chain if qemu allows us to use
blockdev-snapshot on write-only nodes as it can be used to install the
backing chain even for an existing image now.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa d6498be165 qemu: capabilities: Introduce QEMU_CAPS_BLOCKDEV_SNAPSHOT_ALLOW_WRITE_ONLY
The capability is based on qemu's support of using blockdev-snapshot to
install backing chain also for images which are in use by a block-copy
job.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa c2b6bc7377 qemuDomainBlockCopyCommon: Record updated flags to block job
For a long time we've masked out VIR_DOMAIN_BLOCK_COPY_SHALLOW if
there's no backing chain for the copied disk to simplify the code.

One of the refactors of the block copy code caused that we no longer
update the 'flags' variable just the local copies. This was okay until
in ccd4228aff we started storing the job flags in the block job data.

Given that we modify how we call qemu we also should modify @flags so
that the correct value is recorded in the block job data.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa a89ba6524c qemuDomainBlockPivot: Move check prior to executing the pivot steps
Move the check whether the job is already synchronised to the beginning
of the function so that we don't try to do some of the steps necessary
for pivoting prior to actually wanting to pivot.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa 7c7fda15f3 qemu: capabilities: Update qemu-5.0.0 capabilities for x86_64
Update to v4.2.0-2265-g67923a7ea6 to pick up recent addition of
'allow-write-only-overlay' feature of 'blockdev-snapshot' command.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-03-13 13:09:12 +01:00
Peter Krempa 0fdb7385e4 qemuDomainGetGuestInfo: Don't try to free a negative number of entries
'nfs' variable was set to -1 or -2 on agent failure. Cleanup then tried
to free 'nfs' elements of the array which resulted into a crash.

Make 'nfs' size_t and assign it only on successful agent call.

https://bugzilla.redhat.com/show_bug.cgi?id=1812965

Broken by commit 599ae372d8

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-13 08:48:42 +01:00
Peter Krempa da1b193227 qemuAgentFSInfoFormatParams: Remove pointless returned value
The only caller doesn't check the value and also there are no real
errors to report anyways.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-13 08:48:42 +01:00
Peter Krempa 09352cca2a VIR_FREE: Replace internals by g_clear_pointer
Our implementation masks GCC warnings of uninitialized use of the passed
argument. After changing this I got a load of following warnings:

src/conf/virnetworkportdef.c: In function 'virNetworkPortDefSaveStatus':
/usr/include/glib-2.0/glib/gmem.h:136:8: error: 'path' may be used uninitialized in this function [-Werror=maybe-uninitialized]
  136 |     if (_p)                \
      |        ^
src/conf/virnetworkportdef.c:447:11: note: 'path' was declared here
  447 |     char *path;
      |           ^~~~

For the curious, g_clear_pointer is still safe for arguments with
side-effect. Here's the pre-processed output of trying to do a
VIR_FREE(*(test2++)):

 do {
     typedef char _GStaticAssertCompileTimeAssertion_1[(sizeof *(&(*(test2++))) == sizeof (gpointer)) ? 1 : -1] __attribute__((__unused__));
     __typeof__((&(*(test2++)))) _pp = (&(*(test2++)));
     __typeof__(*(&(*(test2++)))) _ptr = *_pp;

     *_pp = ((void *)0);
     if (_ptr)
        (g_free) (_ptr);
 } while (0) ;

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-05 16:22:27 +01:00
Peter Krempa 8a2390f640 testQemuMonitorJSONqemuMonitorJSONGetTargetArch: Fix uninitialized use of 'arch'
Refactor the cleanup control flow and use g_autofree for 'arch' so that
it's mandated that it's initialized.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-05 16:21:47 +01:00
Peter Krempa 1a53b2baab cmdDomHostname: Fix uninitialized use of 'hostname' by refactoring cleanup
Use 'g_autoptr' which mandates initialization for 'hostname' and also
for 'domain' to allow full refactor of the cleanup path.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-05 16:21:47 +01:00
Peter Krempa 140c756f5c virLXCProcessSetupNamespaceName: Fix potential uninitialized free of 'path'
'path' could be accessed uninitialized. Fix it by using g_autofree which
also mandates initialization.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-05 16:21:47 +01:00
Peter Krempa 78ead2529b virNetworkPortDefSaveStatus: Fix potentially uninitialized 'path' by refactoring cleanup
Use 'g_autofree' to clean both 'path' and 'xml' which mandates
initialization and get rid of the 'cleanup' label and 'ret variable.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-05 16:21:47 +01:00
Peter Krempa 7b62dfc737 qemuhotplugtestcpus: Always use 'query-cpus-fast'
Use the new command in the test suite by asserting the capability
and adjusting test data to the correct field names as they changed
compared to 'query-cpus'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-03-05 11:31:38 +01:00
Peter Krempa b5d8d4b025 qemuMonitorJSONSetMigrationParams: Refactor command construction and cleanup
qemuMonitorJSONMakeCommandInternal does the full command construction if
you pass in what would become the value of the 'arguments' key. Refactor
the open-coded implementation to use the helper and use modern cleanup
helpers at the same time.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-03-05 11:31:38 +01:00
Peter Krempa e9153cc604 util: json: Convert virJSONValueNewObject() to g_new0
Make it obvious that the function always returns a valid pointer and fix
all callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-03-05 11:31:38 +01:00
Peter Krempa 95080cc8b4 qemu: Don't request nested entries in qemuBlockGetNamedNodeData
Use the 'flat' flag for 'query-named-block-nodes' if qemu supports
QEMU_CAPS_QMP_QUERY_NAMED_BLOCK_NODES_FLAT in qemuBlockGetNamedNodeData.

We don't need the data so plumb in whether qemu supports the
'flat' output.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:24 +01:00
Peter Krempa 855211bbf3 qemu: monitor: Add 'flat' parameter for qemuMonitorJSONQueryNamedBlockNodes
Modern qemu allows to skip the nested redundant data in the output of
query-named-block-nodes. Plumb in the support for the argument that
enables it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:24 +01:00
Peter Krempa 63610bd5fb qemuCheckpointDiscardBitmaps: Use qemuBlockGetNamedNodeData
Replace qemuMonitorBlockGetNamedNodeData by qemuBlockGetNamedNodeData.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:24 +01:00
Peter Krempa f886c9f330 qemu: monitor: Refactor variable cleanup in qemuMonitorJSONQueryNamedBlockNodes
Use g_autoptr to get rid of the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:24 +01:00
Peter Krempa b7991c903c qemu: capabilities: Add capability for the 'flat' argument of 'query-named-block-nodes'
Detect the presence of the flag and make it available internally as
QEMU_CAPS_QMP_QUERY_NAMED_BLOCK_NODES_FLAT.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:24 +01:00
Peter Krempa 7892748350 qemu: monitor: Remove leftovers from password callback
The monitor password callback was removed long time ago but the callback
type and variable were left around. Finish the cleanup.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:18 +01:00
Peter Krempa 7680f3369a tests: qemucapabilities: Update capabilities of qemu-5.0.0 on x86_64
Update to v4.2.0-1858-gdb736e0437 which contains my commit for 'flat'
output of 'query-named-block-nodes'.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-03-04 14:39:18 +01:00
Peter Krempa 0b0907316d news: Document recent storage improvements
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-27 12:06:25 +01:00
Peter Krempa 82d5b762f1 kbase: backing_chains: Add steps how to securely probe image format
We document steps how to fix images if they are rejected for missing
the 'backing file format' field. Document also how to securely probe
the image format if it's unknown.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-27 12:06:25 +01:00
Peter Krempa d552b93448 kbase: backing_chains: Clarify some aspects of image probing
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-25 21:14:27 +01:00
Peter Krempa ae9e6c2a2b virStorageFileGetMetadataRecurse: Allow format probing under special circumstances
Allow format probing to work around lazy clients which did not specify
their format in the overlay. Format probing will be allowed only, if we
are able to probe the image, the probing result was successful and the
probed image does not have any backing or data file.

This relaxes the restrictions which were imposed in commit 3615e8b39b
in cases when we know that the image probing will not result in security
issues or data corruption.

We perform the image format detection and in the case that we were able
to probe the format and the format does not specify a backing store (or
doesn't support backing store) we can use this format.

With pre-blockdev configurations this will restore the previous
behaviour for the images mentioned above as qemu would probe the format
anyways. It also improves error reporting compared to the old state as
we now report that the backing chain will be broken in case when there
is a backing file.

In blockdev configurations this ensures that libvirt will not cause data
corruption by ending the chain prematurely without notifying the user,
but still allows the old semantics when the users forgot to specify the
format.

Users thus don't have to re-invent when image format detection is safe
to do.

The price for this is that libvirt will need to keep the image format
detector still current and working or replace it by invocation of
qemu-img.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-25 21:14:27 +01:00
Peter Krempa 3c6e6f55a5 qemu: domain: Convert detected 'iso' image format into 'raw'
While our code can detect ISO as a separate format, qemu does not use it
as such and just passes it through as raw. Add conversion for detected
parts of the backing chain so that the validation code does not reject
it right away.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-02-25 21:14:27 +01:00
Peter Krempa c95656c995 virStorageFileGetMetadataFromFD: Remove unused 'backingFormat' argument
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa 57df35aead virStorageFileGetMetadataFromBuf: Remove 'backingFormat' argument
None of the callers actually use it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa fee56942e2 virStorageBackendGlusterRefreshVol: Refactor handling of backing store
Take the format of the backing store from the 'meta' object directly and
use g_steal_pointer to steal the path.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa 264b79c63a virStorageSourceNewFromBacking: Also transfer the format
When we create the new virStorageSource from the definitions stored in
the parent we should also use the 'backingStoreRawFormat' field to
populate the format.

Callers which use virStorageSourceNewFromBacking are also fixed to stop
setting the format manually.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa 62539c5f7d util: storage: Store backing store format in virStorageSource
We store the backing file string in the structure so we should also
store the format so that callers can be simplified.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa 35d1f5bd14 virStorageSourceUpdateCapacity: Drop 'probe' argument
Both callers pass false. Since we frown upon format probing, remove the
unused possibility to do the probing.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa e5c8f6e080 tests: virstorage: Fix backing file format of created image
We create some images for testing our code. We've recorded wrong format
of the backing file for one of the images though.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-24 15:12:32 +01:00
Peter Krempa a570dc6767 virStorageFileGetMetadataRecurse: Remove 'cleanup' label
There's nothing to clean up. Make it obvious what is returned.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa 01adad0932 virStorageFileGetMetadataRecurse: Extract storage access
Extract the code that directly deals with storage. This allows further
simplification and clarification of virStorageFileGetMetadataRecurse.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa e3960f4b6d virStorageFileGetMetadataRecurse: Use virHashHasEntry instead of fake pointers
Replacing virHashLookup by virHashHasEntry allows us to use NULL as the
payload of the hash table rather than putting a fake '1' pointer into
the table.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:09 +01:00
Peter Krempa 157b8722cb virStorageFileGetMetadataRecurse: Expect NULL src->path
The path can be NULL e.g. for NBD disks. Use NULLSTR to prevent use of
NULL in %s.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa b347e5c7dd virStorageFileGetMetadataRecurse: Shuffle around assignment of backing chain depth
Move the assignment to a place where we know that the backing store is
present rather than having to check in the cleanup section.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa 84df98f29e virStorageFileGetMetadataRecurse: Remove impossible error report
We call virStorageFileSupportsBackingChainTraversal which already checks
that the 'storageFileRead' callback is non-NULL, which in turn means
that virStorageFileRead will not return -2.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa 181fccc2ed util: storagefile: Drop image format probing by file suffix
Probing by file suffix was meant to be a last resort if probing by
contents fails or is not supported. For most formats we never specified
any suffix. There's a few formats implementing both magic bytes and
suffix and finally DMG which had only suffix probing. Since suffix
probing is nowhere reliable and only one format depends on in which has a
comment that qemu doesn't do the probing either drop the whole
infrastructure.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-02-20 07:57:08 +01:00
Peter Krempa 9bf9e0ae6a qemuDomainGetStatsIOThread: Don't leak array with 0 iothreads
qemuMonitorGetIOThreads returns a NULL-terminated list even when 0
iothreads are present. The caller didn't perform cleanup if there were 0
iothreads leaking the array.

https://bugzilla.redhat.com/show_bug.cgi?id=1804548

Fixes: d1eac92784
Reported-by: Jing Yan <jiyan@redhat.com>
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-19 10:24:31 +01:00
Peter Krempa e8a819e87f virStorageSourceParseBackingJSONRaw: Parse 'offset' and 'size' attributes
If the parsed 'raw' format JSON string has 'offset' or 'size' attributes
parse them as the format slice.

https://bugzilla.redhat.com/show_bug.cgi?id=1791788

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 293e7750c9 tests: qemu: Add test data for the new <slice> element
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 0e644e6e47 qemu: Add support for slices of type 'storage'
Implement support for the slice of type 'storage' which allows to set
the offset and size which modifies where qemu should look for the start
of the format container inside the image.

Since slicing is done using the 'raw' driver we need to add another
layer into the blockdev tree if there's any non-raw image format driver
used to access the data.

This patch adds the blockdev integration and setup of the image data so
that we can use the slices for any backing image.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 9b804ef5ef tests: qemublock: Add cases for creating image overlays on top of disks with <slice>
Add a set of test data to see whether the backing store strings are
formatted reasonably. Note that we don't support direct creation of such
images so those tests are not enabled.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 73ca201467 qemu: block: Properly format storage slice into backing store strings
When creating overlay images e.g. for snapshots or when merging
snapshots we often specify the backing store string to use. Make the
formatter aware of backing chain entries which have a <slice>
configured so that we record it properly. Otherwise such images
would not work without the XML (when detecting the backing chain).

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa f36d751fa6 qemu: domain: Store nodenames of slice in status XML
The storage slice will require a specific node name in cases when the
image format is not raw. Store and format them in the status XML.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa bbf5d05cfd conf: Implement support for <slices> of disk source
Implement parsing and formatting of the 'storage' slice.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 44f0f76890 docs: Document the new <slices> sub-element of disk's <source>
We are going to add support for specifying offset and size attributes
which will allow controling where the image and where the guest data
itself starts in the source of the disk. This will be represented by
a <slices> element filled with either a <slice type='storage'> for the
offset of the image format data.

Add the XML documentation and RNG schema.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 8c43037688 qemu: block: forbid creation of storage sources with <slice>
Specifically creating such images via libvirt during blockjobs would
be much more hassle than it's worth. Just forbid them for now.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa a6eeda986e qemuDomainValidateStorageSource: Reject unsupported slices
We support explicit storage slices only when using blockdev. Storage
slices expressed via the backing store string are left to qemu to
open correctly.

Reject storage slices configured via the XML for non-blockdev usage.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa c481881283 qemuBlockStorageSourceGetFormatRawProps: format 'offset' and 'size' for slice
If we have a 'format' type slice for a raw driver we can directly format
the values.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 6efa046165 util: virstoragefile: Add data structure for storing storage source slices
Introduce virStorageSourceSlice which will store the 'offset' and 'size'
of a virStorageSource and declare it as 'sliceStorage' and 'sliceFormat'
attributes of a virStorageSource.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 554ae62637 tests: virstorage: Add test data for json specified raw image with offset/size
QEMU allows specifying the offset and size into a raw file to expose a
sub-slice of the image to the guest with the raw driver. Libvirt
currently doesn't support it but we can add test case for future
reference.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 4e93c47576 docs: formatdomain: Close <source> on one of disk examples
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 9fb7ccb3cf qemu: domain: Refactor formatting of node names into status XML
Use virXMLFormatElement to simplify the logic.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-14 16:32:21 +01:00
Peter Krempa 0d0d60ddc5 tests: virstorage: Add test cases for "json:" pseudo-URI without 'file' wrapper
Add few cases that prove the second format of "json:" pseudo-URIs.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa fd70f1b4d3 virStorageSourceParseBackingJSON: Prevent arbitrary nesting with format drivers
Since we parse attributes for 'raw' which is a format driver and thus
has nested 'file' structure we must prevent that this isn't nested
arbitrarily.

Add a flag for the function which allows parsing of 'format' type
drivers only on the first pass.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa f8e097570e virStorageSourceParseBackingJSON: Allow 'json:' pseudo URIs without 'file' wrapper
There are two possibilities:
1) json:{"file":{"driver":...}}
2) json:{"driver":...}

Our code didn't work properly with the second one as it was expecting
the 'file' wrapper. Conditionalize the removal to only the situation
when the top level doesn't have "driver".

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa 7e13ff8dc0 virStorageSourceJSONDriverParser: annotate 'format' drivers
The parser was originally designed only for protocol parsers. Since
we already have 'raw' format driver in the list we'll need to be able
to parse it too. In later patches this will be used to prevent parsing
nested format drivers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa aadb34be34 virStorageSourceParseBackingJSON: Move deflattening of json: URIs out of recursion
Originally virStorageSourceParseBackingJSON didn't recurse, but when
the 'raw' driver support was added we need to parse it's information
which contains nested 'file' object.

Since the deflattening helper recurses already there's no need to call
it again. Move it one level up to the entry point.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa 4a6bc568cd virStorageSourceParseBackingJSON: Pass around original backing file string
There are a few error messages which might want to report the original
backing store string. Pass it around rather than trying to re-generate
it.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
2020-02-10 17:26:27 +01:00
Peter Krempa b71cf8726c qemu: hotplug: Fix handling of the 'copy-on-read' layer with blockdev
My original implementation was completely broken because it attempted to
use object-add/del instead of blockdev-add/del.

https://bugzilla.redhat.com/show_bug.cgi?id=1798366

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00
Peter Krempa db57e9daf5 qemuMonitorBlockdevAdd: Take double pointer argument
Modify qemuMonitorBlockdevAdd so that it takes a double pointer for the
@props argument so that it's cleared inside the call. This allows
writing cleaner callers.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2020-02-10 17:26:27 +01:00