Commit Graph
100 Commits
Author SHA1 Message Date
John Ferlan 40ec4ff658 storage: Alter error message in probe/empty checks
For case VIR_STORAGE_BLKID_PROBE_DIFFERENT, clean up the message to
avoid using the virsh like --overwrite syntax. Additionally provide
a different error message when not writing the label to avoid confusion.
2017-01-14 10:13:05 -05:00
John Ferlan f462f9ad6e storage: Clean up return value checking
Rather than special casing the VIR_STORAGE_BLKID_PROBE_UNKNOWN after
calling virStorageBackendBLKIDFindPart, just allow the switch statement
handle setting ret = -2.
2017-01-14 10:13:05 -05:00
John Ferlan d1f5dfc416 storage: Alter logic when both BLKID and PARTED unavailable
If neither BLKID or PARTED is available and we're not writing, then
just return 0 which allows the underlying storage pool to generate
a failure. If both are unavailable and we're writing, then generate
a more generic error message.
2017-01-14 10:13:05 -05:00
John Ferlan 7b837d6477 docs: Add logical storage pool overwrite change description to news.xml
Add "New Features" entry to describe the overwrite flag for logical backend.
2017-01-11 11:27:03 -05:00
John Ferlan 039d9762a0 docs: Add file system pool overwrite change description to news.xml
Add bug fixes description of overwrite changes for a file system storage pool
2017-01-11 11:27:03 -05:00
John Ferlan 6e032c3c48 docs: Add NPIV/vHBA change description to news.xml
Add "Improvements" for commit id 'bb74a7ffe' and '78be2e8b7' which add
support for using the parent wwnn/wwpn or fabric_name rather than just
using the parent by scsi_hostX name.
2017-01-11 11:27:03 -05:00
John Ferlan 34e22af614 docs: Document the new vHBA/NPIV params for storage
Commit id 'bb74a7ffe' forgot to adjust the storage docs to describe the
new fields.
2017-01-11 11:27:03 -05:00
John Ferlan bdd371c5c5 storage: Fix storage_backend probing when PARTED not installed.
Commit id 'a48c674f' caused problems for systems without PARTED installed.

So move the PARTED probing code back to storage_backend_disk.c and create
a shim within storage_backend.c to call it if WITH_STORAGE_DISK is true;
otherwise, just return -1 with the error.
2017-01-10 10:20:17 -05:00
John Ferlan cb38b6cbc7 storage: Validate the device formats at logical startup
At startup time, rather than blindly trusting the target devices are
still properly formatted, let's check to make sure the pool's target
devices are all properly formatted before attempting to start the pool.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan f573f84eb7 storage: Add overwrite flag checking for logical pool
https://bugzilla.redhat.com/show_bug.cgi?id=1373711

Add support and documentation for the [NO_]OVERWRITE flags for the
logical backend.

Update virsh.pod with a description of the process for usage of
the flags and building of the pool's volume group.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan d5cc5f8997 storage: Extract logical device initialize into a helper
Make the remaining code a bit cleaner.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan 71a08b5a5a storage: Clean up logical pool devices on build failure
If the build fails, then we need to ensure that we've run pvremove
on any devices which we've run pvcreate on; otherwise, a subsequent
build could fail since running pvcreate twice on a device requires
special force arguments.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan a4cb4a74f9 storage: Adjust disk label found to match labels
Currently as long as the disk is formatted using a known parted format
type, the algorithm is happy to continue. However, that leaves a scenario
whereby a disk formatted using "pc98" could be used by a pool that's defined
using "dvh" (or vice versa). Alter the check to be match and different
and adjust the caller.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan a48c674fba storage: Move and rename disk backend label checking
Rather than have the Disk code having to use PARTED to determine if
there's something on the device, let's use the virStorageBackendDeviceProbe.
and only fallback to the PARTED probing if the BLKID code isn't built in.

This will also provide a mechanism for the other current caller (File
System Backend) to utilize a PARTED parsing algorithm in the event that
BLKID isn't built in to at least see if *something* exists on the disk
before blindly trying to use. The PARTED error checking will not find
file system types, but if there is a partition table set on the device,
it will at least cause a failure.

Move virStorageBackendDiskValidLabel and virStorageBackendDiskFindLabel
to storage_backend and rename/rework the code to fit the new model.

Update the virsh.pod description to provide a more generic description
of the process since we could now use either blkid or parted to find
data on the target device.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan a11fd69735 storage: For FS pool check for properly formatted target volume
Prior to starting up, let's be sure the target volume device is
formatted as we expect; otherwise, inhibit the start.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan 19ced38f1c storage: Add writelabel bool for virStorageBackendDeviceProbe
It's possible that the API could be called from a startup path in
order to check whether the label on the device matches what our
format is. In order to handle that condition, add a 'writelabel'
boolean to the API in order to indicate whether a write or just
read is about to happen.

This alters two "error" conditions that would care about knowing.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan a22e1a0032 storage: Add partition type checks for BLKID probing
A device may be formatted using some sort of disk partition format type.
We can check that using the blkid_ API's as well - so alter the logic to
allow checking the device for both a filesystem and a disk partition.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan f23d4bbce3 storage: Fix implementation of no-overwrite for file system backend
https://bugzilla.redhat.com/show_bug.cgi?id=1363586

Commit id '27758859' introduced the "NO_OVERWRITE" flag check for
file system backends; however, the implementation, documentation,
and algorithm was inconsistent. For the "flag" description for the
API the flag was described as "Do not overwrite existing pool";
however, within the storage backend code the flag is described
as "it probes to determine if filesystem already exists on the
target device, renurning an error if exists".

The code itself was implemented using the paradigm to set up the
superblock probe by creating a filter that would cause the code
to only search for the provided format type. If that type wasn't
found, then the algorithm would return success allowing the caller
to format the device. If the format type already existed on the
device, then the code would fail indicating that the a filesystem
of the same type existed on the device.

The result is that if someone had a file system of one type on the
device, it was possible to overwrite it if a different format type
was specified in updated XML effectively trashing whatever was on
the device already.

This patch alters what NO_OVERWRITE does for a file system backend
to be more realistic and consistent with what should be expected when
the caller requests to not overwrite the data on the disk.

Rather than filter results based on the expected format type, the
code will allow success/failure be determined solely on whether the
blkid_do_probe calls finds some known format on the device. This
adjustment also allows removal of the virStoragePoolProbeResult
enum that was under utilized.

If it does find a formatted file system different errors will be
generated indicating a file system of a specific type already exists
or a file system of some other type already exists.

In the original virsh support commit id 'ddcd5674', the description
for '--no-overwrite' within the 'pool-build' command help output
has an ambiguous "of this type" included in the short description.
Compared to the longer description within the "Build a given pool."
section of the virsh.pod file it's more apparent that the meaning
of this flag would cause failure if a probe of the target already
has a filesystem.

So this patch also modifies the short description to just be the
antecedent of the 'overwrite' flag, which matches the API description.
This patch also modifies the grammar in virsh.pod for no-overwrite
as well as reworking the paragraph formats to make it easier to read.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan 553d21da6c storage: Introduce virStorageBackendDeviceIsEmpty
Rename virStorageBackendFileSystemProbe and to virStorageBackendBLKIDFindFS
and move to the more common storage_backend module.

Create a shim virStorageBackendDeviceIsEmpty which will make the call
to the virStorageBackendBLKIDFindFS and check the return value.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-10 08:44:50 -05:00
John Ferlan 78be2e8b74 iscsi: Add parent wwnn/wwpn or fabric capability for createVport
https://bugzilla.redhat.com/show_bug.cgi?id=1349696

As it turns out using only the 'parent' to achieve the goal of a
consistent vHBA parent has issues with reboots where the scsi_hostX
parent could change to scsi_hostY causing either failure to create
the vHBA or usage of the wrong HBA for our vHBA.

Thus add the ability to search for the "parent" by the parent wwnn/
wwpn values or just a fabric_name if someone only cares to ensure
usage of the same SAN for the vHBA.
2017-01-06 17:15:34 -05:00
John Ferlan 8366cb0a20 util: Introduce virGetFCHostNameByFabricWWN
Create a utility routine in order to read the scsi_host fabric_name files
looking for a match to a passed fabric_name
2017-01-06 17:15:34 -05:00
John Ferlan bb74a7ffeb conf: Add more fchost search fields for storage pool vHBA creation
Add new fields to the fchost structure to allow creation of a vHBA via
the storage pool when a parent_wwnn/parent_wwpn or parent_fabric_wwn is
supplied in the storage pool XML.
2017-01-06 17:15:34 -05:00
John Ferlan 2b13361bc7 nodedev: Add the ability to create vHBA by parent wwnn/wwpn or fabric_wwn
https://bugzilla.redhat.com/show_bug.cgi?id=1349696

When creating a vHBA, the process is to feed XML to nodeDeviceCreateXML
that lists the <parent> scsi_hostX to use to create the vHBA. However,
between reboots, it's possible that the <parent> changes its scsi_hostX
to scsi_hostY and saved XML to perform the creation will either fail or
create a vHBA using the wrong parent.

So add the ability to provide "wwnn" and "wwpn" or "fabric_wwn" to
the <parent> instead of a name of the scsi_hostN that is the parent.
The allowed XML will thus be:

  <parent>scsi_host3</parent>  (current)

or

  <parent wwnn='$WWNN' wwpn='$WWPN'/>

or

  <parent fabric_wwn='$WWNN'/>

Using the wwnn/wwpn or fabric_wwn ensures the same 'scsi_hostN' is
selected between hardware reconfigs or host reboots. The fabric_wwn
Using the wwnn/wwpn pair will provide the most specific search option,
while fabric_wwn will at least ensure usage of the same SAN, but maybe
not the same scsi_hostN.

This patch will add the new fields to the nodedev.rng for input purposes
only since the input XML is essentially thrown away, no need to Format
the values since they'd already be printed as part of the scsi_host
data block.

New API virNodeDeviceGetParentHostByWWNs will take the parent "wwnn" and
"wwpn" in order to search the list of devices for matching capability
data fields wwnn and wwpn.

New API virNodeDeviceGetParentHostByFabricWWN will take the parent "fabric_wwn"
in order to search the list of devices for matching capability data field
fabric_wwn.
2017-01-06 17:14:12 -05:00
John Ferlan 1d0fde7ee1 util: Remove need for extra VIR_FREE's in virGetFCHostNameByWWN
Rather than extraneous VIR_FREE's depending on where we are in the code,
move them to the top of the loop and in the cleanup path.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 9fdc8c4269 scsi: Converge more createVport checks
Remove duplicated code - make one simple path through

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 476ecf2a2a scsi: Change order of checks in createVport
Move the check for an already existing vHBA to the top of the function.
No sense in first decoding a provided parent if the next thing we're going
to do is fail if a provided wwnn/wwpn already exists.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 79ab093518 scsi: Clean up createVport exit paths
Use the ret = -1, goto cleanup, etc. rather than current hodgepodge.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 8b629a3c01 nodedev: Add ability to find a vport capable vHBA
If a <parent> is not supplied in the XML used to create a non-persistent
vHBA, then instead of failing, let's try to find a "vports" capable node
device and use that.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 8f3054a0f8 nodedev: Create helpers to search for vport capable nodedevs
Extract out code from virNodeDeviceGetParentHost into helpers - it's
going to be reused in upcoming patches to search on more fields

Create virNodeDeviceFindVPORTCapDef in order to return a virNodeDevCapsDefPtr
of the VPORT_OPS and virNodeDeviceFindFCParentHost to use the function and
generate an error message if the device doesn't have the capability.

Also clean up the processing in virNodeDeviceGetParentHost to remove
need for goto's.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2017-01-04 17:09:59 -05:00
John Ferlan 7f7d990483 qemu: Don't assume secret provided for LUKS encryption
https://bugzilla.redhat.com/show_bug.cgi?id=1405269

If a secret was not provided for what was determined to be a LUKS
encrypted disk (during virStorageFileGetMetadata processing when
called from qemuDomainDetermineDiskChain as a result of hotplug
attach qemuDomainAttachDeviceDiskLive), then do not attempt to
look it up (avoiding a libvirtd crash) and do not alter the format
to "luks" when adding the disk; otherwise, the device_add would
fail with a message such as:

   "unable to execute QEMU command 'device_add': Property 'scsi-hd.drive'
    can't find value 'drive-scsi0-0-0-0'"

because of assumptions that when the format=luks that libvirt would have
provided the secret to decrypt the volume.

Access to unlock the volume will thus be left to the application.
2017-01-03 12:59:18 -05:00
John Ferlan 67882e56d1 docs: Update docs for recently added physical adjustments
Add the description for <physical>... One is a new feature - the API
and the other is an improvement for the volume xml output.
2016-12-22 18:33:32 -05:00
John Ferlan f62e418c86 virsh: Allow display of the physical volume size
Add a new qualifier '--physical' to the 'vol-info' command in order to
dispaly the physical size of the volume. The size can differ from the
allocation value depending on the volume file time. In particular, qcow2
volumes will have a physical value larger than allocation. This also occurs
for sparse files, although for those the capacity is the largest size;
whereas, for qcow2 capacity is the logical size.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-20 13:54:03 -05:00
John Ferlan 0c234889c4 storage: Introduce virStorageVolInfoFlags
https://bugzilla.redhat.com/show_bug.cgi?id=1332019

This function will essentially be a wrapper to virStorageVolInfo in order
to provide a mechanism to have the "physical" size of the volume returned
instead of the "allocation" size. This will provide similar capabilities to
the virDomainBlockInfo which can return both allocation and physical of a
domain storage volume.

NB: Since we're reusing the _virStorageVolInfo and not creating a new
_virStorageVolInfoFlags structure, we'll need to generate the rpc APIs
remoteStorageVolGetInfoFlags and remoteDispatchStorageVolGetInfoFlags
(although both were originally created from gendispatch.pl and then
just copied into daemon/remote.c and src/remote/remote_driver.c).

The new API will allow the usage of a VIR_STORAGE_VOL_GET_PHYSICAL flag
and will make the decision to return the physical or allocation value
into the allocation field.

In order to get that physical value, virStorageBackendUpdateVolTargetInfoFD
adds logic to fill in physical value matching logic in qemuStorageLimitsRefresh
used by virDomainBlockInfo when the domain is inactive.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-20 13:52:39 -05:00
John Ferlan 78661cb1f4 conf: Display <physical> in output of voldef
Although the virStorageBackendUpdateVolTargetInfo will update the
target.physical value, there is no way to provide that information
via the virStorageGetVolInfo API since it only returns the capacity
and allocation of a volume. So as described in commit id '0282ca45',
it should be possible to generate an output only <physical> value
for that purpose.

This patch generates the <physical> value in the volume XML output
for the sole purpose of being able to view/see the value to allow
someone to parse the XML in order to obtain the value.

Update the documentation to describe the output only nature.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-20 13:52:39 -05:00
John Ferlan b9b1aa6392 qemu: Adjust qemuDomainGetBlockInfo data for sparse backed files
According to commit id '0282ca45a' the 'physical' value should
essentially be the last offset of the image or the host physical
size in bytes of the image container. However, commit id '15fa84ac'
refactored the GetBlockInfo to use the same returned data as the
GetStatsBlock API for an active domain. For the 'entry->physical'
that would end up being the "actual-size" as set through the
qemuMonitorJSONBlockStatsUpdateCapacityOne (commit '7b11f5e5').
Digging deeper into QEMU code one finds that actual_size is
filled in using the same algorithm as GetBlockInfo has used for
setting the 'allocation' field when the domain is inactive.

The difference in values is seen primarily in sparse raw files
and other container type files (such as qcow2), which will return
a smaller value via the stat API for 'st_blocks'. Additionally
for container files, the 'capacity' field (populated via the
QEMU "virtual-size" value) may be slightly different (smaller)
in order to accomodate the overhead for the container. For
sparse files, the state 'st_size' field is returned.

This patch thus alters the allocation and physical values for
sparse backed storage files to be more appropriate to the API
contract. The result for GetBlockInfo is the following:

 capacity: logical size in bytes of the image (how much storage
           the guest will see)
 allocation: host storage in bytes occupied by the image (such
             as highest allocated extent if there are no holes,
             similar to 'du')
 physical: host physical size in bytes of the image container
           (last offset, similar to 'ls')

NB: The GetStatsBlock API allows a different contract for the
values:

 "block.<num>.allocation" - offset of the highest written sector
                            as unsigned long long.
 "block.<num>.capacity" - logical size in bytes of the block device
                          backing image as unsigned long long.
 "block.<num>.physical" - physical size in bytes of the container
                          of the backing image as unsigned long long.
2016-12-20 12:56:44 -05:00
John Ferlan 7ed89d7421 docs: Update formatstorage to match code for pool path
Seems commit id '0257d06b' forgot to include formatstorage when updating
the docs to describe allowing zfs as a pool type and to furthermore note
that the pool's target path element will be generated rather than read.

Similarly commit id 'efab27afb' neglected to indicate that the target path
for a logical pool will now be generated by libvirt.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-13 15:17:10 -05:00
John Ferlan d2c1222627 news: Add news entry for qemuDomainGetBlockInfo adjustments 2016-12-12 16:04:17 -05:00
John Ferlan cf436a560d qemu: Fix GetBlockInfo setting allocation from wr_highest_offset
The libvirt-domain.h documentation indicates that for a qcow2 file
in a filesystem being used for a backing store should report the disk
space occupied by a file; however, commit id '15fa84ac' altered the
code to trust that the wr_highest_offset should be used whenever
wr_highest_offset_valid was set.

As it turns out this will lead to indeterminite results. For an active
domain when qemu hasn't yet had the need to find the wr_highest_offset
value, qemu will report 0 even though qemu-img will report the proper
disk size. This causes reporting of the following XML:

  <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source file='/path/to/test-1g.qcow2'/>

to be as follows:

Capacity:       1073741824
Allocation:     0
Physical:       1074139136

with qemu-img indicating:

image: /path/to/test-1g.qcow2
file format: qcow2
virtual size: 1.0G (1073741824 bytes)
disk size: 1.0G

Once the backing source file is opened on the guest, then wr_highest_offset
is updated, but only to the high water mark and not the size of the file.

This patch will adjust the logic to check for the file backed qcow2 image
and enforce setting the allocation to the returned 'physical' value, which
is the 'actual-size' value from a 'query-block' operation.

NB: The other consumer of the wr_highest_offset output (GetAllDomainStats)
has a contract that indicates 'allocation' is the offset of the highest
written sector, so it doesn't need adjustment.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan 9d734b60a7 util: Introduce virStorageSourceUpdateCapacity
Instead of having duplicated code in qemuStorageLimitsRefresh and
virStorageBackendUpdateVolTargetInfo to get capacity specific data
about the storage backing source or volume -- create a common API
to handle the details for both.

As a side effect, virStorageFileProbeFormatFromBuf returns to being
a local/static helper to virstoragefile.c

For the QEMU code - if the probe is done, then the format is saved so
as to avoid future such probes.

For the storage backend code, there is no need to deal with the probe
since we cannot call the new API if target->format == NONE.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan 3039ec962e util: Introduce virStorageSourceUpdateBackingSizes
Instead of having duplicated code in qemuStorageLimitsRefresh and
virStorageBackendUpdateVolTargetInfoFD to fill in the storage backing
source or volume allocation, capacity, and physical values - create a
common API that will handle the details for both.

The common API will fill in "default" capacity values as well - although
those more than likely will be overridden by subsequent code. Having just
one place to make the determination of what the values should be will
make things be more consistent.

For the QEMU code - the data filled in will be for inactive domains
for the GetBlockInfo and DomainGetStatsOneBlock API's. For the storage
backend code - the data will be filled in during the volume updates.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan c5f6151390 util: Introduce virStorageSourceUpdatePhysicalSize
Commit id '8dc27259' introduced virStorageSourceUpdateBlockPhysicalSize
in order to retrieve the physical size for a block backed source device
for an active domain since commit id '15fa84ac' changed to use the
qemuMonitorGetAllBlockStatsInfo and qemuMonitorBlockStatsUpdateCapacity
API's to (essentially) retrieve the "actual-size" from a 'query-block'
operation for the source device.

However, the code only was made functional for a BLOCK backing type
and it neglected to use qemuOpenFile, instead using just open. After
the open the block lseek would find the end of the block and set the
physical value, close the fd and return.

Since the code would return 0 immediately if the source device wasn't
a BLOCK backed device, the physical would be displayed incorrectly,
such as follows in domblkinfo for a file backed source device:

Capacity:       1073741824
Allocation:     0
Physical:       0

This patch will modify the algorithm to get the physical size for other
backing types and it will make use of the qemuDomainStorageOpenStat
helper in order to open/stat the source file depending on its type.
The qemuDomainGetStatsOneBlock will no longer inhibit printing errors,
but it will still ignore them leaving the physical value set to 0.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan a7fea19fcd qemu: Introduce helper qemuDomainStorageUpdatePhysical
Currently just a shim to call virStorageSourceUpdateBlockPhysicalSize

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan 732af77cce qemu: Add helpers to handle stat data for qemuStorageLimitsRefresh
Split out the opening of the file and fetch of the stat buffer into a
helper qemuDomainStorageOpenStat. This will handle either opening the
local or remote storage.

Additionally split out the cleanup of that into a separate helper
qemuDomainStorageCloseStat which will either close the file or
call the virStorageFileDeinit function.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan 7149d1693d qemu: Clean up description for qemuStorageLimitsRefresh
Originally added by commit id '89646e69' prior to commit id '15fa84ac'
and '71d2c172' which ensured that qemuStorageLimitsRefresh was only called
for inactive domains.

Adjust the comment describing the need for FIXME and move all the text
to the function description.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-12 16:04:17 -05:00
John Ferlan e36a0e0cde docs: Adjust pool/vol target output
Looks like the <timestamps> and <encryption> were put in the wrong
place... They're not <pool> elements, rather they are <volume> elements
2016-12-11 08:07:41 -05:00
John Ferlan f17d68067e docs: Replace missing description for perf.cpu_cycles
Lost during merge of commit id '8546adf80' and '585ad00b5'
2016-12-11 07:56:53 -05:00
John Ferlan 03876cf506 nss: Need to check error condition on virJSONValueArraySize
If the 'nleases < 0' on return, then the subsequent call to
findLeaseInJSON will not produce the expected results (passed
in as a size_t, but nleases is a ssize_t).  So check if the
returned value < 0 and if so, goto cleanup.

Found by Coverity as a NEGATIVE_RETURNS event
2016-12-08 14:58:31 -05:00
John Ferlan 80acf4b867 tests: Fix virmacmaptest when allocation fails
If the allocation fails in DO_TEST_FLUSH_PROLOGUE, then 'mgr == NULL',
but the code continues on - which won't be good. So modify the macro
to cause an immediate failure and jump to a cleanup label.

Found by Coverity as FORWARD_NULL event.
2016-12-08 14:58:31 -05:00
John Ferlan 0658184944 news: Add description for perf.branch_instructions 2016-12-08 14:28:58 -05:00
John Ferlan 016b63bdf0 docs: Fix code example formatting for virDomainInterfaceAddresses
Adjust the spacing so that the code examples are display in the code/text
box rather than just as paragraph text.
2016-12-07 06:07:16 -05:00
John Ferlan 585ad00b5b docs: Adjust formatting for virConnectGetAllDomainStats output
Adjust the spacing a bit in order to generate 'cleaner' looking output.
This matches what virDomainMemoryStats does and it creates text/code boxes
in order to list each of the stats for each category.
2016-12-07 06:07:16 -05:00
John Ferlan 55bddd5e4e docs: Rebuild apihtml docs when source changes
When changing one of the src/libvirt-*.c files to alter the docs, the
adjusted files weren't being built.  Added them into APIBUILD_STAMP and
then added that to the html/index.html rule which is used for the
$(apihtml_generated) generated rule.

Also, for clean we can remove the html/*.html files
2016-12-07 06:07:16 -05:00
John Ferlan fbec5b949b docs: Add NEWS description for group_name support in iotune 2016-12-05 18:30:38 -05:00
John Ferlan d92b09cd75 virsh: Add group name to blkdeviotune output
https://bugzilla.redhat.com/show_bug.cgi?id=1336564

Add the ability to set/display the group_name for block device iotune

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-05 18:30:38 -05:00
John Ferlan 1ff38366b8 qemu: Add the group name option to the iotune command line
Add in the block I/O throttling group parameter to the command line
if supported. If not supported, fail command creation.

Add the xml2argvtest for testing.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-05 18:30:38 -05:00
John Ferlan 32d99cb772 conf: Add support for blkiotune group_name option
Modify _virDomainBlockIoTuneInfo and rng schema to support the group_name
option for iotune throttling. Document the new value.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-05 18:30:34 -05:00
John Ferlan c53bd25b13 qemu: Add support for parsing iotune group setting
Add support to read/parse the iotune group setting for qemu.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-05 18:12:08 -05:00
John Ferlan d0f82df205 qemu: Adjust various bool BlockIoTune set_ values into a single mask
Rather than have multiple bool values, create a single enum with bits
representing what fields are set. Fields are generally set in groups
of 3 (read, write, total).
2016-12-05 18:12:08 -05:00
John Ferlan ad9f127302 qemu: Alter qemuMonitorJSONSetBlockIoThrottle command logic
Currently we build the JSON object for the "block_set_io_throttle"
command using the knowledge that a NULL for a support*Options boolean
would essentially ignore the rest of the arguments.

This may not work properly if some capability was backported, plus it just
looks rather ugly. So instead, build the "base" arguments and then if
the support*Option bool capability is set, add in the arguments on the fly.

Then append those arguments to the basic command and send to qemu.
2016-12-05 18:12:08 -05:00
John Ferlan c84ad82a2d qemu: Adjust maxparams logic for qemuDomainGetBlockIoTune
Rather than using negative logic and setting the maxparams to a lesser
value based on which capabilities exist, alter the logic to modify the
maxparams based on a base value plus the found capabilities. Reduces the
chance that some backported feature produces an incorrect value.
2016-12-05 18:12:08 -05:00
John Ferlan d3364dfdc8 caps: Add new capability for the iotune group name
Add the capability to detect if the qemu binary can support the feature
to use throttling.group.
2016-12-05 18:12:08 -05:00
John Ferlan 9001662b9f include: Add new "group_name" definition for iotune throttling
Add the new field to support sharing I/O throttling quota between
multiple drives.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-12-05 18:12:08 -05:00
John Ferlan d3bba70771 storage: Fix type PLOOP type check for storageVolUpload
Commit id '03e750f3' added support for checking the PLOOP type; however,
it used 'target.type' which no storage code ever fills in, so it will
never be set.  Change to just vol->type (could use vol->target.format
as well).
2016-12-05 06:44:04 -05:00
John Ferlan 135e77d32f fs: Add proper switch to create filesystem with overwrite
https://bugzilla.redhat.com/show_bug.cgi?id=1366460

When using the --overwrite switch on a pool-build or pool-create, the
The mkfs.ext{2|3|4} commands use mke2fs which requires using the '-F' switch
in order to force overwriting the current filesystem on the whole disk.

Likewise, the mkfs.vfat command uses mkfs.fat which requires using the '-I'
switch in order to force overwriting the current filesystem on the whole disk.
2016-11-16 06:52:35 -05:00
John Ferlan ec00fc016a qemu: Remove erroneously placed comments for numerical ordering
Commit id '74bbb8c2ec' seems to have mismerged a bit - adding 240 comments
out of place.  Just clean that up.
2016-11-10 10:55:31 -05:00
John Ferlan f694f3ff6b qemu: Only allow 'raw' format for scsi-block using virtio-scsi
https://bugzilla.redhat.com/show_bug.cgi?id=1379196

Add check in qemuCheckDiskConfig for an invalid combination
of using the 'scsi' bus for a block 'lun' device and any disk
source format other than 'raw'.
2016-11-08 06:32:12 -05:00
John Ferlan 06a7b1ff4d vsh: Pass correct values for command line completion
Commit id 'dcfdf341' passes 'opts_need_arg' and 'opts_seen' to
vshCmddefGetData, but that seems to be incorrect as those values
are not initialized properly (something at least one compiler found).
Instead the static 'const_opts_need_arg' and 'const_opts_seen' values
should be passed.

By passing unitialized values leads to not finding possible options
for simpler commands (domfsfreeze for example), where if you're in
a virsh shell using command line completion - you'll get a list of
files in your current directory instead of two options --domain and
--mountpoint (as would happen with this patch applied.
2016-10-28 17:09:46 -04:00
John Ferlan 8550e8585e qemu: Add secret object hotplug for TCP chardev TLS
https://bugzilla.redhat.com/show_bug.cgi?id=1300776

Complete the implementation of support for TLS encryption on
chardev TCP transports by adding the hotplug ability of a secret
to generate the passwordid for the TLS object for chrdev, RNG,
and redirdev.

Fix up the order of object removal on failure to be the inverse
of the attempted attach (for redirdev, chr, rng) - for each the
tls object was being removed before the chardev backend.

Likewise, add the ability to hot unplug that secret object as well
and be sure the order of unplug matches that inverse order of plug.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-26 07:27:48 -04:00
John Ferlan daf5c651f0 qemu: Add a secret object to/for a char source dev
Add the secret object so the 'passwordid=' can be added if the command line
if there's a secret defined in/on the host for TCP chardev TLS objects.

Preparation for the secret involves adding the secinfo to the char source
device prior to command line processing. There are multiple possibilities
for TCP chardev source backend usage.

Add test for at least a serial chardev as an example.
2016-10-26 07:18:25 -04:00
John Ferlan 68808516fe qemu: Need to remove TLS object in RemoveRNGDevice
Commit id '6e6b4bfc' added the object, but forgot the other end.
2016-10-26 07:04:15 -04:00
John Ferlan 502c747aa1 qemu: Fix depedency order in qemuRemoveDiskDevice
Need to remove the drive first, then the secobj and/or encobj if they exist.
This is because the drive has a dependency on secobj (or the secret for
the networked storage server) and/or the encobj (or the secret for the
LUKS encrypted volume).  Deleting either object first leaves an drive
without it's respective objects.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-26 06:56:00 -04:00
John Ferlan 13022ce430 virsh: Add _length parameters to virsh output
https://bugzilla.redhat.com/show_bug.cgi?id=1349898

Add the duration parameters to the virsh input/output for blkdeviotune
command and describe them in the pod file.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-25 17:20:17 -04:00
John Ferlan 49d01f9e3f virsh: Create macros to add IOTUNE values
Rework the repetitive lines to add iotune values into easier to read macros.
One to handle the SCALED values and one to handle the non scaled values.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-25 17:20:17 -04:00
John Ferlan 2db108c766 qemu: Add the length options to the iotune command line
Add in the block I/O throttling length/duration parameter to the command
line if supported. If not supported, fail command creation.

Add the xml2argvtest for testing.
2016-10-25 17:20:17 -04:00
John Ferlan 8dcf355973 conf: Add support for blkiotune "_length" options
Modify _virDomainBlockIoTuneInfo and rng schema to support the _length
options for bps/iops throttling values. Document the new values.
2016-10-25 17:20:17 -04:00
John Ferlan 223438a245 qemu: Add length for bps/iops throttling parameters to driver
Add support for a duration/length for the bps/iops and friends.

Modify the API in order to add the "blkdeviotune." specific definitions
for the iotune throttling duration/length options

    total_bytes_sec_max_length
    write_bytes_sec_max_length
    read_bytes_sec_max_length
    total_iops_sec_max_length
    write_iops_sec_max_length
    read_iops_sec_max_length
2016-10-25 17:20:13 -04:00
John Ferlan d379552b41 caps: Add new capability for the bps/iops throttling length
Add the capability to detect if the qemu binary can support the feature
to use bps-max-length and friends.
2016-10-25 17:16:26 -04:00
John Ferlan 2d3377c3a0 include: Add new definitions for duration for bps/iops throttling
Add new options to allow proving a duration/length in seconds to allow the
bps/iops (and friends) to occur:

    total_bytes_sec_max_length
    write_bytes_sec_max_length
    read_bytes_sec_max_length
    total_iops_sec_max_length
    write_iops_sec_max_length
    read_iops_sec_max_length

Add continue for compiler hint to return to for control
2016-10-25 17:16:26 -04:00
John Ferlan 144947ced6 qemu: Introduce qemuDomainSetBlockIoTuneDefaults
Create a helper to set the bytes/iops iotune default values based on
the current qemu setting for both the live and persistent definitions.
NB: This also fixes an unreported bug where the persistent values for
*_max and size_iops_sec would be set back to 0 if unrelated persistent
values were set.
2016-10-25 17:12:11 -04:00
John Ferlan 1f89039ddb qemu: Move setting of conf_disk in qemuDomainSetBlockIoTune
Since persistent_def is the only place that uses it, let's just keep
it closer to where it's used.
2016-10-25 16:09:24 -04:00
John Ferlan 0ac8b70bb3 qemu: Return real error message for block_set_io_throttle
This patch will also adjust the qemuMonitorJSONSetBlockIoThrottle error
procession so that rather than returning/displaying:

    "error: internal error: Unexpected error"

Fetch the actual error message from qemu and display that
2016-10-25 16:09:24 -04:00
John Ferlan d24835f2ae qemu: Create a macro to handle setting bytes/iops iotune values
Create a macros to hide all the comparisons for each of the fields.

Add a 'continue;' for a compiler hint that we only need to find one
this should be similar enough to the if - elseif - elseif logic.
2016-10-25 16:09:24 -04:00
John Ferlan 1b93def213 qemu: Move TLS object remove from DetachChr to RemoveChr
Commit id '2c32237' added the TLS object removal to the DetachChrDevice
all when it should have been added to the RemoveChrDevice since that's
the norm for similar processing (e.g. disk)

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-25 15:18:17 -04:00
John Ferlan 6e6b4bfcf2 qemu: Add TLS hotplug for qemuDomainAttachRNGDevice
Commit id '2c322378' missed the nuance that the rng backend could be
using a TCP chardev and if TLS is enabled on the host, thus will need
to have the TLS object added.
2016-10-24 07:56:50 -04:00
John Ferlan d27c5c3e0d qemu: Add TLS hotplug for qemuDomainAttachRedirdevDevice
Commit id '2c322378' missed the nuance that the redirdev backend could
be using a TCP chardev and if TLS is enabled on the host, thus will need
to have the TLS object added.
2016-10-24 07:56:35 -04:00
John Ferlan 7300ca2134 qemu: Clean up error path in qemuDomainAttachRedirdevDevice
It's about to get more complicated - let's alter the logic to handle
various failures. Adds saving of the error as well.
2016-10-24 07:46:48 -04:00
John Ferlan 8b82355e51 qemu: Introduce qemuDomainGetChardevTLSObjects for hotplug
As it turns out more than one place will need these objects, so rather
than cut-copy-paste in each, make a helper
2016-10-24 07:44:10 -04:00
John Ferlan 526acb6a7f conf: Use virDomainChrSourceDefNew for virDomainRNGDef allocation
Rather than VIR_ALLOC() the data, use virDomainChrSourceDefNew in order
to get the private data if necessary.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-24 06:44:23 -04:00
John Ferlan 9938226251 conf: Use virDomainChrSourceDefPtr for _virDomainRedirdevDef 'source.chr'
Use a pointer and the virDomainChrSourceDefNew() function in order to
allocate the structure for _virDomainRedirdevDef.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-24 06:44:23 -04:00
John Ferlan 8f67b9ecd2 conf: Use virDomainChrSourceDefPtr for _virDomainSmartcardDef 'passthru'
Use a pointer and the virDomainChrSourceDefNew() function in order to
allocate the structure for _virDomainSmartcardDef.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-24 06:44:23 -04:00
John Ferlan 7bd8312e7f conf: Move the privateData from virDomainChrDef to virDomainChrSourceDef
Commit id '5f2a132786' should have placed the data in the host source
def structure since that's also used by smartcard, redirdev, and rng in
order to provide a backend tcp channel.  The data in the private structure
will be necessary in order to provide the secret properly.

This also renames the previous names from "Chardev" to "ChrSource" for
the private data structures and API's
2016-10-21 16:42:59 -04:00
John Ferlan 77a12987a4 Introduce virDomainChrSourceDefNew for virDomainChrDefPtr
Change the virDomainChrDef to use a pointer to 'source' and allocate
that pointer during virDomainChrDefNew.

This has tremendous "fallout" in the rest of the code which mainly
has to change source.$field to source->$field.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-21 14:03:36 -04:00
John Ferlan 931864a3ea vmx: Use the allocator virDomainChrDefNew
Rather than VIR_ALLOC of the virDomainChrDefPtr
2016-10-21 14:03:30 -04:00
John Ferlan 9c2e539acd bhyve: Need to add parameter to virDomainChrDefNew
Commit id '5f2a13278' missed this one.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-21 08:50:28 -04:00
John Ferlan 5f2a132786 qemu: Introduce qemuDomainChardevPrivatePtr
Modeled after the qemuDomainHostdevPrivatePtr (commit id '27726d8c'),
create a privateData pointer in the _virDomainChardevDef to allow storage
of private data for a hypervisor in order to at least temporarily store
secret data for usage during qemuBuildCommandLine.

NB: Since the qemu_parse_command (qemuParseCommandLine) code is not
expecting to restore the secret data, there's no need to add code
code to handle this new structure there.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-19 15:40:29 -04:00
John Ferlan 3b668bb51a conf: Introduce {default|chardev}_tls_x509_secret_uuid
Add a new qemu.conf variables to store the UUID for the secret that could
be used to present credentials to access the TLS chardev.  Since this will
be a server level and it's possible to use some sort of default, introduce
both the default and chardev logic at the same time making the setting of
the chardev check for it's own value, then if not present checking whether
the default value had been set.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-19 15:40:29 -04:00
John Ferlan 6262a9b282 qemu: Remove unnecessary NULL arg check
qemuDomainSecret{Disk|Hostdev}Prepare has a prototype that checks for
ATTRIBUTE_NONNULL(1) for 'conn'.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-17 15:38:32 -04:00
John Ferlan 40b6f91900 qemu: Add 'verify-peer=yes' test for chardev TCP TLS
Missing the option to set verify-peer to yes

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-17 15:38:32 -04:00
John Ferlan a99d9082ac qemu: Remove unnecessary cfg fetch/unref
qemuProcessPrepareDomain has no need to fetch/unref the cfg, so remove it.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-17 15:38:32 -04:00
John Ferlan f29b13f830 util: Alter return value of virReadFCHost and fix mem leak
https://bugzilla.redhat.com/show_bug.cgi?id=1357416

Rather than return a 0 or -1 and the *result string, return just the result
string to the caller.  Alter all the callers to handle the different return.

As a side effect or result of this, it's much clearer that we cannot just
assign the returned string into the scsi_host wwnn, wwpn, and fabric_wwn
fields - rather we should fetch a temporary string, then as long as our
fetch was good, VIR_FREE what may have been there, and STEAL what we just got.
This fixes a memory leak in the virNodeDeviceCreateXML code path through
find_new_device and nodeDeviceLookupSCSIHostByWWN which will continually
call nodeDeviceSysfsGetSCSIHostCaps until the expected wwnn/wwpn is found
in the device object capabilities.

Signed-off-by: John Ferlan <jferlan@redhat.com>
2016-10-14 06:47:36 -04:00