Block job cancellation can take a while. Now that upstream qemu 1.1
has asynchronous block cancellation, we want to expose that to the user.
Therefore, the following updates are made to the virDomainBlockJob API:
A new block job event type VIR_DOMAIN_BLOCK_JOB_CANCELED is managed by
libvirt. Regardless of the flags used with virDomainBlockJobAbort, this
event will be raised: 1. when using synchronous block_job_cancel (the
event will be synthesized by libvirt), and 2. whenever it is received
from qemu (via asynchronous block-job-cancel). Note that the event
may be detected by libvirt even before the virDomainBlockJobAbort
completes (always true when it is synthesized, but also possible if
cancellation was fast).
A new extension flag VIR_DOMAIN_BLOCK_JOB_ABORT_ASYNC is added to the
virDomainBlockJobAbort API. When enabled, this function will allow
(but not require) asynchronous operation (ie, it returns as soon as
possible, which might be before the job has actually been canceled).
When the API is used in this mode, it is the responsibility of the
caller to wait for a VIR_DOMAIN_BLOCK_JOB_CANCELED event or poll via
the virDomainGetBlockJobInfo API to check the cancellation status.
This patch also exposes the new flag through virsh, and makes virsh
slightly easier to use (--async implies --abort, and lack of any options
implies --info), although it leaves the qemu implementation for later
patches.
Signed-off-by: Adam Litke <agl@us.ibm.com>
Cc: Stefan Hajnoczi <stefanha@gmail.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Commit d09f6ba5fe introduced a regression in event
registration. virDomainEventCallbackListAddID() will only return a positive
integer if the type of event being registered is VIR_DOMAIN_EVENT_ID_LIFECYCLE.
For other event types, 0 is always returned on success. This has the
unfortunate side effect of not enabling remote event callbacks because
remoteDomainEventRegisterAny() uses the return value from the local call to
determine if an event callback needs to be registered on the remote end.
Make sure virDomainEventCallbackListAddID() returns the callback count for the
eventID being registered.
Signed-off-by: Adam Litke <agl@us.ibm.com>
The libvirt BlockPull API supports the use of an initial bandwidth limit but the
qemu block_stream API does not. To get the desired behavior we use the two APIs
strung together: first BlockPull, then BlockJobSetSpeed. We can do this at the
driver level to avoid duplicated code in each monitor path.
Signed-off-by: Adam Litke <agl@us.ibm.com>
Due to an unfortunate precedent in qemu, the units for the bandwidth parameter
to block_job_set_speed are different between the text monitor and the qmp
monitor. While the qmp monitor uses bytes/s, the text monitor expects MB/s.
Correct the units for the text interface.
Signed-off-by: Adam Litke <agl@us.ibm.com>
When an operation started by virDomainBlockPull completes (either with
success or with failure), raise an event to indicate the final status.
This API allow users to avoid polling on virDomainGetBlockJobInfo if
they would prefer to use an event mechanism.
* daemon/remote.c: Dispatch events to client
* include/libvirt/libvirt.h.in: Define event ID and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
src/libvirt_private.syms: Extend API to handle the new event
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
for block_stream completion and emit a libvirt block pull event
* src/remote/remote_driver.c: Receive and dispatch events to application
* src/remote/remote_protocol.x: Wire protocol definition for the event
* src/remote_protocol-structs: structure definitions for protocol verification
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
from QEMU monitor
virDomainGetBlockJobInfo requires manual override since it returns a
custom type.
* python/generator.py: reenable bindings for this entry point
* python/libvirt-override-api.xml python/libvirt-override.c:
manual overrides
Define two new virsh commands:
* blockpull: Initiate a blockPull for the given disk
* blockjob: Retrieve progress info, modify speed, and cancel active block jobs
Share print_job_progress() with the migration code.
* tools/virsh.c: implement the new commands
The virDomainBlockPull* family of commands are enabled by the
following HMP/QMP commands: 'block_stream', 'block_job_cancel',
'info block-jobs' / 'query-block-jobs', and 'block_job_set_speed'.
* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk
streaming by using the proper qemu monitor commands.
* src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor
The generator can handle everything except virDomainGetBlockJobInfo().
* src/remote/remote_protocol.x: provide defines for the new entry points
* src/remote/remote_driver.c daemon/remote.c: implement the client and
server side for virDomainGetBlockJobInfo.
* src/remote_protocol-structs: structure definitions for protocol verification
* src/rpc/gendispatch.pl: Permit some unsigned long parameters
Set up the types for the block pull functions and insert them into the
virDriver structure definition. Symbols are exported in this patch to
prevent
documentation compile failures.
* include/libvirt/libvirt.h.in: new API
* src/driver.h: add the new entry to the driver structure
* python/generator.py: fix compiler errors, the actual python bindings
* are
implemented later
* src/libvirt_public.syms: export symbols
* docs/apibuild.py: Extend 'unsigned long' parameter exception to this
* API
When an operation started by virDomainBlockPullAll completes (either with
success or with failure), raise an event to indicate the final status. This
allows an API user to avoid polling on virDomainBlockPullInfo if they would
prefer to use the event mechanism.
* daemon/remote.c: Dispatch events to client
* include/libvirt/libvirt.h.in: Define event ID and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
src/libvirt_private.syms: Extend API to handle the new event
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
for block_stream completion and emit a libvirt block pull event
* src/remote/remote_driver.c: Receive and dispatch events to application
* src/remote/remote_protocol.x: Wire protocol definition for the event
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
from QEMU monitor
Signed-off-by: Adam Litke <agl@us.ibm.com>
virDomainBlockPullAll and virDomainBlockPullAbort are handled automatically.
virDomainBlockPull and virDomainBlockPullInfo require manual overrides since
they return a custom type.
* python/generator.py: reenable bindings for this entry point
* python/libvirt-override-api.xml python/libvirt-override.c:
manual overrides
Signed-off-by: Adam Litke <agl@us.ibm.com>
Acked-by: Daniel P. Berrange <berrange@redhat.com>
Define two new virsh commands:
* blockpull: Perform block pull operations (incremental plus start
and stop continuous streams)
* blockpullinfo: Retrieve progress info for continuous block pull
Share print_job_progress() with the migration code.
* tools/virsh.c: implement the new commands
Signed-off-by: Adam Litke <agl@us.ibm.com>
The virDomainBlockPull* family of commands are enabled by the
'block_stream' and 'info block_stream' qemu monitor commands.
* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk
streaming by using the stream and info stream text monitor commands
* src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor
Signed-off-by: Adam Litke <agl@us.ibm.com>
Acked-by: Daniel P. Berrange <berrange@redhat.com>
The generator can handle DomainBlockPullAll and DomainBlockPullAbort.
DomainBlockPull and DomainBlockPullInfo must be written by hand.
* src/remote/remote_protocol.x: provide defines for the new entry points
* src/remote/remote_driver.c daemon/remote.c: implement the client and
server side
* src/remote_protocol-structs: structure definitions for protocol verification
Signed-off-by: Adam Litke <agl@us.ibm.com>
Set up the types for the block pull functions and insert them into the
virDriver structure definition. Symbols are exported in this patch to prevent
documentation compile failures.
* include/libvirt/libvirt.h.in: new API
* src/driver.h: add the new entry to the driver structure
* python/generator.py: fix compiler errors, the actual python bindings are
implemented later
* src/libvirt_public.syms: export symbols
Signed-off-by: Adam Litke <agl@us.ibm.com>
This patch makes two corrections to the newly-added QED support patch series:
- Correct the QED header field offsets
- Remove XML parsing for VIR_STORAGE_FILE_AUTO_SAFE
Signed-off-by: Adam Litke <agl@us.ibm.com>
Disk image formats that wish to opt-out of version validation are supposed to
set versionOffset to -1 in their fileTypeInfo entry.
By unconditionally returning False for these formats,
virStorageFileMatchesVersion() incorrectly reports a version mismatch when the
test was actually skipped. The correct behavior is to return True so these
formats can be successfully probed using the magic bytes alone.
Signed-off-by: Adam Litke <agl@us.ibm.com>
I am trying to use a qcow image with libvirt where the backing 'file' is a
qemu-nbd server. Unfortunately virDomainDiskDefForeachPath() assumes that
backingStore is always a real file so something like 'nbd:0:3333' is rejected
because a file with that name cannot be accessed. Note that I am not worried
about directly using nbd images. That would require a new disk type with XML
markup, etc. I only want it to be permitted as a backingStore
The following patch implements danpb's suggestion:
> I think I'm inclined to push the logic for skipping NBD one stage higher.
> I'd rather expect virStorageFileGetMetadata() to return all backing
> stores, even if not files. The virDomainDiskDefForeachPath() method
> should definitely ignore non-file backing stores though.
>
> So what I'm thinking is to extend the virStorageFileMetadata struct and
> just add a 'bool isFile' field to it. Default this field to true, unless
> you see the prefix of nbd: in which case set it to false. The
> virDomainDiskDefForeachPath() method can then skip over any backing
> store with isFile == false
Signed-off-by: Adam Litke <agl@us.ibm.com>
Cc: Daniel P. Berrange <berrange@redhat.com>
When adding domainMemoryStats API support for the qemu driver, I didn't
follow the locking rules exactly. The job condition must be held when
executing monitor commands. This corrects the segfaults I was seeing
when calling domainMemoryStats in a multi-threaded environment.
* src/qemu/qemu_driver.c: in qemudDomainMemoryStats() add missing
calls to qemuDomainObjBeginJob/qemuDomainObjEndJob
Fix a small problem with the qemu memory stats parsing algorithm. If qemu
reports a stat that libvirt does not recognize, skip past it so parsing can
continue. This corrects a potential infinite loop in the parsing code that can
only be triggered if new statistics are added to qemu.
* src/qemu/qemu_monitor_text.c: qemuMonitorParseExtraBalloonInfo add a
skip for extra ','
The behavior for the qemu balloon device has changed. Formerly, a virtio
balloon device was provided by default. Now, '-balloon virtio' must be
specified on the command line to enable it. This patch causes libvirt to
add '-balloon virtio' to the command line whenever the -balloon option is
available.
* src/qemu/qemu_conf.c src/qemu/qemu_conf.h: check for the new flag and
add "-baloon vitio" to qemu command when needed
* tests/qemuhelptest.c: add the new flag for detection
Enable virDomainMemoryStats in the python API. dom.memoryStats() will return a
dictionary containing the supported statistics. A dictionary is required
because the meaining of each quantity cannot be inferred from its index in a
list.
* python/generator.py: reenable bindings for this entry point
* python/libvirt-override-api.xml python/libvirt-override.c: the
generator can't handle this new function, add the new binding,
and the XML description
Define a new command 'dommemstats' to report domain memory statistics. The
output format is inspired by 'domblkstat' and 'domifstat' and consists of
tag/value pairs, one per line. The command can complete successfully and
print no output if virDomainMemoryStats is supported by the driver, but not
the guest operating system.
Sample output:
swap_in 0
swap_out 0
major_fault 54
minor_fault 58259
unused 487680
available 502472
All stats referring to a quantity of memory (eg. all above except major and
minor faults) represent the quantity in KBytes.
* tools/virsh.c: implements the new command
Use a dynamically sized xdr_array to pass memory stats on the wire. This
supports the addition of future memory stats and reduces the message size
since only supported statistics are returned.
* src/remote/remote_protocol.x: provide defines for the new entry point
* src/remote/remote_driver.c daemon/remote.c: implement the client and
server side
* daemon/remote_dispatch_args.h daemon/remote_dispatch_prototypes.h
daemon/remote_dispatch_ret.h daemon/remote_dispatch_table.h
src/remote/remote_protocol.c src/remote/remote_protocol.h: generated
stubs
Support for memory statistics reporting is accepted for qemu inclusion.
Statistics are reported via the monitor command 'info balloon' as a comma
seprated list:
(qemu) info balloon
balloon: actual=1024,mem_swapped_in=0,mem_swapped_out=0,major_page_faults=88,minor_page_faults=105535,free_mem=1017065472,total_mem=1045229568
Libvirt, qemu, and the guest operating system may support a subset of the
statistics defined by the virtio spec. Thus, only statistics recognized by
components will be reported.
* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement the
new entry point by using info balloon monitor command
Set up the types for the domainMemoryStats function and insert it into the
virDriver structure definition. Because of static initializers, update
every driver and set the new field to NULL.
* include/libvirt/libvirt.h.in: new API
* src/driver.h src/*/*_driver.c src/vbox/vbox_tmpl.c: add the new
entry to the driver structure
* python/generator.py: fix compiler errors, the actual python binding is
implemented later
Adam Litke