Fixing the transient definition bug in a0d2ec63b99c4e0f500d6b61a208be48b3e3b396
exposed another issue: the startup filled firmware data is only
available in the live definition. Thus, after shutdown, there is
no information about the VARS file, so `virsh undefine --nvram` cannot
remove it.
Fix that by filling firmware data in the domain XML post-parse stage.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve allows to set eui64 for the NVMe disks:
eui64 IEEE Extended Unique Identifier (8 byte value).
And in a real command line:
-s 5:0,nvme,/data/img/10gb.img,ser=BHYVE-NVME01-F000,eui64=0x01000000efbeadde
Model that using the <wwn> element:
<disk type='file'>
<driver name='file' type='raw'/>
<source file='/path/to/some.img'/>
<target dev='nvme0n1' bus='nvme'/>
<wwn>01000000efbeadde</wwn>
</disk>
As this element does not allow commas, it's not necessary to add
additional validation for that.
Closes: https://gitlab.com/libvirt/libvirt/-/work_items/901
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
bhyve supports setting NVMe controller serial number using
the "ser=<SN>" option. Update bhyveBuildNVMeControllerArgStr()
to set that if "serial" is specified for the controller.
We do it in the controller's disks loop, but as we prohibit more than 1
disk per controller in bhyveDomainDefValidate(), there is no
repetition here.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Bhyve supports virtio-console devices using the following syntax:
-s 2:0,virtio-console,org.qemu.guest_agent.0=/path/to/unix/socket,other.port=/other/socket,...
There are two details about that to consider.
The first one is that only up to 16 ports per console is supported. This
is different from the default (31), so update the code to manually add
the virtio-serial controllers with 16 ports. For the existing
controllers, make sure to set max ports to 16 or error out if ports
count greater than 16 was specified.
The second one is that bhyve does not clean up UNIX sockets for these
devices. So update virBhyveProcessStop() to remove leftover sockets.
Not adding capabilities probing as the virtio-console device is
available on all supported FreeBSD versions and on all supported arches.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
FreeBSD supports resource limiting with the rctl(4) framework.
It supports various resource types, including I/O resources.
It allows to limit resources for users, processes, login classes,
and jails.
To apply blkiotune limits set limits for the bhyve process.
I/O related resources supported by rctl(4) are:
readbps filesystem reads, in bytes per second
writebps filesystem writes, in bytes per second
readiops filesystem reads, in operations per second
writeiops filesystem writes, in operations per second
Thus, the actual commands look like:
rctl -a process:$bhyvepid:writebps:throttle=10000000
rctl -a process:$bhyvepid:readbps:throttle=10000000
rctl -a process:$bhyvepid:writeiops:throttle=20000
rctl -a process:$bhyvepid:readiops:throttle=20000
This is different from the current blkiotune modeling in libvirt as
it requires specific device to apply limits to. To adapt this model
to per-domain I/O limits, update domain schema to specify "*" as a
device name.
The rctl(8) may be not available or not enabled, so add a capability
check for that.
Per process rules get removed when the process disappears, so no special
clean up is necessary.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Bhyve supports NUMA domains configuration using the '-n'
command line argument:
-n id,size,cpus[,domain_policy]
Here, "id" is a numeric NUMA domain id, "size" is the total VM
memory size with units format similar to the "-m" switch,
"cpus" is a cpuset, and "domain_policy" is an optional
domainset(9) memory allocation policy. The "domain_policy"
is currently not used by the libvirt driver.
This argument is repeated for every NUMA domain to be configured, e.g.:
bhyve \
...
-n id=0,size=107,cpus=0-3
-n id=1,size=107,cpus=4-7
To support that:
* Add a corresponding capability; it is considered supported
if the bhyve binary has the '-n' command line switch.
* Generate command line arguments for NUMA from
<cpu><numa>..</numa></cpu> domain configuration.
Additionally, validate that:
* NUMA domains can be only configured with the UEFI loaders.
* No more than 8 domains configured per VM as limited by bhyve.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve on arm64 does not have the bhyveload(8) tool.
That means that it cannot be used as a default if the loader is not
configured for the domain.
To prevent users from getting unusable configurations, handle loader
configuration on arm64 like that:
- if loader is specified in the domain XML, just use it
- if not specified, try to check whether the default uboot loader
is available on the system. In case it is, set is as the loader,
otherwise fail with the error.
Additionally, the loader could be configured in bhyve.conf.
By default, it uses the loader installed by the
sysutils/u-boot-bhyve-arm64 port or a corresponding package.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently, bhyve does not support UTC clock offset on ARM64.
However, when <clock offset= > is not specified in the domain XML,
UTC offset is used by default. That results in an incorrect
configuration for the bhyve ARM64 guests by default.
Workaround is to extend bhyveDomainDefPostParse() to fall back
to the LOCALTIME clock offset when UTC clock offset is not
supported by bhyve.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports virtio-scsi devices using the following syntax:
bhyve ... -s N,virtio-scsi,/dev/cam/ctl[pp.vp][,scsi-device-options]
Where /dev/cam/ctl is a ctl(4) device path.
The optional "scsi-device-options" include "iid" (Initiator ID)
and "bootindex", which are currently not used by libvirt.
Model this device using:
<disk type='ctl'>
<source dev='/dev/cam/ctl'/>
<target dev='sda' bus='scsi'/>
</disk>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports SLIRP networking using the following syntax:
-s 12:0,e1000,slirp,mac=<macaddr>,open
Where "e1000" is a NIC model, "slirp" is the SLIRP backend and "open"
specifies open mode where external network is available to the guest.
The "open" mode is a recent addition in FreeBSD -CURRENT.
Unfortunately, bhyve does not provide a way to probe whether
the open mode is supported, so users will have to make sure
it's supported on their own.
For the reference, without the "open" mode, the guest will have no
outside network connectivity. To make this mode useful,
it is possible to configure forwarding from the host to the guest,
but it is not covered by this patch.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
Bhyve supports the 'wait' option for the VNC device configuration.
When enabled, VM boots only upon a VNC connection.
Sample device configuration looks like this:
-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
bhyve supports queue configuration for the NVMe disks:
maxq Max number of queues.
qsz Max elements in each queue.
Map that to the disk driver's "queues" and "queue_size" attributes
respectfully, so:
<driver name='file' type='raw' queues='2' queue_size='256'/>
results in:
-s N:0,nvme,/tmp/disk.img,maxq=2,qsz=256
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
NVMe devices in bhyve are modeled this way:
-s $pciaddr,nvme,devpath[,opts]
devpath can be a path to the image or the block device. It also can be
"ram=size_in_MiB", but this is not covered by this series.
There could be only a single device per PCI address.
Optional configuration options (such as max number of queues, concurrent
I/O requests, etc) are also not covered by this series.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Recent versions of bhyve support 4 com ports instead of just 2. Thus,
allow to use 4 console devices.
Also, there was a bug previously because the condition was
"if (chr->target.port > 2)", but as target.port start
with 0 and "com" ports start with 1, this condition allows com3 to be
used.
As bhyve supports 4 com ports already long enough, and all supported
FreeBSD versions include this capability, do not introduce driver
capability for that.
Add a couple of tests for that:
- A domain that uses 4 serials, 2 of type 'nmdm'
and the other 2 of type 'tcp'
- A domain that uses unsupported port, such as target.port=4 which
translates into com5.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
In addition to the nmdm consoles, bhyve also supports a tcp console.
It's configured with:
.. -l com1,tcp=127.0.0.1:12345
Then a user could connect to the guest console port 0 by making a tcp
connection to the host's 127.0.0.1:12345.
In the domain XML this configuration is represented as:
<serial type='tcp'>
<source mode='bind' host='127.0.0.1' service='12345'/>
<target type='serial' port='0'/>
</serial>
Also, update domain capabilities to include the TCP console support.
Unfortunately, there's no way to detect that from the bhyve binary
before trying to start a VM, so there's no capability probing for this
feature.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Bhyve supports the Virtio RNG interface. It's always using the
/dev/random device and doesn't have any configuration options.
Thus, in XML it's represented as:
<rng model='virtio'>
<backend model='random'/>
</rng>
So extend the bhyve driver to support that and add a set of tests for
this feature.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently, nmdm console device requires user to specify master and slave
path attributes (such as /dev/nmdm0A and /dev/nmdm0B respectively).
However, making user find a non-occupied device name might be not
convenient, especially for the remote connections.
Update the logic to make these attributes optional. In case if not
specified, use /dev/nmdm$UUID[AB], where $UUID is a domain's UUID.
With this schema it's unlikely nmdm device will clash with other domains
or even other non-bhyve nmdm devices.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Recently virtio-9p support was added to bhyve.
On the host side it looks this way:
bhyve .... -s 25:0,virtio-9p,sharename=/path/to/shared/dir
It could also have ",ro" suffix to make share read-only.
In the Linux guest, this share is mounted with:
mount -t 9p sharename /mnt/sharename
In the guest user will see the same permissions and ownership
information for this directory as on the host. No uid/gid remapping is
supported, so those could resolve to wrong user or group names.
The same applies to the other side: chowning/chmodding in the guest will
set specified ownership and permissions on the host.
In libvirt domain XML it's modeled using the 'filesystem' element:
<filesystem type='mount'>
<source dir='/path/to/shared/dir'/>
<target dir='sharename'/>
</filesystem>
Optional 'readonly' sub-element enables read-only mode.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Support setting a password for the VNC framebuffer using the passwd
attribute on the <graphics/> element, if the driver has the
BHYVE_CAP_VNC_PASSWORD capability.
Note that virsh domxml-from-native does not output the password in the
generated XML, as VIR_DOMAIN_DEF_FORMAT_SECURE is not set when
formatting the domain definition.
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The resolution of the VNC framebuffer can now be set via the resolution
definition introduced in 5.9.0.
Also, add "gop" to the list of model types the <resolution/>
sub-element is valid for.
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Support modeling of the 'isa' controller for bhyve. User can manually
define any PCI slot for the 'isa' controller, including PCI slot 1,
but other devices are not allowed to use this address.
When domain configuration requires the 'isa' controller to be present,
automatically add it on domain post-parse stage.
Now, as this controller is always available when needed, it's not
necessary to implicitly add it to the bhyve command line, so remove
bhyveBuildLPCArgStr().
Also, make bhyveDomainDefNeedsISAController() static as it's no longer
used outside of bhyve_domain.c.
As more than one ISA controller is not supported by bhyve,
and multiple controllers with the same index are forbidden,
so forbid ISA controllers with non-zero index for bhyve.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Allow to map sound playback and recording devices to host devices
using "<audio type='oss'/>" OSS audio backend.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
bhyve supports intel hda sound devices that could be specified
on the command like using "-1:0,hda,play=$play_dev,rec=$rec_dev",
where "1:0" is a PCI address, and "$play_dev" and "$rec_dev"
point to the playback and recording device on the host respectively.
Currently, schema of the 'sound' element doesn't allow specifying
neither playback nor recording devices, so for now hardcode
/dev/dsp0, which is the first audio device on the host.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Implement the MSRs ignore unknown reads and writes feature
that's specified using:
<features>
...
<msrs unknown='ignore'>
...
</features>
in the domain XML.
In bhyve, it's just passing '-w' command line argument to the bhyve(8)
executable.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Implement support for passing custom command line arguments
to bhyve using the 'bhyve:commandline' element:
<bhyve:commandline>
<bhyve:arg value='-newarg'/>
</bhyve:commandline>
* Define virDomainXMLNamespace for the bhyve driver, which
at this point supports only the 'commandline' element
described above,
* Update command generation code to inject these command line
arguments between driver-generated arguments and the vmname
positional argument.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
This patch adds support for automatic VNC port assignment for bhyve guests.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Add support for vgaconf driver configuration. In domain xml it looks like
this:
<video>
<driver vgaconf='io|on|off'>
<model .../>
</video>
It was added with bhyve gop video in mind to allow users control how the
video device is exposed to the guest, specifically, how VGA I/O is
handled.
One can refer to the bhyve manual page to get more detailed description
of the possible VGA configuration options:
https://www.freebsd.org/cgi/man.cgi?query=bhyve&manpath=FreeBSD+12-current
The relevant part could be found using the 'vgaconf' keyword.
Also, add some tests for this new feature.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Along with video and VNC support, bhyve has introduced USB tablet
support as an input device. This tablet is exposed to a guest
as a device on an XHCI controller.
At present, tablet is the only supported device on the XHCI controller
in bhyve, so to make things simple, it's allowed to only have a
single XHCI controller with a single tablet device.
In detail, this commit:
- Introduces a new capability bit for XHCI support in bhyve
- Adds an XHCI controller and tabled support with 1:1 mapping
between them
- Adds a couple of unit tests
Like it usually happens, I fixed one thing and broke another:
in 803966c76 address allocation was fixed for SATA disks, but
broke that for virtio disks, because it dropped disk address
assignment completely. It's not needed for SATA disks anymore,
but still needed for the virtio ones.
Bring that back and add a couple of tests to make sure it won't
happen again.
As bhyve for a long time didn't have a notion of the explicit SATA
controller and created a controller for each drive, the bhyve driver
in libvirt acted in a similar way and didn't care about the SATA
controllers and assigned PCI addresses to drives directly, as
the generated command will look like this anyway:
2:0,ahci-hd,somedisk.img
This no longer makes sense because:
1. After commit c07d1c1c4f it's not possible to assign
PCI addresses to disks
2. Bhyve now supports multiple disk drives for a controller,
so it's going away from 1:1 controller:disk mapping, so
the controller object starts to make more sense now
So, this patch does the following:
- Assign PCI address to SATA controllers (previously we didn't do this)
- Assign disk addresses instead of PCI addresses for disks. Now, when
building a bhyve command, we take PCI address not from the disk
itself but from its controller
- Assign addresses at XML parsing time using the
assignAddressesCallback. This is done mainly for being able to
verify address allocation via xml2xml tests
- Adjust existing bhyvexml2{xml,argv} tests to chase the new
address allocation
This patch is largely based on work of Fabian Freyer.
Add virBhyveDriverCreateXMLConf, a simple wrapper around
virDomainXMLOptionNew that makes it easier to pass bhyveConnPtr
as a private data for parser. It will be used later for device
address allocation at parsing time.
Update consumers to use it instead of direct calls to
virDomainXMLOptionNew.
As we now have proper callbacks connected for the tests, update
test files accordingly to include the automatically generated
PCI root controller.
Roman Bogorodskiy
Fabian Freyer
Alexander Nusov
Michal Privoznik