Bhyve supports vcpu pinning using the `-p vcpu:hostcpu`
argument. This argument can be specified multiple times for the same
vcpu to pin it to multiple hostcpu's.
Bhyve currently does not allow to change vcpu pinning configuration for
the VM that is already running.
Use this to support domain's vcpupin configuration such as:
<cputune>
<vcpupin vcpu="0" cpuset="1,2,3"/>
</cputune>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve on arm64 does not have the bhyveload(8) tool.
That means that it cannot be used as a default if the loader is not
configured for the domain.
To prevent users from getting unusable configurations, handle loader
configuration on arm64 like that:
- if loader is specified in the domain XML, just use it
- if not specified, try to check whether the default uboot loader
is available on the system. In case it is, set is as the loader,
otherwise fail with the error.
Additionally, the loader could be configured in bhyve.conf.
By default, it uses the loader installed by the
sysutils/u-boot-bhyve-arm64 port or a corresponding package.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve used the '-A' flag to enable ACPI until it was deprecated
by commit:
https://cgit.freebsd.org/src/commit/?id=6a0e7f908802b86ca5d1c0b3c404b8391d0f626e
With that, ACPI tables are always generated. As this change is
relatively new and there are likely systems that have bhyve(8) that
requires using the '-A' flag, add a capability probing for that, and
use this flag if it's supported.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The firmwareDir member of driver config is set at the beginning
of mymain(). But then, roughly in the middle of test cases it is
overwritten to fakefirmwareemptydir. But this means the old value
must be freed. Or reassigned back to its original variable which
is freed automatically.
16 bytes in 1 blocks are definitely lost in loss record 190 of 505
at 0x4883224: malloc (vg_replace_malloc.c:451)
by 0x4EE6562: g_malloc (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x4F0100F: g_strdup (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x4013E26: g_strdup_inline (gstrfuncs.h:321)
by 0x4013E26: mymain (bhyvexml2argvtest.c:151)
by 0x40189A2: virTestMain (testutils.c:913)
by 0x4013DE6: main (bhyvexml2argvtest.c:354)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
At the beginning of mymain() the virBhyveDriverConfigNew() is
called which inits driver config with some paths. These are
then overwritten to produce stable test output. Well, the old
ones should be freed first.
128 bytes in 1 blocks are definitely lost in loss record 453 of 508
at 0x4883224: malloc (vg_replace_malloc.c:451)
by 0x506BD16: vasprintf_l (in /lib/libc.so.7)
by 0x4F39073: g_vasprintf (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x4F01288: g_strdup_printf (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x401F75B: virBhyveDriverConfigNew (bhyve_conf.c:62)
by 0x4013FAA: mymain (bhyvexml2argvtest.c:164)
by 0x4018892: virTestMain (testutils.c:913)
by 0x4013DC6: main (bhyvexml2argvtest.c:352)
25 bytes in 1 blocks are definitely lost in loss record 206 of 508
at 0x4883224: malloc (vg_replace_malloc.c:451)
by 0x4EE6562: g_malloc (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x4F0100F: g_strdup (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x401F715: g_strdup_inline (gstrfuncs.h:321)
by 0x401F715: virBhyveDriverConfigNew (bhyve_conf.c:60)
by 0x4013FAA: mymain (bhyvexml2argvtest.c:164)
by 0x4018892: virTestMain (testutils.c:913)
by 0x4013DC6: main (bhyvexml2argvtest.c:352)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Driver capabilities are allocated at the beginning of mymain(),
but roughly in the middle the architecture is switched to aarch64
and capabilities are constructed again. Without freeing the old
ones.
1,583 (288 direct, 1,295 indirect) bytes in 1 blocks are definitely lost in loss record 520 of 536
at 0x4888098: calloc (vg_replace_malloc.c:1682)
by 0x4EE65CA: g_malloc0 (in /usr/local/lib/libglib-2.0.so.0.8400.4)
by 0x53344B8: g_type_create_instance (in /usr/local/lib/libgobject-2.0.so.0.8400.4)
by 0x531D263: ??? (in /usr/local/lib/libgobject-2.0.so.0.8400.4)
by 0x531C75E: g_object_new (in /usr/local/lib/libgobject-2.0.so.0.8400.4)
by 0x4AAC806: virObjectNew (virobject.c:252)
by 0x4AF366A: virCapabilitiesNew (capabilities.c:87)
by 0x401998B: virBhyveCapsBuild (bhyve_capabilities.c:51)
by 0x4013E93: mymain (bhyvexml2argvtest.c:155)
by 0x4018882: virTestMain (testutils.c:913)
by 0x4013DC6: main (bhyvexml2argvtest.c:351)
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Currently, bhyve does not support UTC clock offset on ARM64.
However, when <clock offset= > is not specified in the domain XML,
UTC offset is used by default. That results in an incorrect
configuration for the bhyve ARM64 guests by default.
Workaround is to extend bhyveDomainDefPostParse() to fall back
to the LOCALTIME clock offset when UTC clock offset is not
supported by bhyve.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Just like consoles, bootloader is handled differently on arm64.
It also does not used the LPC bus, and is configured with:
-o bootrom=/usr/local/share/u-boot/u-boot-bhyve-arm64/u-boot.bin
Additionally, fill firmware inforamtion only for amd64.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Console device handling in bhyve is different for amd64 and arm64.
On amd64, it's configured as an LPC device, and multiple consoles are
supported.
On arm64, only a single console can be configured, and the syntax is
different:
-o console=/dev/nmdmguest0A
Update the bhyve command generation accordingly.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports virtio-scsi devices using the following syntax:
bhyve ... -s N,virtio-scsi,/dev/cam/ctl[pp.vp][,scsi-device-options]
Where /dev/cam/ctl is a ctl(4) device path.
The optional "scsi-device-options" include "iid" (Initiator ID)
and "bootindex", which are currently not used by libvirt.
Model this device using:
<disk type='ctl'>
<source dev='/dev/cam/ctl'/>
<target dev='sda' bus='scsi'/>
</disk>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports SLIRP networking using the following syntax:
-s 12:0,e1000,slirp,mac=<macaddr>,open
Where "e1000" is a NIC model, "slirp" is the SLIRP backend and "open"
specifies open mode where external network is available to the guest.
The "open" mode is a recent addition in FreeBSD -CURRENT.
Unfortunately, bhyve does not provide a way to probe whether
the open mode is supported, so users will have to make sure
it's supported on their own.
For the reference, without the "open" mode, the guest will have no
outside network connectivity. To make this mode useful,
it is possible to configure forwarding from the host to the guest,
but it is not covered by this patch.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Laine Stump <laine@redhat.com>
Bhyve supports the 'wait' option for the VNC device configuration.
When enabled, VM boots only upon a VNC connection.
Sample device configuration looks like this:
-s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Martin Kletzander <mkletzan@redhat.com>
Do not allow to configure queues and queue size for non-NVMe disks.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports PCI device passthrough using the following syntax:
bhyve ... -s 4:0,passthru,5/2/0 ...
Where 5/2/0 is PCI address of the device in the host, and "4:0" is the
address in the guest.
Currently, user is responsible for reserving the device for passthrough,
i.e. by configuring pptdevs in loader.conf(5), or using devctl(8) to
detach the device.
Co-authored-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Signed-off-by: Alexander Shursha <kekek2@ya.ru>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports specifying disk rotation rate using the nmrr attribute,
e.g.:
-s 3:0,ahci,hd:/data/img/freebsd.img,nmrr=1
Where 1 means the SSD, 0 (default) means do not report, and other values
specify the actual RPM.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
For domains using NVMe disks make sure that the bhyve binary supports
that by checking capabilities.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
As bhyve does not have explicit notion of controllers, and for NVMe
devices it allows to specify one a single source for for a given PCI
address, it effectively means that there could be only one device per
controller.
Update validation code to check this case.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
NVMe devices in bhyve are modeled this way:
-s $pciaddr,nvme,devpath[,opts]
devpath can be a path to the image or the block device. It also can be
"ram=size_in_MiB", but this is not covered by this series.
There could be only a single device per PCI address.
Optional configuration options (such as max number of queues, concurrent
I/O requests, etc) are also not covered by this series.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
The bhyveload(8) command does not have a native non-interactive mode.
It means that in case of errors, e.g. invalid boot media, it
just drops into a loader prompt and waits for user input. This behaviour
makes it tricky for users to understand what's going on.
To address that, run it with the timeout(1) tool which sends SIGTERM
after a certain timeout, and then optionally sends SIGKILL if the
command keeps hanging.
These timeout values could be configured in the bhyve.conf. Setting
timeout to 0 mean that bhyveload(8) will be executed directly, without
timeout(1).
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Recent versions of bhyve support 4 com ports instead of just 2. Thus,
allow to use 4 console devices.
Also, there was a bug previously because the condition was
"if (chr->target.port > 2)", but as target.port start
with 0 and "com" ports start with 1, this condition allows com3 to be
used.
As bhyve supports 4 com ports already long enough, and all supported
FreeBSD versions include this capability, do not introduce driver
capability for that.
Add a couple of tests for that:
- A domain that uses 4 serials, 2 of type 'nmdm'
and the other 2 of type 'tcp'
- A domain that uses unsupported port, such as target.port=4 which
translates into com5.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
In addition to the nmdm consoles, bhyve also supports a tcp console.
It's configured with:
.. -l com1,tcp=127.0.0.1:12345
Then a user could connect to the guest console port 0 by making a tcp
connection to the host's 127.0.0.1:12345.
In the domain XML this configuration is represented as:
<serial type='tcp'>
<source mode='bind' host='127.0.0.1' service='12345'/>
<target type='serial' port='0'/>
</serial>
Also, update domain capabilities to include the TCP console support.
Unfortunately, there's no way to detect that from the bhyve binary
before trying to start a VM, so there's no capability probing for this
feature.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Extend bhyveFirmwareFillDomain() so that when we find the default edk2
firmware, also look for its matching template file, and use it as a
nvramTemplate if found.
Extend bhyvexml2argvtest to verify various NVRAM configurations.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Add a bunch of device def validation to catch unsupported RNG device
configurations early.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Bhyve supports the Virtio RNG interface. It's always using the
/dev/random device and doesn't have any configuration options.
Thus, in XML it's represented as:
<rng model='virtio'>
<backend model='random'/>
</rng>
So extend the bhyve driver to support that and add a set of tests for
this feature.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently the tests would fail if the bhyve commands are installed in
different path then /usr/bin. Strip the command path to not depend on
the host environment.
Signed-off-by: Pavel Hrdina <phrdina@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
virCommandToString has the possibility to return an already wrapped
string with better format than what we get from the test wrapper script.
The main advantage is that arguments for an option are always on the
same line which makes it more easy to see what changed in a diff and
prevents re-wrapping of the line if a wrapping point moves over the
threshold.
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Pavel Hrdina <phrdina@redhat.com>
Implement "<os firmware='efi'>" support for bhyve driver.
As there are not really lot of options, try to find
"BHYVE_UEFI.fd" firmware which is installed by the
sysutils/uefi-edk2-bhyve FreeBSD port.
If not found, just use the first found firmware
in the firmwares directory (which is configurable via
config file).
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Currently, nmdm console device requires user to specify master and slave
path attributes (such as /dev/nmdm0A and /dev/nmdm0B respectively).
However, making user find a non-occupied device name might be not
convenient, especially for the remote connections.
Update the logic to make these attributes optional. In case if not
specified, use /dev/nmdm$UUID[AB], where $UUID is a domain's UUID.
With this schema it's unlikely nmdm device will clash with other domains
or even other non-bhyve nmdm devices.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Recently virtio-9p support was added to bhyve.
On the host side it looks this way:
bhyve .... -s 25:0,virtio-9p,sharename=/path/to/shared/dir
It could also have ",ro" suffix to make share read-only.
In the Linux guest, this share is mounted with:
mount -t 9p sharename /mnt/sharename
In the guest user will see the same permissions and ownership
information for this directory as on the host. No uid/gid remapping is
supported, so those could resolve to wrong user or group names.
The same applies to the other side: chowning/chmodding in the guest will
set specified ownership and permissions on the host.
In libvirt domain XML it's modeled using the 'filesystem' element:
<filesystem type='mount'>
<source dir='/path/to/shared/dir'/>
<target dir='sharename'/>
</filesystem>
Optional 'readonly' sub-element enables read-only mode.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Michal Privoznik <mprivozn@redhat.com>
Support setting a password for the VNC framebuffer using the passwd
attribute on the <graphics/> element, if the driver has the
BHYVE_CAP_VNC_PASSWORD capability.
Note that virsh domxml-from-native does not output the password in the
generated XML, as VIR_DOMAIN_DEF_FORMAT_SECURE is not set when
formatting the domain definition.
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
The resolution of the VNC framebuffer can now be set via the resolution
definition introduced in 5.9.0.
Also, add "gop" to the list of model types the <resolution/>
sub-element is valid for.
Signed-off-by: Fabian Freyer <fabian.freyer@physik.tu-berlin.de>
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Currently, slot 1 is only allowed to be used by the LPC device.
Relax this requirement and allow to use slot 1 if it was explicitly
specified by the user for any other device type. In this case the LPC
device will have the next available address.
If slot 1 was not used by the user, it'll be reserved for the LPC
device, even if it is not configured to make address assignment
consistent in case the LPC device becomes necessary (e.g. the user
adds a console or a video device which require LPC).
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Support modeling of the 'isa' controller for bhyve. User can manually
define any PCI slot for the 'isa' controller, including PCI slot 1,
but other devices are not allowed to use this address.
When domain configuration requires the 'isa' controller to be present,
automatically add it on domain post-parse stage.
Now, as this controller is always available when needed, it's not
necessary to implicitly add it to the bhyve command line, so remove
bhyveBuildLPCArgStr().
Also, make bhyveDomainDefNeedsISAController() static as it's no longer
used outside of bhyve_domain.c.
As more than one ISA controller is not supported by bhyve,
and multiple controllers with the same index are forbidden,
so forbid ISA controllers with non-zero index for bhyve.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
bhyve supports intel hda sound devices that could be specified
on the command like using "-1:0,hda,play=$play_dev,rec=$rec_dev",
where "1:0" is a PCI address, and "$play_dev" and "$rec_dev"
point to the playback and recording device on the host respectively.
Currently, schema of the 'sound' element doesn't allow specifying
neither playback nor recording devices, so for now hardcode
/dev/dsp0, which is the first audio device on the host.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
There is an inconsistency with VIR_TEST_DEBUG() calls. One half
(roughly) of calls does have the newline character the other one
doesn't. Well, it doesn't have it because it assumed blindly that
new line will be printed, which is not the case.
Signed-off-by: Michal Privoznik <mprivozn@redhat.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
In preparation libtool "-module" flag removal, add lib prefix to all
mock shared objects.
While at it, introduce VIR_TEST_MOCK macros that makes path out of mock
name to be used with VIR_TEST_PRELOAD or VIR_TEST_MAIN_PRELOAD. That,
hopefully, improves readability, reduces line length and allows to
tailor VIR_TEST_MOCK for specific platform if it has shared library
suffix different from ".so".
Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com>
Implement the MSRs ignore unknown reads and writes feature
that's specified using:
<features>
...
<msrs unknown='ignore'>
...
</features>
in the domain XML.
In bhyve, it's just passing '-w' command line argument to the bhyve(8)
executable.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: Cole Robinson <crobinso@redhat.com>
Implement support for passing custom command line arguments
to bhyve using the 'bhyve:commandline' element:
<bhyve:commandline>
<bhyve:arg value='-newarg'/>
</bhyve:commandline>
* Define virDomainXMLNamespace for the bhyve driver, which
at this point supports only the 'commandline' element
described above,
* Update command generation code to inject these command line
arguments between driver-generated arguments and the vmname
positional argument.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
The QEMU command line arguments are very long and currently all written
on a single line to /var/log/libvirt/qemu/$GUEST.log. This introduces
logic to add line breaks after every env variable and "-" optional
argument, and every positional argument. This will create a clearer log
file, which will in turn present better in bug reports when people cut +
paste from the log into a bug comment.
An example log file entry now looks like this:
2018-12-14 12:57:03.677+0000: starting up libvirt version: 5.0.0, qemu version: 3.0.0qemu-3.0.0-1.fc29, kernel: 4.19.5-300.fc29.x86_64, hostname: localhost.localdomain
LC_ALL=C \
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin \
HOME=/home/berrange \
USER=berrange \
LOGNAME=berrange \
QEMU_AUDIO_DRV=none \
/usr/bin/qemu-system-ppc64 \
-name guest=guest,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/home/berrange/.config/libvirt/qemu/lib/domain-33-guest/master-key.aes \
-machine pseries-2.10,accel=tcg,usb=off,dump-guest-core=off \
-m 1024 \
-realtime mlock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid c8a74977-ab18-41d0-ae3b-4041c7fffbcd \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=23,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc \
-no-shutdown \
-boot strict=on \
-device qemu-xhci,id=usb,bus=pci.0,addr=0x1 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x2 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2018-12-14 12:57:03.730+0000: shutting down, reason=failed
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Recently, bhyve started supporting specifying guest CPU topology.
It looks this way:
bhyve -c cpus=C,sockets=S,cores=C,threads=T ...
The old behaviour was bhyve -c C, where C is a number of vCPUs, is
still supported.
So if we have CPU topology in the domain XML, use the new syntax,
otherwise keep the old behaviour.
Also, document this feature in the bhyve driver page.
Signed-off-by: Roman Bogorodskiy <bogorodskiy@gmail.com>
Reviewed-by: John Ferlan <jferlan@redhat.com>
Roman Bogorodskiy
Michal Privoznik
Alexander ShurshaandRoman Bogorodskiy
Pavel Hrdina
Peter Krempa
Fabian Freyer
Ján Tomko
Ryan Moeller
Daniel P. Berrangé
Roman Bolshakov