Merge pull request #1251 from alvistack/master-virtiofs

Add `virtiofs` Support to vagrant-libvirt

README.md

@@ -83,7 +83,7 @@ can help a lot :-)
 * SSH into domains.
 * Setup hostname and network interfaces.
 * Provision domains with any built-in Vagrant provisioner.
-* Synced folder support via `rsync`, `nfs` or `9p`.
+* Synced folder support via `rsync`, `nfs`, `9p` or `virtiofs`.
 * Snapshots via [sahara](https://github.com/jedi4ever/sahara).
 * Package caching via
   [vagrant-cachier](http://fgrehm.viewdocs.io/vagrant-cachier/).
@@ -1416,38 +1416,134 @@ Default is `eth0`.
 
 ## Synced Folders
 
-Vagrant automatically syncs the project folder on the host to `/vagrant` in the guest. You can also configure
+Vagrant automatically syncs the project folder on the host to `/vagrant` in
-additional synced folders.
+the guest. You can also configure additional synced folders.
 
-`vagrant-libvirt` supports bidirectional synced folders via [NFS](https://en.wikipedia.org/wiki/Network_File_System) or [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p or Plan 9](https://en.wikipedia.org/wiki/9P_(protocol))) and
+**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
-unidirectional via rsync. The default is NFS. Difference between NFS and 9p is explained [here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
+public network interface and you must connect to Libvirt via ssh.
 
-You can change the synced folder type for `/vagrant` by explicity configuring
+**NFS**
-it an setting the type, e.g.
 
-```shell
+`vagrant-libvirt` supports
-config.vm.synced_folder './', '/vagrant', type: 'rsync'
+[NFS](https://www.vagrantup.com/docs/synced-folders/nfs) as default with
+bidirectional synced folders.
+
+Example with NFS:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant"
+end
 ```
 
-or
+**RSync**
 
-```shell
+`vagrant-libvirt` supports
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "1000"
+[rsync](https://www.vagrantup.com/docs/synced-folders/rsync) with
+unidirectional synced folders.
+
+Example with rsync:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "rsync"
+end
 ```
 
-or
+**9P**
 
-```shell
+`vagrant-libvirt` supports [VirtFS](http://www.linux-kvm.org/page/VirtFS) ([9p
-config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "mapped", mount: false
+or Plan 9](https://en.wikipedia.org/wiki/9P_\(protocol\))) with bidirectional
-```
+synced folders.
+
+Difference between NFS and 9p is explained
+[here](https://unix.stackexchange.com/questions/240281/virtfs-plan-9-vs-nfs-as-tool-for-share-folder-for-virtual-machine).
 
 For 9p shares, a `mount: false` option allows to define synced folders without
 mounting them at boot.
 
-Further documentation on using 9p can be found in [kernel docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in [QEMU wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly). Please do note that 9p depends on support in the guest and not all distros come with the 9p module by default.
+Example for `accessmode: "squash"` with 9p:
 
-**SECURITY NOTE:** for remote Libvirt, nfs synced folders requires a bridged
+``` ruby
-public network interface and you must connect to Libvirt via ssh.
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "squash", owner: "1000"
+end
+```
+
+Example for `accessmode: "mapped"` with 9p:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.synced_folder "./", "/vagrant", type: "9p", disabled: false, accessmode: "mapped", mount: false
+end
+```
+
+Further documentation on using 9p can be found in [kernel
+docs](https://www.kernel.org/doc/Documentation/filesystems/9p.txt) and in
+[QEMU
+wiki](https://wiki.qemu.org/Documentation/9psetup#Starting_the_Guest_directly).
+
+Please do note that 9p depends on support in the guest and not all distros
+come with the 9p module by default.
+
+**Virtio-fs**
+
+`vagrant-libvirt` supports [Virtio-fs](https://virtio-fs.gitlab.io/) with
+bidirectional synced folders.
+
+For virtiofs shares, a `mount: false` option allows to define synced folders
+without mounting them at boot.
+
+So far, passthrough is the only supported access mode and it requires running
+the virtiofsd daemon as root.
+
+QEMU needs to allocate the backing memory for all the guest RAM as shared
+memory, e.g. [Use file-backed
+memory](https://libvirt.org/kbase/virtiofs.html#host-setup) by enable
+`memory_backing_dir` option in `/etc/libvirt/qemu.conf`:
+
+``` shell
+memory_backing_dir = "/dev/shm"
+```
+
+Example for Libvirt \>= 6.2.0 (e.g. Ubuntu 20.10 with Linux 5.8.0 + QEMU 5.0 +
+Libvirt 6.6.0, i.e. NUMA nodes required) with virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.numa_nodes = [{ :cpus => "0-1", :memory => 8192, :memAccess => "shared" }]
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Example for Libvirt \>= 6.9.0 (e.g. Ubuntu 21.04 with Linux 5.11.0 + QEMU 5.2 +
+Libvirt 7.0.0, or Ubuntu 20.04 + [PPA
+enabled](https://launchpad.net/~savoury1/+archive/ubuntu/virtualisation)) with
+virtiofs:
+
+``` ruby
+Vagrant.configure("2") do |config|
+  config.vm.provider :libvirt do |libvirt|
+    libvirt.cpus = 2
+    libvirt.memory = 8192
+    libvirt.memorybacking :access, :mode => "shared"
+  end
+  config.vm.synced_folder "./", "/vagrant", type: "virtiofs"
+end
+```
+
+Further documentation on using virtiofs can be found in [official
+HowTo](https://virtio-fs.gitlab.io/index.html#howto) and in [Libvirt
+KB](https://libvirt.org/kbase/virtiofs.html).
+
+Please do note that virtiofs depends on:
+
+  - Host: Linux \>= 5.4, QEMU \>= 4.2 and Libvirt \>= 6.2 (e.g. Ubuntu 20.10)
+  - Guest: Linux \>= 5.4 (e.g. Ubuntu 20.04)
 
 ## QEMU Session Support
 

lib/vagrant-libvirt/cap/mount_9p.rb

@@ -4,10 +4,10 @@ require 'vagrant/util/retryable'
 module VagrantPlugins
   module ProviderLibvirt
     module Cap
-      class MountP9
+      class Mount9P
         extend Vagrant::Util::Retryable
 
-        def self.mount_p9_shared_folder(machine, folders)
+        def self.mount_9p_shared_folder(machine, folders)
           folders.each do |_name, opts|
             # Expand the guest path so we can handle things like "~/vagrant"
             expanded_guest_path = machine.guest.capability(

lib/vagrant-libvirt/cap/mount_virtiofs.rb

@@ -0,0 +1,37 @@
+require 'digest/md5'
+require 'vagrant/util/retryable'
+
+module VagrantPlugins
+  module ProviderLibvirt
+    module Cap
+      class MountVirtioFS
+        extend Vagrant::Util::Retryable
+
+        def self.mount_virtiofs_shared_folder(machine, folders)
+          folders.each do |_name, opts|
+            # Expand the guest path so we can handle things like "~/vagrant"
+            expanded_guest_path = machine.guest.capability(
+              :shell_expand_guest_path, opts[:guestpath]
+            )
+
+            # Do the actual creating and mounting
+            machine.communicate.sudo("mkdir -p #{expanded_guest_path}")
+
+            # Mount
+            mount_tag = Digest::MD5.new.update(opts[:hostpath]).to_s[0, 31]
+
+            mount_opts = "-o #{opts[:mount_opts]}" if opts[:mount_opts]
+
+            mount_command = "mount -t virtiofs #{mount_opts} '#{mount_tag}' #{expanded_guest_path}"
+            retryable(on: Vagrant::Errors::LinuxMountFailed,
+                      tries: 5,
+                      sleep: 3) do
+              machine.communicate.sudo(mount_command,
+                                       error_class: Vagrant::Errors::LinuxMountFailed)
+            end
+          end
+        end
+      end
+    end
+  end
+end

lib/vagrant-libvirt/cap/synced_folder_9p.rb

@@ -6,10 +6,9 @@ require 'digest/md5'
 require 'vagrant/util/subprocess'
 require 'vagrant/errors'
 require 'vagrant-libvirt/errors'
-# require_relative "helper"
 
 module VagrantPlugins
-  module SyncedFolder9p
+  module SyncedFolder9P
     class SyncedFolder < Vagrant.plugin('2', :synced_folder)
       include Vagrant::Util
       include VagrantPlugins::ProviderLibvirt::Util::ErbTemplate
@@ -69,10 +68,10 @@ module VagrantPlugins
         end
       end
 
-      # TODO: once up, mount folders
+      # once up, mount folders
       def enable(machine, folders, _opts)
         # Go through each folder and mount
-        machine.ui.info('mounting p9 share in guest')
+        machine.ui.info('mounting 9p share in guest')
         # Only mount folders that have a guest path specified.
         mount_folders = {}
         folders.each do |id, opts|
@@ -83,7 +82,7 @@ module VagrantPlugins
         end
         # Mount the actual folder
         machine.guest.capability(
-          :mount_p9_shared_folder, mount_folders
+          :mount_9p_shared_folder, mount_folders
         )
       end
 

lib/vagrant-libvirt/cap/synced_folder_virtiofs.rb

@@ -0,0 +1,109 @@
+require 'log4r'
+require 'ostruct'
+require 'nokogiri'
+require 'digest/md5'
+
+require 'vagrant/util/subprocess'
+require 'vagrant/errors'
+require 'vagrant-libvirt/errors'
+
+module VagrantPlugins
+  module SyncedFolderVirtioFS
+    class SyncedFolder < Vagrant.plugin('2', :synced_folder)
+      include Vagrant::Util
+      include VagrantPlugins::ProviderLibvirt::Util::ErbTemplate
+
+      def initialize(*args)
+        super
+        @logger = Log4r::Logger.new('vagrant_libvirt::synced_folders::virtiofs')
+      end
+
+      def usable?(machine, _raise_error = false)
+        # bail now if not using Libvirt since checking version would throw error
+        return false unless machine.provider_name == :libvirt
+
+        # virtiofs support introduced since 6.2.0
+        # version number format is major * 1,000,000 + minor * 1,000 + release
+        libvirt_version = machine.provider.driver.connection.client.libversion
+        libvirt_version >= 6_002_000
+      end
+
+      def prepare(machine, folders, _opts)
+        raise Vagrant::Errors::Error('No Libvirt connection') if machine.provider.driver.connection.nil?
+        @conn = machine.provider.driver.connection.client
+
+        begin
+          # loop through folders
+          folders.each do |id, folder_opts|
+            folder_opts.merge!(target: id,
+                               mount: true,
+                               readonly: nil) { |_k, ov, _nv| ov }
+
+            mount_tag = Digest::MD5.new.update(folder_opts[:hostpath]).to_s[0, 31]
+            folder_opts[:mount_tag] = mount_tag
+
+            machine.ui.info "================\nMachine id: #{machine.id}\nShould be mounting folders\n #{id}, opts: #{folder_opts}"
+
+            #xml = to_xml('filesystem', folder_opts)
+            xml = Nokogiri::XML::Builder.new do |xml|
+              xml.filesystem(type: 'mount', accessmode: 'passthrough') do
+                xml.driver(type: 'virtiofs')
+                xml.source(dir: folder_opts[:hostpath])
+                xml.target(dir: mount_tag)
+                xml.readonly unless folder_opts[:readonly].nil?
+              end
+            end.to_xml(
+              save_with: Nokogiri::XML::Node::SaveOptions::NO_DECLARATION |
+                         Nokogiri::XML::Node::SaveOptions::NO_EMPTY_TAGS |
+                         Nokogiri::XML::Node::SaveOptions::FORMAT
+            )
+            # puts "<<<<< XML:\n #{xml}\n >>>>>"
+            @conn.lookup_domain_by_uuid(machine.id).attach_device(xml, 0)
+          end
+        rescue => e
+          machine.ui.error("could not attach device because: #{e}")
+          raise VagrantPlugins::ProviderLibvirt::Errors::AttachDeviceError,
+                error_message: e.message
+        end
+      end
+
+      # once up, mount folders
+      def enable(machine, folders, _opts)
+        # Go through each folder and mount
+        machine.ui.info('mounting virtiofs share in guest')
+        # Only mount folders that have a guest path specified.
+        mount_folders = {}
+        folders.each do |id, opts|
+          next unless opts[:mount] && opts[:guestpath] && !opts[:guestpath].empty?
+          mount_folders[id] = opts.dup
+        end
+        # Mount the actual folder
+        machine.guest.capability(
+          :mount_virtiofs_shared_folder, mount_folders
+        )
+      end
+
+      def cleanup(machine, _opts)
+        if machine.provider.driver.connection.nil?
+          raise Vagrant::Errors::Error('No Libvirt connection')
+        end
+        @conn = machine.provider.driver.connection.client
+        begin
+          if machine.id && machine.id != ''
+            dom = @conn.lookup_domain_by_uuid(machine.id)
+            Nokogiri::XML(dom.xml_desc).xpath(
+              '/domain/devices/filesystem'
+            ).each do |xml|
+              dom.detach_device(xml.to_s)
+              machine.ui.info 'Cleaned up shared folders'
+            end
+          end
+        rescue => e
+          machine.ui.error("could not detach device because: #{e}")
+          raise VagrantPlugins::ProviderLibvirt::Errors::DetachDeviceError,
+                error_message: e.message
+        end
+      end
+    end
+  end
+end

lib/vagrant-libvirt/plugin.rb

@@ -30,9 +30,13 @@ module VagrantPlugins
         hook.after Vagrant::Action::Builtin::BoxRemove, Action.remove_libvirt_image
       end
 
-      guest_capability('linux', 'mount_p9_shared_folder') do
+      guest_capability('linux', 'mount_9p_shared_folder') do
-        require_relative 'cap/mount_p9'
+        require_relative 'cap/mount_9p'
-        Cap::MountP9
+        Cap::Mount9P
+      end
+      guest_capability('linux', 'mount_virtiofs_shared_folder') do
+        require_relative 'cap/mount_virtiofs'
+        Cap::MountVirtioFS
       end
 
       provider_capability(:libvirt, :nic_mac_addresses) do
@@ -48,8 +52,12 @@ module VagrantPlugins
       # lower priority than nfs or rsync
       # https://github.com/vagrant-libvirt/vagrant-libvirt/pull/170
       synced_folder('9p', 4) do
-        require_relative 'cap/synced_folder'
+        require_relative 'cap/synced_folder_9p'
-        VagrantPlugins::SyncedFolder9p::SyncedFolder
+        VagrantPlugins::SyncedFolder9P::SyncedFolder
+      end
+      synced_folder('virtiofs', 5) do
+        require_relative 'cap/synced_folder_virtiofs'
+        VagrantPlugins::SyncedFolderVirtioFS::SyncedFolder
       end
 
       # This initializes the internationalization strings.