mirror of
https://github.com/vagrant-libvirt/vagrant-libvirt.git
synced 2025-02-25 18:55:27 -06:00
Added support for redirected devices and USB filter.
This commit is contained in:
109
README.md
109
README.md
@@ -30,6 +30,7 @@ can help a lot :-)
|
|||||||
- [CDROMs](#cdroms)
|
- [CDROMs](#cdroms)
|
||||||
- [Input](#input)
|
- [Input](#input)
|
||||||
- [PCI device passthrough](#pci-device-passthrough)
|
- [PCI device passthrough](#pci-device-passthrough)
|
||||||
|
- [USB Redirector Devices](#usb-redirector-devices)
|
||||||
- [Random number generator passthrough](#random-number-generator-passthrough)
|
- [Random number generator passthrough](#random-number-generator-passthrough)
|
||||||
- [CPU Features](#cpu-features)
|
- [CPU Features](#cpu-features)
|
||||||
- [No box and PXE boot](#no-box-and-pxe-boot)
|
- [No box and PXE boot](#no-box-and-pxe-boot)
|
||||||
@@ -79,7 +80,33 @@ kvm type virtual machines with `virsh` or `virt-manager`.
|
|||||||
|
|
||||||
Next, you must have [Vagrant
|
Next, you must have [Vagrant
|
||||||
installed](http://docs.vagrantup.com/v2/installation/index.html).
|
installed](http://docs.vagrantup.com/v2/installation/index.html).
|
||||||
Vagrant-libvirt supports Vagrant 1.5, 1.6, 1.7 and 1.8.
|
Vagrant-libvirt supports Vagrant 1.5, 1.6, 1.7 and 1.8.
|
||||||
|
*We only test with the upstream version!* If you decide to install your distros
|
||||||
|
version and you run into problems, as a first step you should switch to upstream.
|
||||||
|
|
||||||
|
Now you need to make sure your have all the build dependencies installed for
|
||||||
|
vagrant-libvirt. This depends on your distro. An overview:
|
||||||
|
|
||||||
|
* Ubuntu 12.04/14.04/16.04, Debian:
|
||||||
|
```shell
|
||||||
|
apt-get build-dep vagrant ruby-libvirt; apt-get install qemu libvirt-bin ebtables dnsmasq
|
||||||
|
```
|
||||||
|
|
||||||
|
* CentOS 6, 7, Fedora 21:
|
||||||
|
```shell
|
||||||
|
yum install qemu libvirt libvirt-devel ruby-devel gcc qemu-kvm
|
||||||
|
```
|
||||||
|
|
||||||
|
* Fedora 22 and up:
|
||||||
|
```shell
|
||||||
|
dnf -y install qemu libvirt libvirt-devel ruby-devel gcc
|
||||||
|
```
|
||||||
|
|
||||||
|
* Arch linux: look at tips and solutions from Arch wiki.
|
||||||
|
```shell
|
||||||
|
pacman -Sy vagrant
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
Now you're ready to install vagrant-libvirt using standard [Vagrant
|
Now you're ready to install vagrant-libvirt using standard [Vagrant
|
||||||
plugin](http://docs.vagrantup.com/v2/plugins/usage.html) installation methods.
|
plugin](http://docs.vagrantup.com/v2/plugins/usage.html) installation methods.
|
||||||
@@ -106,6 +133,8 @@ $ sudo dnf install libxslt-devel libxml2-devel libvirt-devel \
|
|||||||
libguestfs-tools-c ruby-devel gcc
|
libguestfs-tools-c ruby-devel gcc
|
||||||
```
|
```
|
||||||
|
|
||||||
|
On Arch linux it is recommended to follow [steps from ArchWiki](https://wiki.archlinux.org/index.php/Vagrant#vagrant-libvirt).
|
||||||
|
|
||||||
If have problem with installation - check your linker. It should be `ld.gold`:
|
If have problem with installation - check your linker. It should be `ld.gold`:
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
@@ -114,6 +143,11 @@ sudo alternatives --set ld /usr/bin/ld.gold
|
|||||||
sudo ln -fs /usr/bin/ld.gold /usr/bin/ld
|
sudo ln -fs /usr/bin/ld.gold /usr/bin/ld
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you have issues building ruby-libvirt, try the following:
|
||||||
|
```shell
|
||||||
|
CONFIGURE_ARGS='with-ldflags=-L/opt/vagrant/embedded/lib with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib' GEM_HOME=~/.vagrant.d/gems GEM_PATH=$GEM_HOME:/opt/vagrant/embedded/gems PATH=/opt/vagrant/embedded/bin:$PATH vagrant plugin install vagrant-libvirt
|
||||||
|
```
|
||||||
|
|
||||||
## Vagrant Project Preparation
|
## Vagrant Project Preparation
|
||||||
|
|
||||||
### Add Box
|
### Add Box
|
||||||
@@ -233,8 +267,9 @@ end
|
|||||||
mode](https://libvirt.org/formatdomain.html#elementsCPU). Defaults to
|
mode](https://libvirt.org/formatdomain.html#elementsCPU). Defaults to
|
||||||
'host-model' if not set. Allowed values: host-model, host-passthrough,
|
'host-model' if not set. Allowed values: host-model, host-passthrough,
|
||||||
custom.
|
custom.
|
||||||
* `cpu_model` - CPU Model. Defaults to 'qemu64' if not set. This can really
|
* `cpu_model` - CPU Model. Defaults to 'qemu64' if not set and `cpu_mode` is
|
||||||
only be used when setting `cpu_mode` to `custom`.
|
`custom` and to '' otherwise. This can really only be used when setting
|
||||||
|
`cpu_mode` to `custom`.
|
||||||
* `cpu_fallback` - Whether to allow libvirt to fall back to a CPU model close
|
* `cpu_fallback` - Whether to allow libvirt to fall back to a CPU model close
|
||||||
to the specified model if features in the guest CPU are not supported on the
|
to the specified model if features in the guest CPU are not supported on the
|
||||||
host. Defaults to 'allow' if not set. Allowed values: `allow`, `forbid`.
|
host. Defaults to 'allow' if not set. Allowed values: `allow`, `forbid`.
|
||||||
@@ -267,7 +302,7 @@ end
|
|||||||
* `keymap` - Set keymap for vm. default: en-us
|
* `keymap` - Set keymap for vm. default: en-us
|
||||||
* `kvm_hidden` - [Hide the hypervisor from the
|
* `kvm_hidden` - [Hide the hypervisor from the
|
||||||
guest](https://libvirt.org/formatdomain.html#elementsFeatures). Useful for
|
guest](https://libvirt.org/formatdomain.html#elementsFeatures). Useful for
|
||||||
GPU passthrough on stubborn drivers. Default is false.
|
[GPU passthrough](#pci-device-passthrough) on stubborn drivers. Default is false.
|
||||||
* `video_type` - Sets the graphics card type exposed to the guest. Defaults to
|
* `video_type` - Sets the graphics card type exposed to the guest. Defaults to
|
||||||
"cirrus". [Possible
|
"cirrus". [Possible
|
||||||
values](http://libvirt.org/formatdomain.html#elementsVideo) are "vga",
|
values](http://libvirt.org/formatdomain.html#elementsVideo) are "vga",
|
||||||
@@ -564,6 +599,9 @@ provider level.
|
|||||||
* `management_network_address` - Address of network to which all VMs will be
|
* `management_network_address` - Address of network to which all VMs will be
|
||||||
connected. Must include the address and subnet mask. If not specified the
|
connected. Must include the address and subnet mask. If not specified the
|
||||||
default is '192.168.121.0/24'.
|
default is '192.168.121.0/24'.
|
||||||
|
* `management_network_mode` - Network mode for the libvirt management network.
|
||||||
|
Specify one of veryisolated, none, nat or route options. Further documentated
|
||||||
|
under [Private Networks](#private-network-options)
|
||||||
* `management_network_guest_ipv6` - Enable or disable guest-to-guest IPv6
|
* `management_network_guest_ipv6` - Enable or disable guest-to-guest IPv6
|
||||||
communication. See
|
communication. See
|
||||||
[here](https://libvirt.org/formatnetwork.html#examplesPrivate6), and
|
[here](https://libvirt.org/formatnetwork.html#examplesPrivate6), and
|
||||||
@@ -657,7 +695,7 @@ You can specify multiple inputs to the VM via `libvirt.input`. Available
|
|||||||
options are listed below. Note that both options are required:
|
options are listed below. Note that both options are required:
|
||||||
|
|
||||||
* `type` - The type of the input
|
* `type` - The type of the input
|
||||||
* `bus` - The bust of the input
|
* `bus` - The bus of the input
|
||||||
|
|
||||||
```ruby
|
```ruby
|
||||||
Vagrant.configure("2") do |config|
|
Vagrant.configure("2") do |config|
|
||||||
@@ -702,6 +740,63 @@ Vagrant.configure("2") do |config|
|
|||||||
end
|
end
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Note! Above options affect configuration only at domain creation. It won't change VM behaviour on `vagrant reload` after domain was created.
|
||||||
|
|
||||||
|
Don't forget to [set](#domain-specific-options) `kvm_hidden` option to `true` especially if you are passthroughing NVIDIA GPUs. Otherwise GPU is visible from VM but cannot be operated.
|
||||||
|
|
||||||
|
## USB Redirector Devices
|
||||||
|
You can specify multiple redirect devices via `libvirt.redirdev`. There are two types, `tcp` and `spicevmc` supported, for forwarding USB-devices to the guest. Available options are listed below.
|
||||||
|
|
||||||
|
* `type` - The type of the USB redirector device. (`tcp` or `spicevmc`)
|
||||||
|
* `host` - The host where the device is attached to. (mandatory for type `tcp`)
|
||||||
|
* `port` - The port where the device is listening. (mandatory for type `tcp`)
|
||||||
|
|
||||||
|
```ruby
|
||||||
|
Vagrant.configure("2") do |config|
|
||||||
|
config.vm.provider :libvirt do |libvirt|
|
||||||
|
# add two devices using spicevmc channel
|
||||||
|
(1..2).each do
|
||||||
|
libvirt.redirdev :type => "spicevmc"
|
||||||
|
end
|
||||||
|
# add device, provided by localhost:4000
|
||||||
|
libvirt.redirdev :type => "tcp", :host => "localhost", :port => "4000"
|
||||||
|
end
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
### Filter for USB Redirector Devices
|
||||||
|
You can define filter for redirected devices. These filters can be positiv or negative, by setting the mandatory option `allow=yes` or `allow=no`. All available options are listed below. Note the option `allow` is mandatory.
|
||||||
|
|
||||||
|
* `class` - The device class of the USB device. A list of device classes is available on [Wikipedia](https://en.wikipedia.org/wiki/USB#Device_classes).
|
||||||
|
* `vendor` - The vendor of the USB device.
|
||||||
|
* `product` - The product id of the USB device.
|
||||||
|
* `version` - The version of the USB device. Note that this is the version of `bcdDevice`
|
||||||
|
* `allow` - allow or disallow redirecting this device. (mandatory)
|
||||||
|
|
||||||
|
You can extract that information from output of `lsusb` command. Every line contains the information in format `Bus [<bus>] Device [<device>]: ID [<vendor>:[<product>]`. The `version` can be extracted from the detailed output of the device using `lsusb -D /dev/usb/[<bus>]/[<device>]`. For example:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
# get bcdDevice from
|
||||||
|
$: lsusb
|
||||||
|
Bus 001 Device 009: ID 08e6:3437 Gemalto (was Gemplus) GemPC Twin SmartCard Reader
|
||||||
|
|
||||||
|
$: lsusb -D /dev/bus/usb/001/009 | grep bcdDevice
|
||||||
|
bcdDevice 2.00
|
||||||
|
```
|
||||||
|
|
||||||
|
In this case, the USB device with `class 0x0b`, `vendor 0x08e6`, `product 0x3437` and `bcdDevice version 2.00` is allowed to be redirected to the guest. All other devices will be refused.
|
||||||
|
|
||||||
|
```ruby
|
||||||
|
Vagrant.configure("2") do |config|
|
||||||
|
config.vm.provider :libvirt do |libvirt|
|
||||||
|
libvirt.redirdev :type => "spicevmc"
|
||||||
|
libvirt.redirfilter :class => "0x0b" :vendor => "0x08e6" :product => "0x3437" :version => "2.00" :allow => "yes"
|
||||||
|
libvirt.redirfilter :allow => "no"
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
## Random number generator passthrough
|
## Random number generator passthrough
|
||||||
|
|
||||||
You can pass through `/dev/random` to your VM by configuring the domain like this:
|
You can pass through `/dev/random` to your VM by configuring the domain like this:
|
||||||
@@ -842,7 +937,7 @@ config.vm.synced_folder './', '/vagrant', type: 'rsync'
|
|||||||
or
|
or
|
||||||
|
|
||||||
```shell
|
```shell
|
||||||
config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "vagrant"
|
config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmode: "squash", owner: "1000"
|
||||||
```
|
```
|
||||||
|
|
||||||
or
|
or
|
||||||
@@ -854,6 +949,8 @@ config.vm.synced_folder './', '/vagrant', type: '9p', disabled: false, accessmod
|
|||||||
For 9p shares, a `mount: false` option allows to define synced folders without
|
For 9p shares, a `mount: false` option allows to define synced folders without
|
||||||
mounting them at boot.
|
mounting them at boot.
|
||||||
|
|
||||||
|
Further documentation on using 9p can be found [here](https://www.kernel.org/doc/Documentation/filesystems/9p.txt). Please do note that 9p depends on support in the guest and not all distros come with the 9p module by default.
|
||||||
|
|
||||||
**SECURITY NOTE:** for remote libvirt, nfs synced folders requires a bridged
|
**SECURITY NOTE:** for remote libvirt, nfs synced folders requires a bridged
|
||||||
public network interface and you must connect to libvirt via ssh.
|
public network interface and you must connect to libvirt via ssh.
|
||||||
|
|
||||||
|
|||||||
@@ -88,6 +88,10 @@ module VagrantPlugins
|
|||||||
|
|
||||||
# USB device passthrough
|
# USB device passthrough
|
||||||
@usbs = config.usbs
|
@usbs = config.usbs
|
||||||
|
|
||||||
|
# Redirected devices
|
||||||
|
@redirdevs = config.redirdevs
|
||||||
|
@redirfilters = config.redirfilters
|
||||||
|
|
||||||
# RNG device passthrough
|
# RNG device passthrough
|
||||||
@rng =config.rng
|
@rng =config.rng
|
||||||
@@ -235,6 +239,28 @@ module VagrantPlugins
|
|||||||
env[:ui].info(" -- USB passthrough: #{usb_dev.join(', ')}")
|
env[:ui].info(" -- USB passthrough: #{usb_dev.join(', ')}")
|
||||||
end
|
end
|
||||||
|
|
||||||
|
if not @redirdevs.empty?
|
||||||
|
env[:ui].info(" -- Redirected Devices: ")
|
||||||
|
@redirdevs.each do |redirdev|
|
||||||
|
msg = " -> bus=usb, type=#{redirdev[:type]}"
|
||||||
|
env[:ui].info(msg)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
|
if not @redirfilters.empty?
|
||||||
|
env[:ui].info(" -- USB Device filter for Redirected Devices: ")
|
||||||
|
@redirfilters.each do |redirfilter|
|
||||||
|
msg = " -> class=#{redirfilter[:class]}, "
|
||||||
|
msg += "vendor=#{redirfilter[:vendor]}, "
|
||||||
|
msg += "product=#{redirfilter[:product]}, "
|
||||||
|
msg += "version=#{redirfilter[:version]}, "
|
||||||
|
msg += "allow=#{redirfilter[:allow]}"
|
||||||
|
env[:ui].info(msg)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
|
||||||
env[:ui].info(" -- Command line : #{@cmd_line}")
|
env[:ui].info(" -- Command line : #{@cmd_line}")
|
||||||
|
|
||||||
# Create libvirt domain.
|
# Create libvirt domain.
|
||||||
|
|||||||
@@ -51,7 +51,7 @@ module VagrantPlugins
|
|||||||
# don't have any prefix, not even "_"
|
# don't have any prefix, not even "_"
|
||||||
""
|
""
|
||||||
else
|
else
|
||||||
config.default_prefix.to_s.concat("_")
|
config.default_prefix.to_s.dup.concat("_")
|
||||||
end
|
end
|
||||||
domain_name << env[:machine].name.to_s
|
domain_name << env[:machine].name.to_s
|
||||||
domain_name.gsub!(/[^-a-z0-9_\.]/i, '')
|
domain_name.gsub!(/[^-a-z0-9_\.]/i, '')
|
||||||
|
|||||||
@@ -117,6 +117,10 @@ module VagrantPlugins
|
|||||||
# USB device passthrough
|
# USB device passthrough
|
||||||
attr_accessor :usbs
|
attr_accessor :usbs
|
||||||
|
|
||||||
|
# Redirected devices
|
||||||
|
attr_accessor :redirdevs
|
||||||
|
attr_accessor :redirfilters
|
||||||
|
|
||||||
# Suspend mode
|
# Suspend mode
|
||||||
attr_accessor :suspend_mode
|
attr_accessor :suspend_mode
|
||||||
|
|
||||||
@@ -197,6 +201,10 @@ module VagrantPlugins
|
|||||||
|
|
||||||
# USB device passthrough
|
# USB device passthrough
|
||||||
@usbs = UNSET_VALUE
|
@usbs = UNSET_VALUE
|
||||||
|
|
||||||
|
# Redirected devices
|
||||||
|
@redirdevs = UNSET_VALUE
|
||||||
|
@redirfilters = UNSET_VALUE
|
||||||
|
|
||||||
# Suspend mode
|
# Suspend mode
|
||||||
@suspend_mode = UNSET_VALUE
|
@suspend_mode = UNSET_VALUE
|
||||||
@@ -369,6 +377,38 @@ module VagrantPlugins
|
|||||||
})
|
})
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def redirdev(options={})
|
||||||
|
if options[:type].nil?
|
||||||
|
raise 'Type must be specified.'
|
||||||
|
end
|
||||||
|
|
||||||
|
if @redirdevs == UNSET_VALUE
|
||||||
|
@redirdevs = []
|
||||||
|
end
|
||||||
|
|
||||||
|
@redirdevs.push({
|
||||||
|
type: options[:type],
|
||||||
|
})
|
||||||
|
end
|
||||||
|
|
||||||
|
def redirfilter(options={})
|
||||||
|
if options[:allow].nil?
|
||||||
|
raise 'Option allow must be specified.'
|
||||||
|
end
|
||||||
|
|
||||||
|
if @redirfilters == UNSET_VALUE
|
||||||
|
@redirfilters = []
|
||||||
|
end
|
||||||
|
|
||||||
|
@redirfilters.push({
|
||||||
|
class: options[:class] || -1,
|
||||||
|
vendor: options[:class] || -1,
|
||||||
|
product: options[:class] || -1,
|
||||||
|
version: options[:class] || -1,
|
||||||
|
allow: options[:allow],
|
||||||
|
})
|
||||||
|
end
|
||||||
|
|
||||||
# NOTE: this will run twice for each time it's needed- keep it idempotent
|
# NOTE: this will run twice for each time it's needed- keep it idempotent
|
||||||
def storage(storage_type, options={})
|
def storage(storage_type, options={})
|
||||||
if storage_type == :file
|
if storage_type == :file
|
||||||
@@ -502,7 +542,11 @@ module VagrantPlugins
|
|||||||
@memory = 512 if @memory == UNSET_VALUE
|
@memory = 512 if @memory == UNSET_VALUE
|
||||||
@cpus = 1 if @cpus == UNSET_VALUE
|
@cpus = 1 if @cpus == UNSET_VALUE
|
||||||
@cpu_mode = 'host-model' if @cpu_mode == UNSET_VALUE
|
@cpu_mode = 'host-model' if @cpu_mode == UNSET_VALUE
|
||||||
@cpu_model = 'qemu64' if @cpu_model == UNSET_VALUE
|
@cpu_model = if (@cpu_model == UNSET_VALUE and @cpu_mode == 'custom')
|
||||||
|
'qemu64'
|
||||||
|
elsif (@cpu_mode != 'custom')
|
||||||
|
''
|
||||||
|
end
|
||||||
@cpu_fallback = 'allow' if @cpu_fallback == UNSET_VALUE
|
@cpu_fallback = 'allow' if @cpu_fallback == UNSET_VALUE
|
||||||
@cpu_features = [] if @cpu_features == UNSET_VALUE
|
@cpu_features = [] if @cpu_features == UNSET_VALUE
|
||||||
@numa_nodes = @numa_nodes == UNSET_VALUE ? nil : _generate_numa()
|
@numa_nodes = @numa_nodes == UNSET_VALUE ? nil : _generate_numa()
|
||||||
@@ -558,6 +602,10 @@ module VagrantPlugins
|
|||||||
|
|
||||||
# USB device passthrough
|
# USB device passthrough
|
||||||
@usbs = [] if @usbs == UNSET_VALUE
|
@usbs = [] if @usbs == UNSET_VALUE
|
||||||
|
|
||||||
|
# Redirected devices
|
||||||
|
@redirdevs = [] if @redirdevs == UNSET_VALUE
|
||||||
|
@redirfilters = [] if @redirfilters == UNSET_VALUE
|
||||||
|
|
||||||
# Suspend mode
|
# Suspend mode
|
||||||
@suspend_mode = "pause" if @suspend_mode == UNSET_VALUE
|
@suspend_mode = "pause" if @suspend_mode == UNSET_VALUE
|
||||||
|
|||||||
@@ -7,7 +7,7 @@
|
|||||||
|
|
||||||
<cpu mode='<%= @cpu_mode %>'>
|
<cpu mode='<%= @cpu_mode %>'>
|
||||||
<% if @cpu_mode != 'host-passthrough' %>
|
<% if @cpu_mode != 'host-passthrough' %>
|
||||||
<model fallback='<%= @cpu_fallback %>'><%= @cpu_model %></model>
|
<model fallback='<%= @cpu_fallback %>'><% if @cpu_mode == 'custom' %><%= @cpu_model %><% end %></model>
|
||||||
<% if @nested %>
|
<% if @nested %>
|
||||||
<feature policy='optional' name='vmx'/>
|
<feature policy='optional' name='vmx'/>
|
||||||
<feature policy='optional' name='svm'/>
|
<feature policy='optional' name='svm'/>
|
||||||
@@ -170,6 +170,19 @@
|
|||||||
</source>
|
</source>
|
||||||
</hostdev>
|
</hostdev>
|
||||||
<% end %>
|
<% end %>
|
||||||
|
<% unless @redirdevs.empty? %>
|
||||||
|
<% @redirdevs.each do |redirdev| %>
|
||||||
|
<redirdev bus='usb' type='<%= redirdev[:type] %>'>
|
||||||
|
</redirdev>
|
||||||
|
<% end %>
|
||||||
|
<% unless @redirfilters.empty? %>
|
||||||
|
<redirfilter>
|
||||||
|
<% @redirfilters.each do |usbdev| %>
|
||||||
|
<usbdev class='<%= usbdev[:class] %>' vendor='<%= usbdev[:vendor] %>' product='<%= usbdev[:product] %>' version='<%= usbdev[:version] %>' allow='<%= usbdev[:allow] %>'/>
|
||||||
|
<% end %>
|
||||||
|
</redirfilter>
|
||||||
|
<% end %>
|
||||||
|
<% end %>
|
||||||
|
|
||||||
<% if @tpm_path -%>
|
<% if @tpm_path -%>
|
||||||
<%# TPM Device -%>
|
<%# TPM Device -%>
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
module VagrantPlugins
|
module VagrantPlugins
|
||||||
module ProviderLibvirt
|
module ProviderLibvirt
|
||||||
VERSION = '0.0.35'
|
VERSION = '0.0.36'
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
Reference in New Issue
Block a user