DOCS shift to rst - Deploying Your Application with Deployment Manager (#16453)
This commit is contained in:
Sebastian Golebiewski
GitHub
parent
2f69305aa3
commit
066ef694f5
@ -1,202 +1,184 @@
|
||||
# Deploying Your Application with Deployment Manager {#openvino_docs_install_guides_deployment_manager_tool}
|
||||
|
||||
The OpenVINO™ Deployment Manager is a Python command-line tool that creates a deployment package by assembling the model, OpenVINO IR files, your application, and associated dependencies into a runtime package for your target device. This tool is delivered within the Intel® Distribution of OpenVINO™ toolkit for Linux, Windows and macOS release packages. It is available in the `<INSTALL_DIR>/tools/deployment_manager` directory after installation.
|
||||
@sphinxdirective
|
||||
|
||||
The OpenVINO™ Deployment Manager is a Python command-line tool that creates a deployment package by assembling the model, OpenVINO IR files, your application, and associated dependencies into a runtime package for your target device. This tool is delivered within the Intel® Distribution of OpenVINO™ toolkit for Linux, Windows and macOS release packages. It is available in the ``<INSTALL_DIR>/tools/deployment_manager`` directory after installation.
|
||||
|
||||
This article provides instructions on how to create a package with Deployment Manager and then deploy the package to your target systems.
|
||||
|
||||
## Prerequisites
|
||||
Prerequisites
|
||||
####################
|
||||
|
||||
To use the Deployment Manager tool, the following requirements need to be met:
|
||||
* Intel® Distribution of OpenVINO™ toolkit is installed. See the [Installation Guide](../../install_guides/installing-openvino-overview.md) for instructions on different operating systems.
|
||||
|
||||
* Intel® Distribution of OpenVINO™ toolkit is installed. See the :doc:`Installation Guide <openvino_docs_install_guides_overview>` for instructions on different operating systems.
|
||||
* To run inference on a target device other than CPU, device drivers must be pre-installed:
|
||||
* **For GPU**, see [Configurations for Intel® Processor Graphics (GPU)](../../install_guides/configurations-for-intel-gpu.md).
|
||||
* **For GNA**, see [Intel® Gaussian & Neural Accelerator (GNA)](../../install_guides/configurations-for-intel-gna.md)
|
||||
|
||||
> **IMPORTANT**: The operating system on the target system must be the same as the development system on which you are creating the package. For example, if the target system is Ubuntu 18.04, the deployment package must be created from the OpenVINO™ toolkit installed on Ubuntu 18.04.
|
||||
* **For GPU**, see :doc:`Configurations for Intel® Processor Graphics (GPU) <openvino_docs_install_guides_configurations_for_intel_gpu>`.
|
||||
* **For GNA**, see :doc:`Intel® Gaussian & Neural Accelerator (GNA) <openvino_docs_install_guides_configurations_for_intel_gna>`
|
||||
|
||||
> **TIP**: If your application requires additional dependencies, including the Microsoft Visual C++ Redistributable, use the ['--user_data' option](https://docs.openvino.ai/latest/openvino_docs_install_guides_deployment_manager_tool.html#run-standard-cli-mode) to add them to the deployment archive. Install these dependencies on the target host before running inference.
|
||||
.. important::
|
||||
|
||||
## Creating Deployment Package Using Deployment Manager
|
||||
The operating system on the target system must be the same as the development system on which you are creating the package. For example, if the target system is Ubuntu 18.04, the deployment package must be created from the OpenVINO™ toolkit installed on Ubuntu 18.04.
|
||||
|
||||
.. tip::
|
||||
|
||||
If your application requires additional dependencies, including the Microsoft Visual C++ Redistributable, use the `'--user_data' option <#running-deployment-manager-in-standard-cli-mode>`__ to add them to the deployment archive. Install these dependencies on the target host before running inference.
|
||||
|
||||
Creating Deployment Package Using Deployment Manager
|
||||
####################################################
|
||||
|
||||
To create a deployment package that includes inference-related components of OpenVINO™ toolkit, you can run the Deployment Manager tool in either interactive or standard CLI mode .
|
||||
|
||||
### Running Deployment Manager in Interactive Mode
|
||||
Running Deployment Manager in Interactive Mode
|
||||
++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
@sphinxdirective
|
||||
.. dropdown:: Click to expand/collapse
|
||||
|
||||
.. raw:: html
|
||||
The interactive mode provides a user-friendly command-line interface that guides through the process with text prompts.
|
||||
|
||||
<div class="collapsible-section" data-title="Click to expand/collapse">
|
||||
To launch the Deployment Manager in interactive mode, open a new terminal window, go to the Deployment Manager tool directory, and run the tool script without parameters:
|
||||
|
||||
@endsphinxdirective
|
||||
.. tab:: Linux
|
||||
|
||||
The interactive mode provides a user-friendly command-line interface that guides through the process with text prompts.
|
||||
.. code-block:: sh
|
||||
|
||||
To launch the Deployment Manager in interactive mode, open a new terminal window, go to the Deployment Manager tool directory, and run the tool script without parameters:
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
|
||||
./deployment_manager.py
|
||||
|
||||
.. tab:: Windows
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
cd <INSTALL_DIR>\deployment_tools\tools\deployment_manager
|
||||
.\deployment_manager.py
|
||||
|
||||
.. tab:: macOS
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py
|
||||
|
||||
@endsphinxdirective
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
|
||||
The target device selection dialog is displayed:
|
||||
|
||||
|
||||
./deployment_manager.py
|
||||
|
||||
Use the options provided on the screen to complete the selection of the target devices, and press **Enter** to proceed to the package generation dialog. To interrupt the generation process and exit the program, type **q** and press **Enter**.
|
||||
.. tab:: Windows
|
||||
|
||||
Once the selection is accepted, the package generation dialog will appear:
|
||||
|
||||
|
||||
.. code-block:: bat
|
||||
|
||||
The target devices selected in the previous step appear on the screen. To go back and change the selection, type **b** and press **Enter**. Use the default settings, or use the following options to configure the generation process:
|
||||
|
||||
* `o. Change output directory` (optional): the path to the output directory. By default, it is set to your home directory.
|
||||
cd <INSTALL_DIR>\deployment_tools\tools\deployment_manager
|
||||
.\deployment_manager.py
|
||||
|
||||
* `u. Provide (or change) path to folder with user data` (optional): the path to a directory with user data (OpenVINO IR, model, dataset, etc.) files and subdirectories required for inference, which will be added to the deployment archive. By default, it is set to `None`, which means that copying the user data to the target system need to be done separately.
|
||||
.. tab:: macOS
|
||||
|
||||
* `t. Change archive name` (optional): the deployment archive name without extension. By default, it is set to `openvino_deployment_package`.
|
||||
|
||||
After all the parameters are set, type **g** and press **Enter** to generate the package for the selected target devices. To interrupt the generation process and exit the program, type **q** and press **Enter**.
|
||||
.. code-block:: sh
|
||||
|
||||
Once the script has successfully completed, the deployment package is generated in the specified output directory.
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. raw:: html
|
||||
The target device selection dialog is displayed:
|
||||
|
||||
</div>
|
||||
.. image:: _static/images/selection_dialog.png
|
||||
:alt: Deployment Manager selection dialog
|
||||
|
||||
@endsphinxdirective
|
||||
Use the options provided on the screen to complete the selection of the target devices, and press **Enter** to proceed to the package generation dialog. To interrupt the generation process and exit the program, type **q** and press **Enter**.
|
||||
|
||||
### Running Deployment Manager in Standard CLI Mode
|
||||
Once the selection is accepted, the package generation dialog will appear:
|
||||
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/configuration_dialog.png
|
||||
:alt: Deployment Manager configuration dialog
|
||||
|
||||
.. raw:: html
|
||||
The target devices selected in the previous step appear on the screen. To go back and change the selection, type **b** and press **Enter**. Use the default settings, or use the following options to configure the generation process:
|
||||
|
||||
<div class="collapsible-section" data-title="Click to expand/collapse">
|
||||
* ``o. Change output directory`` (optional): the path to the output directory. By default, it is set to your home directory.
|
||||
|
||||
@endsphinxdirective
|
||||
* ``u. Provide (or change) path to folder with user data`` (optional): the path to a directory with user data (OpenVINO IR, model, dataset, etc.) files and subdirectories required for inference, which will be added to the deployment archive. By default, it is set to ``None``, which means that copying the user data to the target system need to be done separately.
|
||||
|
||||
You can also run the Deployment Manager tool in the standard CLI mode. In this mode, specify the target devices and other parameters as command-line arguments of the Deployment Manager Python script. This mode facilitates integrating the tool in an automation pipeline.
|
||||
* ``t. Change archive name`` (optional): the deployment archive name without extension. By default, it is set to ``openvino_deployment_package``.
|
||||
|
||||
To launch the Deployment Manager tool in the standard mode: open a new terminal window, go to the Deployment Manager tool directory, and run the tool command with the following syntax:
|
||||
After all the parameters are set, type **g** and press **Enter** to generate the package for the selected target devices. To interrupt the generation process and exit the program, type **q** and press **Enter**.
|
||||
|
||||
@sphinxdirective
|
||||
Once the script has successfully completed, the deployment package is generated in the specified output directory.
|
||||
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: sh
|
||||
Running Deployment Manager in Standard CLI Mode
|
||||
+++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
.. tab:: Windows
|
||||
.. dropdown:: Click to expand/collapse
|
||||
|
||||
.. code-block:: bat
|
||||
You can also run the Deployment Manager tool in the standard CLI mode. In this mode, specify the target devices and other parameters as command-line arguments of the Deployment Manager Python script. This mode facilitates integrating the tool in an automation pipeline.
|
||||
|
||||
cd <INSTALL_DIR>\tools\deployment_manager
|
||||
.\deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
To launch the Deployment Manager tool in the standard mode: open a new terminal window, go to the Deployment Manager tool directory, and run the tool command with the following syntax:
|
||||
|
||||
.. tab:: macOS
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: sh
|
||||
.. code-block:: sh
|
||||
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
@endsphinxdirective
|
||||
.. tab:: Windows
|
||||
|
||||
The following options are available:
|
||||
.. code-block:: bat
|
||||
|
||||
* `<--targets>` (required): the list of target devices to run inference. To specify more than one target, separate them with spaces, for example, `--targets cpu gpu`.
|
||||
To get a list of currently available targets, run the program with the `-h` option.
|
||||
cd <INSTALL_DIR>\tools\deployment_manager
|
||||
.\deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
* `[--output_dir]` (optional): the path to the output directory. By default, it is set to your home directory.
|
||||
.. tab:: macOS
|
||||
|
||||
* `[--archive_name]` (optional): a deployment archive name without extension. By default, it is set to `openvino_deployment_package`.
|
||||
.. code-block:: sh
|
||||
|
||||
* `[--user_data]` (optional): the path to a directory with user data (OpenVINO IR, model, dataset, etc.) files and subdirectories required for inference, which will be added to the deployment archive. By default, it is set to `None`, which means copying the user data to the target system need to be performed separately.
|
||||
cd <INSTALL_DIR>/tools/deployment_manager
|
||||
./deployment_manager.py <--targets> [--output_dir] [--archive_name] [--user_data]
|
||||
|
||||
Once the script has successfully completed, the deployment package is generated in the output directory specified.
|
||||
|
||||
@sphinxdirective
|
||||
The following options are available:
|
||||
|
||||
.. raw:: html
|
||||
* ``<--targets>`` (required): the list of target devices to run inference. To specify more than one target, separate them with spaces, for example, ``--targets cpu gpu``.
|
||||
To get a list of currently available targets, run the program with the ``-h`` option.
|
||||
|
||||
</div>
|
||||
* ``[--output_dir]`` (optional): the path to the output directory. By default, it is set to your home directory.
|
||||
|
||||
@endsphinxdirective
|
||||
* ``[--archive_name]`` (optional): a deployment archive name without extension. By default, it is set to ``openvino_deployment_package``.
|
||||
|
||||
## Deploying Package on Target Systems
|
||||
* ``[--user_data]`` (optional): the path to a directory with user data (OpenVINO IR, model, dataset, etc.) files and subdirectories required for inference, which will be added to the deployment archive. By default, it is set to ``None``, which means copying the user data to the target system need to be performed separately.
|
||||
|
||||
Once the Deployment Manager has successfully completed, the `.tar.gz` (on Linux or macOS) or `.zip` (on Windows) package is generated in the specified output directory.
|
||||
Once the script has successfully completed, the deployment package is generated in the output directory specified.
|
||||
|
||||
|
||||
Deploying Package on Target Systems
|
||||
###################################
|
||||
|
||||
Once the Deployment Manager has successfully completed, the ``.tar.gz`` (on Linux or macOS) or ``.zip`` (on Windows) package is generated in the specified output directory.
|
||||
|
||||
To deploy the OpenVINO Runtime components from the development machine to the target system, perform the following steps:
|
||||
|
||||
1. Copy the generated archive to the target system by using your preferred method.
|
||||
|
||||
2. Extract the archive to the destination directory on the target system. If the name of your archive is different from the default one shown below, replace `openvino_deployment_package` with your specified name.
|
||||
@sphinxdirective
|
||||
2. Extract the archive to the destination directory on the target system. If the name of your archive is different from the default one shown below, replace ``openvino_deployment_package`` with your specified name.
|
||||
|
||||
.. tab:: Linux
|
||||
.. tab:: Linux
|
||||
|
||||
.. code-block:: sh
|
||||
.. code-block:: sh
|
||||
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
|
||||
.. tab:: Windows
|
||||
.. tab:: Windows
|
||||
|
||||
.. code-block:: bat
|
||||
.. code-block:: bat
|
||||
|
||||
Use the archiver of your choice to unzip the file.
|
||||
Use the archiver of your choice to unzip the file.
|
||||
|
||||
.. tab:: macOS
|
||||
.. tab:: macOS
|
||||
|
||||
.. code-block:: sh
|
||||
.. code-block:: sh
|
||||
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
|
||||
@endsphinxdirective
|
||||
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
|
||||
|
||||
|
||||
Now, the package is extracted to the destination directory. The following files and subdirectories are created:
|
||||
|
||||
* `setupvars.sh` — a copy of `setupvars.sh`.
|
||||
* `runtime` — contains the OpenVINO runtime binary files.
|
||||
* `install_dependencies` — a snapshot of the `install_dependencies` directory from the OpenVINO installation directory.
|
||||
* `<user_data>` — the directory with the user data (OpenVINO IR, model, dataset, etc.) specified while configuring the package.
|
||||
|
||||
3. On a target Linux system, to run inference install additional dependencies by running the `install_openvino_dependencies.sh` script:
|
||||
```sh
|
||||
cd <destination_dir>/openvino/install_dependencies
|
||||
sudo -E ./install_openvino_dependencies.sh
|
||||
```
|
||||
* ``setupvars.sh`` — a copy of ``setupvars.sh``.
|
||||
* ``runtime`` — contains the OpenVINO runtime binary files.
|
||||
* ``install_dependencies`` — a snapshot of the ``install_dependencies`` directory from the OpenVINO installation directory.
|
||||
* ``<user_data>`` — the directory with the user data (OpenVINO IR, model, dataset, etc.) specified while configuring the package.
|
||||
|
||||
3. On a target Linux system, to run inference install additional dependencies by running the ``install_openvino_dependencies.sh`` script:
|
||||
|
||||
.. code-block: sh
|
||||
|
||||
cd <destination_dir>/openvino/install_dependencies
|
||||
sudo -E ./install_openvino_dependencies.sh
|
||||
|
||||
|
||||
4. Set up the environment variables:
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: Linux
|
||||
|
||||
@ -219,7 +201,7 @@ To deploy the OpenVINO Runtime components from the development machine to the ta
|
||||
cd <destination_dir>/openvino/
|
||||
source ./setupvars.sh
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
Now, you have finished the deployment of the OpenVINO Runtime components to the target system.
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Loading…
Reference in New Issue
Block a user