IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update Model Optimizer User Guide (#10759) (#10934)

Browse Source 
* Remove install prerequisites steps, order FWs, and move pre-processing details

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update Introduction: examples of MO CLIs, references to parameters description pages

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update Setting Input Shape section

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update Optimizing Preprocessing Computation page

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Revert location of Additional_Optimizations.md

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Describe layout and FP16 support in MO

* Fix docs issue

* Apply feedback

* Apply review feedback

* Clean-up Resources

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Mention FP16 compression in MO Introduction

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply the first portion of feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply the second portion of feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply review feedback

* Apply review feedback

* Apply the third portion of feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* Apply feedback for FP16 compression documentation

* Apply review for FP16 page

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* Update docs/MO_DG/prepare_model/Additional_Optimizations.md

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* Apply feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Address feedback about tutorials, input_shape option

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Rework Setting Input Shapes section

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update "See also" list

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Correct conversion documents for each FW

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Refactor TensorFlow converting document and expand Embedding Preprocessing document

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix a link to POT

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>

Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
This commit is contained in:
Maxim Vafin
2022-03-14 19:12:34 +03:00
committed by GitHub
parent af5b31c413
commit bf11b965e6
22 changed files with 347 additions and 1011 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
docs/Extensibility_UG/Intro.md
View File

@@ -8,11 +8,12 @@
openvino_docs_Extensibility_UG_add_openvino_ops
openvino_docs_Extensibility_UG_GPU
openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer
@endsphinxdirective
The Intel® Distribution of OpenVINO™ toolkit supports neural network models trained with multiple frameworks including
TensorFlow, Caffe, MXNet, Kaldi, PaddlePaddle, and ONNX. The list of supported operations (layers) is different for
The Intel® Distribution of OpenVINO™ toolkit supports neural network models trained with various frameworks, including
TensorFlow, PyTorch, ONNX, PaddlePaddle, MXNet, Caffe, and Kaldi. The list of supported operations (layers) is different for
each of the supported frameworks. To see the operations supported by your framework, refer to
[Supported Framework Operations](../MO_DG/prepare_model/Supported_Frameworks_Layers.md).
@@ -27,13 +28,16 @@ the Model Optimizer can generate the IR with the operation.
## OpenVINO™ Extensions
An OpenVINO™ provides extensions for:
OpenVINO™ provides extensions for:
* [Custom OpenVINO™ Operation](add_openvino_ops.md):
- Enables the creation of unsupported operations
- Enables the use of `ov::Core::read_model` to read models with unsupported operations
- Provides a shape inference mechanism for custom operations
- Provides an evaluate method which allow to support the operation on CPU or perform constant folding
- Provides an evaluate method that allows you to support the operation on CPU or perform constant folding
* [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md):
- Enables support of new operations to generate IR
- Enables support of custom transformations to replace sub-graphs for performance optimization
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension/new), which demonstrates extension development details. You can review the complete code, which is fully compilable and up-to-date, to see how it works.

710
docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md
View File

@@ -1,4 +1,4 @@
# Model Optimizer Developer Guide {#openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide}
# Model Optimizer User Guide {#openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide}
@sphinxdirective
@@ -9,6 +9,9 @@
:hidden:
openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model
openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model
openvino_docs_MO_DG_Additional_Optimization_Use_Cases
openvino_docs_MO_DG_FP16_Compression
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch
@@ -16,8 +19,6 @@
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe
openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi
openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model
openvino_docs_MO_DG_Additional_Optimization_Use_Cases
openvino_docs_MO_DG_prepare_model_convert_model_tutorials
openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ
@@ -25,11 +26,11 @@
## Introduction
Model Optimizer is a cross-platform command-line tool that facilitates the transition between the training and deployment environment, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
Model Optimizer is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
Model Optimizer process assumes you have a network model trained using supported deep learning frameworks: Caffe*, TensorFlow*, Kaldi*, MXNet* or converted to the ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the network, which can be inferred with the [OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
Using Model Optimizer tool assumes you already have a deep learning model trained using one of the supported frameworks: TensorFlow, PyTorch, PaddlePaddle, MXNet, Caffe, Kaldi, or represented in ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the model, which can be inferred with [OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
> **NOTE**: Model Optimizer does not infer models. Model Optimizer is an offline tool that runs before the inference takes place.
> **NOTE**: Model Optimizer does not infer models. Model Optimizer is an offline tool that converts a model into IR and optimizes before the inference takes place.
The scheme below illustrates the typical workflow for deploying a trained deep learning model:
@@ -41,656 +42,93 @@ The IR is a pair of files describing the model:
* <code>.bin</code> - Contains the weights and biases binary data.
> **NOTE**: The generated IR can be additionally optimized for inference by [Post-training Optimization tool](../../tools/pot/README.md)
> that applies post-training quantization methods.
> **TIP**: You also can work with the Model Optimizer inside the OpenVINO™ [Deep Learning Workbench](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html) (DL Workbench).
> [DL Workbench](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html) is a web-based graphical environment that enables you to optimize, fine-tune, analyze, visualize, and compare performance of deep learning models.
## Install Model Optimizer Pre-Requisites
Before running the Model Optimizer, you must install the Model Optimizer pre-requisites for the framework that was used to train the model.
@sphinxdirective
.. tab:: Using configuration scripts
.. tab:: Linux
.. tab:: All frameworks
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
./install_prerequisites.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
./install_prerequisites.shs
.. tab:: Caffe
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisitess
install_prerequisites_caffe.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_caffe.shs
.. tab:: Tensorflow 1.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_tf.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_tf.sh
.. tab:: Tensorflow 2.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_tf2.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_tf2.sh
.. tab:: MXNet
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_mxnet.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_mxnet.sh
.. tab:: ONNX
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_onnx.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_onnx.sh
.. tab:: Kaldi
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_kaldi.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_kaldi.sh
.. tab:: Windows
.. tab:: All frameworks
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites.bat
.. tab:: Caffe
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_caffe.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_caffe.bat
.. tab:: Tensorflow 1.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_tf.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_tf.bat
.. tab:: Tensorflow 2.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_tf2.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_tf2.bat
.. tab:: MXNet
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_mxnet.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_mxnet.bat
.. tab:: ONNX
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_onnx.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_onnx.bat
.. tab:: Kaldi
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites\
install_prerequisites_kaldi.bat
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer\install_prerequisites
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
install_prerequisites_kaldi.bat
.. tab:: macOS
.. tab:: All frameworks
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
./install_prerequisites.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
./install_prerequisites.shs
.. tab:: Caffe
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisitess
install_prerequisites_caffe.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_caffe.shs
.. tab:: Tensorflow 1.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_tf.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_tf.sh
.. tab:: Tensorflow 2.x
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_tf2.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_tf2.sh
.. tab:: MXNet
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_mxnet.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_mxnet.sh
.. tab:: ONNX
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_onnx.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_onnx.sh
.. tab:: Kaldi
.. tab:: Install globally
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
install_prerequisites_kaldi.sh
.. tab:: Install to virtualenv
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/install_prerequisites
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate # sh, bash, ksh, or zsh
install_prerequisites_kaldi.sh
.. tab:: Using manual configuration process
.. tab:: Linux
.. tab:: All frameworks
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements.txt
.. tab:: Caffe
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_caffe.txt
.. tab:: Tensorflow 1.x
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_tf.txt
.. tab:: Tensorflow 2.x
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_tf2.txt
.. tab:: MXNet
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_mxnet.txt
.. tab:: ONNX
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_onnx.txt
.. tab:: Kaldi
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_kaldi.txt
.. tab:: Windows
.. tab:: All frameworks
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements.txt
.. tab:: Caffe
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_caffe.txt
.. tab:: Tensorflow 1.x
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_tf.txt
.. tab:: Tensorflow 2.x
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_tf2.txt
.. tab:: MXNet
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_mxnet.txt
.. tab:: ONNX
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_onnx.txt
.. tab:: Kaldi
.. code-block:: sh
cd <INSTALL_DIR>\deployment_tools\model_optimizer
virtualenv --system-site-packages -p python .\env
env\Scripts\activate.bat
pip install -r requirements_kaldi.txt
.. tab:: macOS
.. tab:: All frameworks
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements.txt
.. tab:: Caffe
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_caffe.txt
.. tab:: Tensorflow 1.x
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_tf.txt
.. tab:: Tensorflow 2.x
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_tf2.txt
.. tab:: MXNet
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_mxnet.txt
.. tab:: ONNX
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_onnx.txt
.. tab:: Kaldi
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
virtualenv --system-site-packages -p python3 ./venv
source ./venv/bin/activate
pip3 install -r requirements_kaldi.txt
@endsphinxdirective
## Run Model Optimizer
To convert the model to the Intermediate Representation (IR), run Model Optimizer:
To convert the model to IR, run Model Optimizer:
```sh
mo --input_model INPUT_MODEL
```
> **NOTE**: Some models require using additional arguments, such as `--input`, `--input_shape`, and `--output`. To learn about when you need to use these parameters, refer to [General Conversion Parameters](prepare_model/convert_model/Converting_Model.md).
If out-of-the-box conversion (only the `--input_model` parameter is specified) is not succeed,
try to use parameters for overriding input shapes and cutting the model, mentioned below.
To adjust the conversion process, you may use [general parameters](prepare_model/convert_model/Converting_Model.md) and framework-specific parameters for:
* [TensorFlow](prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
* [ONNX](prepare_model/convert_model/Convert_Model_From_ONNX.md)
* [PyTorch](prepare_model/convert_model/Convert_Model_From_PyTorch.md)
* [PaddlePaddle](prepare_model/convert_model/Convert_Model_From_Paddle.md)
* [MXNet](prepare_model/convert_model/Convert_Model_From_MxNet.md)
* [Caffe](prepare_model/convert_model/Convert_Model_From_Caffe.md)
* [Kaldi](prepare_model/convert_model/Convert_Model_From_Kaldi.md)
To override original input shapes for model conversion, Model Optimizer provides two parameters: `--input` and `--input_shape`.
For more information about these parameters, refer to [Setting Input Shapes](prepare_model/convert_model/Converting_Model.md).
To cut off unwanted parts of a model, such as unsupported operations and training sub-graphs,
the `--input` and `--output` parameters can be used, defining new inputs and outputs of the converted model.
For a more detailed description, refer to [Cutting Off Parts of a Model](prepare_model/convert_model/Cutting_Model.md).
Also, you can insert additional input pre-processing sub-graphs into the converted model using
the `--mean_values`, `scales_values`, `--layout`, and other parameters described
in [Embedding Preprocessing Computation](prepare_model/Additional_Optimizations.md).
Model Optimizer's compression parameter `--data_type` allows to generate IR of the `FP16` data type. For more details,
please refer to [Compression of a Model to FP16](prepare_model/FP16_Compression.md).
To get the full list of conversion parameters available in Model Optimizer, run the following command:
```sh
mo --help
```
## Examples of CLI Commands
Launch the Model Optimizer for the Caffe bvlc_alexnet model:
Below is a list of separate examples for different frameworks and Model Optimizer parameters.
1. Launch Model Optimizer for a TensorFlow MobileNet model in the binary protobuf format.
```sh
mo --input_model bvlc_alexnet.caffemodel
mo --input_model MobileNet.pb
```
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format, with three inputs. Explicitly specify input shapes
where the batch size and the sequence length equal 2 and 30 respectively.
```sh
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
```
For more information on TensorFlow model conversion,
refer to [Converting a TensorFlow Model](prepare_model/convert_model/Convert_Model_From_TensorFlow.md).
## Videos
2. Launch Model Optimizer for an ONNX OCR model and explicitly specify new output.
```sh
mo --input_model ocr.onnx --output probabilities
```
For more information on ONNX model conversion,
please refer to [Converting an ONNX Model](prepare_model/convert_model/Convert_Model_From_ONNX.md).
Note that PyTorch models must be exported to the ONNX format before its conversion into IR.
More details can be found in [Converting a PyTorch Model](prepare_model/convert_model/Convert_Model_From_PyTorch.md).
@sphinxdirective
3. Launch Model Optimizer for a PaddlePaddle UNet model and apply mean-scale normalization to the input.
```sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
```
For more information on PaddlePaddle model conversion, please refer to
[Converting a PaddlePaddle Model](prepare_model/convert_model/Convert_Model_From_Paddle.md).
.. list-table::
4. Launch Model Optimizer for an MXNet SSD Inception V3 model and specify first-channel layout for the input.
```sh
mo --input_model ssd_inception_v3-0000.params --layout NCHW
```
For more information on MXNet models conversion, please refer to [Converting an MXNet Model](prepare_model/convert_model/Convert_Model_From_MxNet.md).
* - .. raw:: html
5. Launch Model Optimizer for a Caffe AlexNet model with input channels in the RGB format, which needs to be reversed.
```sh
mo --input_model alexnet.caffemodel --reverse_input_channels
```
For more information on Caffe model conversion, please refer to [Converting a Caffe Model](prepare_model/convert_model/Convert_Model_From_Caffe.md).
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="220"
src="https://www.youtube.com/embed/Kl1ptVb7aI8">
</iframe>
6. Launch Model Optimizer for a Kaldi LibriSpeech nnet2 model.
```sh
mo --input_model librispeech_nnet2.mdl --input_shape [1,140]
```
For more information on Kaldi model conversion,
refer to [Converting a Kaldi Model](prepare_model/convert_model/Convert_Model_From_Kaldi.md).
- .. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="220"
src="https://www.youtube.com/embed/BBt1rseDcy0">
</iframe>
- .. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="220"
src="https://www.youtube.com/embed/RF8ypHyiKrY">
</iframe>
* - **Model Optimizer Concept.**
- **Model Optimizer Basic Operation.**
- **Choosing the Right Precision.**
* - Duration: 3:56
- Duration: 2:57
- Duration: 4:18
@endsphinxdirective
To get conversion recipes for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models,
refer to [Model Conversion Tutorials](prepare_model/convert_model/Convert_Model_Tutorials.md).

106
docs/MO_DG/prepare_model/Additional_Optimizations.md
View File

@@ -1,35 +1,99 @@
# Optimize Preprocessing Computation {#openvino_docs_MO_DG_Additional_Optimization_Use_Cases}
# Embedding Preprocessing Computation {#openvino_docs_MO_DG_Additional_Optimization_Use_Cases}
Model Optimizer performs preprocessing to a model. It is possible to optimize this step and improve first inference time, to do that, follow the tips bellow:
Input data for inference can be different from the training dataset and requires additional preprocessing before inference.
To accelerate the whole pipeline including preprocessing and inference, Model Optimizer provides special parameters such as `--mean_values`,
`--scale_values`, `--reverse_input_channels`, and `--layout`. Based on these parameters, Model Optimizer generates IR with additionally
inserted sub-graph that performs the defined preprocessing. This preprocessing block can perform mean-scale normalization of input data,
reverting data along channel dimension, and changing the data layout. For more details about these parameters, refer to the paragraphs below.
- **Image mean/scale parameters**<br>
Make sure to use the input image mean/scale parameters (`--scale` and `–mean_values`) with the Model Optimizer when you need pre-processing. It allows the tool to bake the pre-processing into the IR to get accelerated by the OpenVINO Runtime.
## When to Specify Layout
- **RGB vs. BGR inputs**<br>
If, for example, your network assumes the RGB inputs, the Model Optimizer can swap the channels in the first convolution using the `--reverse_input_channels` command line option, so you do not need to convert your inputs to RGB every time you get the BGR image, for example, from OpenCV*.
You may need to set input layouts, as it is required by some preprocessing, for example, setting a batch,
applying mean or scales, and reversing input channels (BGR<->RGB).
- **Larger batch size**<br>
Notice that the devices like GPU are doing better with larger batch size. While it is possible to set the batch size in the runtime using the OpenVINO Runtime API [ShapeInference feature](../../OV_Runtime_UG/ShapeInference.md).
Layout defines the meaning of dimensions in shape and can be specified for both inputs and outputs.
For the layout syntax, check the [Layout API overview](../../OV_Runtime_UG/layout_overview.md).
To specify the layout, you can use `--layout` option followed by the layout value.
- **Resulting IR precision**<br>
The resulting IR precision, for instance, `FP16` or `FP32`, directly affects performance. As CPU now supports `FP16` (while internally upscaling to `FP32` anyway) and because this is the best precision for a GPU target, you may want to always convert models to `FP16`. Notice that this is the only precision that Intel&reg; Movidius&trade; Myriad&trade; 2 and Intel&reg; Myriad&trade; X VPUs support.
For example, for Tensorflow\* `nasnet_large` model that was exported to ONNX format and thus has input with `NHWC` layout:
```
mo --input_model tf_nasnet_large.onnx --layout nhwc
```
Additionally, if a model has more than one input or needs both input and output layouts specified,
you need to provide the name of each input or output to which you apply the layout.
For example, for ONNX\* `Yolo v3 Tiny` model that has first input `input_1` in `NCHW` layout and second input `image_shape`
with 2 dimensions: batch and size of the image which can be expressed as `N?` layout:
```
mo --input_model yolov3-tiny.onnx --layout input_1(nchw),image_shape(n?)
```
## How to Change Layout of a Model Inputs and Outputs
Changing the model layout may be necessary if it differs from the one presented by input data.
To change the layout, you can use either `--layout` or `--source_layout` with `--target_layout`.
For example, for the same `nasnet_large` that were mentioned previously we may want to provide data in `NCHW` layout:
```
mo --input_model tf_nasnet_large.onnx --source_layout nhwc --target_layout nchw
mo --input_model tf_nasnet_large.onnx --layout "nhwc->nchw"
```
Again, if a model has more than one input or needs both input and output layouts specified, you need to provide the name of each input or output to which you apply the layout.
For example, to provide data in the `NHWC` layout for the `Yolo v3 Tiny` model mentioned earlier:
```
mo --input_model yolov3-tiny.onnx --source_layout "input_1(nchw),image_shape(n?)" --target_layout "input_1(nhwc)"
mo --input_model yolov3-tiny.onnx --layout "input_1(nchw->nhwc),image_shape(n?)"
```
## When to Specify Mean and Scale Values
Usually neural network models are trained with the normalized input data. This means that the input data values are converted to be in a specific range, for example, `[0, 1]` or `[-1, 1]`. Sometimes the mean values (mean images) are subtracted from the input data values as part of the pre-processing. There are two cases how the input data pre-processing is implemented.
* The input pre-processing operations are a part of a topology. In this case, the application that uses the framework to infer the topology does not pre-process the input.
* The input pre-processing operations are not a part of a topology and the pre-processing is performed within the application which feeds the model with an input data.
Usually neural network models are trained with the normalized input data. This means that the input data values are converted to be in a specific range,
for example, `[0, 1]` or `[-1, 1]`. Sometimes the mean values (mean images) are subtracted from the input data values as part of the pre-processing.
There are two cases of how the input data pre-processing is implemented.
* The input pre-processing operations are a part of a model. In this case, the application does not pre-process the input data as a separate step: everything is embedded into the model itself.
* The input pre-processing operations are not a part of a model and the pre-processing is performed within the application which feeds the model with input data.
In the first case, the Model Optimizer generates the IR with required pre-processing operations and OpenVINO Samples may be used to infer the model.
In the first case, the Model Optimizer generates the IR with required pre-processing operations and no `mean` and `scale` parameters are required.
In the second case, information about mean/scale values should be provided to the Model Optimizer to embed it to the generated IR. Model Optimizer provides a number of command line parameters to specify them: `--mean`, `--scale`, `--scale_values`, `--mean_values`.
In the second case, information about mean/scale values should be provided to the Model Optimizer to embed it to the generated IR.
Model Optimizer provides command-line parameters to specify the values: `--mean_values`, `--scale_values`, `--scale`.
Using these parameters, Model Optimizer embeds the corresponding preprocessing block for mean-value normalization of the input data
and optimizes this block so that the preprocessing takes negligible time for inference.
> **NOTE:** If both mean and scale values are specified, the mean is subtracted first and then scale is applied regardless of the order of options in command line. Input values are *divided* by the scale value(s). If also `--reverse_input_channels` option is used, the reverse_input_channels will be applied first, then mean and after that scale.
For example, run the Model Optimizer for the PaddlePaddle* UNet model and apply mean-scale normalization to the input data.
There is no a universal recipe for determining the mean/scale values for a particular model. The steps below could help to determine them:
* Read the model documentation. Usually the documentation describes mean/scale value if the pre-processing is required.
* Open the example script/application executing the model and track how the input data is read and passed to the framework.
* Open the model in a visualization tool and check for layers performing subtraction or multiplication (like `Sub`, `Mul`, `ScaleShift`, `Eltwise` etc) of the input data. If such layers exist, pre-processing is probably part of the model.
```sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
```
## When to Reverse Input Channels <a name="when_to_reverse_input_channels"></a>
Input data for your application can be of RGB or BRG color input order. For example, OpenVINO Samples load input images in the BGR channels order. However, the model may be trained on images loaded with the opposite order (for example, most TensorFlow\* models are trained with images in RGB order). In this case, inference results using the OpenVINO samples may be incorrect. The solution is to provide `--reverse_input_channels` command line parameter. Taking this parameter, the Model Optimizer performs first convolution or other channel dependent operation weights modification so these operations output will be like the image is passed with RGB channels order.
Sometimes input images for your application can be of the RGB (BGR) format and the model is trained on images of the BGR (RGB) format,
the opposite color channel order. In this case, it is important to preprocess the input images by reverting the color channels before inference.
To embed this preprocessing step into IR, Model Optimizer provides the `--reverse_input_channels` command-line parameter to shuffle the color channels.
The `--reverse_input_channels` parameter applies to an input of the model in two cases.
* Only one dimension in the input shape has a size equal to 3.
* One dimension has an undefined size and is marked as `C` channel using `layout` parameters.
Using the `--reverse_input_channels` parameter, Model Optimizer embeds the corresponding preprocessing block for reverting
the input data along channel dimension and optimizes this block so that the preprocessing takes negligible time for inference.
For example, launch the Model Optimizer for the TensorFlow* AlexNet model and embed `reverse_input_channel` preprocessing block into IR.
```sh
mo --input_model alexnet.pb --reverse_input_channels
```
> **NOTE**: If both mean and scale values are specified, the mean is subtracted first and then the scale is applied regardless of the order of options
in the command line. Input values are *divided* by the scale value(s). If also `--reverse_input_channels` option is used, the `reverse_input_channels`
will be applied first, then `mean` and after that `scale`. The data flow in the model looks as follows:
`Parameter -> ReverseInputChannels -> Mean apply-> Scale apply -> the original body of the model`.
## See Also
* [Overview of Preprocessing API](../../OV_Runtime_UG/preprocessing_overview.md)

20
docs/MO_DG/prepare_model/FP16_Compression.md Normal file
View File

@@ -0,0 +1,20 @@
# Compression of a Model to FP16 {#openvino_docs_MO_DG_FP16_Compression}
Model Optimizer can convert all floating-point weights to `FP16` data type. The resulting IR is called
compressed `FP16` model.
To compress the model, use the `--data_type` option:
```
mo --input_model INPUT_MODEL --data_type FP16
```
> **NOTE**: Using `--data_type FP32` will give no result and will not force `FP32`
> precision in the model. If the model was `FP16` it will have `FP16` precision in IR as well.
The resulting model will occupy about twice as less space in the file system, but it may have some accuracy drop,
although for the majority of models accuracy degradation is negligible. For details on how plugins handle
compressed `FP16` models refer to [Working with devices](../../OV_Runtime_UG/supported_plugins/Device_Plugins.md) page.
> **NOTE**: `FP16` compression is sometimes used as initial step for `INT8` quantization, please refer to
> [Post-Training Optimization tool](../../../tools/pot/README.md) for more information about that.

2
docs/MO_DG/prepare_model/Getting_performance_numbers.md
View File

@@ -9,7 +9,7 @@ When evaluating performance of your model with the OpenVINO Runtime, you must me
- Track separately the operations that happen outside the OpenVINO Runtime, like video decoding.
> **NOTE**: Some image pre-processing can be baked into the IR and accelerated. For more information, refer to [Model Optimizer Knobs Related to Performance](Additional_Optimizations.md)
> **NOTE**: Some image pre-processing can be baked into the IR and accelerated. For more information, refer to [Embedding Preprocessing Computation](Additional_Optimizations.md)
## Tip 2. Getting Credible Performance Numbers

65
docs/MO_DG/prepare_model/Model_Optimization_Techniques.md
View File

@@ -1,65 +0,0 @@
# Model Optimization Techniques {#openvino_docs_MO_DG_prepare_model_Model_Optimization_Techniques}
Optimization offers methods to accelerate inference with the convolution neural networks (CNN) that do not require model retraining.
* * *
## Linear Operations Fusing
Many convolution neural networks includes `BatchNormalization` and `ScaleShift` layers (for example, Resnet\*, Inception\*) that can be presented as a sequence of linear operations: additions and multiplications. For example ScaleShift layer can be presented as Mul → Add sequence. These layers can be fused into previous `Convolution` or `FullyConnected` layers, except when Convolution comes after an Add operation (due to Convolution paddings).
### Usage
In the Model Optimizer, this optimization is turned on by default. To disable it, you can pass `--disable_fusing` parameter to the Model Optimizer.
### Optimization Description
This optimization method consists of three stages:
1. <strong>`BatchNormalization` and `ScaleShift` decomposition</strong>: in this stage, `BatchNormalization` layer is decomposed to `Mul → Add → Mul → Add` sequence, and `ScaleShift` layer is decomposed to `Mul → Add` layers sequence.
2. **Linear operations merge**: in this stage, the `Mul` and `Add` operations are merged into a single `Mul → Add` instance.
For example, if there is a `BatchNormalization → ScaleShift` sequence in the topology, it is replaced with `Mul → Add` in the first stage. In the next stage, the latter is replaced with a `ScaleShift` layer if there is no available `Convolution` or `FullyConnected` layer to fuse into next.
3. **Linear operations fusion**: in this stage, the tool fuses `Mul` and `Add` operations to `Convolution` or `FullyConnected` layers. Notice that it searches for `Convolution` and `FullyConnected` layers both backward and forward in the graph (except for `Add` operation that cannot be fused to `Convolution` layer in forward direction).
### Usage Examples
The picture below shows the depicted part of Caffe\* Resnet269 topology where `BatchNorm` and `ScaleShift` layers will be fused to `Convolution` layers.
![Caffe ResNet269 block before and after optimization generated with Netscope*](../img/optimizations/resnet_269.png)
* * *
## ResNet optimization (stride optimization)
ResNet optimization is a specific optimization that applies to Caffe ResNet topologies such as ResNet50, ResNet101, ResNet152 and to ResNet-based topologies. This optimization is turned on by default, and can be disabled with the `--disable_resnet_optimization` key.
### Optimization Description
In the picture below, you can see the original and optimized parts of a Caffe ResNet50 model. The main idea of this optimization is to move the stride that is greater than 1 from Convolution layers with the kernel size = 1 to upper Convolution layers. In addition, the Model Optimizer adds a Pooling layer to align the input shape for a Eltwise layer, if it was changed during the optimization.
![ResNet50 blocks (original and optimized) from Netscope*](../img/optimizations/resnet_optimization.png)
In this example, the stride from the `res3a_branch1` and `res3a_branch2a` Convolution layers moves to the `res2c_branch2b` Convolution layer. In addition, to align the input shape for `res2c` Eltwise, the optimization inserts the Pooling layer with kernel size = 1 and stride = 2.
* * *
## Grouped Convolution Fusing
Grouped convolution fusing is a specific optimization that applies for TensorFlow\* topologies. The main idea of this optimization is to combine convolutions results for the `Split` outputs and then recombine them using `Concat` operation in the same order as they were out from `Split`.
![Split→Convolutions→Concat block from TensorBoard*](../img/optimizations/groups.png)
* * *
## Disable Fusing
Model Optimizer allows to disable optimizations for specified nodes via `--finegrain_fusing <node_name1>,<node_name2>,...` (regex is also supported). Using this key, you mark nodes that will noy be touched by any optimizations.
### Examples of usage
On the picture below you can see two visualized Intermediate Representations (IR) of TensorFlow InceptionV4 topology.
The first one is original IR that will be produced by the Model Optimizer.
The second one will be produced by the Model Optimizer with key `--finegrain_fusing InceptionV4/InceptionV4/Conv2d_1a_3x3/Conv2D`, where you can see that `Convolution` was not fused with `Mul1_3752` and `Mul1_4061/Fused_Mul_5096/FusedScaleShift_5987` operations.
![TF InceptionV4 block without/with key --finegrain_fusing (from IR visualizer)](../img/optimizations/inception_v4.png)

4
docs/MO_DG/prepare_model/Model_Optimizer_FAQ.md
View File

@@ -589,14 +589,14 @@ For all frameworks:
2. [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md)
or
* Edit network in original framework to exclude cycle.
* Edit model in original framework to exclude cycle.
#### 98. What does the message "Can not transpose attribute '..' with value .. for node '..' .." mean? <a name="question-98"></a>
This message means that model is not supported. It may be caused by using shapes larger than 4-D.
There are two ways to avoid such message:
1. [Cut model part containing such layers in Model Optimizer](convert_model/Cutting_Model.md)
1. [Cutting Off Parts of a Model](convert_model/Cutting_Model.md)
2. Edit network in original framework to exclude such layers.
#### 99. What does the message "Expected token `</ParallelComponent>`, has `...`" mean? <a name="question-99"></a>

31
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md
View File

@@ -1,27 +1,12 @@
# Converting a Caffe* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe}
A summary of the steps for optimizing and deploying a model that was trained with Caffe\*:
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Caffe\*.
2. [Convert a Caffe\* Model](#Convert_From_Caffe) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [OpenVINO samples](../../../OV_Runtime_UG/Samples_Overview.md)
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in your application to deploy the model in the target environment
## Convert a Caffe* Model <a name="Convert_From_Caffe"></a>
To convert a Caffe\* model, run Model Optimizer with the path to the input model `.caffemodel` file and the path to an output directory with write permissions:
To convert a Caffe\* model, run Model Optimizer with the path to the input model `.caffemodel` file:
```sh
mo --input_model <INPUT_MODEL>.caffemodel --output_dir <OUTPUT_MODEL_DIR>
mo --input_model <INPUT_MODEL>.caffemodel
```
Two groups of parameters are available to convert your model:
* Framework-agnostic parameters are used to convert a model trained with any supported framework. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
* [Caffe-specific parameters](#caffe_specific_conversion_params) are used to convert only Caffe\* models.
### Using Caffe\*-Specific Conversion Parameters <a name="caffe_specific_conversion_params"></a>
The following list provides the Caffe\*-specific parameters.
```
@@ -60,16 +45,16 @@ Caffe*-specific parameters:
attributes without flattening nested parameters.
```
#### Command-Line Interface (CLI) Examples Using Caffe\*-Specific Parameters
### Command-Line Interface (CLI) Examples Using Caffe\*-Specific Parameters
* Launching the Model Optimizer for the [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `prototxt` file. This is needed when the name of the Caffe\* model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file. You must have write permissions for the output directory.
* Launching the Model Optimizer for the [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `prototxt` file. This is needed when the name of the Caffe\* model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file.
```sh
mo --input_model bvlc_alexnet.caffemodel --input_proto bvlc_alexnet.prototxt --output_dir <OUTPUT_MODEL_DIR>
mo --input_model bvlc_alexnet.caffemodel --input_proto bvlc_alexnet.prototxt
```
* Launching the Model Optimizer for the [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `CustomLayersMapping` file. This is the legacy method of quickly enabling model conversion if your model has custom layers. This requires the Caffe\* system on the computer.
Optional parameters without default values and not specified by the user in the `.prototxt` file are removed from the Intermediate Representation, and nested parameters are flattened:
```sh
mo --input_model bvlc_alexnet.caffemodel -k CustomLayersMapping.xml --disable_omitting_optional --enable_flattening_nested_params --output_dir <OUTPUT_MODEL_DIR>
mo --input_model bvlc_alexnet.caffemodel -k CustomLayersMapping.xml --disable_omitting_optional --enable_flattening_nested_params
```
This example shows a multi-input model with input layers: `data`, `rois`
```
@@ -91,9 +76,9 @@ Optional parameters without default values and not specified by the user in the
}
```
* Launching the Model Optimizer for a multi-input model with two inputs and providing a new shape for each input in the order they are passed to the Model Optimizer along with a writable output directory. In particular, for data, set the shape to `1,3,227,227`. For rois, set the shape to `1,6,1,1`:
* Launching the Model Optimizer for a multi-input model with two inputs and providing a new shape for each input in the order they are passed to the Model Optimizer. In particular, for data, set the shape to `1,3,227,227`. For rois, set the shape to `1,6,1,1`:
```sh
mo --input_model /path-to/your-model.caffemodel --input data,rois --input_shape (1,3,227,227),[1,6,1,1] --output_dir <OUTPUT_MODEL_DIR>
mo --input_model /path-to/your-model.caffemodel --input data,rois --input_shape (1,3,227,227),[1,6,1,1]
```
## Custom Layer Definition

28
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md
View File

@@ -1,26 +1,14 @@
# Converting a Kaldi* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi}
A summary of the steps for optimizing and deploying a model that was trained with Kaldi\*:
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Kaldi\*.
2. [Convert a Kaldi\* Model](#Convert_From_Kaldi) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [OpenVINO Samples](../../../OV_Runtime_UG/Samples_Overview.md).
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in your application to deploy the model in the target environment.
> **NOTE**: The Model Optimizer supports the [nnet1](http://kaldi-asr.org/doc/dnn1.html) and [nnet2](http://kaldi-asr.org/doc/dnn2.html) formats of Kaldi models. Support of the [nnet3](http://kaldi-asr.org/doc/dnn3.html) format is limited.
## Convert a Kaldi* Model <a name="Convert_From_Kaldi"></a>
To convert a Kaldi\* model, run Model Optimizer with the path to the input model `.nnet` or `.mdl` file:
To convert a Kaldi\* model, run Model Optimizer with the path to the input model `.nnet` or `.mdl` file and to an output directory where you have write permissions:
```sh
mo --input_model <INPUT_MODEL>.nnet --output_dir <OUTPUT_MODEL_DIR>
mo --input_model <INPUT_MODEL>.nnet
```
Two groups of parameters are available to convert your model:
* Framework-agnostic parameters are used to convert a model trained with any supported framework. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
* [Kaldi-specific parameters](#kaldi_specific_conversion_params) are used to convert only Kaldi\* models.
### Using Kaldi\*-Specific Conversion Parameters <a name="kaldi_specific_conversion_params"></a>
The following list provides the Kaldi\*-specific parameters.
@@ -35,14 +23,14 @@ Kaldi-specific parameters:
### Examples of CLI Commands
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with the specified `.nnet` file and an output directory where you have write permissions:
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with the specified `.nnet` file:
```sh
mo --input_model wsj_dnn5b_smbr.nnet --output_dir <OUTPUT_MODEL_DIR>
mo --input_model wsj_dnn5b_smbr.nnet
```
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with existing file that contains counts for the last layer with biases and a writable output directory:
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with existing file that contains counts for the last layer with biases:
```sh
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --output_dir <OUTPUT_MODEL_DIR>
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts
```
* The Model Optimizer normalizes сounts in the following way:
@@ -56,12 +44,12 @@ Kaldi-specific parameters:
\f$|C|\f$ - number of elements in the counts array;
* The normalized counts are subtracted from biases of the last or next to last layer (if last layer is SoftMax).
> **NOTE:** Model Optimizer will show warning if model contains counts values inside model and `--counts` option is not used.
> **NOTE**: Model Optimizer will show warning if model contains counts values inside model and `--counts` option is not used.
* If you want to remove the last SoftMax layer in the topology, launch the Model Optimizer with the
`--remove_output_softmax` flag:
```sh
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax --output_dir <OUTPUT_MODEL_DIR>_
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax
```
The Model Optimizer finds the last layer of the topology and removes this layer only if it is a SoftMax layer.

20
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md
View File

@@ -1,26 +1,12 @@
# Converting an MXNet* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet}
A summary of the steps for optimizing and deploying a model that was trained with the MXNet\* framework:
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for MXNet* (MXNet was used to train your model)
2. [Convert a MXNet model](#ConvertMxNet) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [OpenVINO Samples](../../../OV_Runtime_UG/Samples_Overview.md)
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in your application to deploy the model in the target environment
## Convert an MXNet* Model <a name="ConvertMxNet"></a>
To convert an MXNet\* model, run Model Optimizer with a path to the input model `.params` file and to an output directory where you have write permissions:
To convert an MXNet\* model, run Model Optimizer with a path to the input model `.params` file:
```sh
mo --input_model model-file-0000.params --output_dir <OUTPUT_MODEL_DIR>
mo --input_model model-file-0000.params
```
Two groups of parameters are available to convert your model:
* Framework-agnostic parameters are used to convert a model trained with any supported framework. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
* [MXNet-specific parameters](#mxnet_specific_conversion_params) are used to convert only MXNet models.
### Using MXNet\*-Specific Conversion Parameters <a name="mxnet_specific_conversion_params"></a>
The following list provides the MXNet\*-specific parameters.
@@ -54,7 +40,7 @@ MXNet-specific parameters:
Internally, when you run the Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in this list of known layers, the Model Optimizer classifies them as custom.
## Supported MXNet\* Layers
Refer to [Supported Framework Layers ](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
Refer to [Supported Framework Layers](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Frequently Asked Questions (FAQ)

7
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md
View File

@@ -1,16 +1,17 @@
# Converting a ONNX* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX}
## Introduction to ONNX
[ONNX*](https://github.com/onnx/onnx) is a representation format for deep learning models. ONNX allows AI developers easily transfer models between different frameworks that helps to choose the best combination for them. Today, PyTorch\*, Caffe2\*, Apache MXNet\*, Microsoft Cognitive Toolkit\* and other tools are developing ONNX support.
## Convert an ONNX* Model <a name="Convert_From_ONNX"></a>
The Model Optimizer process assumes you have an ONNX model that was directly downloaded from a public repository or converted from any framework that supports exporting to the ONNX format.
To convert an ONNX\* model, run Model Optimizer with the path to the input model `.nnet` file and an output directory where you have write permissions:
To convert an ONNX\* model, run Model Optimizer with the path to the input model `.onnx` file:
```sh
mo --input_model <INPUT_MODEL>.onnx --output_dir <OUTPUT_MODEL_DIR>
mo --input_model <INPUT_MODEL>.onnx
```
There are no ONNX\* specific parameters, so only framework-agnostic parameters are available to convert your model. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
## Supported ONNX\* Layers

24
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Paddle.md
View File

@@ -1,31 +1,17 @@
# Converting a PaddlePaddle Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Paddle}
A summary of the steps for optimizing and deploying a model trained with PaddlePaddle:
1. [Configure Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for PaddlePaddle.
2. [Convert a PaddlePaddle Model](#Convert_From_Paddle) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases.
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [OpenVINO Samples](../../../OV_Runtime_UG/Samples_Overview.md).
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in your application to deploy the model in the target environment.
# Converting a PaddlePaddle* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Paddle}
## Convert a PaddlePaddle Model <a name="Convert_From_Paddle"></a>
To convert a PaddlePaddle model, use the `mo` script and specify the path to the input model `.pdmodel` file:
To convert a PaddlePaddle model:
1. Activate environment with installed OpenVINO™ if needed
2. Use the `mo` script to simply convert a model, specifying the framework, the path to the input model `.pdmodel` file and the path to an output directory with write permissions:
```sh
mo --input_model <INPUT_MODEL>.pdmodel --output_dir <OUTPUT_MODEL_DIR> --framework=paddle
mo --input_model <INPUT_MODEL>.pdmodel
```
Parameters to convert your model:
* [Framework-agnostic parameters](Converting_Model.md): These parameters are used to convert a model trained with any supported framework.
> **NOTE:** `--scale`, `--scale_values`, `--mean_values` are not supported in the current version of mo_paddle.
### Example of Converting a PaddlePaddle Model
Below is the example command to convert yolo v3 PaddlePaddle network to OpenVINO IR network with Model Optimizer.
```sh
mo --model_name yolov3_darknet53_270e_coco --output_dir <OUTPUT_MODEL_DIR> --framework=paddle --data_type=FP32 --reverse_input_channels --input_shape=[1,3,608,608],[1,2],[1,2] --input=image,im_shape,scale_factor --output=save_infer_model/scale_0.tmp_1,save_infer_model/scale_1.tmp_1 --input_model=yolov3.pdmodel
mo --input_model=yolov3.pdmodel --input=image,im_shape,scale_factor --input_shape=[1,3,608,608],[1,2],[1,2] --reverse_input_channels --output=save_infer_model/scale_0.tmp_1,save_infer_model/scale_1.tmp_1
```
## Supported PaddlePaddle Layers

12
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_PyTorch.md
View File

@@ -1,23 +1,19 @@
# Converting a PyTorch* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch}
## Typical Steps to Convert PyTorch Model <a name="typical-pytorch"></a>
PyTorch* framework is supported through export to ONNX\* format. A summary of the steps for optimizing and deploying a model that was trained with the PyTorch\* framework:
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for ONNX\*.
2. [Export PyTorch model to ONNX\*](#export-to-onnx).
3. [Convert an ONNX\* model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
4. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
5. [Integrate OpenVINO Runtime](../../../OV_Runtime_UG/integrate_with_your_application.md) in your application to deploy the model in the target environment.
1. [Export PyTorch model to ONNX\*](#export-to-onnx).
2. [Convert an ONNX\* model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
## Export PyTorch\* Model to ONNX\* Format <a name="export-to-onnx"></a>
PyTorch models are defined in a Python\* code, to export such models use `torch.onnx.export()` method. Usually code to
evaluate or test the model is provided with the model code and can be used to initialize and export model.
Only the basics will be covered here, the step to export to ONNX\* is crucial but it is covered by PyTorch\* framework.
For more information, please refer to [PyTorch\* documentation](https://pytorch.org/docs/stable/onnx.html).
For more information, please refer to [Exporting PyTorch models to ONNX format](https://pytorch.org/docs/stable/onnx.html).
To export a PyTorch\* model you need to obtain the model as an instance of `torch.nn.Module` class and call the `export` function.
```python
import torch

208
docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md
View File

@@ -1,65 +1,40 @@
# Converting a TensorFlow* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow}
A summary of the steps for optimizing and deploying a model that was trained with the TensorFlow\* framework:
## Convert TensorFlow 1 Models <a name="Convert_From_TF2X"></a>
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for TensorFlow\* (TensorFlow was used to train your model).
2. [Freeze the TensorFlow model](#freeze-the-tensorflow-model) if your model is not already frozen or skip this step and use the [instruction](#loading-nonfrozen-models) to a convert a non-frozen model.
3. [Convert a TensorFlow\* model](#Convert_From_TF) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
4. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/openvino_intro.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
5. [Integrate OpenVINO Runtime](../../../OV_Runtime_UG/integrate_with_your_application.md) in your application to deploy the model in the target environment.
### Convert Frozen Model Format <a name="Convert_From_TF"></a>
To convert a TensorFlow model, use the `mo` script to simply convert a model with the path to the input model `.pb` file:
## Loading Non-Frozen Models to the Model Optimizer <a name="loading-nonfrozen-models"></a>
There are three ways to store non-frozen TensorFlow models and load them to the Model Optimizer:
1. Checkpoint:
In this case, a model consists of two files:
- `inference_graph.pb` or `inference_graph.pbtxt`
- `checkpoint_file.ckpt`
If you do not have an inference graph file, refer to [Freezing Custom Models in Python](#freeze-the-tensorflow-model).
To convert such a TensorFlow model:
1. Go to the `<INSTALL_DIR>/tools/model_optimizer` directory
2. Run the `mo` script with the path to the checkpoint file to convert a model and an output directory where you have write permissions:
* If input model is in `.pb` format:<br>
```sh
mo --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT> --output_dir <OUTPUT_MODEL_DIR>
```
* If input model is in `.pbtxt` format:<br>
```sh
mo --input_model <INFERENCE_GRAPH>.pbtxt --input_checkpoint <INPUT_CHECKPOINT> --input_model_is_text --output_dir <OUTPUT_MODEL_DIR>
mo --input_model <INPUT_MODEL>.pb
```
2. MetaGraph:
### Convert Non-Frozen Model Formats <a name="loading-nonfrozen-models"></a>
There are three ways to store non-frozen TensorFlow models and convert them by Model Optimizer:
In this case, a model consists of three or four files stored in the same directory:
- `model_name.meta`
- `model_name.index`
- `model_name.data-00000-of-00001` (digit part may vary)
- `checkpoint` (optional)
To convert such TensorFlow model:
1. Go to the `<INSTALL_DIR>/tools/model_optimizer` directory
2. Run the `mo` script with a path to the MetaGraph `.meta` file and a writable output directory to convert a model:<br>
1. **Checkpoint**. In this case, a model consists of two files: `inference_graph.pb` (or `inference_graph.pbtxt`) and `checkpoint_file.ckpt`.
If you do not have an inference graph file, refer to [Freezing Custom Models in Python](#freeze-the-tensorflow-model).
To convert the model with the inference graph in `.pb` format, run the `mo` script with the path to the checkpoint file to convert a model:
```sh
mo --input_meta_graph <INPUT_META_GRAPH>.meta --output_dir <OUTPUT_MODEL_DIR>
mo --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT>
```
To convert the model with the inference graph in `.pbtxt` format, run the `mo` script with the path to the checkpoint file to convert a model:
```sh
mo --input_model <INFERENCE_GRAPH>.pbtxt --input_checkpoint <INPUT_CHECKPOINT> --input_model_is_text
```
3. SavedModel format of TensorFlow 1.x and 2.x versions:
In this case, a model consists of a special directory with a `.pb` file and several subfolders: `variables`, `assets`, and `assets.extra`. For more information about the SavedModel directory, refer to the [README](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/saved_model#components) file in the TensorFlow repository.
To convert such TensorFlow model:
1. Go to the `<INSTALL_DIR>/tools/model_optimizer` directory
2. Run the `mo` script with a path to the SavedModel directory and a writable output directory to convert a model:<br>
2. **MetaGraph**. In this case, a model consists of three or four files stored in the same directory: `model_name.meta`, `model_name.index`,
`model_name.data-00000-of-00001` (digit part may vary), and `checkpoint` (optional).
To convert such TensorFlow model, run the `mo` script with a path to the MetaGraph `.meta` file:
```sh
mo --saved_model_dir <SAVED_MODEL_DIRECTORY> --output_dir <OUTPUT_MODEL_DIR>
mo --input_meta_graph <INPUT_META_GRAPH>.meta
```
3. **SavedModel format**. In this case, a model consists of a special directory with a `.pb` file
and several subfolders: `variables`, `assets`, and `assets.extra`. For more information about the SavedModel directory, refer to the [README](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/python/saved_model#components) file in the TensorFlow repository.
To convert such TensorFlow model, run the `mo` script with a path to the SavedModel directory:
```sh
mo --saved_model_dir <SAVED_MODEL_DIRECTORY>
```
You can convert TensorFlow 1.x SavedModel format in the environment that has a 1.x or 2.x version of TensorFlow. However, TensorFlow 2.x SavedModel format strictly requires the 2.x version of TensorFlow.
@@ -67,12 +42,12 @@ If a model contains operations currently unsupported by OpenVINO, prune these op
To determine custom input nodes, display a graph of the model in TensorBoard. To generate TensorBoard logs of the graph, use the `--tensorboard_logs` option.
TensorFlow 2.x SavedModel format has a specific graph due to eager execution. In case of pruning, find custom input nodes in the `StatefulPartitionedCall/*` subgraph of TensorFlow 2.x SavedModel format.
## Freezing Custom Models in Python\* <a name="freeze-the-tensorflow-model"></a>
### Freezing Custom Models in Python\* <a name="freeze-the-tensorflow-model"></a>
When a network is defined in Python\* code, you have to create an inference graph file. Usually graphs are built in a form
that allows model training. That means that all trainable parameters are represented as variables in the graph.
To be able to use such graph with Model Optimizer such graph should be frozen.
The graph is frozen and dumped to a file with the following code:
```python
import tensorflow as tf
from tensorflow.python.framework import graph_io
@@ -91,97 +66,17 @@ Where:
* `inference_graph.pb` is the name of the generated inference graph file.
* `as_text` specifies whether the generated file should be in human readable text format or binary.
## Convert a TensorFlow* Model <a name="Convert_From_TF"></a>
To convert a TensorFlow model:
1. Go to the `<INSTALL_DIR>/tools/model_optimizer` directory
2. Use the `mo` script to simply convert a model with the path to the input model `.pb` file and a writable output directory:
```sh
mo --input_model <INPUT_MODEL>.pb --output_dir <OUTPUT_MODEL_DIR>
```
Two groups of parameters are available to convert your model:
* Framework-agnostic parameters are used to convert a model trained with any supported framework. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
* [TensorFlow-specific parameters](#tensorflow_specific_conversion_params): Parameters used to convert only TensorFlow models.
> **NOTE**: The color channel order (RGB or BGR) of an input data should match the channel order of the model training dataset. If they are different, perform the `RGB<->BGR` conversion specifying the command-line parameter: `--reverse_input_channels`. Otherwise, inference results may be incorrect. For more information about the parameter, refer to **When to Reverse Input Channels** section of [Converting a Model to Intermediate Representation (IR)](Converting_Model.md).
### Using TensorFlow\*-Specific Conversion Parameters <a name="tensorflow_specific_conversion_params"></a>
The following list provides the TensorFlow\*-specific parameters.
```
TensorFlow*-specific parameters:
--input_model_is_text
TensorFlow*: treat the input model file as a text
protobuf format. If not specified, the Model Optimizer
treats it as a binary file by default.
--input_checkpoint INPUT_CHECKPOINT
TensorFlow*: variables file to load.
--input_meta_graph INPUT_META_GRAPH
Tensorflow*: a file with a meta-graph of the model
before freezing
--saved_model_dir SAVED_MODEL_DIR
TensorFlow*: directory with a model in SavedModel format
of TensorFlow 1.x or 2.x version
--saved_model_tags SAVED_MODEL_TAGS
Group of tag(s) of the MetaGraphDef to load, in string
format, separated by ','. For tag-set contains
multiple tags, all tags must be passed in.
--tensorflow_custom_operations_config_update TENSORFLOW_CUSTOM_OPERATIONS_CONFIG_UPDATE
TensorFlow*: update the configuration file with node
name patterns with input/output nodes information.
--tensorflow_object_detection_api_pipeline_config TENSORFLOW_OBJECT_DETECTION_API_PIPELINE_CONFIG
TensorFlow*: path to the pipeline configuration file
used to generate model created with help of Object
Detection API.
--tensorboard_logdir TENSORBOARD_LOGDIR
TensorFlow*: dump the input graph to a given directory
that should be used with TensorBoard.
--tensorflow_custom_layer_libraries TENSORFLOW_CUSTOM_LAYER_LIBRARIES
TensorFlow*: comma separated list of shared libraries
with TensorFlow* custom operations implementation.
--disable_nhwc_to_nchw
[DEPRECATED] Disables default translation from NHWC to NCHW. Since 2022.1
this option is deprecated and used only to maintain backward compatibility
with previous releases.
```
> **NOTE**: Models produces with TensorFlow\* usually have not fully defined shapes (contain `-1` in some dimensions). It is necessary to pass explicit shape for the input using command line parameter `--input_shape` or `-b` to override just batch dimension. If the shape is fully defined, then there is no need to specify either `-b` or `--input_shape` options.
#### Command-Line Interface (CLI) Examples Using TensorFlow\*-Specific Parameters
* Launching the Model Optimizer for Inception V1 frozen model when model file is a plain text protobuf, specifying a writable output directory:
```sh
mo --input_model inception_v1.pbtxt --input_model_is_text -b 1 --output_dir <OUTPUT_MODEL_DIR>
```
* Launching the Model Optimizer for Inception V1 frozen model and dump information about the graph to TensorBoard log dir `/tmp/log_dir`
```sh
mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir --output_dir <OUTPUT_MODEL_DIR>
```
* Launching the Model Optimizer for a model with custom TensorFlow operations (refer to the [TensorFlow* documentation](https://www.tensorflow.org/extend/adding_an_op)) implemented in C++ and compiled into the shared library `my_custom_op.so`. Model Optimizer falls back to TensorFlow to infer output shape of operations implemented in the library if a custom TensorFlow operation library is provided. If it is not provided, a custom operation with an inference function is needed. For more information about custom operations, refer to the [OpenVINO™ Extensibility Mechanism](../../../Extensibility_UG/Intro.md).
```sh
mo --input_model custom_model.pb --tensorflow_custom_layer_libraries ./my_custom_op.so --output_dir <OUTPUT_MODEL_DIR>
```
## Convert TensorFlow* 2 Models <a name="Convert_From_TF2X"></a>
In order to convert TensorFlow* 2 models, installation of dependencies from `requirements_tf.txt` is required.
## Convert TensorFlow 2 Models <a name="Convert_From_TF2X"></a>
To convert TensorFlow* 2 models, ensure that `openvino-dev[tensorflow2]` is installed via `pip`.
TensorFlow* 2.X officially supports two model formats: SavedModel and Keras H5 (or HDF5).
Below are the instructions on how to convert each of them.
### SavedModel Format
A model in the SavedModel format consists of a directory with a `saved_model.pb` file and two subfolders: `variables` and `assets`.
To convert such a model:
1. Go to the `<INSTALL_DIR>/tools/model_optimizer` directory.
2. Run the `mo` script with a path to the SavedModel directory and a writable output directory:
To convert such a model, run the `mo` script with a path to the SavedModel directory:
```sh
mo --saved_model_dir <SAVED_MODEL_DIRECTORY> --output_dir <OUTPUT_MODEL_DIR>
mo --saved_model_dir <SAVED_MODEL_DIRECTORY>
```
TensorFlow* 2 SavedModel format strictly requires the 2.x version of TensorFlow installed in the
@@ -198,9 +93,9 @@ TensorFlow* 2 SavedModel format has a specific graph structure due to eager exec
pruning, find custom input nodes in the `StatefulPartitionedCall/*` subgraph.
### Keras H5
If you have a model in the HDF5 format, load the model using TensorFlow* 2 and serialize it in the
SavedModel format. Here is an example of how to do it:
```python
import tensorflow as tf
model = tf.keras.models.load_model('model.h5')
@@ -209,6 +104,7 @@ tf.saved_model.save(model,'model')
The Keras H5 model with a custom layer has specifics to be converted into SavedModel format.
For example, the model with a custom layer `CustomLayer` from `custom_layer.py` is converted as follows:
```python
import tensorflow as tf
from custom_layer import CustomLayer
@@ -220,42 +116,36 @@ Then follow the above instructions for the SavedModel format.
> **NOTE**: Do not use other hacks to resave TensorFlow* 2 models into TensorFlow* 1 formats.
## Command-Line Interface (CLI) Examples Using TensorFlow\*-Specific Parameters
* Launching the Model Optimizer for Inception V1 frozen model when model file is a plain text protobuf:
## Custom Layer Definition
```sh
mo --input_model inception_v1.pbtxt --input_model_is_text -b 1
```
Internally, when you run the Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in this list of known layers, the Model Optimizer classifies them as custom.
* Launching the Model Optimizer for Inception V1 frozen model and dump information about the graph to TensorBoard log dir `/tmp/log_dir`
See [Custom Layers in the Model Optimizer](../customize_model_optimizer/Customize_Model_Optimizer.md) for information about:
```sh
mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir
```
* Model Optimizer internal procedure for working with custom layers
* How to convert a TensorFlow model that has custom layers
* Custom layer implementation details
* Launching the Model Optimizer for BERT model in the SavedModel format, with three inputs. Explicitly specify input shapes
where the batch size and the sequence length equal 2 and 30 respectively.
```sh
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
```
## Supported TensorFlow\* and TensorFlow 2 Keras\* Layers
Refer to [Supported Framework Layers ](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Frequently Asked Questions (FAQ)
The Model Optimizer provides explanatory messages if it is unable to run to completion due to issues like typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md). The FAQ has instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
## Video: Converting a TensorFlow Model
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315"
src="https://www.youtube.com/embed/QW6532LtiTc">
</iframe>
@endsphinxdirective
## Summary
In this document, you learned:
* Basic information about how the Model Optimizer works with TensorFlow\* models
* Basic information about how the Model Optimizer works with TensorFlow models
* Which TensorFlow models are supported
* How to freeze a TensorFlow model
* How to convert a trained TensorFlow model using the Model Optimizer with both framework-agnostic and TensorFlow-specific command-line options

6
docs/MO_DG/prepare_model/convert_model/Convert_Model_Tutorials.md
View File

@@ -36,6 +36,8 @@
@endsphinxdirective
This section provides you with a set of tutorials that demonstrate conversion steps for specific TensorFlow, ONNX, PyTorch, MXNet and Kaldi models.
This section provides you with a set of tutorials that demonstrate conversion steps for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models.
It contains conversion recipes for concrete models, that unnecessarily cover your case.
Try to convert the model out-of-the-box, meaning only the `--input_model` parameter is specified in the command line, before studying the tutorials.
You can also find a collection of [Python tutorials](../../../tutorials.md) written for running on Jupyter* notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference.
You can also find a collection of [Python tutorials](../../../tutorials.md) written for running on Jupyter* notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference.

86
docs/MO_DG/prepare_model/convert_model/Converting_Model.md
View File

@@ -1,32 +1,74 @@
# General Conversion Parameters {#openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model}
# Setting Input Shapes {#openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model}
To get the full list of general (framework-agnostic) conversion parameters available in Model Optimizer, run the following command:
Paragraphs below provide details about specifying input shapes for model conversion.
## When to Specify --input_shape Command-line Parameter <a name="when_to_specify_input_shapes"></a>
Model Optimizer supports conversion of models with input dynamic shapes that contain undefined dimensions.
However, if the shape of inference data is not going to change from one inference request to another,
it is recommended to set up static shapes (when all dimensions are fully defined) for the inputs.
It can be beneficial from a performance perspective and memory consumption.
To set up static shapes, Model Optimizer provides the `--input_shape` parameter. This is an offline approach to set static shapes and
can save time on calling `reshape` method.
For more information about the `reshape` method and dynamic shapes, refer to [Dynamic Shapes](../../../OV_Runtime_UG/ov_dynamic_shapes.md)
OpenVINO Runtime API can have limitations to infer models with undefined dimensions on some hardware.
In this case, the `--input_shape` parameter and the `reshape` method can help resolving undefined dimensions.
Sometimes Model Optimizer is unable to convert models out-of-the-box (only the `--input_model` parameter is specified).
Such problem can relate to models with inputs of undefined ranks and a case of cutting off parts of a model.
In this case, user has to specify input shapes explicitly using `--input_shape` parameter.
For example, run the Model Optimizer for the TensorFlow* MobileNet model with the single input
and specify input shape `[2,300,300,3]`.
```sh
mo --help
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
```
Paragraphs below provide useful details on relevant parameters.
If a model has multiple inputs, `--input_shape` must be used in conjunction with `--input` parameter.
The parameter `--input` contains a list of input names for which shapes in the same order are defined via `--input_shape`.
For example, launch the Model Optimizer for the ONNX* OCR model with a pair of inputs `data` and `seq_len`
and specify shapes `[3,150,200,1]` and `[3]` for them.
## When to Specify --input_shape Command Line Parameter <a name="when_to_specify_input_shapes"></a>
There are situations when Model Optimizer is unable to deduce input shapes of the model, for example, in case of model cutting due to unsupported operations.
The solution is to provide input shapes of a static rank explicitly.
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
```
## When to Specify --static_shape Command Line Parameter
If the `--static_shape` command line parameter is specified the Model Optimizer evaluates shapes of all operations in the model (shape propagation) for a fixed input(s) shape(s). During the shape propagation the Model Optimizer evaluates operations *Shape* and removes them from the computation graph. With that approach, the initial model which can consume inputs of different shapes may be converted to IR working with the input of one fixed shape only. For example, consider the case when some blob is reshaped from 4D of a shape *[N, C, H, W]* to a shape *[N, C, H \* W]*. During the model conversion the Model Optimize calculates output shape as a constant 1D blob with values *[N, C, H \* W]*. So if the input shape changes to some other value *[N,C,H1,W1]* (it is possible scenario for a fully convolutional model) then the reshape layer becomes invalid.
Resulting Intermediate Representation will not be resizable with the help of OpenVINO Runtime API.
The alternative way to specify input shapes is to use the `--input` parameter as follows:
## Parameters for Pre-Processing
Input data may require pre-processing such as `RGB<->BGR` conversion and mean and scale normalization. To learn about Model Optimizer parameters used for pre-processing, refer to [Optimize Preprocessing Computation](../Additional_Optimizations.md).
```sh
mo --input_model ocr.onnx --input data[3 150 200 1],seq_len[3]
```
The parameter `--input_shape` allows overriding original input shapes to the shapes compatible with a given model.
Dynamic shapes, i.e. with dynamic dimensions, in the original model can be replaced with static shapes for the converted model, and vice versa.
The dynamic dimension can be marked in Model Optimizer command-line as `-1` or `?`.
For example, launch the Model Optimizer for the ONNX* OCR model and specify dynamic batch dimension for inputs.
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
```
To optimize memory consumption for models with undefined dimensions in run-time, Model Optimizer provides the capability to define boundaries of dimensions.
The boundaries of undefined dimension can be specified with ellipsis.
For example, launch the Model Optimizer for the ONNX* OCR model and specify a boundary for the batch dimension.
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
```
## When to Specify --static_shape Command-line Parameter
Model Optimizer provides the `--static_shape` parameter that allows evaluating shapes of all operations in the model for fixed input shapes
and to fold shape computing sub-graphs into constants. The resulting IR can be more compact in size and the loading time for such IR can be decreased.
However, the resulting IR will not be reshape-able with the help of the `reshape` method from OpenVINO Runtime API.
It is worth noting that the `--input_shape` parameter does not affect reshape-ability of the model.
For example, launch the Model Optimizer for the ONNX* OCR model using `--static_shape`.
```sh
mo --input_model ocr.onnx --input data[3 150 200 1],seq_len[3] --static_shape
```
## See Also
* [Configuring the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md)
* [Model Cutting](Cutting_Model.md)
* [Optimize Preprocessing Computation](../Additional_Optimizations.md)
* [Convert TensorFlow Models](Convert_Model_From_TensorFlow.md)
* [Convert ONNX Models](Convert_Model_From_ONNX.md)
* [Convert PyTorch Models](Convert_Model_From_PyTorch.md)
* [Convert PaddlePaddle Models](Convert_Model_From_Paddle.md)
* [Convert MXNet Models](Convert_Model_From_MxNet.md)
* [Convert Caffe Models](Convert_Model_From_Caffe.md)
* [Convert Kaldi Models](Convert_Model_From_Kaldi.md)
* [Introduction](../../Deep_Learning_Model_Optimizer_DevGuide.md)
* [Cutting Off Parts of a Model](Cutting_Model.md)

2
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_YOLO_From_Tensorflow.md
View File

@@ -35,7 +35,7 @@ python keras-YOLOv3-model-set/tools/model_converter/convert.py <path_to_cfg_file
4. Run Model Optimizer to converter the model from the TensorFlow 2 format to an IR:
> **NOTE:** Before you run the conversion, make sure you have installed all the Model Optimizer dependencies for TensorFlow 2.
> **NOTE**: Before you run the conversion, make sure you have installed all the Model Optimizer dependencies for TensorFlow 2.
```sh
mo --saved_model_dir yolov4 --output_dir models/IRs --input_shape [1,608,608,3] --model_name yolov4
```

3
docs/OV_Runtime_UG/ov_dynamic_shapes.md
View File

@@ -73,7 +73,8 @@ If the input model has undefined dimensions that you are not going to change dur
From the API perspective any combination of dynamic and static dimensions can be configured.
Model Optimizer provides capability to reshape the model during the conversion, including specifying dynamic dimensions.
Use this capability to save time on calling `reshape` method in the end application. <TODO: Link to MO setting shape doc>
Use this capability to save time on calling `reshape` method in the end application.
To get information about setting input shapes using Model Optimizer, refer to [Setting Input Shapes](../MO_DG/prepare_model/convert_model/Converting_Model.md)
### Dimension Bounds

2
docs/OV_Runtime_UG/supported_plugins/GPU.md
View File

@@ -81,7 +81,7 @@ Selected precision of each primitive depends on the operation precision in IR, q
u1/u8/i8 data types are used for quantized operations only, i.e. those are not selected automatically for non-quantized operations.
See [low-precision optimization guide](@ref pot_docs_LowPrecisionOptimizationGuide) for more details on how to get quantized model.
Floating-point precision of a GPU primitive is selected based on operation precision in IR except [compressed f16 IR form](../model_representation.md) which is executed in f16 precision.
Floating-point precision of a GPU primitive is selected based on operation precision in IR except [compressed f16 IR form](../../MO_DG/prepare_model/FP16_Compression.md) which is executed in f16 precision.
> **NOTE**: Harware acceleration for i8/u8 precision may be unavailable on some platforms. In that case model is executed in floating-point precision taken from IR. Hardware support of u8/i8 acceleration can be queried via `ov::device::capabilities` property.

2
docs/optimization_guide/model_optimization_guide.md
View File

@@ -17,7 +17,7 @@
- **Neural Network Compression Framework [(NNCF)](./nncf_introduction.md)** provides a suite of advanced algorithms for Neural Networks inference optimization with minimal accuracy drop, for example, quantization, pruning algorithms.
- **Model Optimizer** implements optimization to a model, most of them added by default, but you can configure mean/scale values, batch size RGB vs BGR input channels, and other parameters to speed-up preprocess of a model ([Additional Optimization Use Cases](../MO_DG/prepare_model/Additional_Optimizations.md))
- **Model Optimizer** implements optimization to a model, most of them added by default, but you can configure mean/scale values, batch size, RGB vs BGR input channels, and other parameters to speed-up preprocess of a model ([Embedding Preprocessing Computation](../MO_DG/prepare_model/Additional_Optimizations.md))
## Detailed workflow:

6
docs/resources/resources.md
View File

@@ -10,23 +10,21 @@
Release Notes <https://software.intel.com/content/www/us/en/develop/articles/openvino-relnotes.html>
openvino_docs_IE_DG_supported_plugins_Supported_Devices
openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers
openvino_docs_OV_Glossary
openvino_docs_Legal_Information
.. toctree::
:maxdepth: 1
:caption: Developer Documentation
:caption: Intermediate Representation and Operations Sets
:hidden:
openvino_docs_MO_DG_IR_and_opsets
openvino_docs_ops_opset
openvino_docs_ops_broadcast_rules
openvino_docs_operations_specifications
openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers
openvino_docs_MO_DG_prepare_model_Model_Optimization_Techniques
openvino_docs_MO_DG_prepare_model_convert_model_IR_suitable_for_INT8_inference
openvino_docs_MO_DG_prepare_model_customize_model_optimizer_Customize_Model_Optimizer
.. toctree::

2
tools/mo/openvino/tools/mo/utils/cli_parser.py
View File

@@ -289,7 +289,7 @@ def get_common_cli_parser(parser: argparse.ArgumentParser = None):
'parameter, this scale ' +
'is not applied for any input that does not match with ' +
'the original input of the model.' +
'If both --mean and --scale are specified, ' +
'If both --mean_values and --scale are specified, ' +
'the mean is subtracted first and then scale is applied ' +
'regardless of the order of options in command line.')
common_group.add_argument('--reverse_input_channels',