[DOCS] Document Model Optimizer Python API (#14380)
* Added MO convert_model() documentation. * Apply suggestions from code review Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> * Updated Convert_Model pages for PyTorch and TF with PythonAPI info. United TF2 and TF formats lists. * Added info on flag params, example_input formats list, small corrections. * Moved MO python API to separate doc. Small text corrections. * Added TF types conversion description. * Removed duplicating info. * Added description of InputCutInfo types and default onnx opset. * Small correction. * Changed type table to bullets, added blank lines. * Added quote marks. * Removed redunrant bullets. * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Apply suggestions from code review. * Added new line. * Apply comments from review.d * Update docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Added description of lists of parameters. * Update docs/MO_DG/prepare_model/MO_Python_API.md Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> * Added details about input_shape, example_input. * Updated PyTorch page. * Corrected input_signature description. * Format correction. * Format correction. * Format correction. * Format correction. * Small correction. * Small correction. * Removed input_signature param description. * Updated text. * Small correction. * Small correction. * Removed not needed examples. * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Added new line. * Update docs/MO_DG/prepare_model/MO_Python_API.md Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> * Added titles of examples. * Update docs/MO_DG/prepare_model/MO_Python_API.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/MO_DG/prepare_model/MO_Python_API.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/MO_DG/prepare_model/convert_model/Convert_Model_From_PyTorch.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> --------- Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
This commit is contained in:
4 files changed
+271
-3
No files matched your search
@@ -13,6 +13,7 @@
|
||||
openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model
|
||||
openvino_docs_MO_DG_Additional_Optimization_Use_Cases
|
||||
openvino_docs_MO_DG_FP16_Compression
|
||||
openvino_docs_MO_DG_Python_API
|
||||
openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# Model Optimizer Python API {#openvino_docs_MO_DG_Python_API}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
Model Optimizer (MO) has a Python API for model conversion, which is represented by the ``convert_model()`` method in the openvino.tools.mo namespace.
|
||||
|
||||
``convert_model()`` has all the functionality available from the command-line tool, plus the ability to pass Python model objects, such as a Pytorch model or TensorFlow Keras model directly, without saving them into files and without leaving the training environment (Jupyter Notebook or training scripts). In addition to input models consumed directly from Python, ``convert_model`` can take OpenVINO extension objects constructed directly in Python for easier conversion of operations that are not supported in OpenVINO.
|
||||
``convert_model()`` returns an openvino.runtime.Model object which can be compiled and inferred or serialized to IR.
|
||||
|
||||
Example of converting a PyTorch model directly from memory:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import torchvision
|
||||
|
||||
model = torchvision.models.resnet50(pretrained=True)
|
||||
ov_model = convert_model(model)
|
||||
|
||||
The following types are supported as an input model for ``convert_model()``:
|
||||
|
||||
* PyTorch - ``torch.nn.Module``, ``torch.jit.ScriptModule``, ``torch.jit.ScriptFunction``. Refer to the :doc:`Converting a PyTorch Model<openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch>` article for more details.
|
||||
* TensorFlow / TensorFlow 2 / Keras - ``tf.keras.Model``, ``tf.keras.layers.Layer``, ``tf.compat.v1.Graph``, ``tf.compat.v1.GraphDef``, ``tf.Module``, ``tf.function``, ``tf.compat.v1.session``, ``tf.train.checkpoint``. Refer to the :doc:`Converting a TensorFlow Model<openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` article for more details.
|
||||
|
||||
``convert_model()`` accepts all parameters available in the MO command-line tool. Parameters can be specified by Python classes or string analogs, similar to the command-line tool.
|
||||
|
||||
Example of using native Python classes to set ``input_shape``, ``mean_values`` and ``layout``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from openvino.runtime import PartialShape, Layout
|
||||
|
||||
ov_model = convert_model(model, input_shape=PartialShape([1,3,100,100]), mean_values=[127, 127, 127], layout=Layout("NCHW"))
|
||||
|
||||
Example of using strings for setting ``input_shape``, ``mean_values`` and ``layout``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ov_model = convert_model(model, input_shape="[1,3,100,100]", mean_values="[127,127,127]", layout="NCHW")
|
||||
|
||||
|
||||
The ``input`` parameter can be set by a ``tuple`` with a name, shape, and type. The input name of the type string is required in the tuple. The shape and type are optional.
|
||||
The shape can be a ``list`` or ``tuple`` of dimensions (``int`` or ``openvino.runtime.Dimension``), or ``openvino.runtime.PartialShape``, or ``openvino.runtime.Shape``. The type can be of numpy type or ``openvino.runtime.Type``.
|
||||
|
||||
Example of using a tuple in the ``input`` parameter to cut a model:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ov_model = convert_model(model, input=("input_name", [3], np.float32))
|
||||
|
||||
For complex cases, when a value needs to be set in the ``input`` parameter, the ``InputCutInfo`` class can be used. ``InputCutInfo`` accepts four parameters: ``name``, ``shape``, ``type``, and ``value``.
|
||||
|
||||
``InputCutInfo("input_name", [3], np.float32, [0.5, 2.1, 3.4])`` is equivalent of ``InputCutInfo(name="input_name", shape=[3], type=np.float32, value=[0.5, 2.1, 3.4])``.
|
||||
|
||||
Supported types for ``InputCutInfo``:
|
||||
|
||||
* name: ``string``.
|
||||
* shape: ``list`` or ``tuple`` of dimensions (``int`` or ``openvino.runtime.Dimension``), ``openvino.runtime.PartialShape``, ``openvino.runtime.Shape``.
|
||||
* type: ``numpy type``, ``openvino.runtime.Type``.
|
||||
* value: ``numpy.ndarray``, ``list`` of numeric values, ``bool``.
|
||||
|
||||
Example of using ``InputCutInfo`` to freeze an input with value:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from openvino.tools.mo import convert_model, InputCutInfo
|
||||
|
||||
ov_model = convert_model(model, input=InputCutInfo("input_name", [3], np.float32, [0.5, 2.1, 3.4]))
|
||||
|
||||
To set parameters for models with multiple inputs, use ``list`` of parameters.
|
||||
Parameters supporting ``list``:
|
||||
|
||||
* input
|
||||
* input_shape
|
||||
* layout
|
||||
* source_layout
|
||||
* dest_layout
|
||||
* mean_values
|
||||
* scale_values
|
||||
|
||||
Example of using lists to set shapes, types and layout for multiple inputs:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ov_model = convert_model(model, input=[("input1", [1,3,100,100], np.float32), ("input2", [1,3,100,100], np.float32)], layout=[Layout("NCHW"), LayoutMap("NCHW", "NHWC")])
|
||||
|
||||
``layout``, ``source_layout`` and ``dest_layout`` accept an ``openvino.runtime.Layout`` object or ``string``.
|
||||
|
||||
Example of using the ``Layout`` class to set the layout of a model input:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from openvino.runtime import Layout
|
||||
from openvino.tools.mo import convert_model
|
||||
|
||||
ov_model = convert_model(model, source_layout=Layout("NCHW"))
|
||||
|
||||
To set both source and destination layouts in the ``layout`` parameter, use the ``LayoutMap`` class. ``LayoutMap`` accepts two parameters: ``source_layout`` and ``target_layout``.
|
||||
|
||||
``LayoutMap("NCHW", "NHWC")`` is equivalent to ``LayoutMap(source_layout="NCHW", target_layout="NHWC")``.
|
||||
|
||||
Example of using the ``LayoutMap`` class to change the layout of a model input:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
from openvino.tools.mo import convert_model, LayoutMap
|
||||
|
||||
ov_model = convert_model(model, layout=LayoutMap("NCHW", "NHWC"))
|
||||
|
||||
@endsphinxdirective
|
||||
@@ -2,14 +2,75 @@
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
The PyTorch framework is supported through export to the ONNX format. In order to optimize and deploy a model that was trained with it:
|
||||
This page provides instructions on how to convert a model from the PyTorch format to the OpenVINO IR format using Model Optimizer.
|
||||
Model Optimizer Python API allows the conversion of PyTorch models using the ``convert_model()`` method.
|
||||
|
||||
1. `Export a PyTorch model to ONNX <#exporting-a-pytorch-model-to-onnx-format>`__.
|
||||
2. :doc:`Convert the ONNX model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX>` to produce an optimized :doc:`Intermediate Representation <openvino_docs_MO_DG_IR_and_opsets>` of the model based on the trained network topology, weights, and biases values.
|
||||
(Experimental) Converting a PyTorch model with PyTorch Frontend
|
||||
###############################################################
|
||||
|
||||
Example of PyTorch model conversion:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import torchvision
|
||||
import torch
|
||||
from openvino.tools.mo import convert_model
|
||||
|
||||
model = torchvision.models.resnet50(pretrained=True)
|
||||
ov_model = convert_model(model)
|
||||
|
||||
Following PyTorch model formats are supported:
|
||||
|
||||
* ``torch.nn.Module``
|
||||
* ``torch.jit.ScriptModule``
|
||||
* ``torch.jit.ScriptFunction``
|
||||
|
||||
Converting certain PyTorch models may require model tracing, which needs ``input_shape`` or ``example_input`` parameters to be set.
|
||||
|
||||
``example_input`` is used as example input for model tracing.
|
||||
``input_shape`` is used for constructing a float zero-filled torch.Tensor for model tracing.
|
||||
|
||||
Example of using ``example_input``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import torchvision
|
||||
import torch
|
||||
from openvino.tools.mo import convert_model
|
||||
|
||||
model = torchvision.models.resnet50(pretrained=True)
|
||||
ov_model = convert_model(model, example_input=torch.zeros(1, 3, 100, 100))
|
||||
|
||||
``example_input`` accepts the following formats:
|
||||
|
||||
* ``openvino.runtime.Tensor``
|
||||
* ``torch.Tensor``
|
||||
* ``np.ndarray``
|
||||
* ``list`` or ``tuple`` with tensors (``openvino.runtime.Tensor`` / ``torch.Tensor`` / ``np.ndarray``)
|
||||
* ``dictionary`` where key is the input name, value is the tensor (``openvino.runtime.Tensor`` / ``torch.Tensor`` / ``np.ndarray``)
|
||||
|
||||
If ``use_legacy_frontend`` is set, it enables PyTorch model conversion using a temporary ONNX model.
|
||||
ONNX opset version can be set using an optional ``onnx_opset_version`` parameter.
|
||||
|
||||
If ``onnx_opset_version`` is not set, the default opset from ``torch.onnx.export()`` is used.
|
||||
|
||||
Example of PyTorch model conversion using a temporary ONNX model:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import torchvision
|
||||
|
||||
model = torchvision.models.resnet50(pretrained=True)
|
||||
ov_model = convert_model(model, input_shape=[1,3,100,100], use_legacy_frontend=True, onnx_opset_version=13)
|
||||
|
||||
Exporting a PyTorch Model to ONNX Format
|
||||
########################################
|
||||
|
||||
Currently, the most robust method of converting PyTorch models is exporting a PyTorch model to ONNX and then converting it to IR. To convert and deploy a PyTorch model, follow these steps:
|
||||
|
||||
1. `Export a PyTorch model to ONNX <#exporting-a-pytorch-model-to-onnx-format>`__.
|
||||
2. :doc:`Convert the ONNX model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX>` to produce an optimized :doc:`Intermediate Representation <openvino_docs_MO_DG_IR_and_opsets>` of the model based on the trained network topology, weights, and biases values.
|
||||
|
||||
PyTorch models are defined in Python. To export them, use the ``torch.onnx.export()`` method. The code to
|
||||
evaluate or test the model is usually provided with its code and can be used for its initialization and export.
|
||||
The export to ONNX is crucial for this process, but it is covered by PyTorch framework, therefore, It will not be covered here in detail.
|
||||
|
||||
@@ -167,6 +167,103 @@ Command-Line Interface (CLI) Examples Using TensorFlow-Specific Parameters
|
||||
|
||||
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
|
||||
|
||||
Conversion of TensorFlow models from memory using Python API
|
||||
############################################################
|
||||
|
||||
MO Python API supports passing TensorFlow/TensorFlow2 models directly from memory.
|
||||
|
||||
* ``tf.keras.Model``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
model = tf.keras.applications.ResNet50(weights="imagenet")
|
||||
ov_model = convert_model(model)
|
||||
|
||||
|
||||
* ``tf.keras.layers.Layer``. Requires setting the "input_shape".
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
import tensorflow_hub as hub
|
||||
|
||||
model = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/classification/5")
|
||||
ov_model = convert_model(model, input_shape=[-1, 224, 224, 3])
|
||||
|
||||
* ``tf.Module``. Requires setting the "input_shape".
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class MyModule(tf.Module):
|
||||
def __init__(self, name=None):
|
||||
super().__init__(name=name)
|
||||
self.variable1 = tf.Variable(5.0, name="var1")
|
||||
self.variable2 = tf.Variable(1.0, name="var2")
|
||||
def __call__(self, x):
|
||||
return self.variable1 * x + self.variable2
|
||||
|
||||
model = MyModule(name="simple_module")
|
||||
ov_model = convert_model(model, input_shape=[-1])
|
||||
|
||||
* ``tf.compat.v1.Graph``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with tf.compat.v1.Session() as sess:
|
||||
inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
|
||||
inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
|
||||
output = tf.nn.relu(inp1 + inp2, name='Relu')
|
||||
tf.compat.v1.global_variables_initializer()
|
||||
model = sess.graph
|
||||
|
||||
ov_model = convert_model(model)
|
||||
|
||||
* ``tf.compat.v1.GraphDef``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with tf.compat.v1.Session() as sess:
|
||||
inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
|
||||
inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
|
||||
output = tf.nn.relu(inp1 + inp2, name='Relu')
|
||||
tf.compat.v1.global_variables_initializer()
|
||||
model = sess.graph_def
|
||||
|
||||
ov_model = convert_model(model)
|
||||
|
||||
* ``tf.function``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
@tf.function(
|
||||
input_signature=[tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32),
|
||||
tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32)])
|
||||
def func(x, y):
|
||||
return tf.nn.sigmoid(tf.nn.relu(x + y))
|
||||
|
||||
ov_model = convert_model(func)
|
||||
|
||||
* ``tf.compat.v1.session``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
with tf.compat.v1.Session() as sess:
|
||||
inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1')
|
||||
inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2')
|
||||
output = tf.nn.relu(inp1 + inp2, name='Relu')
|
||||
tf.compat.v1.global_variables_initializer()
|
||||
|
||||
ov_model = convert_model(sess)
|
||||
|
||||
* ``tf.train.checkpoint``
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
model = tf.keras.Model(...)
|
||||
checkpoint = tf.train.Checkpoint(model)
|
||||
save_path = checkpoint.save(save_directory)
|
||||
# ...
|
||||
checkpoint.restore(save_path)
|
||||
ov_model = convert_model(checkpoint)
|
||||
|
||||
Supported TensorFlow and TensorFlow 2 Keras Layers
|
||||
##################################################
|
||||
|
||||
Reference in new issue
Block a user