DOCS shift to rst - Model Optimizer Usage (#16630)
This commit is contained in:
Sebastian Golebiewski authored and 
GitHub committed
1 parent
712d1b99d1
commit
3573a38e0b
4 files changed
+144
-107
No files matched your search
@@ -15,102 +15,128 @@
|
||||
openvino_docs_MO_DG_FP16_Compression
|
||||
openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
Model Optimizer is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
|
||||
|
||||
To use it, you need a pre-trained deep learning model in one of the supported formats: TensorFlow, PyTorch, PaddlePaddle, MXNet, Caffe, Kaldi, or ONNX. Model Optimizer converts the model to the OpenVINO Intermediate Representation format (IR), which you can infer later with [OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
|
||||
To use it, you need a pre-trained deep learning model in one of the supported formats: TensorFlow, PyTorch, PaddlePaddle, MXNet, Caffe, Kaldi, or ONNX. Model Optimizer converts the model to the OpenVINO Intermediate Representation format (IR), which you can infer later with :doc:`OpenVINO™ Runtime <openvino_docs_OV_UG_OV_Runtime_User_Guide>`.
|
||||
|
||||
Note that Model Optimizer does not infer models.
|
||||
|
||||
The figure below illustrates the typical workflow for deploying a trained deep learning model:
|
||||
|
||||
|
||||
.. image:: _static/images/BASIC_FLOW_MO_simplified.svg
|
||||
|
||||
where IR is a pair of files describing the model:
|
||||
|
||||
* <code>.xml</code> - Describes the network topology.
|
||||
* ``.xml`` - Describes the network topology.
|
||||
|
||||
* <code>.bin</code> - Contains the weights and biases binary data.
|
||||
* ``.bin`` - Contains the weights and biases binary data.
|
||||
|
||||
The OpenVINO IR can be additionally optimized for inference by [Post-training optimization](../../tools/pot/docs/Introduction.md) that applies post-training quantization methods.
|
||||
The OpenVINO IR can be additionally optimized for inference by :doc:`Post-training optimization <pot_introduction>` that applies post-training quantization methods.
|
||||
|
||||
## How to Run Model Optimizer
|
||||
How to Run Model Optimizer
|
||||
##########################
|
||||
|
||||
To convert a model to IR, you can run Model Optimizer by using the following command:
|
||||
|
||||
```sh
|
||||
mo --input_model INPUT_MODEL
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
If the out-of-the-box conversion (only the `--input_model` parameter is specified) is not successful, use the parameters mentioned below to override input shapes and cut the model:
|
||||
mo --input_model INPUT_MODEL
|
||||
|
||||
- Model Optimizer provides two parameters to override original input shapes for model conversion: `--input` and `--input_shape`.
|
||||
For more information about these parameters, refer to the [Setting Input Shapes](prepare_model/convert_model/Converting_Model.md) guide.
|
||||
|
||||
If the out-of-the-box conversion (only the ``--input_model`` parameter is specified) is not successful, use the parameters mentioned below to override input shapes and cut the model:
|
||||
|
||||
- Model Optimizer provides two parameters to override original input shapes for model conversion: ``--input`` and ``--input_shape``.
|
||||
For more information about these parameters, refer to the :doc:`Setting Input Shapes <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>` guide.
|
||||
|
||||
- To cut off unwanted parts of a model (such as unsupported operations and training sub-graphs),
|
||||
use the `--input` and `--output` parameters to define new inputs and outputs of the converted model.
|
||||
For a more detailed description, refer to the [Cutting Off Parts of a Model](prepare_model/convert_model/Cutting_Model.md) guide.
|
||||
use the ``--input`` and ``--output`` parameters to define new inputs and outputs of the converted model.
|
||||
For a more detailed description, refer to the :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>` guide.
|
||||
|
||||
You can also insert additional input pre-processing sub-graphs into the converted model by using
|
||||
the `--mean_values`, `scales_values`, `--layout`, and other parameters described
|
||||
in the [Embedding Preprocessing Computation](prepare_model/Additional_Optimizations.md) article.
|
||||
the ``--mean_values``, ``scales_values``, ``--layout``, and other parameters described
|
||||
in the :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_Additional_Optimization_Use_Cases>` article.
|
||||
|
||||
The `--compress_to_fp16` compression parameter in Model Optimizer allows generating IR with constants (for example, weights for convolutions and matrix multiplications) compressed to `FP16` data type. For more details, refer to the [Compression of a Model to FP16](prepare_model/FP16_Compression.md) guide.
|
||||
The ``--compress_to_fp16`` compression parameter in Model Optimizer allows generating IR with constants (for example, weights for convolutions and matrix multiplications) compressed to ``FP16`` data type. For more details, refer to the :doc:`Compression of a Model to FP16 <openvino_docs_MO_DG_FP16_Compression>` guide.
|
||||
|
||||
To get the full list of conversion parameters available in Model Optimizer, run the following command:
|
||||
|
||||
```sh
|
||||
mo --help
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
## Examples of CLI Commands
|
||||
mo --help
|
||||
|
||||
|
||||
Examples of CLI Commands
|
||||
########################
|
||||
|
||||
Below is a list of separate examples for different frameworks and Model Optimizer parameters:
|
||||
|
||||
1. Launch Model Optimizer for a TensorFlow MobileNet model in the binary protobuf format:
|
||||
```sh
|
||||
mo --input_model MobileNet.pb
|
||||
```
|
||||
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format with three inputs. Specify input shapes explicitly
|
||||
where the batch size and the sequence length equal 2 and 30 respectively:
|
||||
```sh
|
||||
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
|
||||
```
|
||||
For more information, refer to the [Converting a TensorFlow Model](prepare_model/convert_model/Convert_Model_From_TensorFlow.md) guide.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model MobileNet.pb
|
||||
|
||||
|
||||
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format with three inputs. Specify input shapes explicitly where the batch size and the sequence length equal 2 and 30 respectively:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
|
||||
|
||||
For more information, refer to the :doc:`Converting a TensorFlow Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow>` guide.
|
||||
|
||||
2. Launch Model Optimizer for an ONNX OCR model and specify new output explicitly:
|
||||
```sh
|
||||
mo --input_model ocr.onnx --output probabilities
|
||||
```
|
||||
For more information, refer to the [Converting an ONNX Model](prepare_model/convert_model/Convert_Model_From_ONNX.md) guide.
|
||||
|
||||
> **NOTE**: PyTorch models must be exported to the ONNX format before conversion into IR. More information can be found in [Converting a PyTorch Model](prepare_model/convert_model/Convert_Model_From_PyTorch.md).
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model ocr.onnx --output probabilities
|
||||
|
||||
|
||||
For more information, refer to the :doc:`Converting an ONNX Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX>` guide.
|
||||
|
||||
.. note::
|
||||
|
||||
PyTorch models must be exported to the ONNX format before conversion into IR. More information can be found in :doc:`Converting a PyTorch Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch>`.
|
||||
|
||||
3. Launch Model Optimizer for a PaddlePaddle UNet model and apply mean-scale normalization to the input:
|
||||
```sh
|
||||
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
|
||||
```
|
||||
For more information, refer to the [Converting a PaddlePaddle Model](prepare_model/convert_model/Convert_Model_From_Paddle.md) guide.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
|
||||
|
||||
|
||||
For more information, refer to the :doc:`Converting a PaddlePaddle Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Paddle>` guide.
|
||||
|
||||
4. Launch Model Optimizer for an Apache MXNet SSD Inception V3 model and specify first-channel layout for the input:
|
||||
```sh
|
||||
mo --input_model ssd_inception_v3-0000.params --layout NCHW
|
||||
```
|
||||
For more information, refer to the [Converting an Apache MXNet Model](prepare_model/convert_model/Convert_Model_From_MxNet.md) guide.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model ssd_inception_v3-0000.params --layout NCHW
|
||||
|
||||
|
||||
For more information, refer to the :doc:`Converting an Apache MXNet Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet>` guide.
|
||||
|
||||
5. Launch Model Optimizer for a Caffe AlexNet model with input channels in the RGB format which needs to be reversed:
|
||||
```sh
|
||||
mo --input_model alexnet.caffemodel --reverse_input_channels
|
||||
```
|
||||
For more information, refer to the [Converting a Caffe Model](prepare_model/convert_model/Convert_Model_From_Caffe.md) guide.
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model alexnet.caffemodel --reverse_input_channels
|
||||
|
||||
|
||||
For more information, refer to the :doc:`Converting a Caffe Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe>` guide.
|
||||
|
||||
6. Launch Model Optimizer for a Kaldi LibriSpeech nnet2 model:
|
||||
```sh
|
||||
mo --input_model librispeech_nnet2.mdl --input_shape [1,140]
|
||||
```
|
||||
For more information, refer to the [Converting a Kaldi Model](prepare_model/convert_model/Convert_Model_From_Kaldi.md) guide.
|
||||
|
||||
- To get conversion recipes for specific TensorFlow, ONNX, PyTorch, Apache MXNet, and Kaldi models,
|
||||
refer to the [Model Conversion Tutorials](prepare_model/convert_model/Convert_Model_Tutorials.md).
|
||||
- For more information about IR, see [Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™](IR_and_opsets.md).
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model librispeech_nnet2.mdl --input_shape [1,140]
|
||||
|
||||
|
||||
For more information, refer to the :doc:`Converting a Kaldi Model <openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi>` guide.
|
||||
|
||||
- To get conversion recipes for specific TensorFlow, ONNX, PyTorch, Apache MXNet, and Kaldi models, refer to the :doc:`Model Conversion Tutorials <openvino_docs_MO_DG_prepare_model_convert_model_tutorials>`.
|
||||
- For more information about IR, see :doc:`Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™ <openvino_docs_MO_DG_IR_and_opsets>`.
|
||||
|
||||
@endsphinxdirective
|
||||
@@ -6,86 +6,93 @@ With Model Optimizer you can increase your model's efficiency by providing an ad
|
||||
|
||||
.. _when_to_specify_input_shapes:
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Specifying --input_shape Command-line Parameter
|
||||
###############################################
|
||||
|
||||
## Specifying --input_shape Command-line Parameter
|
||||
Model Optimizer supports conversion of models with dynamic input shapes that contain undefined dimensions.
|
||||
However, if the shape of data is not going to change from one inference request to another,
|
||||
it is recommended to set up static shapes (when all dimensions are fully defined) for the inputs.
|
||||
Doing it at this stage, instead of during inference in runtime, can be beneficial in terms of performance and memory consumption.
|
||||
To set up static shapes, Model Optimizer provides the `--input_shape` parameter.
|
||||
For more information on input shapes under runtime, refer to the [Changing input shapes](../../../OV_Runtime_UG/ShapeInference.md) guide.
|
||||
To learn more about dynamic shapes in runtime, refer to the [Dynamic Shapes](../../../OV_Runtime_UG/ov_dynamic_shapes.md) guide.
|
||||
To set up static shapes, Model Optimizer provides the ``--input_shape`` parameter.
|
||||
For more information on input shapes under runtime, refer to the :doc:`Changing input shapes <openvino_docs_OV_UG_ShapeInference>` guide.
|
||||
To learn more about dynamic shapes in runtime, refer to the :doc:`Dynamic Shapes <openvino_docs_OV_UG_DynamicShapes>` guide.
|
||||
|
||||
The OpenVINO Runtime API may present certain limitations in inferring models with undefined dimensions on some hardware. See the [Features support matrix](../../../OV_Runtime_UG/supported_plugins/Device_Plugins.md) for reference.
|
||||
In this case, the `--input_shape` parameter and the [reshape method](../../../OV_Runtime_UG/ShapeInference.md) can help to resolve undefined dimensions.
|
||||
The OpenVINO Runtime API may present certain limitations in inferring models with undefined dimensions on some hardware. See the :doc:`Features support matrix <openvino_docs_OV_UG_Working_with_devices>` for reference.
|
||||
In this case, the ``--input_shape`` parameter and the :doc:`reshape method <openvino_docs_OV_UG_ShapeInference>` can help to resolve undefined dimensions.
|
||||
|
||||
Sometimes, Model Optimizer is unable to convert models out-of-the-box (only the `--input_model` parameter is specified).
|
||||
Sometimes, Model Optimizer is unable to convert models out-of-the-box (only the ``--input_model`` parameter is specified).
|
||||
Such problem can relate to models with inputs of undefined ranks and a case of cutting off parts of a model.
|
||||
In this case, input shapes must be specified explicitly with the `--input_shape` parameter.
|
||||
In this case, input shapes must be specified explicitly with the ``--input_shape`` parameter.
|
||||
|
||||
For example, run Model Optimizer for the TensorFlow MobileNet model with the single input
|
||||
and specify the input shape of `[2,300,300,3]`:
|
||||
and specify the input shape of ``[2,300,300,3]``:
|
||||
|
||||
```sh
|
||||
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
If a model has multiple inputs, `--input_shape` must be used in conjunction with `--input` parameter.
|
||||
The `--input` parameter contains a list of input names, for which shapes in the same order are defined via `--input_shape`.
|
||||
For example, launch Model Optimizer for the ONNX OCR model with a pair of inputs `data` and `seq_len`
|
||||
and specify shapes `[3,150,200,1]` and `[3]` for them:
|
||||
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
|
||||
|
||||
```sh
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
|
||||
```
|
||||
|
||||
Alternatively, specify input shapes, using the `--input` parameter as follows:
|
||||
If a model has multiple inputs, ``--input_shape`` must be used in conjunction with ``--input`` parameter.
|
||||
The ``--input`` parameter contains a list of input names, for which shapes in the same order are defined via ``--input_shape``.
|
||||
For example, launch Model Optimizer for the ONNX OCR model with a pair of inputs ``data`` and ``seq_len``
|
||||
and specify shapes ``[3,150,200,1]`` and ``[3]`` for them:
|
||||
|
||||
```sh
|
||||
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3]
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
The `--input_shape` parameter allows overriding original input shapes to ones compatible with a given model.
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
|
||||
|
||||
|
||||
Alternatively, specify input shapes, using the ``--input`` parameter as follows:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3]
|
||||
|
||||
|
||||
The ``--input_shape`` parameter allows overriding original input shapes to ones compatible with a given model.
|
||||
Dynamic shapes, i.e. with dynamic dimensions, can be replaced in the original model with static shapes for the converted model, and vice versa.
|
||||
The dynamic dimension can be marked in Model Optimizer command-line as `-1`* or *`?`.
|
||||
The dynamic dimension can be marked in Model Optimizer command-line as ``-1``* or *``?``.
|
||||
For example, launch Model Optimizer for the ONNX OCR model and specify dynamic batch dimension for inputs:
|
||||
|
||||
```sh
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
|
||||
|
||||
|
||||
To optimize memory consumption for models with undefined dimensions in run-time, Model Optimizer provides the capability to define boundaries of dimensions.
|
||||
The boundaries of undefined dimension can be specified with ellipsis.
|
||||
For example, launch Model Optimizer for the ONNX OCR model and specify a boundary for the batch dimension:
|
||||
|
||||
```sh
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
Practically, some models are not ready for input shapes change.
|
||||
In this case, a new input shape cannot be set via Model Optimizer.
|
||||
For more information about shape follow the :doc:`inference troubleshooting <troubleshooting_reshape_errors>`
|
||||
and :ref:`ways to relax shape inference flow <how-to-fix-non-reshape-able-model>` guides.
|
||||
|
||||
@endsphinxdirective
|
||||
Specifying --static_shape Command-line Parameter
|
||||
################################################
|
||||
|
||||
|
||||
## Specifying --static_shape Command-line Parameter
|
||||
Model Optimizer provides the `--static_shape` parameter that allows evaluating shapes of all operations in the model for fixed input shapes
|
||||
Model Optimizer provides the ``--static_shape`` parameter that allows evaluating shapes of all operations in the model for fixed input shapes
|
||||
and folding shape computing sub-graphs into constants. The resulting IR may be more compact in size and the loading time for such IR may decrease.
|
||||
However, the resulting IR will not be reshape-able with the help of the [reshape method](../../../OV_Runtime_UG/ShapeInference.md) from OpenVINO Runtime API.
|
||||
It is worth noting that the `--input_shape` parameter does not affect reshapeability of the model.
|
||||
However, the resulting IR will not be reshape-able with the help of the :doc:`reshape method <openvino_docs_OV_UG_ShapeInference>` from OpenVINO Runtime API.
|
||||
It is worth noting that the ``--input_shape`` parameter does not affect reshapeability of the model.
|
||||
|
||||
For example, launch Model Optimizer for the ONNX OCR model using `--static_shape`:
|
||||
For example, launch Model Optimizer for the ONNX OCR model using ``--static_shape``:
|
||||
|
||||
```sh
|
||||
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3] --static_shape
|
||||
```
|
||||
.. code-block:: sh
|
||||
|
||||
## Additional Resources
|
||||
* [Introduction to converting models with Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md)
|
||||
* [Cutting Off Parts of a Model](Cutting_Model.md)
|
||||
mo --input_model ocr.onnx --input data[3,150,200,1],seq_len[3] --static_shape
|
||||
|
||||
|
||||
Additional Resources
|
||||
####################
|
||||
|
||||
* :doc:`Introduction to converting models with Model Optimizer <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
|
||||
* :doc:`Cutting Off Parts of a Model <openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model>`
|
||||
|
||||
@endsphinxdirective
|
||||
@@ -1,13 +1,17 @@
|
||||
# Model Inputs and Outputs, Shapes and Layouts {#openvino_docs_model_inputs_outputs}
|
||||
|
||||
Users interact with a model by passing data to its _inputs_ before the inference and retrieving data from its _outputs_ after the inference. A model may have one or multiple inputs and outputs. Normally, in OpenVINO™ toolkit, all inputs and outputs in the converted model are identified in the same way as in the original framework model.
|
||||
@sphinxdirective
|
||||
|
||||
OpenVINO uses the _names of tensors_ for identification. Depending on the framework, the names of tensors are formed differently.
|
||||
Users interact with a model by passing data to its *inputs* before the inference and retrieving data from its *outputs* after the inference. A model may have one or multiple inputs and outputs. Normally, in OpenVINO™ toolkit, all inputs and outputs in the converted model are identified in the same way as in the original framework model.
|
||||
|
||||
A model accepts inputs and produces outputs of some _shape_. Shape defines the number of dimensions in a tensor and their order. For example, an image classification model can accept tensor of shape [1, 3, 240, 240] and produces tensor of shape [1, 1000].
|
||||
OpenVINO uses the *names of tensors* for identification. Depending on the framework, the names of tensors are formed differently.
|
||||
|
||||
The meaning of each dimension in the shape is specified by its _layout_. Layout is an interpretation of shape dimensions. OpenVINO toolkit conversion tools and APIs keep all dimensions and their order unchanged and aligned with the original framework model. Usually, original models do not contain layout information explicitly, but in various pre-processing and post-processing scenarios in the OpenVINO Runtime API, sometimes it is required to have the layout specified explicitly. We recommend specifying layouts for inputs/outputs during the model conversion.
|
||||
A model accepts inputs and produces outputs of some *shape*. Shape defines the number of dimensions in a tensor and their order. For example, an image classification model can accept tensor of shape [1, 3, 240, 240] and produces tensor of shape [1, 1000].
|
||||
|
||||
OpenVINO also supports _partially defined shapes_, where part of the dimensions is undefined. Undefined dimensions are also kept intact in the final IR file and you can define them later, during runtime. Undefined dimensions can be used as [dynamic dimensions](../../OV_Runtime_UG/ov_dynamic_shapes.md) for certain hardware and models, which enables you to change shapes of input data dynamically in each infer request. For example, the sequence length dimension in the BERT model can be left undefined and variously sized data along this dimension can be fed on the CPU.
|
||||
The meaning of each dimension in the shape is specified by its *layout*. Layout is an interpretation of shape dimensions. OpenVINO toolkit conversion tools and APIs keep all dimensions and their order unchanged and aligned with the original framework model. Usually, original models do not contain layout information explicitly, but in various pre-processing and post-processing scenarios in the OpenVINO Runtime API, sometimes it is required to have the layout specified explicitly. We recommend specifying layouts for inputs/outputs during the model conversion.
|
||||
|
||||
To learn about how the model is represented in OpenVINO™ Runtime, see the [Model Representation in OpenVINO™ Runtime](../../OV_Runtime_UG/model_representation.md).
|
||||
OpenVINO also supports *partially defined shapes*, where part of the dimensions is undefined. Undefined dimensions are also kept intact in the final IR file and you can define them later, during runtime. Undefined dimensions can be used as :doc:`dynamic dimensions <openvino_docs_OV_UG_DynamicShapes>` for certain hardware and models, which enables you to change shapes of input data dynamically in each infer request. For example, the sequence length dimension in the BERT model can be left undefined and variously sized data along this dimension can be fed on the CPU.
|
||||
|
||||
To learn about how the model is represented in OpenVINO™ Runtime, see the :doc:`Model Representation in OpenVINO™ Runtime <openvino_docs_OV_UG_Model_Representation>`.
|
||||
|
||||
@endsphinxdirective
|
||||
Vendored
File renamed without changes.
Reference in new issue
Block a user