From 4270dca591e573df151b4567274356ca73ed68e0 Mon Sep 17 00:00:00 2001 From: Sebastian Golebiewski Date: Tue, 13 Jun 2023 07:59:27 +0200 Subject: [PATCH] [DOCS] Fixing formatting issues in articles (#17994) * fixing-formatting --- .../Documentation/inference_modes_overview.md | 2 +- .../convert_model/Convert_Model_From_Kaldi.md | 34 ++-- .../Convert_Model_From_TensorFlow.md | 142 +++++++-------- .../mxnet_specific/Convert_GluonCV_Models.md | 20 +-- .../pytorch_specific/Convert_YOLACT.md | 2 +- .../Convert_BERT_From_Tensorflow.md | 6 +- .../Convert_CRNN_From_Tensorflow.md | 20 +-- .../Convert_DeepSpeech_From_Tensorflow.md | 6 +- .../Convert_NCF_From_Tensorflow.md | 3 +- docs/OV_Runtime_UG/Model_caching_overview.md | 14 +- docs/OV_Runtime_UG/Python_API_inference.md | 3 +- docs/OV_Runtime_UG/ShapeInference.md | 164 +++++++++--------- .../conditional_compilation_deployment.md | 58 +++---- .../supported_plugins/Device_Plugins.md | 28 +-- docs/OV_Runtime_UG/supported_plugins/GNA.md | 44 ++--- .../supported_plugins/GPU_RemoteTensor_API.md | 16 +- .../configurations-for-intel-gna.md | 16 +- .../configurations-for-intel-gpu.md | 15 +- .../installing-openvino-brew.md | 2 +- tools/pot/openvino/tools/pot/api/README.md | 2 +- 20 files changed, 292 insertions(+), 305 deletions(-) diff --git a/docs/Documentation/inference_modes_overview.md b/docs/Documentation/inference_modes_overview.md index 7372a047466..52bb346f4f4 100644 --- a/docs/Documentation/inference_modes_overview.md +++ b/docs/Documentation/inference_modes_overview.md @@ -17,7 +17,7 @@ OpenVINO Runtime offers multiple inference modes to allow optimum hardware utili The remaining modes assume certain levels of automation in selecting devices for inference. Using them in the deployed solution may potentially increase its performance and portability. The automated modes are: * :doc:`Automatic Device Selection (AUTO) ` -* :doc:``Multi-Device Execution (MULTI) ` +* :doc:`Multi-Device Execution (MULTI) ` * :doc:`Heterogeneous Execution (HETERO) ` * :doc:`Automatic Batching Execution (Auto-batching) ` diff --git a/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md b/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md index 8f1615e3af3..89113e2a249 100644 --- a/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md +++ b/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md @@ -7,17 +7,17 @@ Note that OpenVINO support for Kaldi is currently being deprecated and will be removed entirely in the future. -.. note:: +.. note:: Model conversion API supports the `nnet1 `__ and `nnet2 `__ formats of Kaldi models. The support of the `nnet3 `__ format is limited. - + To convert a Kaldi model, run model conversion with the path to the input model ``.nnet`` or ``.mdl`` file: .. code-block:: cpp mo --input_model .nnet -Using Kaldi-Specific Conversion Parameters +Using Kaldi-Specific Conversion Parameters ########################################## The following list provides the Kaldi-specific parameters. @@ -34,43 +34,43 @@ Examples of CLI Commands ######################## * To launch model conversion for the ``wsj_dnn5b_smbr`` model with the specified ``.nnet`` file: - + .. code-block:: cpp mo --input_model wsj_dnn5b_smbr.nnet - + * To launch model conversion for the ``wsj_dnn5b_smbr`` model with the existing file that contains counts for the last layer with biases: .. code-block:: cpp mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts - + * The model conversion normalizes сounts in the following way: - + .. math:: - + S = \frac{1}{\sum_{j = 0}^{|C|}C_{j}} - + .. math:: - + C_{i}=log(S\*C_{i}) - - where :math:`C` - the counts array, :math:`C_{i} - i^{th}` element of the counts array, :math:`|C|` - number of elements in the counts array; + + where :math:`C` - the counts array, :math:`C_{i} - i^{th}` element of the counts array, :math:`|C|` - number of elements in the counts array; * The normalized counts are subtracted from biases of the last or next to last layer (if last layer is SoftMax). - + .. note:: Model conversion API will show a warning if a model contains values of counts and the ``counts`` option is not used. * If you want to remove the last SoftMax layer in the topology, launch the model conversion with the ``remove_output_softmax`` flag: -.. code-block:: cpp + .. code-block:: cpp - mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax + mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax -Model conversion API finds the last layer of the topology and removes this layer only if it is a SoftMax layer. + Model conversion API finds the last layer of the topology and removes this layer only if it is a SoftMax layer. -.. note:: Model conversion can remove SoftMax layer only if the topology has one output. + .. note:: Model conversion can remove SoftMax layer only if the topology has one output. * You can use the *OpenVINO Speech Recognition* sample application for the sample inference of Kaldi models. This sample supports models with only one output. If your model has several outputs, specify the desired one with the ``output`` option. diff --git a/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md b/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md index 30be55882c0..0b875a6eee8 100644 --- a/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md +++ b/docs/MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md @@ -6,10 +6,10 @@ This page provides general instructions on how to run model conversion from a Te To use model conversion API, install OpenVINO Development Tools by following the :doc:`installation instructions `. -Converting TensorFlow 1 Models +Converting TensorFlow 1 Models ############################### -Converting Frozen Model Format +Converting Frozen Model Format +++++++++++++++++++++++++++++++ To convert a TensorFlow model, use the ``*mo*`` script to simply convert a model with a path to the input model ``*.pb*`` file: @@ -19,7 +19,7 @@ To convert a TensorFlow model, use the ``*mo*`` script to simply convert a model mo --input_model .pb -Converting Non-Frozen Model Formats +Converting Non-Frozen Model Formats +++++++++++++++++++++++++++++++++++ There are three ways to store non-frozen TensorFlow models and convert them by model conversion API: @@ -62,14 +62,14 @@ If a model contains operations currently unsupported by OpenVINO, prune these op To determine custom input nodes, display a graph of the model in TensorBoard. To generate TensorBoard logs of the graph, use the ``--tensorboard_logs`` option. TensorFlow 2.x SavedModel format has a specific graph due to eager execution. In case of pruning, find custom input nodes in the ``StatefulPartitionedCall/*`` subgraph of TensorFlow 2.x SavedModel format. -Freezing Custom Models in Python +Freezing Custom Models in Python ++++++++++++++++++++++++++++++++ When a network is defined in Python code, you have to create an inference graph file. Graphs are usually built in a form that allows model training. That means all trainable parameters are represented as variables in the graph. To be able to use such graph with model conversion API, it should be frozen and dumped to a file with the following code: -.. code-block:: python +.. code-block:: python import tensorflow as tf from tensorflow.python.framework import graph_io @@ -84,7 +84,7 @@ Where: * ``inference_graph.pb`` is the name of the generated inference graph file. * ``as_text`` specifies whether the generated file should be in human readable text format or binary. -Converting TensorFlow 2 Models +Converting TensorFlow 2 Models ############################### To convert TensorFlow 2 models, ensure that `openvino-dev[tensorflow2]` is installed via `pip`. @@ -114,7 +114,7 @@ pruning, find custom input nodes in the ``StatefulPartitionedCall/*`` subgraph. Since the 2023.0 release, direct pruning of models in SavedModel format is not supported. It is essential to freeze the model before pruning. Use the following code snippet for model freezing: -.. code-block:: python +.. code-block:: python import tensorflow as tf from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2 @@ -155,7 +155,7 @@ For example, the model with a custom layer ``CustomLayer`` from ``custom_layer.p Then follow the above instructions for the SavedModel format. -.. note:: +.. note:: Do not use other hacks to resave TensorFlow 2 models into TensorFlow 1 formats. @@ -164,23 +164,23 @@ Command-Line Interface (CLI) Examples Using TensorFlow-Specific Parameters * Launching model conversion for Inception V1 frozen model when model file is a plain text protobuf: -.. code-block:: sh + .. code-block:: sh - mo --input_model inception_v1.pbtxt --input_model_is_text -b 1 + mo --input_model inception_v1.pbtxt --input_model_is_text -b 1 * Launching model conversion for Inception V1 frozen model and dump information about the graph to TensorBoard log dir ``/tmp/log_dir`` -.. code-block:: sh + .. code-block:: sh - mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir + mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir * Launching model conversion for BERT model in the SavedModel format, with three inputs. Specify explicitly the input shapes where the batch size and the sequence length equal 2 and 30 respectively. -.. code-block:: sh + .. code-block:: sh - mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30] + mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30] Conversion of TensorFlow models from memory using Python API ############################################################ @@ -189,96 +189,96 @@ Model conversion API supports passing TensorFlow/TensorFlow2 models directly fro * ``tf.keras.Model`` -.. code-block:: python + .. code-block:: python - model = tf.keras.applications.ResNet50(weights="imagenet") - ov_model = convert_model(model) + model = tf.keras.applications.ResNet50(weights="imagenet") + ov_model = convert_model(model) * ``tf.keras.layers.Layer``. Requires setting the "input_shape". -.. code-block:: python + .. code-block:: python - import tensorflow_hub as hub + import tensorflow_hub as hub - model = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/classification/5") - ov_model = convert_model(model, input_shape=[-1, 224, 224, 3]) + model = hub.KerasLayer("https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/classification/5") + ov_model = convert_model(model, input_shape=[-1, 224, 224, 3]) * ``tf.Module``. Requires setting the "input_shape". -.. code-block:: python + .. code-block:: python - class MyModule(tf.Module): - def __init__(self, name=None): - super().__init__(name=name) - self.variable1 = tf.Variable(5.0, name="var1") - self.variable2 = tf.Variable(1.0, name="var2") - def __call__(self, x): - return self.variable1 * x + self.variable2 + class MyModule(tf.Module): + def __init__(self, name=None): + super().__init__(name=name) + self.variable1 = tf.Variable(5.0, name="var1") + self.variable2 = tf.Variable(1.0, name="var2") + def __call__(self, x): + return self.variable1 * x + self.variable2 - model = MyModule(name="simple_module") - ov_model = convert_model(model, input_shape=[-1]) + model = MyModule(name="simple_module") + ov_model = convert_model(model, input_shape=[-1]) * ``tf.compat.v1.Graph`` -.. code-block:: python + .. code-block:: python - with tf.compat.v1.Session() as sess: - inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') - inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') - output = tf.nn.relu(inp1 + inp2, name='Relu') - tf.compat.v1.global_variables_initializer() - model = sess.graph - - ov_model = convert_model(model) + with tf.compat.v1.Session() as sess: + inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') + inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') + output = tf.nn.relu(inp1 + inp2, name='Relu') + tf.compat.v1.global_variables_initializer() + model = sess.graph + + ov_model = convert_model(model) * ``tf.compat.v1.GraphDef`` -.. code-block:: python + .. code-block:: python - with tf.compat.v1.Session() as sess: - inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') - inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') - output = tf.nn.relu(inp1 + inp2, name='Relu') - tf.compat.v1.global_variables_initializer() - model = sess.graph_def - - ov_model = convert_model(model) + with tf.compat.v1.Session() as sess: + inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') + inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') + output = tf.nn.relu(inp1 + inp2, name='Relu') + tf.compat.v1.global_variables_initializer() + model = sess.graph_def + + ov_model = convert_model(model) * ``tf.function`` -.. code-block:: python + .. code-block:: python - @tf.function( - input_signature=[tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32), - tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32)]) - def func(x, y): - return tf.nn.sigmoid(tf.nn.relu(x + y)) - - ov_model = convert_model(func) + @tf.function( + input_signature=[tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32), + tf.TensorSpec(shape=[1, 2, 3], dtype=tf.float32)]) + def func(x, y): + return tf.nn.sigmoid(tf.nn.relu(x + y)) + + ov_model = convert_model(func) * ``tf.compat.v1.session`` -.. code-block:: python + .. code-block:: python - with tf.compat.v1.Session() as sess: - inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') - inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') - output = tf.nn.relu(inp1 + inp2, name='Relu') - tf.compat.v1.global_variables_initializer() + with tf.compat.v1.Session() as sess: + inp1 = tf.compat.v1.placeholder(tf.float32, [100], 'Input1') + inp2 = tf.compat.v1.placeholder(tf.float32, [100], 'Input2') + output = tf.nn.relu(inp1 + inp2, name='Relu') + tf.compat.v1.global_variables_initializer() - ov_model = convert_model(sess) + ov_model = convert_model(sess) * ``tf.train.checkpoint`` -.. code-block:: python + .. code-block:: python - model = tf.keras.Model(...) - checkpoint = tf.train.Checkpoint(model) - save_path = checkpoint.save(save_directory) - # ... - checkpoint.restore(save_path) - ov_model = convert_model(checkpoint) + model = tf.keras.Model(...) + checkpoint = tf.train.Checkpoint(model) + save_path = checkpoint.save(save_directory) + # ... + checkpoint.restore(save_path) + ov_model = convert_model(checkpoint) Supported TensorFlow and TensorFlow 2 Keras Layers ################################################## diff --git a/docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_GluonCV_Models.md b/docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_GluonCV_Models.md index 0d556998d15..ec9687d1655 100644 --- a/docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_GluonCV_Models.md +++ b/docs/MO_DG/prepare_model/convert_model/mxnet_specific/Convert_GluonCV_Models.md @@ -21,25 +21,25 @@ This article provides the instructions and examples on how to convert `GluonCV S 2. Run model conversion API, specifying the ``enable_ssd_gluoncv`` option. Make sure the ``input_shape`` parameter is set to the input shape layout of your model (NHWC or NCHW). The examples below illustrate running model conversion for the SSD and YOLO-v3 models trained with the NHWC layout and located in the ````: -* **For GluonCV SSD topologies:** + * **For GluonCV SSD topologies:** - .. code-block:: sh + .. code-block:: sh - mo --input_model /ssd_512_mobilenet1.0.params --enable_ssd_gluoncv --input_shape [1,512,512,3] --input data --output_dir + mo --input_model /ssd_512_mobilenet1.0.params --enable_ssd_gluoncv --input_shape [1,512,512,3] --input data --output_dir -* **For YOLO-v3 topology:** + * **For YOLO-v3 topology:** - * To convert the model: + * To convert the model: - .. code-block:: sh + .. code-block:: sh - mo --input_model /yolo3_mobilenet1.0_voc-0000.params --input_shape [1,255,255,3] --output_dir + mo --input_model /yolo3_mobilenet1.0_voc-0000.params --input_shape [1,255,255,3] --output_dir - * To convert the model with replacing the subgraph with RegionYolo layers: + * To convert the model with replacing the subgraph with RegionYolo layers: - .. code-block:: sh + .. code-block:: sh - mo --input_model /models/yolo3_mobilenet1.0_voc-0000.params --input_shape [1,255,255,3] --transformations_config "front/mxnet/yolo_v3_mobilenet1_voc. json" --output_dir + mo --input_model /models/yolo3_mobilenet1.0_voc-0000.params --input_shape [1,255,255,3] --transformations_config "front/mxnet/ yolo_v3_mobilenet1_voc. json" --output_dir @endsphinxdirective diff --git a/docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_YOLACT.md b/docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_YOLACT.md index d7e000e3089..36c45ff8f54 100644 --- a/docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_YOLACT.md +++ b/docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_YOLACT.md @@ -184,7 +184,7 @@ Converting a YOLACT Model to the OpenVINO IR format mo --input_model /path/to/yolact.onnx -**Step 4**. Embed input preprocessing into the IR: +**Step 5**. Embed input preprocessing into the IR: To get performance gain by offloading to the OpenVINO application of mean/scale values and RGB->BGR conversion, use the following model conversion API parameters: diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md index 8ba678d3abc..0bbf0f7c825 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_BERT_From_Tensorflow.md @@ -79,13 +79,13 @@ Follow these steps to make a pretrained TensorFlow BERT model reshapable over ba * For UNIX-like systems, run the following command: - .. code-block:: sh + .. code-block:: sh - wget https://gist.githubusercontent.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e/raw/17b8dd0d724281ed7c3b2aeeda662b92809aadd5/download_glue_data.py + wget https://gist.githubusercontent.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e/raw/17b8dd0d724281ed7c3b2aeeda662b92809aadd5/download_glue_data.py * For Windows systems: - Download the `Python script `__ to the current working directory. + Download the `Python script `__ to the current working directory. 6. Download GLUE data by running: diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md index 4fc4be30889..251ac8da42b 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_CRNN_From_Tensorflow.md @@ -15,19 +15,19 @@ If you have another implementation of CRNN model, it can be converted to OpenVIN 1. Clone the repository: .. code-block:: sh - + git clone https://github.com/MaybeShewill-CV/CRNN_Tensorflow.git 2. Go to the ``CRNN_Tensorflow`` directory of the cloned repository: .. code-block:: sh - + cd path/to/CRNN_Tensorflow 3. Check out the necessary commit: .. code-block:: sh - + git checkout 64f1f1867bffaacfeacc7a80eebf5834a5726122 @@ -35,24 +35,22 @@ If you have another implementation of CRNN model, it can be converted to OpenVIN **Step 3.** Create an inference graph: - + 1. Add the ``CRNN_Tensorflow`` folder to ``PYTHONPATH``. * For Linux: - .. code-block:: sh + .. code-block:: sh - export PYTHONPATH="${PYTHONPATH}:/path/to/CRNN_Tensorflow/" + export PYTHONPATH="${PYTHONPATH}:/path/to/CRNN_Tensorflow/" * For Windows, add ``/path/to/CRNN_Tensorflow/`` to the ``PYTHONPATH`` environment variable in settings. - - 2. Edit the ``tools/demo_shadownet.py`` script. After ``saver.restore(sess=sess, save_path=weights_path)`` line, add the following code: .. code-block:: python - + from tensorflow.python.framework import graph_io frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['shadow/LSTMLayers/transpose_time_major']) graph_io.write_graph(frozen, '.', 'frozen_graph.pb', as_text=False) @@ -60,12 +58,12 @@ If you have another implementation of CRNN model, it can be converted to OpenVIN 3. Run the demo with the following command: .. code-block:: sh - + python tools/demo_shadownet.py --image_path data/test_images/test_01.jpg --weights_path model/shadownet/shadownet_2017-10-17-11-47-46.ckpt-199999 If you want to use your checkpoint, replace the path in the ``--weights_path`` parameter with a path to your checkpoint. - + 4. In the ``CRNN_Tensorflow`` directory, you will find the inference CRNN graph ``frozen_graph.pb``. You can use this graph with OpenVINO to convert the model to IR and then run inference. **Step 4.** Convert the model to IR: diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md index 7958595d513..1b224bb21af 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_DeepSpeech_From_Tensorflow.md @@ -19,10 +19,10 @@ To download the model, follow the instruction below: * For UNIX-like systems, run the following command: -.. code-block:: sh + .. code-block:: sh - wget -O - https://github.com/mozilla/DeepSpeech/archive/v0.8.2.tar.gz | tar xvfz - - wget -O - https://github.com/mozilla/DeepSpeech/releases/download/v0.8.2/deepspeech-0.8.2-checkpoint.tar.gz | tar xvfz - + wget -O - https://github.com/mozilla/DeepSpeech/archive/v0.8.2.tar.gz | tar xvfz - + wget -O - https://github.com/mozilla/DeepSpeech/releases/download/v0.8.2/deepspeech-0.8.2-checkpoint.tar.gz | tar xvfz - * For Windows systems: diff --git a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md index a1e8dbe7409..b427b2f0084 100644 --- a/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md +++ b/docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_NCF_From_Tensorflow.md @@ -27,8 +27,7 @@ This tutorial explains how to convert Neural Collaborative Filtering (NCF) model where ``rating/BiasAdd`` is an output node. -3. Convert the model to the OpenVINO format. If you look at your frozen model, you can see that -it has one input that is split into four ``ResourceGather`` layers. (Click image to zoom in.) +3. Convert the model to the OpenVINO format. If you look at your frozen model, you can see that it has one input that is split into four ``ResourceGather`` layers. (Click image to zoom in.) .. image:: ./_static/images/NCF_start.svg diff --git a/docs/OV_Runtime_UG/Model_caching_overview.md b/docs/OV_Runtime_UG/Model_caching_overview.md index dd80f193ea6..0cbe9c52ea0 100644 --- a/docs/OV_Runtime_UG/Model_caching_overview.md +++ b/docs/OV_Runtime_UG/Model_caching_overview.md @@ -40,16 +40,16 @@ To enable model caching, the application must specify a folder to store the cach .. tab-item:: C++ :sync: cpp - .. doxygensnippet:: docs/snippets/ov_caching.cpp - :language: cpp - :fragment: [ov:caching:part0] - + .. doxygensnippet:: docs/snippets/ov_caching.cpp + :language: cpp + :fragment: [ov:caching:part0] + .. tab-item:: Python :sync: py - .. doxygensnippet:: docs/snippets/ov_caching.py - :language: py - :fragment: [ov:caching:part0] + .. doxygensnippet:: docs/snippets/ov_caching.py + :language: py + :fragment: [ov:caching:part0] With this code, if the device specified by ``device_name`` supports import/export model capability, diff --git a/docs/OV_Runtime_UG/Python_API_inference.md b/docs/OV_Runtime_UG/Python_API_inference.md index 965f56d6ed7..6638ae4e6f9 100644 --- a/docs/OV_Runtime_UG/Python_API_inference.md +++ b/docs/OV_Runtime_UG/Python_API_inference.md @@ -74,7 +74,8 @@ Example usage: "Postponed Return" is a practice to omit overhead of ``OVDict``, which is always returned from synchronous calls. "Postponed Return" could be applied when: -* only a part of output data is required. For example, only one specific output is significant + +* only a part of output data is required. For example, only one specific output is significant in a given pipeline step and all outputs are large, thus, expensive to copy. * data is not required "now". For example, it can be later extracted inside the pipeline as a part of latency hiding. diff --git a/docs/OV_Runtime_UG/ShapeInference.md b/docs/OV_Runtime_UG/ShapeInference.md index 245c13946e5..823e1c383fa 100644 --- a/docs/OV_Runtime_UG/ShapeInference.md +++ b/docs/OV_Runtime_UG/ShapeInference.md @@ -9,24 +9,24 @@ troubleshooting_reshape_errors -OpenVINO™ enables you to change model input shape during the application runtime. -It may be useful when you want to feed the model an input that has different size than the model input shape. +OpenVINO™ enables you to change model input shape during the application runtime. +It may be useful when you want to feed the model an input that has different size than the model input shape. The following instructions are for cases where you need to change the model input shape repeatedly. .. note:: - If you need to do this only once, prepare a model with updated shapes via - :doc:`model conversion API `. + If you need to do this only once, prepare a model with updated shapes via + :doc:`model conversion API `. For more information, refer to the :ref:`Specifying input_shape Parameter ` article. The reshape method ++++++++++++++++++++ -The reshape method is used as ``ov::Model::reshape`` in C++ and -`Model.reshape `__ -in Python. The method updates input shapes and propagates them down to the outputs -of the model through all intermediate layers. The code below is an example of how +The reshape method is used as ``ov::Model::reshape`` in C++ and +`Model.reshape `__ +in Python. The method updates input shapes and propagates them down to the outputs +of the model through all intermediate layers. The code below is an example of how to set a new batch size with the ``reshape`` method: .. tab-set:: @@ -45,7 +45,7 @@ to set a new batch size with the ``reshape`` method: :language: Python :fragment: picture_snippet -The diagram below presents the results of using the method, where the size of +The diagram below presents the results of using the method, where the size of model input is changed with an image input: .. image:: _static/images/original_vs_reshaped_model.svg @@ -55,111 +55,111 @@ When using the ``reshape`` method, you may take one of the approaches: .. _usage_of_reshape_method: -1. You can pass a new shape to the method in order to change the input shape of -the model with a single input. See the example of adjusting spatial dimensions to the input image: +1. You can pass a new shape to the method in order to change the input shape of + the model with a single input. See the example of adjusting spatial dimensions to the input image: -.. tab-set:: + .. tab-set:: - .. tab-item:: C++ - :sync: cpp + .. tab-item:: C++ + :sync: cpp - .. doxygensnippet:: docs/snippets/ShapeInference.cpp - :language: cpp - :fragment: spatial_reshape + .. doxygensnippet:: docs/snippets/ShapeInference.cpp + :language: cpp + :fragment: spatial_reshape - .. tab-item:: Python - :sync: py + .. tab-item:: Python + :sync: py - .. doxygensnippet:: docs/snippets/ShapeInference.py - :language: python - :fragment: simple_spatials_change + .. doxygensnippet:: docs/snippets/ShapeInference.py + :language: python + :fragment: simple_spatials_change -To do the opposite - to resize input image to match the input shapes of the model, -use the :doc:`pre-processing API `. + To do the opposite - to resize input image to match the input shapes of the model, + use the :doc:`pre-processing API `. 2. You can express a reshape plan, specifying the input by the port, the index, and the tensor name: -.. tab-set:: + .. tab-set:: - .. tab-item:: Port + .. tab-item:: Port - .. tab-set:: + .. tab-set:: - .. tab-item:: C++ - :sync: cpp + .. tab-item:: C++ + :sync: cpp - ``map, ov::PartialShape`` specifies input by passing actual input port: + ``map, ov::PartialShape`` specifies input by passing actual input port: - .. doxygensnippet:: docs/snippets/ShapeInference.cpp - :language: cpp - :fragment: [obj_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.cpp + :language: cpp + :fragment: [obj_to_shape] - .. tab-item:: Python - :sync: py + .. tab-item:: Python + :sync: py - ``openvino.runtime.Output`` dictionary key specifies input by passing actual input object. - Dictionary values representing new shapes could be ``PartialShape``: + ``openvino.runtime.Output`` dictionary key specifies input by passing actual input object. + Dictionary values representing new shapes could be ``PartialShape``: - .. doxygensnippet:: docs/snippets/ShapeInference.py - :language: python - :fragment: [obj_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.py + :language: python + :fragment: [obj_to_shape] - .. tab-item:: Index + .. tab-item:: Index - .. tab-set:: + .. tab-set:: - .. tab-item:: C++ - :sync: cpp + .. tab-item:: C++ + :sync: cpp - ``map`` specifies input by its index: + ``map`` specifies input by its index: - .. doxygensnippet:: docs/snippets/ShapeInference.cpp - :language: cpp - :fragment: [idx_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.cpp + :language: cpp + :fragment: [idx_to_shape] - .. tab-item:: Python - :sync: py + .. tab-item:: Python + :sync: py - ``int`` dictionary key specifies input by its index. - Dictionary values representing new shapes could be ``tuple``: + ``int`` dictionary key specifies input by its index. + Dictionary values representing new shapes could be ``tuple``: - .. doxygensnippet:: docs/snippets/ShapeInference.py - :language: python - :fragment: [idx_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.py + :language: python + :fragment: [idx_to_shape] - .. tab-item:: Tensor Name + .. tab-item:: Tensor Name - .. tab-set:: + .. tab-set:: - .. tab-item:: C++ - :sync: cpp + .. tab-item:: C++ + :sync: cpp - ``map`` specifies input by its name: + ``map`` specifies input by its name: - .. doxygensnippet:: docs/snippets/ShapeInference.cpp - :language: cpp - :fragment: [name_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.cpp + :language: cpp + :fragment: [name_to_shape] - .. tab-item:: Python - :sync: py + .. tab-item:: Python + :sync: py - ``str`` dictionary key specifies input by its name. - Dictionary values representing new shapes could be ``str``: + ``str`` dictionary key specifies input by its name. + Dictionary values representing new shapes could be ``str``: - .. doxygensnippet:: docs/snippets/ShapeInference.py - :language: python - :fragment: [name_to_shape] + .. doxygensnippet:: docs/snippets/ShapeInference.py + :language: python + :fragment: [name_to_shape] -You can find the usage scenarios of the ``reshape`` method in +You can find the usage scenarios of the ``reshape`` method in :doc:`Hello Reshape SSD Samples `. .. note:: - In some cases, models may not be ready to be reshaped. Therefore, a new input - shape cannot be set neither with :doc:`Model Optimizer ` + In some cases, models may not be ready to be reshaped. Therefore, a new input + shape cannot be set neither with :doc:`Model Optimizer ` nor the ``reshape`` method. The set_batch method @@ -185,24 +185,24 @@ To change the batch dimension of the model, :ref:`set the layout ` or :ref:`model representation ` in OpenVINO Runtime. .. important:: - Shape-changing functionality could be used to turn dynamic model input into a - static one and vice versa. Always set static shapes when the shape of data is - NOT going to change from one inference to another. Setting static shapes can - avoid memory and runtime overheads for dynamic shapes which may vary depending - on hardware plugin and model used. For more information, refer to the + Shape-changing functionality could be used to turn dynamic model input into a + static one and vice versa. Always set static shapes when the shape of data is + NOT going to change from one inference to another. Setting static shapes can + avoid memory and runtime overheads for dynamic shapes which may vary depending + on hardware plugin and model used. For more information, refer to the :doc:`Dynamic Shapes `. diff --git a/docs/OV_Runtime_UG/deployment/conditional_compilation_deployment.md b/docs/OV_Runtime_UG/deployment/conditional_compilation_deployment.md index 3a2266cae20..7813be3e8f7 100644 --- a/docs/OV_Runtime_UG/deployment/conditional_compilation_deployment.md +++ b/docs/OV_Runtime_UG/deployment/conditional_compilation_deployment.md @@ -161,41 +161,41 @@ Considering that JIT kernels can be affected by L1/L2/L3 cache size and the numb - L2/L3 cache emulation - Hack the function of get cache size: + Hack the function of get cache size ``unsigned int dnnl::impl::cpu::platform::get_per_core_cache_size(int level)`` to make it return emulated cache size in analyzed stage, the simplest way is to leverage environment variable to pass the emulated cache size, for example: - .. code-block:: cpp + .. code-block:: cpp - #if defined(SELECTIVE_BUILD_ANALYZER) - if (level == 2) { - const char* L2_cache_size = std::getenv("OV_CC_L2_CACHE_SIZE"); - if (L2_cache_size) { - int size = std::atoi(L2_cache_size); - if (size > 0) { - return size; - } - } - } else if (level == 3) { - const char* L3_cache_size = std::getenv("OV_CC_L3_CACHE_SIZE"); - if (L3_cache_size) { - int size = std::atoi(L3_cache_size); - if (size > 0) { - return size; - } - } - } else if (level == 1) { - const char* L1_cache_size = std::getenv("OV_CC_L1_CACHE_SIZE"); - if (L1_cache_size) { - int size = std::atoi(L1_cache_size); - if (size > 0) { - return size; - } - } - } - #endif + #if defined(SELECTIVE_BUILD_ANALYZER) + if (level == 2) { + const char* L2_cache_size = std::getenv("OV_CC_L2_CACHE_SIZE"); + if (L2_cache_size) { + int size = std::atoi(L2_cache_size); + if (size > 0) { + return size; + } + } + } else if (level == 3) { + const char* L3_cache_size = std::getenv("OV_CC_L3_CACHE_SIZE"); + if (L3_cache_size) { + int size = std::atoi(L3_cache_size); + if (size > 0) { + return size; + } + } + } else if (level == 1) { + const char* L1_cache_size = std::getenv("OV_CC_L1_CACHE_SIZE"); + if (L1_cache_size) { + int size = std::atoi(L1_cache_size); + if (size > 0) { + return size; + } + } + } + #endif - CPU core number emulation diff --git a/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md b/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md index e002214aa79..bd4518e60d9 100644 --- a/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md +++ b/docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md @@ -29,20 +29,20 @@ Feature Support Matrix The table below demonstrates support of key features by OpenVINO device plugins. - ========================================================================================= ============================ =============== =============== - Capability CPU GPU GNA - ========================================================================================= ============================ =============== =============== - :doc:`Heterogeneous execution ` Yes Yes No - :doc:`Multi-device execution ` Yes Yes Partial - :doc:`Automatic batching ` No Yes No - :doc:`Multi-stream execution ` Yes (Intel® x86-64 only) Yes No - :doc:`Models caching ` Yes Partial Yes - :doc:`Dynamic shapes ` Yes Partial No - :doc:`Import/Export ` Yes No Yes - :doc:`Preprocessing acceleration ` Yes Yes No - :doc:`Stateful models ` Yes No Yes - :doc:`Extensibility ` Yes Yes No - ========================================================================================= ============================ =============== =============== +========================================================================================= ============================ =============== =============== + Capability CPU GPU GNA +========================================================================================= ============================ =============== =============== + :doc:`Heterogeneous execution ` Yes Yes No + :doc:`Multi-device execution ` Yes Yes Partial + :doc:`Automatic batching ` No Yes No + :doc:`Multi-stream execution ` Yes (Intel® x86-64 only) Yes No + :doc:`Models caching ` Yes Partial Yes + :doc:`Dynamic shapes ` Yes Partial No + :doc:`Import/Export ` Yes No Yes + :doc:`Preprocessing acceleration ` Yes Yes No + :doc:`Stateful models ` Yes No Yes + :doc:`Extensibility ` Yes Yes No +========================================================================================= ============================ =============== =============== For more details on plugin-specific feature limitations, see the corresponding plugin pages. diff --git a/docs/OV_Runtime_UG/supported_plugins/GNA.md b/docs/OV_Runtime_UG/supported_plugins/GNA.md index 4e912a95b1c..f9b9c141b6d 100644 --- a/docs/OV_Runtime_UG/supported_plugins/GNA.md +++ b/docs/OV_Runtime_UG/supported_plugins/GNA.md @@ -78,39 +78,29 @@ Starting with the 2021.4.1 release of OpenVINO™ and the 03.00.00.1363 version In this mode, the GNA driver automatically falls back on CPU for a particular infer request if the HW queue is not empty. Therefore, there is no need for explicitly switching between GNA and CPU. +.. tab-set:: + .. tab-item:: C++ + :sync: cpp + .. doxygensnippet:: docs/snippets/gna/configure.cpp + :language: cpp + :fragment: [include] + .. doxygensnippet:: docs/snippets/gna/configure.cpp + :language: cpp + :fragment: [ov_gna_exec_mode_hw_with_sw_fback] + .. tab-item:: Python + :sync: py + .. doxygensnippet:: docs/snippets/gna/configure.py + :language: py + :fragment: [import] - - .. tab-set:: - - .. tab-item:: C++ - :sync: cpp - - .. doxygensnippet:: docs/snippets/gna/configure.cpp - :language: cpp - :fragment: [include] - - .. doxygensnippet:: docs/snippets/gna/configure.cpp - :language: cpp - :fragment: [ov_gna_exec_mode_hw_with_sw_fback] - - .. tab-item:: Python - :sync: py - - .. doxygensnippet:: docs/snippets/gna/configure.py - :language: py - :fragment: [import] - - .. doxygensnippet:: docs/snippets/gna/configure.py - :language: py - :fragment: [ov_gna_exec_mode_hw_with_sw_fback] - - - + .. doxygensnippet:: docs/snippets/gna/configure.py + :language: py + :fragment: [ov_gna_exec_mode_hw_with_sw_fback] .. note:: diff --git a/docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md b/docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md index 0eac844e4c7..38a035145e4 100644 --- a/docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md +++ b/docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md @@ -428,12 +428,12 @@ on waiting for the completion of inference. The pseudo-code may look as follows: Limitations +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ - - Some primitives in the GPU plugin may block the host thread on waiting for the previous primitives before adding its kernels - to the command queue. In such cases, the ``ov::InferRequest::start_async()`` call takes much more time to return control to the calling thread - as internally it waits for a partial or full network completion. - Examples of operations: Loop, TensorIterator, DetectionOutput, NonMaxSuppression - - Synchronization of pre/post processing jobs and inference pipeline inside a shared queue is user's responsibility. - - Throughput mode is not available when queue sharing is used, i.e., only a single stream can be used for each compiled model. +- Some primitives in the GPU plugin may block the host thread on waiting for the previous primitives before adding its kernels + to the command queue. In such cases, the ``ov::InferRequest::start_async()`` call takes much more time to return control to the calling thread + as internally it waits for a partial or full network completion. + Examples of operations: Loop, TensorIterator, DetectionOutput, NonMaxSuppression +- Synchronization of pre/post processing jobs and inference pipeline inside a shared queue is user's responsibility. +- Throughput mode is not available when queue sharing is used, i.e., only a single stream can be used for each compiled model. Low-Level Methods for RemoteContext and RemoteTensor Creation ##################################################################### @@ -490,7 +490,7 @@ To see pseudo-code of usage examples, refer to the sections below. See Also ####################################### -* ov::Core -* ov::RemoteTensor +* ``:ref:`ov::Core ``` +* ``:ref:`ov::RemoteTensor ``` @endsphinxdirective diff --git a/docs/install_guides/configurations-for-intel-gna.md b/docs/install_guides/configurations-for-intel-gna.md index fc3579eb88c..157a33ed4d5 100644 --- a/docs/install_guides/configurations-for-intel-gna.md +++ b/docs/install_guides/configurations-for-intel-gna.md @@ -68,17 +68,17 @@ What’s Next? Now you are ready to try out OpenVINO™. You can use the following tutorials to write your applications using Python and C++. -Developing in Python: +* Developing in Python: -* `Start with tensorflow models with OpenVINO™ `__ -* `Start with ONNX and PyTorch models with OpenVINO™ `__ -* `Start with PaddlePaddle models with OpenVINO™ `__ + * `Start with tensorflow models with OpenVINO™ `__ + * `Start with ONNX and PyTorch models with OpenVINO™ `__ + * `Start with PaddlePaddle models with OpenVINO™ `__ -Developing in C++: +* Developing in C++: -* :doc:`Image Classification Async C++ Sample ` -* :doc:`Hello Classification C++ Sample ` -* :doc:`Hello Reshape SSD C++ Sample ` + * :doc:`Image Classification Async C++ Sample ` + * :doc:`Hello Classification C++ Sample ` + * :doc:`Hello Reshape SSD C++ Sample ` @endsphinxdirective diff --git a/docs/install_guides/configurations-for-intel-gpu.md b/docs/install_guides/configurations-for-intel-gpu.md index 400cfeb017e..05421dd7bc5 100644 --- a/docs/install_guides/configurations-for-intel-gpu.md +++ b/docs/install_guides/configurations-for-intel-gpu.md @@ -83,21 +83,20 @@ What’s Next? You can try out the toolkit with: -`Python Quick Start Example `_ to estimate depth in a scene using an OpenVINO monodepth model in a Jupyter Notebook inside your web browser. +* `Python Quick Start Example `_ to estimate depth in a scene using an OpenVINO monodepth model in a Jupyter Notebook inside your web browser. + + Visit the :ref:`Tutorials ` page for more Jupyter Notebooks to get you started with OpenVINO, such as: - Visit the :ref:`Tutorials ` page for more Jupyter Notebooks to get you started with OpenVINO, such as: - * `OpenVINO Python API Tutorial `__ * `Basic image classification program with Hello Image Classification `__ * `Convert a PyTorch model and use it for image background removal `__ +* `C++ Quick Start Example `__ for step-by-step instructions on building and running a basic image classification C++ application. - `C++ Quick Start Example `__ for step-by-step instructions on building and running a basic image classification C++ application. + Visit the :ref:`Samples ` page for other C++ example applications to get you started with OpenVINO, such as: - Visit the :ref:`Samples ` page for other C++ example applications to get you started with OpenVINO, such as: - - * `Basic object detection with the Hello Reshape SSD C++ sample `_ - * `Automatic speech recognition C++ sample `_ + * `Basic object detection with the Hello Reshape SSD C++ sample `_ + * `Automatic speech recognition C++ sample `_ @endsphinxdirective diff --git a/docs/install_guides/installing-openvino-brew.md b/docs/install_guides/installing-openvino-brew.md index 359ae36a8ac..945ecce9959 100644 --- a/docs/install_guides/installing-openvino-brew.md +++ b/docs/install_guides/installing-openvino-brew.md @@ -30,7 +30,7 @@ See `Installing Additional Components <#optional-installing-additional-component * `Homebrew `_ * `CMake 3.13 or higher `__ (choose "macOS 10.13 or later"). Add ``/Applications/CMake.app/Contents/bin`` to path (for default installation). - * `Python 3.7 - 3.11 `__ (choose 3.7 - 3.10). Install and add it to path. + * `Python 3.7 - 3.11 `__ . Install and add it to path. * Apple Xcode Command Line Tools. In the terminal, run ``xcode-select --install`` from any directory to install it. * (Optional) Apple Xcode IDE (not required for OpenVINO™, but useful for development) diff --git a/tools/pot/openvino/tools/pot/api/README.md b/tools/pot/openvino/tools/pot/api/README.md index 12af76c6230..1dd47251417 100644 --- a/tools/pot/openvino/tools/pot/api/README.md +++ b/tools/pot/openvino/tools/pot/api/README.md @@ -107,7 +107,7 @@ All subclasses should override the following methods: - `sampler` - `Sampler` instance that provides a way to iterate over the dataset. (See details below). - `metric_per_sample` - if `Metric` is specified and this parameter is set to True, then the metric value should be - calculated for each data sample, otherwise for the whole dataset. + calculated for each data sample, otherwise for the whole dataset. - `print_progress` - print inference progress. *Returns*