IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Changing file structure of Learn Openvino section (#20337)

Browse Source
This commit is contained in:
Sebastian Golebiewski
2023-10-10 08:34:53 +02:00
committed by GitHub
parent c417d15432
commit 1e05949888
29 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
samples/python/benchmark/bert_benchmark/README.md
-82
View File
@@ -1,82 +0,0 @@
# Bert Benchmark Python Sample {#openvino_inference_engine_ie_bridges_python_sample_bert_benchmark_README}
@sphinxdirective
.. meta::
:description: Learn how to estimate performance of a Bert model using Asynchronous Inference Request (Python) API.
This sample demonstrates how to estimate performance of a Bert model using Asynchronous Inference Request API. Unlike :doc:`demos <omz_demos>` this sample doesn't have configurable command line arguments. Feel free to modify sample's source code to try out different options.
The following Python API is used in the application:
.. tab-set::
.. tab-item:: Python API
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Feature | API | Description |
+================================+=================================================+==============================================+
| OpenVINO Runtime Version | [openvino.runtime.get_version] | Get Openvino API version. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Basic Infer Flow | [openvino.runtime.Core], | Common API to do inference: compile a model. |
| | [openvino.runtime.Core.compile_model] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], | Do asynchronous inference. |
| | [openvino.runtime.AsyncInferQueue.start_async], | |
| | [openvino.runtime.AsyncInferQueue.wait_all] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Model Operations | [openvino.runtime.CompiledModel.inputs] | Get inputs of a model. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/benchmark/bert_benchmark/bert_benchmark.py
:language: python
How It Works
####################
The sample downloads a model and a tokenizer, export the model to onnx, reads the exported model and reshapes it to enforce dynamic input shapes, compiles the resulting model, downloads a dataset and runs benchmarking on the dataset.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
####################
Install the ``openvino`` Python package:
.. code-block:: sh
python -m pip install openvino
Install packages from ``requirements.txt``:
.. code-block:: sh
python -m pip install -r requirements.txt
Run the sample
.. code-block:: sh
python bert_benchmark.py
Sample Output
####################
The sample outputs how long it takes to process a dataset.
See Also
####################
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/benchmark/sync_benchmark/README.md
-141
View File
@@ -1,141 +0,0 @@
# Sync Benchmark Python Sample {#openvino_inference_engine_ie_bridges_python_sample_sync_benchmark_README}
@sphinxdirective
.. meta::
:description: Learn how to estimate performance of a model using Synchronous Inference Request (Python) API.
This sample demonstrates how to estimate performance of a model using Synchronous Inference Request API. It makes sense to use synchronous inference only in latency oriented scenarios. Models with static input shapes are supported. Unlike :doc:`demos <omz_demos>` this sample doesn't have other configurable command line arguments. Feel free to modify sample's source code to try out different options.
.. tab-set::
.. tab-item:: Requirements
+--------------------------------+------------------------------------------------------------------------------+
| Options | Values |
+================================+==============================================================================+
| Validated Models | :doc:`alexnet <omz_models_model_alexnet>`, |
| | :doc:`googlenet-v1 <omz_models_model_googlenet_v1>`, |
| | :doc:`yolo-v3-tf <omz_models_model_yolo_v3_tf>`, |
| | :doc:`face-detection-0200 <omz_models_model_face_detection_0200>` |
+--------------------------------+------------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation |
| | (\*.xml + \*.bin), ONNX (\*.onnx) |
+--------------------------------+------------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+--------------------------------+------------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_sync_benchmark_README>` |
+--------------------------------+------------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Feature | API | Description |
+================================+=================================================+==============================================+
| OpenVINO Runtime Version | [openvino.runtime.get_version] | Get Openvino API version. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Basic Infer Flow | [openvino.runtime.Core], | Common API to do inference: compile a model, |
| | [openvino.runtime.Core.compile_model], | configure input tensors. |
| | [openvino.runtime.InferRequest.get_tensor] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Synchronous Infer | [openvino.runtime.InferRequest.infer], | Do synchronous inference. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Model Operations | [openvino.runtime.CompiledModel.inputs] | Get inputs of a model. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Tensor Operations | [openvino.runtime.Tensor.get_shape], | Get a tensor shape and its data. |
| | [openvino.runtime.Tensor.data] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/benchmark/sync_benchmark/sync_benchmark.py
:language: python
How It Works
####################
The sample compiles a model for a given device, randomly generates input data, performs synchronous inference multiple times for a given number of seconds. Then processes and reports performance results.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
####################
.. code-block:: sh
python sync_benchmark.py <path_to_model>
To run the sample, you need to specify a model:
- You can use :doc:`public <omz_models_group_public>` or doc:`Intel's <omz_models_group_intel>` pre-trained models from the Open Model Zoo. The models can be downloaded using the :doc:`Model Downloader <omz_tools_downloader>`.
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
++++++++++++++++++++
1. Install the ``openvino-dev`` Python package to use Open Model Zoo Tools:
.. code-block:: sh
python -m pip install openvino-dev[caffe]
2. Download a pre-trained model using:
.. code-block:: sh
omz_downloader --name googlenet-v1
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
.. code-block:: sh
omz_converter --name googlenet-v1
4. Perform benchmarking using the ``googlenet-v1`` model on a ``CPU``:
.. code-block:: sh
python sync_benchmark.py googlenet-v1.xml
Sample Output
####################
The application outputs performance results.
.. code-block:: sh
[ INFO ] OpenVINO:
[ INFO ] Build ................................. <version>
[ INFO ] Count: 2333 iterations
[ INFO ] Duration: 10003.59 ms
[ INFO ] Latency:
[ INFO ] Median: 3.90 ms
[ INFO ] Average: 4.29 ms
[ INFO ] Min: 3.30 ms
[ INFO ] Max: 10.11 ms
[ INFO ] Throughput: 233.22 FPS
See Also
####################
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/benchmark/throughput_benchmark/README.md
-147
View File
@@ -1,147 +0,0 @@
# Throughput Benchmark Python Sample {#openvino_inference_engine_ie_bridges_python_sample_throughput_benchmark_README}
@sphinxdirective
.. meta::
:description: Learn how to estimate performance of a model using Asynchronous Inference Request (Python) API in throughput mode.
This sample demonstrates how to estimate performance of a model using Asynchronous Inference Request API in throughput mode. Unlike :doc:`demos <omz_demos>` this sample doesn't have other configurable command line arguments. Feel free to modify sample's source code to try out different options.
The reported results may deviate from what :doc:`benchmark_app <openvino_inference_engine_tools_benchmark_tool_README>` reports. One example is model input precision for computer vision tasks. benchmark_app sets uint8, while the sample uses default model precision which is usually float32.
.. tab-set::
.. tab-item:: Requirements
+--------------------------------+------------------------------------------------------------------------------+
| Options | Values |
+================================+==============================================================================+
| Validated Models | :doc:`alexnet <omz_models_model_alexnet>`, |
| | :doc:`googlenet-v1 <omz_models_model_googlenet_v1>`, |
| | :doc:`yolo-v3-tf <omz_models_model_yolo_v3_tf>`, |
| | :doc:`face-detection-0200 <omz_models_model_face_detection_0200>` |
+--------------------------------+------------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation |
| | (\*.xml + \*.bin), ONNX (\*.onnx) |
+--------------------------------+------------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+--------------------------------+------------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_throughput_benchmark_README>` |
+--------------------------------+------------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Feature | API | Description |
+================================+=================================================+==============================================+
| OpenVINO Runtime Version | [openvino.runtime.get_version] | Get Openvino API version. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Basic Infer Flow | [openvino.runtime.Core], | Common API to do inference: compile a model, |
| | [openvino.runtime.Core.compile_model] | configure input tensors. |
| | [openvino.runtime.InferRequest.get_tensor] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], | Do asynchronous inference. |
| | [openvino.runtime.AsyncInferQueue.start_async], | |
| | [openvino.runtime.AsyncInferQueue.wait_all], | |
| | [openvino.runtime.InferRequest.results] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Model Operations | [openvino.runtime.CompiledModel.inputs] | Get inputs of a model. |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
| Tensor Operations | [openvino.runtime.Tensor.get_shape], | Get a tensor shape and its data. |
| | [openvino.runtime.Tensor.data] | |
+--------------------------------+-------------------------------------------------+----------------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/benchmark/throughput_benchmark/throughput_benchmark.py
:language: python
How It Works
####################
The sample compiles a model for a given device, randomly generates input data, performs asynchronous inference multiple times for a given number of seconds. Then processes and reports performance results.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
####################
.. code-block:: sh
python throughput_benchmark.py <path_to_model>
To run the sample, you need to specify a model:
- You can use :doc:`public <omz_models_group_public>` or :doc:`Intel's <omz_models_group_intel>` pre-trained models from the Open Model Zoo. The models can be downloaded using the :doc:`Model Downloader <omz_tools_downloader>`.
.. note::
Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
++++++++++++++++++++
1. Install the ``openvino-dev`` Python package to use Open Model Zoo Tools:
.. code-block:: sh
python -m pip install openvino-dev[caffe]
2. Download a pre-trained model using:
.. code-block:: sh
omz_downloader --name googlenet-v1
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
.. code-block:: sh
omz_converter --name googlenet-v1
4. Perform benchmarking using the ``googlenet-v1`` model on a ``CPU``:
.. code-block:: sh
python throughput_benchmark.py googlenet-v1.xml
Sample Output
####################
The application outputs performance results.
.. code-block:: sh
[ INFO ] OpenVINO:
[ INFO ] Build ................................. <version>
[ INFO ] Count: 2817 iterations
[ INFO ] Duration: 10012.65 ms
[ INFO ] Latency:
[ INFO ] Median: 13.80 ms
[ INFO ] Average: 14.10 ms
[ INFO ] Min: 8.35 ms
[ INFO ] Max: 28.38 ms
[ INFO ] Throughput: 281.34 FPS
See Also
####################
* :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
* :doc:`Using OpenVINO Samples <openvino_docs_OV_UG_Samples_Overview>`
* :doc:`Model Downloader <omz_tools_downloader>`
* :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/classification_sample_async/README.md
-187
View File
@@ -1,187 +0,0 @@
# Image Classification Async Python Sample {#openvino_inference_engine_ie_bridges_python_sample_classification_sample_async_README}
@sphinxdirective
.. meta::
:description: Learn how to do inference of image classification models
using Asynchronous Inference Request (Python) API.
This sample demonstrates how to do inference of image classification models using Asynchronous Inference Request API.
Models with only 1 input and output are supported.
.. tab-set::
.. tab-item:: Requirements
+----------------------------+-----------------------------------------------------------------------------------+
| Options | Values |
+============================+===================================================================================+
| Validated Models | :doc:`alexnet <omz_models_model_alexnet>` |
+----------------------------+-----------------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
+----------------------------+-----------------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+----------------------------+-----------------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_classification_sample_async_README>` |
+----------------------------+-----------------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+
| Feature | API | Description |
+====================+===========================================================================================================================================================================================================+===========================+
| Asynchronous Infer | `openvino.runtime.AsyncInferQueue <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.AsyncInferQueue.html>`__ , | Do asynchronous inference |
| | `openvino.runtime.AsyncInferQueue.set_callback <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.AsyncInferQueue.html#openvino.runtime.AsyncInferQueue.set_callback>`__ , | |
| | `openvino.runtime.AsyncInferQueue.start_async <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.AsyncInferQueue.html#openvino.runtime.AsyncInferQueue.start_async>`__ , | |
| | `openvino.runtime.AsyncInferQueue.wait_all <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.AsyncInferQueue.html#openvino.runtime.AsyncInferQueue.wait_all>`__ , | |
| | `openvino.runtime.InferRequest.results <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.results>`__ | |
+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------+
Basic OpenVINO™ Runtime API is covered by :doc:`Hello Classification Python Sample <openvino_inference_engine_ie_bridges_python_sample_hello_classification_README>`.
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/classification_sample_async/classification_sample_async.py
:language: python
How It Works
############
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
#######
Run the application with the ``-h`` option to see the usage message:
.. code-block:: sh
python classification_sample_async.py -h
Usage message:
.. code-block:: sh
usage: classification_sample_async.py [-h] -m MODEL -i INPUT [INPUT ...]
[-d DEVICE]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to an .xml or .onnx file with a trained
model.
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Path to an image file(s).
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
To run the sample, you need specify a model and image:
- You can use :doc:`public <omz_models_group_public>` or :doc:`Intel's <omz_models_group_intel>` pre-trained models from the Open Model Zoo. The models can be downloaded using the :doc:`Model Downloader <omz_tools_downloader>`.
- You can use images from the media files collection available `here <https://storage.openvinotoolkit.org/data/test_data>`__ .
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
- Stating flags that take only single option like `-m` multiple times, for example `python classification_sample_async.py -m model.xml -m model2.xml`, results in only the last value being used.
- The sample supports NCHW model layout only.
Example
+++++++
1. Install the ``openvino-dev`` Python package to use Open Model Zoo Tools:
.. code-block:: sh
python -m pip install openvino-dev[caffe]
2. Download a pre-trained model:
.. code-block:: sh
omz_downloader --name alexnet
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
.. code-block:: sh
omz_converter --name alexnet
4. Perform inference of ``banana.jpg`` and ``car.bmp`` using the ``alexnet`` model on a ``GPU``, for example:
.. code-block:: sh
python classification_sample_async.py -m alexnet.xml -i banana.jpg car.bmp -d GPU
Sample Output
#############
The sample application logs each step in a standard output stream and outputs top-10 inference results.
.. code-block:: sh
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/alexnet.xml
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in asynchronous mode
[ INFO ] Image path: /test_data/images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9707602
[ INFO ] 666 0.0216788
[ INFO ] 659 0.0032558
[ INFO ] 435 0.0008082
[ INFO ] 809 0.0004359
[ INFO ] 502 0.0003860
[ INFO ] 618 0.0002867
[ INFO ] 910 0.0002866
[ INFO ] 951 0.0002410
[ INFO ] 961 0.0002193
[ INFO ]
[ INFO ] Image path: /test_data/images/car.bmp
[ INFO ] Top 10 results:
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 656 0.5120340
[ INFO ] 874 0.1142275
[ INFO ] 654 0.0697167
[ INFO ] 436 0.0615163
[ INFO ] 581 0.0552262
[ INFO ] 705 0.0304179
[ INFO ] 675 0.0151660
[ INFO ] 734 0.0151582
[ INFO ] 627 0.0148493
[ INFO ] 757 0.0120964
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
See Also
########
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/hello_classification/README.md
-150
View File
@@ -1,150 +0,0 @@
# Hello Classification Python Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_classification_README}
@sphinxdirective
.. meta::
:description: Learn how to do inference of image classification
models using Synchronous Inference Request (Python) API.
This sample demonstrates how to do inference of image classification models using Synchronous Inference Request API.
Models with only 1 input and output are supported.
.. tab-set::
.. tab-item:: Requirements
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Options | Values |
+===================================+===================================================================================================================================================================+
| Validated Models | :doc:`alexnet <omz_models_model_alexnet>`, :doc:`googlenet-v1 <omz_models_model_googlenet_v1>` |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_hello_classification_README>`, :doc:`C <openvino_inference_engine_ie_bridges_c_samples_hello_classification_README>` |
+-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Feature | API | Description |
+=============================+===========================================================================================================================================================================================================================================+============================================================================================================================================================================================+
| Basic Infer Flow | `openvino.runtime.Core <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html>`__ , | |
| | `openvino.runtime.Core.read_model <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html#openvino.runtime.Core.read_model>`__ , | |
| | `openvino.runtime.Core.compile_model <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html#openvino.runtime.Core.compile_model>`__ | Common API to do inference |
+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Synchronous Infer | `openvino.runtime.CompiledModel.infer_new_request <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.CompiledModel.html#openvino.runtime.CompiledModel.infer_new_request>`__ | Do synchronous inference |
+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Model Operations | `openvino.runtime.Model.inputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.inputs>`__ , | Managing of model |
| | `openvino.runtime.Model.outputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.outputs>`__ | |
+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Preprocessing | `openvino.preprocess.PrePostProcessor <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.PrePostProcessor.html>`__ , | Set image of the original size as input for a model with other input size. Resize and layout conversions will be performed automatically by the corresponding plugin just before inference |
| | `openvino.preprocess.InputTensorInfo.set_element_type <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.InputTensorInfo.html#openvino.preprocess.InputTensorInfo.set_element_type>`__ , | |
| | `openvino.preprocess.InputTensorInfo.set_layout <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.InputTensorInfo.html#openvino.preprocess.InputTensorInfo.set_layout>`__ , | |
| | `openvino.preprocess.InputTensorInfo.set_spatial_static_shape <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.InputTensorInfo.html#openvino.preprocess.InputTensorInfo.set_spatial_static_shape>`__ , | |
| | `openvino.preprocess.PreProcessSteps.resize <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.PreProcessSteps.html#openvino.preprocess.PreProcessSteps.resize>`__ , | |
| | `openvino.preprocess.InputModelInfo.set_layout <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.InputModelInfo.html#openvino.preprocess.InputModelInfo.set_layout>`__ , | |
| | `openvino.preprocess.OutputTensorInfo.set_element_type <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.OutputTensorInfo.html#openvino.preprocess.OutputTensorInfo.set_element_type>`__ , | |
| | `openvino.preprocess.PrePostProcessor.build <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.preprocess.PrePostProcessor.html#openvino.preprocess.PrePostProcessor.build>`__ | |
+-----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/hello_classification/hello_classification.py
:language: python
How It Works
############
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
#######
.. code-block:: console
python hello_classification.py <path_to_model> <path_to_image> <device_name>
To run the sample, you need to specify a model and image:
- You can use :doc:`public <omz_models_group_public>` or :doc:`Intel's <omz_models_group_intel>` pre-trained models from the Open Model Zoo. The models can be downloaded using the :doc:`Model Downloader <omz_tools_downloader>`.
- You can use images from the media files collection available at `the storage <https://storage.openvinotoolkit.org/data/test_data>`__.
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
+++++++
1. Install the ``openvino-dev`` Python package to use Open Model Zoo Tools:
.. code-block:: console
python -m pip install openvino-dev[caffe]
2. Download a pre-trained model:
.. code-block:: console
omz_downloader --name alexnet
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
.. code-block:: console
omz_converter --name alexnet
4. Perform inference of ``banana.jpg`` using the ``alexnet`` model on a ``GPU``, for example:
.. code-block:: console
python hello_classification.py alexnet.xml banana.jpg GPU
Sample Output
#############
The sample application logs each step in a standard output stream and outputs top-10 inference results.
.. code-block:: console
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/alexnet/alexnet.xml
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: /images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9703885
[ INFO ] 666 0.0219518
[ INFO ] 659 0.0033120
[ INFO ] 435 0.0008246
[ INFO ] 809 0.0004433
[ INFO ] 502 0.0003852
[ INFO ] 618 0.0002906
[ INFO ] 910 0.0002848
[ INFO ] 951 0.0002427
[ INFO ] 961 0.0002213
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
See Also
########
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/hello_query_device/README.md
-120
View File
@@ -1,120 +0,0 @@
# Hello Query Device Python Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_query_device_README}
@sphinxdirective
.. meta::
:description: Learn how to show metrics and default
configuration values of inference devices using Query
Device (Python) API feature.
This sample demonstrates how to show OpenVINO™ Runtime devices and prints their metrics and default configuration values using :doc:`Query Device API feature <openvino_docs_OV_UG_query_api>`.
.. tab-set::
.. tab-item:: Requirements
+-------------------------------------------------------+--------------------------------------------------------------------------+
| Options | Values |
+=======================================================+==========================================================================+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+-------------------------------------------------------+--------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_hello_query_device_README>` |
+-------------------------------------------------------+--------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+
| Feature | API | Description |
+=======================================+============================================================================================================================================================================================+========================================+
| Basic | `openvino.runtime.Core <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html>`__ | Common API |
+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+
| Query Device | `openvino.runtime.Core.available_devices <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html#openvino.runtime.Core.available_devices>`__ , | Get device properties |
| | `openvino.runtime.Core.get_metric <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.inference_engine.IECore.html#openvino.inference_engine.IECore.get_metric>`__ , | |
| | `openvino.runtime.Core.get_config <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.inference_engine.IECore.html#openvino.inference_engine.IECore.get_config>`__ | |
+---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------+
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/hello_query_device/hello_query_device.py
:language: python
How It Works
############
The sample queries all available OpenVINO™ Runtime devices and prints their supported metrics and plugin configuration parameters.
Running
#######
The sample has no command-line parameters. To see the report, run the following command:
.. code-block:: console
python hello_query_device.py
Sample Output
#############
The application prints all available devices with their supported metrics and default values for configuration parameters.
For example:
.. code-block:: console
[ INFO ] Available devices:
[ INFO ] CPU :
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES:
[ INFO ] FULL_DEVICE_NAME: Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
[ INFO ] OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1
[ INFO ] RANGE_FOR_STREAMS: 1, 8
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CACHE_DIR:
[ INFO ] CPU_BIND_THREAD: NO
[ INFO ] CPU_THREADS_NUM: 0
[ INFO ] CPU_THROUGHPUT_STREAMS: 1
[ INFO ] DUMP_EXEC_GRAPH_AS_DOT:
[ INFO ] ENFORCE_BF16: NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] PERFORMANCE_HINT:
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ] PERF_COUNT: NO
[ INFO ]
[ INFO ] GNA :
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES: GNA_SW
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ] FULL_DEVICE_NAME: GNA_SW
[ INFO ] GNA_LIBRARY_FULL_VERSION: 3.0.0.1455
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] GNA_COMPACT_MODE: YES
[ INFO ] GNA_COMPILE_TARGET:
[ INFO ] GNA_DEVICE_MODE: GNA_SW_EXACT
[ INFO ] GNA_EXEC_TARGET:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE_GENERATION:
[ INFO ] GNA_LIB_N_THREADS: 1
[ INFO ] GNA_PRECISION: I16
[ INFO ] GNA_PWL_MAX_ERROR_PERCENT: 1.000000
[ INFO ] GNA_PWL_UNIFORM_DESIGN: NO
[ INFO ] GNA_SCALE_FACTOR: 1.000000
[ INFO ] GNA_SCALE_FACTOR_0: 1.000000
[ INFO ] LOG_LEVEL: LOG_NONE
[ INFO ] PERF_COUNT: NO
[ INFO ] SINGLE_THREAD: YES
See Also
########
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
@endsphinxdirective
samples/python/hello_reshape_ssd/README.md
-133
View File
@@ -1,133 +0,0 @@
# Hello Reshape SSD Python Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_reshape_ssd_README}
@sphinxdirective
.. meta::
:description: Learn how to do inference of object detection
models using shape inference feature and Synchronous
Inference Request (Python) API.
This sample demonstrates how to do synchronous inference of object detection models using :doc:`Shape Inference feature <openvino_docs_OV_UG_ShapeInference>`.
Models with only 1 input and output are supported.
.. tab-set::
.. tab-item:: Requirements
+------------------------------------+---------------------------------------------------------------------------+
| Options | Values |
+====================================+===========================================================================+
| Validated Models | :doc:`mobilenet-ssd <omz_models_model_mobilenet_ssd>` |
+------------------------------------+---------------------------------------------------------------------------+
| Validated Layout | NCHW |
+------------------------------------+---------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
+------------------------------------+---------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+------------------------------------+---------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_hello_reshape_ssd_README>` |
+------------------------------------+---------------------------------------------------------------------------+
.. tab-item:: Python API
The following Python API is used in the application:
+------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+
| Feature | API | Description |
+====================================+================================================================================================================================================================================+======================================+
| Model Operations | `openvino.runtime.Model.reshape <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.reshape>`__ , | Managing of model |
| | `openvino.runtime.Model.input <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.input>`__ , | |
| | `openvino.runtime.Output.get_any_name <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Output.html#openvino.runtime.Output.get_any_name>`__ , | |
| | `openvino.runtime.PartialShape <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.PartialShape.html>`__ | |
+------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+
Basic OpenVINO™ Runtime API is covered by :doc:`Hello Classification Python* Sample <openvino_inference_engine_ie_bridges_python_sample_hello_classification_README>`.
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/hello_reshape_ssd/hello_reshape_ssd.py
:language: python
How It Works
############
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data.
As a result, the program creates an output image, logging each step in a standard output stream.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
#######
.. code-block:: console
python hello_reshape_ssd.py <path_to_model> <path_to_image> <device_name>
To run the sample, you need to specify a model and image:
- You can use :doc:`public <omz_models_group_public>` or :doc:`Intel's <omz_models_group_intel>` pre-trained models from the Open Model Zoo. The models can be downloaded using the :doc:`Model Downloader <omz_tools_downloader>`.
- You can use images from the media files collection available at `the storage <https://storage.openvinotoolkit.org/data/test_data>`.
.. note::
- By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using model conversion API with ``reverse_input_channels`` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of :doc:`Embedding Preprocessing Computation <openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model>`.
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Example
+++++++
1. Install the ``openvino-dev`` Python package to use Open Model Zoo Tools:
.. code-block:: console
python -m pip install openvino-dev[caffe]
2. Download a pre-trained model:
.. code-block:: console
omz_downloader --name mobilenet-ssd
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
.. code-block:: console
omz_converter --name mobilenet-ssd
4. Perform inference of ``banana.jpg`` using ``ssdlite_mobilenet_v2`` model on a ``GPU``, for example:
.. code-block:: console
python hello_reshape_ssd.py mobilenet-ssd.xml banana.jpg GPU
Sample Output
#############
The sample application logs each step in a standard output stream and creates an output image, drawing bounding boxes for inference results with an over 50% confidence.
.. code-block:: console
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/mobilenet-ssd.xml
[ INFO ] Reshaping the model to the height and width of the input image
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Found: class_id = 52, confidence = 0.98, coords = (21, 98), (276, 210)
[ INFO ] Image out.bmp was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
See Also
########
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/model_creation_sample/README.md
-176
View File
@@ -1,176 +0,0 @@
# Model Creation Python Sample {#openvino_inference_engine_ie_bridges_python_sample_model_creation_sample_README}
@sphinxdirective
.. meta::
:description: Learn how to create a model on the fly with a
provided weights file and infer it later using Synchronous
Inference Request (Python) API.
This sample demonstrates how to run inference using a :doc:`model <openvino_docs_OV_UG_Model_Representation>` built on the fly that uses weights from the LeNet classification model, which is known to work well on digit classification tasks. You do not need an XML file, the model is created from the source code on the fly.
.. tab-set::
.. tab-item:: Requirements
+------------------------------------------------+-----------------------------------------------------------------------------+
| Options | Values |
+================================================+=============================================================================+
| Validated Models | LeNet |
+------------------------------------------------+-----------------------------------------------------------------------------+
| Model Format | Model weights file (\*.bin) |
+------------------------------------------------+-----------------------------------------------------------------------------+
| Supported devices | :doc:`All <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+------------------------------------------------+-----------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_model_creation_sample_README>` |
+------------------------------------------------+-----------------------------------------------------------------------------+
.. tab-item:: Python API
The following OpenVINO Python API is used in the application:
+------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
| Feature | API | Description |
+==========================================+==============================================================================================================================================================+====================================================================================+
| Model Operations | `openvino.runtime.Model <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html>`__ , | Managing of model |
| | `openvino.runtime.set_batch <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.set_batch.html>`__ , | |
| | `openvino.runtime.Model.input <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.input>`__ | |
+------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
| Opset operations | `openvino.runtime.op.Parameter <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.op.Parameter.html>`__ , | Description of a model topology using OpenVINO Python API |
| | `openvino.runtime.op.Constant <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.op.Constant.html>`__ , | |
| | `openvino.runtime.opset8.convolution <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.convolution.html>`__ , | |
| | `openvino.runtime.opset8.add <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.add.html>`__ , | |
| | `openvino.runtime.opset1.max_pool <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset1.max_pool.html>`__ , | |
| | `openvino.runtime.opset8.reshape <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.reshape.html>`__ , | |
| | `openvino.runtime.opset8.matmul <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.matmul.html>`__ , | |
| | `openvino.runtime.opset8.relu <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.relu.html>`__ , | |
| | `openvino.runtime.opset8.softmax <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.opset8.softmax.html>`__ | |
+------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------+
Basic OpenVINO™ Runtime API is covered by :doc:`Hello Classification Python* Sample <openvino_inference_engine_ie_bridges_python_sample_hello_classification_README>`.
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/model_creation_sample/model_creation_sample.py
:language: python
How It Works
############
At startup, the sample application does the following:
- Reads command line parameters
- :doc:`Build a Model <openvino_docs_OV_UG_Model_Representation>` and passed weights file
- Loads the model and input data to the OpenVINO™ Runtime plugin
- Performs synchronous inference and processes output data, logging each step in a standard output stream
You can see the explicit description of each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
Running
#######
To run the sample, you need to specify model weights and device.
.. code-block:: console
python model_creation_sample.py <path_to_model> <device_name>
.. note::
- This sample supports models with FP32 weights only.
- The ``lenet.bin`` weights file was generated by :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>` from the public LeNet model with the ``input_shape [64,1,28,28]`` parameter specified.
- The original model is available in the `Caffe* repository <https://github.com/BVLC/caffe/tree/master/examples/mnist>`__ on GitHub\*.
For example:
.. code-block:: console
python model_creation_sample.py lenet.bin GPU
Sample Output
#############
The sample application logs each step in a standard output stream and outputs 10 inference results.
.. code-block:: console
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Loading the model using ngraph function with weights from lenet.bin
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Top 1 results:
[ INFO ] Image 0
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 0 1.0000000 0
[ INFO ]
[ INFO ] Image 1
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 1 1.0000000 1
[ INFO ]
[ INFO ] Image 2
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 2 1.0000000 2
[ INFO ]
[ INFO ] Image 3
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 3 1.0000000 3
[ INFO ]
[ INFO ] Image 4
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 4 1.0000000 4
[ INFO ]
[ INFO ] Image 5
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 5 1.0000000 5
[ INFO ]
[ INFO ] Image 6
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 6 1.0000000 6
[ INFO ]
[ INFO ] Image 7
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 7 1.0000000 7
[ INFO ]
[ INFO ] Image 8
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 8 1.0000000 8
[ INFO ]
[ INFO ] Image 9
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 9 1.0000000 9
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
See Also
########
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective
samples/python/speech_sample/README.md
-399
View File
@@ -1,399 +0,0 @@
# Automatic Speech Recognition Python Sample {#openvino_inference_engine_ie_bridges_python_sample_speech_sample_README}
@sphinxdirective
.. meta::
:description: Learn how to infer an acoustic model based on Kaldi
neural networks and speech feature vectors using Asynchronous
Inference Request (Python) API.
.. note::
This sample is being deprecated and will no longer be maintained after
OpenVINO 2023.2 (LTS). The main reason for it is the outdated state of
the sample and its extensive usage of GNA, which is not going to be
supported by OpenVINO beyond 2023.2.
This sample demonstrates how to do a Synchronous Inference of acoustic model based on Kaldi\* neural models and speech feature vectors.
The sample works with Kaldi ARK or Numpy* uncompressed NPZ files, so it does not cover an end-to-end speech recognition scenario (speech to text), requiring additional preprocessing (feature extraction) to get a feature vector from a speech signal, as well as postprocessing (decoding) to produce text from scores.
.. tab-set::
.. tab-item:: Requirements
+----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Options | Values |
+======================================================================+======================================================================================================================================================================+
| Validated Models | Acoustic model based on Kaldi* neural models (see :ref:`Model Preparation <model-preparation-speech-python>` section) |
+----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin) |
+----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Supported devices | See :ref:`Execution Modes <execution-modes-speech-python>` section below and :doc:`List Supported Devices <openvino_docs_OV_UG_supported_plugins_Supported_Devices>` |
+----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Other language realization | :doc:`C++ <openvino_inference_engine_samples_speech_sample_README>` |
+----------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+
.. tab-item:: Python API
Automatic Speech Recognition Python sample application demonstrates how to use the following Python API in applications:
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| Feature | API | Description |
+===================================================================+================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+=======================================================================+
| Import/Export Model | `openvino.runtime.Core.import_model <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Core.html#openvino.runtime.Core.import_model>`__ , `openvino.runtime.CompiledModel.export_model <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.CompiledModel.html#openvino.runtime.CompiledModel.export_model>`__ | The GNA plugin supports loading and saving of the GNA-optimized model |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| Model Operations | `openvino.runtime.Model.add_outputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.Model.html#openvino.runtime.Model.add_outputs>`__ , `openvino.runtime.set_batch <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.html#openvino.runtime.set_batch>`__ , `openvino.runtime.CompiledModel.inputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.CompiledModel.html#openvino.runtime.CompiledModel.inputs>`__ , `openvino.runtime.CompiledModel.outputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.CompiledModel.html#openvino.runtime.CompiledModel.outputs>`__ , `openvino.runtime.ConstOutput.any_name <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.ConstOutput.html#openvino.runtime.ConstOutput.any_name>`__ | Managing of model: configure batch_size, input and output tensors |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| Synchronous Infer | `openvino.runtime.CompiledModel.create_infer_request <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.CompiledModel.html#openvino.runtime.CompiledModel.create_infer_request>`__ , `openvino.runtime.InferRequest.infer <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.infer>`__ | Do synchronous inference |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| InferRequest Operations | `openvino.runtime.InferRequest.get_input_tensor <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.get_input_tensor>`__ , `openvino.runtime.InferRequest.model_outputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.model_outputs>`__ , `openvino.runtime.InferRequest.model_inputs <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.model_inputs>`__ , | Get info about model using infer request API |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| InferRequest Operations | `openvino.runtime.InferRequest.query_state <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.query_state>`__ , `openvino.runtime.VariableState.reset <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.inference_engine.VariableState.html#openvino.inference_engine.VariableState.reset>`__ | Gets and resets CompiledModel state control |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
| Profiling | `openvino.runtime.InferRequest.profiling_info <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.InferRequest.html#openvino.runtime.InferRequest.profiling_info>`__ , `openvino.runtime.ProfilingInfo.real_time <https://docs.openvino.ai/2023.1/api/ie_python_api/_autosummary/openvino.runtime.ProfilingInfo.html#openvino.runtime.ProfilingInfo.real_time>`__ | Get infer request profiling info |
+-------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+
Basic OpenVINO™ Runtime API is covered by :doc:`Hello Classification Python* Sample <openvino_inference_engine_ie_bridges_python_sample_hello_classification_README>`.
.. tab-item:: Sample Code
.. doxygensnippet:: samples/python/speech_sample/speech_sample.py
:language: python
How It Works
############
At startup, the sample application reads command-line parameters, loads a specified model and input data to the OpenVINO™ Runtime plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream.
You can see the explicit description of
each sample step at :doc:`Integration Steps <openvino_docs_OV_UG_Integrate_OV_with_your_application>` section of "Integrate OpenVINO™ Runtime with Your Application" guide.
GNA-specific details
####################
Quantization
++++++++++++
If the GNA device is selected (for example, using the ``-d`` GNA flag), the GNA OpenVINO™ Runtime plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
Several neural model quantization modes:
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural model is quantized to accommodate the scaled input dynamic range.
- *user-defined* - The user may specify a scale factor via the ``-sf`` flag that will be used for static quantization.
The ``-qb`` flag provides a hint to the GNA plugin regarding the preferred target weight resolution for all layers.
For example, when ``-qb 8`` is specified, the plugin will use 8-bit weights wherever possible in the
model.
.. note::
It is not always possible to use 8-bit weights due to GNA hardware limitations. For example, convolutional layers always use 16-bit weights (GNA hardware version 1 and 2). This limitation will be removed in GNA hardware version 3 and higher.
.. _execution-modes-speech-python:
Execution Modes
+++++++++++++++
Several execution modes are supported via the ``-d`` flag:
- ``CPU`` - All calculations are performed on CPU device using CPU Plugin.
- ``GPU`` - All calculations are performed on GPU device using GPU Plugin.
- ``NPU`` - All calculations are performed on NPU device using NPU Plugin.
- ``GNA_AUTO`` - GNA hardware is used if available and the driver is installed. Otherwise, the GNA device is emulated in fast-but-not-bit-exact mode.
- ``GNA_HW`` - GNA hardware is used if available and the driver is installed. Otherwise, an error will occur.
- ``GNA_SW`` - Deprecated. The GNA device is emulated in fast-but-not-bit-exact mode.
- ``GNA_SW_FP32`` - Substitutes parameters and calculations from low precision to floating point (FP32).
- ``GNA_SW_EXACT`` - GNA device is emulated in bit-exact mode.
Loading and Saving Models
+++++++++++++++++++++++++
The GNA plugin supports loading and saving of the GNA-optimized model (non-IR) via the ``-rg`` and ``-wg`` flags.
Thereby, it is possible to avoid the cost of full model quantization at run time.
The GNA plugin also supports export of firmware-compatible embedded model images for the Intel® Speech Enabling Developer Kit and Amazon Alexa* Premium Far-Field Voice Development Kit via the ``-we`` flag (save only).
In addition to performing inference directly from a GNA model file, these options make it possible to:
- Convert from IR format to GNA format model file (``-m``, ``-wg``)
- Convert from IR format to embedded format model file (``-m``, ``-we``)
- Convert from GNA format to embedded format model file (``-rg``, ``-we``)
Running
#######
Run the application with the ``-h`` option to see the usage message:
.. code-block:: sh
python speech_sample.py -h
Usage message:
.. code-block:: console
usage: speech_sample.py [-h] (-m MODEL | -rg IMPORT_GNA_MODEL) -i INPUT [-o OUTPUT] [-r REFERENCE] [-d DEVICE] [-bs [1-8]]
[-layout LAYOUT] [-qb [8, 16]] [-sf SCALE_FACTOR] [-wg EXPORT_GNA_MODEL]
[-we EXPORT_EMBEDDED_GNA_MODEL] [-we_gen [GNA1, GNA3]]
[--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]] [-pc] [-a [CORE, ATOM]] [-iname INPUT_LAYERS]
[-oname OUTPUT_LAYERS] [-cw_l CONTEXT_WINDOW_LEFT] [-cw_r CONTEXT_WINDOW_RIGHT] [-pwl_me PWL_ME]
optional arguments:
-m MODEL, --model MODEL
Path to an .xml file with a trained model (required if -rg is missing).
-rg IMPORT_GNA_MODEL, --import_gna_model IMPORT_GNA_MODEL
Read GNA model from file using path/filename provided (required if -m is missing).
Options:
-h, --help Show this help message and exit.
-i INPUT, --input INPUT
Required. Path(s) to input file(s).
Usage for a single file/layer: <input_file.ark> or <input_file.npz>.
Example of usage for several files/layers: <layer1>:<port_num1>=<input_file1.ark>,<layer2>:<port_num2>=<input_file2.ark>.
-o OUTPUT, --output OUTPUT
Optional. Output file name(s) to save scores (inference results).
Usage for a single file/layer: <output_file.ark> or <output_file.npz>.
Example of usage for several files/layers: <layer1>:<port_num1>=<output_file1.ark>,<layer2>:<port_num2>=<output_file2.ark>.
-r REFERENCE, --reference REFERENCE
Read reference score file(s) and compare inference results with reference scores.
Usage for a single file/layer: <reference_file.ark> or <reference_file.npz>.
Example of usage for several files/layers: <layer1>:<port_num1>=<reference_file1.ark>,<layer2>:<port_num2>=<reference_file2.ark>.
-d DEVICE, --device DEVICE
Optional. Specify a target device to infer on. CPU, GPU, NPU, GNA_AUTO, GNA_HW, GNA_SW_FP32,
GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g.
HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified.
Default value is CPU.
-bs [1-8], --batch_size [1-8]
Optional. Batch size 1-8.
-layout LAYOUT Optional. Custom layout in format: "input0[value0],input1[value1]" or "[value]" (applied to all
inputs)
-qb [8, 16], --quantization_bits [8, 16]
Optional. Weight resolution in bits for GNA quantization: 8 or 16 (default 16).
-sf SCALE_FACTOR, --scale_factor SCALE_FACTOR
Optional. User-specified input scale factor for GNA quantization.
If the model contains multiple inputs, provide scale factors by separating them with commas.
For example: <layer1>:<sf1>,<layer2>:<sf2> or just <sf> to be applied to all inputs.
-wg EXPORT_GNA_MODEL, --export_gna_model EXPORT_GNA_MODEL
Optional. Write GNA model to file using path/filename provided.
-we EXPORT_EMBEDDED_GNA_MODEL, --export_embedded_gna_model EXPORT_EMBEDDED_GNA_MODEL
Optional. Write GNA embedded model to file using path/filename provided.
-we_gen [GNA1, GNA3], --embedded_gna_configuration [GNA1, GNA3]
Optional. GNA generation configuration string for embedded export. Can be GNA1 (default) or GNA3.
--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]
Optional. Specify GNA execution target generation. By default, generation corresponds to the GNA HW
available in the system or the latest fully supported generation by the software. See the GNA
Plugin's GNA_EXEC_TARGET config option description.
-pc, --performance_counter
Optional. Enables performance report (specify -a to ensure arch accurate results).
-a [CORE, ATOM], --arch [CORE, ATOM]
Optional. Specify architecture. CORE, ATOM with the combination of -pc.
-cw_l CONTEXT_WINDOW_LEFT, --context_window_left CONTEXT_WINDOW_LEFT
Optional. Number of frames for left context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-cw_r CONTEXT_WINDOW_RIGHT, --context_window_right CONTEXT_WINDOW_RIGHT
Optional. Number of frames for right context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-pwl_me PWL_ME Optional. The maximum percent of error for PWL function. The value must be in <0, 100> range. The
default value is 1.0.
.. _model-preparation-speech-python:
Model Preparation
#################
You can use the following model conversion command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
.. code-block:: sh
mo --framework kaldi --input_model wsj_dnn5b.nnet --counts wsj_dnn5b.counts --remove_output_softmax --output_dir <OUTPUT_MODEL_DIR>
The following pre-trained models are available:
- rm_cnn4a_smbr
- rm_lstm4f
- wsj_dnn5b_smbr
All of them can be downloaded from `the storage <https://storage.openvinotoolkit.org/models_contrib/speech/2021.2>`.
Speech Inference
################
You can do inference on Intel® Processors with the GNA co-processor (or emulation library):
.. code-block:: sh
python speech_sample.py -m wsj_dnn5b.xml -i dev93_10.ark -r dev93_scores_10.ark -d GNA_AUTO -o result.npz
.. note::
- Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using :doc:`model conversion API <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`.
- The sample supports input and output in numpy file format (.npz)
- Stating flags that take only single option like `-m` multiple times, for example `python classification_sample_async.py -m model.xml -m model2.xml`, results in only the last value being used.
Sample Output
#############
The sample application logs each step in a standard output stream.
.. code-block:: sh
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/wsj_dnn5b_smbr_fp32.xml
[ INFO ] Using scale factor(s) calculated from first utterance
[ INFO ] For input 0 using scale factor of 2175.4322418
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ]
[ INFO ] Utterance 0:
[ INFO ] Total time in Infer (HW and SW): 6326.06ms
[ INFO ] Frames in utterance: 1294
[ INFO ] Average Infer time per frame: 4.89ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7051840
[ INFO ] avg error: 0.0448388
[ INFO ] avg rms error: 0.0582387
[ INFO ] stdev error: 0.0371650
[ INFO ]
[ INFO ] Utterance 1:
[ INFO ] Total time in Infer (HW and SW): 4526.57ms
[ INFO ] Frames in utterance: 1005
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7575974
[ INFO ] avg error: 0.0452166
[ INFO ] avg rms error: 0.0586013
[ INFO ] stdev error: 0.0372769
[ INFO ]
[ INFO ] Utterance 2:
[ INFO ] Total time in Infer (HW and SW): 6636.56ms
[ INFO ] Frames in utterance: 1471
[ INFO ] Average Infer time per frame: 4.51ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7191710
[ INFO ] avg error: 0.0472226
[ INFO ] avg rms error: 0.0612991
[ INFO ] stdev error: 0.0390846
[ INFO ]
[ INFO ] Utterance 3:
[ INFO ] Total time in Infer (HW and SW): 3927.01ms
[ INFO ] Frames in utterance: 845
[ INFO ] Average Infer time per frame: 4.65ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7436461
[ INFO ] avg error: 0.0477581
[ INFO ] avg rms error: 0.0621334
[ INFO ] stdev error: 0.0397457
[ INFO ]
[ INFO ] Utterance 4:
[ INFO ] Total time in Infer (HW and SW): 3891.49ms
[ INFO ] Frames in utterance: 855
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7071600
[ INFO ] avg error: 0.0449147
[ INFO ] avg rms error: 0.0585048
[ INFO ] stdev error: 0.0374897
[ INFO ]
[ INFO ] Utterance 5:
[ INFO ] Total time in Infer (HW and SW): 3378.61ms
[ INFO ] Frames in utterance: 699
[ INFO ] Average Infer time per frame: 4.83ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.8870468
[ INFO ] avg error: 0.0479243
[ INFO ] avg rms error: 0.0625490
[ INFO ] stdev error: 0.0401951
[ INFO ]
[ INFO ] Utterance 6:
[ INFO ] Total time in Infer (HW and SW): 4034.31ms
[ INFO ] Frames in utterance: 790
[ INFO ] Average Infer time per frame: 5.11ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7648273
[ INFO ] avg error: 0.0482702
[ INFO ] avg rms error: 0.0629734
[ INFO ] stdev error: 0.0404429
[ INFO ]
[ INFO ] Utterance 7:
[ INFO ] Total time in Infer (HW and SW): 2854.04ms
[ INFO ] Frames in utterance: 622
[ INFO ] Average Infer time per frame: 4.59ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7389560
[ INFO ] avg error: 0.0465543
[ INFO ] avg rms error: 0.0604941
[ INFO ] stdev error: 0.0386294
[ INFO ]
[ INFO ] Utterance 8:
[ INFO ] Total time in Infer (HW and SW): 2493.28ms
[ INFO ] Frames in utterance: 548
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6680136
[ INFO ] avg error: 0.0439341
[ INFO ] avg rms error: 0.0574614
[ INFO ] stdev error: 0.0370353
[ INFO ]
[ INFO ] Utterance 9:
[ INFO ] Total time in Infer (HW and SW): 1654.67ms
[ INFO ] Frames in utterance: 368
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6550579
[ INFO ] avg error: 0.0467643
[ INFO ] avg rms error: 0.0605045
[ INFO ] stdev error: 0.0383914
[ INFO ]
[ INFO ] Total sample time: 39722.60ms
[ INFO ] File result.npz was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
See Also
########
- :doc:`Integrate the OpenVINO™ Runtime with Your Application <openvino_docs_OV_UG_Integrate_OV_with_your_application>`
- :doc:`Using OpenVINO™ Toolkit Samples <openvino_docs_OV_UG_Samples_Overview>`
- :doc:`Model Downloader <omz_tools_downloader>`
- :doc:`Convert a Model <openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide>`
@endsphinxdirective