IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[IE Python Sample] Update docs (#9807)

Browse Source 
* update hello_classification readme

* update classification_async readme

* update hello_query_device readme

* Fix hello_classification launch line

* Update hello_reshape_ssd readme

* update speech sample docs

* update ngraph sample docs

* fix launch command

* refactor py ngraph imports

* Replace `network` with `model`

* update example section with openvino-dev

* Update samples/python/classification_sample_async/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/classification_sample_async/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/hello_classification/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/hello_classification/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/hello_reshape_ssd/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/ngraph_function_creation_sample/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/ngraph_function_creation_sample/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/ngraph_function_creation_sample/README.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update samples/python/ngraph_function_creation_sample/README.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>

* Replace `Inference Engine` with `OpenVINO`

* fix ngraph ref

* Replace `Inference Engine` by `OpenVINO™ Runtime`

* Fix IR mentions

Co-authored-by: Vladimir Dudnik <vladimir.dudnik@intel.com>
Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
This commit is contained in:
Dmitry Pigasin
2022-02-14 19:03:45 +03:00
committed by GitHub
parent 310eb81403
commit 3a5d821219
12 changed files with 567 additions and 548 deletions
Download Patch File Download Diff File Expand all files Collapse all files

156
samples/python/classification_sample_async/README.md
View File

@@ -1,45 +1,43 @@
# Image Classification Async Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_classification_sample_async_README}
This sample demonstrates how to do inference of image classification networks using Asynchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Asynchronous Inference Request API.
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------------- | :-------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Asynchronous Infer | [InferRequest.async_infer], [InferRequest.wait], [Blob.buffer] | Do asynchronous inference |
| Custom Extension Kernels | [IECore.add_extension], [IECore.set_config] | Load extension library and config to the device |
| Feature | API | Description |
| :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------ |
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], [openvino.runtime.AsyncInferQueue.set_callback], [openvino.runtime.AsyncInferQueue.start_async], [openvino.runtime.AsyncInferQueue.wait_all], [openvino.runtime.InferRequest.results] | Do asynchronous inference |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/classification_sample_async/README.md) |
| Options | Values |
| :------------------------- | :----------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/classification_sample_async/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/classification_sample_async.py -h
python classification_sample_async.py -h
```
Usage message:
```
usage: classification_sample_async.py [-h] -m MODEL -i INPUT [INPUT ...]
[-l EXTENSION] [-c CONFIG] [-d DEVICE]
[--labels LABELS] [-nt NUMBER_TOP]
[-d DEVICE]
Options:
-h, --help Show this help message and exit.
@@ -48,22 +46,11 @@ Options:
model.
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Path to an image file(s).
-l EXTENSION, --extension EXTENSION
Optional. Required by the CPU Plugin for executing the
custom operation on a CPU. Absolute path to a shared
library with the kernels implementations.
-c CONFIG, --config CONFIG
Optional. Required by GPU or VPU Plugins for the
custom operation kernel. Absolute path to operation
description file (.xml).
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
```
To run the sample, you need specify a model and image:
@@ -73,28 +60,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` and `cat.jpg` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name alexnet
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/classification_sample_async.py -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp <path_to_image>/cat.jpg -d GPU
omz_converter --name alexnet
```
4. Perform inference of `banana.jpg` and `car.bmp` using the `alexnet` model on a `GPU`, for example:
```
python classification_sample_async.py -m alexnet.xml -i banana.jpg car.bmp -d GPU
```
## Sample Output
@@ -102,66 +96,52 @@ python <path_to_sample>/classification_sample_async.py -m <path_to_model>/alexne
The sample application logs each step in a standard output stream and outputs top-10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/alexnet.xml
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\car.bmp is resized from (637, 749) to (227, 227)
[ WARNING ] Image c:\images\cat.jpg is resized from (300, 300) to (227, 227)
[ INFO ] Starting inference in asynchronous mode
[ INFO ] Infer request 0 returned 0
[ INFO ] Image path: c:\images\car.bmp
[ INFO ] Image path: /test_data/images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 656 0.6645315
[ INFO ] 654 0.1121185
[ INFO ] 581 0.0698451
[ INFO ] 874 0.0334973
[ INFO ] 436 0.0259718
[ INFO ] 817 0.0173190
[ INFO ] 675 0.0109321
[ INFO ] 511 0.0109075
[ INFO ] 569 0.0083093
[ INFO ] 717 0.0063173
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9707602
[ INFO ] 666 0.0216788
[ INFO ] 659 0.0032558
[ INFO ] 435 0.0008082
[ INFO ] 809 0.0004359
[ INFO ] 502 0.0003860
[ INFO ] 618 0.0002867
[ INFO ] 910 0.0002866
[ INFO ] 951 0.0002410
[ INFO ] 961 0.0002193
[ INFO ]
[ INFO ] Infer request 1 returned 0
[ INFO ] Image path: c:\images\cat.jpg
[ INFO ] Image path: /test_data/images/car.bmp
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 876 0.1320105
[ INFO ] 435 0.1210389
[ INFO ] 285 0.0712640
[ INFO ] 282 0.0570528
[ INFO ] 281 0.0319335
[ INFO ] 999 0.0285931
[ INFO ] 94 0.0270323
[ INFO ] 36 0.0240510
[ INFO ] 335 0.0198461
[ INFO ] 186 0.0183939
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 656 0.5120340
[ INFO ] 874 0.1142275
[ INFO ] 654 0.0697167
[ INFO ] 436 0.0615163
[ INFO ] 581 0.0552262
[ INFO ] 705 0.0304179
[ INFO ] 675 0.0151660
[ INFO ] 734 0.0151582
[ INFO ] 627 0.0148493
[ INFO ] 757 0.0120964
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.add_extension]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a8a4b671a9928c7c059bd1e76d2333967
[IECore.set_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a2c738cee90fca27146e629825c039a05
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[InferRequest.async_infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InferRequest.html#a95ebe0368cdf4d5d64f9fddc8ee1cd0e
[InferRequest.wait]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InferRequest.html#a936fa50a7531e2f9a9e9c3d45afc9b43
<!-- TODO replace by python API link -->
[Blob.buffer]:https://docs.openvino.ai/latest/classInferenceEngine_1_1Blob.html#a0cad47b43204b115b4017b6b2564fa7e
<!-- [openvino.runtime.AsyncInferQueue]:
[openvino.runtime.AsyncInferQueue.set_callback]:
[openvino.runtime.AsyncInferQueue.start_async]:
[openvino.runtime.AsyncInferQueue.wait_all]:
[openvino.runtime.InferRequest.results]: -->

2
samples/python/classification_sample_async/classification_sample_async.py
View File

@@ -64,7 +64,7 @@ def main() -> int:
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {args.model}')
log.info(f'Reading the model: {args.model}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(args.model)

144
samples/python/hello_classification/README.md
View File

@@ -1,59 +1,35 @@
# Hello Classification Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_classification_README}
This sample demonstrates how to do inference of image classification networks using Synchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Synchronous Inference Request API.
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------- | :-------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Basic Infer Flow | [IECore], [IECore.read_network], [IECore.load_network] | Common API to do inference |
| Synchronous Infer | [ExecutableNetwork.infer] | Do synchronous inference |
| Network Operations | [IENetwork.input_info], [IENetwork.outputs], [InputInfoPtr.precision], [DataPtr.precision], [InputInfoPtr.input_data.shape] | Managing of network: configure input and output blobs |
| Feature | API | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Basic Infer Flow | [openvino.runtime.Core], [openvino.runtime.Core.read_model], [openvino.runtime.Core.compile_model] | Common API to do inference |
| Synchronous Infer | [openvino.runtime.CompiledModel.infer_new_request] | Do synchronous inference |
| Model Operations | [openvino.runtime.Model.inputs], [openvino.runtime.Model.outputs] | Managing of model |
| Preprocessing | [openvino.preprocess.PrePostProcessor], [openvino.preprocess.InputTensorInfo.set_element_type],[openvino.preprocess.InputTensorInfo.set_layout],[openvino.preprocess.InputTensorInfo.set_spatial_static_shape],[openvino.preprocess.PreProcessSteps.resize],[openvino.preprocess.InputModelInfo.set_layout],[openvino.preprocess.OutputTensorInfo.set_element_type],[openvino.preprocess.PrePostProcessor.build] | Set image of the original size as input for a model with other input size. Resize and layout conversions will be performed automatically by the corresponding plugin just before inference |
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Options | Values |
| :------------------------- | :------------------------------------------------------------------------------------------------------ |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_classification/README.md), [C](../../c/hello_classification/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/hello_classification.py -h
```
Usage message:
```
usage: hello_classification.py [-h] -m MODEL -i INPUT [-d DEVICE]
[--labels LABELS] [-nt NUMBER_TOP]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to an .xml or .onnx file with a trained
model.
-i INPUT, --input INPUT
Required. Path to an image file.
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
python hello_classification.py <path_to_model> <path_to_image> <device_name>
```
To run the sample, you need specify a model and image:
@@ -62,28 +38,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name alexnet
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/hello_classification.py -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp -d GPU
omz_converter --name alexnet
```
4. Perform inference of `banana.jpg` using the `alexnet` model on a `GPU`, for example:
```
python hello_classification.py alexnet.xml banana.jpg GPU
```
## Sample Output
@@ -91,43 +74,46 @@ python <path_to_sample>/hello_classification.py -m <path_to_model>/alexnet.xml -
The sample application logs each step in a standard output stream and outputs top-10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/alexnet/alexnet.xml
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\car.bmp is resized from (637, 749) to (227, 227)
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: c:\images\car.bmp
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 656 0.6645315
[ INFO ] 654 0.1121185
[ INFO ] 581 0.0698451
[ INFO ] 874 0.0334973
[ INFO ] 436 0.0259718
[ INFO ] 817 0.0173190
[ INFO ] 675 0.0109321
[ INFO ] 511 0.0109075
[ INFO ] 569 0.0083093
[ INFO ] 717 0.0063173
[ INFO ] Image path: /images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9703885
[ INFO ] 666 0.0219518
[ INFO ] 659 0.0033120
[ INFO ] 435 0.0008246
[ INFO ] 809 0.0004433
[ INFO ] 502 0.0003852
[ INFO ] 618 0.0002906
[ INFO ] 910 0.0002848
[ INFO ] 951 0.0002427
[ INFO ] 961 0.0002213
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- [openvino.runtime.Core]:
[openvino.runtime.Core.read_model]:
[openvino.runtime.Core.compile_model]:
[openvino.runtime.CompiledModel.infer_new_request]:
[openvino.runtime.Model.inputs]:
[openvino.runtime.Model.outputs]:
[openvino.preprocess.PrePostProcessor]:
[openvino.preprocess.InputTensorInfo.set_element_type]:
[openvino.preprocess.InputTensorInfo.set_layout]:
[openvino.preprocess.InputTensorInfo.set_spatial_static_shape]:
[openvino.preprocess.PreProcessSteps.resize]:
[openvino.preprocess.InputModelInfo.set_layout]:
[openvino.preprocess.OutputTensorInfo.set_element_type]:
[openvino.preprocess.PrePostProcessor.build]: -->

4
samples/python/hello_classification/hello_classification.py
View File

@@ -17,7 +17,7 @@ def main():
# Parsing and validation of input arguments
if len(sys.argv) != 4:
log.info('Usage: <path_to_model> <path_to_image> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <path_to_image> <device_name>')
return 1
model_path = sys.argv[1]
@@ -29,7 +29,7 @@ def main():
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {model_path}')
log.info(f'Reading the model: {model_path}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(model_path)

79
samples/python/hello_query_device/README.md
View File

@@ -1,37 +1,37 @@
# Hello Query Device Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_query_device_README}
This sample demonstrates how to show Inference Engine devices and prints their metrics and default configuration values using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
This sample demonstrates how to show OpenVINO™ Runtime devices and prints their metrics and default configuration values using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------- | :--------------------------------------- | :-------------------- |
| Basic | [IECore] | Common API |
| Query Device | [IECore.get_metric], [IECore.get_config] | Get device properties |
| Feature | API | Description |
| :----------- | :---------------------------------------------------------------------------------------------------------------- | :-------------------- |
| Basic | [openvino.runtime.Core] | Common API |
| Query Device | [openvino.runtime.Core.available_devices], [openvino.runtime.Core.get_metric], [openvino.runtime.Core.get_config] | Get device properties |
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------- |
| Options | Values |
| :------------------------- | :---------------------------------------------------------------- |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_query_device/README.md) |
| Other language realization | [C++](../../../samples/cpp/hello_query_device/README.md) |
## How It Works
The sample queries all available Inference Engine devices and prints their supported metrics and plugin configuration parameters.
The sample queries all available OpenVINO™ Runtime devices and prints their supported metrics and plugin configuration parameters.
## Running
The sample has no command-line parameters. To see the report, run the following command:
```
python <path_to_sample>/hello_query_device.py
python hello_query_device.py
```
## Sample Output
The application prints all available devices with their supported metrics and default values for configuration parameters. (Some lines are not shown due to length.) For example:
The application prints all available devices with their supported metrics and default values for configuration parameters.
For example:
```
[ INFO ] Creating Inference Engine
[ INFO ] Available devices:
[ INFO ] CPU :
[ INFO ] SUPPORTED_METRICS:
@@ -40,9 +40,11 @@ The application prints all available devices with their supported metrics and de
[ INFO ] OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1
[ INFO ] RANGE_FOR_STREAMS: 1, 8
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CPU_BIND_THREAD: NUMA
[ INFO ] CACHE_DIR:
[ INFO ] CPU_BIND_THREAD: NO
[ INFO ] CPU_THREADS_NUM: 0
[ INFO ] CPU_THROUGHPUT_STREAMS: 1
[ INFO ] DUMP_EXEC_GRAPH_AS_DOT:
@@ -50,6 +52,8 @@ The application prints all available devices with their supported metrics and de
[ INFO ] DYN_BATCH_LIMIT: 0
[ INFO ] ENFORCE_BF16: NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] PERFORMANCE_HINT:
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ] PERF_COUNT: NO
[ INFO ]
[ INFO ] GNA :
@@ -57,54 +61,33 @@ The application prints all available devices with their supported metrics and de
[ INFO ] AVAILABLE_DEVICES: GNA_SW
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ] FULL_DEVICE_NAME: GNA_SW
[ INFO ] GNA_LIBRARY_FULL_VERSION: 2.0.0.1047
[ INFO ] GNA_LIBRARY_FULL_VERSION: 3.0.0.1455
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] GNA_COMPACT_MODE: NO
[ INFO ] GNA_COMPACT_MODE: YES
[ INFO ] GNA_COMPILE_TARGET:
[ INFO ] GNA_DEVICE_MODE: GNA_SW_EXACT
[ INFO ] GNA_EXEC_TARGET:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE_GENERATION:
[ INFO ] GNA_LIB_N_THREADS: 1
[ INFO ] GNA_PRECISION: I16
[ INFO ] GNA_PWL_MAX_ERROR_PERCENT: 1.000000
[ INFO ] GNA_PWL_UNIFORM_DESIGN: NO
[ INFO ] GNA_SCALE_FACTOR: 1.000000
[ INFO ] GNA_SCALE_FACTOR_0: 1.000000
[ INFO ] LOG_LEVEL: LOG_NONE
[ INFO ] PERF_COUNT: NO
[ INFO ] SINGLE_THREAD: YES
[ INFO ]
[ INFO ] GPU :
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES: 0
[ INFO ] FULL_DEVICE_NAME: Intel(R) UHD Graphics 620 (iGPU)
[ INFO ] OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1
[ INFO ] RANGE_FOR_STREAMS: 1, 2
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CACHE_DIR:
[ INFO ] CLDNN_ENABLE_FP16_FOR_QUANTIZED_MODELS: YES
[ INFO ] CLDNN_GRAPH_DUMPS_DIR:
[ INFO ] CLDNN_MEM_POOL: YES
[ INFO ] CLDNN_NV12_TWO_INPUTS: NO
[ INFO ] CLDNN_PLUGIN_PRIORITY: 0
[ INFO ] CLDNN_PLUGIN_THROTTLE: 0
[ INFO ] CLDNN_SOURCES_DUMPS_DIR:
[ INFO ] CONFIG_FILE:
[ INFO ] DEVICE_ID:
[ INFO ] DUMP_KERNELS: NO
[ INFO ] DYN_BATCH_ENABLED: NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] GPU_THROUGHPUT_STREAMS: 1
[ INFO ] PERF_COUNT: NO
[ INFO ] TUNING_FILE:
[ INFO ] TUNING_MODE: TUNING_DISABLED
[ INFO ]
```
## See Also
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.get_metric]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#af1cdf2ecbea6399c556957c2c2fdf8eb
[IECore.get_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a48764dec7c235d2374af8b8ef53c6363
<!-- [openvino.runtime.Core]:
[openvino.runtime.Core.available_devices]:
[openvino.runtime.Core.get_metric]:
[openvino.runtime.Core.get_config]: -->

119
samples/python/hello_reshape_ssd/README.md
View File

@@ -1,67 +1,35 @@
# Hello Reshape SSD Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_reshape_ssd_README}
This sample demonstrates how to do synchronous inference of object detection networks using [Shape Inference feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
This sample demonstrates how to do synchronous inference of object detection models using [Shape Inference feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------------- | :-------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Network Operations | [IENetwork.reshape] | Managing of network: configure input and output blobs |
| Custom Extension Kernels | [IECore.add_extension], [IECore.set_config] | Load extension library and config to the device |
| Feature | API | Description |
| :--------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :---------------- |
| Model Operations | [openvino.runtime.Model.reshape], [openvino.runtime.Model.input], [openvino.runtime.Output.get_any_name], [openvino.runtime.PartialShape] | Managing of model |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------------------------- |
| Validated Models | [mobilenet-ssd](@ref omz_models_model_mobilenet_ssd) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_reshape_ssd/README.md) |
| Options | Values |
| :------------------------- | :----------------------------------------------------------------------- |
| Validated Models | [ssdlite_mobilenet_v2](@ref omz_models_model_ssdlite_mobilenet_v2) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_reshape_ssd/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the Inference Engine plugin, performs synchronous inference, and processes output data.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data.
As a result, the program creates an output image, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/hello_reshape_ssd.py -h
```
Usage message:
```
usage: hello_reshape_ssd.py [-h] -m MODEL -i INPUT [-l EXTENSION] [-c CONFIG]
[-d DEVICE] [--labels LABELS]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to an .xml or .onnx file with a trained
model.
-i INPUT, --input INPUT
Required. Path to an image file.
-l EXTENSION, --extension EXTENSION
Optional. Required by the CPU Plugin for executing the
custom operation on a CPU. Absolute path to a shared
library with the kernels implementations.
-c CONFIG, --config CONFIG
Optional. Required by GPU or VPU Plugins for the
custom operation kernel. Absolute path to operation
description file (.xml).
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
python hello_reshape_ssd.py <path_to_model> <path_to_image> <device_name>
```
To run the sample, you need specify a model and image:
@@ -70,28 +38,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name mobilenet-ssd
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name mobilenet-ssd
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `mobilenet-ssd` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name ssdlite_mobilenet_v2
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/hello_reshape_ssd.py -m <path_to_model>/mobilenet-ssd.xml -i <path_to_image>/car.bmp -d GPU
omz_converter --name ssdlite_mobilenet_v2
```
4. Perform inference of `banana.jpg` using `ssdlite_mobilenet_v2` model on a `GPU`, for example:
```
python hello_reshape_ssd.py ssdlite_mobilenet_v2.xml banana.jpg GPU
```
## Sample Output
@@ -99,34 +74,24 @@ python <path_to_sample>/hello_reshape_ssd.py -m <path_to_model>/mobilenet-ssd.xm
The sample application logs each step in a standard output stream and creates an output image, drawing bounding boxes for inference results with an over 50% confidence.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\mobilenet-ssd\FP32\mobilenet-ssd.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Reshaping the network to the height and width of the input image
[ INFO ] Input shape before reshape: [1, 3, 300, 300]
[ INFO ] Input shape after reshape: [1, 3, 637, 749]
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/ssdlite_mobilenet_v2.xml
[ INFO ] Reshaping the model to the height and width of the input image
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Found: label = 7, confidence = 0.99, coords = (283, 166), (541, 472)
[ INFO ] Found: class_id = 52, confidence = 0.98, coords = (21, 98), (276, 210)
[ INFO ] Image out.bmp was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.add_extension]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a8a4b671a9928c7c059bd1e76d2333967
[IECore.set_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a2c738cee90fca27146e629825c039a05
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[IENetwork.reshape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a6683f0291db25f908f8d6720ab2f221a
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- [openvino.runtime.Model.reshape]:
[openvino.runtime.Model.input]:
[openvino.runtime.Output.get_any_name]:
[openvino.runtime.PartialShape]: -->

6
samples/python/hello_reshape_ssd/hello_reshape_ssd.py
View File

@@ -18,7 +18,7 @@ def main():
# Parsing and validation of input arguments
if len(sys.argv) != 4:
log.info('Usage: <path_to_model> <path_to_image> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <path_to_image> <device_name>')
return 1
model_path = sys.argv[1]
@@ -30,7 +30,7 @@ def main():
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {model_path}')
log.info(f'Reading the model: {model_path}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(model_path)
@@ -48,7 +48,7 @@ def main():
# Add N dimension
input_tensor = np.expand_dims(image, 0)
log.info('Reshaping the network to the height and width of the input image')
log.info('Reshaping the model to the height and width of the input image')
n, h, w, c = input_tensor.shape
model.reshape({model.input().get_any_name(): PartialShape((n, c, h, w))})

196
samples/python/ngraph_function_creation_sample/README.md
View File

@@ -1,66 +1,43 @@
# nGraph Function Creation Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_ngraph_function_creation_sample_README}
This sample demonstrates how to execute an inference using [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) to create a network that uses weights from LeNet classification network, which is known to work well on digit classification tasks. So you don't need an XML file, the model will be created from the source code on the fly.
This sample demonstrates how to run inference using a [model](../../../docs/OV_Runtime_UG/model_representation.md) built on the fly that uses weights from the LeNet classification model, which is known to work well on digit classification tasks. You do not need an XML file, the model is created from the source code on the fly.
In addition to regular grayscale images with a digit, the sample also supports single-channel `ubyte` images as an input.
The following Python API is used in the application:
The following Inference Engine Python API is used in the application:
| Feature | API | Description |
| :--------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------ |
| Model Operations | [openvino.runtime.Model], [openvino.runtime.set_batch], [openvino.runtime.Model.input] | Managing of model |
| nGraph Functions | [openvino.runtime.op.Parameter], [openvino.runtime.op.Constant], [openvino.runtime.opset8.convolution], [openvino.runtime.opset8.add], [openvino.runtime.opset1.max_pool], [openvino.runtime.opset8.reshape], [openvino.runtime.opset8.matmul], [openvino.runtime.opset8.relu], [openvino.runtime.opset8.softmax] | Description of a model topology using nGraph Python API |
| Feature | API | Description |
| :----------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Network Operations | [IENetwork], [IENetwork.batch_size] | Managing of network |
| nGraph Functions | [ngraph.impl.Function], [ngraph.parameter], [ngraph.constant], [ngraph.convolution], [ngraph.add], [ngraph.max_pool], [ngraph.reshape], [ngraph.matmul], [ngraph.relu], [ngraph.softmax], [ngraph.result], ngraph.impl.Function.to_capsule | Description of a network using nGraph Python API |
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------- |
| Validated Models | LeNet |
| Model Format | Network weights file (\*.bin) |
| Validated images | The sample uses OpenCV\* to [read input grayscale image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png) or single-channel `ubyte` image |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------- |
| Validated Models | LeNet |
| Model Format | Model weights file (\*.bin) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/ngraph_function_creation_sample/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, creates a network using [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file, loads the network and image(s) to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application does the following:
- Reads command line parameters
- [Build a Model](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file
- Loads the model and input data to the OpenVINO™ Runtime plugin
- Performs synchronous inference and processes output data, logging each step in a standard output stream
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```sh
python <path_to_sample>/ngraph_function_creation_sample.py -h
```
Usage message:
To run the sample, you need to specify model weights and device.
```
usage: ngraph_function_creation_sample.py [-h] -m MODEL -i INPUT [INPUT ...]
[-d DEVICE] [--labels LABELS]
[-nt NUMBER_TOP]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to a file with network weights.
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Path to an image file.
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
python ngraph_function_creation_sample.py <path_to_model> <device_name>
```
To run the sample, you need specify a model weights and image. You can use images from the media files collection available at https://storage.openvinotoolkit.org/data/test_data.
> **NOTE**:
>
> - This sample supports models with FP32 weights only.
@@ -68,73 +45,102 @@ To run the sample, you need specify a model weights and image. You can use image
> - The `lenet.bin` weights file was generated by the [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) tool from the public LeNet model with the `--input_shape [64,1,28,28]` parameter specified.
>
> - The original model is available in the [Caffe* repository](https://github.com/BVLC/caffe/tree/master/examples/mnist) on GitHub\*.
>
> - The white over black images will be automatically inverted in color for a better predictions.
For example, you can do inference of [3.png](https://storage.openvinotoolkit.org/data/test_data/images/3.png) using the pre-trained model on a `GPU`:
For example:
```sh
python <path_to_sample>/ngraph_function_creation_sample.py -m <path_to_sample>/lenet.bin -i <path_to_image>/3.png -d GPU
```
python ngraph_function_creation_sample.py lenet.bin GPU
```
## Sample Output
The sample application logs each step in a standard output stream and outputs top-10 inference results.
The sample application logs each step in a standard output stream and outputs 10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Loading the network using ngraph function with weights from c:\openvino\samples\python\ngraph_function_creation_sample\lenet.bin
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Loading the model using ngraph function with weights from lenet.bin
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\3.png is inverted to white over black
[ WARNING ] Image c:\images\3.png is resized from (351, 353) to (28, 28)
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: c:\images\3.png
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 3 1.0000000
[ INFO ] 9 0.0000000
[ INFO ] 8 0.0000000
[ INFO ] 7 0.0000000
[ INFO ] 6 0.0000000
[ INFO ] 5 0.0000000
[ INFO ] 4 0.0000000
[ INFO ] 2 0.0000000
[ INFO ] 1 0.0000000
[ INFO ] 0 0.0000000
[ INFO ] Top 1 results:
[ INFO ] Image 0
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 0 1.0000000 0
[ INFO ]
[ INFO ] Image 1
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 1 1.0000000 1
[ INFO ]
[ INFO ] Image 2
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 2 1.0000000 2
[ INFO ]
[ INFO ] Image 3
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 3 1.0000000 3
[ INFO ]
[ INFO ] Image 4
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 4 1.0000000 4
[ INFO ]
[ INFO ] Image 5
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 5 1.0000000 5
[ INFO ]
[ INFO ] Image 6
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 6 1.0000000 6
[ INFO ]
[ INFO ] Image 7
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 7 1.0000000 7
[ INFO ]
[ INFO ] Image 8
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 8 1.0000000 8
[ INFO ]
[ INFO ] Image 9
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 9 1.0000000 9
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IENetwork]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html
[IENetwork.batch_size]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a79a647cb1b49645616eaeb2ca255ef2e
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- TODO: Replace the link by another one pointing to the Python API, if available -->
[ngraph.impl.Function]:https://docs.openvino.ai/latest/ngraph_cpp_api/classngraph_1_1Function.html
<!-- [ngraph.impl.Function.to_capsule]: -->
[ngraph.parameter]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a709acd09288f5a76ed8d07492efc3d13
[ngraph.constant]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a5b6c4e416026e007a4107b3f510d0c27
[ngraph.convolution]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a3143ff55f68428afc1b6c802ee9381e8
[ngraph.add]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#abfa0373c10ced1b1f129594d9bd8a159
[ngraph.max_pool]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#ac60b4459ad23b296086925abce6acd2d
[ngraph.reshape]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a38e1ead9435c4b75c1d891ba2dd6a62e
[ngraph.matmul]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a403b5e10e1f75aeb7569024237e85071
[ngraph.relu]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a70b9b3faf58d85e43d27fef5028117e3
[ngraph.softmax]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a632cc9a31ecaefa2a982d039ecad8d26
[ngraph.result]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a94f8bf6ab8910dfd461d09cb6c6edd11
<!-- [openvino.runtime.Model]:
[openvino.runtime.set_batch]:
[openvino.runtime.Model.input]:
[openvino.runtime.op.Parameter]:
[openvino.runtime.op.Constant]:
[openvino.runtime.opset8.convolution]:
[openvino.runtime.opset8.add]:
[openvino.runtime.opset1.max_pool]:
[openvino.runtime.opset8.reshape]:
[openvino.runtime.opset8.matmul]:
[openvino.runtime.opset8.relu]:
[openvino.runtime.opset8.softmax]: -->

90
samples/python/ngraph_function_creation_sample/ngraph_function_creation_sample.py
View File

@@ -9,13 +9,14 @@ from functools import reduce
import numpy as np
from openvino.preprocess import PrePostProcessor
from openvino.runtime import Core, Layout, Type, Model, Shape, PartialShape
import openvino
from openvino.runtime import (Core, Layout, Model, Shape, Type, op, opset1,
opset8, set_batch)
from data import digits
def create_ngraph_function(model_path: str) -> Model:
"""Create a network on the fly from the source code using ngraph"""
"""Create a model on the fly from the source code using ngraph"""
def shape_and_length(shape: list) -> typing.Tuple[list, int]:
length = reduce(lambda x, y: x * y, shape)
@@ -27,42 +28,42 @@ def create_ngraph_function(model_path: str) -> Model:
# input
input_shape = [64, 1, 28, 28]
param_node = openvino.runtime.op.Parameter(Type.f32, Shape(input_shape))
param_node = op.Parameter(Type.f32, Shape(input_shape))
# convolution 1
conv_1_kernel_shape, conv_1_kernel_length = shape_and_length([20, 1, 5, 5])
conv_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(conv_1_kernel_shape), weights[0:conv_1_kernel_length].tolist())
conv_1_kernel = op.Constant(Type.f32, Shape(conv_1_kernel_shape), weights[0:conv_1_kernel_length].tolist())
weights_offset += conv_1_kernel_length
conv_1_node = openvino.runtime.opset8.convolution(param_node, conv_1_kernel, [1, 1], padding_begin, padding_end, [1, 1])
conv_1_node = opset8.convolution(param_node, conv_1_kernel, [1, 1], padding_begin, padding_end, [1, 1])
# add 1
add_1_kernel_shape, add_1_kernel_length = shape_and_length([1, 20, 1, 1])
add_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_1_kernel_shape),
weights[weights_offset : weights_offset + add_1_kernel_length])
add_1_kernel = op.Constant(Type.f32, Shape(add_1_kernel_shape),
weights[weights_offset : weights_offset + add_1_kernel_length])
weights_offset += add_1_kernel_length
add_1_node = openvino.runtime.opset8.add(conv_1_node, add_1_kernel)
add_1_node = opset8.add(conv_1_node, add_1_kernel)
# maxpool 1
maxpool_1_node = openvino.runtime.opset1.max_pool(add_1_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
maxpool_1_node = opset1.max_pool(add_1_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
# convolution 2
conv_2_kernel_shape, conv_2_kernel_length = shape_and_length([50, 20, 5, 5])
conv_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(conv_2_kernel_shape),
weights[weights_offset : weights_offset + conv_2_kernel_length],
)
conv_2_kernel = op.Constant(Type.f32, Shape(conv_2_kernel_shape),
weights[weights_offset : weights_offset + conv_2_kernel_length],
)
weights_offset += conv_2_kernel_length
conv_2_node = openvino.runtime.opset8.convolution(maxpool_1_node, conv_2_kernel, [1, 1], padding_begin, padding_end, [1, 1])
conv_2_node = opset8.convolution(maxpool_1_node, conv_2_kernel, [1, 1], padding_begin, padding_end, [1, 1])
# add 2
add_2_kernel_shape, add_2_kernel_length = shape_and_length([1, 50, 1, 1])
add_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_2_kernel_shape),
weights[weights_offset : weights_offset + add_2_kernel_length],
)
add_2_kernel = op.Constant(Type.f32, Shape(add_2_kernel_shape),
weights[weights_offset : weights_offset + add_2_kernel_length],
)
weights_offset += add_2_kernel_length
add_2_node = openvino.runtime.opset8.add(conv_2_node, add_2_kernel)
add_2_node = opset8.add(conv_2_node, add_2_kernel)
# maxpool 2
maxpool_2_node = openvino.runtime.opset1.max_pool(add_2_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
maxpool_2_node = opset1.max_pool(add_2_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
# reshape 1
reshape_1_dims, reshape_1_length = shape_and_length([2])
@@ -71,52 +72,52 @@ def create_ngraph_function(model_path: str) -> Model:
weights[weights_offset : weights_offset + 2 * reshape_1_length],
dtype=np.int64,
)
reshape_1_kernel = openvino.runtime.op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_1_kernel = op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
weights_offset += 2 * reshape_1_length
reshape_1_node = openvino.runtime.opset8.reshape(maxpool_2_node, reshape_1_kernel, True)
reshape_1_node = opset8.reshape(maxpool_2_node, reshape_1_kernel, True)
# matmul 1
matmul_1_kernel_shape, matmul_1_kernel_length = shape_and_length([500, 800])
matmul_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(matmul_1_kernel_shape),
weights[weights_offset : weights_offset + matmul_1_kernel_length],
)
matmul_1_kernel = op.Constant(Type.f32, Shape(matmul_1_kernel_shape),
weights[weights_offset : weights_offset + matmul_1_kernel_length],
)
weights_offset += matmul_1_kernel_length
matmul_1_node = openvino.runtime.opset8.matmul(reshape_1_node, matmul_1_kernel, False, True)
matmul_1_node = opset8.matmul(reshape_1_node, matmul_1_kernel, False, True)
# add 3
add_3_kernel_shape, add_3_kernel_length = shape_and_length([1, 500])
add_3_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_3_kernel_shape),
weights[weights_offset : weights_offset + add_3_kernel_length],
)
add_3_kernel = op.Constant(Type.f32, Shape(add_3_kernel_shape),
weights[weights_offset : weights_offset + add_3_kernel_length],
)
weights_offset += add_3_kernel_length
add_3_node = openvino.runtime.opset8.add(matmul_1_node, add_3_kernel)
add_3_node = opset8.add(matmul_1_node, add_3_kernel)
# ReLU
relu_node = openvino.runtime.opset8.relu(add_3_node)
relu_node = opset8.relu(add_3_node)
# reshape 2
reshape_2_kernel = openvino.runtime.op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_2_node = openvino.runtime.opset8.reshape(relu_node, reshape_2_kernel, True)
reshape_2_kernel = op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_2_node = opset8.reshape(relu_node, reshape_2_kernel, True)
# matmul 2
matmul_2_kernel_shape, matmul_2_kernel_length = shape_and_length([10, 500])
matmul_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(matmul_2_kernel_shape),
weights[weights_offset : weights_offset + matmul_2_kernel_length],
)
matmul_2_kernel = op.Constant(Type.f32, Shape(matmul_2_kernel_shape),
weights[weights_offset : weights_offset + matmul_2_kernel_length],
)
weights_offset += matmul_2_kernel_length
matmul_2_node = openvino.runtime.opset8.matmul(reshape_2_node, matmul_2_kernel, False, True)
matmul_2_node = opset8.matmul(reshape_2_node, matmul_2_kernel, False, True)
# add 4
add_4_kernel_shape, add_4_kernel_length = shape_and_length([1, 10])
add_4_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_4_kernel_shape),
weights[weights_offset : weights_offset + add_4_kernel_length],
)
add_4_kernel = op.Constant(Type.f32, Shape(add_4_kernel_shape),
weights[weights_offset : weights_offset + add_4_kernel_length],
)
weights_offset += add_4_kernel_length
add_4_node = openvino.runtime.opset8.add(matmul_2_node, add_4_kernel)
add_4_node = opset8.add(matmul_2_node, add_4_kernel)
# softmax
softmax_axis = 1
softmax_node = openvino.runtime.opset8.softmax(add_4_node, softmax_axis)
softmax_node = opset8.softmax(add_4_node, softmax_axis)
return Model(softmax_node, [param_node], 'lenet')
@@ -125,7 +126,7 @@ def main():
log.basicConfig(format='[ %(levelname)s ] %(message)s', level=log.INFO, stream=sys.stdout)
# Parsing and validation of input arguments
if len(sys.argv) != 3:
log.info('Usage: <path_to_model> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <device_name>')
return 1
model_path = sys.argv[1]
@@ -137,7 +138,7 @@ def main():
core = Core()
# ---------------------------Step 2. Read a model in OpenVINO Intermediate Representation------------------------------
log.info(f'Loading the network using ngraph function with weights from {model_path}')
log.info(f'Loading the model using ngraph function with weights from {model_path}')
model = create_ngraph_function(model_path)
# ---------------------------Step 3. Apply preprocessing----------------------------------------------------------
# Get names of input and output blobs
@@ -160,8 +161,7 @@ def main():
model = ppp.build()
# Set a batch size equal to number of input images
model.reshape({model.input().get_any_name(): PartialShape((digits.shape[0], model.input().shape[1], model.input().shape[2], model.input().shape[3]))})
set_batch(model, digits.shape[0])
# ---------------------------Step 4. Loading model to the device-------------------------------------------------------
log.info('Loading the model to the plugin')
compiled_model = core.compile_model(model, device_name)

307
samples/python/speech_sample/README.md
View File

@@ -1,48 +1,50 @@
# Automatic Speech Recognition Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_speech_sample_README}
This sample demonstrates how to do a Synchronous Inference of acoustic model based on Kaldi\* neural networks and speech feature vectors.
This sample demonstrates how to do a Synchronous Inference of acoustic model based on Kaldi\* neural models and speech feature vectors.
The sample works with Kaldi ARK or Numpy* uncompressed NPZ files, so it does not cover an end-to-end speech recognition scenario (speech to text), requiring additional preprocessing (feature extraction) to get a feature vector from a speech signal, as well as postprocessing (decoding) to produce text from scores.
Automatic Speech Recognition Python sample application demonstrates how to use the following Inference Engine Python API in applications:
Automatic Speech Recognition Python sample application demonstrates how to use the following Python API in applications:
| Feature | API | Description |
| :------------------ | :---------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| Import/Export Model | [IECore.import_network], [ExecutableNetwork.export] | The GNA plugin supports loading and saving of the GNA-optimized model |
| Network Operations | [IENetwork.batch_size], [CDataPtr.shape], [ExecutableNetwork.input_info], [ExecutableNetwork.outputs] | Managing of network: configure input and output blobs |
| Network Operations | [IENetwork.add_outputs] | Managing of network: Change names of output layers in the network |
| InferRequest Operations|InferRequest.query_state, VariableState.reset| Gets and resets state control interface for given executable network |
| Feature | API | Description |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| Import/Export Model | [openvino.runtime.Core.import_model], [openvino.runtime.CompiledModel.export_model] | The GNA plugin supports loading and saving of the GNA-optimized model |
| Model Operations | [openvino.runtime.Model.add_outputs], [openvino.runtime.set_batch], [openvino.runtime.CompiledModel.inputs], [openvino.runtime.CompiledModel.outputs], [openvino.runtime.ConstOutput.any_name] | Managing of model: configure batch_size, input and output tensors |
| Synchronous Infer | [openvino.runtime.CompiledModel.create_infer_request], [openvino.runtime.InferRequest.infer] | Do synchronous inference |
| InferRequest Operations | [openvino.runtime.InferRequest.get_input_tensor], [openvino.runtime.InferRequest.model_outputs], [openvino.runtime.InferRequest.model_inputs], | Get info about model using infer request API |
| InferRequest Operations | [openvino.runtime.InferRequest.query_state], [openvino.runtime.VariableState.reset] | Gets and resets CompiledModel state control |
| Profiling | [openvino.runtime.InferRequest.profiling_info], [openvino.runtime.ProfilingInfo.real_time] | Get infer request profiling info |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------------------------------------- |
| Validated Models | Acoustic model based on Kaldi* neural networks (see [Model Preparation](#model-preparation) section) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin) |
| Options | Values |
| :------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| Validated Models | Acoustic model based on Kaldi* neural models (see [Model Preparation](#model-preparation) section) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin) |
| Supported devices | See [Execution Modes](#execution-modes) section below and [List Supported Devices](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/speech_sample/README.md) |
| Other language realization | [C++](../../../samples/cpp/speech_sample/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, loads a specified model and input data to the Inference Engine plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, loads a specified model and input data to the OpenVINO™ Runtime plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## GNA-specific details
### Quantization
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA Inference Engine plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA OpenVINO™ Runtime plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
Several neural network quantization modes:
Several neural model quantization modes:
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural network is quantized to accommodate the scaled input dynamic range.
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural model is quantized to accommodate the scaled input dynamic range.
- *user-defined* - The user may specify a scale factor via the `-sf` flag that will be used for static quantization.
The `-qb` flag provides a hint to the GNA plugin regarding the preferred target weight resolution for all layers.
For example, when `-qb 8` is specified, the plugin will use 8-bit weights wherever possible in the
network.
model.
> **NOTE**:
>
@@ -79,29 +81,23 @@ In addition to performing inference directly from a GNA model file, these option
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/speech_sample.py -h
python speech_sample.py -h
```
Usage message:
```
usage: speech_sample.py [-h] (-m MODEL | -rg IMPORT_GNA_MODEL) -i INPUT
[-o OUTPUT] [-r REFERENCE] [-d DEVICE] [-bs [1-8]]
[-qb [8, 16]] [-sf SCALE_FACTOR]
[-wg EXPORT_GNA_MODEL] [-we EXPORT_EMBEDDED_GNA_MODEL]
[-we_gen [GNA1, GNA3]]
[--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]] [-pc]
[-a [CORE, ATOM]] [-iname INPUT_LAYERS]
[-oname OUTPUT_LAYERS] [-cw_l CONTEXT_WINDOW_LEFT]
[-cw_r CONTEXT_WINDOW_RIGHT]
usage: speech_sample.py [-h] (-m MODEL | -rg IMPORT_GNA_MODEL) -i INPUT [-o OUTPUT] [-r REFERENCE] [-d DEVICE] [-bs [1-8]]
[-qb [8, 16]] [-sf SCALE_FACTOR] [-wg EXPORT_GNA_MODEL] [-we EXPORT_EMBEDDED_GNA_MODEL]
[-we_gen [GNA1, GNA3]] [--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]] [-pc] [-a [CORE, ATOM]]
[-iname INPUT_LAYERS] [-oname OUTPUT_LAYERS] [-cw_l CONTEXT_WINDOW_LEFT] [-cw_r CONTEXT_WINDOW_RIGHT]
[-pwl_me PWL_ME]
optional arguments:
-m MODEL, --model MODEL
Path to an .xml file with a trained model (required if
-rg is missing).
Path to an .xml file with a trained model (required if -rg is missing).
-rg IMPORT_GNA_MODEL, --import_gna_model IMPORT_GNA_MODEL
Read GNA model from file using path/filename provided
(required if -m is missing).
Read GNA model from file using path/filename provided (required if -m is missing).
Options:
-h, --help Show this help message and exit.
@@ -110,70 +106,52 @@ Options:
-o OUTPUT, --output OUTPUT
Optional. Output file name to save inference results (.ark or .npz).
-r REFERENCE, --reference REFERENCE
Optional. Read reference score file and compare
scores.
Optional. Read reference score file and compare scores.
-d DEVICE, --device DEVICE
Optional. Specify a target device to infer on. CPU,
GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32,
GNA_SW_EXACT and HETERO with combination of GNA as the
primary device and CPU as a secondary (e.g.
HETERO:GNA,CPU) are supported. The sample will look
for a suitable plugin for device specified. Default
Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32,
GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g.
HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified. Default
value is CPU.
-bs [1-8], --batch_size [1-8]
Optional. Batch size 1-8 (default 1).
-qb [8, 16], --quantization_bits [8, 16]
Optional. Weight bits for quantization: 8 or 16
(default 16).
Optional. Weight bits for quantization: 8 or 16 (default 16).
-sf SCALE_FACTOR, --scale_factor SCALE_FACTOR
Optional. The user-specified input scale factor for
quantization. If the network contains multiple inputs,
provide scale factors by separating them with commas.
Optional. The user-specified input scale factor for quantization. If the model contains multiple
inputs, provide scale factors by separating them with commas.
-wg EXPORT_GNA_MODEL, --export_gna_model EXPORT_GNA_MODEL
Optional. Write GNA model to file using path/filename
provided.
Optional. Write GNA model to file using path/filename provided.
-we EXPORT_EMBEDDED_GNA_MODEL, --export_embedded_gna_model EXPORT_EMBEDDED_GNA_MODEL
Optional. Write GNA embedded model to file using
path/filename provided.
Optional. Write GNA embedded model to file using path/filename provided.
-we_gen [GNA1, GNA3], --embedded_gna_configuration [GNA1, GNA3]
Optional. GNA generation configuration string for
embedded export. Can be GNA1 (default) or GNA3.
Optional. GNA generation configuration string for embedded export. Can be GNA1 (default) or GNA3.
--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]
Optional. Specify GNA execution target generation. By
default, generation corresponds to the GNA HW
available in the system or the latest fully supported
generation by the software. See the GNA Plugin's
Optional. Specify GNA execution target generation. By default, generation corresponds to the GNA HW
available in the system or the latest fully supported generation by the software. See the GNA Plugin's
GNA_EXEC_TARGET config option description.
-pc, --performance_counter
Optional. Enables performance report (specify -a to
ensure arch accurate results).
Optional. Enables performance report (specify -a to ensure arch accurate results).
-a [CORE, ATOM], --arch [CORE, ATOM]
Optional. Specify architecture. CORE, ATOM with the
combination of -pc.
Optional. Specify architecture. CORE, ATOM with the combination of -pc.
-iname INPUT_LAYERS, --input_layers INPUT_LAYERS
Optional. Layer names for input blobs. The names are
separated with ",". Allows to change the order of
Optional. Layer names for input blobs. The names are separated with ",". Allows to change the order of
input layers for -i flag. Example: Input1,Input2
-oname OUTPUT_LAYERS, --output_layers OUTPUT_LAYERS
Optional. Layer names for output blobs. The names are
separated with ",". Allows to change the order of
output layers for -o flag. Example:
Output1:port,Output2:port.
Optional. Layer names for output blobs. The names are separated with ",". Allows to change the order of
output layers for -o flag. Example: Output1:port,Output2:port.
-cw_l CONTEXT_WINDOW_LEFT, --context_window_left CONTEXT_WINDOW_LEFT
Optional. Number of frames for left context windows
(default is 0). Works only with context window
networks. If you use the cw_l or cw_r flag, then batch
size argument is ignored.
Optional. Number of frames for left context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-cw_r CONTEXT_WINDOW_RIGHT, --context_window_right CONTEXT_WINDOW_RIGHT
Optional. Number of frames for right context windows
(default is 0). Works only with context window
networks. If you use the cw_l or cw_r flag, then batch
size argument is ignored.
Optional. Number of frames for right context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-pwl_me PWL_ME Optional. The maximum percent of error for PWL function. The value must be in <0, 100> range. The
default value is 1.0.
```
## Model Preparation
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural network to Inference Engine Intermediate Representation format:
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
```sh
mo --framework kaldi --input_model wsj_dnn5b.nnet --counts wsj_dnn5b.counts --remove_output_softmax --output_dir <OUTPUT_MODEL_DIR>
@@ -192,12 +170,12 @@ All of them can be downloaded from [https://storage.openvinotoolkit.org/models_c
You can do inference on Intel® Processors with the GNA co-processor (or emulation library):
```
python <path_to_sample>/speech_sample.py -m <path_to_model>/wsj_dnn5b.xml -i <path_to_ark>/dev93_10.ark -r <path_to_ark>/dev93_scores_10.ark -d GNA_AUTO -o result.npz
python speech_sample.py -m wsj_dnn5b.xml -i dev93_10.ark -r dev93_scores_10.ark -d GNA_AUTO -o result.npz
```
> **NOTES**:
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample supports input and output in numpy file format (.npz)
@@ -206,47 +184,168 @@ python <path_to_sample>/speech_sample.py -m <path_to_model>/wsj_dnn5b.xml -i <pa
The sample application logs each step in a standard output stream.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: wsj_dnn5b.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/wsj_dnn5b_smbr_fp32.xml
[ INFO ] Using scale factor(s) calculated from first utterance
[ INFO ] For input 0 using scale factor of 2175.4322418
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Utterance 0 (4k0c0301)
[ INFO ] Output blob name: affinetransform14/Fused_Add_
[ INFO ]
[ INFO ] Utterance 0:
[ INFO ] Total time in Infer (HW and SW): 6326.06ms
[ INFO ] Frames in utterance: 1294
[ INFO ] Total time in Infer (HW and SW): 6211.45ms
[ INFO ] max error: 0.7051840
[ INFO ] avg error: 0.0448388
[ INFO ] avg rms error: 0.0582387
[ INFO ] stdev error: 0.0371650
[ INFO ] Average Infer time per frame: 4.89ms
[ INFO ]
[ INFO ] Utterance 1 (4k0c0302)
[ INFO ] Output blob name: affinetransform14/Fused_Add_
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7051840
[ INFO ] avg error: 0.0448388
[ INFO ] avg rms error: 0.0582387
[ INFO ] stdev error: 0.0371650
[ INFO ]
[ INFO ] Utterance 1:
[ INFO ] Total time in Infer (HW and SW): 4526.57ms
[ INFO ] Frames in utterance: 1005
[ INFO ] Total time in Infer (HW and SW): 4742.27ms
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7575974
[ INFO ] avg error: 0.0452166
[ INFO ] avg error: 0.0452166
[ INFO ] avg rms error: 0.0586013
[ INFO ] stdev error: 0.0372769
...
[ INFO ] Total sample time: 40219.99ms
[ INFO ] stdev error: 0.0372769
[ INFO ]
[ INFO ] Utterance 2:
[ INFO ] Total time in Infer (HW and SW): 6636.56ms
[ INFO ] Frames in utterance: 1471
[ INFO ] Average Infer time per frame: 4.51ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7191710
[ INFO ] avg error: 0.0472226
[ INFO ] avg rms error: 0.0612991
[ INFO ] stdev error: 0.0390846
[ INFO ]
[ INFO ] Utterance 3:
[ INFO ] Total time in Infer (HW and SW): 3927.01ms
[ INFO ] Frames in utterance: 845
[ INFO ] Average Infer time per frame: 4.65ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7436461
[ INFO ] avg error: 0.0477581
[ INFO ] avg rms error: 0.0621334
[ INFO ] stdev error: 0.0397457
[ INFO ]
[ INFO ] Utterance 4:
[ INFO ] Total time in Infer (HW and SW): 3891.49ms
[ INFO ] Frames in utterance: 855
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7071600
[ INFO ] avg error: 0.0449147
[ INFO ] avg rms error: 0.0585048
[ INFO ] stdev error: 0.0374897
[ INFO ]
[ INFO ] Utterance 5:
[ INFO ] Total time in Infer (HW and SW): 3378.61ms
[ INFO ] Frames in utterance: 699
[ INFO ] Average Infer time per frame: 4.83ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.8870468
[ INFO ] avg error: 0.0479243
[ INFO ] avg rms error: 0.0625490
[ INFO ] stdev error: 0.0401951
[ INFO ]
[ INFO ] Utterance 6:
[ INFO ] Total time in Infer (HW and SW): 4034.31ms
[ INFO ] Frames in utterance: 790
[ INFO ] Average Infer time per frame: 5.11ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7648273
[ INFO ] avg error: 0.0482702
[ INFO ] avg rms error: 0.0629734
[ INFO ] stdev error: 0.0404429
[ INFO ]
[ INFO ] Utterance 7:
[ INFO ] Total time in Infer (HW and SW): 2854.04ms
[ INFO ] Frames in utterance: 622
[ INFO ] Average Infer time per frame: 4.59ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7389560
[ INFO ] avg error: 0.0465543
[ INFO ] avg rms error: 0.0604941
[ INFO ] stdev error: 0.0386294
[ INFO ]
[ INFO ] Utterance 8:
[ INFO ] Total time in Infer (HW and SW): 2493.28ms
[ INFO ] Frames in utterance: 548
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6680136
[ INFO ] avg error: 0.0439341
[ INFO ] avg rms error: 0.0574614
[ INFO ] stdev error: 0.0370353
[ INFO ]
[ INFO ] Utterance 9:
[ INFO ] Total time in Infer (HW and SW): 1654.67ms
[ INFO ] Frames in utterance: 368
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6550579
[ INFO ] avg error: 0.0467643
[ INFO ] avg rms error: 0.0605045
[ INFO ] stdev error: 0.0383914
[ INFO ]
[ INFO ] Total sample time: 39722.60ms
[ INFO ] File result.npz was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IENetwork.batch_size]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a79a647cb1b49645616eaeb2ca255ef2e
[IENetwork.add_outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#ae8024b07f3301d6d5de5c0d153e2e6e6
[CDataPtr.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1CDataPtr.html#aa6fd459edb323d1c6215dc7a970ebf7f
[ExecutableNetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#ac76a04c2918607874018d2e15a2f274f
[ExecutableNetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#a4a631776df195004b1523e6ae91a65c1
[IECore.import_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#afdeac5192bb1d9e64722f1071fb0a64a
[ExecutableNetwork.export]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#afa78158252f0d8070181bafec4318413
<!-- [openvino.runtime.Core.import_model]:
[openvino.runtime.CompiledModel.export_model]:
[openvino.runtime.Model.add_outputs]:
[openvino.runtime.set_batch]:
[openvino.runtime.CompiledModel.inputs]:
[openvino.runtime.CompiledModel.outputs]:
[openvino.runtime.ConstOutput.any_name]:
[openvino.runtime.CompiledModel.create_infer_request]:
[openvino.runtime.InferRequest.infer]:
[openvino.runtime.InferRequest.get_input_tensor]:
[openvino.runtime.InferRequest.model_outputs]:
[openvino.runtime.InferRequest.model_inputs]:
[openvino.runtime.InferRequest.query_state]:
[openvino.runtime.VariableState.reset]:
[openvino.runtime.InferRequest.profiling_info]:
[openvino.runtime.ProfilingInfo.real_time]: -->

6
samples/python/speech_sample/arg_parser.py
View File

@@ -33,7 +33,7 @@ def build_arg_parser() -> argparse.ArgumentParser:
help='Optional. Weight bits for quantization: 8 or 16 (default 16).')
args.add_argument('-sf', '--scale_factor', type=str,
help='Optional. The user-specified input scale factor for quantization. '
'If the network contains multiple inputs, provide scale factors by separating them with commas.')
'If the model contains multiple inputs, provide scale factors by separating them with commas.')
args.add_argument('-wg', '--export_gna_model', type=str,
help='Optional. Write GNA model to file using path/filename provided.')
args.add_argument('-we', '--export_embedded_gna_model', type=str,
@@ -59,11 +59,11 @@ def build_arg_parser() -> argparse.ArgumentParser:
'Allows to change the order of output layers for -o flag. Example: Output1:port,Output2:port.')
args.add_argument('-cw_l', '--context_window_left', type=int, default=0,
help='Optional. Number of frames for left context windows (default is 0). '
'Works only with context window networks. '
'Works only with context window models. '
'If you use the cw_l or cw_r flag, then batch size argument is ignored.')
args.add_argument('-cw_r', '--context_window_right', type=int, default=0,
help='Optional. Number of frames for right context windows (default is 0). '
'Works only with context window networks. '
'Works only with context window models. '
'If you use the cw_l or cw_r flag, then batch size argument is ignored.')
args.add_argument('-pwl_me', type=float, default=1.0,
help='Optional. The maximum percent of error for PWL function. '

6
samples/python/speech_sample/speech_sample.py
View File

@@ -183,7 +183,7 @@ def main():
input_file_names = re.split(', |,', args.input)
if len(input_layer_names) != len(input_file_names):
log.error(f'Number of network inputs ({len(compiled_model.inputs)}) is not equal '
log.error(f'Number of model inputs ({len(compiled_model.inputs)}) is not equal '
f'to number of ark files ({len(input_file_names)})')
sys.exit(-3)
@@ -209,14 +209,14 @@ def main():
output_file_names = re.split(', |,', args.output)
if len(output_layer_names) != len(output_file_names):
log.error('The number of output files is not equal to the number of network outputs.')
log.error('The number of output files is not equal to the number of model outputs.')
sys.exit(-6)
if args.reference:
reference_file_names = re.split(', |,', args.reference)
if len(output_layer_names) != len(reference_file_names):
log.error('The number of reference files is not equal to the number of network outputs.')
log.error('The number of reference files is not equal to the number of model outputs.')
sys.exit(-5)
reference_file_data = [read_utterance_file(file_name) for file_name in reference_file_names]