IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[IE Samples] Update docs for C++ samples (#9937)

Browse Source 
* update hello classification readme

* update hello classification readme

* update classification async readme

* replace `network` with `model`

* update example section with openvino-dev

* update hello query device readme

* Update hello reshape readme

* Update ngraph func creation readme

* update speech sample readme

* update hello nv12 readme

* Apply suggestions from code review

review comments accepted

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Replace `Inference Engine` with `OpenVINO`

* fix model ref

* Replace `Inference Engine` by `OpenVINO™ Runtime`

* Fix IR mentions

Co-authored-by: Vladimir Dudnik <vladimir.dudnik@intel.com>
Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
This commit is contained in:
Dmitry Pigasin
2022-02-14 19:03:19 +03:00
committed by GitHub
parent 3fcff15166
commit 310eb81403
12 changed files with 598 additions and 504 deletions
Download Patch File Download Diff File Expand all files Collapse all files

198
samples/cpp/classification_sample_async/README.md
View File

@@ -1,33 +1,32 @@
# Image Classification Async C++ Sample {#openvino_inference_engine_samples_classification_sample_async_README}
This sample demonstrates how to execute an inference of image classification networks like AlexNet and GoogLeNet using Asynchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Asynchronous Inference Request API.
Models with only one input and output are supported.
In addition to regular images, the sample also supports single-channel `ubyte` images as an input for LeNet model.
Image Classification Async C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
|OpenVINO Runtime Version| `ov::get_openvino_version` | Get Openvino API version
|Available Devices| `ov::Core::get_available_devices`| Get version information of the devices for inference
| Asynchronous Infer | `ov::InferRequest::start_async`, `ov::InferRequest::set_callback` | Do asynchronous inference with callback
|Custom Extension Kernels| `ov::Core::add_extension`, `ov::Core::set_config`| Load extension library and config to the device
| Model Operations | `ov::set_batch`, `ov::get_batch` | Managing of model, operate with its batch size. Setting batch size using input image count.
| Feature | API | Description |
| :--- | :--- | :--- |
| Asynchronous Infer | `ov::InferRequest::start_async`, `ov::InferRequest::set_callback` | Do asynchronous inference with callback. |
| Model Operations | `ov::Output::get_shape`, `ov::set_batch` | Manage the model, operate with its batch size. Set batch size using input image count. |
| Infer Request Operations | `ov::InferRequest::get_input_tensor` | Get an input tensor. |
| Tensor Operations | `ov::shape_size`, `ov::Tensor::data` | Get a tensor shape size and its data. |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx)
| Validated images | The sample uses OpenCV\* to [read input image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png), single-channel `ubyte` images.
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/classification_sample_async/README.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/classification_sample_async/README.md) |
## How It Works
Upon the start-up, the sample application reads command line parameters and loads specified network and input images (or a
folder with images) to the Inference Engine plugin. The batch size of the network is set according to the number of read images. The batch mode is an independent attribute on the asynchronous mode. Asynchronous mode works efficiently with any batch size.
At startup, the sample application reads command line parameters and loads the specified model and input images (or a
folder with images) to the OpenVINO™ Runtime plugin. The batch size of the model is set according to the number of read images. The batch mode is an independent attribute on the asynchronous mode. Asynchronous mode works efficiently with any batch size.
Then, the sample creates an inference request object and assigns completion callback for it. In scope of the completion callback
handling the inference request is executed again.
@@ -37,14 +36,37 @@ After that, the application starts inference for the first infer request and wai
When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
Run the application with the `-h` option to see the usage instructions:
```
classification_sample_async -h
```
Usage instructions:
```
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
classification_sample_async [OPTION]
Options:
-h Print usage instructions.
-m "<path>" Required. Path to an .xml file with a trained model.
-i "<path>" Required. Path to a folder with images or path to image files: a .ubyte file for LeNet and a .bmp file for other models.
-d "<device>" Optional. Specify the target device to infer on (the list of available devices is shown below). Default value is CPU. Use "-d HETERO:<comma_separated_devices_list>" format to specify the HETERO plugin. Sample will look for a suitable plugin for the device specified.
Available target devices: <devices>
```
To run the sample, you need specify a model and image:
- you can use [public](@ref omz_models_group_public) or [Intel's](@ref omz_models_group_intel) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader).
@@ -52,82 +74,72 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Running the application with the `-h` option yields the following usage message:
```
<path_to_sample>/classification_sample_async -h
InferenceEngine:
API version ............ <version>
Build .................. <build>
Description ....... API
classification_sample_async [OPTION]
Options:
-h Print a usage message.
-m "<path>" Required. Path to an .xml file with a trained model.
-i "<path>" Required. Path to a folder with images or path to an image files: a .ubyte file for LeNet and a .bmp file for the other networks.
-l "<absolute_path>" Required for CPU plugin custom layers. Absolute path to a shared library with the kernels implementations
Or
-c "<absolute_path>" Required for GPU, MYRIAD, HDDL custom kernels. Absolute path to the .xml config file with the kernels descriptions.
-d "<device>" Optional. Specify the target device to infer on (the list of available devices is shown below). Default value is CPU. Use "-d HETERO:<comma_separated_devices_list>" format to specify HETERO plugin. Sample will look for a suitable plugin for device specified.
-nt "<integer>" Optional. Number of top results. Default value is 10.
Available target devices: <devices>
```
Running the application with the empty list of options yields the usage message given above and an error message.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model using:
```
<path_to_sample>/classification_sample_async -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp -d GPU
omz_downloader --name googlenet-v1
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
omz_converter --name googlenet-v1
```
4. Perform inference of `dog.bmp` using `googlenet-v1` model on a `GPU`, for example:
```
classification_sample_async -m googlenet-v1.xml -i dog.bmp -d GPU
```
## Sample Output
By default the application outputs top-10 inference results for each infer request.
```
[ INFO ] InferenceEngine:
IE version ......... 2021.4.0
Build ........... 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] C:\images\car.bmp
[ INFO ] Loading Inference Engine
[ INFO ] Device info:
GPU
clDNNPlugin version ......... 2021.4.0
Build ........... 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Loading network files:
[ INFO ] C:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (749, 637) to (227, 227)
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[ INFO ] /images/dog.bmp
[ INFO ] Loading model files:
[ INFO ] /models/googlenet-v1.xml
[ INFO ] model name: GoogleNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 224, 224}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
[ INFO ] Read input images
[ INFO ] Set batch size 1
[ INFO ] model name: GoogleNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: u8
[ INFO ] input shape: {1, 224, 224, 3}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
[ INFO ] Loading model to the device GPU
[ INFO ] Create infer request
[ INFO ] Start inference (10 asynchronous executions)
[ INFO ] Start inference (asynchronous executions)
[ INFO ] Completed 1 async request execution
[ INFO ] Completed 2 async request execution
[ INFO ] Completed 3 async request execution
@@ -138,33 +150,29 @@ By default the application outputs top-10 inference results for each infer reque
[ INFO ] Completed 8 async request execution
[ INFO ] Completed 9 async request execution
[ INFO ] Completed 10 async request execution
[ INFO ] Processing output blobs
[ INFO ] Completed async requests execution
Top 10 results:
Image C:\images\car.bmp
Image /images/dog.bmp
classid probability
------- -----------
656 0.6645315
654 0.1121185
581 0.0698451
874 0.0334973
436 0.0259718
817 0.0173190
675 0.0109321
511 0.0109075
569 0.0083093
717 0.0063173
[ INFO ] Execution successful
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
156 0.8935547
218 0.0608215
215 0.0217133
219 0.0105667
212 0.0018835
217 0.0018730
152 0.0018730
157 0.0015745
154 0.0012817
220 0.0010099
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

1
samples/cpp/classification_sample_async/classification_sample_async.h
View File

@@ -52,5 +52,6 @@ static void show_usage() {
std::cout << std::endl;
std::cout << " -h " << help_message << std::endl;
std::cout << " -m \"<path>\" " << model_message << std::endl;
std::cout << " -i \"<path>\" " << image_message << std::endl;
std::cout << " -d \"<device>\" " << target_device_message << std::endl;
}

4
samples/cpp/classification_sample_async/main.cpp
View File

@@ -87,8 +87,8 @@ int main(int argc, char* argv[]) {
std::shared_ptr<ov::Model> model = core.read_model(FLAGS_m);
printInputAndOutputsInfo(*model);
OPENVINO_ASSERT(model->get_parameters().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->get_results().size() == 1, "Sample supports models with 1 output only");
OPENVINO_ASSERT(model->inputs().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->outputs().size() == 1, "Sample supports models with 1 output only");
// -------- Step 3. Configure preprocessing --------
const ov::Layout tensor_layout{"NHWC"};