openvino/samples/cpp/ngraph_function_creation_sample/README.md

nGraph Function Creation C++ Sample

This sample demonstrates how to execute an synchronous inference using model built on the fly which uses weights from LeNet classification model, which is known to work well on digit classification tasks.

You do not need an XML file to create a model. The API of ngraph::Function allows creating a model on the fly from the source code.

The following C++ API is used in the application:

Feature	API	Description
OpenVINO Runtime Info	ov::Core::get_versions	Get device plugins versions
Shape Operations	ov::Output::get_shape, ov::Shape::size, ov::shape_size	Operate with shape
Tensor Operations	ov::Tensor::get_byte_size, ov::Tensor:data	Get tensor byte size and its data
Model Operations	ov::set_batch	Operate with model batch size
Infer Request Operations	ov::InferRequest::get_input_tensor	Get a input tensor
nGraph Functions	ov::opset8::Parameter, ov::Node::output, ov::opset8::Constant, ov::opset8::Convolution, ov::opset8::Add, ov::opset1::MaxPool, ov::opset8::Reshape, ov::opset8::MatMul, ov::opset8::Relu, ov::opset8::Softmax, ov::descriptor::Tensor::set_names, ov::opset8::Result, ov::Model, ov::ParameterVector::vector	Used to construct an nGraph function

Basic OpenVINO™ Runtime API is covered by Hello Classification C++ sample.

Options	Values
Validated Models	LeNet
Model Format	model weights file (*.bin)
Validated images	single-channel MNIST ubyte images
Supported devices	All
Other language realization	Python

How It Works

At startup, the sample application does the following:

• Reads command line parameters
• Build a Model and passed weights file
• Loads the model and input data to the OpenVINO™ Runtime plugin
• Performs synchronous inference and processes output data, logging each step in a standard output stream

You can see the explicit description of each sample step at Integration Steps section of "Integrate the OpenVINO™ Runtime with Your Application" guide.

Building

To build the sample, please use instructions available at Build the Sample Applications section in OpenVINO™ Toolkit Samples guide.

Running

ngraph_function_creation_sample <path_to_lenet_weights> <device>

NOTES:

• you can use LeNet model weights in the sample folder: lenet.bin with FP32 weights file
• The lenet.bin with FP32 weights file was generated by the Model Optimizer tool from the public LeNet model with the --input_shape [64,1,28,28] parameter specified.

The original model is available in the Caffe* repository on GitHub*.

You can do inference of an image using a pre-trained model on a GPU using the following command:

ngraph_function_creation_sample lenet.bin GPU

Sample Output

The sample application logs each step in a standard output stream and outputs top-10 inference results.

[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Device info:
[ INFO ] GPU
[ INFO ] Intel GPU plugin version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ]
[ INFO ] Create model from weights: lenet.bin
[ INFO ] model name: lenet
[ INFO ]     inputs
[ INFO ]         input name: NONE
[ INFO ]         input type: f32
[ INFO ]         input shape: {64, 1, 28, 28}
[ INFO ]     outputs
[ INFO ]         output name: output_tensor
[ INFO ]         output type: f32
[ INFO ]         output shape: {64, 10}
[ INFO ] Batch size is 10
[ INFO ] model name: lenet
[ INFO ]     inputs
[ INFO ]         input name: NONE
[ INFO ]         input type: u8
[ INFO ]         input shape: {10, 28, 28, 1}
[ INFO ]     outputs
[ INFO ]         output name: output_tensor
[ INFO ]         output type: f32
[ INFO ]         output shape: {10, 10}
[ INFO ] Compiling a model for the GPU device
[ INFO ] Create infer request
[ INFO ] Combine images in batch and set to input tensor
[ INFO ] Start sync inference
[ INFO ] Processing output tensor

Top 1 results:

Image 0

classid probability label
------- ----------- -----
0       1.0000000   0

Image 1

classid probability label
------- ----------- -----
1       1.0000000   1

Image 2

classid probability label
------- ----------- -----
2       1.0000000   2

Image 3

classid probability label
------- ----------- -----
3       1.0000000   3

Image 4

classid probability label
------- ----------- -----
4       1.0000000   4

Image 5

classid probability label
------- ----------- -----
5       1.0000000   5

Image 6

classid probability label
------- ----------- -----
6       1.0000000   6

Image 7

classid probability label
------- ----------- -----
7       1.0000000   7

Image 8

classid probability label
------- ----------- -----
8       1.0000000   8

Image 9

classid probability label
------- ----------- -----
9       1.0000000   9

Deprecation Notice

Deprecation Begins	June 1, 2020
Removal Date	December 1, 2020

Starting with the OpenVINO™ toolkit 2020.2 release, all of the features previously available through nGraph have been merged into the OpenVINO™ toolkit. As a result, all the features previously available through ONNX RT Execution Provider for nGraph have been merged with ONNX RT Execution Provider for OpenVINO™ toolkit.

Therefore, ONNX RT Execution Provider for nGraph will be deprecated starting June 1, 2020 and will be completely removed on December 1, 2020. Users are recommended to migrate to the ONNX RT Execution Provider for OpenVINO™ toolkit as the unified solution for all AI inferencing on Intel® hardware.

See Also

• Integrate the OpenVINO™ Runtime with Your Application
• Using OpenVINO™ Toolkit Samples
• Model Optimizer