From d670e77d976a98d67b2d24ab4a004fc54751da22 Mon Sep 17 00:00:00 2001 From: Ilya Churaev Date: Wed, 2 Mar 2022 20:07:52 +0300 Subject: [PATCH] Docs: Changed OpenVINO Runtime User Guide integration (#10187) * Changed C++ OpenVINO Runtime User Guide integration * Remove IE from C++ guide * Fixed comments * Additional fix * Fixed some comments * Some new documents * Fixed some comments * Added Python snippets * Added sphinx tabs * Removed tabs * Removed group-tab * Added additional lines * Fixed typo * Fixed comments and build * Try to fix complex tabs * Fixed some typos * Added python code for model representation * Added more python code * Added serialize/visualize python examples * Simplify integration pipeline * Fixed typo * Try to fix tabs * Extend CompiledModel guide * Resolve merge conflict * Added separate infer request guide * Fixed build * Added cancel infer request method * Update docs/snippets/ov_model_snippets.py Co-authored-by: Jan Iwaszkiewicz * Fixed comments * Fixed typo * Extend visualize pass * Fixed comments * Fixed build * Fixed typo * Update docs/snippets/ov_infer_request.py Co-authored-by: Jan Iwaszkiewicz * Update docs/snippets/ov_infer_request.py Co-authored-by: Jan Iwaszkiewicz * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Andrey Zaytsev * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Andrey Zaytsev * Fixed comments * Fixed doc * Fixed merge Co-authored-by: Jan Iwaszkiewicz Co-authored-by: Andrey Zaytsev --- README.md | 8 +- docs/CMakeLists.txt | 4 +- ...grate_with_customer_application_new_API.md | 490 ------------------ docs/OV_Runtime_UG/PythonPackage_Overview.md | 14 + .../integrate_with_your_application.md | 265 ++++++++++ docs/OV_Runtime_UG/model_representation.md | 181 +++++-- docs/OV_Runtime_UG/network_state_intro.md | 4 +- docs/OV_Runtime_UG/openvino_intro.md | 3 +- docs/OV_Runtime_UG/ov_infer_request.md | 277 ++++++++++ .../supported_plugins/Supported_Devices.md | 2 +- docs/index.rst | 4 +- docs/snippets/CMakeLists.txt | 19 + ...rate_with_customer_application_new_API.cpp | 133 ----- docs/snippets/example_ngraph_utils.cpp | 60 --- docs/snippets/ov_infer_request.cpp | 112 ++++ docs/snippets/ov_infer_request.py | 97 ++++ docs/snippets/ov_model_snippets.cpp | 110 +++- docs/snippets/ov_model_snippets.py | 88 ++++ docs/snippets/src/main.cpp | 69 +++ docs/snippets/src/main.py | 58 +++ samples/c/hello_classification/README.md | 6 +- .../hello_nv12_input_classification/README.md | 6 +- .../cpp/classification_sample_async/README.md | 4 +- samples/cpp/hello_classification/README.md | 4 +- .../hello_nv12_input_classification/README.md | 4 +- samples/cpp/hello_query_device/README.md | 2 +- samples/cpp/hello_reshape_ssd/README.md | 4 +- samples/cpp/model_creation_sample/README.md | 4 +- samples/cpp/speech_sample/README.md | 4 +- .../classification_sample_async/README.md | 4 +- samples/python/hello_classification/README.md | 4 +- samples/python/hello_reshape_ssd/README.md | 4 +- .../python/model_creation_sample/README.md | 4 +- samples/python/speech_sample/README.md | 4 +- tools/pot/openvino/tools/pot/api/README.md | 2 +- 35 files changed, 1286 insertions(+), 772 deletions(-) delete mode 100644 docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md create mode 100644 docs/OV_Runtime_UG/PythonPackage_Overview.md create mode 100644 docs/OV_Runtime_UG/integrate_with_your_application.md create mode 100644 docs/OV_Runtime_UG/ov_infer_request.md delete mode 100644 docs/snippets/Integrate_with_customer_application_new_API.cpp create mode 100644 docs/snippets/ov_infer_request.cpp create mode 100644 docs/snippets/ov_infer_request.py create mode 100644 docs/snippets/ov_model_snippets.py create mode 100644 docs/snippets/src/main.cpp create mode 100644 docs/snippets/src/main.py diff --git a/README.md b/README.md index fc47dbe49d4..95431e98dce 100644 --- a/README.md +++ b/README.md @@ -16,12 +16,11 @@ source and public models in popular formats such as TensorFlow, ONNX, PaddlePadd * [OpenVINO™ Runtime] * [Model Optimizer] * [Post-Training Optimization Tool] - +* [Samples] ## License -Deep Learning Deployment Toolkit is licensed under [Apache License Version 2.0](LICENSE). -By contributing to the project, you agree to the license and copyright terms therein -and release your contribution under these terms. +OpenVINO™ Toolkit is licensed under [Apache License Version 2.0](LICENSE). +By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms. ## Resources * Docs: https://docs.openvino.ai/ @@ -46,5 +45,6 @@ Please report questions, issues and suggestions using: [OpenVINO™ Runtime]:https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_User_Guide.html [Model Optimizer]:https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html [Post-Training Optimization Tool]:https://docs.openvino.ai/latest/pot_README.html +[Samples]:https://github.com/openvinotoolkit/openvino/tree/master/samples [tag on StackOverflow]:https://stackoverflow.com/search?q=%23openvino diff --git a/docs/CMakeLists.txt b/docs/CMakeLists.txt index 75421a3c972..75dbeb180c6 100644 --- a/docs/CMakeLists.txt +++ b/docs/CMakeLists.txt @@ -7,8 +7,6 @@ if(NOT ENABLE_DOCKER) ie_add_compiler_flags(-Wall) endif() - add_subdirectory(snippets) - # Detect OpenVINO find_package(OpenVINO QUIET PATHS "${CMAKE_BINARY_DIR}" @@ -17,6 +15,8 @@ if(NOT ENABLE_DOCKER) set(OpenVINO_DIR ${CMAKE_BINARY_DIR}) endif() + add_subdirectory(snippets) + add_subdirectory(template_extension) set(all_docs_targets diff --git a/docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md b/docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md deleted file mode 100644 index ebbec0e3c6e..00000000000 --- a/docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md +++ /dev/null @@ -1,490 +0,0 @@ -# Integrate Inference Engine {#openvino_docs_IE_DG_Integrate_with_customer_application_new_API} - -## Integrate Inference Engine with Your C++ Application - -@sphinxdirective -.. raw:: html - -
C++
-@endsphinxdirective - -The following diagram illustrates the typical Inference Engine С++ API workflow: - -![ie_api_flow_cpp] - -Read the sections below to learn about each item. - -> **NOTE**: Before start using Inference Engine, make sure you set all environment variables during the installation. If you did not, follow the instructions from the _Set the Environment Variables_ section in the installation guides: -> * [For Windows* 10](../install_guides/installing-openvino-windows.md) -> * [For Linux*](../install_guides/installing-openvino-linux.md) -> * [For macOS*](../install_guides/installing-openvino-macos.md) -> * To build an open source version, use the [Inference Engine Build Instructions](https://github.com/openvinotoolkit/openvino/wiki/BuildingCode). - -### Link with Inference Library - -1. **Create a structure** for the project: - ``` sh - project/ - ├── CMakeLists.txt - CMake file to build - ├── ... - Additional folders like includes/ - └── src/ - source folder - └── main.cpp - build/ - build directory - ... - ``` - -2. **Include Inference Engine, nGraph and OpenCV libraries** in `project/CMakeLists.txt` -[OpenCV](https://docs.opencv.org/master/db/df5/tutorial_linux_gcc_cmake.html) integration is needed mostly for pre-processing input data and model representation in OpenVINO™ Runtime for more complex applications using [OpenVINO Model API](../OV_Runtime_UG/model_representation.md). - ``` cmake - cmake_minimum_required(VERSION 3.0.0) - project(project_name) - find_package(OpenVINO REQUIRED) - add_executable(${PROJECT_NAME} src/main.cpp) - target_link_libraries(${PROJECT_NAME} PRIVATE openvino::runtime) - ``` - -### Use Inference Engine API to Implement Inference Pipeline - -This section provides step-by-step instructions to implement a typical inference pipeline with the Inference Engine C++ API: - -![ie_api_use_cpp] -#### Step 1. Create Inference Engine Core - -Use the following code to create Inference Engine Core to manage available devices and read network objects: - -@snippet snippets/Integrate_with_customer_application_new_API.cpp part0 - -#### Step 2 (Optional). Configure Input and Output of the Model - -@sphinxdirective -.. raw:: html - -
-@endsphinxdirective - - -Optionally, configure input and output of the model using the steps below: - -1. Load a model to a Core object: - @sphinxdirective - - .. tab:: IR - - .. code-block:: c - - auto network = core.ReadNetwork("model.xml"); - - .. tab:: ONNX - - .. code-block:: c - - auto network = core.ReadNetwork("model.onnx"); - - .. tab:: nGraph - - .. code-block:: c - - std::shared_ptr createNetwork() { - // To construct a network, please follow - // https://docs.openvino.ai/latest/openvino_docs_nGraph_DG_build_function.html - } - auto network = CNNNetwork(createNetwork()); - - @endsphinxdirective - -2. Request input and output information using `InferenceEngine::CNNNetwork::getInputsInfo()`, and `InferenceEngine::CNNNetwork::getOutputsInfo()` methods: - ```cpp - /** Take information about all topology inputs **/ - InferenceEngine::InputsDataMap input_info = network.getInputsInfo(); - /** Iterate over all input info**/ - for (auto &item : input_info) { - auto input_data = item.second; - // Add your input configuration steps here - } - - /** Take information about all topology outputs **/ - InferenceEngine::OutputsDataMap output_info = network.getOutputsInfo(); - /** Iterate over all output info**/ - for (auto &item : output_info) { - auto output_data = item.second; - // Add your output configuration steps here - } - ``` - Configuring options: - 1. **Set precision** (number format): FP16, FP32, INT8, etc. Refer to the Supported Configurations section on the [Supported Devices](supported_plugins/Supported_Devices.md) page to choose the relevant configuration.
- For input (*iterate over all input info*): - ```cpp - input_data->setPrecision(InferenceEngine::Precision::U8); - ``` - For output (*iterate over all output info*): - ```cpp - output_data->setPrecision(InferenceEngine::Precision::FP32); - ``` - **By default**, the input and output precision is set to `Precision::FP32`. - - 2. **Set layout** (NCHW, ).
- For input (*iterate over all input info*): - ```cpp - input_data->setLayout(InferenceEngine::Layout::NCHW); - ``` - **By default**, the input layout is set to `Layout::NCHW`.
- For output (*iterate over all output info*): - ```cpp - output_data->setLayout(InferenceEngine::Layout::NC); - ``` - **By default**, the output layout depends on a number of its dimensions:
- |Number of dimensions | 5 | 4 | 3 | 2 | 1 | - |:--------------------|-------|------|-----|----|----| - |Layout | NCDHW | NCHW | CHW | NC | C | - 3. **Set resize algorithm for inputs** (Bilinear). You can allow input of any size. To do this, mark each input as resizable by setting a desired resize algorithm (e.g. `BILINEAR`) inside of the appropriate input info (*Iterate over all input info*): - ```cpp - input_data->getPreProcess().setResizeAlgorithm(InferenceEngine::RESIZE_BILINEAR); - ``` - **By default**, no resize algorithm is set for inputs. - - 4. **Set color format** (BGR, RGB, NV12). Basic color format conversions are supported as well. **By default**, the Inference Engine assumes that the input color format is BGR and color format conversions are disabled. Set `ColorFormat::RAW` input color format if the input does not need color conversions. The Inference Engine supports the following color format conversions: - * RGB->BGR - * RGBX->BGR - * BGRX->BGR - * NV12->BGR - where X is a channel that will be ignored during inference. To enable the conversions, set a desired color format (for example, RGB) for each input inside of the appropriate input info (*iterate over all input info*): - ```cpp - input_data->getPreProcess().setColorFormat(InferenceEngine::ColorFormat::RGB); - ``` - > **NOTE**: NV12 input color format pre-processing differs from other color conversions. In case of NV12, Inference Engine expects two separate image planes (Y and UV). You must use a specific `InferenceEngine::NV12Blob` object instead of default blob object and set this blob to the Inference Engine Infer Request using `InferenceEngine::InferRequest::SetBlob()`. Refer to [Hello NV12 Input Classification C++ Sample](../../samples/cpp/hello_nv12_input_classification/README.md) for more details. - - 5. **Run on multiple images** with setting batch. If you want to run inference for multiple images at once, you can use the built-in batch pre-processing functionality. - - **NOTE** : Batch pre-processing is not supported if input color format is set to `ColorFormat::NV12`. - -@sphinxdirective -.. raw:: html - -
-@endsphinxdirective - -#### Step 3. Load the Model to the Device - -Load the model to the device using `InferenceEngine::Core::LoadNetwork()`: - - -@sphinxdirective - -.. tab:: IR - - .. code-block:: c - - executable_network = core.LoadNetwork("model.xml", "CPU"); - -.. tab:: ONNX - - .. code-block:: c - - executable_network = core.LoadNetwork("model.onnx", "CPU"); - -.. tab:: nGraph - - .. code-block:: c - - std::shared_ptr createNetwork() { - // To construct a network, please follow - // https://docs.openvino.ai/latest/openvino_docs_nGraph_DG_build_function.html - } - auto network = CNNNetwork(createNetwork()); - executable_network = core.LoadNetwork(network, "CPU"); - -.. tab:: Model From Step 2 - - Follow this step only if you went through optional "Step 2 (Optional). Configure Input and Output of the Model", otherwise use another tab for your model type: IR (OpenVINO Intermediate Representation), ONNX or nGraph. - - .. code-block:: c - - executable_network = core.LoadNetwork(network, "CPU"); - -@endsphinxdirective - - -It creates an executable network from a network object. The executable network is associated with single hardware device. -It is possible to create as many networks as needed and to use them simultaneously (up to the limitation of the hardware resources). - -Third parameter is a configuration for plugin. It is map of pairs: (parameter name, parameter value). Choose device from -[Supported devices](supported_plugins/Supported_Devices.md) page for more details about supported configuration parameters. - -@snippet snippets/Integrate_with_customer_application_new_API.cpp part6 - -#### Step 4. Create an Inference Request - -Create an infer request using the following code: - -@snippet snippets/Integrate_with_customer_application_new_API.cpp part7 - -#### Step 5. Prepare Input - -You can use one of the following options to prepare input: - -* **Optimal way for a single network.** Get blobs allocated by an infer request using `InferenceEngine::InferRequest::GetBlob()` and feed an image and the input data to the blobs. In this case, input data must be aligned (resized manually) with a given blob size and have a correct color format. - - @snippet snippets/Integrate_with_customer_application_new_API.cpp part8 - -* **Optimal way for a cascade of networks (output of one network is input for another).** Get output blob from the first request using `InferenceEngine::InferRequest::GetBlob()` and set it as input for the second request using `InferenceEngine::InferRequest::SetBlob()`. - - @snippet snippets/Integrate_with_customer_application_new_API.cpp part9 - -* **Optimal way to handle ROI (a ROI object located inside of input of one network is input for another).** It is possible to re-use shared input by several networks. You do not need to allocate separate input blob for a network if it processes a ROI object located inside of already allocated input of a previous network. For instance, when first network detects objects on a video frame (stored as input blob) and second network accepts detected bounding boxes (ROI inside of the frame) as input. In this case, it is allowed to re-use pre-allocated input blob (used by first network) by second network and just crop ROI without allocation of new memory using `InferenceEngine::make_shared_blob()` with passing of `InferenceEngine::Blob::Ptr` and `InferenceEngine::ROI` as parameters. - - @snippet snippets/Integrate_with_customer_application_new_API.cpp part10 - - Make sure that shared input is kept valid during execution of each network. Otherwise, ROI blob may be corrupted if the original input blob (that ROI is cropped from) has already been rewritten. - -* Allocate input blobs of the appropriate types and sizes, feed an image and the input data to the blobs, and call `InferenceEngine::InferRequest::SetBlob()` to set these blobs for an infer request: - - @snippet snippets/Integrate_with_customer_application_new_API.cpp part11 - -A blob can be filled before and after `SetBlob()`. - -> **NOTE**: -> -> * The `SetBlob()` method compares precision and layout of an input blob with the ones defined in step 3 and -> throws an exception if they do not match. It also compares a size of the input blob with input -> size of the read network. But if input was configured as resizable, you can set an input blob of -> any size (for example, any ROI blob). Input resize will be invoked automatically using resize -> algorithm configured on step 3. Similarly to the resize, color format conversions allow the color -> format of an input blob to differ from the color format of the read network. Color format -> conversion will be invoked automatically using color format configured on step 3. -> -> * `GetBlob()` logic is the same for pre-processable and not pre-processable input. Even if it is -> called with input configured as resizable or as having specific color format, a blob allocated by -> an infer request is returned. Its size and color format are already consistent with the -> corresponding values of the read network. No pre-processing will happen for this blob. If you -> call `GetBlob()` after `SetBlob()`, you will get the blob you set in `SetBlob()`. - -#### Step 6. Start Inference - -Start inference in asynchronous or synchronous mode. Async API usage can improve overall frame-rate of the application, because rather than wait for inference to complete, the app can continue doing things on the host, while accelerator is busy. - -* For synchronous inference request: - ```cpp - infer_request.Infer(); - ``` - -* For asynchronous inference request: - ```cpp - infer_request.StartAsync(); - infer_request.Wait(InferenceEngine::InferRequest::WaitMode::RESULT_READY); - ``` - `StartAsync` returns immediately and starts inference without blocking main thread, `Infer` blocks main thread and returns when inference is completed. Call `Wait` for waiting result to become available for asynchronous request. - - There are three ways to use it: - * specify maximum duration in milliseconds to block for. The method is blocked until the specified timeout has elapsed, or the result becomes available, whichever comes first. - * `InferenceEngine::InferRequest::WaitMode::RESULT_READY` - waits until inference result becomes available - * `InferenceEngine::InferRequest::WaitMode::STATUS_ONLY` - immediately returns request status.It does not - block or interrupts current thread. - - -Both requests are thread-safe: can be called from different threads without fearing corruption and failures. - -Multiple requests for single `ExecutableNetwork` are executed sequentially one by one in FIFO order. - -While request is ongoing, all its methods except `InferenceEngine::InferRequest::Wait` would throw an -exception. - -#### Step 7. Process the Inference Results - -Go over the output blobs and process the inference results. Note that casting `Blob` to `TBlob` via `std::dynamic_pointer_cast` is not the recommended way. It's better to access data via the `buffer()` and `as()` methods as follows: - -@snippet snippets/Integrate_with_customer_application_new_API.cpp part14 - -### Build Your Application - -For details about building your application, refer to the CMake files for the sample applications. -All samples source code is located in the `/samples` directory, where `INSTALL_DIR` is the OpenVINO™ installation directory. - -To build your project using CMake with the default build tools currently available on your machine, execute the following commands: - -> **NOTE**: Make sure you set environment variables first by running `/setupvars.sh` (or `setupvars.bat` for Windows). Otherwise the `InferenceEngine_DIR` and `OpenCV_DIR` variables won't be configured properly to pass `find_package` calls. - -```sh -cd build/ -cmake ../project -cmake --build . -``` -It's allowed to specify additional build options (e.g. to build CMake project on Windows with a specific build tools). Please refer to the [CMake page](https://cmake.org/cmake/help/latest/manual/cmake.1.html#manual:cmake(1)) for details. - -### Run Your Application - -> **NOTE**: Before running, make sure you completed **Set the Environment Variables** section in [OpenVINO Installation](../../samples/cpp/hello_nv12_input_classification/README.md) document so that the application can find the libraries. - -To run compiled applications on Microsoft* Windows* OS, make sure that Microsoft* Visual C++ 2017 -Redistributable and Intel® C++ Compiler 2017 Redistributable packages are installed and -`/bin/intel64/Release/*.dll` files are placed to the -application folder or accessible via `%PATH%` environment variable. - -## Integrate Inference Engine with Your Python Application - -@sphinxdirective -.. raw:: html - -
Python
-@endsphinxdirective - -This document explains how to integrate and use the Inference Engine API with your Python application. - -The following diagram illustrates the typical Inference Engine Python API workflow: -![ie_api_flow_python] - -Read the sections below to learn about each item. - -### Import Inference Module - -To make use of the Inference Engine functionality, import IECore to your application: - -```py -from openvino.inference_engine import IECore -``` - -### Use Inference Engine API - -This section provides step-by-step instructions to implement a typical inference pipeline with the Inference Engine API: - -![ie_api_use_python] - -#### Step 1. Create Inference Engine Core - -Use the following code to create Inference Engine Core to manage available devices and read network objects: -```py -ie = IECore() -``` -#### Step 2 (Optional). Read model. Configure Input and Output of the Model - -@sphinxdirective -.. raw:: html - -
-@endsphinxdirective - -Optionally, configure input and output of the model using the steps below: - -1. Read model - @sphinxdirective - - .. tab:: IR - - .. code-block:: python - - net = ie.read_network(model="model.xml") - - .. tab:: ONNX - - .. code-block:: python - - net = ie.read_network(model="model.onnx") - - .. tab:: nGraph - - .. code-block:: python - - #Basic example of nGraph model creation - param = Parameter(Type.f32, Shape([1, 3, 22, 22])) - relu = ng.relu(param) - func = Function([relu], [param], 'test') - caps = Function.to_capsule(func) - net = IENetwork(caps) - - @endsphinxdirective - -2. Request input and output information using input_info, outputs - ```py - inputs = net.input_info - input_name = next(iter(net.input_info)) - - outputs = net.outputs - output_name = next(iter(net.outputs)) - ``` - Information for this input layer is stored in input_info. The next cell prints the input layout, precision and shape. - ```py - print("Inputs:") - for name, info in net.input_info.items(): - print("\tname: {}".format(name)) - print("\tshape: {}".format(info.tensor_desc.dims)) - print("\tlayout: {}".format(info.layout)) - print("\tprecision: {}\n".format(info.precision)) - ``` - This cell output tells us that the model expects inputs with a shape of [1,3,224,224], and that this is in NCHW layout. This means that the model expects input data with a batch size (N) of 1, 3 channels (C), and images of a height (H) and width (W) of 224. The input data is expected to be of FP32 (floating point) precision. - - Getting the output layout, precision and shape is similar to getting the input layout, precision and shape. - ```py - print("Outputs:") - for name, info in net.outputs.items(): - print("\tname: {}".format(name)) - print("\tshape: {}".format(info.shape)) - print("\tlayout: {}".format(info.layout)) - print("\tprecision: {}\n".format(info.precision)) - ``` - This cell output shows that the model returns outputs with a shape of [1, 1001], where 1 is the batch size (N) and 1001 the number of classes (C). The output is returned as 32-bit floating point. - -@sphinxdirective -.. raw:: html - -
-@endsphinxdirective - -#### Step 3. Load model to the Device - -Load the model to the device using `load_network()`: - -@sphinxdirective - -.. tab:: IR - - .. code-block:: python - - exec_net = ie.load_network(network= "model.xml", device_name="CPU") -.. tab:: ONNX - - .. code-block:: python - - exec_net = ie.load_network(network= "model.onnx", device_name="CPU") - -.. tab:: Model from step 2 - - .. code-block:: python - - exec_net = ie.load_network(network=net, device_name="CPU") - -@endsphinxdirective - -This example is designed for CPU device, refer to the [Supported Devices](../OV_Runtime_UG/supported_plugins/Supported_Devices.md) page to read about more devices. - -#### Step 4. Prepare input -```py -import cv2 -import numpy as np - -image = cv2.imread("image.png") - -# Resize with OpenCV your image if needed to match with net input shape -# N, C, H, W = net.input_info[input_name].tensor_desc.dims -# image = cv2.resize(src=image, dsize=(W, H)) - -# Converting image to NCHW format with FP32 type -input_data = np.expand_dims(np.transpose(image, (2, 0, 1)), 0).astype(np.float32) -``` - -#### Step 5. Start Inference -```py -result = exec_net.infer({input_name: input_data}) -``` - -#### Step 6. Process the Inference Results -```py -output = result[output_name] -``` - -### Run Your Application - -Congratulations, you have made your first Python application with OpenVINO™ toolkit, now you may run it. - -[ie_api_flow_cpp]: img/BASIC_IE_API_workflow_Cpp.svg -[ie_api_use_cpp]: img/IMPLEMENT_PIPELINE_with_API_C.svg -[ie_api_flow_python]: img/BASIC_IE_API_workflow_Python.svg -[ie_api_use_python]: img/IMPLEMENT_PIPELINE_with_API_Python.svg diff --git a/docs/OV_Runtime_UG/PythonPackage_Overview.md b/docs/OV_Runtime_UG/PythonPackage_Overview.md new file mode 100644 index 00000000000..5e03eb3295c --- /dev/null +++ b/docs/OV_Runtime_UG/PythonPackage_Overview.md @@ -0,0 +1,14 @@ +# OpenVINO™ Python* Package + +OpenVINO™ Python\* package includes types to measure model and calibrate to low precision. + +The OpenVINO™ Python\* package available in the `/python/python3.X` directory. + +The OpenVINO™ Python\* package includes the following sub-packages: + + - [openvino.inference_engine](../../src/bindings/python/docs/api_overview.md) - Python\* wrapper on OpenVINO™ Inference Engine. + - `openvino.tools.accuracy_checker` - Measure accuracy. + - `openvino.tools.benchmark` - Measure latency and throughput. + +## See Also +* [Integrate with Customer Application New API](integrate_with_your_application.md) diff --git a/docs/OV_Runtime_UG/integrate_with_your_application.md b/docs/OV_Runtime_UG/integrate_with_your_application.md new file mode 100644 index 00000000000..6472e9ec8c5 --- /dev/null +++ b/docs/OV_Runtime_UG/integrate_with_your_application.md @@ -0,0 +1,265 @@ +# Integrate OpenVINO™ with Your Application {#openvino_docs_Integrate_OV_with_your_application} + +@sphinxdirective + +.. toctree:: + :maxdepth: 1 + :hidden: + + openvino_docs_OV_Runtime_UG_Model_Representation + openvino_docs_OV_Runtime_UG_Infer_request + +@endsphinxdirective + +> **NOTE**: Before start using OpenVINO™ Runtime, make sure you set all environment variables during the installation. If you did not, follow the instructions from the _Set the Environment Variables_ section in the installation guides: +> * [For Windows* 10](../install_guides/installing-openvino-windows.md) +> * [For Linux*](../install_guides/installing-openvino-linux.md) +> * [For macOS*](../install_guides/installing-openvino-macos.md) +> * To build an open source version, use the [OpenVINO™ Runtime Build Instructions](https://github.com/openvinotoolkit/openvino/wiki/BuildingCode). + +## Use OpenVINO™ Runtime API to Implement Inference Pipeline + +This section provides step-by-step instructions to implement a typical inference pipeline with the OpenVINO™ Runtime C++ API: + +![ie_api_use_cpp] + +### Step 1. Create OpenVINO™ Runtime Core + +Include next files to work with OpenVINO™ Runtime: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [include] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [import] + +@endsphinxdirective + +Use the following code to create OpenVINO™ Core to manage available devices and read model objects: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part1] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part1] + +@endsphinxdirective + +### Step 2. Compile the Model + +`ov::CompiledModel` class represents a device specific compiled model. `ov::CompiledModel` allows you to get information inputs or output ports by a tensor name or index. + +Compile the model for a specific device using `ov::Core::compile_model()`: + +@sphinxdirective + +.. tab:: C++ + + .. tab:: IR + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part2_1] + + .. tab:: ONNX + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part2_2] + + .. tab:: PaddlePaddle + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part2_3] + + .. tab:: ov::Model + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part2_4] + +.. tab:: Python + + .. tab:: IR + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part2_1] + + .. tab:: ONNX + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part2_2] + + .. tab:: PaddlePaddle + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part2_3] + + .. tab:: ov::Model + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part2_4] + +@endsphinxdirective + +The `ov::Model` object represents any models inside the OpenVINO™ Runtime. +For more details please read article about [OpenVINO™ Model representation](model_representation.md). + +The code above creates a compiled model associated with a single hardware device from the model object. +It is possible to create as many compiled models as needed and use them simultaneously (up to the limitation of the hardware resources). +To learn how to change the device configuration, read the [Query device properties](./supported_plugins/config_properties.md) article. + +### Step 3. Create an Inference Request + +`ov::InferRequest` class provides methods for model inference in the OpenVINO™ Runtime. +This section demonstrates a simple pipeline, to get more information about other use cases, read the [InferRequest documentation](./ov_infer_request.md) dedicated article. +Create an infer request using the following code: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part3] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part3] + +@endsphinxdirective + +### Step 4. Set Inputs + +You can use external memory to create `ov::Tensor` and use the `ov::InferRequest::set_input_tensor` method to put this tensor on the device: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part4] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part4] + +@endsphinxdirective + +### Step 5. Start Inference + +OpenVINO™ Runtime supports inference in asynchronous or synchronous mode. Async API usage can improve overall frame-rate of the application, because rather than wait for inference to complete, the app can continue doing things on the host, while the accelerator is busy. You can use `ov::InferRequest::start_async()` to start model inference in the asynchronous mode and call `ov::InferRequest::wait()` to wait for the inference results: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part5] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part5] + +@endsphinxdirective + +The asynchronous mode supports two methods to get the inference results: + * `ov::InferRequest::wait_for()` - Waits until the specified timeout (in milliseconds) has elapsed or the inference result becomes available, whichever comes first. + * `ov::InferRequest::wait()` - Waits until the inference result becomes available. + +Both requests are thread-safe, which means they can be called from different threads without exposing erroneous behavior or producing unpredictable results. + +While the request is ongoing, all its methods except `ov::InferRequest::cancel`, `ov::InferRequest::wait` or `ov::InferRequest::wait_for` throw +the `ov::Busy` exception indicating the request is busy with computations. + +### Step 6. Process the Inference Results + +Go over the output tensors and process the inference results. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/src/main.cpp + :language: cpp + :fragment: [part6] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/src/main.py + :language: python + :fragment: [part6] + +@endsphinxdirective + +## Link and Build Your C++ Application with OpenVINO™ Runtime + +The example uses CMake for project configuration. + +1. **Create a structure** for the project: + ``` sh + project/ + ├── CMakeLists.txt - CMake file to build + ├── ... - Additional folders like includes/ + └── src/ - source folder + └── main.cpp + build/ - build directory + ... + ``` + +2. **Include OpenVINO™ Runtime libraries** in `project/CMakeLists.txt` + + @snippet snippets/CMakeLists.txt cmake:integration_example + +To build your project using CMake with the default build tools currently available on your machine, execute the following commands: + +> **NOTE**: Make sure you set environment variables first by running `/setupvars.sh` (or `setupvars.bat` for Windows). Otherwise the `OpenVINO_DIR` variable won't be configured properly to pass `find_package` calls. + +```sh +cd build/ +cmake ../project +cmake --build . +``` +It's allowed to specify additional build options (e.g. to build CMake project on Windows with a specific build tools). Please refer to the [CMake page](https://cmake.org/cmake/help/latest/manual/cmake.1.html#manual:cmake(1)) for details. + +## Run Your Application + +Congratulations, you have made your first application with OpenVINO™ toolkit, now you may run it. + +## See also + + - [OpenVINO™ Runtime Preprocessing](./preprocessing_overview.md) + +[ie_api_flow_cpp]: img/BASIC_IE_API_workflow_Cpp.svg +[ie_api_use_cpp]: img/IMPLEMENT_PIPELINE_with_API_C.svg +[ie_api_flow_python]: img/BASIC_IE_API_workflow_Python.svg +[ie_api_use_python]: img/IMPLEMENT_PIPELINE_with_API_Python.svg diff --git a/docs/OV_Runtime_UG/model_representation.md b/docs/OV_Runtime_UG/model_representation.md index f3edf5e2f24..0e932278373 100644 --- a/docs/OV_Runtime_UG/model_representation.md +++ b/docs/OV_Runtime_UG/model_representation.md @@ -2,8 +2,6 @@ In OpenVINO™ Runtime a model is represented by the `ov::Model` class. -## Model Representation - The `ov::Model` object stores shared pointers to `ov::op::v0::Parameter`, `ov::op::v0::Result` and `ov::op::Sink` operations that are inputs, outputs and sinks of the graph. Sinks of the graph have no consumers and are not included in the results vector. All other operations hold each other via shared pointers: child operation holds its parent (hard link). If an operation has no consumers and it's not the `Result` or `Sink` operation (shared pointer counter is zero), then it will be destructed and won't be accessible anymore. @@ -12,24 +10,63 @@ Each operation in `ov::Model` has the `std::shared_ptr` type. For details on how to build a model in OpenVINO™ Runtime, see the [Build a Model in OpenVINO™ Runtime](@ref build_model) section. +OpenVINO™ Runtime allows using tensor names or indexes to work wit model inputs/outputs. To get model input/output ports, use the `ov::Model::inputs()` or `ov::Model::outputs()` methods respectively. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [all_inputs_ouputs] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [all_inputs_ouputs] + +@endsphinxdirective + +OpenVINO™ Runtime model representation uses special classes to work with model data types and shapes. For data types the `ov::element::Type` is used. + +## Shapes Representation + +OpenVINO™ Runtime provides two types for shape representation: + +* `ov::Shape` - Represents static (fully defined) shapes. + +* `ov::PartialShape` - Represents dynamic shapes. That means that the rank or some of dimensions are dynamic (dimension defines an interval or undefined). `ov::PartialShape` can be converted to `ov::Shape` using the `get_shape()` method if all dimensions are static; otherwise the conversion raises an exception. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:partial_shape] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [ov:partial_shape] + +@endsphinxdirective + + But in most cases before getting static shape using `get_shape()` method, you need to check that shape is static. + ## Operations -The `ov::Op` class represents any abstract operation in the model representation. Use this class to create [custom operations](../Extensibility_UG/add_openvino_ops). +The `ov::Op` class represents any abstract operation in the model representation. Use this class to create [custom operations](../Extensibility_UG/add_openvino_ops.md). ## Operation Sets Operation set (opset) is a collection of operations that can be used to construct a model. The `ov::OpSet` class provides a functionality to work with operation sets. -For each operation set, OpenVINO™ Runtime provides a separate namespace, for example `opset8`. +For each operation set, OpenVINO™ Runtime provides a separate namespace, for example `opset8`. Each OpenVINO™ Release release introduces new operations and add these operations to a new operation set. New operation sets help to introduce a new version of operations that change behavior of previous operations. Using operation sets allows you to avoid changes in your application if new operations have been introduced. - -## Static and Partial Shapes - -There are two types used for shape representation: - -* `ov::Shape` - Represents static (fully defined) shapes. - -* `ov::PartialShape` - Represents dynamic shapes. That means that the rank or some of dimensions are dynamic (undefined). `ov::PartialShape` can be converted to `ov::Shape` using the `get_shape()` method if all dimensions are static; otherwise the conversion raises an exception. - +For a complete list of operation sets supported in OpenVINO™ toolkit, see [Available Operations Sets](../ops/opset.md). +To add support of custom operations, see the [Add Custom OpenVINO Operations](../Extensibility_UG/Intro.md) document. ## Build a Model in OpenVINO™ Runtime {#build_model} @@ -37,55 +74,109 @@ You can create a model from source. This section illustrates how to construct a Operation set `opsetX` integrates a list of pre-compiled operations that work for this purpose. In other words, `opsetX` defines a set of operations for building a graph. -For a complete list of operation sets supported in OpenVINO™ toolkit, see [Available Operations Sets](../ops/opset.md). - -To add support of custom operations, see the [Add Custom OpenVINO Operations](../Extensibility_UG/Intro.md) document. - To build an `ov::Model` instance from `opset8` operations, include the following files: - - C++ - @snippet example_ngraph_utils.cpp ov:include +@sphinxdirective - - Python - ```python - import openvino.runtime.opset8 as ov - from openvino.runtime import Model - ``` +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:include] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [import] + +@endsphinxdirective The following code demonstrates how to create a simple model: - - C++ - @snippet example_ngraph_utils.cpp ov:create_simple_model +@sphinxdirective - - Python - ```python - TBD - ``` +.. tab:: C++ + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:create_simple_model] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [ov:create_simple_model] + +@endsphinxdirective The following code creates a model with several outputs: - - C++ - @snippet example_ngraph_utils.cpp ov:create_advanced_model +@sphinxdirective - - Python - ```python - TBD - ``` +.. tab:: C++ -## FAQ + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:create_advanced_model] -### Does OpenVINO™ Runtime provide any capabilities to debug the model structure and model modification? - - To receive additional messages about applied graph modifications, rebuild the OpenVINO™ Runtime library with the `-DENABLE_OPENVINO_DEBUG=ON` option. - - A model can be visualized to image from the xDot format: - @snippet example_ngraph_utils.cpp ov:visualize - - A model can be serialized to IR: - @snippet example_ngraph_utils.cpp ov:serialize +.. tab:: Python -### How can I develop my own transformation pass? - See the [Transformations Developer Guide](./../Extensibility_UG/ov_transformations.md). + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [ov:create_advanced_model] + +@endsphinxdirective + +## Model debug capabilities + +OpenVINO™ provides several debug capabilities: + - To receive additional messages about applied model modifications, rebuild the OpenVINO™ Runtime library with the `-DENABLE_OPENVINO_DEBUG=ON` option. + - Model can be visualized to image from the xDot format: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:visualize] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [ov:visualize] + + @endsphinxdirective + + `ov::pass::VisualizeTree` can be parametrized via environment variables: + + OV_VISUALIZE_TREE_OUTPUT_SHAPES=1 - visualize shapes + OV_VISUALIZE_TREE_OUTPUT_TYPES=1 - visualize types + OV_VISUALIZE_TREE_MIN_MAX_DENORMAL=1 - pretty denormal values + OV_VISUALIZE_TREE_RUNTIME_INFO=1 - print runtime information + OV_VISUALIZE_TREE_IO=1 - print I/O ports + OV_VISUALIZE_TREE_MEMBERS_NAME=1 - print member names + + - Also model can be serialized to IR: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_model_snippets.cpp + :language: cpp + :fragment: [ov:serialize] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_model_snippets.py + :language: python + :fragment: [ov:serialize] + + @endsphinxdirective ## See Also * [Available Operation Sets](../ops/opset.md) * [OpenVINO™ Runtime Extensibility Developer Guide](../Extensibility_UG/Intro.md) +* [Transformations Developer Guide](../Extensibility_UG/ov_transformations.md). diff --git a/docs/OV_Runtime_UG/network_state_intro.md b/docs/OV_Runtime_UG/network_state_intro.md index 5d39b56d32d..cf76e83ac41 100644 --- a/docs/OV_Runtime_UG/network_state_intro.md +++ b/docs/OV_Runtime_UG/network_state_intro.md @@ -243,7 +243,7 @@ After applying the transformation, ReadValue operations can receive other operat 1. Get CNNNetwork. Either way is acceptable: - * [from IR or ONNX model](./Integrate_with_customer_application_new_API.md) + * [from IR or ONNX model](./integrate_with_your_application.md) * [from ov::Model](../OV_Runtime_UG/model_representation.md) 2. Change the number of iterations inside TensorIterator/Loop nodes in the network using the [Reshape](ShapeInference.md) feature. @@ -347,7 +347,7 @@ After applying the transformation, ReadValue operations can receive other operat 1. Get CNNNetwork. Either way is acceptable: - * [from IR or ONNX model](./Integrate_with_customer_application_new_API.md) + * [from IR or ONNX model](./integrate_with_your_application.md) * [from ov::Model](../OV_Runtime_UG/model_representation.md) 2. [Reshape](ShapeInference.md) the CNNNetwork network if necessary. **Necessary case:** where the sequence_lengths dimension of input > 1, it means TensorIterator layer will have number_iterations > 1. We should reshape the inputs of the network to set sequence_dimension to exactly 1. diff --git a/docs/OV_Runtime_UG/openvino_intro.md b/docs/OV_Runtime_UG/openvino_intro.md index 10aeb252855..d5f79b4be5f 100644 --- a/docs/OV_Runtime_UG/openvino_intro.md +++ b/docs/OV_Runtime_UG/openvino_intro.md @@ -8,9 +8,8 @@ :maxdepth: 1 :hidden: - openvino_docs_IE_DG_Integrate_with_customer_application_new_API + openvino_docs_Integrate_OV_with_your_application - openvino_docs_OV_Runtime_UG_Model_Representation openvino_docs_IE_DG_ShapeInference openvino_docs_OV_UG_Working_with_devices openvino_docs_OV_Runtime_UG_Preprocessing_Overview diff --git a/docs/OV_Runtime_UG/ov_infer_request.md b/docs/OV_Runtime_UG/ov_infer_request.md new file mode 100644 index 00000000000..c984a4e6a92 --- /dev/null +++ b/docs/OV_Runtime_UG/ov_infer_request.md @@ -0,0 +1,277 @@ +# OpenVINO™ Inference Request {#openvino_docs_OV_Runtime_UG_Infer_request} + +OpenVINO™ Runtime uses Infer Request mechanism which allows to run models on different devices in asynchronous or synchronous manners. +`ov::InferRequest` class is used for this purpose inside the OpenVINO™ Runtime. +This class allows to set and get data for model inputs, outputs and run inference for the model. + +## Creating Infer Request + +`ov::InferRequest` can be created from the `ov::CompiledModel`: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [create_infer_request] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [create_infer_request] + +@endsphinxdirective + +## Run inference + +`ov::InferRequest` supports synchronous and asynchronous modes for inference. + +### Synchronous mode + +You can use `ov::InferRequest::infer()`, which blocks the application execution, to infer model in synchronous mode: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [sync_infer] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [sync_infer] + +@endsphinxdirective + +### Asynchronous mode + +Asynchronous mode can improve overall frame-rate of the application, because rather than wait for inference to complete, the app can continue doing things on the host, while accelerator is busy. You can use `ov::InferRequest::start_async()` to infer model in asynchronous mode: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [async_infer] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [async_infer] + +@endsphinxdirective + +Asynchronous mode supports two ways to wait inference results: + * `ov::InferRequest::wait_for()` - specify maximum duration in milliseconds to block for. The method is blocked until the specified timeout has elapsed, or the result becomes available, whichever comes first. + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [wait_for] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [wait_for] + + @endsphinxdirective + * `ov::InferRequest::wait()` - waits until inference result becomes available + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [wait] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [wait] + + @endsphinxdirective + +Both requests are thread-safe: can be called from different threads without fearing corruption and failures. + +Also InferRequest provides an functionality which allows to avoid a call of `ov::InferRequest::wait()`, in order to do it, you can use `ov::InferRequest::set_callback()` method. This method allows to set callback which will be called after completing run of InferRequest, please use weak reference of infer_request (`ov::InferRequest*`, `ov::InferRequest&`, `std::weal_ptr` and etc) in the callback, it is needed to avoid cyclic references. +For more details please take a look too [Classification Sample Async](../../samples/cpp/classification_sample_async/README.md). + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [set_callback] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [set_callback] + +@endsphinxdirective + +You can use `ov::InferRequest::cancel()` method in case if you want to cancel the current inference request: + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [cancel] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [cancel] + +@endsphinxdirective + +## Working with Input and Output tensors + +`ov::InferRequest` allows to get input/output tensors by tensor name, index, port and without any arguments in case if model has only one input or output. + + * `ov::InferRequest::get_input_tensor()`, `ov::InferRequest::set_input_tensor()`, `ov::InferRequest::get_output_tensor()`, `ov::InferRequest::set_output_tensor()` methods without arguments can be used to get or set input/output tensor for model with only one input/output: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [get_set_one_tensor] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [get_set_one_tensor] + + @endsphinxdirective + + * `ov::InferRequest::get_input_tensor()`, `ov::InferRequest::set_input_tensor()`, `ov::InferRequest::get_output_tensor()`, `ov::InferRequest::set_output_tensor()` methods with argument can be used to get or set input/output tensor by input/output index: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [get_set_index_tensor] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [get_set_index_tensor] + + @endsphinxdirective + + * `ov::InferRequest::get_tensor()`, `ov::InferRequest::set_tensor()` methods can be used to get or set input/output tensor by tensor name: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [get_set_tensor] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [get_set_tensor] + + @endsphinxdirective + + * `ov::InferRequest::get_tensor()`, `ov::InferRequest::set_tensor()` methods can be used to get or set input/output tensor by port: + @sphinxdirective + + .. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [get_set_tensor_by_port] + + .. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [get_set_tensor_by_port] + + @endsphinxdirective + +## Examples of InferRequest usages + +### Cascade of models + +`ov::InferRequest` can be used to organize cascade of models. You need to have infer requests for each model. +In this case you can get output tensor from the first request using `ov::InferRequest::get_tensor()` and set it as input for the second request using `ov::InferRequest::set_tensor()`. But be careful, shared tensors across compiled models can be rewritten by the first model if the first infer request is run once again, while the second model has not started yet. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [cascade_models] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [cascade_models] + +@endsphinxdirective + +### Using of ROI tensors + +It is possible to re-use shared input by several models. You do not need to allocate separate input tensor for a model if it processes a ROI object located inside of already allocated input of a previous model. For instance, when first model detects objects on a video frame (stored as input tensor) and second model accepts detected bounding boxes (ROI inside of the frame) as input. In this case, it is allowed to re-use pre-allocated input tensor (used by first model) by second model and just crop ROI without allocation of new memory using `ov::Tensor()` with passing of `ov::Tensor` and `ov::Coordinate` as parameters. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [roi_tensor] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [roi_tensor] + +@endsphinxdirective + +### Using of remote tensors + +You can create a remote tensor to work with remote device memory. `ov::RemoteContext` allows to create remote tensor. + +@sphinxdirective + +.. tab:: C++ + + .. doxygensnippet:: docs/snippets/ov_infer_request.cpp + :language: cpp + :fragment: [remote_tensor] + +.. tab:: Python + + .. doxygensnippet:: docs/snippets/ov_infer_request.py + :language: python + :fragment: [remote_tensor] + +@endsphinxdirective diff --git a/docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md b/docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md index c8914472035..0c200186708 100644 --- a/docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md +++ b/docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md @@ -116,7 +116,7 @@ the supported output precision depends on the actual underlying devices. _Gener |Layout |NCDHW|NCHW |CHW |NC |C | For setting relevant configuration, refer to the -[Integrate with Customer Application New Request API](../Integrate_with_customer_application_new_API.md) topic +[Integrate with Customer Application](../integrate_with_your_application.md) topic (step 3 "Configure input and output"). ### Supported Layers diff --git a/docs/index.rst b/docs/index.rst index 46b71ca485d..bd472b359b7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -73,8 +73,8 @@ OpenVINO™ Documentation

Learn about the alternative, web-based version of OpenVINO. DL Workbench container installation Required.

-

Inference Engine

-

Learn about OpenVINO's inference mechanism which executes the IR and ONNX models on target devices.

+

OpenVINO™ Runtime

+

Learn about OpenVINO's inference mechanism which executes the IR, ONNX, Paddle models on target devices.

Tune & Optimize

diff --git a/docs/snippets/CMakeLists.txt b/docs/snippets/CMakeLists.txt index ce32a90200e..889936e8579 100644 --- a/docs/snippets/CMakeLists.txt +++ b/docs/snippets/CMakeLists.txt @@ -58,3 +58,22 @@ if(NOT MSVC) endif() target_link_libraries(${TARGET_NAME} PRIVATE openvino::runtime openvino::runtime::dev) + +set(TARGET_NAME "ov_integration_snippet") +# [cmake:integration_example] +cmake_minimum_required(VERSION 3.10) +set(CMAKE_CXX_STANDARD 11) + + +find_package(OpenVINO REQUIRED) + +add_executable(${TARGET_NAME} src/main.cpp) + +target_link_libraries(${TARGET_NAME} PRIVATE openvino::runtime) +# [cmake:integration_example] +if(NOT MSVC) + target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-variable) + if(CMAKE_COMPILER_IS_GNUCXX) + target_compile_options(${TARGET_NAME} PRIVATE -Wno-unused-but-set-variable) + endif() +endif() diff --git a/docs/snippets/Integrate_with_customer_application_new_API.cpp b/docs/snippets/Integrate_with_customer_application_new_API.cpp deleted file mode 100644 index 3bee3d4df4c..00000000000 --- a/docs/snippets/Integrate_with_customer_application_new_API.cpp +++ /dev/null @@ -1,133 +0,0 @@ -#include - -int main() { -const std::string output_name = "output_name"; -const std::string input_name = "input_name"; -//! [part0] -InferenceEngine::Core core; -InferenceEngine::CNNNetwork network; -InferenceEngine::ExecutableNetwork executable_network; -//! [part0] - -//! [part1] -network = core.ReadNetwork("Model.xml"); -//! [part1] - -//! [part2] -network = core.ReadNetwork("model.onnx"); -//! [part2] - -//! [part3] -/** Take information about all topology inputs **/ -InferenceEngine::InputsDataMap input_info = network.getInputsInfo(); -/** Take information about all topology outputs **/ -InferenceEngine::OutputsDataMap output_info = network.getOutputsInfo(); -//! [part3] - -//! [part4] -/** Iterate over all input info**/ -for (auto &item : input_info) { - auto input_data = item.second; - input_data->setPrecision(InferenceEngine::Precision::U8); - input_data->setLayout(InferenceEngine::Layout::NCHW); - input_data->getPreProcess().setResizeAlgorithm(InferenceEngine::RESIZE_BILINEAR); - input_data->getPreProcess().setColorFormat(InferenceEngine::ColorFormat::RGB); -} -/** Iterate over all output info**/ -for (auto &item : output_info) { - auto output_data = item.second; - output_data->setPrecision(InferenceEngine::Precision::FP32); - output_data->setLayout(InferenceEngine::Layout::NC); -} -//! [part4] - -//! [part5] -executable_network = core.LoadNetwork(network, "CPU"); -//! [part5] - -//! [part6] -/** Optional config. E.g. this enables profiling of performance counters. **/ -std::map config = {{ InferenceEngine::PluginConfigParams::KEY_PERF_COUNT, InferenceEngine::PluginConfigParams::YES }}; -executable_network = core.LoadNetwork(network, "CPU", config); -//! [part6] - -//! [part7] -auto infer_request = executable_network.CreateInferRequest(); -//! [part7] - -auto infer_request1 = executable_network.CreateInferRequest(); -auto infer_request2 = executable_network.CreateInferRequest(); - -//! [part8] -/** Iterate over all input blobs **/ -for (auto & item : input_info) { - auto input_name = item.first; - /** Get input blob **/ - auto input = infer_request.GetBlob(input_name); - /** Fill input tensor with planes. First b channel, then g and r channels **/ -// ... -} -//! [part8] - -//! [part9] -auto output = infer_request1.GetBlob(output_name); -infer_request2.SetBlob(input_name, output); -//! [part9] - -//! [part10] -/** inputBlob points to input of a previous network and - cropROI contains coordinates of output bounding box **/ -InferenceEngine::Blob::Ptr inputBlob; -InferenceEngine::ROI cropRoi; -//... - -/** roiBlob uses shared memory of inputBlob and describes cropROI - according to its coordinates **/ -auto roiBlob = InferenceEngine::make_shared_blob(inputBlob, cropRoi); -infer_request2.SetBlob(input_name, roiBlob); -//! [part10] - -//! [part11] -/** Iterate over all input blobs **/ -for (auto & item : input_info) { - auto input_data = item.second; - /** Create input blob **/ - InferenceEngine::TBlob::Ptr input; - // assuming input precision was asked to be U8 in prev step - input = InferenceEngine::make_shared_blob( - InferenceEngine::TensorDesc(InferenceEngine::Precision::U8, input_data->getTensorDesc().getDims(), - input_data->getTensorDesc().getLayout())); - input->allocate(); - infer_request.SetBlob(item.first, input); - - /** Fill input tensor with planes. First b channel, then g and r channels **/ -// ... -} -//! [part11] - -//! [part12] -infer_request.StartAsync(); -infer_request.Wait(InferenceEngine::InferRequest::WaitMode::RESULT_READY); -//! [part12] - -auto sync_infer_request = executable_network.CreateInferRequest(); - -//! [part13] -sync_infer_request.Infer(); -//! [part13] - -//! [part14] - for (auto &item : output_info) { - auto output_name = item.first; - auto output = infer_request.GetBlob(output_name); - { - auto const memLocker = output->cbuffer(); // use const memory locker - // output_buffer is valid as long as the lifetime of memLocker - const float *output_buffer = memLocker.as(); - /** output_buffer[] - accessing output blob data **/ - } - } -//! [part14] - -return 0; -} diff --git a/docs/snippets/example_ngraph_utils.cpp b/docs/snippets/example_ngraph_utils.cpp index 185e9f425cf..6cf9faf21a4 100644 --- a/docs/snippets/example_ngraph_utils.cpp +++ b/docs/snippets/example_ngraph_utils.cpp @@ -20,56 +20,6 @@ #include // ! [ov:include] - -// ! [ov:create_simple_model] -std::shared_ptr create_simple_function() { - // This example shows how to create ov::Function - // - // Parameter--->Multiply--->Add--->Result - // Constant---' / - // Constant---' - - // Create opset8::Parameter operation with static shape - auto data = std::make_shared(ov::element::f32, ov::Shape{3, 1, 2}); - - auto mul_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {1.5}); - auto mul = std::make_shared(data, mul_constant); - - auto add_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {0.5}); - auto add = std::make_shared(mul, add_constant); - - // Create opset8::Result operation - auto res = std::make_shared(mul); - - // Create nGraph function - return std::make_shared(ov::ResultVector{res}, ov::ParameterVector{data}); -} -// ! [ov:create_simple_model] - -// ! [ov:create_advanced_model] -std::shared_ptr create_advanced_function() { - // Advanced example with multi output operation - // - // Parameter->Split---0-->Result - // | `--1-->Relu-->Result - // `----2-->Result - - auto data = std::make_shared(ov::element::f32, ov::Shape{1, 3, 64, 64}); - - // Create Constant for axis value - auto axis_const = ov::opset8::Constant::create(ov::element::i64, ov::Shape{}/*scalar shape*/, {1}); - - // Create opset8::Split operation that splits input to three slices across 1st dimension - auto split = std::make_shared(data, axis_const, 3); - - // Create opset8::Relu operation that takes 1st Split output as input - auto relu = std::make_shared(split->output(1)/*specify explicit output*/); - - // Results operations will be created automatically based on provided OutputVector - return std::make_shared(ov::OutputVector{split->output(0), relu, split->output(2)}, ov::ParameterVector{data}); -} -// ! [ov:create_advanced_model] - bool ngraph_api_examples(std::shared_ptr node) { { // ! [ngraph:ports_example] @@ -95,16 +45,6 @@ auto consumers = output.get_target_inputs(); // ! [ngraph:ports_example] } -{ -// ! [ngraph:shape] -auto partial_shape = node->input(0).get_partial_shape(); // get zero input partial shape -if (partial_shape.is_dynamic() /* or !partial_shape.is_static() */) { - return false; -} -auto static_shape = partial_shape.get_shape(); -// ! [ngraph:shape] -} - { // ! [ngraph:shape_check] auto partial_shape = node->input(0).get_partial_shape(); // get zero input partial shape diff --git a/docs/snippets/ov_infer_request.cpp b/docs/snippets/ov_infer_request.cpp new file mode 100644 index 00000000000..42be537252e --- /dev/null +++ b/docs/snippets/ov_infer_request.cpp @@ -0,0 +1,112 @@ +// Copyright (C) 2020 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +// ! [ov:include] +#include +// ! [ov:include] + +int main() { +ov::Core core; +std::shared_ptr model; +ov::CompiledModel compiled_model; + +//! [create_infer_request] +auto infer_request = compiled_model.create_infer_request(); +//! [create_infer_request] + +//! [sync_infer] +infer_request.infer(); +//! [sync_infer] + +//! [async_infer] +infer_request.start_async(); +//! [async_infer] + +//! [wait] +infer_request.wait(); +//! [wait] + +//! [wait_for] +infer_request.wait_for(std::chrono::milliseconds(10)); +//! [wait_for] + +//! [set_callback] +infer_request.set_callback([&](std::exception_ptr ex_ptr) { + if (!ex_ptr) { + // all done. Output data can be processed. + // You can fill the input data and run inference one more time: + infer_request.start_async(); + } else { + // Something wrong, you can analyze exception_ptr + } +}); +//! [set_callback] + +//! [cancel] +infer_request.cancel(); +//! [cancel] + +{ +//! [get_set_one_tensor] +auto input_tensor = infer_request.get_input_tensor(); +auto output_tensor = infer_request.get_output_tensor(); +//! [get_set_one_tensor] +} + +{ +//! [get_set_index_tensor] +auto input_tensor = infer_request.get_input_tensor(0); +auto output_tensor = infer_request.get_output_tensor(1); +//! [get_set_index_tensor] +} + +//! [get_set_tensor] +auto tensor1 = infer_request.get_tensor("tensor_name1"); +ov::Tensor tensor2; +infer_request.set_tensor("tensor_name2", tensor2); +//! [get_set_tensor] + +{ +//! [get_set_tensor_by_port] +auto input_port = model->input(0); +auto output_port = model->output("tensor_name"); +ov::Tensor input_tensor; +infer_request.set_tensor(input_port, input_tensor); +auto output_tensor = infer_request.get_tensor(output_port); +//! [get_set_tensor_by_port] +} + +auto infer_request1 = compiled_model.create_infer_request(); +auto infer_request2 = compiled_model.create_infer_request(); + +//! [cascade_models] +auto output = infer_request1.get_output_tensor(0); +infer_request2.set_input_tensor(0, output); +//! [cascade_models] + +//! [roi_tensor] +/** input_tensor points to input of a previous network and + cropROI contains coordinates of output bounding box **/ +ov::Tensor input_tensor(ov::element::f32, ov::Shape({1, 3, 20, 20})); +ov::Coordinate begin({0, 0, 0, 0}); +ov::Coordinate end({1, 2, 3, 3}); +//... + +/** roi_tensor uses shared memory of input_tensor and describes cropROI + according to its coordinates **/ +ov::Tensor roi_tensor(input_tensor, begin, end); +infer_request2.set_tensor("input_name", roi_tensor); +//! [roi_tensor] + +{ +//! [remote_tensor] +ov::RemoteContext context = core.get_default_context("GPU"); +auto input_port = compiled_model.input("tensor_name"); +ov::RemoteTensor remote_tensor = context.create_tensor(input_port.get_element_type(), input_port.get_shape()); +infer_request.set_tensor(input_port, remote_tensor); +//! [remote_tensor] +} + +return 0; +} diff --git a/docs/snippets/ov_infer_request.py b/docs/snippets/ov_infer_request.py new file mode 100644 index 00000000000..c18ea6316f2 --- /dev/null +++ b/docs/snippets/ov_infer_request.py @@ -0,0 +1,97 @@ +# Copyright (C) 2018-2022 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import numpy as np +#! [import] +import openvino.runtime as ov +#! [import] + +core = ov.Core() +model = core.read_model("model.xml") +compiled_model = core.compile_model(model, "AUTO") + +#! [create_infer_request] +infer_request = compiled_model.create_infer_request() +#! [create_infer_request] + +#! [sync_infer] +infer_request.infer() +#! [sync_infer] + +#! [async_infer] +infer_request.start_async() +#! [async_infer] + +#! [wait] +infer_request.wait() +#! [wait] + +#! [wait_for] +infer_request.wait_for(10) +#! [wait_for] + +#! [set_callback] +def callback(request, userdata): + request.start_async() + +infer_request.set_callback(callback) +#! [set_callback] + +#! [cancel] +infer_request.cancel() +#! [cancel] + +#! [get_set_one_tensor] +input_tensor = infer_request.get_input_tensor() +output_tensor = infer_request.get_output_tensor() +#! [get_set_one_tensor] + +#! [get_set_index_tensor] +input_tensor = infer_request.get_input_tensor(0) +output_tensor = infer_request.get_output_tensor(1) +#! [get_set_index_tensor] + +#! [get_set_name_tensor] +input_tensor = infer_request.get_tensor("input_name") +output_tensor = infer_request.get_tensor("output_name") +#! [get_set_name_tensor] + +#! [get_set_tensor] +tensor1 = infer_request.get_tensor("tensor_name1") +tensor2 = ov.Tensor() +infer_request.set_tensor("tensor_name2", tensor2) +#! [get_set_tensor] + +#! [get_set_tensor_by_port] +input_port = model.input(0) +output_port = model.input("tensor_name") +input_tensor = ov.Tensor() +infer_request.set_tensor(input_port, input_tensor) +output_tensor = infer_request.get_tensor(output_port) +#! [get_set_tensor_by_port] + +infer_request1 = compiled_model.create_infer_request() +infer_request2 = compiled_model.create_infer_request() + +#! [cascade_models] +output = infer_request1.get_output_tensor(0) +infer_request2.set_input_tensor(0, output) +#! [cascade_models] + +#! [roi_tensor] +# input_tensor points to input of a previous network and +# cropROI contains coordinates of output bounding box **/ +input_tensor = ov.Tensor(type=ov.Type.f32, shape=ov.Shape([1, 3, 20, 20])) +begin = [0, 0, 0, 0] +end = [1, 2, 3, 3] +# ... + +# roi_tensor uses shared memory of input_tensor and describes cropROI +# according to its coordinates **/ +roi_tensor = ov.Tensor(input_tensor, begin, end) +infer_request2.set_tensor("input_name", roi_tensor) +#! [roi_tensor] + +#! [remote_tensor] +# NOT SUPPORTED +#! [remote_tensor] diff --git a/docs/snippets/ov_model_snippets.cpp b/docs/snippets/ov_model_snippets.cpp index e5ccca530c6..12a8c732fc3 100644 --- a/docs/snippets/ov_model_snippets.cpp +++ b/docs/snippets/ov_model_snippets.cpp @@ -1,7 +1,14 @@ -// Copyright (C) 2018-2021 Intel Corporation +// Copyright (C) 2020 Intel Corporation // SPDX-License-Identifier: Apache-2.0 // +// ! [ov:include] +#include +#include +// ! [ov:include] + +#include +#include #include #include #include @@ -9,6 +16,107 @@ #include #include + +// ! [ov:create_simple_model] +std::shared_ptr create_simple_model() { + // This example shows how to create ov::Model + // + // Parameter--->Multiply--->Add--->Result + // Constant---' / + // Constant---' + + // Create opset8::Parameter operation with static shape + auto data = std::make_shared(ov::element::f32, ov::Shape{3, 1, 2}); + + auto mul_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {1.5}); + auto mul = std::make_shared(data, mul_constant); + + auto add_constant = ov::opset8::Constant::create(ov::element::f32, ov::Shape{1}, {0.5}); + auto add = std::make_shared(mul, add_constant); + + // Create opset8::Result operation + auto res = std::make_shared(mul); + + // Create nGraph function + return std::make_shared(ov::ResultVector{res}, ov::ParameterVector{data}); +} +// ! [ov:create_simple_model] + +// ! [ov:create_advanced_model] +std::shared_ptr create_advanced_model() { + // Advanced example with multi output operation + // + // Parameter->Split---0-->Result + // | `--1-->Relu-->Result + // `----2-->Result + + auto data = std::make_shared(ov::element::f32, ov::Shape{1, 3, 64, 64}); + + // Create Constant for axis value + auto axis_const = ov::opset8::Constant::create(ov::element::i64, ov::Shape{} /*scalar shape*/, {1}); + + // Create opset8::Split operation that splits input to three slices across 1st dimension + auto split = std::make_shared(data, axis_const, 3); + + // Create opset8::Relu operation that takes 1st Split output as input + auto relu = std::make_shared(split->output(1) /*specify explicit output*/); + + // Results operations will be created automatically based on provided OutputVector + return std::make_shared(ov::OutputVector{split->output(0), relu, split->output(2)}, + ov::ParameterVector{data}); +} +// ! [ov:create_advanced_model] + +void ov_api_examples() { + std::shared_ptr node = std::make_shared(ov::element::f32, ov::PartialShape{ov::Dimension::dynamic(), 3, 64, 64}); + + // ! [ov:partial_shape] + ov::Shape static_shape; + ov::PartialShape partial_shape = node->output(0).get_partial_shape(); // get zero output partial shape + if (!partial_shape.is_dynamic() /* or partial_shape.is_static() */) { + static_shape = partial_shape.get_shape(); + } + // ! [ov:partial_shape] +} + +// ! [ov:serialize] +void serialize_example(const std::shared_ptr& f) { + // Need include: + // * openvino/pass/manager.hpp + // * openvino/pass/serialize.hpp + ov::pass::Manager manager; + + // Serialize ov::Model to IR + manager.register_pass("/path/to/file/model.xml", "/path/to/file/model.bin"); + + manager.run_passes(f); +} +// ! [ov:serialize] + +// ! [ov:visualize] +void visualize_example(const std::shared_ptr& m) { + // Need include: + // * openvino/pass/manager.hpp + // * openvino/pass/visualize_tree.hpp + ov::pass::Manager manager; + + // Serialize ov::Model to before.svg file before transformation + manager.register_pass("image.svg"); + + manager.run_passes(m); +} +// ! [ov:visualize] + +void model_inputs() { +std::shared_ptr model; +//! [all_inputs_ouputs] +/* Take information about all topology inputs */ +auto inputs = model->inputs(); +/* Take information about all topology outputs */ +auto outputs = model->outputs(); +//! [all_inputs_ouputs] +} + void pattern_matcher_examples(std::shared_ptr node) { { // ! [pattern:simple_example] diff --git a/docs/snippets/ov_model_snippets.py b/docs/snippets/ov_model_snippets.py new file mode 100644 index 00000000000..575ab07e80e --- /dev/null +++ b/docs/snippets/ov_model_snippets.py @@ -0,0 +1,88 @@ +# Copyright (C) 2018-2022 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import numpy as np +#! [import] +import openvino.runtime as ov +#! [import] +import openvino.runtime.passes as passes + +# ! [ov:create_simple_model] +def create_simple_model(): + # This example shows how to create ov::Function + # + # Parameter--->Multiply--->Add--->Result + # Constant---' / + # Constant---' + data = ov.opset8.parameter([3, 1, 2], ov.Type.f32) + mul_constant = ov.opset8.constant([1.5], ov.Type.f32) + mul = ov.opset8.multiply(data, mul_constant) + add_constant = ov.opset8.constant([0.5], ov.Type.f32) + add = ov.opset8.add(mul, add_constant) + res = ov.opset8.result(add) + return ov.Model([res], [data], "model") +# ! [ov:create_simple_model] + +# ! [ov:create_advanced_model] +def create_advanced_model(): + # Advanced example with multi output operation + # + # Parameter->Split---0-->Result + # | `--1-->Relu-->Result + # `----2-->Result + data = ov.opset8.parameter(ov.Shape([1, 3, 64, 64]), ov.Type.f32) + # Create Constant for axis value + axis_const = ov.opset8.constant(ov.Type.i64, ov.Shape({}), [1]) + + # Create opset8::Split operation that splits input to three slices across 1st dimension + split = ov.opset8.split(data, axis_const, 3) + + # Create opset8::Relu operation that takes 1st Split output as input + relu = ov.opset8.relu(split.output(1)) + + # Results operations will be created automatically based on provided OutputVector + return ov.Model([split.output(0), relu, split.output[2]], [data], "model") +# ! [ov:create_advanced_model] + +def ov_api_examples(): + # Doesn't work + # node = ov.opset8.parameter(ov.PartialShape([ov.Dimension.dynamic(), 3, 64, 64]), np.float32) + node = ov.opset8.parameter(ov.PartialShape([ov.Dimension.dynamic(), ov.Dimension(3), ov.Dimension(64), ov.Dimension(64)]), np.float32) + + # it doesn't work: + # static_shape = ov.Shape() + # ! [ov:partial_shape] + partial_shape = node.output(0).get_partial_shape() # get zero output partial shape + if not partial_shape.is_dynamic: # or partial_shape.is_static + static_shape = partial_shape.get_shape() + # ! [ov:partial_shape] + +# ! [ov:serialize] +def serialize_example(m : ov.Model): + # Need import: + # * import openvino.runtime.passes as passes + pass_manager = passes.Manager() + pass_manager.register_pass(pass_name="Serialize", xml_path='model.xml', bin_path='model.bin') + pass_manager.run_passes(m) +# ! [ov:serialize] + +# ! [ov:visualize] +def visualize_example(m : ov.Model): + # Need import: + # * import openvino.runtime.passes as passes + pass_manager = passes.Manager() + pass_manager.register_pass(pass_name="VisualTree", file_name='image.svg') + pass_manager.run_passes(m) +# ! [ov:visualize] + +def model_inputs_outputs(model : ov.Model): + #! [all_inputs_ouputs] + inputs = model.inputs + outputs = model.outputs + #! [all_inputs_ouputs] + + +if __name__ == '__main__': + ov_api_examples() + create_simple_model() + create_advanced_model() diff --git a/docs/snippets/src/main.cpp b/docs/snippets/src/main.cpp new file mode 100644 index 00000000000..c600e063865 --- /dev/null +++ b/docs/snippets/src/main.cpp @@ -0,0 +1,69 @@ +// Copyright (C) 2018-2022 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +//! [include] +#include +//! [include] + +int main() { +//! [part1] +ov::Core core; +//! [part1] + +ov::CompiledModel compiled_model; +{ +//! [part2_1] +ov::CompiledModel compiled_model = core.compile_model("model.xml", "AUTO"); +//! [part2_1] +} +{ +//! [part2_2] +ov::CompiledModel compiled_model = core.compile_model("model.onnx", "AUTO"); +//! [part2_2] +} +{ +//! [part2_3] +ov::CompiledModel compiled_model = core.compile_model("model.pdmodel", "AUTO"); +//! [part2_3] +} +{ +//! [part2_4] +auto create_model = []() { + std::shared_ptr model; + // To construct a model, please follow + // https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_UG_Model_Representation.html + return model; +}; +std::shared_ptr model = create_model(); +compiled_model = core.compile_model(model, "AUTO"); +//! [part2_4] +} + +//! [part3] +ov::InferRequest infer_request = compiled_model.create_infer_request(); +//! [part3] + +void * memory_ptr = nullptr; +//! [part4] +// Get input port for model with one input +auto input_port = compiled_model.input(); +// Create tensor from external memory +ov::Tensor input_tensor(input_port.get_element_type(), input_port.get_shape(), memory_ptr); +// Set input tensor for model with one input +infer_request.set_input_tensor(input_tensor); +//! [part4] + +//! [part5] +infer_request.start_async(); +infer_request.wait(); +//! [part5] + +//! [part6] +// Get output tensor by tensor name +auto output = infer_request.get_tensor("tensor_name"); +const float *output_buffer = output.data(); +/* output_buffer[] - accessing output tensor data */ +//! [part6] +return 0; +} diff --git a/docs/snippets/src/main.py b/docs/snippets/src/main.py new file mode 100644 index 00000000000..a86986d56be --- /dev/null +++ b/docs/snippets/src/main.py @@ -0,0 +1,58 @@ +# Copyright (C) 2018-2022 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import numpy as np +#! [import] +import openvino.runtime as ov +#! [import] + +#! [part1] +core = ov.Core() +#! [part1] + +#! [part2_1] +compiled_model = core.compile_model("model.xml", "AUTO") +#! [part2_1] +#! [part2_2] +compiled_model = core.compile_model("model.onnx", "AUTO") +#! [part2_2] +#! [part2_3] +compiled_model = core.compile_model("model.pdmodel", "AUTO") +#! [part2_3] +#! [part2_4] +def create_model(): + # This example shows how to create ov::Function + # + # To construct a model, please follow + # https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_UG_Model_Representation.html + data = ov.opset8.parameter([3, 1, 2], ov.Type.f32) + res = ov.opset8.result(data) + return ov.Model([res], [data], "model") + +model = create_model() +compiled_model = core.compile_model(model, "AUTO") +#! [part2_4] + +#! [part3] +infer_request = compiled_model.create_infer_request() +#! [part3] + +memory = np.array([1, 2, 3, 4]) +#! [part4] +# Create tensor from external memory +input_tensor = ov.Tensor(array=memory, shared_memory=True) +# Set input tensor for model with one input +infer_request.set_input_tensor(input_tensor) +#! [part4] + +#! [part5] +infer_request.start_async() +infer_request.wait() +#! [part5] + +#! [part6] +# Get output tensor for model with one output +output = infer_request.get_output_tensor() +output_buffer = output.data +# output_buffer[] - accessing output tensor data +#! [part6] diff --git a/samples/c/hello_classification/README.md b/samples/c/hello_classification/README.md index f9daa132d69..a77c10f5f5f 100644 --- a/samples/c/hello_classification/README.md +++ b/samples/c/hello_classification/README.md @@ -26,7 +26,7 @@ Upon the start-up, the sample application reads command line parameters, loads s Then, the sample creates an synchronous inference request object. When inference is done, the application outputs data to the standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -92,8 +92,8 @@ This sample is an API example, for any performance measurements please use the d ## See Also -- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) -- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) +- [Integrate OpenVINO™ into Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) +- [Using OpenVINO™ Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/c/hello_nv12_input_classification/README.md b/samples/c/hello_nv12_input_classification/README.md index a3879a129d0..da71ed74e7f 100644 --- a/samples/c/hello_nv12_input_classification/README.md +++ b/samples/c/hello_nv12_input_classification/README.md @@ -25,7 +25,7 @@ image in the NV12 color format to an Inference Engine plugin. Then, the sample c application outputs data to the standard output stream. You can see the explicit description of -each sample step at [Integration Steps](https://docs.openvino.ai/latest/openvino_docs_IE_DG_Integrate_with_customer_application_new_API.html) section of "Integrate the Inference Engine with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -107,8 +107,8 @@ This sample is an API example, for any performance measurements please use the d ## See Also -- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) -- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) +- [Integrate the OpenVINO™ into Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) +- [Using OpenVINO™ Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/classification_sample_async/README.md b/samples/cpp/classification_sample_async/README.md index c9f601cc4c8..fbddfc4df2b 100644 --- a/samples/cpp/classification_sample_async/README.md +++ b/samples/cpp/classification_sample_async/README.md @@ -36,7 +36,7 @@ After that, the application starts inference for the first infer request and wai When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -172,7 +172,7 @@ classid probability ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/hello_classification/README.md b/samples/cpp/hello_classification/README.md index 27af315a355..f0ee0f343b0 100644 --- a/samples/cpp/hello_classification/README.md +++ b/samples/cpp/hello_classification/README.md @@ -26,7 +26,7 @@ The following C++ API is used in the application: At startup, the sample application reads command line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin and performs synchronous inference. Then processes output data and write it to a standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -116,7 +116,7 @@ classid probability ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/hello_nv12_input_classification/README.md b/samples/cpp/hello_nv12_input_classification/README.md index 83fd42793a0..c1a78d56893 100644 --- a/samples/cpp/hello_nv12_input_classification/README.md +++ b/samples/cpp/hello_nv12_input_classification/README.md @@ -25,7 +25,7 @@ Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](.. At startup, the sample application reads command line parameters, loads the specified model and an image in the NV12 color format to an OpenVINO™ Runtime plugin. Then, the sample creates an synchronous inference request object. When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output. -You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -130,7 +130,7 @@ classid probability ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/hello_query_device/README.md b/samples/cpp/hello_query_device/README.md index d783f234627..202f733542b 100644 --- a/samples/cpp/hello_query_device/README.md +++ b/samples/cpp/hello_query_device/README.md @@ -90,5 +90,5 @@ The application prints all available devices with their supported metrics and de ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) diff --git a/samples/cpp/hello_reshape_ssd/README.md b/samples/cpp/hello_reshape_ssd/README.md index ef9d21ba169..cdd0dcfe421 100644 --- a/samples/cpp/hello_reshape_ssd/README.md +++ b/samples/cpp/hello_reshape_ssd/README.md @@ -27,7 +27,7 @@ Upon the start-up the sample application reads command line parameters, loads sp Engine plugin. Then, the sample creates an synchronous inference request object. When inference is done, the application creates output image and output data to the standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -116,7 +116,7 @@ This sample is an API example, for any performance measurements please use the d ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/model_creation_sample/README.md b/samples/cpp/model_creation_sample/README.md index e0edc2c3625..542d6d82ec0 100644 --- a/samples/cpp/model_creation_sample/README.md +++ b/samples/cpp/model_creation_sample/README.md @@ -33,7 +33,7 @@ At startup, the sample application does the following: - Loads the model and input data to the OpenVINO™ Runtime plugin - Performs synchronous inference and processes output data, logging each step in a standard output stream -You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Building @@ -178,6 +178,6 @@ classid probability label ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/cpp/speech_sample/README.md b/samples/cpp/speech_sample/README.md index dde7d168d04..0f440ba767d 100644 --- a/samples/cpp/speech_sample/README.md +++ b/samples/cpp/speech_sample/README.md @@ -31,7 +31,7 @@ At startup, the sample application reads command-line parameters, loads a specif If the `-r` option is given, error statistics are provided for each speech utterance as shown above. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ### GNA-specific details @@ -237,7 +237,7 @@ All of mentioned files can be downloaded from [https://storage.openvinotoolkit.o ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/python/classification_sample_async/README.md b/samples/python/classification_sample_async/README.md index 98c38a47166..24f4095001a 100644 --- a/samples/python/classification_sample_async/README.md +++ b/samples/python/classification_sample_async/README.md @@ -23,7 +23,7 @@ Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Running @@ -135,7 +135,7 @@ The sample application logs each step in a standard output stream and outputs to ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/python/hello_classification/README.md b/samples/python/hello_classification/README.md index 71f15ba8315..a5dd902fc1b 100644 --- a/samples/python/hello_classification/README.md +++ b/samples/python/hello_classification/README.md @@ -24,7 +24,7 @@ The following Python API is used in the application: At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Running @@ -98,7 +98,7 @@ The sample application logs each step in a standard output stream and outputs to ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/python/hello_reshape_ssd/README.md b/samples/python/hello_reshape_ssd/README.md index ef16766af1f..f7bc8e6aba7 100644 --- a/samples/python/hello_reshape_ssd/README.md +++ b/samples/python/hello_reshape_ssd/README.md @@ -24,7 +24,7 @@ At startup, the sample application reads command-line parameters, prepares input As a result, the program creates an output image, logging each step in a standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Running @@ -86,7 +86,7 @@ The sample application logs each step in a standard output stream and creates an ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/python/model_creation_sample/README.md b/samples/python/model_creation_sample/README.md index d50cbf20a8e..068cb25894c 100644 --- a/samples/python/model_creation_sample/README.md +++ b/samples/python/model_creation_sample/README.md @@ -28,7 +28,7 @@ At startup, the sample application does the following: You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## Running @@ -127,7 +127,7 @@ The sample application logs each step in a standard output stream and outputs 10 ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/samples/python/speech_sample/README.md b/samples/python/speech_sample/README.md index 1f460379557..3c8dfa99b4c 100644 --- a/samples/python/speech_sample/README.md +++ b/samples/python/speech_sample/README.md @@ -29,7 +29,7 @@ Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample At startup, the sample application reads command-line parameters, loads a specified model and input data to the OpenVINO™ Runtime plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream. You can see the explicit description of -each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide. +each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) section of "Integrate OpenVINO™ Runtime with Your Application" guide. ## GNA-specific details @@ -328,7 +328,7 @@ The sample application logs each step in a standard output stream. ## See Also -- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) +- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/integrate_with_your_application.md) - [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md) - [Model Downloader](@ref omz_tools_downloader) - [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) diff --git a/tools/pot/openvino/tools/pot/api/README.md b/tools/pot/openvino/tools/pot/api/README.md index 728711284f3..06b9af6a165 100644 --- a/tools/pot/openvino/tools/pot/api/README.md +++ b/tools/pot/openvino/tools/pot/api/README.md @@ -52,7 +52,7 @@ The POT Python* API for model optimization can be used in the following cases: - [Accuracy Checker](@ref omz_tools_accuracy_checker) tool does not support the model or dataset. - POT does not support the model in the [Simplified Mode](@ref pot_docs_BestPractices) or produces the optimized model with low accuracy in this mode. -- You already have the Python* script to validate the accuracy of the model using the [OpenVINO™ Runtime](@ref openvino_docs_OV_Runtime_User_Guide). +- You already have the Python* script to validate the accuracy of the model using the [OpenVINO™ Runtime](@ref openvino_docs_OV_Runtime_User_Guide). ## Examples