Publishing 2019 R2 content (#223)

* Publishing 2019 R1.1 content and Myriad plugin sources

.gitmodules

@@ -1,3 +1,8 @@
 [submodule "inference-engine/thirdparty/ade"]
 	path = inference-engine/thirdparty/ade
 	url = https://github.com/opencv/ade.git
+	ignore = dirty
+[submodule "inference-engine/thirdparty/ngraph"]
+	path = inference-engine/thirdparty/ngraph
+	url = https://github.com/NervanaSystems/ngraph.git
+	ignore = dirty

README.md

@@ -1,5 +1,5 @@
 # [OpenVINO™ Toolkit](https://01.org/openvinotoolkit) - Deep Learning Deployment Toolkit repository
-[![Stable release](https://img.shields.io/badge/version-2019.R1-green.svg)](https://github.com/opencv/dldt/releases/tag/2019_R1)
+[![Stable release](https://img.shields.io/badge/version-2019.R2-green.svg)](https://github.com/opencv/dldt/releases/tag/2019_R2)
 [![Apache License Version 2.0](https://img.shields.io/badge/license-Apache_2.0-green.svg)](LICENSE)
 
 This toolkit allows developers to deploy pre-trained deep learning models through a high-level C++ Inference Engine API integrated with application logic. 
@@ -15,7 +15,12 @@ Deep Learning Deployment Toolkit is licensed under [Apache License Version 2.0](
 
 ## Documentation
 * [OpenVINO™ Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
-* Inference Engine [build instructions](inference-engine/README.md)
+* [Inference Engine build instructions](inference-engine/README.md)
+* [Get Started with Deep Learning Deployment Toolkit on Linux*](get-started-linux.md)
+* [Introduction to Deep Learning Deployment Toolkit](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Introduction.html)
+* [Inference Engine Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html)
+* [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
+
 
 ## How to Contribute
 We welcome community contributions to the Deep Learning Deployment Toolkit repository. If you have an idea how to improve the product, please share it with us doing the following steps:
@@ -29,7 +34,7 @@ Deep Learning Deployment Toolkit is licensed under Apache License, Version 2.0.
 
 ## Support
 Please report questions, issues and suggestions using:
-* [\#dldt](https://stackoverflow.com/search?q=%23dldt) tag on StackOverflow*
+* [\#openvino](https://stackoverflow.com/search?q=%23openvino) tag on StackOverflow*
 * [GitHub* Issues](https://github.com/opencv/dldt/issues) 
 * [Forum](https://software.intel.com/en-us/forums/computer-vision)
 

get-started-linux.md

@@ -0,0 +1,203 @@
+# Get Started with OpenVINO™ Deep Learning Deployment Toolkit (DLDT) on Linux*
+
+This guide provides you with the information that will help you to start using the DLDT on Linux*. With this guide you will learn how to:
+
+1. [Configure the Model Optimizer](#configure-the-model-optimizer)
+2. [Prepare a model for sample inference:](#prepare-a-model-for-sample-inference)
+   1. [Download a pre-trained model](#download-a-trained-model)
+   2. [Convert the model to an Intermediate Representation (IR) with the Model Optimizer](#convert-the-model-to-an-intermediate-representation-with-the-model-optimizer)
+3. [Run the Image Classification Sample Application with the model](#run-the-image-classification-sample-application)
+
+## Prerequisites
+1. This guide assumes that you have already cloned the `dldt` repo and successfully built the Inference Engine and Samples using the [build instructions](inference-engine/README.md). 
+2. The original structure of the repository directories is kept unchanged.
+
+> **NOTE**: Below, the directory to which the `dldt` repository is cloned is referred to as `<DLDT_DIR>`.  
+
+## Configure the Model Optimizer
+
+The Model Optimizer is a Python\*-based command line tool for importing trained models from popular deep learning frameworks such as Caffe\*, TensorFlow\*, Apache MXNet\*, ONNX\* and Kaldi\*.
+
+You cannot perform inference on your trained model without running the model through the Model Optimizer. When you run a pre-trained model through the Model Optimizer, your output is an Intermediate Representation (IR) of the network. The Intermediate Representation is a pair of files that describe the whole model:
+
+- `.xml`: Describes the network topology
+- `.bin`: Contains the weights and biases binary data
+
+For more information about the Model Optimizer, refer to the [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html). 
+
+### Model Optimizer Configuration Steps
+
+You can choose to either configure all supported frameworks at once **OR** configure one framework at a time. Choose the option that best suits your needs. If you see error messages, make sure you installed all dependencies.
+
+> **NOTE**: Since the TensorFlow framework is not officially supported on CentOS*, the Model Optimizer for TensorFlow can't be configured and ran on those systems.  
+
+> **IMPORTANT**: The Internet access is required to execute the following steps successfully. If you have access to the Internet through the proxy server only, please make sure that it is configured in your OS environment.
+
+**Option 1: Configure all supported frameworks at the same time**
+
+1.  Go to the Model Optimizer prerequisites directory:
+```sh
+cd <DLDT_DIR>/model_optimizer/install_prerequisites
+```
+2.  Run the script to configure the Model Optimizer for Caffe,
+    TensorFlow, MXNet, Kaldi\*, and ONNX:
+```sh
+sudo ./install_prerequisites.sh
+```
+
+**Option 2: Configure each framework separately**
+
+Configure individual frameworks separately **ONLY** if you did not select **Option 1** above.
+
+1.  Go to the Model Optimizer prerequisites directory:
+```sh
+cd <DLDT_DIR>/model_optimizer/install_prerequisites
+```
+2.  Run the script for your model framework. You can run more than one script:
+
+   - For **Caffe**:
+   ```sh
+   sudo ./install_prerequisites_caffe.sh
+   ```
+
+   - For **TensorFlow**:
+   ```sh
+   sudo ./install_prerequisites_tf.sh
+   ```
+
+   - For **MXNet**:
+   ```sh
+   sudo ./install_prerequisites_mxnet.sh
+   ```
+
+   - For **ONNX**:
+   ```sh
+   sudo ./install_prerequisites_onnx.sh
+   ```
+
+   - For **Kaldi**:
+   ```sh
+   sudo ./install_prerequisites_kaldi.sh
+   ```
+The Model Optimizer is configured for one or more frameworks. Continue to the next session to download and prepare a model for running a sample inference.
+
+## Prepare a Model for Sample Inference
+
+This paragraph contains the steps to get the pre-trained model for sample inference and to prepare the model's optimized Intermediate Representation that Inference Engine uses.
+
+### Download a Trained Model
+
+To run the Image Classification Sample you'll need a pre-trained model to run the inference on. This guide will use the public SqueezeNet 1.1 Caffe* model. You can find and download this model manually or use the OpenVINO™ [Model Downloader](https://github.com/opencv/open_model_zoo/tree/master/model_downloader). 
+
+With the Model Downloader, you can download other popular public deep learning topologies and the [OpenVINO™ pre-trained models](https://github.com/opencv/open_model_zoo/tree/master/intel_models) prepared for running inference for a wide list of inference scenarios: object detection, object recognition, object re-identification, human pose estimation, action recognition and others.
+
+To download the SqueezeNet 1.1 Caffe* model to a models folder with the Model Downloader:
+1. Install the [prerequisites](https://github.com/opencv/open_model_zoo/tree/master/model_downloader#prerequisites).
+2. Run the `downloader.py` with specifying the topology name and a `<models_dir>` path. For example to download the model to the `~/public_models` directory:
+   ```sh
+   ./downloader.py --name squeezenet1.1 --output_dir ~/public_models
+   ```
+   When the model files are successfully downloaded the output similar to the following is printed:
+   ```sh
+   ###############|| Downloading topologies ||###############
+
+   ========= Downloading /home/username/public_models/classification/squeezenet/1.1/caffe/squeezenet1.1.prototxt
+   
+   ========= Downloading /home/username/public_models/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel
+   ... 100%, 4834 KB, 3157 KB/s, 1 seconds passed
+
+   ###############|| Post processing ||###############
+
+   ========= Changing input dimensions in squeezenet1.1.prototxt =========
+   ```
+
+### Convert the model to an Intermediate Representation with the Model Optimizer
+
+> **NOTE**: This section assumes that you have configured the Model Optimizer using the instructions from the [Configure the Model Optimizer](#configure-the-model-optimizer) section.
+
+1. Create a `<ir_dir>` directory that will contains the Intermediate Representation (IR) of the model. 
+
+2. Inference Engine can perform inference on a [list of supported devices](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html) using specific device plugins. Different plugins support models of [different precision formats](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html#supported_model_formats), such as FP32, FP16, INT8. To prepare an IR to run inference on a particular hardware, run the Model Optimizer with the appropriate `--data_type` options:
+
+   **For CPU (FP32):**
+   ```sh  
+   python3 <DLDT_DIR>/model_optimizer/mo.py --input_model <models_dir>/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel --data_type FP32 --output_dir <ir_dir>
+   ```
+
+   **For GPU and MYRIAD (FP16):**
+   ```sh  
+   python3 <DLDT_DIR>/model_optimizer/mo.py --input_model <models_dir>/classification/squeezenet/1.1/caffe/squeezenet1.1.caffemodel --data_type FP16 --output_dir <ir_dir>
+   ``` 
+   After the Model Optimizer script is completed, the produced IR files (`squeezenet1.1.xml`, `squeezenet1.1.bin`) are in the specified `<ir_dir>` directory.
+
+3. Copy the `squeezenet1.1.labels` file from the `<DLDT_DIR>/inference-engine/samples/sample_data/` to the model IR directory. This file contains the classes that ImageNet uses so that the inference results show text instead of classification numbers:
+   ```sh
+   cp <DLDT_DIR>/inference-engine/samples/sample_data/squeezenet1.1.labels <ir_dir>
+   ```
+
+Now you are ready to run the Image Classification Sample Application.
+
+## Run the Image Classification Sample Application
+
+The Inference Engine sample applications are automatically compiled when you built the Inference Engine using the [build instructions](inference-engine/README.md). The binary files are located in the `<DLDT_DIR>/inference-engine/bin/intel64/Release` directory.
+
+Follow the steps below to run the Image Classification sample application on the prepared IR and with an input image: 
+
+1. Go to the samples build directory:
+   ```sh
+   cd <DLDT_DIR>/inference-engine/bin/intel64/Release
+   ```
+2. Run the sample executable with specifying the `car.png` file from the `<DLDT_DIR>/inference-engine/samples/sample_data/` directory as an input image, the IR of your model and a plugin for a hardware device to perform inference on:
+
+   **For CPU:**
+   ```sh
+   ./classification_sample -i <DLDT_DIR>/inference-engine/samples/sample_data/car.png -m <ir_dir>/squeezenet1.1.xml -d CPU
+   ```
+
+   **For GPU:**
+   ```sh
+   ./classification_sample -i <DLDT_DIR>/inference-engine/samples/sample_data/car.png -m <ir_dir>/squeezenet1.1.xml -d GPU
+   ```
+   
+   **For MYRIAD:** 
+   >**NOTE**: Running inference on VPU devices (Intel® Movidius™ Neural Compute Stick or Intel® Neural Compute Stick 2) with the MYRIAD plugin requires performing [additional hardware configuration steps](inference-engine/README.md#optional-additional-installation-steps-for-the-intel-movidius-neural-compute-stick-and-neural-compute-stick-2).
+   ```sh
+   ./classification_sample -i <DLDT_DIR>/inference-engine/samples/sample_data/car.png -m <ir_dir>/squeezenet1.1.xml -d MYRIAD
+   ```
+
+When the Sample Application completes, you will have the label and confidence for the top-10 categories printed on the screen. Below is a sample output with inference results on CPU:    
+```sh
+Top 10 results:
+
+Image /home/user/dldt/inference-engine/samples/sample_data/car.png
+
+classid probability label
+------- ----------- -----
+817     0.8363345   sports car, sport car
+511     0.0946488   convertible
+479     0.0419131   car wheel
+751     0.0091071   racer, race car, racing car
+436     0.0068161   beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
+656     0.0037564   minivan
+586     0.0025741   half track
+717     0.0016069   pickup, pickup truck
+864     0.0012027   tow truck, tow car, wrecker
+581     0.0005882   grille, radiator grille
+
+
+total inference time: 2.6642941
+Average running time of one iteration: 2.6642941 ms
+
+Throughput: 375.3339402 FPS
+
+[ INFO ] Execution successful
+```
+
+## Additional Resources
+
+* [OpenVINO™ Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
+* [Inference Engine build instructions](inference-engine/README.md)
+* [Introduction to Intel® Deep Learning Deployment Toolkit](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Introduction.html)
+* [Inference Engine Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html)
+* [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
+* [Inference Engine Samples Overview](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Samples_Overview.html). 

inference-engine/CMakeLists.txt

@@ -2,149 +2,52 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
-cmake_minimum_required(VERSION 3.8 FATAL_ERROR)
+if (APPLE)
+    # due to https://cmake.org/cmake/help/v3.12/policy/CMP0068.html
+    cmake_minimum_required(VERSION 3.9 FATAL_ERROR)
+else()
+    cmake_minimum_required(VERSION 3.7.2 FATAL_ERROR)
+endif()
 
 project(InferenceEngine)
 
-set(DEV_BUILD TRUE)
+set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake" ${CMAKE_MODULE_PATH})
+set(IE_MAIN_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 
 include(CTest)
+include(features)
 
-## WA for problem with gtest submodule. It cannot detect uint32 type.
-## remove Gtest submodule and this two lines together
-include (CheckTypeSize)
-check_type_size (uint32_t uint32_t LANGUAGE CXX)
+# include developer package
+include(developer_package)
 
-if (UNIX AND NOT APPLE)
-set(LINUX TRUE)
-endif()
+# These options are shared with 3rdparty plugins
+# by means of developer package
+include(check_features)
 
-option (OS_FOLDER "create OS dedicated folder in output" OFF)
-
-if(CMAKE_SYSTEM_PROCESSOR STREQUAL "armv7l")
-    set (ARCH_FOLDER armv7l)
-elseif("${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
-    set (ARCH_FOLDER intel64)
-else()
-    set (ARCH_FOLDER  ia32)
-endif()
-
-if (OS_FOLDER)
-	message ("**** OS FOLDER IS: [${OS_FOLDER}]")
-	if ("${OS_FOLDER}" STREQUAL "ON")
-		message ("**** USING OS FOLDER: [${CMAKE_SYSTEM_NAME}]")
-		set (BIN_FOLDER bin/${CMAKE_SYSTEM_NAME}/${ARCH_FOLDER})
-	else()
-		set (BIN_FOLDER bin/${OS_FOLDER}/${ARCH_FOLDER})
-	endif()
-else()
-    set (BIN_FOLDER bin/${ARCH_FOLDER})
-endif()
-
-set (IE_MAIN_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
-set (CMAKE_MODULE_PATH "${IE_MAIN_SOURCE_DIR}/cmake" ${CMAKE_MODULE_PATH})
-
-#printing debug messages
-include (debug)
-
-if("${CMAKE_BUILD_TYPE}" STREQUAL "")
-    debug_message(STATUS "CMAKE_BUILD_TYPE not defined, 'Release' will be used")
-    set(CMAKE_BUILD_TYPE "Release")
-endif()
-message(STATUS "BUILD_CONFIGURATION: ${CMAKE_BUILD_TYPE}")
-
-if(COVERAGE)
-    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0")
-endif()
-
-if (UNIX)
-    SET(LIB_DL ${CMAKE_DL_LIBS})
-endif()
-
-set (OUTPUT_ROOT ${IE_MAIN_SOURCE_DIR})
-
-include(os_flags)
-
-#resolving dependencies for the project
-include (dependencies)
-
-set(CMAKE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX})
-set(CMAKE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX})
-
-if (WIN32)
-    # Support CMake multiconfiguration for Visual Studio build
-    set(IE_BUILD_POSTFIX $<$<CONFIG:Debug>:${IE_DEBUG_POSTFIX}>$<$<CONFIG:Release>:${IE_RELEASE_POSTFIX}>)
-    set(IE_BUILD_CONFIGURATION $<CONFIG>)
-else ()
-    if (${CMAKE_BUILD_TYPE} STREQUAL "Debug" )
-        set(IE_BUILD_POSTFIX ${IE_DEBUG_POSTFIX})
-    else()
-        set(IE_BUILD_POSTFIX ${IE_RELEASE_POSTFIX})
-    endif()
-    set(IE_BUILD_CONFIGURATION ${CMAKE_BUILD_TYPE})
-endif()
-
-add_definitions(-DIE_BUILD_POSTFIX=\"${IE_BUILD_POSTFIX}\")
-
-if(NOT(UNIX))
-    if (WIN32)
-        #set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
-        #set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
-    endif()
-    set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (CMAKE_LIBRARY_PATH ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
-    set (LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}) # compatibility issue: linux uses LIBRARY_OUTPUT_PATH, windows uses LIBRARY_OUTPUT_DIRECTORY
-else ()
-    set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
-    set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
-    set (CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
-    set (CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
-    set (CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
-    set (LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
-    set (LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}/lib)
-endif()
-
-if (APPLE)
-    set(CMAKE_MACOSX_RPATH 1)
-endif(APPLE)
-
-#rpath fully disabled
-if (NOT ENABLE_PLUGIN_RPATH)
-    SET (CMAKE_SKIP_RPATH TRUE)
-endif()
-
-#Use solution folders.
-set_property(GLOBAL PROPERTY USE_FOLDERS ON)
-#message("=====================> ${CMAKE_BUILD_TYPE} <=====================")
+# resolving dependencies for the project
+include(dependencies)
 
 message (STATUS "PROJECT ............................... " ${PROJECT_NAME})
 message (STATUS "CMAKE_BINARY_DIR ...................... " ${CMAKE_BINARY_DIR})
 message (STATUS "IE_MAIN_SOURCE_DIR .................... " ${IE_MAIN_SOURCE_DIR})
 message (STATUS "CMAKE_GENERATOR ....................... " ${CMAKE_GENERATOR})
 message (STATUS "CMAKE_C_COMPILER_ID ................... " ${CMAKE_C_COMPILER_ID})
-
-include(sdl)
-
-set (CMAKE_POSITION_INDEPENDENT_CODE ON)
-
-include (sanitizer)
-
-include(CheckCXXCompilerFlag)
-
-include(cpplint)
+message (STATUS "CMAKE_BUILD_TYPE ...................... " ${CMAKE_BUILD_TYPE})
 
 add_subdirectory(src)
-add_subdirectory(tests)
-add_subdirectory(thirdparty)
-set(InferenceEngine_DIR "${CMAKE_BINARY_DIR}")
 
-#to be able to link
-set (LIB_FOLDER ${IE_MAIN_SOURCE_DIR}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
+if(ENABLE_TESTS)
+    add_subdirectory(tests)
+endif()
+
+add_subdirectory(thirdparty)
+
+add_subdirectory(tools)
+
+if (ENABLE_SAMPLES)
+    # hint for find_package(InferenceEngine in the samples folder)
+    set(InferenceEngine_DIR "${CMAKE_BINARY_DIR}")
+endif()
 
 # gflags and format_reader targets are kept inside of samples directory and
 # they must be built even if samples build is disabled (required for tests and tools).

inference-engine/README.md

@@ -1,5 +1,34 @@
-## Repository components
+# Build Inference Engine
 
+## Contents
+
+- [Introduction](#introduction)
+- [Build on Linux* Systems](#build-on-linux-systems)
+  - [Software Requirements](#software-requirements)
+  - [Build Steps](#build-steps)
+  - [Additional Build Options](#additional-build-options)
+- [Build for Raspbian* Stretch OS](#build-for-raspbian-stretch-os)
+  - [Hardware Requirements](#hardware-requirements)
+  - [Native Compilation](#native-compilation)
+  - [Cross Compilation Using Docker*](#cross-compilation-using-docker)
+  - [Additional Build Options](#additional-build-options-1)
+- [Build on Windows* Systems](#build-on-windows-systems)
+  - [Software Requirements](#software-requirements-1)
+  - [Build Steps](#build-steps-1)
+  - [Additional Build Options](#additional-build-options-2)
+  - [Building Inference Engine with Ninja* Build System](#building-inference-engine-with-ninja-build-system)
+- [Build on macOS* Systems](#build-on-macos-systems)
+  - [Software Requirements](#software-requirements-2)
+  - [Build Steps](#build-steps-2)
+  - [Additional Build Options](#additional-build-options-3)
+- [Use Custom OpenCV Builds for Inference Engine](#use-custom-opencv-builds-for-inference-engine)
+- [(Optional) Additional Installation Steps for the Intel® Movidius™ Neural Compute Stick and Neural Compute Stick 2](#optional-additional-installation-steps-for-the-intel-movidius-neural-compute-stick-and-neural-compute-stick-2)
+  - [For Linux, Raspbian Stretch* OS](#for-linux-raspbian-stretch-os)
+  - [For Windows](#for-windows-1)
+- [Next Steps](#next-steps)
+- [Additional Resources](#additional-resources)
+
+## Introduction
 The Inference Engine can infer models in different formats with various input and output formats.
 
 The open source version of Inference Engine includes the following plugins:
@@ -9,21 +38,22 @@ The open source version of Inference Engine includes the following plugins:
 | CPU plugin           | Intel® Xeon® with Intel® AVX2 and AVX512, Intel® Core™ Processors with Intel® AVX2, Intel® Atom® Processors with Intel® SSE |
 | GPU plugin           | Intel® Processor Graphics, including Intel® HD Graphics and Intel® Iris® Graphics |
 | GNA plugin           | Intel® Speech Enabling Developer Kit, Amazon Alexa* Premium Far-Field Developer Kit, Intel® Pentium® Silver processor J5005, Intel® Celeron® processor J4005, Intel® Core™ i3-8121U processor |
+| MYRIAD plugin        | Intel® Movidius™ Neural Compute Stick powered by the Intel® Movidius™ Myriad™ 2, Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X |
 | Heterogeneous plugin | Heterogeneous plugin enables computing for inference on one network on several Intel® devices. |
 
-Inference Engine plugins for Intel® FPGA and Intel® Movidius™ Neural Compute Stick are distributed only in a binary form as a part of [Intel® Distribution of OpenVINO™](https://software.intel.com/en-us/openvino-toolkit).
+Inference Engine plugin for Intel® FPGA is distributed only in a binary form as a part of [Intel® Distribution of OpenVINO™](https://software.intel.com/en-us/openvino-toolkit).
 
-## Build on Linux\* Systems
+## Build on Linux* Systems
 
 The software was validated on:
 - Ubuntu\* 16.04 (64-bit) with default GCC\* 5.4.0
 - CentOS\* 7.4 (64-bit) with default GCC\* 4.8.5
-- [Intel® Graphics Compute Runtime for OpenCL™ Driver package 18.28.11080](https://github.com/intel/compute-runtime/releases/tag/18.28.11080).
 
 ### Software Requirements
-- [CMake\*](https://cmake.org/download/) 3.9 or higher
+- [CMake\*](https://cmake.org/download/) 3.5 or higher
 - GCC\* 4.8 or higher to build the Inference Engine
 - Python 2.7 or higher for Inference Engine Python API wrapper
+- (Optional) [Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.04.12237](https://github.com/intel/compute-runtime/releases/tag/19.04.12237).
 
 ### Build Steps
 1. Clone submodules:
@@ -33,34 +63,42 @@ The software was validated on:
     git submodule update --recursive
     ```
 2. Install build dependencies using the `install_dependencies.sh` script in the project root folder.
-3. Create a build folder:
+3. By default, the build enables the Inference Engine GPU plugin to infer models on your Intel® Processor Graphics. This requires you to [Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.04.12237](https://github.com/intel/compute-runtime/releases/tag/19.04.12237) before running the build. If you don't want to use the GPU plugin, use the `-DENABLE_CLDNN=OFF` CMake build option and skip the installation of the Intel® Graphics Compute Runtime for OpenCL™ Driver.
+4. Create a build folder:
 ```sh
-  mkdir build
+  mkdir build && cd build
 ```
-4. Inference Engine uses a CMake-based build system. In the created `build` directory, run `cmake` to fetch project dependencies and create Unix makefiles, then run `make` to build the project:
+5. Inference Engine uses a CMake-based build system. In the created `build` directory, run `cmake` to fetch project dependencies and create Unix makefiles, then run `make` to build the project:
 ```sh
   cmake -DCMAKE_BUILD_TYPE=Release ..
-  make -j16
+  make --jobs=$(nproc --all)
 ```
-You can use the following additional build options:
-- Internal JIT GEMM implementation is used by default.
-- To switch to OpenBLAS\* implementation, use `GEMM=OPENBLAS` option and `BLAS_INCLUDE_DIRS` and `BLAS_LIBRARIES` cmake options to specify path to OpenBLAS headers and library, for example use the following options on CentOS\*: `-DGEMM=OPENBLAS -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0`
 
-- To switch to the optimized MKL-ML\* GEMM implementation, use `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` cmake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded [here](https://github.com/intel/mkl-dnn/releases/download/v0.17/mklml_lnx_2019.0.1.20180928.tgz)
+### Additional Build Options
+
+You can use the following additional build options:
+
+- Internal JIT GEMM implementation is used by default.
+
+- To switch to OpenBLAS\* implementation, use the `GEMM=OPENBLAS` option and `BLAS_INCLUDE_DIRS` and `BLAS_LIBRARIES` CMake options to specify path to the OpenBLAS headers and library. For example use the following options on CentOS\*: `-DGEMM=OPENBLAS -DBLAS_INCLUDE_DIRS=/usr/include/openblas -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0`.
+
+- To switch to the optimized MKL-ML\* GEMM implementation, use `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` CMake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded from the [MKL-DNN repository](https://github.com/intel/mkl-dnn/releases/download/v0.19/mklml_lnx_2019.0.5.20190502.tgz).
 
 - Threading Building Blocks (TBB) is used by default. To build the Inference Engine with OpenMP* threading, set the `-DTHREADING=OMP` option.
 
-- Required versions of TBB and OpenCV packages are downloaded automatically by the CMake-based script. If you already have installed TBB or OpenCV packages configured in your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR` environment variables before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed.
+- Required versions of TBB and OpenCV packages are downloaded automatically by the CMake-based script. If you want to use the automatically downloaded packages but you already have installed TBB or OpenCV packages configured in your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR` environment variables before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed. 
+
+- If the CMake-based build script can not find and download the OpenCV package that is supported on your platform, or if you want to use a custom build of the OpenCV library, refer to the [Use Custom OpenCV Builds](#use-custom-opencv-builds-for-inference-engine) section for details. 
 
 - To build the Python API wrapper, use the `-DENABLE_PYTHON=ON` option. To specify an exact Python version, use the following options:
-```sh
-  -DPYTHON_EXECUTABLE=`which python3.7` \
-  -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.7m.so \
-  -DPYTHON_INCLUDE_DIR=/usr/include/python3.7
-```
-
-- To switch on/off the CPU and GPU plugins, use `cmake` options `-DENABLE_MKL_DNN=ON/OFF` and `-DENABLE_CLDNN=ON/OFF`.
+   ```sh
+   -DPYTHON_EXECUTABLE=`which python3.7` \
+   -DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.7m.so \
+   -DPYTHON_INCLUDE_DIR=/usr/include/python3.7
+   ```
 
+- To switch off/on the CPU and GPU plugins, use the `cmake` options `-DENABLE_MKL_DNN=ON/OFF` and `-DENABLE_CLDNN=ON/OFF` respectively. 
+  
 5. Adding to your project
 
     For CMake projects, set an environment variable `InferenceEngine_DIR`:
@@ -79,16 +117,179 @@ You can use the following additional build options:
     target_link_libraries(${PROJECT_NAME} ${InferenceEngine_LIBRARIES} dl)
     ```
 
-## Build on Windows\* Systems:
+## Build for Raspbian Stretch* OS
+
+> **NOTE**: Only the MYRIAD plugin is supported.
+
+### Hardware Requirements
+* Raspberry Pi\* 2 or 3 with Raspbian\* Stretch OS (32-bit). Check that it's CPU supports ARMv7 instruction set (`uname -m` command returns `armv7l`).
+
+  > **NOTE**: Despite the Raspberry Pi\* CPU is ARMv8, 32-bit OS detects ARMv7 CPU instruction set. The default `gcc` compiler applies ARMv6 architecture flag for compatibility with lower versions of boards. For more information, run the `gcc -Q --help=target` command and refer to the description of the `-march=` option.
+
+You can compile the Inference Engine for Raspberry Pi\* in one of the two ways:
+* [Native Compilation](#native-compilation), which is the simplest way, but time-consuming
+* [Cross Compilation Using Docker*](#cross-compilation-using-docker), which is the recommended way
+
+### Native Compilation
+Native compilation of the Inference Engine is the most straightforward solution. However, it might take at least one hour to complete on Raspberry Pi\* 3.
+
+1. Install dependencies:
+
+  ```bash
+  sudo apt-get update
+  sudo apt-get install -y git cmake libusb-1.0-0-dev
+  ```
+
+2. Go to the `inference-engine` directory of the cloned `dldt` repository:
+
+  ```bash
+  cd dldt/inference-engine
+  ```
+
+3. Initialize submodules:
+
+  ```bash
+  git submodule init
+  git submodule update --recursive
+  ```
+
+4. Create a build folder:
+
+  ```bash
+  mkdir build && cd build
+  ```
+
+5. Build the Inference Engine:
+
+  ```bash
+  cmake -DCMAKE_BUILD_TYPE=Release \
+        -DENABLE_SSE42=OFF \
+        -DTHREADING=SEQ \
+        -DENABLE_GNA=OFF .. && make
+  ```
+
+### Cross Compilation Using Docker*
+
+  This compilation was tested on the following configuration:
+
+  * Host: Ubuntu\* 16.04 (64-bit, Intel® Core™ i7-6700K CPU @ 4.00GHz × 8)
+  * Target: Raspbian\* Stretch (32-bit, ARMv7, Raspberry Pi\* 3)
+
+1. Install Docker\*:
+
+  ```bash
+  sudo apt-get install -y docker.io
+  ```
+
+2. Add a current user to `docker` group:
+
+  ```bash
+  sudo usermod -a -G docker $USER
+  ```
+
+  Log out and log in for this to take effect.
+
+3. Create a directory named `ie_cross_armhf` and add a text file named `Dockerfile`
+with the following content:
+
+  ```docker
+  FROM debian:stretch
+
+  USER root
+
+  RUN dpkg --add-architecture armhf && \
+      apt-get update && \
+      apt-get install -y --no-install-recommends \
+      build-essential \
+      crossbuild-essential-armhf \
+      git \
+      wget \
+      libusb-1.0-0-dev:armhf \
+      libgtk-3-dev:armhf \
+      libavcodec-dev:armhf \
+      libavformat-dev:armhf \
+      libswscale-dev:armhf \
+      libgstreamer1.0-dev:armhf \
+      libgstreamer-plugins-base1.0-dev:armhf \
+      libpython3-dev:armhf \
+      python3-pip
+      
+  RUN wget https://www.cmake.org/files/v3.14/cmake-3.14.3.tar.gz && \
+      tar xf cmake-3.14.3.tar.gz && \
+      (cd cmake-3.14.3 && ./bootstrap --parallel=$(nproc --all) && make --jobs=$(nproc --all) && make install) && \
+      rm -rf cmake-3.14.3 cmake-3.14.3.tar.gz
+
+  ```
+
+  It uses the Debian\* Stretch (Debian 9) OS for compilation because it is a base of the Raspbian\* Stretch.
+
+4. Build a Docker\* image:
+
+  ```bash
+  docker image build -t ie_cross_armhf ie_cross_armhf
+  ```
+
+5. Run Docker\* container with mounted source code folder from host:
+
+  ```bash
+  docker run -it -v /absolute/path/to/dldt:/dldt ie_cross_armhf /bin/bash
+  ```
+
+6. While in the container:
+
+    1. Go to the `inference-engine` directory of the cloned `dldt` repository:
+
+      ```bash
+      cd dldt/inference-engine
+      ```
+
+    2. Create a build folder:
+
+      ```bash
+      mkdir build && cd build
+      ```
+
+    3. Build the Inference Engine:
+
+      ```bash
+      cmake -DCMAKE_BUILD_TYPE=Release \
+          -DCMAKE_TOOLCHAIN_FILE="../cmake/arm.toolchain.cmake" \
+          -DTHREADS_PTHREAD_ARG="-pthread" \
+          -DENABLE_SSE42=OFF \
+          -DTHREADING=SEQ \
+          -DENABLE_GNA=OFF .. && make --jobs=$(nproc --all)
+      ```
+
+7. Press "Ctrl"+"D" to exit from Docker\*. You can find the resulting binaries in the `dldt/inference-engine/bin/armv7l/` directory and the OpenCV* installation in the `dldt/inference-engine/temp`.
+   
+>**NOTE**: Native applications that link to cross-compiled Inference Engine library require an extra compilation flag `-march=armv7-a`.
+
+### Additional Build Options
+
+You can use the following additional build options:
+
+- Required versions of OpenCV packages are downloaded automatically by the CMake-based script. If you want to use the automatically downloaded packages but you already have installed OpenCV packages configured in your environment, you may need to clean the `OpenCV_DIR` environment variable before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed. 
+
+- If the CMake-based build script can not find and download the OpenCV package that is supported on your platform, or if you want to use a custom build of the OpenCV library, refer to the [Use Custom OpenCV Builds](#use-custom-opencv-builds-for-inference-engine) section for details.
+
+- To build Python API wrapper, install `libpython3-dev:armhf` and `python3-pip` packages using `apt-get`, then install `numpy` and `cython` python modules using `pip3` command and add the following cmake options:
+```sh
+  -DENABLE_PYTHON=ON \
+  -DPYTHON_EXECUTABLE=/usr/bin/python3.5 \
+  -DPYTHON_LIBRARY=/usr/lib/arm-linux-gnueabihf/libpython3.5m.so \
+  -DPYTHON_INCLUDE_DIR=/usr/include/python3.5
+```
+
+## Build on Windows* Systems
 
 The software was validated on:
 - Microsoft\* Windows\* 10 (64-bit) with Visual Studio 2017 and Intel® C++ Compiler 2018 Update 3
-- [Intel® Graphics Driver for Windows* [24.20] driver package](https://downloadcenter.intel.com/download/27803/Graphics-Intel-Graphics-Driver-for-Windows-10?v=t).
 
 ### Software Requirements
-- [CMake\*](https://cmake.org/download/) 3.9 or higher
+- [CMake\*](https://cmake.org/download/) 3.5 or higher
 - [OpenBLAS\*](https://sourceforge.net/projects/openblas/files/v0.2.14/OpenBLAS-v0.2.14-Win64-int64.zip/download) and [mingw64\* runtime dependencies](https://sourceforge.net/projects/openblas/files/v0.2.14/mingw64_dll.zip/download).
 - [Intel® C++ Compiler](https://software.intel.com/en-us/intel-parallel-studio-xe) 18.0 to build the Inference Engine on Windows.
+- (Optional) [Intel® Graphics Driver for Windows* [25.20] driver package](https://downloadcenter.intel.com/download/28646/Intel-Graphics-Windows-10-DCH-Drivers?product=80939).
 - Python 3.4 or higher for Inference Engine Python API wrapper
 
 ### Build Steps
@@ -101,11 +302,12 @@ The software was validated on:
 3. Install OpenBLAS:
     1. Download [OpenBLAS\*](https://sourceforge.net/projects/openblas/files/v0.2.14/OpenBLAS-v0.2.14-Win64-int64.zip/download)
     2. Unzip the downloaded package to a directory on your machine. In this document, this directory is referred to as `<OPENBLAS_DIR>`.
-4. Create build directory:
+4. By default, the build enables the Inference Engine GPU plugin to infer models on your Intel® Processor Graphics. This requires you to [download and install the Intel® Graphics Driver for Windows* [25.20] driver package](https://downloadcenter.intel.com/download/28646/Intel-Graphics-Windows-10-DCH-Drivers?product=80939) before running the build. If you don't want to use the GPU plugin, use the `-DENABLE_CLDNN=OFF` CMake build option and skip the installation of the Intel® Graphics Driver.    
+5. Create build directory:
     ```sh
     mkdir build
     ```
-5. In the `build` directory, run `cmake` to fetch project dependencies and generate a Visual Studio solution:
+6. In the `build` directory, run `cmake` to fetch project dependencies and generate a Visual Studio solution:
 ```sh
 cd build
 cmake -G "Visual Studio 15 2017 Win64" -T "Intel C++ Compiler 18.0" ^
@@ -113,26 +315,32 @@ cmake -G "Visual Studio 15 2017 Win64" -T "Intel C++ Compiler 18.0" ^
     -DICCLIB="C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018\windows\compiler\lib" ..
 ```
 
+7. Build generated solution in Visual Studio 2017 or run `cmake --build . --config Release` to build from the command line.
+
+8. Before running the samples, add paths to TBB and OpenCV binaries used for the build to the `%PATH%` environment variable. By default, TBB binaries are downloaded by the CMake-based script to the `<dldt_repo>/inference-engine/temp/tbb/lib` folder, OpenCV binaries - to the `<dldt_repo>/inference-engine/temp/opencv_4.1.0/bin` folder.
+
+### Additional Build Options
+
 - Internal JIT GEMM implementation is used by default.
-- To switch to OpenBLAS GEMM implementation, use -DGEMM=OPENBLAS cmake option and specify path to OpenBLAS using `-DBLAS_INCLUDE_DIRS=<OPENBLAS_DIR>\include` and `-DBLAS_LIBRARIES=<OPENBLAS_DIR>\lib\libopenblas.dll.a` options. Prebuilt OpenBLAS\* package can be downloaded [here](https://sourceforge.net/projects/openblas/files/v0.2.14/OpenBLAS-v0.2.14-Win64-int64.zip/download), mingw64* runtime dependencies [here](https://sourceforge.net/projects/openblas/files/v0.2.14/mingw64_dll.zip/download)
-- To switch to the optimized MKL-ML\* GEMM implementation, use `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` cmake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded [here](https://github.com/intel/mkl-dnn/releases/download/v0.17/mklml_win_2019.0.1.20180928.zip)
+- To switch to OpenBLAS GEMM implementation, use the `-DGEMM=OPENBLAS` CMake option and specify path to OpenBLAS using the `-DBLAS_INCLUDE_DIRS=<OPENBLAS_DIR>\include` and `-DBLAS_LIBRARIES=<OPENBLAS_DIR>\lib\libopenblas.dll.a` options. Prebuilt OpenBLAS\* package can be downloaded [here](https://sourceforge.net/projects/openblas/files/v0.2.14/OpenBLAS-v0.2.14-Win64-int64.zip/download). mingw64* runtime dependencies can be downloaded [here](https://sourceforge.net/projects/openblas/files/v0.2.14/mingw64_dll.zip/download).
+- To switch to the optimized MKL-ML\* GEMM implementation, use the `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` CMake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded from the [MKL-DNN repository](https://github.com/intel/mkl-dnn/releases/download/v0.19/mklml_win_2019.0.5.20190502.zip).
 
 - Threading Building Blocks (TBB) is used by default. To build the Inference Engine with OpenMP* threading, set the `-DTHREADING=OMP` option.
 
-- Required versions of TBB and OpenCV packages are downloaded automatically by the CMake-based script. If you already have installed TBB or OpenCV packages configured in your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR` environment variables before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed.
+- Required versions of TBB and OpenCV packages are downloaded automatically by the CMake-based script. If you want to use the automatically downloaded packages but you already have installed TBB or OpenCV packages configured in your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR` environment variables before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed.
+
+- If the CMake-based build script can not find and download the OpenCV package that is supported on your platform, or if you want to use a custom build of the OpenCV library, refer to the [Use Custom OpenCV Builds](#use-custom-opencv-builds-for-inference-engine) section for details.
+
+- To switch off/on the CPU and GPU plugins, use the `cmake` options `-DENABLE_MKL_DNN=ON/OFF` and `-DENABLE_CLDNN=ON/OFF` respectively.
 
 - To build the Python API wrapper, use the `-DENABLE_PYTHON=ON` option. To specify an exact Python version, use the following options:
-```sh
-  -DPYTHON_EXECUTABLE="C:\Program Files\Python37\python.exe" ^
-  -DPYTHON_LIBRARY="C:\Program Files\Python37\libs\python37.lib" ^
-  -DPYTHON_INCLUDE_DIR="C:\Program Files\Python37\include"
-```
+   ```sh
+   -DPYTHON_EXECUTABLE="C:\Program Files\Python37\python.exe" ^
+   -DPYTHON_LIBRARY="C:\Program Files\Python37\libs\python37.lib" ^
+   -DPYTHON_INCLUDE_DIR="C:\Program Files\Python37\include"
+   ```
 
-6. Build generated solution in Visual Studio 2017 or run `cmake --build . --config Release` to build from the command line.
-
-7. Before running the samples, add paths to TBB and OpenCV binaries used for the build to the %PATH% environment variable. By default, TBB binaries are downloaded by the CMake-based script to the `<dldt_repo>/inference-engine/temp/tbb/lib` folder, OpenCV binaries - to the `<dldt_repo>/inference-engine/temp/opencv_4.1.0/bin` folder.
-
-### Building Inference Engine with Ninja
+### Building Inference Engine with Ninja* Build System
 
 ```sh
 call "C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018\windows\bin\ipsxe-comp-vars.bat" intel64 vs2017
@@ -144,13 +352,15 @@ cmake -G Ninja -Wno-dev -DCMAKE_BUILD_TYPE=Release ..
 cmake --build . --config Release
 ```
 
-## Build on macOS\* Systems
+## Build on macOS* Systems
+
+> **NOTE**: The current version of the OpenVINO™ toolkit for macOS* supports inference on Intel CPUs only.
 
 The software was validated on:
 - macOS\* 10.14, 64-bit
 
 ### Software Requirements
-- [CMake\*](https://cmake.org/download/) 3.9 or higher
+- [CMake\*](https://cmake.org/download/) 3.5 or higher
 - Clang\* compiler from Xcode\* 10.1
 - Python\* 3.4 or higher for the Inference Engine Python API wrapper
 
@@ -169,14 +379,20 @@ The software was validated on:
 4. Inference Engine uses a CMake-based build system. In the created `build` directory, run `cmake` to fetch project dependencies and create Unix makefiles, then run `make` to build the project:
 ```sh
   cmake -DCMAKE_BUILD_TYPE=Release ..
-  make -j16
+  make --jobs=$(nproc --all)
 ```
+### Additional Build Options
+
 You can use the following additional build options:
 - Internal JIT GEMM implementation is used by default.
-- To switch to the optimized MKL-ML\* GEMM implementation, use `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` cmake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded [here](https://github.com/intel/mkl-dnn/releases/download/v0.17.1/mklml_mac_2019.0.1.20180928.tgz)
+- To switch to the optimized MKL-ML\* GEMM implementation, use `-DGEMM=MKL` and `-DMKLROOT=<path_to_MKL>` cmake options to specify a path to unpacked MKL-ML with the `include` and `lib` folders. MKL-ML\* package can be downloaded [here](https://github.com/intel/mkl-dnn/releases/download/v0.19/mklml_mac_2019.0.5.20190502.tgz)
 
 - Threading Building Blocks (TBB) is used by default. To build the Inference Engine with OpenMP* threading, set the `-DTHREADING=OMP` option.
 
+- Required versions of TBB and OpenCV packages are downloaded automatically by the CMake-based script. If you want to use the automatically downloaded packages but you already have installed TBB or OpenCV packages configured in your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR` environment variables before running the `cmake` command, otherwise they won't be downloaded and the build may fail if incompatible versions were installed.
+
+- If the CMake-based build script can not find and download the OpenCV package that is supported on your platform, or if you want to use a custom build of the OpenCV library, refer to the [Use Custom OpenCV Builds](#use-custom-opencv-builds-for-inference-engine) section for details.
+
 - To build the Python API wrapper, use the `-DENABLE_PYTHON=ON` option. To specify an exact Python version, use the following options:
 ```sh
   -DPYTHON_EXECUTABLE=/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 \
@@ -184,6 +400,82 @@ You can use the following additional build options:
   -DPYTHON_INCLUDE_DIR=/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m
 ```
 
+## Use Custom OpenCV Builds for Inference Engine
+
+> **NOTE**: The recommended and tested version of OpenCV is 4.1. The minimum supported version is 3.4.0.  
+
+Required versions of OpenCV packages are downloaded automatically during the building Inference Engine library. If the build script can not find and download the OpenCV package that is supported on your platform, you can use one of the following options:
+
+* Download the most suitable version from the list of available pre-build packages from [https://download.01.org/opencv/2019/openvinotoolkit](https://download.01.org/opencv/2019/openvinotoolkit) from the `<release_version>/inference_engine` directory.
+
+* Use a system provided OpenCV package (e.g with running the `apt install libopencv-dev` command). The following modules must be enabled: `imgcodecs`, `videoio`, `highgui`.
+
+* Get the OpenCV package using a package manager: pip, conda, conan etc. The package must have the development components included (header files and CMake scripts).
+
+* Build OpenCV from source using the [build instructions](https://docs.opencv.org/master/df/d65/tutorial_table_of_content_introduction.html) on the OpenCV site. 
+  
+After you got the built OpenCV library, perform the following preparation steps before running the Inference Engine build:
+  
+1. Set the `OpenCV_DIR` environment variable to the directory where the `OpenCVConfig.cmake` file of you custom OpenCV build is located.
+2. Disable the package automatic downloading with using the `-DENABLE_OPENCV=OFF` option for CMake-based build script for Inference Engine.
+
+## (Optional) Additional Installation Steps for the Intel® Movidius™ Neural Compute Stick and Neural Compute Stick 2
+
+> **NOTE**: These steps are only required if you want to perform inference on Intel® Movidius™ Neural Compute Stick or the Intel® Neural Compute Stick 2 using the Inference Engine MYRIAD Plugin. See also [Intel® Neural Compute Stick 2 Get Started](https://software.intel.com/en-us/neural-compute-stick/get-started)
+
+### For Linux, Raspbian\* Stretch OS
+
+1. Add the current Linux user to the `users` group:
+```sh
+sudo usermod -a -G users "$(whoami)"
+```
+Log out and log in for it to take effect.
+
+2. To perform inference on Intel® Movidius™ Neural Compute Stick and Intel® Neural Compute Stick 2, install the USB rules as follows:
+```sh
+cat <<EOF > 97-myriad-usbboot.rules
+SUBSYSTEM=="usb", ATTRS{idProduct}=="2150", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+SUBSYSTEM=="usb", ATTRS{idProduct}=="2485", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+SUBSYSTEM=="usb", ATTRS{idProduct}=="f63b", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
+EOF
+```
+```sh
+sudo cp 97-myriad-usbboot.rules /etc/udev/rules.d/
+```
+```sh
+sudo udevadm control --reload-rules
+```
+```sh
+sudo udevadm trigger
+```
+```sh
+sudo ldconfig
+```
+```sh
+rm 97-myriad-usbboot.rules
+```
+
+### For Windows
+
+For Intel® Movidius™ Neural Compute Stick and Intel® Neural Compute Stick 2, install the Movidius™ VSC driver:
+1. Go to the `<DLDT_ROOT_DIR>/inference-engine/thirdparty/movidius/MovidiusDriver` directory, where the `DLDT_ROOT_DIR` is the directory to which the DLDT repository was cloned. 
+2. Right click on the `Movidius_VSC_Device.inf` file and choose **Install** from the pop up menu.
+
+You have installed the driver for your Intel® Movidius™ Neural Compute Stick or Intel® Neural Compute Stick 2.
+
+## Next Steps
+
+Congratulations, you have built the Inference Engine. To get started with the OpenVINO™ DLDT, proceed to the Get Started guides:
+
+* [Get Started with Deep Learning Deployment Toolkit on Linux*](../get-started-linux.md)
+
+## Additional Resources
+
+* [OpenVINO™ Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
+* [Introduction to Intel® Deep Learning Deployment Toolkit](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Introduction.html)
+* [Inference Engine Samples Overview](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Samples_Overview.html)
+* [Inference Engine Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html)
+* [Model Optimizer Developer Guide](https://docs.openvinotoolkit.org/latest/_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)
 
 ---
-\* Other names and brands may be claimed as the property of others.
+\* Other names and brands may be claimed as the property of others.

inference-engine/cmake/FindITT.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -11,13 +10,15 @@ if(NOT DEFINED INTEL_VTUNE_DIR AND DEFINED ENV{INTEL_VTUNE_DIR})
 endif()
 if(NOT DEFINED INTEL_VTUNE_DIR)
     if(EXISTS "/opt/intel/vtune_amplifier_xe/include")
-    	set(INTEL_VTUNE_DIR "/opt/intel/vtune_amplifier_xe")
+        set(INTEL_VTUNE_DIR "/opt/intel/vtune_amplifier_xe")
     elseif(EXISTS "/opt/intel/vtune_amplifier/include")
         set(INTEL_VTUNE_DIR "/opt/intel/vtune_amplifier")
     elseif (EXISTS "C:/Program Files (x86)/IntelSWTools/VTune Amplifier XE")
         set(INTEL_VTUNE_DIR "C:/Program Files (x86)/IntelSWTools/VTune Amplifier XE")
     elseif (EXISTS "C:/Program Files (x86)/IntelSWTools/VTune Amplifier")
         set(INTEL_VTUNE_DIR "C:/Program Files (x86)/IntelSWTools/VTune Amplifier")
+    elseif (EXISTS "$ENV{HOME}/intel/vtune_amplifier_2019")
+        set(INTEL_VTUNE_DIR "$ENV{HOME}/intel/vtune_amplifier_2019")
     endif()
 endif()
 
@@ -33,7 +34,7 @@ if(DEFINED INTEL_VTUNE_DIR)
     "libittnotify${CMAKE_STATIC_LIBRARY_SUFFIX}"
     PATHS ${INTEL_VTUNE_DIR}/lib64)
 
-    set(Located_ITT_LIBS ${ITT_LIB} ${CMAKE_DL_LIBS})
+    set(Located_ITT_LIBS ${ITT_LIB})
     set(Located_ITT_INCLUDE_DIRS ${ITT_INCLUDE_DIR})
 else()
     message(STATUS "INTEL_VTUNE_DIR is not defined")
@@ -46,17 +47,11 @@ find_package_handle_standard_args(INTEL_ITT
         Located_ITT_INCLUDE_DIRS
         Located_ITT_LIBS)
 
-if(ENABLE_PROFILING_ITT AND INTEL_ITT_FOUND)
-    add_definitions(-DENABLE_PROFILING_ITT=1)
+if(INTEL_ITT_FOUND)
+    add_library(ittnotify STATIC IMPORTED GLOBAL)
+    set_target_properties(ittnotify PROPERTIES IMPORTED_LOCATION "${Located_ITT_LIBS}"
+                                               INTERFACE_INCLUDE_DIRECTORIES ${Located_ITT_INCLUDE_DIRS}
+                                               INTERFACE_COMPILE_DEFINITIONS ENABLE_PROFILING_ITT)
 
-    set(INTEL_ITT_LIBS ${Located_ITT_LIBS})
-    set(INTEL_ITT_INCLUDE_DIRS ${Located_ITT_INCLUDE_DIRS})
-
-    message(STATUS "INTEL_ITT_INCLUDE_DIRS: ${INTEL_ITT_INCLUDE_DIRS}")
-    include_directories(${INTEL_ITT_INCLUDE_DIRS})
-    message(STATUS "INTEL_ITT_LIBS: ${INTEL_ITT_LIBS}")
-else()
-    add_definitions(-DENABLE_PROFILING_ITT=0)
-    message(STATUS "INTEL_ITT is disabled")
+    set(INTEL_ITT_LIBS ittnotify ${CMAKE_DL_LIBS})
 endif()
-

inference-engine/cmake/FindlibGNA.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/arm.toolchain.cmake

@@ -1,3 +1,7 @@
+# Copyright (C) 2018-2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
 set(CMAKE_SYSTEM_NAME Linux)
 set(CMAKE_SYSTEM_PROCESSOR armv7l)
 

inference-engine/cmake/arm64.toolchain.cmake

@@ -0,0 +1,14 @@
+# Copyright (C) 2018-2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+set(CMAKE_SYSTEM_NAME Linux)
+set(CMAKE_SYSTEM_PROCESSOR aarch64)
+
+set(CMAKE_C_COMPILER aarch64-linux-gnu-gcc)
+set(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++)
+
+set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
+set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
+set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

inference-engine/cmake/check_features.cmake

@@ -1,12 +1,7 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
-include("features")
-include("mode")
-include("itt")
-
 #64 bits platform
 if ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
     message(STATUS "Detected 64 bit architecture")
@@ -18,8 +13,7 @@ else()
     SET(ARCH_32 ON)
 endif()
 
-if (ARCH_64)
-else()
+if (NOT ARCH_64)
     if (UNIX OR APPLE)
         SET(ENABLE_CLDNN OFF)
     endif()
@@ -30,6 +24,8 @@ endif()
 if (APPLE)
     set(ENABLE_GNA OFF)
     set(ENABLE_CLDNN OFF)
+    SET(ENABLE_MYRIAD OFF)
+    SET(ENABLE_VPU OFF)
 endif()
 
 
@@ -41,25 +37,14 @@ if (WIN32)
     endif()
 endif()
 
-# Linux specific - not all OS'es are supported
-if (LINUX)
-    include("linux_name")
-    get_linux_name(LINUX_OS_NAME)
-    if (LINUX_OS_NAME)
-        if (NOT(
-                ${LINUX_OS_NAME} STREQUAL "Ubuntu 14.04" OR
-                ${LINUX_OS_NAME} STREQUAL "Ubuntu 16.04" OR
-                ${LINUX_OS_NAME} STREQUAL "CentOS 7"))
-        endif()
-    else ()
-        message(WARNING "Cannot detect Linux OS via reading /etc/*-release:\n ${release_data}")
-    endif ()
-endif ()
-
 if (NOT ENABLE_MKL_DNN)
     set(ENABLE_MKL OFF)
 endif()
 
+if (NOT ENABLE_VPU)
+    set(ENABLE_MYRIAD OFF)
+endif()
+
 #next section set defines to be accesible in c++/c code for certain feature
 if (ENABLE_PROFILING_RAW)
     add_definitions(-DENABLE_PROFILING_RAW=1)
@@ -69,10 +54,22 @@ if (ENABLE_CLDNN)
     add_definitions(-DENABLE_CLDNN=1)
 endif()
 
+if (ENABLE_MYRIAD)
+    add_definitions(-DENABLE_MYRIAD=1)
+endif()
+
+if (ENABLE_MYRIAD_NO_BOOT AND ENABLE_MYRIAD )
+    add_definitions(-DENABLE_MYRIAD_NO_BOOT=1)
+endif()
+
 if (ENABLE_MKL_DNN)
     add_definitions(-DENABLE_MKL_DNN=1)
 endif()
 
+if (ENABLE_UNICODE_PATH_SUPPORT)
+    add_definitions(-DENABLE_UNICODE_PATH_SUPPORT=1)
+endif()
+
 if (ENABLE_GNA)
     add_definitions(-DENABLE_GNA)
 endif()

inference-engine/cmake/config.cmake.in

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/cppcheck.cmake

@@ -0,0 +1,28 @@
+# Copyright (C) 2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+if(ENABLE_CPPCHECK)
+    find_program(CPPCHECK_EXECUTABLE cppcheck)
+
+    if(NOT CPPCHECK_EXECUTABLE)
+        message(WARNING "cppcheck was not found : disable static analysis")
+        set(ENABLE_CPPCHECK OFF)
+    endif()
+endif()
+
+function(add_cppcheck)
+    if(NOT ENABLE_CPPCHECK)
+        return()
+    endif()
+
+    set_property(
+        TARGET ${ARGN}
+        PROPERTY CXX_CPPCHECK
+            ${CPPCHECK_EXECUTABLE}
+            "--suppress=*:*/temp/*"
+            "--suppress=*:*/thirdparty/*"
+            "--error-exitcode=1"
+            "--template={file}:{line}: error: [cppcheck:{severity}] {message}"
+            "--quiet")
+endfunction()

inference-engine/cmake/cpplint.cmake

@@ -1,13 +1,12 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
 if(ENABLE_CPPLINT)
     find_package(PythonInterp 2.7 EXACT)
 
-    if(NOT PYTHONINTERP_FOUND)
-        message(WARNING "Python was not found (required for cpplint check)")
+    if(NOT PYTHONINTERP_FOUND OR NOT PYTHON_VERSION_MAJOR EQUAL 2)
+        message(WARNING "Python 2.7 was not found (required for cpplint check)")
         set(ENABLE_CPPLINT OFF)
     endif()
 endif()
@@ -127,7 +126,7 @@ function(add_cpplint_report_target)
             -D "OUTPUT_FILE=${cppcheck_output_file}"
             -P "${IE_MAIN_SOURCE_DIR}/cmake/cpplint_to_cppcheck_xml.cmake"
         DEPENDS
-            ${cpplint_output_file}
+            "${cpplint_output_file}"
             "${IE_MAIN_SOURCE_DIR}/scripts/cpplint_to_cppcheckxml.py"
             "${IE_MAIN_SOURCE_DIR}/cmake/cpplint_to_cppcheck_xml.cmake"
         COMMENT

inference-engine/cmake/cpplint_html.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -22,9 +21,9 @@ execute_process(
 
 file(READ "${REPORT_DIR}/index.html" cur_file_content)
 
-string(REPLACE "Cppcheck" "cpplint" cur_file_content ${cur_file_content})
-string(REPLACE "a tool for static C/C++ code analysis" "an open source lint-like tool from Google" cur_file_content ${cur_file_content})
-string(REPLACE "http://cppcheck.sourceforge.net" "http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py" cur_file_content ${cur_file_content})
-string(REPLACE "IRC: <a href=\"irc://irc.freenode.net/cppcheck\">irc://irc.freenode.net/cppcheck</a>" " " cur_file_content ${cur_file_content})
+string(REPLACE "Cppcheck" "cpplint" cur_file_content "${cur_file_content}")
+string(REPLACE "a tool for static C/C++ code analysis" "an open source lint-like tool from Google" cur_file_content "${cur_file_content}")
+string(REPLACE "http://cppcheck.sourceforge.net" "http://google-styleguide.googlecode.com/svn/trunk/cpplint/cpplint.py" cur_file_content "${cur_file_content}")
+string(REPLACE "IRC: <a href=\"irc://irc.freenode.net/cppcheck\">irc://irc.freenode.net/cppcheck</a>" " " cur_file_content "${cur_file_content}")
 
 file(WRITE "${REPORT_DIR}/index.html" "${cur_file_content}")

inference-engine/cmake/cpplint_merge.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/cpplint_run.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -22,12 +21,12 @@ execute_process(
 message("${output}")
 
 # Store cpplint output to file (replace problematic symbols)
-string(REPLACE "\"" "&quot\;" output ${output})
-string(REPLACE "<" "&lt\;" output ${output})
-string(REPLACE ">" "&gt\;" output ${output})
-string(REPLACE "'" "&apos\;" output ${output})
-string(REPLACE "&" "&amp\;" output ${output})
-file(WRITE "${OUTPUT_FILE}" ${output})
+string(REPLACE "\"" "&quot\;" output "${output}")
+string(REPLACE "<" "&lt\;" output "${output}")
+string(REPLACE ">" "&gt\;" output "${output}")
+string(REPLACE "'" "&apos\;" output "${output}")
+string(REPLACE "&" "&amp\;" output "${output}")
+file(WRITE "${OUTPUT_FILE}" "${output}")
 
 if(NOT SKIP_RETURN_CODE)
     # Pass through the cpplint return code

inference-engine/cmake/cpplint_to_cppcheck_xml.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/debug.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -9,7 +8,6 @@ function (debug_message)
     endif()
 endfunction()
 
-
 function(clean_message type)
   string (REPLACE ";" "" output_string "${ARGN}")
   execute_process(COMMAND ${CMAKE_COMMAND} -E echo "${output_string}")
@@ -47,7 +45,12 @@ function (log_rpath_remove_top component component_remove_top lib lib_remove_top
 
 #  debug_message(STATUS "LIB-OUT=${lib_dir}")
 #  debug_message(STATUS "TOPLIB-OUT=${top_lib_dir}")
- 
+
+  if (WIN32)
+    string (TOLOWER "${top_lib_dir}" top_lib_dir)
+    string (TOLOWER "${lib_dir}" lib_dir)
+  endif()
+
   string (REPLACE "${top_lib_dir}" "" component_dir "${lib_dir}")
 
   set(RPATH_INFO "${component}=${component_dir}")
@@ -56,9 +59,7 @@ function (log_rpath_remove_top component component_remove_top lib lib_remove_top
 endfunction()
 
 function (log_rpath_from_dir component lib_dir)
-  if(NOT APPLE)
-    log_rpath_remove_top("${component}" TRUE "${lib_dir}" FALSE)
-  endif()
+  log_rpath_remove_top("${component}" TRUE "${lib_dir}" FALSE)
 endfunction()
 
 function (log_rpath component lib_path)

inference-engine/cmake/dependencies.cmake

@@ -1,33 +1,13 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
 cmake_policy(SET CMP0054 NEW)
 
-#features trigger supported by build system
-include(check_features)
-include(debug)
-
 #we have number of dependencies stored on ftp
 include(dependency_solver)
 
-#prepare temporary folder
-if (DEFINED ENV{${DL_SDK_TEMP}} AND NOT $ENV{${DL_SDK_TEMP}} STREQUAL "")
-    if (WIN32)
-        string(REPLACE "\\" "\\\\" TEMP $ENV{${DL_SDK_TEMP}})
-    else(WIN32)
-        set(TEMP $ENV{${DL_SDK_TEMP}})
-    endif(WIN32)
-
-    if (ENABLE_ALTERNATIVE_TEMP)
-        set(ALTERNATIVE_PATH ${IE_MAIN_SOURCE_DIR}/temp)
-    endif()
-else ()
-    message(STATUS "DL_SDK_TEMP envionment not set")
-    set(TEMP ${IE_MAIN_SOURCE_DIR}/temp)
-endif ()
-
+set_temp_directory(TEMP "${IE_MAIN_SOURCE_DIR}")
 
 include(ExternalProject)
 
@@ -37,6 +17,29 @@ else()
     set(MODELS_BRANCH "master")
 endif()
 
+include(linux_name)
+if(COMMAND get_linux_name)
+    get_linux_name(LINUX_OS_NAME)
+endif()
+
+if (ENABLE_MYRIAD)
+    RESOLVE_DEPENDENCY(VPU_FIRMWARE_MA2450
+            ARCHIVE_UNIFIED firmware_ma2450_676.zip
+            TARGET_PATH "${TEMP}/vpu/firmware/ma2450"
+            ENVIRONMENT "VPU_FIRMWARE_MA2450"
+            FOLDER)
+    debug_message(STATUS "ma2450=" ${VPU_FIRMWARE_MA2450})
+endif ()
+
+if (ENABLE_MYRIAD)
+    RESOLVE_DEPENDENCY(VPU_FIRMWARE_MA2X8X
+            ARCHIVE_UNIFIED firmware_ma2x8x_mdk_R8_9.zip
+            TARGET_PATH "${TEMP}/vpu/firmware/ma2x8x"
+            ENVIRONMENT "VPU_FIRMWARE_MA2X8X"
+            FOLDER)
+    debug_message(STATUS "ma2x8x=" ${VPU_FIRMWARE_MA2X8X})
+endif ()
+
 ## enable cblas_gemm from OpenBLAS package
 if (GEMM STREQUAL "OPENBLAS")
 if(NOT BLAS_LIBRARIES OR NOT BLAS_INCLUDE_DIRS)
@@ -85,7 +88,7 @@ debug_message(STATUS "intel_omp=" ${OMP})
 endif ()
 
 ## TBB package
-if (THREADING STREQUAL "TBB")
+if (THREADING STREQUAL "TBB" OR THREADING STREQUAL "TBB_AUTO")
 if (WIN32)
     #TODO: add target_path to be platform specific as well, to avoid following if
     RESOLVE_DEPENDENCY(TBB
@@ -110,57 +113,53 @@ debug_message(STATUS "tbb=" ${TBB})
 endif ()
 
 if (ENABLE_OPENCV)
+  set(OPENCV_VERSION "4.1.1")
+  set(OPENCV_BUILD "595")
+  set(OPENCV_SUFFIX "")
 if (WIN32)
     RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_WIN "opencv_4.1.0-0437.zip"
-            TARGET_PATH "${TEMP}/opencv_4.1.0"
+            ARCHIVE_WIN "opencv_${OPENCV_VERSION}-${OPENCV_BUILD}.zip"
+            TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}"
             ENVIRONMENT "OpenCV_DIR"
             VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "\\opencv_4.1.0\\bin")
+    log_rpath_from_dir(OPENCV "\\opencv_${OPENCV_VERSION}\\bin")
     set( ENV{OpenCV_DIR} ${OPENCV}/cmake )
 elseif(APPLE)
     RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_MAC "opencv_4.1.0-0437_osx.tar.xz"
-            TARGET_PATH "${TEMP}/opencv_4.1.0_osx"
+            ARCHIVE_MAC "opencv_${OPENCV_VERSION}-${OPENCV_BUILD}_osx.tar.xz"
+            TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}_osx"
             ENVIRONMENT "OpenCV_DIR"
             VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "opencv_4.1.0_osx/lib")
+    log_rpath_from_dir(OPENCV "opencv_${OPENCV_VERSION}_osx/lib")
     set( ENV{OpenCV_DIR} ${OPENCV}/cmake )
 elseif(LINUX)
-if (${LINUX_OS_NAME} STREQUAL "Ubuntu 16.04")
-    RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_LIN "opencv_4.1.0-0437_ubuntu16.tar.xz"
-            TARGET_PATH "${TEMP}/opencv_4.1.0_ubuntu16"
-            ENVIRONMENT "OpenCV_DIR"
-            VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "opencv_4.1.0_ubuntu16/lib")
-elseif (${LINUX_OS_NAME} STREQUAL "Ubuntu 18.04")
-    RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_LIN "opencv_4.1.0-0437_ubuntu18.tar.xz"
-            TARGET_PATH "${TEMP}/opencv_4.1.0_ubuntu18"
-            ENVIRONMENT "OpenCV_DIR"
-            VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "opencv_4.1.0_ubuntu18/lib")
-elseif (${LINUX_OS_NAME} STREQUAL "CentOS 7")
-    RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_LIN "opencv_4.1.0-0437_centos7.tar.xz"
-            TARGET_PATH "${TEMP}/opencv_4.1.0_centos"
-            ENVIRONMENT "OpenCV_DIR"
-            VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "opencv_4.1.0_centos/lib")
-elseif (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "armv7l" AND
-        (${LINUX_OS_NAME} STREQUAL "Debian 9" OR
-         ${LINUX_OS_NAME} STREQUAL "Raspbian 9"))
-    RESOLVE_DEPENDENCY(OPENCV
-            ARCHIVE_LIN "opencv_4.1.0-0437_debian9arm.tar.xz"
-            TARGET_PATH "${TEMP}/opencv_4.1.0_debian9arm"
-            ENVIRONMENT "OpenCV_DIR"
-            VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
-    log_rpath_from_dir(OPENCV "opencv_4.1.0_debian9arm/lib")
+    if (${LINUX_OS_NAME} STREQUAL "Ubuntu 16.04")
+        set(OPENCV_SUFFIX "ubuntu16")
+    elseif (${LINUX_OS_NAME} STREQUAL "Ubuntu 18.04")
+        set(OPENCV_SUFFIX "ubuntu18")
+    elseif (${LINUX_OS_NAME} STREQUAL "CentOS 7")
+        set(OPENCV_SUFFIX "centos7")
+    elseif (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "armv7l" AND
+            (${LINUX_OS_NAME} STREQUAL "Debian 9" OR
+             ${LINUX_OS_NAME} STREQUAL "Raspbian 9" OR
+             ${LINUX_OS_NAME} STREQUAL "Debian 10" OR
+             ${LINUX_OS_NAME} STREQUAL "Raspbian 10"))
+        set(OPENCV_SUFFIX "debian9arm")
+    endif()
 endif()
+
+if (OPENCV_SUFFIX)
+    RESOLVE_DEPENDENCY(OPENCV
+            ARCHIVE_LIN "opencv_${OPENCV_VERSION}-${OPENCV_BUILD}_${OPENCV_SUFFIX}.tar.xz"
+            TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}_${OPENCV_SUFFIX}"
+            ENVIRONMENT "OpenCV_DIR"
+            VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*")
+    log_rpath_from_dir(OPENCV "opencv_${OPENCV_VERSION}_${OPENCV_SUFFIX}/lib")
     set( ENV{OpenCV_DIR} ${OPENCV}/cmake )
 endif()
+
 debug_message(STATUS "opencv=" ${OPENCV})
+set(OpenCV_DIR "${OPENCV}" CACHE PATH "Path to OpenCV in temp directory")
 endif()
 
 
@@ -173,16 +172,16 @@ if (ENABLE_GNA)
 endif()
 
 configure_file(
-        "${CMAKE_SOURCE_DIR}/cmake/share/InferenceEngineConfig.cmake.in"
+        "${PROJECT_SOURCE_DIR}/cmake/share/InferenceEngineConfig.cmake.in"
         "${CMAKE_BINARY_DIR}/share/InferenceEngineConfig.cmake"
         @ONLY)
 
 configure_file(
-        "${CMAKE_SOURCE_DIR}/cmake/share/InferenceEngineConfig-version.cmake.in"
+        "${PROJECT_SOURCE_DIR}/cmake/share/InferenceEngineConfig-version.cmake.in"
         "${CMAKE_BINARY_DIR}/share/InferenceEngineConfig-version.cmake"
         COPYONLY)
 
 configure_file(
-        "${CMAKE_SOURCE_DIR}/cmake/ie_parallel.cmake"
+        "${PROJECT_SOURCE_DIR}/cmake/ie_parallel.cmake"
         "${CMAKE_BINARY_DIR}/share/ie_parallel.cmake"
         COPYONLY)

inference-engine/cmake/dependency_solver.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -106,6 +105,8 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
     set (FOLDER FALSE)
   endif()
 
+
+
   #for each dependency type have to do separate things
   if (ARCHIVE_WIN OR ARCHIVE_LIN OR ARCHIVE_MAC OR ARCHIVE OR ARCHIVE_UNIFIED)
     if (NOT DEFINED TARGET_PATH)

inference-engine/cmake/developer_package.cmake

@@ -0,0 +1,161 @@
+# Copyright (C) 2018 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+# printing debug messages
+include(debug)
+
+if (UNIX AND NOT APPLE)
+    set(LINUX TRUE)
+endif()
+
+string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} ARCH_FOLDER)
+if(ARCH_FOLDER STREQUAL "x86_64" OR ARCH_FOLDER STREQUAL "amd64") # Windows detects Intel's 64-bit CPU as AMD64
+    set(ARCH_FOLDER intel64)
+elseif(ARCH_FOLDER STREQUAL "i386")
+    set(ARCH_FOLDER ia32)
+endif()
+
+if(OS_FOLDER)
+	message ("**** OS FOLDER IS: [${OS_FOLDER}]")
+	if("${OS_FOLDER}" STREQUAL "ON")
+		message ("**** USING OS FOLDER: [${CMAKE_SYSTEM_NAME}]")
+		set(BIN_FOLDER "bin/${CMAKE_SYSTEM_NAME}/${ARCH_FOLDER}")
+	else()
+		set(BIN_FOLDER "bin/${OS_FOLDER}/${ARCH_FOLDER}")
+	endif()
+else()
+    set(BIN_FOLDER "bin/${ARCH_FOLDER}")
+endif()
+
+if("${CMAKE_BUILD_TYPE}" STREQUAL "")
+    debug_message(STATUS "CMAKE_BUILD_TYPE not defined, 'Release' will be used")
+    set(CMAKE_BUILD_TYPE "Release")
+endif()
+
+if(COVERAGE)
+    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fprofile-arcs -ftest-coverage -O0")
+endif()
+
+if(UNIX)
+    SET(LIB_DL ${CMAKE_DL_LIBS})
+endif()
+
+set(OUTPUT_ROOT ${IE_MAIN_SOURCE_DIR})
+
+# Enable postfixes for Debug/Release builds
+set(IE_DEBUG_POSTFIX_WIN "d")
+set(IE_RELEASE_POSTFIX_WIN "")
+set(IE_DEBUG_POSTFIX_LIN "")
+set(IE_RELEASE_POSTFIX_LIN "")
+set(IE_DEBUG_POSTFIX_MAC "d")
+set(IE_RELEASE_POSTFIX_MAC "")
+
+if(WIN32)
+    set(IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_WIN})
+    set(IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_WIN})
+elseif(APPLE)
+    set(IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_MAC})
+    set(IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_MAC})
+else()
+    set(IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_LIN})
+    set(IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_LIN})
+endif()
+
+set(CMAKE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX})
+set(CMAKE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX})
+
+if (WIN32)
+    # Support CMake multiconfiguration for Visual Studio build
+    set(IE_BUILD_POSTFIX $<$<CONFIG:Debug>:${IE_DEBUG_POSTFIX}>$<$<CONFIG:Release>:${IE_RELEASE_POSTFIX}>)
+    set(IE_BUILD_CONFIGURATION $<CONFIG>)
+else ()
+    if (${CMAKE_BUILD_TYPE} STREQUAL "Debug" )
+        set(IE_BUILD_POSTFIX ${IE_DEBUG_POSTFIX})
+    else()
+        set(IE_BUILD_POSTFIX ${IE_RELEASE_POSTFIX})
+    endif()
+    set(IE_BUILD_CONFIGURATION ${CMAKE_BUILD_TYPE})
+endif()
+message(STATUS "BUILD_CONFIGURATION: ${IE_BUILD_CONFIGURATION}")
+
+add_definitions(-DIE_BUILD_POSTFIX=\"${IE_BUILD_POSTFIX}\")
+
+if(NOT UNIX)
+    if (WIN32)
+        # set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} /MT")
+        # set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /MTd")
+    endif()
+    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(CMAKE_LIBRARY_PATH ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
+    set(LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}) # compatibility issue: linux uses LIBRARY_OUTPUT_PATH, windows uses LIBRARY_OUTPUT_DIRECTORY
+else()
+    set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
+    set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
+    set(CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
+    set(CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
+    set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION})
+    set(LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/${IE_BUILD_CONFIGURATION}/lib)
+    set(LIBRARY_OUTPUT_PATH ${LIBRARY_OUTPUT_DIRECTORY}/lib)
+endif()
+
+if(APPLE)
+	set(CMAKE_MACOSX_RPATH 1)
+endif(APPLE)
+
+# rpath fully disabled
+if (NOT ENABLE_PLUGIN_RPATH)
+    set(CMAKE_SKIP_RPATH TRUE)
+endif()
+
+# prepare temporary folder
+function(set_temp_directory temp_variable source_tree_dir)
+    if (DEFINED ENV{${DL_SDK_TEMP}} AND NOT $ENV{${DL_SDK_TEMP}} STREQUAL "")
+        if (WIN32)
+            string(REPLACE "\\" "\\\\" temp $ENV{${DL_SDK_TEMP}})
+        else(WIN32)
+            set(temp $ENV{${DL_SDK_TEMP}})
+        endif(WIN32)
+
+        if (ENABLE_ALTERNATIVE_TEMP)
+            set(ALTERNATIVE_PATH ${source_tree_dir}/temp)
+        endif()
+    else ()
+        message(STATUS "DL_SDK_TEMP envionment not set")
+        set(temp ${source_tree_dir}/temp)
+    endif()
+
+    set("${temp_variable}" "${temp}" PARENT_SCOPE)
+    if(ALTERNATIVE_PATH)
+        set(ALTERNATIVE_PATH "${ALTERNATIVE_PATH}" PARENT_SCOPE)
+    endif()
+endfunction()
+
+# Use solution folders
+set_property(GLOBAL PROPERTY USE_FOLDERS ON)
+
+set(CMAKE_POSITION_INDEPENDENT_CODE ON)
+
+include(os_flags)
+include(sdl)
+include(sanitizer)
+include(cpplint)
+include(cppcheck)
+
+function(set_ci_build_number)
+    set(IE_MAIN_SOURCE_DIR "${CMAKE_SOURCE_DIR}")
+    include(version)
+    set(CI_BUILD_NUMBER "${CI_BUILD_NUMBER}" PARENT_SCOPE)
+endfunction()
+set_ci_build_number()
+
+if(ENABLE_PROFILING_ITT)
+    find_package(ITT REQUIRED)
+endif()
+
+include(plugins/plugins)

inference-engine/cmake/developer_package_config.cmake.in

@@ -0,0 +1,48 @@
+# Copyright (C) 2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+set(IE_MAIN_SOURCE_DIR "@CMAKE_SOURCE_DIR@")
+file(TO_CMAKE_PATH "${CMAKE_CURRENT_LIST_DIR}" cache_path)
+
+# inherit OpenCV from main IE project
+load_cache("${cache_path}" READ_WITH_PREFIX "" OpenCV_DIR)
+find_package(OpenCV COMPONENTS imgcodecs)
+
+# Targets
+
+include("${CMAKE_CURRENT_LIST_DIR}/targets_developer.cmake")
+
+# add additional interface include directories needed for plugin development
+if(NOT TARGET IE::inference_engine)
+	message(FATAL_ERROR "The target IE::inference_engine does not exist")
+endif()
+
+set(ie_plugin_headers "${IE_MAIN_SOURCE_DIR}/src/inference_engine")
+set_property(TARGET IE::inference_engine APPEND PROPERTY INTERFACE_INCLUDE_DIRECTORIES "${ie_plugin_headers}")
+set_property(TARGET IE::inference_engine PROPERTY IMPORTED_GLOBAL TRUE)
+
+get_target_property(InferenceEngine_INCLUDE_DIRS IE::inference_engine INTERFACE_INCLUDE_DIRECTORIES)
+set(InferenceEngine_LIBRARIES IE::inference_engine)
+
+# Variables to export in plugin's projects
+
+set(ie_options "@IE_OPTIONS@;CMAKE_BUILD_TYPE")
+
+load_cache("${cache_path}" READ_WITH_PREFIX "" ${ie_options})
+
+message(STATUS "The following CMake options are exported from Inference Engine Developer package")
+message("")
+foreach(option IN LISTS ie_options)
+	message("    ${option}: ${${option}}")
+endforeach()
+message("")
+
+#
+# Common cmake includes
+#
+
+list(APPEND CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake;${IE_MAIN_SOURCE_DIR}/cmake")
+
+# generic stuff from developer package
+include(developer_package)

inference-engine/cmake/download.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/download_and_apply.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/download_and_check.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/download_and_extract.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -146,7 +145,7 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
   if(DEFINED ENV{IE_PATH_TO_DEPS})
     set(URL "$ENV{IE_PATH_TO_DEPS}/${RELATIVE_URL}")
   else()
-    set(URL "https://download.01.org/opencv/2019/openvinotoolkit/R1/inference_engine/${RELATIVE_URL}")
+    set(URL "https://download.01.org/opencv/2019/openvinotoolkit/R2/inference_engine/${RELATIVE_URL}")
   endif()
 
   #no message on recursive calls

inference-engine/cmake/extract.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/features.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
@@ -8,7 +7,6 @@ include (options)
 #this options are aimed to optimize build time on development system
 
 #backed targets
-
 ie_option (ENABLE_GNA "GNA support for inference engine" ON)
 
 ie_option (ENABLE_MKL_DNN "MKL-DNN plugin for inference engine" ON)
@@ -31,49 +29,37 @@ list (APPEND IE_OPTIONS GEMM)
 
 # "MKL-DNN library based on OMP or TBB or Sequential implementation: TBB|OMP|SEQ"
 if (NOT THREADING STREQUAL "TBB"
+        AND NOT THREADING STREQUAL "TBB_AUTO"
         AND NOT THREADING STREQUAL "OMP"
         AND NOT THREADING STREQUAL "SEQ")
     set (THREADING "TBB")
-    message(STATUS "THREADING should be set to TBB, OMP or SEQ. Default option is " ${THREADING})
+    message(STATUS "THREADING should be set to TBB, TBB_AUTO, OMP or SEQ. Default option is " ${THREADING})
 endif()
 set(THREADING "${THREADING}" CACHE STRING "Threading" FORCE)
 list (APPEND IE_OPTIONS THREADING)
 
-# Enable postfixes for Debug/Release builds
-set (IE_DEBUG_POSTFIX_WIN "d")
-set (IE_RELEASE_POSTFIX_WIN "")
-set (IE_DEBUG_POSTFIX_LIN "")
-set (IE_RELEASE_POSTFIX_LIN "")
-set (IE_DEBUG_POSTFIX_MAC "d")
-set (IE_RELEASE_POSTFIX_MAC "")
+ie_option (ENABLE_VPU "vpu targeted plugins for inference engine" ON)
 
-if (WIN32)
-    set (IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_WIN})
-    set (IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_WIN})
-elseif(APPLE)
-    set (IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_MAC})
-    set (IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_MAC})
-else()
-    set (IE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX_LIN})
-    set (IE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX_LIN})
-endif()
-set(IE_DEBUG_POSTFIX "${IE_DEBUG_POSTFIX}" CACHE STRING "Debug postfix" FORCE)
-list (APPEND IE_OPTIONS IE_DEBUG_POSTFIX)
-set(IE_RELEASE_POSTFIX "${IE_RELEASE_POSTFIX}" CACHE STRING "Release postfix" FORCE)
-list (APPEND IE_OPTIONS IE_RELEASE_POSTFIX)
+ie_option (ENABLE_MYRIAD "myriad targeted plugin for inference engine" ON)
+
+ie_option (ENABLE_MYRIAD_NO_BOOT "myriad plugin will skip device boot" OFF)
 
 ie_option (ENABLE_TESTS "unit and functional tests" OFF)
 
-ie_option (ENABLE_GAPI_TESTS "unit tests for GAPI kernels" OFF)
+ie_option (ENABLE_GAPI_TESTS "tests for GAPI kernels" OFF)
 
 ie_option (GAPI_TEST_PERF "if GAPI unit tests should examine performance" OFF)
 
+ie_option (ENABLE_MYRIAD_MVNC_TESTS "functional and behavior tests for mvnc api" OFF)
+
 ie_option (ENABLE_SAMPLES "console samples are part of inference engine package" ON)
 
 ie_option (ENABLE_SAMPLES_CORE "console samples core library" ON)
 
 ie_option (ENABLE_SANITIZER "enable checking memory errors via AddressSanitizer" OFF)
 
+ie_option (ENABLE_FUZZING "instrument build for fuzzing" OFF)
+
 ie_option (COVERAGE "enable code coverage" OFF)
 
 ie_option (ENABLE_STRESS_UNIT_TESTS "stress unit tests" OFF)
@@ -100,10 +86,28 @@ ie_option (ENABLE_DEBUG_SYMBOLS "generates symbols for debugging" OFF)
 
 ie_option (ENABLE_PYTHON "enables ie python bridge build" OFF)
 
+ie_option (DEVELOPMENT_PLUGIN_MODE "Disabled build of all plugins" OFF)
+
 ie_option (TREAT_WARNING_AS_ERROR "Treat build warnings as errors" ON)
 
-ie_option(ENABLE_CPPLINT "Enable cpplint checks during the build" OFF)
-ie_option(ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF)
+ie_option (ENABLE_UNICODE_PATH_SUPPORT "Enable loading models from Unicode paths" ON)
+
+if (UNIX AND NOT APPLE AND CMAKE_COMPILER_IS_GNUCC AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 5.3)
+    set(ENABLE_UNICODE_PATH_SUPPORT OFF)
+endif()
+
+if (UNIX AND NOT APPLE)
+    ie_option(ENABLE_CPPLINT "Enable cpplint checks during the build" ON)
+    ie_option(ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF)
+else()
+    set(ENABLE_CPPLINT OFF)
+endif()
+
+if (UNIX AND NOT APPLE AND CMAKE_VERSION VERSION_GREATER_EQUAL 3.10)
+    ie_option(ENABLE_CPPCHECK "Enable cppcheck during the build" ON)
+else()
+    set(ENABLE_CPPCHECK OFF)
+endif()
 
 #environment variables used
 

inference-engine/cmake/fuzzing.cmake

@@ -0,0 +1,30 @@
+# Copyright (C) 2018-2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+function(enable_fuzzing)
+    # Enable (libFuzzer)[https://llvm.org/docs/LibFuzzer.html] if supported.
+    if(CMAKE_CXX_COMPILER_ID MATCHES "Clang" AND NOT WIN32)
+        # Communicate libfuzzer is enabled
+        set(WITH_LIBFUZZER ON PARENT_SCOPE)
+        add_compile_definitions(WITH_LIBFUZZER)
+
+        # Enable libfuzzer and code coverage
+        set(FUZZING_COMPILER_FLAGS "-fsanitize=fuzzer-no-link -fprofile-instr-generate -fcoverage-mapping")
+        set(FUZZING_LINKER_FLAGS "-fsanitize-coverage=trace-pc-guard -fprofile-instr-generate")
+
+        set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${FUZZING_COMPILER_FLAGS}" PARENT_SCOPE)
+        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${FUZZING_COMPILER_FLAGS}" PARENT_SCOPE)
+        set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${FUZZING_LINKER_FLAGS}" PARENT_SCOPE)        
+        set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${FUZZING_LINKER_FLAGS}")
+    endif()
+endfunction(enable_fuzzing)
+
+
+function(add_fuzzer FUZZER_EXE_NAME FUZZER_SOURCES)
+    add_executable(${FUZZER_EXE_NAME} ${FUZZER_SOURCES})
+    if(WITH_LIBFUZZER)
+        set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -fsanitize=fuzzer" PARENT_SCOPE)
+    endif()
+    target_link_libraries(${FUZZER_EXE_NAME} PRIVATE fuzz-testhelper)
+endfunction(add_fuzzer)

inference-engine/cmake/ie_parallel.cmake

@@ -1,12 +1,11 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
 function(set_ie_threading_interface_for TARGET_NAME)
     set(IE_THREAD_DEFINE "IE_THREAD_SEQ")
 
-    if (THREADING STREQUAL "TBB")
+    if (THREADING STREQUAL "TBB" OR THREADING STREQUAL "TBB_AUTO")
         if (NOT (IE_MAIN_SOURCE_DIR))
             set(incl_path ${IE_EXTERNAL_DIR}/tbb/include)
             if (WIN32)
@@ -22,7 +21,7 @@ function(set_ie_threading_interface_for TARGET_NAME)
             set(lib_dbg_path ${lib_rel_path})
         endif ()
 
-        if (NOT TBB_INCLUDE_DIRS OR NOT TBB_LIBRARIES_RELEASE OR NOT TBB_LIBRARIES_DEBUG)
+        if (NOT TBB_INCLUDE_DIRS OR NOT TBB_LIBRARIES_RELEASE)
             find_path(TBB_INCLUDE_DIRS tbb/tbb.h ${incl_path} NO_DEFAULT_PATH)
             find_library(TBB_LIBRARIES_RELEASE tbb ${lib_rel_path} NO_DEFAULT_PATH)
             find_library(TBB_LIBRARIES_DEBUG tbb_debug ${lib_dbg_path} NO_DEFAULT_PATH)
@@ -31,20 +30,31 @@ function(set_ie_threading_interface_for TARGET_NAME)
             ext_message(STATUS "TBB Debug lib: ${TBB_LIBRARIES_DEBUG}")
         endif ()
 
-        if (NOT TBB_INCLUDE_DIRS OR NOT TBB_LIBRARIES_RELEASE OR NOT TBB_LIBRARIES_DEBUG)
+        if (NOT TBB_INCLUDE_DIRS OR NOT TBB_LIBRARIES_RELEASE)
             ext_message(WARNING "TBB not found. TBB support will be disabled. ${IE_THREAD_DEFINE} is defined")
         else ()
             set(IE_THREAD_DEFINE "IE_THREAD_TBB")
+
             target_include_directories(${TARGET_NAME} PUBLIC ${TBB_INCLUDE_DIRS})
             if (WIN32)
                 target_link_libraries(${TARGET_NAME} PUBLIC "-nodefaultlib:vcomp")
-                target_link_libraries(${TARGET_NAME} PUBLIC "$<$<CONFIG:DEBUG>:${TBB_LIBRARIES_DEBUG}>;$<$<NOT:$<CONFIG:DEBUG>>:${TBB_LIBRARIES_RELEASE}>")
-            else()
-                if ("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
-                    target_link_libraries(${TARGET_NAME} PUBLIC ${TBB_LIBRARIES_DEBUG})
-                else()
-                    target_link_libraries(${TARGET_NAME} PUBLIC ${TBB_LIBRARIES_RELEASE})
+            endif ()
+
+            # Debug binaries are optional.
+            if (TBB_LIBRARIES_DEBUG)
+                if (WIN32)
+                    target_link_libraries(${TARGET_NAME} PUBLIC "$<$<CONFIG:DEBUG>:${TBB_LIBRARIES_DEBUG}>;$<$<NOT:$<CONFIG:DEBUG>>:${TBB_LIBRARIES_RELEASE}>")
+                else ()
+                    if ("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
+                        target_link_libraries(${TARGET_NAME} PUBLIC ${TBB_LIBRARIES_DEBUG})
+                    else()
+                        target_link_libraries(${TARGET_NAME} PUBLIC ${TBB_LIBRARIES_RELEASE})
+                    endif ()
                 endif ()
+            else ()
+                # Link Release library to all configurations.
+                ext_message(WARNING "TBB Debug binaries are missed.")
+                target_link_libraries(${TARGET_NAME} PUBLIC ${TBB_LIBRARIES_RELEASE})
             endif ()
         endif ()
     elseif (THREADING STREQUAL "OMP")
@@ -67,31 +77,41 @@ function(set_ie_threading_interface_for TARGET_NAME)
             set(lib_dbg_path ${lib_rel_path})
         endif ()
 
-        if (NOT OMP_LIBRARIES_RELEASE OR NOT OMP_LIBRARIES_DEBUG)
+        if (NOT OMP_LIBRARIES_RELEASE)
             find_library(OMP_LIBRARIES_RELEASE ${omp_lib_name} ${lib_rel_path} NO_DEFAULT_PATH)
             find_library(OMP_LIBRARIES_DEBUG ${omp_lib_name} ${lib_dbg_path} NO_DEFAULT_PATH)
             ext_message(STATUS "OMP Release lib: ${OMP_LIBRARIES_RELEASE}")
             ext_message(STATUS "OMP Debug lib: ${OMP_LIBRARIES_DEBUG}")
         endif ()
 
-        if (NOT OMP_LIBRARIES_RELEASE OR NOT OMP_LIBRARIES_DEBUG)
+        if (NOT OMP_LIBRARIES_RELEASE)
             ext_message(WARNING "Intel OpenMP not found. Intel OpenMP support will be disabled. ${IE_THREAD_DEFINE} is defined")
         else ()
             set(IE_THREAD_DEFINE "IE_THREAD_OMP")
-            
+
             if (WIN32)
                 target_compile_options(${TARGET_NAME} PUBLIC ${OpenMP_CXX_FLAGS} /openmp)
                 target_compile_options(${TARGET_NAME} PUBLIC ${OpenMP_CXX_FLAGS} /Qopenmp)
-
                 target_link_libraries(${TARGET_NAME} PUBLIC "-nodefaultlib:vcomp")
-                target_link_libraries(${TARGET_NAME} PUBLIC "$<$<CONFIG:DEBUG>:${OMP_LIBRARIES_DEBUG}>;$<$<NOT:$<CONFIG:DEBUG>>:${OMP_LIBRARIES_RELEASE}>")
             else()
                 target_compile_options(${TARGET_NAME} PUBLIC ${OpenMP_CXX_FLAGS} -fopenmp)
-                if ("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
-                    target_link_libraries(${TARGET_NAME} PUBLIC ${OMP_LIBRARIES_DEBUG})
+            endif ()
+
+            # Debug binaries are optional.
+            if (OMP_LIBRARIES_DEBUG)
+                if (WIN32)
+                    target_link_libraries(${TARGET_NAME} PUBLIC "$<$<CONFIG:DEBUG>:${OMP_LIBRARIES_DEBUG}>;$<$<NOT:$<CONFIG:DEBUG>>:${OMP_LIBRARIES_RELEASE}>")
                 else()
-                    target_link_libraries(${TARGET_NAME} PUBLIC ${OMP_LIBRARIES_RELEASE})
+                    if ("${CMAKE_BUILD_TYPE}" STREQUAL "Debug")
+                        target_link_libraries(${TARGET_NAME} PUBLIC ${OMP_LIBRARIES_DEBUG})
+                    else()
+                        target_link_libraries(${TARGET_NAME} PUBLIC ${OMP_LIBRARIES_RELEASE})
+                    endif ()
                 endif ()
+            else ()
+                # Link Release library to all configurations.
+                ext_message(WARNING "OMP Debug binaries are missed.")
+                target_link_libraries(${TARGET_NAME} PUBLIC ${OMP_LIBRARIES_RELEASE})
             endif ()
         endif ()
 

inference-engine/cmake/linux_name.cmake

@@ -1,9 +1,8 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
-if (UNIX)
+if (LINUX)
     function(get_linux_name res_var)
         if (NOT EXISTS "/etc/lsb-release")
             execute_process(COMMAND find -L /etc/ -maxdepth 1 -type f -name *-release -exec cat {} \;

inference-engine/cmake/mode.cmake

@@ -1,6 +0,0 @@
-# Copyright (C) 2018-2019 Intel Corporation
-#
-# SPDX-License-Identifier: Apache-2.0
-#
-
-option(DEVELOPMENT_PLUGIN_MODE "Disabled build of all plugins" OFF)

inference-engine/cmake/options.cmake

@@ -1,21 +1,27 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 # Usage: ie_option(<option_variable> "description" <initial value or boolean expression> [IF <condition>])
 
 function (ie_option variable description value)
-    option(${variable} "${description}" ${value})
-    list (APPEND IE_OPTIONS "${variable}")
+	list(FIND IE_OPTIONS "${variable}" result)
 
-    set (IE_OPTIONS "${IE_OPTIONS}" PARENT_SCOPE)
+	if(${result} EQUAL -1)
+	    option(${variable} "${description}" ${value})
+	    list (APPEND IE_OPTIONS "${variable}")
+
+	    set (IE_OPTIONS "${IE_OPTIONS}" PARENT_SCOPE)
+	endif()
 endfunction()
 
 include(version)
 
 function (print_enabled_features)
-    message(STATUS "CI_BUILD_NUMBER: ${CI_BUILD_NUMBER}")
+	message(STATUS "Inference Engine enabled features: ")
+    message("")
+    message("    CI_BUILD_NUMBER: ${CI_BUILD_NUMBER}")
     foreach(_var ${IE_OPTIONS})
-        message(STATUS "${_var} = ${${_var}}")
+        message("    ${_var} = ${${_var}}")
     endforeach()
+    message("")
 endfunction()

inference-engine/cmake/os_flags.cmake

@@ -1,14 +1,35 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
+macro(disable_deprecated_warnings)
+    if(WIN32)
+        if("${CMAKE_CXX_COMPILER_ID}" MATCHES Intel)
+            set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /Qdiag-warning:1478")
+        elseif("${CMAKE_CXX_COMPILER_ID}" STREQUAL MSVC)
+            set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /wd4996") # disable warning on deprecated API
+        endif()
+    else()
+        set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Wno-deprecated-declarations")
+    endif()
+endmacro()
+
 if (WIN32)
     set_property(DIRECTORY APPEND PROPERTY COMPILE_DEFINITIONS _CRT_SECURE_NO_WARNINGS)
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_SCL_SECURE_NO_WARNINGS")
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /EHsc") #no asynchronous structured exception handling
     set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /LARGEADDRESSAWARE")
 
+    if (TREAT_WARNING_AS_ERROR)
+        if(CMAKE_CXX_COMPILER_ID MATCHES Intel)
+            set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /WX")
+            set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /Qdiag-warning:2586,177,3180,1740,1786,47,161")
+        elseif (CMAKE_CXX_COMPILER_ID MATCHES MSVC)
+#            set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /WX") # Too many warnings
+        endif()
+    endif()
+
+
     set(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /Z7")
     set(CMAKE_CXX_FLAGS_DEBUG "${CMAKE_CXX_FLAGS_DEBUG} /Z7")
 
@@ -26,7 +47,6 @@ if (WIN32)
         set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${DEBUG_SYMBOLS_LINKER_FLAGS}")
         set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${DEBUG_SYMBOLS_LINKER_FLAGS}")
     endif()
-
 else()
     set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror -Werror=return-type ")
     if (APPLE)

inference-engine/cmake/plugins/create_plugin_file.cmake

@@ -0,0 +1,27 @@
+# Copyright (C) 2018-2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+set(newContent "        <plugin name=\"${IE_DEVICE_NAME}\" location=\"${IE_PLUGIN_LIBRARY_NAME}\">")
+
+if(IE_PLUGIN_PROPERTIES)
+    set(newContent "${newContent}
+            <properties>")
+
+    foreach(props IN LISTS IE_PLUGIN_PROPERTIES)
+        string(REPLACE "," ";" props "${props}")
+
+        list(GET props 0 key)
+        list(GET props 1 value)
+        set(newContent "${newContent}
+                <property key=\"${key}\" value=\"${value}\"/>")
+    endforeach()
+
+    set(newContent "${newContent}
+            </properties>")
+endif()
+
+set(newContent "${newContent}
+        </plugin>")
+
+file(WRITE "${IE_CONFIG_OUTPUT_FILE}" "${newContent}")

inference-engine/cmake/plugins/plugins.cmake

@@ -0,0 +1,132 @@
+# Copyright (C) 2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+include(CMakeParseArguments)
+
+set(PLUGIN_FILES "" CACHE INTERNAL "")
+
+function(get_shared_library_name target_name library_name)
+    set(LIB_PREFIX "${CMAKE_SHARED_LIBRARY_PREFIX}")
+    set(LIB_SUFFIX "${IE_BUILD_POSTFIX}${CMAKE_SHARED_LIBRARY_SUFFIX}")
+
+    set("${library_name}" "${LIB_PREFIX}${target_name}${LIB_SUFFIX}" PARENT_SCOPE)
+endfunction()
+
+if(NOT TARGET ie_plugins)
+    add_custom_target(ie_plugins)
+endif()
+
+#
+# ie_add_plugin(NAME <targetName>
+#               DEVICE_NAME <deviceName>
+#               SOURCES <sources>
+#               VERSION_DEFINES_FOR <source>
+#               )
+#
+function(ie_add_plugin)
+    set(options)
+    set(oneValueArgs NAME DEVICE_NAME VERSION_DEFINES_FOR)
+    set(multiValueArgs SOURCES)
+    cmake_parse_arguments(IE_PLUGIN "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
+
+    if(NOT IE_PLUGIN_NAME)
+        message(FATAL_ERROR "Please, specify plugin target name")
+    endif()
+
+    if(NOT IE_PLUGIN_DEVICE_NAME)
+        message(FATAL_ERROR "Please, specify device name for ${IE_PLUGIN_NAME}")
+    endif()
+
+    # create and configure target
+
+    if(IE_PLUGIN_VERSION_DEFINES_FOR)
+        addVersionDefines(${IE_PLUGIN_VERSION_DEFINES_FOR} CI_BUILD_NUMBER)
+    endif()
+
+    add_library(${IE_PLUGIN_NAME} SHARED ${IE_PLUGIN_SOURCES})
+    target_compile_definitions(${IE_PLUGIN_NAME} PRIVATE IMPLEMENT_INFERENCE_ENGINE_PLUGIN)
+
+    if(WIN32)
+        set_target_properties(${IE_PLUGIN_NAME} PROPERTIES COMPILE_PDB_NAME ${TARGET_NAME})
+    endif()
+
+    add_cpplint_target(${IE_PLUGIN_NAME}_cpplint FOR_TARGETS ${IE_PLUGIN_NAME})
+
+    # append plugin to the list to register
+
+    list(APPEND PLUGIN_FILES "${IE_PLUGIN_DEVICE_NAME}:${IE_PLUGIN_NAME}")
+    list(REMOVE_DUPLICATES PLUGIN_FILES)
+    set(PLUGIN_FILES "${PLUGIN_FILES}" CACHE INTERNAL "" FORCE)
+
+    add_dependencies(ie_plugins ${IE_PLUGIN_NAME})
+endfunction()
+
+#
+# ie_register_plugins(MAIN_TARGET <main target name>
+#                     POSSIBLE_PLUGINS <list of plugins which can be build by this repo>)
+#
+macro(ie_register_plugins)
+    set(options)
+    set(oneValueArgs MAIN_TARGET)
+    set(multiValueArgs POSSIBLE_PLUGINS)
+    cmake_parse_arguments(IE_REGISTER "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
+
+    if(NOT IE_REGISTER_MAIN_TARGET)
+        message(FATAL_ERROR "Please, define MAIN_TARGET")
+    endif()
+
+    set(plugins_to_remove ${IE_REGISTER_POSSIBLE_PLUGINS})
+    set(plugin_files_local)
+    set(config_output_file "$<TARGET_FILE_DIR:${IE_REGISTER_MAIN_TARGET}>/plugins.xml")
+
+    foreach(plugin IN LISTS plugins_to_remove)
+        add_custom_command(TARGET ${IE_REGISTER_MAIN_TARGET} POST_BUILD
+                  COMMAND
+                    "${CMAKE_COMMAND}"
+                    -D "IE_CONFIG_OUTPUT_FILE=${config_output_file}"
+                    -D "IE_PLUGIN_NAME=${plugin}"
+                    -D "IE_CONFIGS_DIR=${CMAKE_BINARY_DIR}/plugins"
+                    -P "${IE_MAIN_SOURCE_DIR}/cmake/plugins/unregister_plugin_cmake.cmake"
+                  COMMENT
+                    "Remove ${plugin} from the plugins.xml file"
+                  VERBATIM)
+    endforeach()
+
+    foreach(name IN LISTS PLUGIN_FILES)
+        string(REPLACE ":" ";" name "${name}")
+        list(LENGTH name length)
+        if(NOT ${length} EQUAL 2)
+            message(FATAL_ERROR "Unexpected error, please, contact developer of this script")
+        endif()
+        list(GET name 0 device_name)
+        list(GET name 1 name)
+
+        # create plugin file
+        set(config_file_name "${CMAKE_BINARY_DIR}/plugins/${name}.xml")
+        get_shared_library_name(${name} library_name)
+
+        add_custom_command(TARGET ${IE_REGISTER_MAIN_TARGET} POST_BUILD
+           COMMAND
+              "${CMAKE_COMMAND}"
+              -D "IE_CONFIG_OUTPUT_FILE=${config_file_name}"
+              -D "IE_DEVICE_NAME=${device_name}"
+              -D "IE_PLUGIN_LIBRARY_NAME=${library_name}"
+              -P "${IE_MAIN_SOURCE_DIR}/cmake/plugins/create_plugin_file.cmake"
+          COMMENT "Register ${name} plugin"
+          VERBATIM)
+
+        list(APPEND plugin_files_local "${config_file_name}")
+    endforeach()
+
+    add_custom_command(TARGET ${IE_REGISTER_MAIN_TARGET} POST_BUILD
+                      COMMAND
+                        "${CMAKE_COMMAND}"
+                        -D "CMAKE_SHARED_LIBRARY_PREFIX=${CMAKE_SHARED_LIBRARY_PREFIX}"
+                        -D "IE_CONFIG_OUTPUT_FILE=${config_output_file}"
+                        -D "IE_CONFIGS_DIR=${CMAKE_BINARY_DIR}/plugins"
+                        -P "${IE_MAIN_SOURCE_DIR}/cmake/plugins/register_plugin_cmake.cmake"
+                      COMMENT
+                        "Registering plugins to plugins.xml config file"
+                      VERBATIM)
+endmacro()

inference-engine/cmake/plugins/register_plugin_cmake.cmake

@@ -0,0 +1,65 @@
+# Copyright (C) 2018-2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+set(file_content
+"<ie>
+    <plugins>
+    </plugins>
+</ie>")
+
+if(NOT EXISTS "${IE_CONFIG_OUTPUT_FILE}")
+    file(WRITE "${IE_CONFIG_OUTPUT_FILE}" "${file_content}")
+endif()
+
+# get list of plugin files
+file(GLOB plugin_files "${IE_CONFIGS_DIR}/*.xml")
+
+function(check_plugin_exists plugin_name outvar)
+	set(${outvar} OFF PARENT_SCOPE)
+
+	# check if config file already has this plugin
+	file(STRINGS "${IE_CONFIG_OUTPUT_FILE}" content REGEX "plugin .*=\"")
+
+	foreach(line IN LISTS content)
+	    string(REGEX MATCH "location=\"([^\"]*)\"" location "${line}")
+	    get_filename_component(location "${CMAKE_MATCH_1}" NAME_WE)
+
+	    if("${CMAKE_SHARED_LIBRARY_PREFIX}${plugin_name}" MATCHES "${location}")
+	        # plugin has already registered
+			set(${outvar} ON PARENT_SCOPE)
+	    endif()
+	endforeach()
+endfunction()
+
+set(plugin_files_to_add)
+foreach(plugin_file IN LISTS plugin_files)
+	get_filename_component(plugin_name "${plugin_file}" NAME_WE)
+	check_plugin_exists("${plugin_name}" exists)
+
+	if(NOT exists)
+		list(APPEND plugin_files_to_add "${plugin_file}")
+	endif()
+endforeach()
+
+# add plugin
+set(newContent "")
+file(STRINGS "${IE_CONFIG_OUTPUT_FILE}" content)
+
+foreach(line IN LISTS content)
+    if("${line}" MATCHES "</plugins>")
+    	foreach(plugin_file IN LISTS plugin_files_to_add)
+    		file(READ "${plugin_file}" content)
+        	set(newContent "${newContent}
+${content}")
+		endforeach()
+    endif()
+
+    if(newContent)
+        set(newContent "${newContent}\n${line}")
+    else()
+        set(newContent "${line}")
+    endif()
+endforeach()
+
+file(WRITE "${IE_CONFIG_OUTPUT_FILE}" "${newContent}")

inference-engine/cmake/plugins/unregister_plugin_cmake.cmake

@@ -0,0 +1,35 @@
+# Copyright (C) 2019 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+if(NOT EXISTS "${IE_CONFIG_OUTPUT_FILE}")
+	return()
+endif()
+
+# remove plugin file
+file(REMOVE "${IE_CONFIGS_DIR}/${IE_PLUGIN_NAME}.xml")
+
+# remove plugin
+set(newContent "")
+file(STRINGS "${IE_CONFIG_OUTPUT_FILE}" content)
+
+set(skip_plugin OFF)
+foreach(line IN LISTS content)
+    if("${line}" MATCHES "${IE_PLUGIN_NAME}")
+    	set(skip_plugin ON)
+    endif()
+
+    if(NOT skip_plugin)
+	    if(newContent)
+	        set(newContent "${newContent}\n${line}")
+	    else()
+	        set(newContent "${line}")
+	    endif()
+    endif()
+
+    if("${line}" MATCHES "</plugin>")
+    	set(skip_plugin OFF)
+    endif()
+endforeach()
+
+file(WRITE "${IE_CONFIG_OUTPUT_FILE}" "${newContent}")

inference-engine/cmake/sanitizer.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/sdl.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/cmake/share/InferenceEngineConfig-version.cmake.in

@@ -1,9 +1,8 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 
-set(InferenceEngine_VERSION 1.6.0)
+set(InferenceEngine_VERSION 2.0.0)
 set(PACKAGE_VERSION ${InferenceEngine_VERSION})
 
 set(PACKAGE_VERSION_EXACT False)

inference-engine/cmake/share/InferenceEngineConfig.cmake.in

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 #
@@ -42,11 +41,10 @@ else()
     if (WIN32)
         set(_ARCH intel64)
     else()
-        if(CMAKE_SYSTEM_PROCESSOR STREQUAL "armv7l")
-            set(_ARCH armv7l)
-        elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86_64")
+        string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} _ARCH)
+        if(_ARCH STREQUAL "x86_64" OR _ARCH STREQUAL "amd64") # Windows detects Intel's 64-bit CPU as AMD64
             set(_ARCH intel64)
-        elseif(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "i386")
+        elseif(_ARCH STREQUAL "i386")
             set(_ARCH ia32)
         endif()
     endif()
@@ -70,59 +68,32 @@ else()
         endif()
     endif()
 
-    if(NOT IE_ROOT_DIR)  
-        ext_message(FATAL_ERROR "inference_engine directory is not found")
+    if(NOT IE_ROOT_DIR)
+        ext_message(FATAL_ERROR "inference_engine root directory is not found")
     endif()
 
+    find_path(IE_INCLUDE_DIR inference_engine.hpp "${IE_ROOT_DIR}/include" NO_DEFAULT_PATH)
+    find_path(IE_SRC_DIR extension "${IE_ROOT_DIR}/src" NO_DEFAULT_PATH)
 
-    if(IE_INCLUDE_DIR AND NOT "${IE_ROOT_DIR}/include" EQUAL "${IE_INCLUDE_DIR}")
-        unset(IE_INCLUDE_DIR CACHE)
-    endif()
-
-    if(IE_SRC_DIR AND NOT "${IE_ROOT_DIR}/src" EQUAL "${IE_SRC_DIR}")
-        unset(IE_SRC_DIR CACHE)
-    endif()
-
-    if(IE_LIBRARY AND NOT "${IE_ROOT_DIR}/lib/${_ARCH}" EQUAL "${IE_LIBRARY}")
-        unset(IE_LIBRARY CACHE)
-    endif()
-
-    set(_IE_ROOT_INCLUDE_DIR "${IE_ROOT_DIR}/include")
-    set(_IE_ROOT_SRC_DIR "${IE_ROOT_DIR}/src")
-    set(_IE_ROOT_LIBRARY "${IE_ROOT_DIR}/lib/${_ARCH}")
-
-    find_path(IE_INCLUDE_DIR inference_engine.hpp "${_IE_ROOT_INCLUDE_DIR}")
-    find_path(IE_SRC_DIR extension "${_IE_ROOT_SRC_DIR}")
-
-    set(IE_LIB_DIR "${_IE_ROOT_LIBRARY}")
+    set(IE_LIB_DIR "${IE_ROOT_DIR}/lib/${_ARCH}")
     set(IE_LIB_REL_DIR "${IE_LIB_DIR}/Release")
     set(IE_LIB_DBG_DIR "${IE_LIB_DIR}/Debug")
-    set(IE_EXTERNAL_DIR "${IE_ROOT_DIR}/external")
 
     include(FindPackageHandleStandardArgs)
 
-    if (WIN32)
-        find_library(IE_RELEASE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_WIN@ "${IE_LIB_REL_DIR}")
-        find_library(IE_DEBUG_LIBRARY inference_engine@IE_DEBUG_POSTFIX_WIN@ "${IE_LIB_DBG_DIR}")
-        find_package_handle_standard_args(  InferenceEngine
-                                            FOUND_VAR INFERENCEENGINE_FOUND
-                                            REQUIRED_VARS IE_RELEASE_LIBRARY IE_DEBUG_LIBRARY IE_INCLUDE_DIR
-                                            FAIL_MESSAGE "Inference Engine cannot be found at ${_IE_ROOT_LIBRARY}. Please consult InferenceEgnineConfig.cmake module's help page.")
-    elseif (APPLE)
-        find_library(IE_RELEASE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_MAC@ "${IE_LIB_DIR}")
-        find_library(IE_DEBUG_LIBRARY inference_engine@IE_DEBUG_POSTFIX_MAC@ "${IE_LIB_DIR}")
-        find_package_handle_standard_args(  InferenceEngine
-                                            FOUND_VAR INFERENCEENGINE_FOUND
-                                            REQUIRED_VARS IE_RELEASE_LIBRARY IE_DEBUG_LIBRARY IE_INCLUDE_DIR
-                                            FAIL_MESSAGE "Inference Engine cannot be found at ${_IE_ROOT_LIBRARY}. Please consult InferenceEgnineConfig.cmake module's help page.")
-
+    if(WIN32)
+        find_library(IE_RELEASE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_WIN@ "${IE_LIB_REL_DIR}" NO_DEFAULT_PATH)
+    elseif(APPLE)
+        find_library(IE_RELEASE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_MAC@ "${IE_LIB_DIR}" NO_DEFAULT_PATH)
     else()
-        find_library(IE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_LIN@ "${IE_LIB_DIR}")
-        find_package_handle_standard_args(  InferenceEngine
-                                            FOUND_VAR INFERENCEENGINE_FOUND
-                                            REQUIRED_VARS IE_LIBRARY IE_INCLUDE_DIR
-                                            FAIL_MESSAGE "Inference Engine cannot be found at ${_IE_ROOT_LIBRARY}. Please consult InferenceEgnineConfig.cmake module's help page.")
+        find_library(IE_RELEASE_LIBRARY inference_engine@IE_RELEASE_POSTFIX_LIN@ "${IE_LIB_DIR}" NO_DEFAULT_PATH)
     endif()
+
+    find_package_handle_standard_args(  InferenceEngine
+                                        FOUND_VAR INFERENCEENGINE_FOUND
+                                        REQUIRED_VARS IE_RELEASE_LIBRARY IE_INCLUDE_DIR
+                                        FAIL_MESSAGE "Some of mandatory Inference Engine components are not found. Please consult InferenceEgnineConfig.cmake module's help page.")
+
     if(INFERENCEENGINE_FOUND)
         # to keep this line for successful execution in CMake 2.8
         set(InferenceEngine_FOUND TRUE)
@@ -130,25 +101,42 @@ else()
         add_library(IE::inference_engine SHARED IMPORTED GLOBAL)
 
         if (WIN32)
-            set_property(TARGET IE::inference_engine APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG)
-            set_property(TARGET IE::inference_engine APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE)
-
             set_target_properties(IE::inference_engine PROPERTIES
+                    IMPORTED_CONFIGURATIONS RELEASE
                     IMPORTED_IMPLIB_RELEASE    "${IE_RELEASE_LIBRARY}"
-                    IMPORTED_IMPLIB_DEBUG      "${IE_DEBUG_LIBRARY}"
-                    MAP_IMPORTED_CONFIG_DEBUG Debug
                     MAP_IMPORTED_CONFIG_RELEASE Release
                     MAP_IMPORTED_CONFIG_RELWITHDEBINFO Release
                     INTERFACE_INCLUDE_DIRECTORIES "${IE_INCLUDE_DIR}")
+
+            # Debug binaries are optional
+            find_library(IE_DEBUG_LIBRARY inference_engine@IE_DEBUG_POSTFIX_WIN@ "${IE_LIB_DBG_DIR}" NO_DEFAULT_PATH)
+            if (IE_DEBUG_LIBRARY)
+                set_property(TARGET IE::inference_engine APPEND PROPERTY IMPORTED_CONFIGURATIONS DEBUG)
+                set_target_properties(IE::inference_engine PROPERTIES
+                        IMPORTED_IMPLIB_DEBUG "${IE_DEBUG_LIBRARY}"
+                        MAP_IMPORTED_CONFIG_DEBUG Debug)
+            else()
+                ext_message(WARNING "Inference Engine DEBUG binaries are missed.")
+            endif()
         elseif (APPLE)
             set_target_properties(IE::inference_engine PROPERTIES
                     IMPORTED_LOCATION_RELEASE "${IE_RELEASE_LIBRARY}"
-                    IMPORTED_LOCATION_DEBUG   "${IE_DEBUG_LIBRARY}"
                     INTERFACE_INCLUDE_DIRECTORIES "${IE_INCLUDE_DIR}")
+
+            # Debug binaries are optional
+            find_library(IE_DEBUG_LIBRARY inference_engine@IE_DEBUG_POSTFIX_MAC@ "${IE_LIB_DIR}" NO_DEFAULT_PATH)
+            if (IE_DEBUG_LIBRARY)
+                set_target_properties(IE::inference_engine PROPERTIES
+                        IMPORTED_LOCATION_DEBUG "${IE_DEBUG_LIBRARY}")
+            else()
+                ext_message(WARNING "Inference Engine DEBUG binaries are missed")
+            endif()
+
             target_link_libraries(IE::inference_engine INTERFACE ${CMAKE_DL_LIBS})
         else()
+            # Only Release binaries are distributed for Linux systems
             set_target_properties(IE::inference_engine PROPERTIES
-                    IMPORTED_LOCATION "${IE_LIBRARY}"
+                    IMPORTED_LOCATION "${IE_RELEASE_LIBRARY}"
                     INTERFACE_INCLUDE_DIRECTORIES "${IE_INCLUDE_DIR}")
             target_link_libraries(IE::inference_engine INTERFACE ${CMAKE_DL_LIBS})
         endif()
@@ -156,6 +144,7 @@ else()
         set(InferenceEngine_INCLUDE_DIRS ${IE_INCLUDE_DIR})
         set(InferenceEngine_LIBRARIES IE::inference_engine)
 
+        set(IE_EXTERNAL_DIR "${IE_ROOT_DIR}/external")
         include("${IE_ROOT_DIR}/share/ie_parallel.cmake")
 
         add_subdirectory(${IE_SRC_DIR}/extension EXCLUDE_FROM_ALL ie_cpu_extension)

inference-engine/cmake/version.cmake

@@ -1,5 +1,4 @@
 # Copyright (C) 2018-2019 Intel Corporation
-#
 # SPDX-License-Identifier: Apache-2.0
 #
 

inference-engine/ie_bridges/python/CMakeLists.txt

@@ -5,15 +5,13 @@ cmake_minimum_required (VERSION 3.3)
 project (ie_python_api)
 set (CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} ${CMAKE_CURRENT_LIST_DIR}/cmake)
 
-if (CMAKE_SYSTEM_PROCESSOR STREQUAL "armv7l")
-    set (ARCH armv7l)
-elseif ("${CMAKE_SIZEOF_VOID_P}" EQUAL "8")
-    set (ARCH intel64)
-else()
-    set (ARCH ia32)
+string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} ARCH)
+if(ARCH STREQUAL "x86_64" OR ARCH STREQUAL "amd64") # Windows detects Intel's 64-bit CPU as AMD64
+    set(ARCH intel64)
+elseif(ARCH STREQUAL "i386")
+    set(ARCH ia32)
 endif()
 
-
 # in case of independent python api build (out of Inference Engine root Cmake)
 if (NOT(IE_MAIN_SOURCE_DIR))
     if("${CMAKE_BUILD_TYPE}" STREQUAL "")
@@ -50,4 +48,5 @@ endif()
 find_package (InferenceEngine REQUIRED)
 
 set (PYTHON_BRIDGE_SRC_ROOT ${CMAKE_CURRENT_SOURCE_DIR})
-add_subdirectory (src/openvino/inference_engine)
+add_subdirectory (src/openvino/inference_engine)
+add_subdirectory (src/openvino/tools/statistics_collector)

inference-engine/ie_bridges/python/docs/api_overview.md

@@ -11,35 +11,225 @@ This API provides a simplified interface for Inference Engine functionality that
 
 ## Supported OSes
 
-Currently the Inference Engine Python\* API is supported on Ubuntu* 16.04, Microsoft Windows* 10 and CentOS* 7.3 OSes.   
+Currently the Inference Engine Python\* API is supported on Ubuntu\* 16.04 and 18.04, Windows\* 10, macOS\* 10.x and 
+CentOS\* 7.3 OSes.   
 Supported Python* versions:  
 
-* On Ubuntu 16.04: 2.7, 3.5, 3.6  
-* On Windows 10: 3.5, 3.6
-* On CentOS 7.3: 3.4, 3.5, 3.6
+| Operating System | Supported Python\* versions: |
+|:----- | :----- |
+| Ubuntu\* 16.04  |  2.7, 3.5, 3.6, 3.7 |
+| Ubuntu\* 18.04  |  2.7, 3.5, 3.6, 3.7 |
+| Windows\* 10 |  3.5, 3.6, 3.7 |
+| CentOS\* 7.3 | 3.4, 3.5, 3.6, 3.7  |
+| macOS\* 10.x  | 3.5, 3.6, 3.7 |   
+
 
 ## Setting Up the Environment
 
 To configure the environment for the Inference Engine Python\* API, run:
- * On Ubuntu 16.04: `source <INSTALL_DIR>/bin/setupvars.sh .`
- * On Windows 10: `call <INSTALL_DIR>\deployment_tools\inference_engine\python_api\setenv.bat`
+ * On Ubuntu\* 16.04 or 18.04, CentOS\* 7.4 or macOS\* 10.x: `source <INSTALL_DIR>/bin/setupvars.sh .`
+ * On Windows\* 10: `call <INSTALL_DIR>\deployment_tools\inference_engine\python_api\setenv.bat`
 
 The script automatically detects latest installed Python\* version and configures required environment if the version is supported.  
 If you want to use certain version of Python\*, set the environment variable `PYTHONPATH=<INSTALL_DIR>/deployment_tools/inference_engine/python_api/<desired_python_version>`
 after running the environment configuration script.
 
+## <a name="iecore-class"></a>IECore
+
+This class represents an Inference Engine entity and allows you to manipulate with plugins using unified interfaces.
+
+### <a name="iecore-constructor"></a>Class Constructor
+
+`__init__(xml_config_file: str = "")`
+* Parameters:    
+    
+   * `xml_config_file` - A full path to `.xml` file containing plugins configuration. 
+   If the parameter is not specified, the default configuration is handled automatically.
+* Usage examples:
+    
+   * Initialize an `IECore` object with default configuration:
+   ```py
+   ie = IECore()
+   ```
+   * Initialize an `IECore` object with a custom configuration location specified:
+   ```py
+   ie = IECore("/localdisk/plugins/my_custom_cfg.xml")
+   ```
+`.xml` file has the following structure:
+```xml
+    <ie>
+        <plugins>
+            <plugin name="" location="" optional="yes/no">
+                <extensions>
+                    <extension location=""/>
+                </extensions>
+                <properties>
+                    <property key="" value=""/>
+                </properties>
+            </plugin>
+        </plugin>
+    </ie>
+
+```
+  
+### <a name="iecore-attributes"></a>Class Attributes
+* `available_devices` - A vector of devices. The devices are returned as \[CPU, FPGA.0, FPGA.1, MYRIAD\]. 
+                        If there are more than one device of a specific type, they all are listed followed by a dot and a number.
+
+### <a name="iecore-methods"></a>Instance Methods
+
+* `get_versions(device_name: str)`
+    * Description: Returns a `namedtuple` object with versions of the plugin specified
+    * Parameters:
+        * `device_name` - Name of the the registered plugin 
+    * Return value:
+        Dictionary mapping a plugin name and `Versions` `namedtuple` object with the following fields:
+        * `major` - major plugin integer version
+        * `minor` - minor plugin integer version
+        * `build_number` - plugin build number string 
+        * `description` - plugin description string
+    * Usage example:
+```py
+ie = IECore()
+ver = ie.get_versions("CPU")["CPU"]
+print("{descr}: {maj}.{min}.{num}".format(descr=ver.description, maj=ver.major, min=ver.minor, num=ver.build_number))
+```  
+
+*  `load_network(network: IENetwork, device_name: str, config=None, num_requests: int=1)`
+    * Description: Loads a network that was read from the Intermediate Representation (IR) to the plugin with specified device name and creates an `ExecutableNetwork` object of the `IENetwork` class.
+        You can create as many networks as you need and use them simultaneously (up to the limitation of the hardware
+        resources).
+    * Parameters:
+        * `network` - A valid `IENetwork` instance
+        * `device_name` - A device name of a target plugin
+        * `num_requests` - A positive integer value of infer requests to be created. Number of infer requests is limited
+        by device capabilities.        
+        * `config` - A dictionary of plugin configuration keys and their values
+    * Return value: An  <a href="#executablenetwork">`ExecutableNetwork`</a> object
+    * Usage example:
+```py
+net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+ie = IECore()
+exec_net = plugin.load_network(network=net, device_name="CPU", num_requsts=2)
+```
+
+*  `query_network(network: IENetwork, device_name: str, config=None)`
+    * Description:   
+        Queries the plugin with specified device name what network layers are supported in the current configuration.
+        Please note that layers support depends on plugin configuration and loaded extensions.
+    * Parameters:
+        * `network` - A valid `IENetwork` instance
+        * `device_name` - A device name of a target plugin
+        * `config` - A dictionary of plugin configuration keys and their values
+    * Return value: A dictionary mapping layers and device names on which they are supported
+    * Usage example:
+```py
+net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+ie = IECore()
+exec_net = plugin.query_network(network=net, device_name="HETERO:GPU,CPU")
+```
+
+* `set_config(config: dict,  device_name: str)`
+    * Description: Sets a configuration for a plugin
+    * Parameters:
+        * `config` - a dictionary of configuration parameters as keys and their values
+        * `device_name` - a device name of a target plugin
+    * Return value: None
+    * Usage examples:
+	See the `set_affinity` method of the <a href="#ienetwork-class">`IENetwork` class</a>.
+
+* `register_plugin(plugin_name: str, device_name: str = "")`
+    * Description: Registers a new device and a plugin which implement this device inside Inference Engine.
+    * Parameters:
+        * `plugin_name` -  A name of a plugin. Depending on a platform, plugin_name is wrapped with a shared 
+                           library suffix and a prefix to identify a full name of the library
+        * `device_name` - A target device name for the plugin. If not specified, the method registers 
+                          a plugin with the default name.
+    * Return value: None
+    * Usage examples:
+```py
+ie = IECore()
+ie.register_plugin(plugin="MKLDNNPlugin", device_name="MY_NEW_PLUGIN")
+```
+
+* `register_plugins(xml_config_file: str)`
+    * Description: Registers plugins specified in an `.xml` configuration file
+    * Parameters:
+        * `xml_config_file` - A full path to `.xml` file containing plugins configuration
+    * Return value: None
+    * Usage examples:
+```py
+ie = IECore()
+ie.register_plugins("/localdisk/plugins/my_custom_cfg.xml")
+```
+
+* `unregister_plugin(device_name: str = "")`
+    * Description: Unregisters a plugin with a specified device name
+    * Parameters:
+        * `device_name` - A device name of the plugin to unregister
+    * Return value: None
+    * Usage examples:
+```py
+ie = IECore()
+plugin = IEPlugin("GPU")
+ie.register_plugin(plugin=plugin, device_name="MY_NEW_GPU")
+ie.unregister_plugin(device_name="GPU")
+```
+
+* `add_extension(extension_path: str, device_name: str)`
+    * Description: Loads extension library to the plugin with a specified device name 
+    * Parameters:
+        * `extension_path` - Path to the extensions library file to load to a plugin
+        * `device_name` - A device name of a plugin to load the extensions to
+    * Return value: None
+    * Usage examples:
+```py
+ie = IECore()
+ie.add_extension(extension_path="/some_dir/libcpu_extension_avx2.so", device_name="CPU")
+```
+
+* `get_metric(device_name: str, metric_name: str)`
+    * Description: Gets a general runtime metric for dedicated hardware. Enables to request common device properties,
+      which are <a href="#executablenetwork">`ExecutableNetwork`</a> agnostic, such as device name, 
+      temperature, and other devices-specific values.
+
+    * Parameters:
+        * device_name - A name of a device to get a metric value.
+        * metric_name - A metric name to request.
+    * Return value: A metric value corresponding to a metric key.
+    * Usage example
+```py
+ie = IECore()
+ie.get_metric(metric_name="SUPPORTED_METRICS", device_name="CPU")
+```
+
+* `get_config(device_name: str, metric_name: str)`
+    * Description: Gets a configuration dedicated to device behavior. The method targets to extract information 
+                   which can be set via SetConfig method.
+    * Parameters:
+        * device_name - A name of a device to get a metric value.
+        * metric_name - A metric name to request.
+    * Return value: A metric value corresponding to a metric key.
+    * Usage example
+```py
+ie = IECore()
+ie.get_config(metric_name="CPU_BIND_THREAD", device_name="CPU")
+```
+
 ## <a name="ienetlayer-class"></a>IENetLayer
+
 This class stores main information about the layer and allow to modify some layer parameters
-### Class attributes:
+
+### <a name="ienetlayer-attributes"></a>Class Attributes
 
 * `name` - Name of the layer
 * `type`- Layer type
 * `precision` - Layer base operating precision. Provides getter and setter interfaces.
-* `layout` - Returns the layout of shape of the layer.
-* `shape` -  Return the list of the shape of the layer.
-* `parents` - Returns a list, which contains names of layers preceding this layer.
-* `children` - Returns a list, which contains names of layers following this layer.
-* `affinity` - Layer affinity set by user or a default affinity set by the `IEPlugin.set_initial_affinity()` method.             
+* `layout` - Returns the layout of shape of the layer
+* `shape` -  Return the list of the shape of the layer
+* `parents` - Returns a list, which contains names of layers preceding this layer
+* `children` - Returns a list, which contains names of layers following this layer
+* `affinity` - Layer affinity set by user or a default affinity set by the `IEPlugin.set_initial_affinity()` method.            
                The affinity attribute provides getter and setter interfaces, so the layer affinity can be modified directly.
                For example:                          
 ```py
@@ -50,27 +240,23 @@ This class stores main information about the layer and allow to modify some laye
 >>> for l in net.layers.values():
 ...     if l.type == "Convolution":
 ...         l.affinity = "CPU"
-
 ```
+	To correctly set affinity for the network, you must first initialize and properly configure the HETERO plugin.
+	`set_config({"TARGET_FALLBACK": "HETERO:FPGA,GPU"})` function configures the plugin fallback devices and their order.
+	`plugin.set_initial_affinity(net)` function sets affinity parameter of model layers according to its support
+	on specified devices.
 
-To correctly set affinity for the network, you must first initialize and properly configure the HETERO plugin.
-`set_config({"TARGET_FALLBACK": "HETERO:FPGA,GPU"})` function configures the plugin fallback devices and their order.
-`plugin.set_initial_affinity(net)` function sets affinity parameter of model layers according to its support
-on specified devices.
+	After default affinity is set by the plugin, override the default values by setting affinity manually how it's
+	described in example above  
 
-After default affinity is set by the plugin, override the default values by setting affinity manually how it's
-described in example above  
+	To understand how default and non-default affinities are set:
 
-To understand how default and non-default affinities are set:
-
-1. Call `net.layers` function right after model loading and check that layer affinity parameter is empty.
-2. Call `plugin.set_default_affinity(net)`.
-3. Call `net.layers` and check layer affinity parameters to see how plugin set a default affinity
-4. Set layer affinity how it's described above
-5. Call `net.layers` again and check layer affinity parameters to see how it was changed after manual affinity
-   setting
-
-Please refer to `affinity_setting_demo.py` to see the full usage pipeline.
+	1. Call `net.layers` function right after model loading and check that layer affinity parameter is empty.
+	2. Call `plugin.set_default_affinity(net)`.
+	3. Call `net.layers` and check layer affinity parameters to see how plugin set a default affinity
+	4. Set layer affinity how it's described above
+	5. Call `net.layers` again and check layer affinity parameters to see how it was changed after manual affinity
+	   setting
 
 * `weights`- Dictionary with layer weights, biases or custom blobs if any
 * `params` - Layer specific parameters. Provides getter and setter interfaces to get and modify layer parameters.
@@ -83,17 +269,40 @@ Please refer to `affinity_setting_demo.py` to see the full usage pipeline.
 This class contains the information about the network model read from IR and allows you to manipulate with some model parameters such as
 layers affinity and output layers.
 
-### Class Constructor
+### <a name="ienetwork-constructor"></a>Class Constructor
 
-* `__init__(model: str, weights: str)`
-    * Parameters:
-        * model - Path to `.xml` file  of the IR
-        * weights - Path to `.bin` file  of the IR
+`__init__(model: [bytes, str], weights: [bytes, str], init_from_buffer: bool=False, ngrpah_compatibility: bool=False)`
 
-### Class attributes:
+* Parameters:    
+    
+   * `model` - An `.xml` file of the IR. Depending on `init_from_buffer` value, can be a string path or bytes with file content.
+   * `weights` - A `.bin` file of the IR. Depending on `init_from_buffer` value, can be a string path or bytes with file content.
+   * `init_from_buffer` - Defines the way of how `model` and `weights` attributes are interpreted. 
+   If  `True`, attributes are interpreted as strings with paths to .xml and .bin files of IR. If `False`, they are 
+   interpreted as Python `bytes` object with .xml and .bin files content.
+   * `ngrpah_compatibility` - Default value: `False`. If `IENetwork` initializes from 
+                              [experimental IR V7](./docs/OperationsSpecification-V7.md), set to `True`
+
+* Usage examples:
+    
+   * Initializing `IENetwork` object from IR files:
+   ```py
+   net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+   ```
+   
+   * Initializing `IENetwork` object bytes with content of IR files:    
+   ```py
+   with open(path_to_bin_file, 'rb') as f:
+       bin = f.read()
+   with open(path_to_xml_file, 'rb') as f:
+       xml = f.read()
+   net = IENetwork(model=xml, weights=bin, init_from_buffer=True)
+   ``` 
+
+### <a name="ienetwork-attributes"></a>Class Attributes
 
 * `name` - Name of the loaded network
-* `inputs` - A dictionary that maps input layer names to <a name="inputinfo-class"></a>InputInfo objects.
+* `inputs` - A dictionary that maps input layer names to <a href="#inputinfo-class">InputInfo</a> objects.
              For example, to get a shape of the input layer:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -102,7 +311,7 @@ layers affinity and output layers.
 >>> net.inputs['data'].shape
 [1, 3, 224, 224]
 ```
-* `outputs` - A dictionary that maps output layer names to <a name="inputinfo-class"></a>OutputInfo objects
+* `outputs` - A dictionary that maps output layer names to <a href="#outputinfo-class">OutputInfo</a> objects
               For example, to get a shape of the output layer:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -124,7 +333,7 @@ layers affinity and output layers.
 >>> net.inputs['data'].shape
     [4, 3, 224, 224]
 ```
-* `layers` - Return dictionary that maps network layer names to <a name="ienetlayer-class"></a>`IENetLayer`
+* `layers` - Return dictionary that maps network layer names to <a href="#ienetlayer-class">`IENetLayer`</a>
              objects containing layer properties in topological order. For example, to list all network layers:
 ```py
  >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -134,7 +343,7 @@ layers affinity and output layers.
  }
  ```
  * `stats` - Returns `LayersStatsMap` object containing dictionary that maps network layer names to calibration statistics
-            represented by <a name="layerstats-class"></a> `LayerStats` objects.
+            represented by <a href="#layerstats-class">`LayerStats`</a>  objects.
             `LayersStatsMap` class inherited from built-in python `dict` and overrides default `update()`method to allow
             to set or modify layers calibration statistics.
 ```py
@@ -147,17 +356,15 @@ layers affinity and output layers.
 For more details about low precision inference please refer to "Low-Precision 8-bit Integer Inference"
 section in Inference Engine Developers Guide documentation.
 
-### Class Methods
+### <a name="ienetwork-methods"></a>Class Methods
 
 * `from_ir(model: str, weights: str)`
-> **NOTE:** The function is deprecated. Please use `IENetwork()` class constructor to create valid instance of `IENetwork`
-    * Description:   
-        The class method serves to read the model from the `.xml` and `.bin` files of the IR.
+> **NOTE:** The function is deprecated. Please use the `IENetwork()` class constructor to create valid instance of `IENetwork`.
+    * Description: Reads the model from the `.xml` and `.bin` files of the IR.
     * Parameters:
         * model - Path to `.xml` file  of the IR
         * weights - Path to `.bin` file  of the IR
-    * Return value:
-        An instance of the `IENetwork` class
+    * Return value: An instance of the `IENetwork` class
     * Usage example:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -165,16 +372,16 @@ section in Inference Engine Developers Guide documentation.
 <inference_engine.ie_api.IENetwork object at 0x7fd7dbce54b0>
 ```
 
-### Instance Methods
+### <a name="ienetwork-inference-methods"></a>Instance Methods
 
-* `add_outputs(outputs)`:
-    * Description:        
-        The method serves to mark any intermediate layer as output layer to retrieve the inference results
-        from the specified layers.
+* `add_outputs(outputs)`
+    * Description: Marks any intermediate layer as output layer to retrieve the inference results
+      from the specified layers.
     * Parameters:
-        * `outputs` - List of layer names to be set as model outputs. In case of setting one layer as output, string with one layer can be provided.
-    * Return value:
-        None
+        * `outputs` - List of layer to be set as model outputs. The list can contain strings with layer names to be set 
+                      as outputs or tuples with layer name as first element and output port id as second element.
+                      In case of setting one layer as output, string or tuple with one layer can be provided.
+    * Return value: None
     * Usage example:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -186,14 +393,12 @@ section in Inference Engine Developers Guide documentation.
 > by default. In the case above, `prob` layer is a default output and `conv5_1/dwise`, `conv2_1/expand` are user-defined
 > outputs.
 
-* `reshape(input_shapes: dict)`:
-    * Description:
-        The method reshapes the network to change spatial dimensions, batch size, or any dimension.
-> **Note:** Before using this method, make sure that the target shape is applicable for the network. Changing the network shape to an arbitrary value may lead to unpredictable behaviour.
+* `reshape(input_shapes: dict)`
+    * Description: Reshapes the network to change spatial dimensions, batch size, or any dimension.
+> **NOTE:** Before using this method, make sure that the target shape is applicable for the network. Changing the network shape to an arbitrary value may lead to unpredictable behaviour.
     * Parameters:
-        * `input_shapes` - The dictionary that maps input layer names to tuples with the target shape
-    * Return value:  
-        None           
+        * `input_shapes` - A dictionary that maps input layer names to tuples with the target shape
+    * Return value: None           
     * Usage example:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -201,12 +406,12 @@ section in Inference Engine Developers Guide documentation.
 >>> n, c, h, w = net.inputs[input_layer]
 >>> net.reshape({input_layer: (n, c, h*2, w*2)}]
 ```
-* `serialize(path_to_xml, path_to_bin)`:    
+* `serialize(path_to_xml, path_to_bin)`   
     * Description:         
-        The method serializes the network and stores it in files.        
+        Serializes the network and stores it in files.        
     * Parameters:  
-        * `path_to_xml` - path to a file, where a serialized model will be stored.
-        * `path_to_bin` - path to a file, where serialized weights will be stored.
+        * `path_to_xml` - Path to a file, where a serialized model will be stored
+        * `path_to_bin` - Path to a file, where serialized weights will be stored
     * Return value:
         None     
     * Usage example:
@@ -219,18 +424,18 @@ section in Inference Engine Developers Guide documentation.
 
 Layer calibration statistic container.
 
-### Class Constructor
+### <a name="layerstats-constructor"></a>Class Constructor
 
 * `__init__(min: tuple = (), max: tuple = ())`
     * Parameters:
-        * min - Tuple with per-channel minimum layer activation values
-        * max - Tuple with per-channel maximum layer activation values
+        * `min` - Tuple with per-channel minimum layer activation values
+        * `max` - Tuple with per-channel maximum layer activation values
 
 ## <a name="inputinfo-class"></a>InputInfo
 
 This class contains the information about the network input layers
 
-### Class attributes:
+### <a name="inputinfo-attributes"></a>Class Attributes
 
 * `precision` - Precision of the input data provided by user. Provides setter and getter interfaces
                 to get and modify input layer precision.
@@ -245,7 +450,7 @@ This class contains the information about the network input layers
 
 This class contains the information about the network input layers
 
-### Class attributes:
+### <a name="outputinfo-attributes"></a>Class Attributes
 
 * `precision` - Precision of the output data. Provides setter and getter interfaces  
                 to get and modify output layer precision.          
@@ -256,23 +461,22 @@ This class contains the information about the network input layers
 
 This class is the main plugin interface and serves to initialize and configure the plugin.
 
-### Class Constructor
+### <a name="ieplugin-constructor"></a>Class Constructor
 
 * `__init__(device: str, plugin_dirs=None)`
     * Parameters:
         * `device` - Target device name. Supported devices: CPU, GPU, FPGA, MYRIAD, HETERO
         * `plugin_dirs` - List of paths to plugin directories
 
-### Properties
+### <a name="ieplugin-properties"></a>Properties
 
 * `device` - a name of the device that was specified to initialize IEPlugin
 * `version` -  a version of the plugin
 
-### Instance Methods
+### <a name="ieplugin-instance-methods"></a>Instance Methods
 
-*  ```load(network: IENetwork, num_requests: int=1, config=None)```
-    * Description:   
-        Loads a network that was read from the IR to the plugin and creates an executable network from a network object.
+*  `load(network: IENetwork, num_requests: int=1, config=None)`
+    * Description: Loads a network that was read from the IR to the plugin and creates an executable network from a network object.
         You can create as many networks as you need and use them simultaneously (up to the limitation of the hardware
         resources).
     * Parameters:
@@ -280,8 +484,7 @@ This class is the main plugin interface and serves to initialize and configure t
         * `num_requests` - A positive integer value of infer requests to be created. Number of infer requests may be limited
         by device capabilities.        
         * `config` - A dictionary of plugin configuration keys and their values
-    * Return value:  
-        None
+    * Return value: None
     * Usage example:
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -290,60 +493,54 @@ This class is the main plugin interface and serves to initialize and configure t
 >>> exec_net
 <inference_engine.ie_api.ExecutableNetwork object at 0x7f5140bbcd38>
 ```
+
 * `set_initial_affinity(net: IENetwork)`
-    * Description:
-        Sets initial affinity for model layers according to the HETERO plugin logic. Applicable only if
-        IEPlugin was initialized for HETERO device.
+    * Description: Sets initial affinity for model layers according to the HETERO plugin logic. Applicable only if
+      `IEPlugin` was initialized for a HETERO device.
     * Parameters:
         * `net` - A valid instance of IENetwork
-    * Return value:  
-        None   
+    * Return value: None   
     * Usage example:
 	See `affinity` attribute of the `IENetLayer` class.
 * `add_cpu_extension(extension_path: str)`
-    * Description:
-        Loads extensions library to the plugin. Applicable only for CPU device and HETERO device with CPU  
+    * Description: Loads extensions library to the plugin. Applicable only for a CPU device and a HETERO device with CPU  
     * Parameters:
         * `extension_path` - A full path to CPU extensions library    
-     * Return value:
-        None
+     * Return value: None
     * Usage example:
 ```py
 >>> plugin = IEPlugin(device="CPU")
 >>> plugin.add_cpu_extenstions(ext_lib_path)
 ```    
+
 * `set_config(config: dict)`
-    * Description:
-        Sets a configuration for the plugin. Refer to `SetConfig()` in Inference Engine C++ documentation for acceptable
+    * Description: Sets a configuration for the plugin. Refer to `SetConfig()` in Inference Engine C++ documentation for acceptable
         keys and values list.
     * Parameters:
         * `config` - A dictionary of keys and values of acceptable configuration parameters
-    * Return value:
-        None
+    * Return value: None
     * Usage examples:
 	See `set_affinity` method of the `IENetwork` class.
+
 * `get_supported_layers(net: IENetwork)`
-    * Description:
-        Returns the set of layers supported by the plugin. Please note that in case of CPU plugin support of
-        a layer may depends on extension loaded by `add_cpu_extenstion()` method
+    * Description: Returns the set of layers supported by the plugin. Please note that for the CPU plugin, support of
+        a layer may depends on extension loaded by `add_cpu_extenstion()` method.
     * Parameters:
         * `net` - A valid instance of IENetwork
-    * Return value:
-        Set of layers supported by the plugin
-    * Usage example:
-	See `affinity` attribute of the `IENetLayer` class.
+    * Return value: Set of layers supported by the plugin
+    * Usage example: See `affinity` attribute of the `IENetLayer` class.
 
 ## <a name="executablenetwork"></a>ExecutableNetwork Class
 
 This class represents a network instance loaded to plugin and ready for inference.
 
-### Class Constructor
+### <a name="executablenetwork-contructor"></a>Class Constructor
 
 There is no explicit class constructor. To make a valid instance of `ExecutableNetwork`, use `load()` method of the `IEPlugin` class.
 
-### Class attributes
+### <a name="executablenetwork-attributes"></a>Class Attributes
 
-* `requests` - A tuple of InferRequest instances
+* `requests` - A tuple of `InferRequest` instances
     * Usage example:    
 ```py
 >>> net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
@@ -355,7 +552,7 @@ There is no explicit class constructor. To make a valid instance of `ExecutableN
 <inference_engine.ie_api.InferRequest object at 0x7f66f56c5900>)
 ```
 
-### Instance Methods
+### <a name="executablenetwork-methods"></a>Instance Methods
 
 * `infer(inputs=None)`
     * Description:
@@ -379,16 +576,15 @@ There is no explicit class constructor. To make a valid instance of `ExecutableN
                  ......
               ]])}
 ```
-	For illustration of input data preparation, please see samples (for example, `classification_sample.py`).
+	For illustration of input data preparation, please see the samples (for example, `classification_sample.py`).
+
 * `start_async(request_id, inputs=None)`
-    * Description:
-        Starts asynchronous inference for specified infer request.
-        Wraps `async_infer()` method of the `InferRequest` class
+    * Description: Starts asynchronous inference for specified infer request.
+        Wraps `async_infer()` method of the `InferRequest` class.
     * Parameters:
         * `request_id` - Index of infer request to start inference
         * `inputs` - A dictionary that maps input layer names to `numpy.ndarray` objects of proper shape with input data for the layer
-    * Return value:
-        A handler of specified infer request, which is an instance of the `InferRequest` class.
+    * Return value: A handler of specified infer request, which is an instance of the `InferRequest` class.
     * Usage example:
 ```py
 >>> infer_request_handle = exec_net.start_async(request_id=0, inputs={input_blob: image})
@@ -399,21 +595,66 @@ There is no explicit class constructor. To make a valid instance of `ExecutableN
 For more details about infer requests processing, see `classification_sample_async.py` (simplified case) and
 `object_detection_demo_ssd_async.py` (real asynchronous use case) samples.
 
+* `get_exec_graph_info()`
+    * Description: Gets executable graph information from a device
+    * Parameters: None
+    * Return value: An instance of <a href="#ienetwork-class">`IENetwork`</a>
+    * Usage_example:
+
+```py
+net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+plugin = IEPlugin(device="CPU")
+exec_net = plugin.load(network=net, num_requsts=2)
+exec_graph = exec_net.get_exec_graph_info()
+``` 
+
+* `get_metric(metric_name: str)`
+    * Description: - Gets general runtime metric for an executable network. It can be network name, actual device ID on
+      which executable network is running or all other properties which cannot be changed dynamically.
+
+    * Parameters:
+        * metric_name - A metric name to request.
+    * Return value: A metric value corresponding to a metric key.
+    * Usage example
+```py
+ie = IECore()
+net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+exec_net = ie.load_network(net, "CPU")
+exec_net.get_metric("NETWORK_NAME")
+```
+
+* `get_config(metric_config: str)`
+    * Description: - Gets configuration for current executable network. The method is responsible to extract information
+      * which affects executable network execution
+
+    * Parameters:
+        * config_name - A configuration parameter name to request.
+    * Return value: A configuration value corresponding to a configuration key.
+    * Usage example
+```py
+ie = IECore()
+net = IENetwork(model=path_to_xml_file, weights=path_to_bin_file)
+exec_net = ie.load_network(net, "CPU")
+exec_net.get_metric("DEVICE_ID")
+```
+
+
 ## <a name="inferrequest"></a>InferRequest Class
 
 This class provides an interface to infer requests of `ExecutableNetwork` and serves to handle infer requests execution
 and to set and get output data.   
 
-### Class Constructor
+### <a name="inferrequest-constructor"></a>Class Constructor
 
 There is no explicit class constructor. To make a valid `InferRequest` instance, use `load()` method of the `IEPlugin`
 class with specified number of requests to get `ExecutableNetwork` instance which stores infer requests.
 
-### Class attributes
+### <a name="inferrequest-attributes"></a>Class Attributes
 
 * `inputs` - A dictionary that maps input layer names to `numpy.ndarray` objects of proper shape with input data for the layer
 * `outputs` - A dictionary that maps output layer names to `numpy.ndarray` objects with output data of the layer
-    * Usage example:
+
+	Usage example:
 ```py    
 >>> exec_net.requests[0].inputs['data'][:] = image
 >>> exec_net.requests[0].infer()
@@ -424,18 +665,16 @@ array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
        2.26027006e-03, 2.12283316e-03 ...])
 ```
 
-### Instance Methods
+### <a name="inferrequest-methods"></a>Instance Methods
 
 It is not recommended to run inference directly on `InferRequest` instance.
 To run inference, please use simplified methods `infer()` and `start_async()` of `ExecutableNetwork`.
 
 * `infer(inputs=None)`
-    * Description:
-         Starts synchronous inference of the infer request and fill outputs array		 
+    * Description: Starts synchronous inference of the infer request and fill outputs array		 
      * Parameters:	 
         * `inputs` - A dictionary that maps input layer names to `numpy.ndarray` objects of proper shape with input data for the layer        
-    * Return value:        
-        None        
+    * Return value: None        
     * Usage example:    
 ```py
 >>> exec_net = plugin.load(network=net, num_requests=2)
@@ -445,14 +684,13 @@ To run inference, please use simplified methods `infer()` and `start_async()` of
 array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
        5.45198545e-02, 2.44456064e-02, 5.41366823e-03, 3.42589128e-03,
        2.26027006e-03, 2.12283316e-03 ...])
-```                          
+```    
+                      
 * `async_infer(inputs=None)`
-    * Description:     
-        Starts asynchronous inference of the infer request and fill outputs array		
+    * Description: Starts asynchronous inference of the infer request and fill outputs array		
      * Parameters:	 
         * `inputs` - A dictionary that maps input layer names to `numpy.ndarray` objects of proper shape with input data for the layer        
-    * Return value:        
-        None        
+    * Return value: None        
     * Usage example:    
 ```py
 >>> exec_net = plugin.load(network=net, num_requests=2)
@@ -463,10 +701,10 @@ array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
 array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
        5.45198545e-02, 2.44456064e-02, 5.41366823e-03, 3.42589128e-03,
        2.26027006e-03, 2.12283316e-03 ...])
-```			
+```
+
 * `wait(timeout=-1)`
-    * Description:        
-        Waits for the result to become available. Blocks until specified timeout elapses or the result
+    * Description:  Waits for the result to become available. Blocks until specified timeout elapses or the result
         becomes available, whichever comes first.      
 > **NOTE:** There are special values of the timeout parameter:
         * 0 - Immediately returns the inference status. It does not block or interrupt execution.
@@ -475,14 +713,12 @@ array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
     * Parameters:
         * `timeout` - Time to wait in milliseconds or special (0, -1) cases described above.
           If not specified, `timeout` value is set to -1 by default.
-    * Usage example:
-	See `async_infer()` method of the the `InferRequest` class.
+    * Usage example: See `async_infer()` method of the the `InferRequest` class.
+
 * `get_perf_counts()`
-    * Description:
-        Queries performance measures per layer to get feedback of what is the most time consuming layer.
+    * Description: Queries performance measures per layer to get feedback of what is the most time consuming layer.
 > **NOTE**: Performance counters data and format depends on the plugin
-    * Parameters:
-        None
+    * Parameters: None
     * Usage example:
 ```py
 >>> exec_net = plugin.load(network=net, num_requests=2)
@@ -501,16 +737,40 @@ array([4.85416055e-01, 1.70385033e-01, 1.21873841e-01, 1.18894853e-01,
 ...
 }
 ```
+
 * `set_batch(size)`
     * Description:   
        Sets new batch size for certain infer request when dynamic batching is enabled in executable network that created this request.
 > **NOTE:** Support of dynamic batch size depends on the target plugin.          
     * Parameters:
-        * `batch` - new batch size to be used by all the following inference calls for this request.
+        * `batch` - New batch size to be used by all the following inference calls for this request
     * Usage example:
 ```py
 >>> plugin.set_config({"DYN_BATCH_ENABLED": "YES"})
 >>> exec_net = plugin.load(network=net)
 >>> exec_net.requests[0].set_batch(inputs_count)
 ```
-Please refer to `dynamic_batch_demo.py` to see the full usage example.
+
+* `set_completion_callback(py_callback, py_data = None)`
+    * Description: Sets a callback function that is called on success or failure of an asynchronous request
+    * Parameters:
+        * `py_callback` - Any defined or lambda function
+        * `py_data` - Data that is passed to the callback function
+    * Return value: None
+    * Usage example:
+
+```py
+
+callback = lambda status, py_data: print("Request with id {} finished with status {}".format(py_data, status))
+
+net = IENetwork("./model.xml", "./model.bin")
+ie = IECore()
+exec_net = ie.load_network(net, "CPU", num_requests=4)
+
+for id, req in enumerate(exec_net.requests):
+    req.set_completion_callback(py_callback=callback, py_data=id)
+
+for req in exec_net.requests:
+    req.async_infer({"data": img})
+    
+```  

inference-engine/ie_bridges/python/requirements.txt

@@ -1,3 +1,4 @@
 opencv-python
 numpy
-cython
+cython
+progress

inference-engine/ie_bridges/python/sample/benchmark_app/README.md

@@ -1,19 +1,22 @@
-# Benchmark Application Python* Demo
+# Benchmark Python* Application
 
 This topic demonstrates how to run the Benchmark Application demo, which performs inference using convolutional networks.
 
 ## How It Works
 
-> **NOTE:** To achieve benchmark results similar to the official published results, set CPU frequency to 2.9GHz and GPU frequency to 1GHz.
+Upon start-up, the application reads command-line parameters and loads a network and images/binary files to the Inference Engine
+plugin, which is chosen depending on a specified device. The number of infer requests and execution approach depend
+on the mode defined with the `-api` command-line parameter.
 
-Upon the start-up, the application reads command-line parameters and loads a network and images to the Inference Engine plugin. The number of infer requests and execution approach depend on a mode defined with the `-api` command-line parameter.
-
-> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Specify Input Shapes** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
+> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
 
 ### Synchronous API
+
 For synchronous mode, the primary metric is latency. The application creates one infer request and executes the `Infer` method. A number of executions is defined by one of the two values:
 * Number of iterations defined with the `-niter` command-line argument
-* Predefined duration if `-niter` is skipped. Predefined duration value depends on device.
+* Time duration specified with the `-t` command-line argument
+* Both of them (execution will continue until both conditions are met)
+* Predefined duration if `-niter` and `-t` are not specified. Predefined duration value depends on device.
 
 During the execution, the application collects two types of metrics:
 * Latency for each infer request executed with `Infer` method
@@ -24,72 +27,103 @@ Reported latency value is calculated as mean value of all collected latencies. R
 ### Asynchronous API
 For asynchronous mode, the primary metric is throughput in frames per second (FPS). The application creates a certain number of infer requests and executes the `StartAsync` method. A number of infer is specified with the `-nireq` command-line parameter. A number of executions is defined by one of the two values:
 * Number of iterations defined with the `-niter` command-line argument
-* Predefined duration if `-niter` is skipped. Predefined duration value depends on device.
+* Time duration specified with the `-t` command-line argument
+* Both of them (execution will continue until both conditions are met)
+* Predefined duration if `-niter` and `-t` are not specified. Predefined duration value depends on device.
 
-The infer requests are executed asynchronously. `Wait` method is used to wait for previous execution to complete. The application measures all infer requests executions and reports the throughput metric based on batch size and total execution duration.
+The infer requests are executed asynchronously. Callback is used to wait for previous execution to complete. The application measures all infer requests executions and reports the throughput metric based on batch size and total execution duration.
 
 ## Running
+Notice that the benchmark_app usually produces optimal performance for any device out of the box.
+
+**So in most cases you don't need to play the app options explicitly and the plain device name is enough**, e.g.:
+```
+$benchmark_app -m <model> -i <input> -d CPU
+```
 
 Running the application with the `-h` or `--help`' option yields the following usage message:
-```python3 benchmark_app.py -h```
 
-The command yields the following usage message:
 ```
-   usage: benchmark_app.py [-h] -i PATH_TO_IMAGES -m PATH_TO_MODEL
-                        [-c PATH_TO_CLDNN_CONFIG] [-l PATH_TO_EXTENSION]
-                        [-api {sync,async}] [-d TARGET_DEVICE]
-                        [-niter NUMBER_ITERATIONS]
-                        [-nireq NUMBER_INFER_REQUESTS]
-                        [-nthreads NUMBER_THREADS] [-b BATCH_SIZE]
-                        [-pin {YES,NO}]
+usage: benchmark_app.py [-h] [-i PATH_TO_INPUT] -m PATH_TO_MODEL
+                        [-pp PLUGIN_DIR] [-d TARGET_DEVICE]
+                        [-l PATH_TO_EXTENSION] [-c PATH_TO_CLDNN_CONFIG]
+                        [-api {sync,async}] [-niter NUMBER_ITERATIONS]
+                        [-nireq NUMBER_INFER_REQUESTS] [-b BATCH_SIZE]
+                        [-stream_output [STREAM_OUTPUT]] [-t TIME]
+                        [-progress [PROGRESS]] [-nstreams NUMBER_STREAMS]
+                        [-nthreads NUMBER_THREADS] [-pin {YES,NO}]
+                        [--exec_graph_path EXEC_GRAPH_PATH]
+                        [-pc [PERF_COUNTS]]
 
 Options:
   -h, --help            Show this help message and exit.
-  -i PATH_TO_IMAGES, --path_to_images PATH_TO_IMAGES
-                        Required. Path to a folder with images or to image
-                        files.
+  -i PATH_TO_INPUT, --path_to_input PATH_TO_INPUT
+                        Optional. Path to a folder with images and/or binaries
+                        or to specific image or binary file.
   -m PATH_TO_MODEL, --path_to_model PATH_TO_MODEL
                         Required. Path to an .xml file with a trained model.
-  -c PATH_TO_CLDNN_CONFIG, --path_to_cldnn_config PATH_TO_CLDNN_CONFIG
-                        Optional. Required for GPU custom kernels. Absolute
-                        path to an .xml file with the kernels description.
+  -pp PLUGIN_DIR, --plugin_dir PLUGIN_DIR
+                        Optional. Path to a plugin folder.
+  -d TARGET_DEVICE, --target_device TARGET_DEVICE
+                        Optional. Specify a target device to infer on: CPU,
+                        GPU, FPGA, HDDL or MYRIAD.
+                        Use "-d HETERO:<comma separated devices list>" format to specify HETERO plugin.
   -l PATH_TO_EXTENSION, --path_to_extension PATH_TO_EXTENSION
+                        Optional. Required for CPU custom layers. Absolute
+                        path to a shared library with the kernels
+                        implementations.
+  -c PATH_TO_CLDNN_CONFIG, --path_to_cldnn_config PATH_TO_CLDNN_CONFIG
                         Optional. Required for GPU custom kernels. Absolute
                         path to an .xml file with the kernels description.
   -api {sync,async}, --api_type {sync,async}
                         Optional. Enable using sync/async API. Default value
-                        is sync
-  -d TARGET_DEVICE, --target_device TARGET_DEVICE
-                        Optional. Specify a target device to infer on: CPU,
-                        GPU, FPGA, HDDL or MYRIAD. Use "-d HETERO:<comma
-                        separated devices list>" format to specify HETERO
-                        plugin. The application looks for a suitable plugin
-                        for the specified device.
+                        is async.
   -niter NUMBER_ITERATIONS, --number_iterations NUMBER_ITERATIONS
                         Optional. Number of iterations. If not specified, the
                         number of iterations is calculated depending on a
                         device.
   -nireq NUMBER_INFER_REQUESTS, --number_infer_requests NUMBER_INFER_REQUESTS
-                        Optional. Number of infer requests (default value is
-                        2).
-  -nthreads NUMBER_THREADS, --number_threads NUMBER_THREADS
-                        Number of threads to use for inference on the CPU
-                        (including Hetero cases).
+                        Optional. Number of infer requests. Default value is
+                        determined automatically for device.
   -b BATCH_SIZE, --batch_size BATCH_SIZE
                         Optional. Batch size value. If not specified, the
                         batch size value is determined from IR
+  -stream_output [STREAM_OUTPUT]
+                        Optional. Print progress as a plain text. When
+                        specified, an interactive progress bar is replaced
+                        with a multiline output.
+  -t TIME, --time TIME  Optional. Time in seconds to execute topology.
+  -progress [PROGRESS]  Optional. Show progress bar (can affect performance
+                        measurement). Default values is "False".
+  -nstreams NUMBER_STREAMS, --number_streams NUMBER_STREAMS
+                       Optional. Number of streams to use for inference on the CPU/GPU in throughput mode
+                       (for HETERO device case use format <device1>:<nstreams1>,<device2>:<nstreams2> or just <nstreams>).
+  -nthreads NUMBER_THREADS, --number_threads NUMBER_THREADS
+                        Number of threads to use for inference on the CPU
+                        (including HETERO case).
   -pin {YES,NO}, --infer_threads_pinning {YES,NO}
                         Optional. Enable ("YES" is default value) or disable
                         ("NO")CPU threads pinning for CPU-involved inference.
+  --exec_graph_path EXEC_GRAPH_PATH
+                        Optional. Path to a file where to store executable
+                        graph information serialized.
+  -pc [PERF_COUNTS], --perf_counts [PERF_COUNTS]
+                        Optional. Report performance counters.
+
 ```
 
 Running the application with the empty list of options yields the usage message given above and an error message.
 
+Application supports topologies with one or more inputs. If a topology is not data sensitive, you can skip the input parameter. In this case, inputs are filled with random values.
+If a model has only image input(s), please a provide folder with images or a path to an image as input.
+If a model has some specific input(s) (not images), please prepare a binary file(s), which is filled with data of appropriate precision and provide a path to them as input.
+If a model has mixed input types, input folder should contain all required files. Image inputs are filled with image files one by one. Binary inputs are filled with binary inputs one by one.
+
 To run the demo, you can use public or pre-trained models. To download the pre-trained models, use the OpenVINO [Model Downloader](https://github.com/opencv/open_model_zoo/tree/2018/model_downloader) or go to [https://download.01.org/opencv/](https://download.01.org/opencv/).
 
 > **NOTE**: Before running the demo with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](./docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
 
-For example, to do inference on an image using a trained network with multiple outputs on CPU, run the following command:
+For example, to do inference of an image using a trained network with multiple outputs on CPU, run the following command:
 
 ```
 python3 benchmark_app.py -i <path_to_image>/inputImage.bmp -m <path_to_model>/multiple-output.xml -d CPU
@@ -97,17 +131,22 @@ python3 benchmark_app.py -i <path_to_image>/inputImage.bmp -m <path_to_model>/mu
 
 ## Demo Output
 
-Application output depends on a used API. For synchronous API, the application outputs latency and throughput:
-```
-[ INFO ] Start inference synchronously (10 s duration)
-[BENCHMARK RESULT] Latency is 15.5520 msec
-[BENCHMARK RESULT] Throughput is 1286.0082 FPS
-```
+The application outputs number of executed iterations, total duration of execution, latency and throughput.
+Additionally, if you set the `-pc` parameter, the application outputs performance counters.
+If you set `-exec_graph_path`, the application reports executable graph information serialized.
 
-For asynchronous API, the application outputs only throughput:
 ```
-[ INFO ] Start inference asynchronously (10 s duration, 8 inference requests in parallel)
-[BENCHMARK RESULT] Throughput is 1444.2591 FPS
+[Step 8/9] Measuring performance (Start inference asyncronously, 60000 ms duration, 4 inference requests in parallel using 4 streams)
+Progress: |................................| 100.00%
+
+[Step 9/9] Dumping statistics report
+Progress: |................................| 100.00%
+
+Count:      4408 iterations
+Duration:   60153.52 ms
+Latency:    51.8244 ms
+Throughput: 73.28 FPS
+
 ```
 
 ## See Also

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/__init__.py

@@ -15,4 +15,3 @@
 """
 
 from .benchmark import main
-from .utils.constants import HELP_MESSAGES

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/benchmark.py

@@ -15,48 +15,79 @@
 """
 
 from statistics import median
-from openvino.inference_engine import IENetwork, IEPlugin
+from openvino.inference_engine import IENetwork, IECore, get_version
 
-from .utils.benchmark_utils import *
+from .utils.parameters import *
+from .utils.inputs_filling import *
+from .utils.utils import *
+from .utils.infer_request_wrap import *
+from .utils.progress_bar import *
+
+def getDurationInMilliseconds(duration):
+    return duration * 1000
+
+def static_vars(**kwargs):
+    def decorate(func):
+        for k in kwargs:
+            setattr(func, k, kwargs[k])
+        return func
+    return decorate
+
+@static_vars(step_id = 0)
+def next_step(additional_info = ""):
+    step_names = {
+        1  : "Parsing and validating input arguments",
+        2  : "Loading Inference Engine",
+        3  : "Read the Intermediate Representation of the network",
+        4  : "Resizing network to match image sizes and given batch",
+        5  : "Configuring input of the model",
+        6  : "Setting device configuration",
+        7  : "Loading the model to the device",
+        8  : "Setting optimal runtime parameters",
+        9  : "Creating infer requests and filling input blobs with images",
+        10 : "Measuring performance",
+        11 : "Dumping statistics report",
+    }
+
+    next_step.step_id += 1
+    if (next_step.step_id not in step_names.keys()):
+        raise Exception("Step ID " + str(next_step.step_id) + " is out of total steps number " + len(step_names))
+
+    print("[Step {}/{}] {}".format(next_step.step_id, len(step_names), step_names[next_step.step_id]) + (" (" + additional_info + ")" if len(additional_info) else ""))
 
 def main(args=None):
     try:
-        if args is None:
+        # ------------------------------ 1. Parsing and validating input arguments -------------------------------------
+        next_step()
+
+        if not args:
             args = parse_args()
 
-        validate_args(args)
+        # ------------------------------ 2. Loading Inference Engine ---------------------------------------------------
+        next_step()
 
-        # --------------------------------- 1. Load Plugin for inference engine ---------------------------------
-        logging.info("Loading plugin")
-        plugin = IEPlugin(args.target_device)
+        device_name = args.target_device.upper()
 
-        config = dict()
-        if CPU_DEVICE_NAME in args.target_device:
+        ie = IECore()
+
+        if CPU_DEVICE_NAME in device_name:
             if args.path_to_extension:
-                plugin.add_cpu_extension(args.path_to_extension)
-            # limit threading for CPU portion of inference
-            if args.number_threads is not None:
-                config.update({'CPU_THREADS_NUM': str(args.number_threads)})
-            # pin threads for CPU portion of inference
-            config.update({'CPU_BIND_THREAD': args.infer_threads_pinning})
-            # for pure CPU execution, more throughput-oriented execution via streams
-            if args.api_type == 'async' and CPU_DEVICE_NAME in args.target_device:
-                config.update({'CPU_THROUGHPUT_STREAMS': str(args.number_infer_requests)})
-        elif GPU_DEVICE_NAME in args.target_device:
+                ie.add_extension(extension_path=args.path_to_extension, device_name=CPU_DEVICE_NAME)
+        if GPU_DEVICE_NAME in device_name:
             if args.path_to_cldnn_config:
-                config.update({'CONFIG_FILE': args.path_to_cldnn_config})
+                ie.set_config({'CONFIG_FILE' : args.path_to_cldnn_config}, GPU_DEVICE_NAME)
                 logger.info("GPU extensions is loaded {}".format(args.path_to_cldnn_config))
-        elif MYRIAD_DEVICE_NAME in args.target_device:
-            config.update({'LOG_LEVEL': 'LOG_INFO'})
-            config.update({'VPU_LOG_LEVEL': 'LOG_INFO'})
 
-        plugin.set_config(config)
+        logger.info("InferenceEngine:\n{: <9}{}".format("",get_version()))
+        version_string = "Device is {}\n".format(device_name)
+        for device, version in ie.get_versions(device_name).items():
+          version_string += "{: <9}{}\n".format("", device)
+          version_string += "{: <9}{:.<24}{} {}.{}\n".format("",version.description," version", version.major, version.minor)
+          version_string += "{: <9}{:.<24} {}\n".format("","Build", version.build_number)
+        logger.info(version_string)
 
-        logger.info("Device is {}".format(plugin.device))
-        logger.info("Plugin version is {}".format(plugin.version))
-
-        # --------------------- 2. Read IR Generated by ModelOptimizer (.xml and .bin files) ---------------------
-        logger.info("Loading network files")
+        # --------------------- 3. Read the Intermediate Representation of the network ---------------------------------
+        next_step()
 
         xml_filename = os.path.abspath(args.path_to_model)
         head, tail = os.path.splitext(xml_filename)
@@ -68,132 +99,245 @@ def main(args=None):
 
         if len(input_info) == 0:
             raise AttributeError('No inputs info is provided')
-        elif len(input_info) != 1:
-            raise AttributeError("only one input layer network is supported")
 
-        # -------------------------------------- 3. Change network batch_size  -------------------------------------
+        # --------------------- 4. Resizing network to match image sizes and given batch -------------------------------
+        next_step()
+
         batch_size = ie_network.batch_size
-        key = list(input_info.keys()).pop()
-        precision = input_info[key].precision
+        precision = ie_network.precision
 
         if args.batch_size and args.batch_size != ie_network.batch_size:
-            # deepcopy input_info
-            shape = input_info[key].shape
-            # We support models having only one input layers
-            if input_info[key].layout != LAYOUT_TYPE:
-                raise Exception('Unsupported model for batch size changing in automatic mode')
-            shape[BATCH_SIZE_ELEM] = args.batch_size
-            ie_network.reshape({key: shape})
+            new_shapes = {}
+            for key in input_info.keys():
+                shape = input_info[key].shape
+                layout = input_info[key].layout
 
-            input_info = ie_network.inputs
+                batchIndex = -1
+                if ((layout == 'NCHW') or (layout == 'NCDHW') or
+                    (layout == 'NHWC') or (layout == 'NDHWC') or
+                    (layout == 'NC')):
+                    batchIndex = 0
+                elif (layout == 'CN'):
+                    batchIndex = 1
+
+                if ((batchIndex != -1) and (shape[batchIndex] != args.batch_size)):
+                    shape[batchIndex] = args.batch_size
+                    new_shapes[key] = shape
+
+            if (len(new_shapes) > 0):
+                logger.info("Resizing network to batch = {}".format(args.batch_size))
+                ie_network.reshape(new_shapes)
 
             batch_size = args.batch_size
 
+        logger.info("Network batch size: {}, precision {}".format(batch_size, precision))
 
-        logger_message = "Network batch size was changed to: " if args.batch_size is not None else "Network batch size: "
-        logger_message += " {}, precision: {}".format(batch_size, precision)
-        logger.info(logger_message)
+        # --------------------- 5. Configuring input of the model ------------------------------------------------------
+        next_step()
 
-        # ------------------------------------- 4. Loading model to the plugin -------------------------------------
-        logger.info("Loading model to the plugin")
-        exe_network = plugin.load(ie_network, args.number_infer_requests)
+        for key in input_info.keys():
+            if (isImage(input_info[key])):
+                # Set the precision of input data provided by the user
+                # Should be called before load of the network to the plugin
+                input_info[key].precision = 'U8'
 
-        # ------------------------------------ 5. Performance measurements stuff -----------------------------------
-        inputs = get_images(os.path.abspath(args.path_to_images), batch_size)
+        # --------------------- 6. Setting device configuration --------------------------------------------------------
+        next_step()
 
-        if batch_size < len(inputs):
-            logger.warn("Network batch size {} is less then images count  {}"
-                        ", some input files will be ignored".format(batch_size, len(inputs)))
+        devices = parseDevices(device_name)
+        device_nstreams = parseValuePerDevice(devices, args.number_streams)
+        for device in devices:
+          if device == CPU_DEVICE_NAME: ## CPU supports few special performance-oriented keys
+            ## limit threading for CPU portion of inference
+            if args.number_threads:
+              ie.set_config({'CPU_THREADS_NUM': str(args.number_threads)}, device)
 
-        input_images = {key: fill_blob_with_image(inputs, input_info[key].shape)}
+            # pin threads for CPU portion of inference
+            ie.set_config({'CPU_BIND_THREAD': args.infer_threads_pinning}, device)
 
-        times = list()
-        duration = 0
+            ## for CPU execution, more throughput-oriented execution via streams
+            # for pure CPU execution, more throughput-oriented execution via streams
+            if args.api_type == 'async':
+                ie.set_config({'CPU_THROUGHPUT_STREAMS': str(device_nstreams.get(device))
+                                                         if device in device_nstreams.keys()
+                                                         else 'CPU_THROUGHPUT_AUTO' }, device)
+            device_nstreams[device] = int(ie.get_config(device, 'CPU_THROUGHPUT_STREAMS'))
 
-        if args.number_iterations is None:
-            duration = get_duration_in_secs(args.target_device)
+          elif device == GPU_DEVICE_NAME:
+            if args.api_type == 'async':
+                ie.set_config({'GPU_THROUGHPUT_STREAMS' : str(device_nstreams.get(device))
+                                                          if device in device_nstreams.keys()
+                                                          else 'GPU_THROUGHPUT_AUTO'}, device)
+            device_nstreams[device] = int(ie.get_config(device, 'GPU_THROUGHPUT_STREAMS'))
 
-        if args.api_type == 'sync':
+          elif device == MYRIAD_DEVICE_NAME:
+            ie.set_config({'LOG_LEVEL': 'LOG_INFO',
+                           'VPU_LOG_LEVEL': 'LOG_WARNING'}, MYRIAD_DEVICE_NAME)
 
-            # warming up - out of scope
-            exe_network.infer(input_images)
+        # --------------------- 7. Loading the model to the device -----------------------------------------------------
+        next_step()
 
-            if args.number_iterations is not None:
-                logger.info(
-                    "Start inference synchronously ({}) sync inference executions".format(args.number_iterations))
-                for iteration in range(args.number_iterations):
-                    sync_infer_request(exe_network, times, input_images)
+        config = { 'PERF_COUNT' : ('YES' if args.perf_counts else 'NO')}
 
-            else:
-                logger.info("Start inference synchronously ({} s duration)".format(duration))
-                start_time = datetime.now()
-                current_time = start_time
-                while (current_time - start_time).total_seconds() < duration:
-                    current_time = sync_infer_request(exe_network, times, input_images)
+        exe_network = ie.load_network(ie_network,
+                                      device_name,
+                                      config=config,
+                                      num_requests=args.number_infer_requests if args.number_infer_requests else 0)
 
-            times.sort()
-            latency = median(times)
-            fps = batch_size / latency
+        # --------------------- 8. Setting optimal runtime parameters --------------------------------------------------
+        next_step()
 
-            print("[BENCHMARK RESULT] Latency is {:.4f} msec".format(latency * 1e3))
-            print("[BENCHMARK RESULT] Throughput is {:.4f} FPS".format(fps))
+        ## Number of requests
+        infer_requests = exe_network.requests
+        nireq = len(infer_requests)
+
+        ## Iteration limit
+        niter = args.number_iterations
+        if niter and args.api_type == 'async':
+          niter = (int)((niter + nireq - 1)/nireq)*nireq
+          if (args.number_iterations != niter):
+            logger.warn("Number of iterations was aligned by request number "
+                        "from {} to {} using number of requests {}".format(args.number_iterations, niter, nireq))
+
+        ## Time limit
+        duration_seconds = 0
+        if args.time:
+          ## time limit
+          duration_seconds = args.time
+        elif not args.number_iterations:
+          ## default time limit
+          duration_seconds = get_duration_in_secs(device)
+
+        # ------------------------------------ 8. Creating infer requests and filling input blobs ----------------------
+        next_step()
+
+        request_queue = InferRequestsQueue(infer_requests)
+
+        path_to_input = os.path.abspath(args.path_to_input) if args.path_to_input else None
+        requests_input_data = getInputs(path_to_input, batch_size, ie_network.inputs, infer_requests)
+
+        # ------------------------------------ 9. Measuring performance ------------------------------------------------
+
+        progress_count = 0
+        progress_bar_total_count = 10000
+
+        output_string = "Start inference {}ronously".format(args.api_type)
+        if (args.api_type == "async"):
+            if output_string != "":
+                output_string += ", "
+
+            output_string += str(nireq) + " inference requests"
+            device_ss = ''
+            for device, nstreams in device_nstreams.items():
+                if device_ss != '':
+                    device_ss += ', '
+                device_ss += "{} streams for {}".format(str(nstreams), device)
+            if device_ss != '':
+                output_string += " using " + device_ss
+
+        output_string += ", limits: "
+        if niter:
+            if not duration_seconds:
+                progress_bar_total_count = niter
+            output_string += str(niter) + " iterations"
+
+        if duration_seconds:
+            if niter:
+                output_string += ", "
+            output_string += str(getDurationInMilliseconds(duration_seconds)) + " ms duration"
+
+        next_step(output_string)
+
+        ## warming up - out of scope
+        infer_request = request_queue.getIdleRequest()
+        if not infer_request:
+            raise Exception("No idle Infer Requests!")
+
+        if (args.api_type == 'sync'):
+            infer_request.infer(requests_input_data[infer_request.id])
         else:
-            infer_requests = exe_network.requests
+            infer_request.startAsync(requests_input_data[infer_request.id])
 
-            if args.number_iterations is not None:
-                logger.info("Start inference asynchronously ({}"
-                            " async inference executions, {} "
-                            " inference requests in parallel".format(args.number_iterations,
-                                                                       args.number_infer_requests))
+        request_queue.waitAll()
+        request_queue.resetTimes()
+
+        start_time = datetime.now()
+        exec_time = (datetime.now() - start_time).total_seconds()
+        iteration = 0
+
+        progress_bar = ProgressBar(progress_bar_total_count, args.stream_output, args.progress)
+
+        ## Start inference & calculate performance
+        ## to align number if iterations to guarantee that last infer requests are executed in the same conditions **/
+        while ((niter and iteration < niter) or
+               (duration_seconds and exec_time < duration_seconds) or
+               (args.api_type == "async" and iteration % nireq != 0)):
+            infer_request = request_queue.getIdleRequest()
+            if not infer_request:
+                raise Exception("No idle Infer Requests!")
+
+            if (args.api_type == 'sync'):
+                infer_request.infer(requests_input_data[infer_request.id])
             else:
-                logger.info("Start inference asynchronously ({} s duration, "
-                            "{} inference requests in parallel)".format(duration, args.number_infer_requests))
+                infer_request.startAsync(requests_input_data[infer_request.id])
+            iteration += 1
 
-            current_inference = 0
-            required_inference_requests_were_executed = False
-            previous_inference = 1 - args.number_infer_requests
-            step = 0
-            steps_count = args.number_infer_requests - 1
-            if args.number_iterations is not None:
-                steps_count += args.number_iterations
+            exec_time = (datetime.now() - start_time).total_seconds()
 
-            # warming up - out of scope
-            infer_requests[0].async_infer(input_images)
-            infer_requests[0].wait()
+            if niter:
+                progress_bar.add_progress(1)
+            else:
+                ## calculate how many progress intervals are covered by current iteration.
+                ## depends on the current iteration time and time of each progress interval.
+                ## Previously covered progress intervals must be skipped.
+                progress_interval_time = duration_seconds / progress_bar_total_count
+                new_progress = (int) (exec_time / progress_interval_time - progress_count)
+                progress_bar.add_progress(new_progress)
+                progress_count += new_progress
 
-            start_time = datetime.now()
-            while not required_inference_requests_were_executed or step < steps_count or \
-                    args.number_iterations is None and (datetime.now() - start_time).total_seconds() < duration:
-                exe_network.start_async(current_inference, input_images)
+        ## wait the latest inference executions
+        request_queue.waitAll()
 
-                if previous_inference >= 0:
-                    status = infer_requests[previous_inference].wait()
-                    if status is not 0:
-                        raise Exception("Infer request not completed successfully")
+        total_duration_sec = request_queue.getDurationInSeconds()
+        times = request_queue.times
+        times.sort()
+        latency_ms = median(times)
+        fps = batch_size * 1000 / latency_ms if args.api_type == 'sync' else batch_size * iteration / total_duration_sec
 
-                current_inference += 1
-                if current_inference >= args.number_infer_requests:
-                    current_inference = 0
-                    required_inference_requests_were_executed = True
+        progress_bar.finish()
 
-                previous_inference += 1
-                if previous_inference >= args.number_infer_requests:
-                    previous_inference = 0
+        # ------------------------------------ 10. Dumping statistics report -------------------------------------------
+        next_step()
 
-                step += 1
+        if args.exec_graph_path:
+            try:
+              exec_graph_info = exe_network.get_exec_graph_info()
+              exec_graph_info.serialize(args.exec_graph_path)
+              logger.info("Executable graph is stored to {}".format(args.exec_graph_path))
+              del exec_graph_info
+            except Exception as e:
+                logging.exception(e)
 
-            # wait the latest inference executions
-            for not_completed_index in range(args.number_infer_requests):
-                if infer_requests[not_completed_index].wait(0) != 0:
-                    infer_requests[not_completed_index].wait()
+        if args.perf_counts:
+            for ni in range(int(nireq)):
+                perf_counts = exe_network.requests[ni].get_perf_counts()
+                logger.info("Pefrormance counts for {}-th infer request".format(ni))
+                for layer, stats in perf_counts.items():
+                    max_layer_name = 30
+                    print("{:<30}{:<15}{:<30}{:<20}{:<20}{:<20}".format(layer[:max_layer_name - 4] + '...' if (len(layer) >= max_layer_name) else layer,
+                                                                        stats['status'],
+                                                                        'layerType: ' + str(stats['layer_type']),
+                                                                        'realTime: ' + str(stats['real_time']),
+                                                                        'cpu: ' + str(stats['cpu_time']),
+                                                                        'execType: ' + str(stats['exec_type'])))
 
-            total_duration = (datetime.now() - start_time).total_seconds()
-            fps = batch_size * step / total_duration
-
-            print("[BENCHMARK RESULT] Throughput is {:.4f} FPS".format(fps))
+        print("Count:      {} iterations".format(iteration))
+        print("Duration:   {:.2f} ms".format(getDurationInMilliseconds(total_duration_sec)))
+        print("Latency:    {:.4f} ms".format(latency_ms))
+        print("Throughput: {:.2f} FPS".format(fps))
 
         del exe_network
-        del plugin
-
+        del ie
+        next_step.step_id = 0
     except Exception as e:
         logging.exception(e)

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/benchmark_utils.py

@@ -1,124 +0,0 @@
-"""
- Copyright (C) 2018-2019 Intel Corporation
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
-      http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-"""
-
-import logging
-import argparse
-import os
-import cv2
-import numpy as np
-import sys
-
-from glob import glob
-from random import choice
-from datetime import datetime
-from fnmatch import fnmatch
-
-from .constants import *
-
-logging.basicConfig(format="[ %(levelname)s ] %(message)s", level=logging.INFO, stream=sys.stdout)
-logger = logging.getLogger('BenchmarkApp')
-
-
-def validate_args(args):
-    if args.number_iterations is not None and args.number_iterations < 0:
-        raise Exception("Number of iterations should be positive (invalid -niter option value)")
-    if args.number_infer_requests < 0:
-        raise Exception("Number of inference requests should be positive (invalid -nireq option value)")
-    if not fnmatch(args.path_to_model, XML_EXTENSION_PATTERN):
-        raise Exception('Path {} is not xml file.')
-
-
-def parse_args():
-    parser = argparse.ArgumentParser(add_help=False)
-    args = parser.add_argument_group('Options')
-    args.add_argument('-h', '--help', action='help', default=argparse.SUPPRESS, help=HELP_MESSAGES["HELP"])
-    args.add_argument('-i', '--path_to_images', type=str, required=True, help=HELP_MESSAGES['IMAGE_MESSAGE'])
-    args.add_argument('-m', '--path_to_model', type=str, required=True, help=HELP_MESSAGES['MODEL_MESSAGE'])
-    args.add_argument('-c', '--path_to_cldnn_config', type=str, required=False,
-                      help=HELP_MESSAGES['CUSTOM_GPU_LIBRARY_MESSAGE'])
-    args.add_argument('-l', '--path_to_extension', type=str, required=False, default=None,
-                      help=HELP_MESSAGES['CUSTOM_GPU_LIBRARY_MESSAGE'])
-    args.add_argument('-api', '--api_type', type=str, required=False, default='async', choices=['sync', 'async'],
-                      help=HELP_MESSAGES['API_MESSAGE'])
-    args.add_argument('-d', '--target_device', type=str, required=False, default="CPU",
-                      help=HELP_MESSAGES['TARGET_DEVICE_MESSAGE'])
-    args.add_argument('-niter', '--number_iterations', type=int, required=False, default=None,
-                      help=HELP_MESSAGES['ITERATIONS_COUNT_MESSAGE'])
-    args.add_argument('-nireq', '--number_infer_requests', type=int, required=False, default=2,
-                      help=HELP_MESSAGES['INFER_REQUESTS_COUNT_MESSAGE'])
-    args.add_argument('-nthreads', '--number_threads', type=int, required=False, default=None,
-                      help=HELP_MESSAGES['INFER_NUM_THREADS_MESSAGE'])
-    args.add_argument('-b', '--batch_size', type=int, required=False, default=None,
-                      help=HELP_MESSAGES['BATCH_SIZE_MESSAGE'])
-    args.add_argument('-pin', '--infer_threads_pinning', type=str, required=False, default='YES',
-                      choices=['YES', 'NO'], help=HELP_MESSAGES['INFER_THREADS_PINNING_MESSAGE'])
-    return parser.parse_args()
-
-
-def get_images(path_to_images, batch_size):
-    images = list()
-    if os.path.isfile(path_to_images):
-        while len(images) != batch_size:
-            images.append(path_to_images)
-    else:
-        path = os.path.join(path_to_images, '*')
-        files = glob(path, recursive=True)
-        for file in files:
-            file_extension = file.rsplit('.').pop().upper()
-            if file_extension in IMAGE_EXTENSIONS:
-                images.append(file)
-        if len(images) == 0:
-            raise Exception("No images found in {}".format(path_to_images))
-        if len(images) < batch_size:
-            while len(images) != batch_size:
-                images.append(choice(images))
-    return images
-
-
-def get_duration_in_secs(target_device):
-    duration = 0
-    for device in DEVICE_DURATION_IN_SECS:
-        if device in target_device:
-            duration = max(duration, DEVICE_DURATION_IN_SECS[device])
-
-    if duration == 0:
-        duration = DEVICE_DURATION_IN_SECS[UNKNOWN_DEVICE_TYPE]
-        logger.warn("Default duration {} seconds for unknown device {} is used".format(duration, target_device))
-
-    return duration
-
-
-def fill_blob_with_image(images_path, shape):
-    images = np.ndarray(shape)
-    for item in range(shape[0]):
-        image = cv2.imread(images_path[item])
-
-        new_im_size = tuple(shape[2:])
-        if image.shape[:-1] != new_im_size:
-            logger.warn("Image {} is resize from ({}) to ({})".format(images_path[item], image.shape[:-1], new_im_size))
-            image = cv2.resize(image, new_im_size)
-
-        image = image.transpose((2, 0, 1))
-        images[item] = image
-    return images
-
-
-def sync_infer_request(exe_network, times, images):
-    iteration_start_time = datetime.now()
-    exe_network.infer(images)
-    current_time = datetime.now()
-    times.append((current_time - iteration_start_time).total_seconds())
-    return current_time

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/constants.py

@@ -1,65 +0,0 @@
-"""
- Copyright (C) 2018-2019 Intel Corporation
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
-      http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-"""
-
-HELP_MESSAGES = {
-    'HELP': "Show this help message and exit.",
-    'IMAGE_MESSAGE': "Required. Path to a folder with images or to image files.",
-    'MULTI_INPUT_MESSAGE': "Optional. Path to multi input file containing.",
-    'MODEL_MESSAGE': "Required. Path to an .xml file with a trained model.",
-    'PLUGIN_PATH_MESSAGE': "Optional. Path to a plugin folder.",
-    'API_MESSAGE': "Optional. Enable using sync/async API. Default value is sync",
-    'TARGET_DEVICE_MESSAGE': "Optional. Specify a target device to infer on: CPU, GPU, FPGA, HDDL or MYRIAD. "
-                           "Use \"-d HETERO:<comma separated devices list>\" format to specify HETERO plugin. "
-    "The application looks for a suitable plugin for the specified device.",
-    'ITERATIONS_COUNT_MESSAGE': "Optional. Number of iterations. "
-    "If not specified, the number of iterations is calculated depending on a device.",
-    'INFER_REQUESTS_COUNT_MESSAGE': "Optional. Number of infer requests (default value is 2).",
-    'INFER_NUM_THREADS_MESSAGE': "Number of threads to use for inference on the CPU "
-                                 "(including Hetero cases).",
-    'CUSTOM_CPU_LIBRARY_MESSAGE': "Optional. Required for CPU custom layers. "
-                                  "Absolute path to a shared library with the kernels implementations.",
-    'CUSTOM_GPU_LIBRARY_MESSAGE': "Optional. Required for GPU custom kernels. Absolute path to an .xml file with the "
-                                  "kernels description.",
-    'BATCH_SIZE_MESSAGE': "Optional. Batch size value. If not specified, the batch size value is determined from IR",
-    'INFER_THREADS_PINNING_MESSAGE': "Optional. Enable (\"YES\" is default value) or disable (\"NO\")"
-                                     "CPU threads pinning for CPU-involved inference."
-}
-
-DEVICE_DURATION_IN_SECS = {
-    "CPU": 60,
-    "GPU": 60,
-    "VPU": 60,
-    "MYRIAD": 60,
-    "FPGA": 120,
-    "HDDL": 60,
-    "UNKNOWN": 120
-}
-
-IMAGE_EXTENSIONS = ['JPEG', 'JPG', 'PNG', 'BMP']
-
-MYRIAD_DEVICE_NAME = "MYRIAD"
-CPU_DEVICE_NAME = "CPU"
-GPU_DEVICE_NAME = "GPU"
-UNKNOWN_DEVICE_TYPE = "UNKNOWN"
-
-BATCH_SIZE_ELEM = 0
-
-LAYOUT_TYPE = 'NCHW'
-
-XML_EXTENSION = ".xml"
-BIN_EXTENSION = ".bin"
-
-XML_EXTENSION_PATTERN = '*' + XML_EXTENSION

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/infer_request_wrap.py

@@ -0,0 +1,81 @@
+"""
+ Copyright (C) 2018-2019 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+from ctypes import *
+from datetime import datetime
+import threading
+
+class InferReqWrap:
+    def __init__(self, request, id, callbackQueue):
+        self.id = id
+        self.request = request
+        self.request.set_completion_callback(self.callback, self.id)
+        self.callbackQueue = callbackQueue
+
+    def callback(self, statusCode, userdata):
+        if (userdata != self.id):
+            print("Request ID {} does not correspond to user data {}".format(self.id, userdata))
+        elif statusCode != 0:
+            print("Request {} failed with status code {}".format(self.id, statusCode))
+        self.callbackQueue(self.id, self.request.latency)
+
+    def startAsync(self, input_data):
+        self.request.async_infer(input_data)
+
+    def infer(self, input_data):
+        self.request.infer(input_data)
+        self.callbackQueue(self.id, self.request.latency);
+
+class InferRequestsQueue:
+    def __init__(self, requests):
+      self.idleIds = []
+      self.requests = []
+      self.times = []
+      for id in range(0, len(requests)):
+          self.requests.append(InferReqWrap(requests[id], id, self.putIdleRequest))
+          self.idleIds.append(id)
+      self.startTime = datetime.max
+      self.endTime = datetime.min
+      self.cv = threading.Condition()
+
+    def resetTimes(self):
+      self.times.clear()
+
+    def getDurationInSeconds(self):
+      return (self.endTime - self.startTime).total_seconds()
+
+    def putIdleRequest(self, id, latency):
+      self.cv.acquire()
+      self.times.append(latency)
+      self.idleIds.append(id)
+      self.endTime = max(self.endTime, datetime.now())
+      self.cv.notify()
+      self.cv.release()
+
+    def getIdleRequest(self):
+        self.cv.acquire()
+        while len(self.idleIds) == 0:
+            self.cv.wait()
+        id = self.idleIds.pop();
+        self.startTime = min(datetime.now(), self.startTime);
+        self.cv.release()
+        return self.requests[id]
+
+    def waitAll(self):
+        self.cv.acquire()
+        while len(self.idleIds) != len(self.requests):
+            self.cv.wait()
+        self.cv.release()

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/inputs_filling.py

@@ -0,0 +1,194 @@
+"""
+ Copyright (C) 2018-2019 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+import logging
+import os
+import cv2
+import numpy as np
+import sys
+
+from glob import glob
+from random import choice
+
+from .logging import logger
+
+IMAGE_EXTENSIONS = ['JPEG', 'JPG', 'PNG', 'BMP']
+BINARY_EXTENSIONS = ['BIN']
+
+def isImage(blob):
+    if (blob.layout != "NCHW"):
+        return False
+    channels = blob.shape[1]
+    return (channels == 3)
+
+def isImageInfo(blob):
+    if (blob.layout != "NC"):
+        return False
+    channels = blob.shape[1]
+    return (channels >= 2)
+
+def getInputs(path_to_input, batch_size, input_info, requests):
+  input_image_sizes = {}
+  for key in input_info.keys():
+      if (isImage(input_info[key])):
+          input_image_sizes[key] = (input_info[key].shape[2], input_info[key].shape[3])
+      logger.info("Network input '{}' precision {}, dimensions ({}): {}".format(key,
+                                                                         input_info[key].precision,
+                                                                         input_info[key].layout,
+                                                                         " ".join(str(x) for x in input_info[key].shape)))
+
+  images_count = len(input_image_sizes.keys())
+  binaries_count = len(input_info) - images_count
+
+  image_files = list()
+  binary_files = list()
+
+  if (path_to_input):
+    image_files = get_files_by_extensions(path_to_input, IMAGE_EXTENSIONS)
+    image_files.sort()
+    binary_files = get_files_by_extensions(path_to_input, BINARY_EXTENSIONS)
+    binary_files.sort()
+
+  if (len(image_files) == 0) and (len(binary_files) == 0):
+      logger.warn("No input files were given: all inputs will be filled with random values!")
+  else:
+      binary_to_be_used = binaries_count*batch_size*len(requests)
+      if binary_to_be_used > 0 and len(binary_files) == 0:
+          logger.warn("No supported binary inputs found! Please check your file extensions: {}".format(",".join(BINARY_EXTENSIONS)))
+      elif binary_to_be_used > len(binary_files):
+          logger.warn("Some binary input files will be duplicated: {} files are required, but only {} were provided".format(binary_to_be_used, len(binary_files)))
+      elif binary_to_be_used < len(binary_files):
+          logger.warn("Some binary input files will be ignored: only {} files are required from {}".format(binary_to_be_used, len(binary_files)))
+
+      images_to_be_used = images_count*batch_size*len(requests)
+      if images_to_be_used > 0 and len(image_files) == 0:
+          logger.warn("No supported image inputs found! Please check your file extensions: {}".format(",".join(IMAGE_EXTENSIONS)))
+      elif images_to_be_used > len(image_files):
+          logger.warn("Some image input files will be duplicated: {} files are required, but only {} were provided".format(images_to_be_used, len(image_files)))
+      elif images_to_be_used < len(image_files):
+          logger.warn("Some image input files will be ignored: only {} files are required from {}".format(images_to_be_used, len(image_files)))
+
+  requests_input_data = []
+  for request_id in range(0, len(requests)):
+      logger.info("Infer Request {} filling".format(request_id))
+      input_data = {}
+      keys = list(input_info.keys())
+      for key in keys:
+          if isImage(input_info[key]):
+              # input is image
+              if (len(image_files) > 0):
+                  input_data[key] = fill_blob_with_image(image_files, request_id, batch_size, keys.index(key), len(keys), input_info[key].shape)
+                  continue
+
+          # input is binary
+          if (len(binary_files) > 0):
+              input_data[key] = fill_blob_with_binary(binary_files, input_info[key].shape)
+              continue
+
+          # most likely input is image info
+          if isImageInfo(input_info[key]) and len(input_image_sizes) == 1:
+              image_size = input_image_sizes[list(input_image_sizes.keys()).pop()]
+              logger.info("Fill input '" + key + "' with image size " + str(image_size[0]) + "x" +
+                                                                        str(image_size[1]))
+              input_data[key] = fill_blob_with_image_info(image_size, input_info[key].shape)
+              continue
+
+          # fill with random data
+          logger.info("Fill input '{}' with random values ({} is expected)".format(key, "image" if isImage(input_info[key]) else "some binary data"))
+          input_data[key] = fill_blob_with_random(input_info[key].precision, input_info[key].shape)
+
+      requests_input_data.append(input_data)
+
+  return requests_input_data
+
+def get_files_by_extensions(path_to_input, extensions):
+    input_files = list()
+    if os.path.isfile(path_to_input):
+        input_files.append(path_to_input)
+    else:
+        path = os.path.join(path_to_input, '*')
+        files = glob(path, recursive=True)
+        for file in files:
+            file_extension = file.rsplit('.').pop().upper()
+            if file_extension in extensions:
+                input_files.append(file)
+    return input_files
+
+def fill_blob_with_image(image_paths, request_id, batch_size, input_id, input_size, shape):
+    images = np.ndarray(shape)
+    image_index = request_id*batch_size*input_size + input_id
+    for b in range(batch_size):
+        image_index %= len(image_paths)
+        image_filename = image_paths[image_index]
+        image = cv2.imread(image_filename)
+
+        new_im_size = tuple(shape[2:])
+        if image.shape[:-1] != new_im_size:
+            logger.warn("Image {} is resized from ({}) to ({})".format(image_filename, image.shape[:-1], new_im_size))
+            image = cv2.resize(image, new_im_size)
+
+        image = image.transpose((2, 1, 0))
+        images[b] = image
+
+        image_index += input_size
+    return images
+
+def fill_blob_with_binary(binary_paths, request_id, batch_size, input_id, input_size, shape):
+  binaries = np.ndarray(shape)
+  binary_index = request_id*batch_size*input_size + input_id
+  for b in range(batch_size):
+      binary_index %= len(image_paths)
+      binary_filename = binary_paths[binary_index]
+
+      binary_file_size = os.path.getsize(binary_file)
+      input_size = np.prod(shape)/batch_size
+      if (input_size != binary_file_size):
+          raise Exception("File " + binary_filename + " contains " << str(binary_file_size) + " bytes " +
+                          "but network expects " + str(input_size))
+
+      with open(binary_file, 'r') as f:
+          binary_data = f.read()
+
+      binaries[b] = binary_data
+      binary_index += input_size
+
+  return binaries
+
+def fill_blob_with_image_info(image_size, shape):
+    im_info = np.ndarray(shape)
+    for b in range(shape[0]):
+      for i in range(shape[1]):
+        im_info[b][i] = image_size[i] if i in [0, 1] else 1
+
+    return im_info
+
+def fill_blob_with_random(precision, shape):
+    if precision == "FP32":
+      return np.random.rand(*shape).astype(np.float32)
+    elif precision == "FP16":
+      return np.random.rand(*shape).astype(np.float16)
+    elif precision == "I32":
+      return np.random.rand(*shape).astype(np.int32)
+    elif precision == "U8":
+      return np.random.rand(*shape).astype(np.uint8)
+    elif precision == "I8":
+      return np.random.rand(*shape).astype(np.int8)
+    elif precision == "U16":
+      return np.random.rand(*shape).astype(np.uint16)
+    elif precision == "I16":
+      return np.random.rand(*shape).astype(np.int16)
+    else:
+      raise Exception("Input precision is not supported: " + precision)

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/logging.py

@@ -1,5 +1,5 @@
 """
- Copyright (c) 2018-2019 Intel Corporation
+ Copyright (C) 2018-2019 Intel Corporation
 
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
@@ -14,11 +14,8 @@
  limitations under the License.
 """
 
-from mo.front.common.partial_infer.expand_dims import tf_expand_dims_infer
+import logging
+import sys
 
-
-def tf_expand_dims_ext(pb):
-    return {
-        'type': 'Reshape',
-        'infer': tf_expand_dims_infer
-    }
+logging.basicConfig(format="[ %(levelname)s ] %(message)s", level=logging.INFO, stream=sys.stdout)
+logger = logging.getLogger('BenchmarkApp')

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/parameters.py

@@ -0,0 +1,92 @@
+"""
+ Copyright (C) 2018-2019 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+import argparse
+from fnmatch import fnmatch
+
+XML_EXTENSION = ".xml"
+BIN_EXTENSION = ".bin"
+
+XML_EXTENSION_PATTERN = '*' + XML_EXTENSION
+
+def validate_args(args):
+    if args.number_iterations is not None and args.number_iterations < 0:
+        raise Exception("Number of iterations should be positive (invalid -niter option value)")
+    if args.number_infer_requests and args.number_infer_requests < 0:
+        raise Exception("Number of inference requests should be positive (invalid -nireq option value)")
+    if not fnmatch(args.path_to_model, XML_EXTENSION_PATTERN):
+        raise Exception('Path {} is not xml file.')
+
+def str2bool(v):
+  if v.lower() in ('yes', 'true', 't', 'y', '1'):
+    return True
+  elif v.lower() in ('no', 'false', 'f', 'n', '0'):
+    return False
+  else:
+    raise argparse.ArgumentTypeError('Boolean value expected.')
+
+def parse_args():
+    parser = argparse.ArgumentParser(add_help=False)
+    args = parser.add_argument_group('Options')
+    args.add_argument('-h', '--help', action='help', default=argparse.SUPPRESS,
+                      help="Show this help message and exit.")
+    args.add_argument('-i', '--path_to_input', type=str, required=False,
+                      help="Optional. Path to a folder with images and/or binaries or to specific image or binary file.")
+    args.add_argument('-m', '--path_to_model', type=str, required=True,
+                      help="Required. Path to an .xml file with a trained model.")
+    args.add_argument('-d', '--target_device', type=str, required=False, default="CPU",
+                      help="Optional. Specify a target device to infer on: CPU, GPU, FPGA, HDDL or MYRIAD. "
+                           "Use \"-d HETERO:<comma separated devices list>\" format to specify HETERO plugin. ")
+    args.add_argument('-l', '--path_to_extension', type=str, required=False, default=None,
+                      help="Optional. Required for CPU custom layers. "
+                           "Absolute path to a shared library with the kernels implementations.")
+    args.add_argument('-c', '--path_to_cldnn_config', type=str, required=False,
+                      help="Optional. Required for GPU custom kernels. Absolute path to an .xml file with the "
+                           "kernels description.")
+    args.add_argument('-api', '--api_type', type=str, required=False, default='async', choices=['sync', 'async'],
+                      help="Optional. Enable using sync/async API. Default value is async.")
+    args.add_argument('-niter', '--number_iterations', type=int, required=False, default=None,
+                      help="Optional. Number of iterations. "
+                           "If not specified, the number of iterations is calculated depending on a device.")
+    args.add_argument('-nireq', '--number_infer_requests', type=int, required=False, default=None,
+                      help="Optional. Number of infer requests. Default value is determined automatically for device.")
+    args.add_argument('-b', '--batch_size', type=int, required=False, default=None,
+                      help="Optional. Batch size value. If not specified, the batch size value is determined from Intermediate Representation")
+    args.add_argument('-stream_output', type=str2bool, required=False, default=False, nargs='?', const=True,
+                      help="Optional. Print progress as a plain text. When specified, an interactive progress bar is replaced with a "
+                           "multiline output.")
+    args.add_argument('-t', '--time', type=int, required=False, default=None,
+                      help="Optional. Time in seconds to execute topology.")
+    args.add_argument('-progress', type=str2bool, required=False, default=False, nargs='?', const=True,
+                      help="Optional. Show progress bar (can affect performance measurement). Default values is \"False\".")
+    args.add_argument('-nstreams', '--number_streams', type=str, required=False, default=None,
+                      help="Optional. Number of streams to use for inference on the CPU/GPU in throughput mode "
+                           "(for HETERO device case use format <device1>:<nstreams1>,<device2>:<nstreams2> or just <nstreams>).")
+    args.add_argument('-nthreads', '--number_threads', type=int, required=False, default=None,
+                      help="Number of threads to use for inference on the CPU "
+                           "(including HETERO case).")
+    args.add_argument('-pin', '--infer_threads_pinning', type=str, required=False, default='YES', choices=['YES', 'NO'],
+                      help="Optional. Enable (\"YES\" is default value) or disable (\"NO\")"
+                      "CPU threads pinning for CPU-involved inference.")
+    args.add_argument('--exec_graph_path', type=str, required=False,
+                      help="Optional. Path to a file where to store executable graph information serialized.")
+    args.add_argument("-pc", "--perf_counts", type=str2bool, required=False, default=False, nargs='?', const=True,
+                      help="Optional. Report performance counters.", )
+    parsed_args = parser.parse_args()
+
+    validate_args(parsed_args)
+
+    return parsed_args

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/progress_bar.py

@@ -0,0 +1,51 @@
+"""
+ Copyright (C) 2018-2019 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+from progress.bar import Bar
+
+class ProgressBar:
+    def __init__(self, total_num, stream_output=False, progress_enabled=False):
+        self.stream_output = stream_output
+        self.is_finished = True
+        self.progress_enabled = progress_enabled
+        self.reset(total_num)
+
+    def add_progress(self, num):
+        self.is_finished = False
+        if self.progress_enabled:
+           for i in range(num):
+              self.bar.next()
+              if self.stream_output:
+                  print()
+
+    def finish(self, num = 0):
+        if (num > 0):
+            self.add_progress(num)
+
+        self.is_finished = True
+        if self.progress_enabled:
+            self.bar.finish()
+            print()
+
+    def reset(self, total_num):
+        if self.progress_enabled:
+            self.bar = Bar('Progress:', max = total_num, fill = '.', suffix='%(percent).2f%%')
+
+    def new_bar(self, total_num):
+        if self.is_finished:
+            self.reset(total_num)
+        else:
+           raise Exception("Cannot create a new bar. Current bar is still in progress")

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark/utils/utils.py

@@ -0,0 +1,99 @@
+"""
+ Copyright (C) 2018-2019 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+import multiprocessing
+from .logging import logger
+
+VPU_DEVICE_NAME = "VPU"
+MYRIAD_DEVICE_NAME = "MYRIAD"
+HDDL_DEVICE_NAME = "HDDL"
+FPGA_DEVICE_NAME = "FPGA"
+CPU_DEVICE_NAME = "CPU"
+GPU_DEVICE_NAME = "GPU"
+HETERO_DEVICE_NAME = "HETERO"
+UNKNOWN_DEVICE_TYPE = "UNKNOWN"
+
+DEVICE_DURATION_IN_SECS = {
+    CPU_DEVICE_NAME: 60,
+    GPU_DEVICE_NAME: 60,
+    VPU_DEVICE_NAME: 60,
+    MYRIAD_DEVICE_NAME: 60,
+    HDDL_DEVICE_NAME: 60,
+    FPGA_DEVICE_NAME: 120,
+    UNKNOWN_DEVICE_TYPE: 120
+}
+
+DEVICE_NIREQ_ASYNC = {
+    CPU_DEVICE_NAME: 2,
+    GPU_DEVICE_NAME: 2,
+    VPU_DEVICE_NAME: 4,
+    MYRIAD_DEVICE_NAME: 4,
+    HDDL_DEVICE_NAME: 100,
+    FPGA_DEVICE_NAME: 3,
+    UNKNOWN_DEVICE_TYPE: 1
+}
+
+def get_duration_in_secs(target_device):
+    duration = 0
+    for device in DEVICE_DURATION_IN_SECS:
+        if device in target_device:
+            duration = max(duration, DEVICE_DURATION_IN_SECS[device])
+
+    if duration == 0:
+        duration = DEVICE_DURATION_IN_SECS[UNKNOWN_DEVICE_TYPE]
+        logger.warn("Default duration {} seconds is used for unknown device {}".format(duration, target_device))
+
+    return duration
+
+def get_nireq(target_device):
+    nireq = 0
+    for device in DEVICE_NIREQ_ASYNC:
+        if device in target_device:
+            nireq = max(nireq, DEVICE_NIREQ_ASYNC[device])
+
+    if nireq == 0:
+        nireq = DEVICE_NIREQ_ASYNC[UNKNOWN_DEVICE_TYPE]
+        logger.warn("Default number of requests {} is used for unknown device {}".format(duration, target_device))
+
+    return nireq
+
+def parseDevices(device_string):
+    devices = device_string
+    if ':' in devices:
+        devices = devices.partition(':')[2]
+    return [ d[:d.index('(')] if '(' in d else d for d in devices.split(',') ]
+
+def parseValuePerDevice(devices, values_string):
+    ## Format: <device1>:<value1>,<device2>:<value2> or just <value>
+    result = {}
+    if not values_string:
+      return result
+    device_value_strings = values_string.upper().split(',')
+    for device_value_string in device_value_strings:
+        device_value_vec = device_value_string.split(':')
+        if len(device_value_vec) == 2:
+            for device in devices:
+                if device == device_value_vec[0]:
+                    value = int(device_value_vec[1])
+                    result[device_value_vec[0]] = value
+                    break
+        elif len(device_value_vec) == 1:
+            value = int(device_value_vec[0])
+            for device in devices:
+                result[device] = value
+        elif not device_value_vec:
+            raise Exception("Unknown string format: " + values_string)
+    return result

inference-engine/ie_bridges/python/sample/benchmark_app/benchmark_app.py

@@ -1,37 +1,4 @@
 import benchmark
 
-from argparse import ArgumentParser, SUPPRESS
-
-
-def parse_args():
-    parser = ArgumentParser(add_help=False)
-    args = parser.add_argument_group('Options')
-    args.add_argument('-h', '--help', action='help', default=SUPPRESS, help=benchmark.HELP_MESSAGES["HELP"])
-    args.add_argument('-i', '--path_to_images', type=str, required=True,
-                      help=benchmark.HELP_MESSAGES['IMAGE_MESSAGE'])
-    args.add_argument('-m', '--path_to_model', type=str, required=True,
-                      help=benchmark.HELP_MESSAGES['MODEL_MESSAGE'])
-    args.add_argument('-c', '--path_to_cldnn_config', type=str, required=False,
-                      help=benchmark.HELP_MESSAGES['CUSTOM_GPU_LIBRARY_MESSAGE'])
-    args.add_argument('-l', '--path_to_extension', type=str, required=False, default=None,
-                      help=benchmark.HELP_MESSAGES['CUSTOM_GPU_LIBRARY_MESSAGE'])
-    args.add_argument('-api', '--api_type', type=str, required=False, default='async', choices=['sync', 'async'],
-                      help=benchmark.HELP_MESSAGES['API_MESSAGE'])
-    args.add_argument('-d', '--target_device', type=str, required=False, default="CPU",
-                      help=benchmark.HELP_MESSAGES['TARGET_DEVICE_MESSAGE'])
-    args.add_argument('-niter', '--number_iterations', type=int, required=False, default=None,
-                      help=benchmark.HELP_MESSAGES['ITERATIONS_COUNT_MESSAGE'])
-    args.add_argument('-nireq', '--number_infer_requests', type=int, required=False, default=2,
-                      help=benchmark.HELP_MESSAGES['INFER_REQUESTS_COUNT_MESSAGE'])
-    args.add_argument('-nthreads', '--number_threads', type=int, required=False, default=None,
-                      help=benchmark.HELP_MESSAGES['INFER_NUM_THREADS_MESSAGE'])
-    args.add_argument('-b', '--batch_size', type=int, required=False, default=None,
-                      help=benchmark.HELP_MESSAGES['BATCH_SIZE_MESSAGE'])
-    args.add_argument('-pin', '--infer_threads_pinning', type=str, required=False, default='YES',
-                      choices=['YES', 'NO'], help=benchmark.HELP_MESSAGES['INFER_THREADS_PINNING_MESSAGE'])
-    return parser.parse_args()
-
-
 if __name__ == "__main__":
-    args = parse_args()
-    benchmark.main(args)
+    benchmark.main()

inference-engine/ie_bridges/python/sample/classification_sample/README.md

@@ -3,13 +3,13 @@
 This topic demonstrates how to run the Image Classification sample application, which performs
 inference using image classification networks such as AlexNet and GoogLeNet.
 
-### How It Works
+## How It Works
 
 Upon the start-up, the sample application reads command line parameters and loads a network and an image to the Inference
 Engine plugin. When inference is done, the application creates an
 output image and outputs data to the standard output stream.
 
-> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Specify Input Shapes** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
+> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
 
 ## Running
 
@@ -22,7 +22,6 @@ The command yields the following usage message:
 usage: classification_sample.py [-h] -m MODEL -i INPUT [INPUT ...]
                                 [-l CPU_EXTENSION] [-pp PLUGIN_DIR]
                                 [-d DEVICE] [--labels LABELS] [-nt NUMBER_TOP]
-                                [-ni NUMBER_ITER] [-pc]
 
 Options:
   -h, --help            Show this help message and exit.
@@ -45,9 +44,6 @@ Options:
   --labels LABELS       Optional. Path to a labels mapping file
   -nt NUMBER_TOP, --number_top NUMBER_TOP
                         Optional. Number of top results
-  -ni NUMBER_ITER, --number_iter NUMBER_ITER
-                        Optional. Number of inference iterations
-  -pc, --perf_counts    Optional. Report performance counters
 ```
 
 Running the application with the empty list of options yields the usage message given above.
@@ -62,18 +58,16 @@ For example, to perform inference of an AlexNet model (previously converted to t
     python3 classification_sample.py -i <path_to_image>/cat.bmp -m <path_to_model>/alexnet_fp32.xml
 ```
 
-### Sample Output
+## Sample Output
 
 By default the application outputs top-10 inference results.
 Add the `-nt` option to the previous command to modify the number of top output results.
 For example, to get the top-5 results on GPU, run the following command:
 ```
-    python3 classification_sample.py<path_to_image>/cat.bmp -m <path_to_model>/alexnet_fp32.xml -nt 5 -d GPU
+    python3 classification_sample.py -i <path_to_image>/cat.bmp -m <path_to_model>/alexnet_fp32.xml -nt 5 -d GPU
 ```
 
 ## See Also
 * [Using Inference Engine Samples](./docs/IE_DG/Samples_Overview.md)
 * [Model Optimizer tool](./docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
 * [Model Downloader](https://github.com/opencv/open_model_zoo/tree/2018/model_downloader)
-
-

inference-engine/ie_bridges/python/sample/classification_sample/classification_sample.py

@@ -22,7 +22,7 @@ import cv2
 import numpy as np
 import logging as log
 from time import time
-from openvino.inference_engine import IENetwork, IEPlugin
+from openvino.inference_engine import IENetwork, IECore
 
 
 def build_argparser():
@@ -38,7 +38,6 @@ def build_argparser():
                       help="Optional. Required for CPU custom layers. "
                            "MKLDNN (CPU)-targeted custom layers. Absolute path to a shared library with the"
                            " kernels implementations.", type=str, default=None)
-    args.add_argument("-pp", "--plugin_dir", help="Optional. Path to a plugin folder", type=str, default=None)
     args.add_argument("-d", "--device",
                       help="Optional. Specify the target device to infer on; CPU, GPU, FPGA, HDDL, MYRIAD or HETERO: is "
                            "acceptable. The sample will look for a suitable plugin for device specified. Default "
@@ -46,9 +45,6 @@ def build_argparser():
                       default="CPU", type=str)
     args.add_argument("--labels", help="Optional. Path to a labels mapping file", default=None, type=str)
     args.add_argument("-nt", "--number_top", help="Optional. Number of top results", default=10, type=int)
-    args.add_argument("-ni", "--number_iter", help="Optional. Number of inference iterations", default=1, type=int)
-    args.add_argument("-pc", "--perf_counts", help="Optional. Report performance counters", default=False,
-                      action="store_true")
 
     return parser
 
@@ -60,19 +56,20 @@ def main():
     model_bin = os.path.splitext(model_xml)[0] + ".bin"
 
     # Plugin initialization for specified device and load extensions library if specified
-    plugin = IEPlugin(device=args.device, plugin_dirs=args.plugin_dir)
+    log.info("Creating Inference Engine")
+    ie = IECore()
     if args.cpu_extension and 'CPU' in args.device:
-        plugin.add_cpu_extension(args.cpu_extension)
+        ie.add_extension(args.cpu_extension, "CPU")
     # Read IR
     log.info("Loading network files:\n\t{}\n\t{}".format(model_xml, model_bin))
     net = IENetwork(model=model_xml, weights=model_bin)
 
-    if plugin.device == "CPU":
-        supported_layers = plugin.get_supported_layers(net)
+    if "CPU" in args.device:
+        supported_layers = ie.query_network(net, "CPU")
         not_supported_layers = [l for l in net.layers.keys() if l not in supported_layers]
         if len(not_supported_layers) != 0:
             log.error("Following layers are not supported by the plugin for specified device {}:\n {}".
-                      format(plugin.device, ', '.join(not_supported_layers)))
+                      format(args.device, ', '.join(not_supported_layers)))
             log.error("Please try to specify cpu extensions library path in sample's command line parameters using -l "
                       "or --cpu_extension command line argument")
             sys.exit(1)
@@ -99,23 +96,11 @@ def main():
 
     # Loading model to the plugin
     log.info("Loading model to the plugin")
-    exec_net = plugin.load(network=net)
+    exec_net = ie.load_network(network=net, device_name=args.device)
 
     # Start sync inference
-    log.info("Starting inference ({} iterations)".format(args.number_iter))
-    infer_time = []
-    for i in range(args.number_iter):
-        t0 = time()
-        res = exec_net.infer(inputs={input_blob: images})
-        infer_time.append((time() - t0) * 1000)
-    log.info("Average running time of one iteration: {} ms".format(np.average(np.asarray(infer_time))))
-    if args.perf_counts:
-        perf_counts = exec_net.requests[0].get_perf_counts()
-        log.info("Performance counters:")
-        print("{:<70} {:<15} {:<15} {:<15} {:<10}".format('name', 'layer_type', 'exet_type', 'status', 'real_time, us'))
-        for layer, stats in perf_counts.items():
-            print("{:<70} {:<15} {:<15} {:<15} {:<10}".format(layer, stats['layer_type'], stats['exec_type'],
-                                                              stats['status'], stats['real_time']))
+    log.info("Starting inference in synchronous mode")
+    res = exec_net.infer(inputs={input_blob: images})
 
     # Processing output blob
     log.info("Processing output blob")
@@ -144,7 +129,7 @@ def main():
                                           ' ' * space_num_after, ' ' * space_num_before_prob,
                                           probs[id]))
         print("\n")
-
+    log.info("This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool\n")
 
 if __name__ == '__main__':
     sys.exit(main() or 0)

inference-engine/ie_bridges/python/sample/classification_sample_async/README.md

@@ -1,45 +1,40 @@
 # Image Classification Python* Sample Async
 
-This sample demonstrates how to build and execute inference in pipelined mode on example of classifications networks.
+This sample demonstrates how to run the Image Classification sample application with inference executed in the asynchronous mode.
 
-The pipelined mode might increase the throughput of the pictures. The latency of one inference will be the same as for synchronous execution.
-<br>
-The throughput increases due to follow reasons:
-* Some plugins have heterogeneity inside themselves: data transferring, execution on remote device, pre-processing and post-processing on the host.
-* Using of explicit heterogeneous plugin with execution of different parts of network on different devices, for example HETERO:CPU,GPU.
+The sample demonstrates how to use the new Infer Request API of Inference Engine in applications.
+Refer to [Integrate the Inference Engine New Request API with Your Application](./docs/IE_DG/Integrate_with_customer_application_new_API.md) for details.
+The sample demonstrates how to build and execute an inference request 10 times in the asynchronous mode on example of classifications networks.
+The asynchronous mode might increase the throughput of the pictures.
 
-When two or more devices process one image, creating several infer requests and starting asynchronous inference allow for using devices in the most efficient way.
-If two devices are involved in execution, the most optimal value for `-nireq` option is 2.
-To process infer requests more efficiently, Classification Sample Async uses round-robin algorithm. It starts execution of the current infer request and switches to waiting for results of the previous one. After finishing of waiting, it switches infer requests and repeat the procedure.
+The batch mode is an independent attribute on the asynchronous mode. Asynchronous mode works efficiently with any batch size.
 
-Another required aspect of good throughput is a number of iterations. Only with big number of iterations you can emulate the real application work and get good performance.
+## How It Works
 
-The batch mode is an independent attribute on the pipelined mode. Pipelined mode works efficiently with any batch size.
+Upon the start-up, the sample application reads command line parameters and loads specified network and input images (or a
+folder with images) to the Inference Engine plugin. The batch size of the network is set according to the number of read images.
 
-### How It Works
+Then, the sample creates an inference request object and assigns completion callback for it. In scope of the completion callback
+handling the inference request is executed again.
 
-Upon the start-up, the sample application reads command line parameters and loads a network and an image to the Inference
-Engine plugin.
-Then application creates several infer requests pointed in `-nireq` parameter and loads images for inference.
-
-Then in a loop it starts inference for the current infer request and switches to waiting for the previous one. When results are ready, it swaps infer requests.
+After that, the application starts inference for the first infer request and waits of 10th inference request execution being completed.
 
 When inference is done, the application outputs data to the standard output stream.
 
-> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Specify Input Shapes** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
+> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
 
 ## Running
 
 Running the application with the <code>-h</code> option yields the following usage message:
 ```
-python3 classification_sample_async.py -h 
+python3 classification_sample_async.py -h
 ```
 The command yields the following usage message:
 ```
 usage: classification_sample_async.py [-h] -m MODEL -i INPUT [INPUT ...]
                                       [-l CPU_EXTENSION] [-pp PLUGIN_DIR]
                                       [-d DEVICE] [--labels LABELS]
-                                      [-nt NUMBER_TOP] [-ni NUMBER_ITER] [-pc]
+                                      [-nt NUMBER_TOP]
 
 Options:
   -h, --help            Show this help message and exit.
@@ -62,10 +57,6 @@ Options:
   --labels LABELS       Optional. Labels mapping file
   -nt NUMBER_TOP, --number_top NUMBER_TOP
                         Optional. Number of top results
-  -ni NUMBER_ITER, --number_iter NUMBER_ITER
-                        Optional. Number of inference iterations
-  -pc, --perf_counts    Optional. Report performance counters
-
 ```
 
 Running the application with the empty list of options yields the usage message given above and an error message.
@@ -75,12 +66,12 @@ To run the sample, you can use AlexNet and GoogLeNet or other image classificati
 > **NOTE**: Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](./docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
 
 
-You can do inference on an image using a trained AlexNet network on FPGA with fallback to CPU using the following command:
+You can do inference of an image using a trained AlexNet network on FPGA with fallback to CPU using the following command:
 ```
     python3 classification_sample_async.py -i <path_to_image>/cat.bmp -m <path_to_model>/alexnet_fp32.xml -nt 5 -d HETERO:FPGA,CPU -nireq 2 -ni 200
 ```
 
-### Sample Output
+## Sample Output
 
 By default, the application outputs top-10 inference results for each infer request.
 It also provides throughput value measured in frames per seconds.

inference-engine/ie_bridges/python/sample/classification_sample_async/classification_sample_async.py

@@ -22,7 +22,56 @@ import cv2
 import numpy as np
 import logging as log
 from time import time
-from openvino.inference_engine import IENetwork, IEPlugin
+from openvino.inference_engine import IENetwork, IECore
+import threading
+
+
+class InferReqWrap:
+    def __init__(self, request, id, num_iter):
+        self.id = id
+        self.request = request
+        self.num_iter = num_iter
+        self.cur_iter = 0
+        self.cv = threading.Condition()
+        self.request.set_completion_callback(self.callback, self.id)
+
+    def callback(self, statusCode, userdata):
+        if (userdata != self.id):
+            log.error("Request ID {} does not correspond to user data {}".format(self.id, userdata))
+        elif statusCode != 0:
+            log.error("Request {} failed with status code {}".format(self.id, statusCode))
+        self.cur_iter += 1
+        log.info("Completed {} Async request execution".format(self.cur_iter))
+        if self.cur_iter < self.num_iter:
+            # here a user can read output containing inference results and put new input
+            # to repeat async request again
+            self.request.async_infer(self.input)
+        else:
+            # continue sample execution after last Asynchronous inference request execution
+            self.cv.acquire()
+            self.cv.notify()
+            self.cv.release()
+
+    def execute(self, mode, input_data):
+        if (mode == "async"):
+            log.info("Start inference ({} Asynchronous executions)".format(self.num_iter))
+            self.input = input_data
+            # Start async request for the first time. Wait all repetitions of the async request
+            self.request.async_infer(input_data)
+            self.cv.acquire()
+            self.cv.wait()
+            self.cv.release()
+        elif (mode == "sync"):
+            log.info("Start inference ({} Synchronous executions)".format(self.num_iter))
+            for self.cur_iter in range(self.num_iter):
+                # here we start inference synchronously and wait for
+                # last inference request execution
+                self.request.infer(input_data)
+                log.info("Completed {} Sync request execution".format(self.cur_iter + 1))
+        else:
+            log.error("wrong inference mode is chosen. Please use \"sync\" or \"async\" mode")
+            sys.exit(1)
+
 
 
 def build_argparser():
@@ -36,20 +85,15 @@ def build_argparser():
     args.add_argument("-l", "--cpu_extension",
                       help="Optional. Required for CPU custom layers. Absolute path to a shared library with the"
                            " kernels implementations.", type=str, default=None)
-    args.add_argument("-pp", "--plugin_dir", help="Optional. Path to a plugin folder", type=str, default=None)
     args.add_argument("-d", "--device",
                       help="Optional. Specify the target device to infer on; CPU, GPU, FPGA, HDDL or MYRIAD is "
                            "acceptable. The sample will look for a suitable plugin for device specified. Default value is CPU",
                       default="CPU", type=str)
     args.add_argument("--labels", help="Optional. Labels mapping file", default=None, type=str)
     args.add_argument("-nt", "--number_top", help="Optional. Number of top results", default=10, type=int)
-    args.add_argument("-ni", "--number_iter", help="Optional. Number of inference iterations", default=1, type=int)
-    args.add_argument("-pc", "--perf_counts", help="Optional. Report performance counters",
-                      default=False, action="store_true")
 
     return parser
 
-
 def main():
     log.basicConfig(format="[ %(levelname)s ] %(message)s", level=log.INFO, stream=sys.stdout)
     args = build_argparser().parse_args()
@@ -57,19 +101,20 @@ def main():
     model_bin = os.path.splitext(model_xml)[0] + ".bin"
 
     # Plugin initialization for specified device and load extensions library if specified
-    plugin = IEPlugin(device=args.device, plugin_dirs=args.plugin_dir)
+    log.info("Creating Inference Engine")
+    ie = IECore()
     if args.cpu_extension and 'CPU' in args.device:
-        plugin.add_cpu_extension(args.cpu_extension)
+        ie.add_extension(args.cpu_extension, "CPU")
     # Read IR
     log.info("Loading network files:\n\t{}\n\t{}".format(model_xml, model_bin))
     net = IENetwork(model=model_xml, weights=model_bin)
 
-    if plugin.device == "CPU":
-        supported_layers = plugin.get_supported_layers(net)
+    if "CPU" in args.device:
+        supported_layers = ie.query_network(net, "CPU")
         not_supported_layers = [l for l in net.layers.keys() if l not in supported_layers]
         if len(not_supported_layers) != 0:
             log.error("Following layers are not supported by the plugin for specified device {}:\n {}".
-                      format(plugin.device, ', '.join(not_supported_layers)))
+                      format(args.device, ', '.join(not_supported_layers)))
             log.error("Please try to specify cpu extensions library path in sample's command line parameters using -l "
                       "or --cpu_extension command line argument")
             sys.exit(1)
@@ -95,27 +140,20 @@ def main():
 
     # Loading model to the plugin
     log.info("Loading model to the plugin")
-    exec_net = plugin.load(network=net)
+    exec_net = ie.load_network(network=net, device_name=args.device)
+
+    # create one inference request for asynchronous execution
+    request_id = 0
+    infer_request = exec_net.requests[request_id];
+
+    num_iter = 10
+    request_wrap = InferReqWrap(infer_request, request_id, num_iter)
+    # Start inference request execution. Wait for last execution being completed
+    request_wrap.execute("sync", {input_blob: images})
 
-    # Start sync inference
-    log.info("Starting inference ({} iterations)".format(args.number_iter))
-    infer_time = []
-    for i in range(args.number_iter):
-        t0 = time()
-        infer_request_handle = exec_net.start_async(request_id=0, inputs={input_blob: images})
-        infer_request_handle.wait()
-        infer_time.append((time() - t0) * 1000)
-    log.info("Average running time of one iteration: {} ms".format(np.average(np.asarray(infer_time))))
-    if args.perf_counts:
-        perf_counts = infer_request_handle.get_perf_counts()
-        log.info("Performance counters:")
-        print("{:<70} {:<15} {:<15} {:<15} {:<10}".format('name', 'layer_type', 'exet_type', 'status', 'real_time, us'))
-        for layer, stats in perf_counts.items():
-            print("{:<70} {:<15} {:<15} {:<15} {:<10}".format(layer, stats['layer_type'], stats['exec_type'],
-                                                              stats['status'], stats['real_time']))
     # Processing output blob
     log.info("Processing output blob")
-    res = infer_request_handle.outputs[out_blob]
+    res = infer_request.outputs[out_blob]
     log.info("Top {} results: ".format(args.number_top))
     if args.labels:
         with open(args.labels, 'r') as f:
@@ -140,7 +178,7 @@ def main():
                                           ' ' * space_num_after, ' ' * space_num_before_prob,
                                           probs[id]))
         print("\n")
-
+    log.info("This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool\n")
 
 if __name__ == '__main__':
     sys.exit(main() or 0)

inference-engine/ie_bridges/python/sample/hello_query_device/README.md

@@ -0,0 +1,50 @@
+# Hello Query Device Python* Sample
+
+This topic demonstrates how to run the Hello Query Device sample application, which queries Inference Engine 
+devices and prints their metrics and default configuration values. The sample shows 
+how to use Query Device API feature. 
+
+
+## How It Works
+
+The sample queries all available Inference Engine devices and prints their supported metrics and plugin configuration parameters.
+ 
+
+## Running
+
+The sample has no command-line parameters. To see the report, run the following command:
+
+```
+python3 hello_query_device.py
+```
+
+## Sample Output
+
+The application prints all available devices with their supported metrics and default values for configuration parameters. For example:
+
+```
+Available devices:
+	Device: CPU
+	Metrics:
+		AVAILABLE_DEVICES: 0
+		SUPPORTED_METRICS: AVAILABLE_DEVICES, SUPPORTED_METRICS, FULL_DEVICE_NAME, OPTIMIZATION_CAPABILITIES, SUPPORTED_CONFIG_KEYS, RANGE_FOR_ASYNC_INFER_REQUESTS, RANGE_FOR_STREAMS
+		FULL_DEVICE_NAME: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
+		OPTIMIZATION_CAPABILITIES: WINOGRAD, FP32, INT8, BIN
+		SUPPORTED_CONFIG_KEYS: CPU_BIND_THREAD, CPU_THREADS_NUM, CPU_THROUGHPUT_STREAMS, DUMP_EXEC_GRAPH_AS_DOT, DYN_BATCH_ENABLED, DYN_BATCH_LIMIT, EXCLUSIVE_ASYNC_REQUESTS, PERF_COUNT, RANGE_FOR_ASYNC_INFER_REQUESTS, RANGE_FOR_STREAMS
+		RANGE_FOR_ASYNC_INFER_REQUESTS: 0, 6, 1
+		RANGE_FOR_STREAMS: 1, 12
+
+	Default values for device configuration keys:
+		CPU_BIND_THREAD: YES
+		CPU_THREADS_NUM: 0
+		CPU_THROUGHPUT_STREAMS: 1
+		DUMP_EXEC_GRAPH_AS_DOT: 
+		DYN_BATCH_ENABLED: NO
+		DYN_BATCH_LIMIT: 0
+		EXCLUSIVE_ASYNC_REQUESTS: NO
+		PERF_COUNT: NO
+		RANGE_FOR_ASYNC_INFER_REQUESTS: 1
+		RANGE_FOR_STREAMS: 6
+```
+## See Also
+* [Using Inference Engine Samples](./docs/IE_DG/Samples_Overview.md)

inference-engine/ie_bridges/python/sample/hello_query_device/hello_query_device.py

@@ -0,0 +1,35 @@
+import sys
+
+from openvino.inference_engine import IECore
+
+
+def param_to_string(metric):
+    if isinstance(metric, (list, tuple)):
+        return ", ".join([str(val) for val in metric])
+    elif isinstance(metric, dict):
+        str_param_repr = ""
+        for k, v in metric.items():
+            str_param_repr += "{}: {}\n".format(k, v)
+        return str_param_repr
+    else:
+        return str(metric)
+
+
+def main():
+    ie = IECore()
+    print("Available devices:")
+    for device in ie.available_devices:
+        print("\tDevice: {}".format(device))
+        print("\tMetrics:")
+        for metric in ie.get_metric(device, "SUPPORTED_METRICS"):
+            metric_val = ie.get_metric(device, metric)
+            print("\t\t{}: {}".format(metric, param_to_string(metric_val)))
+
+        print("\n\tDefault values for device configuration keys:")
+        for cfg in ie.get_metric(device, "SUPPORTED_CONFIG_KEYS"):
+            cfg_val = ie.get_config(device, cfg)
+            print("\t\t{}: {}".format(cfg, param_to_string(cfg_val)))
+
+
+if __name__ == '__main__':
+    sys.exit(main() or 0)

inference-engine/ie_bridges/python/sample/style_transfer_sample/README.md

@@ -7,7 +7,7 @@ inference of style transfer models.
 
 ## How It Works
 
-> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Specify Input Shapes** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
+> **NOTE**: By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model Using General Conversion Parameters](./docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md).
 
 ## Running
 
@@ -18,11 +18,11 @@ python3 style_transfer_sample.py --help
 The command yields the following usage message:
 ```
 usage: style_transfer_sample.py [-h] -m MODEL -i INPUT [INPUT ...]
-                                [-l CPU_EXTENSION] [-pp PLUGIN_DIR]
-                                [-d DEVICE] [-nt NUMBER_TOP] [-ni NUMBER_ITER]
+                                [-l CPU_EXTENSION] [-d DEVICE] 
+                                [-nt NUMBER_TOP] [-ni NUMBER_ITER]
                                 [--mean_val_r MEAN_VAL_R]
                                 [--mean_val_g MEAN_VAL_G]
-                                [--mean_val_b MEAN_VAL_B] [-pc]
+                                [--mean_val_b MEAN_VAL_B]
 
 Options:
   -h, --help            Show this help message and exit.
@@ -34,16 +34,12 @@ Options:
                         Optional. Required for CPU custom layers. Absolute
                         MKLDNN (CPU)-targeted custom layers. Absolute path to
                         a shared library with the kernels implementations
-  -pp PLUGIN_DIR, --plugin_dir PLUGIN_DIR
-                        Path to a plugin folder
   -d DEVICE, --device DEVICE
                         Specify the target device to infer on; CPU, GPU, FPGA,
                         HDDL or MYRIAD is acceptable. Sample will look for a
                         suitable plugin for device specified. Default value is CPU
   -nt NUMBER_TOP, --number_top NUMBER_TOP
                         Number of top results
-  -ni NUMBER_ITER, --number_iter NUMBER_ITER
-                        Number of inference iterations
   --mean_val_r MEAN_VAL_R, -mean_val_r MEAN_VAL_R
                         Mean value of red chanel for mean value subtraction in
                         postprocessing
@@ -53,13 +49,11 @@ Options:
   --mean_val_b MEAN_VAL_B, -mean_val_b MEAN_VAL_B
                         Mean value of blue chanel for mean value subtraction
                         in postprocessing
-  -pc, --perf_counts    Report performance counters
-
 ```
 
 Running the application with the empty list of options yields the usage message given above and an error message.
 
-To perform inference on an image using a trained model of NST network on Intel® CPUs, use the following command:
+To perform inference of an image using a trained model of NST network on Intel® CPUs, use the following command:
 ```
     python3 style_transfer_sample.py -i <path_to_image>/cat.bmp -m <path_to_model>/1_decoder_FP32.xml
 ```

inference-engine/ie_bridges/python/sample/style_transfer_sample/style_transfer_sample.py

@@ -22,7 +22,7 @@ import cv2
 import numpy as np
 import logging as log
 from time import time
-from openvino.inference_engine import IENetwork, IEPlugin
+from openvino.inference_engine import IENetwork, IECore
 
 
 def build_argparser():
@@ -36,13 +36,11 @@ def build_argparser():
                       help="Optional. Required for CPU custom layers. "
                            "Absolute MKLDNN (CPU)-targeted custom layers. Absolute path to a shared library with the "
                            "kernels implementations", type=str, default=None)
-    args.add_argument("-pp", "--plugin_dir", help="Path to a plugin folder", type=str, default=None)
     args.add_argument("-d", "--device",
                       help="Specify the target device to infer on; CPU, GPU, FPGA, HDDL or MYRIAD is acceptable. Sample "
                            "will look for a suitable plugin for device specified. Default value is CPU", default="CPU",
                       type=str)
     args.add_argument("-nt", "--number_top", help="Number of top results", default=10, type=int)
-    args.add_argument("-ni", "--number_iter", help="Number of inference iterations", default=1, type=int)
     args.add_argument("--mean_val_r", "-mean_val_r",
                       help="Mean value of red chanel for mean value subtraction in postprocessing ", default=0,
                       type=float)
@@ -52,8 +50,6 @@ def build_argparser():
     args.add_argument("--mean_val_b", "-mean_val_b",
                       help="Mean value of blue chanel for mean value subtraction in postprocessing ", default=0,
                       type=float)
-    args.add_argument("-pc", "--perf_counts", help="Report performance counters", default=False, action="store_true")
-
     return parser
 
 
@@ -64,19 +60,20 @@ def main():
     model_bin = os.path.splitext(model_xml)[0] + ".bin"
 
     # Plugin initialization for specified device and load extensions library if specified
-    plugin = IEPlugin(device=args.device, plugin_dirs=args.plugin_dir)
+    log.info("Creating Inference Engine")
+    ie = IECore()
     if args.cpu_extension and 'CPU' in args.device:
-        plugin.add_cpu_extension(args.cpu_extension)
+        ie.add_extension(args.cpu_extension, "CPU")
     # Read IR
     log.info("Loading network files:\n\t{}\n\t{}".format(model_xml, model_bin))
     net = IENetwork(model=model_xml, weights=model_bin)
 
-    if plugin.device == "CPU":
-        supported_layers = plugin.get_supported_layers(net)
+    if "CPU" in args.device:
+        supported_layers = ie.query_network(net, "CPU")
         not_supported_layers = [l for l in net.layers.keys() if l not in supported_layers]
         if len(not_supported_layers) != 0:
             log.error("Following layers are not supported by the plugin for specified device {}:\n {}".
-                      format(plugin.device, ', '.join(not_supported_layers)))
+                      format(args.device, ', '.join(not_supported_layers)))
             log.error("Please try to specify cpu extensions library path in sample's command line parameters using -l "
                       "or --cpu_extension command line argument")
             sys.exit(1)
@@ -103,23 +100,12 @@ def main():
 
     # Loading model to the plugin
     log.info("Loading model to the plugin")
-    exec_net = plugin.load(network=net)
+    exec_net = ie.load_network(network=net, device_name=args.device)
 
     # Start sync inference
-    log.info("Starting inference ({} iterations)".format(args.number_iter))
-    infer_time = []
-    for i in range(args.number_iter):
-        t0 = time()
-        res = exec_net.infer(inputs={input_blob: images})
-        infer_time.append((time() - t0) * 1000)
-    log.info("Average running time of one iteration: {} ms".format(np.average(np.asarray(infer_time))))
-    if args.perf_counts:
-        perf_counts = exec_net.requests[0].get_perf_counts()
-        log.info("Performance counters:")
-        print("{:<70} {:<15} {:<15} {:<15} {:<10}".format('name', 'layer_type', 'exet_type', 'status', 'real_time, us'))
-        for layer, stats in perf_counts.items():
-            print("{:<70} {:<15} {:<15} {:<15} {:<10}".format(layer, stats['layer_type'], stats['exec_type'],
-                                                              stats['status'], stats['real_time']))
+    log.info("Starting inference")
+    res = exec_net.infer(inputs={input_blob: images})
+
     # Processing output blob
     log.info("Processing output blob")
     res = res[out_blob]
@@ -135,6 +121,7 @@ def main():
         out_img = os.path.join(os.path.dirname(__file__), "out_{}.bmp".format(batch))
         cv2.imwrite(out_img, data)
         log.info("Result image was saved to {}".format(out_img))
+    log.info("This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool\n")
 
 
 if __name__ == '__main__':

inference-engine/ie_bridges/python/setup.py

@@ -1,200 +0,0 @@
-import subprocess
-from pathlib import Path
-import platform
-import sys
-from itertools import chain
-from distutils.command.build_py import build_py as _build_py
-from distutils.command.clean import clean as _clean
-
-import shutil
-from setuptools import setup, Extension, find_packages
-from setuptools.command.build_ext import build_ext as _build_ext
-from setuptools.command.install import install as _install
-
-IS_WINDOWS = (platform.system() == 'Windows')
-IS_DARWIN = (platform.system() == 'Darwin')
-IS_LINUX = (platform.system() == 'Linux')
-
-REQUIREMENTS_FILE = 'requirements.txt'
-PACKAGE_NAME = 'inference_engine'
-
-PACKAGE = Path(PACKAGE_NAME)
-C_LIB_NAME = '{}._C'.format(PACKAGE_NAME)
-
-_build_cmd = ['cmake', '--build', '.']
-
-INFERENCE_ENGINE_DIR = None
-BUNDLE_INFERENCE_ENGINE = False
-
-
-def parse_command_line_options(cls):
-    """Propagates command line options to sub-commands.
-    Allows to run install command with build_ext options"""
-
-    base_user_options = getattr(cls, 'user_options', [])
-    base_boolean_options = getattr(cls, 'boolean_options', [])
-    base_run = cls.run
-    base_init_options = cls.initialize_options
-
-    cls.user_options = base_user_options + [
-        ('copy-ie-libs', None, 'Copy Inference Engine Libraries to package directory'),
-        ('inference-engine-dir=', None, 'Path to Inference Engine directory')
-    ]
-
-    cls.boolean_options = base_boolean_options + [
-        'copy-ie-libs'
-    ]
-
-    def initialize_options(self):
-        self.copy_ie_libs = False
-        self.inference_engine_dir = None
-        base_init_options(self)
-
-    def run(self):
-        global INFERENCE_ENGINE_DIR
-        global BUNDLE_INFERENCE_ENGINE
-
-        if self.copy_ie_libs:
-            BUNDLE_INFERENCE_ENGINE = True
-
-        if self.inference_engine_dir:
-            INFERENCE_ENGINE_DIR = self.inference_engine_dir
-
-        base_run(self)
-
-    cls.initialize_options = initialize_options
-    cls.run = run
-    return cls
-
-
-@parse_command_line_options
-class install(_install):
-    pass
-
-
-@parse_command_line_options
-class build_py(_build_py):
-    pass
-
-
-@parse_command_line_options
-class build_ext(_build_ext):
-    def run(self):
-        if not self.extensions:
-            return
-
-        for i, ext in enumerate(self.extensions):
-            if ext.name == C_LIB_NAME:
-                self._build_cmake()
-                self.extensions.pop(i)
-                break
-
-        super().run()
-
-    def _build_cmake(self):
-        print("Building C++ extension")
-        if Path.cwd().joinpath("Makefile").is_file():
-            # in build directory, run make only
-            subprocess.call(_build_cmd)
-        else:
-            # compile extension library and
-            self.build_cmake_lib()
-        print("Built C++ extension")
-
-    def build_cmake_lib(self):
-        def save_call(*args, error_msg=None, **kwargs):
-            if subprocess.call(*args, **kwargs) != 0:
-                if error_msg:
-                    print(error_msg)
-                shutil.rmtree(tmp_build_dir.as_posix(), ignore_errors=True)
-                sys.exit(1)
-
-        tmp_build_dir = Path("tmp_build")
-        destination = Path(self.build_lib) / PACKAGE_NAME if not self.inplace else Path(PACKAGE_NAME)
-        tmp_build_dir.mkdir(exist_ok=False)
-
-        _python_executable_opt = ['-DPYTHON_EXECUTABLE={}'.format(sys.executable)]
-        _build_type_opt = ['-DCMAKE_BUILD_TYPE=Release']
-        _generator_opt = ['-G', 'NMake Makefiles' if IS_WINDOWS else "Unix Makefiles"]
-
-        _optional = []
-        if BUNDLE_INFERENCE_ENGINE:
-            _optional.append('-DCOPY_IE_LIBS=ON')
-
-        if INFERENCE_ENGINE_DIR:
-            _optional.append('-DInferenceEngine_DIR={}'.format(INFERENCE_ENGINE_DIR))
-
-        _cmake_cmd = list(chain(['cmake'], _generator_opt, _build_type_opt, _python_executable_opt, _optional, ['..']))
-
-        save_call(_cmake_cmd, cwd=tmp_build_dir.as_posix(), error_msg="Cmake generator failed")
-        save_call(_build_cmd, cwd=tmp_build_dir.as_posix(), error_msg="Build command failed")
-
-        build_ext.copy_compiled_libs(tmp_build_dir / PACKAGE_NAME, destination)
-        shutil.rmtree(tmp_build_dir.as_posix(), ignore_errors=False)
-
-    @staticmethod
-    def copy_compiled_libs(source_dir, destination):
-        extensions = ['so', 'dll', 'pyd']
-        for path in chain.from_iterable(source_dir.glob("*.%s" % ext) for ext in extensions):
-            shutil.copy(path.as_posix(), destination.as_posix())
-
-
-class clean(_clean):
-    def run(self):
-        shutil.rmtree("tmp_build", ignore_errors=True)
-        extensions = ['so', 'dll', 'pyd']
-        for path in chain.from_iterable(PACKAGE.glob("*.%s" % ext) for ext in extensions):
-            path.unlink()
-        super().run()
-
-
-def paths_to_str(paths):
-    return [p.as_posix() for p in paths]
-
-
-with open(REQUIREMENTS_FILE) as reqs:
-    requirements = set(reqs.read().splitlines())
-
-# do not spoil pre-installed opencv (in case it was built from source)
-_opencv_package = "opencv-python"
-try:
-    import cv2
-
-    if _opencv_package in requirements:
-        requirements.remove(_opencv_package)
-except ImportError:
-    requirements.add(_opencv_package)
-
-
-c_sources = [
-    PACKAGE / 'ie_api_impl.cpp',
-    PACKAGE / 'ie_api_impl.hpp',
-
-    PACKAGE / 'ie_api_impl_defs.pxd',
-    PACKAGE / 'ie_api.pyx',
-    PACKAGE / 'ie_api.pxd',
-]
-
-extensions = [
-    Extension(C_LIB_NAME, paths_to_str(c_sources))
-]
-
-cmdclass = {
-    'build_ext': build_ext,
-    'build_py': build_py,
-    'clean': clean,
-    'install': install,
-}
-
-setup(
-    name="src",
-    version='1.0',
-    description='Python inference for Inference Engine',
-    packages=find_packages(exclude=['tests']),
-    package_data={PACKAGE_NAME: ['*.so', '*.dll', '*dylib*', '*.pyd']},
-    include_package_data=True,
-    ext_modules=extensions,
-    cmdclass=cmdclass,
-    install_requires=list(requirements),
-    zip_safe=False,
-)

inference-engine/ie_bridges/python/src/openvino/inference_engine/CMakeLists.txt

@@ -13,7 +13,7 @@ file(GLOB SOURCE
 set_source_files_properties(${SOURCE} PROPERTIES CYTHON_IS_CXX TRUE
 )
 
-## Compatibility with python 2.7 which has depricated "register" specifier
+## Compatibility with python 2.7 which has deprecated "register" specifier
 if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
     add_definitions("-Wno-register")
 endif()

inference-engine/ie_bridges/python/src/openvino/inference_engine/__init__.py

@@ -1,3 +1,4 @@
 from .ie_api import *
+__all__ = ['IENetwork', "IEPlugin", "IECore", "get_version"]
 __version__ = get_version()
-__all__ = ['IENetwork', "IEPlugin", "IENetReader"]
+

inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pxd

@@ -25,8 +25,9 @@ cdef class InferRequest:
     cpdef async_infer(self, inputs = ?)
     cpdef wait(self, timeout = ?)
     cpdef get_perf_counts(self)
+    cdef void user_callback(self, int status) with gil
     cdef public:
-        _inputs_list, _outputs_list
+        _inputs_list, _outputs_list, _py_callback, _py_data, _py_callback_used, _py_callback_called
 
 cdef class IENetwork:
     cdef C.IENetwork impl
@@ -34,8 +35,9 @@ cdef class IENetwork:
 cdef class ExecutableNetwork:
     cdef unique_ptr[C.IEExecNetwork] impl
     cdef C.IEPlugin plugin_impl
+    cdef C.IECore ie_core_impl
     cdef public:
-        _requests, inputs, outputs
+        _requests, _infer_requests, inputs, outputs
 
 cdef class IEPlugin:
     cdef C.IEPlugin impl
@@ -56,3 +58,7 @@ cdef class OutputInfo:
 
 cdef class LayersStatsMap(dict):
     cdef C.IENetwork net_impl
+
+cdef class IECore:
+    cdef C.IECore impl
+    cpdef ExecutableNetwork load_network(self, IENetwork network, str device_name, config = ?, int num_requests = ?)

inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx

@@ -7,12 +7,16 @@ from libcpp.vector cimport vector
 from libcpp.pair cimport pair
 from libcpp.map cimport map
 from libcpp.memory cimport unique_ptr
-from libc.stdint cimport int64_t
+from libc.stdlib cimport malloc, free
+from libc.stdint cimport int64_t, uint8_t
+from libc.string cimport memcpy, strcpy
 import os
 import numpy as np
 from copy import deepcopy
 import warnings
+from collections import OrderedDict, namedtuple
 from collections import OrderedDict
+import threading
 
 cdef extern from "<utility>" namespace "std" nogil:
     cdef unique_ptr[C.IEExecNetwork] move(unique_ptr[C.IEExecNetwork])
@@ -31,13 +35,103 @@ cdef dict_to_c_map(py_dict):
         c_map[k.encode()] = v.encode()
     return c_map
 
-supported_precisions = ["FP32", "FP16", "Q78", "I32", "I16", "I8", "U32", "U16"]
+cdef c_map_to_dict(map[string, string] c_map):
+    py_dict = {}
+    for v in c_map:
+        py_dict[v.first.decode()] = v.second.decode()
+    return py_dict
+
+supported_precisions = ["FP32", "FP16", "Q78", "I32", "I16", "I8", "U32", "U16", "U8"]
+
 supported_layouts = ["NCHW", "NHWC", "OIHW", "C", "CHW", "HW", "NC", "CN", "BLOCKED", "NCDHW"]
 known_plugins = ['CPU', 'GPU', 'FPGA', 'MYRIAD', 'HETERO', 'HDDL']
 
+ctypedef enum StatusCode:
+    OK = 0
+    GENERAL_ERROR = -1
+    NOT_IMPLEMENTED = -2
+    NETWORK_NOT_LOADED = -3
+    PARAMETER_MISMATCH = -4
+    NOT_FOUND = -5
+    OUT_OF_BOUNDS = -6
+    UNEXPECTED = -7
+    REQUEST_BUSY = -8
+    RESULT_NOT_READY = -9
+    NOT_ALLOCATED = -10
+    INFER_NOT_STARTED = -11
+    NETWORK_NOT_READ = -12
+
 def get_version():
     return C.get_version().decode()
 
+cdef  class IECore:
+    def __cinit__(self, xml_config_file: str = ""):
+        self.impl = C.IECore(xml_config_file.encode())
+
+    def get_versions(self, device_name: str):
+        cdef  map[string, C.Version] versions_
+        versions_ = self.impl.getVersions(device_name.encode())
+        versions = {}
+        for v in versions_:
+            device = v.first.decode()
+            ver = v.second
+            versions[device] = namedtuple("Versions", ["major", "minor", "build_number", "description"])
+            versions[device].build_number = ver.buildNumber.decode()
+            versions[device].description = ver.description.decode()
+            versions[device].minor = ver.apiVersion.minor
+            versions[device].major = ver.apiVersion.major
+        return versions
+
+    cpdef ExecutableNetwork load_network(self, IENetwork network, str device_name, config=None, int num_requests=1):
+        cdef ExecutableNetwork exec_net = ExecutableNetwork()
+        cdef map[string, string] c_config
+
+        if config:
+            c_config = dict_to_c_map(config)
+        exec_net.ie_core_impl = self.impl
+        exec_net.impl = move(self.impl.loadNetwork(network.impl, device_name.encode(), c_config, num_requests))
+        exec_net.inputs = network.inputs.keys()
+        exec_net.outputs = list(network.outputs.keys())
+        return exec_net
+
+    def query_network(self, IENetwork network, str device_name, config=None):
+        cdef map[string, string] c_config
+        if config:
+            c_config = dict_to_c_map(config)
+        res = self.impl.queryNetwork(network.impl, device_name.encode(), c_config)
+        return c_map_to_dict(res)
+
+    def set_config(self, config: dict, device_name: str):
+        cdef map[string, string] c_config = dict_to_c_map(config)
+        self.impl.setConfig(c_config, device_name.encode())
+
+    def register_plugin(self, plugin_name: str, device_name: str = ""):
+        self.impl.registerPlugin(plugin_name.encode(), device_name.encode())
+
+    def register_plugins(self, xml_config_file: str):
+        self.impl.registerPlugins(xml_config_file.encode())
+
+    def unregister_plugin(self, device_name: str):
+        self.impl.unregisterPlugin(device_name.encode())
+
+    def add_extension(self, extension_path: str, device_name: str):
+        self.impl.addExtension(extension_path.encode(), device_name.encode())
+
+    def get_metric(self, device_name: str, metric_name: str):
+        return self.impl.getMetric(device_name.encode(), metric_name.encode())
+
+    def get_config(self, device_name: str, config_name: str):
+        return self.impl.getConfig(device_name.encode(), config_name.encode())
+
+
+    @property
+    def available_devices(self):
+        cdef vector[string] c_devices = self.impl.getAvailableDevices()
+        return [d.decode() for d in c_devices]
+
+    # TODO: Add import network functionality
+    # TODO: Extend API for query config and attributes when it will be merged in C++ API
+
 cdef class IENetLayer:
     @property
     def name(self):
@@ -137,6 +231,7 @@ cdef class OutputInfo:
 
 cdef class ExecutableNetwork:
     def __init__(self):
+        self._infer_requests = []
         self._requests = []
         self.inputs = []
         self.outputs = []
@@ -155,19 +250,53 @@ cdef class ExecutableNetwork:
 
     @property
     def requests(self):
-        requests = []
-        for i in range(deref(self.impl).infer_requests.size()):
-            infer_request = InferRequest()
-            infer_request.impl = &(deref(self.impl).infer_requests[i])
-            infer_request._inputs_list = self.inputs
-            infer_request._outputs_list = self.outputs
-            requests.append(infer_request)
-        return requests
+        if (len(self._infer_requests) == 0):
+            for i in range(deref(self.impl).infer_requests.size()):
+                infer_request = InferRequest()
+                infer_request.impl = &(deref(self.impl).infer_requests[i])
+                self._infer_requests.append(infer_request)
+
+        if (len(self._infer_requests) != deref(self.impl).infer_requests.size()):
+            raise Exception("Mismatch of infer requests number!")
+
+        for i in range(len(self._infer_requests)):
+            self._infer_requests[i]._inputs_list = self.inputs
+            self._infer_requests[i]._outputs_list = self.outputs
+
+        return self._infer_requests
+
+    def get_exec_graph_info(self):
+        ie_network = IENetwork()
+        ie_network.impl = deref(self.impl).GetExecGraphInfo()
+        return ie_network
+
+    def get_metric(self, metric_name: str):
+        return deref(self.impl).getMetric(metric_name.encode())
+
+    def get_config(self, config_name: str):
+        return deref(self.impl).getConfig(config_name.encode())
+
+ctypedef extern void (*cb_type)(void*, int) with gil
 
 cdef class InferRequest:
     def __init__(self):
         self._inputs_list = []
         self._outputs_list = []
+        self._py_callback = lambda *args, **kwargs: None
+        self._py_callback_used = False
+        self._py_callback_called = threading.Event()
+        self._py_data = None
+
+    cdef void user_callback(self, int status) with gil:
+        if self._py_callback:
+            self._py_callback(status, self._py_data)
+            self._py_callback_called.set()
+
+    def set_completion_callback(self, py_callback, py_data = None):
+        self._py_callback = py_callback
+        self._py_data = py_data
+        self._py_callback_used = True
+        deref(self.impl).setCyCallback(<cb_type>self.user_callback, <void *>self)
 
     cpdef BlobBuffer _get_blob_buffer(self, const string & blob_name):
         cdef BlobBuffer buffer = BlobBuffer()
@@ -185,13 +314,19 @@ cdef class InferRequest:
     cpdef async_infer(self, inputs=None):
         if inputs is not None:
             self._fill_inputs(inputs)
-
+        self._py_callback_called.clear()
         deref(self.impl).infer_async()
 
     cpdef wait(self, timeout=None):
-        if timeout is None:
-            timeout = -1
-        return deref(self.impl).wait(<int64_t> timeout)
+        if self._py_callback_used:
+            while not self._py_callback_called.is_set():
+                if not self._py_callback_called.wait(timeout):
+                    return StatusCode.REQUEST_BUSY
+            return StatusCode.OK
+        else:
+            if timeout is None:
+                timeout = -1
+            return deref(self.impl).wait(<int64_t> timeout)
 
     cpdef get_perf_counts(self):
         cdef map[string, C.ProfileInfo] c_profile = deref(self.impl).getPerformanceCounts()
@@ -258,19 +393,30 @@ cdef class LayersStatsMap(dict):
         self.net_impl.setStats(c_stats_map)
 
 cdef class IENetwork:
-    def __cinit__(self, model: str="", weights: str=""):
+    def __cinit__(self, model: [str, bytes] ="", weights: [str, bytes] ="", init_from_buffer: bool=False,
+                  ngraph_compatibility: bool = False):
+        cdef char* xml_buffer = <char*>malloc(len(model))
+        cdef uint8_t* bin_buffer = <uint8_t *>malloc(len(weights))
         cdef string model_
         cdef string weights_
-        if model and weights:
-            if not os.path.isfile(model):
-                raise Exception("Path to the model {} doesn't exists or it's a directory".format(model))
-            if not os.path.isfile(weights):
-                raise Exception("Path to the weights {} doesn't exists or it's a directory".format(weights))
-            model_ = model.encode()
-            weights_ = weights.encode()
-            self.impl = C.IENetwork(model_, weights_)
-        else:
+        if init_from_buffer:
+            strcpy(xml_buffer, model)
+            memcpy(bin_buffer, <uint8_t *>weights, len(weights))
             self.impl = C.IENetwork()
+            self.impl.load_from_buffer(xml_buffer, len(model), bin_buffer, len(weights))
+        else:
+            if model and weights:
+                if not os.path.isfile(model):
+                    raise Exception("Path to the model {} doesn't exists or it's a directory".format(model))
+                if not os.path.isfile(weights):
+                    raise Exception("Path to the weights {} doesn't exists or it's a directory".format(weights))
+                model_ = model.encode()
+                weights_ = weights.encode()
+                self.impl = C.IENetwork(model_, weights_, ngraph_compatibility)
+            else:
+                self.impl = C.IENetwork()
+        free(xml_buffer)
+
     @property
     def name(self):
         name = bytes(self.impl.name)
@@ -302,6 +448,10 @@ cdef class IENetwork:
     def batch_size(self):
         return self.impl.batch_size
 
+    @property
+    def precision(self):
+        return self.impl.precision.decode()
+
     @batch_size.setter
     def batch_size(self, batch: int):
         if batch <= 0:
@@ -343,20 +493,26 @@ cdef class IENetwork:
         cdef IENetwork net = IENetwork(model, weights)
         return net
 
-    # TODO: Use enum with precision type instead of srting parameter when python2 support will not be required.
+        # TODO: Use enum with precision type instead of srting parameter when python2 support will not be required.
     def add_outputs(self, outputs, precision="FP32"):
         if precision.upper() not in supported_precisions:
             raise AttributeError(
                 "Unsupported precision {}! List of supported precisions: {}".format(precision, supported_precisions))
         if not isinstance(outputs, list):
             outputs = [outputs]
-        cdef vector[string] _outputs
-        for l in outputs:
-            _outputs.push_back(l.encode())
-        self.impl.addOutputs(_outputs, precision.upper().encode())
+        for i, l in enumerate(outputs):
+            if isinstance(l, str):
+                self.impl.addOutput(l.encode(), 0, precision.upper().encode())
+            elif isinstance(l, tuple) and len(l) == 2:
+                self.impl.addOutput(l[0].encode(), l[1], precision.upper().encode())
+            else:
+                raise TypeError("Incorrect type {type} for layer to add at index {ind}. "
+                                "Expected string with layer name or tuple with two elements: layer name as "
+                                "first element and port id as second".format(type=type(l), ind=i))
 
-    def serialize(self, path_to_xml, path_to_bin):
+    def serialize(self, path_to_xml, path_to_bin: str = ""):
         self.impl.serialize(path_to_xml.encode(), path_to_bin.encode())
+
     def reshape(self, input_shapes: dict):
         cdef map[string, vector[size_t]] c_input_shapes;
         cdef vector[size_t] c_shape
@@ -364,7 +520,7 @@ cdef class IENetwork:
         for input, shape in input_shapes.items():
             c_shape = []
             if input not in net_inputs:
-                raise AttributeError("Specified {} layer not in network inputs {}! ".format(input, net_inputs))
+                raise AttributeError("Specified '{}' layer not in network inputs '{}'! ".format(input, net_inputs))
             for v in shape:
                 c_shape.push_back(v)
             c_input_shapes[input.encode()] = c_shape
@@ -393,12 +549,11 @@ cdef class IEPlugin:
         self.impl = C.IEPlugin(device_, dirs_)
 
     cpdef ExecutableNetwork load(self, IENetwork network, int num_requests=1, config=None):
-        if num_requests <= 0:
-            raise ValueError(
-                "Incorrect number of requests specified: {}. Expected positive integer number.".format(num_requests))
         cdef ExecutableNetwork exec_net = ExecutableNetwork()
         cdef map[string, string] c_config
-
+        if num_requests < 0:
+            raise ValueError("Incorrect number of requests specified: {}. Expected positive integer number "
+                               "or zero for auto detection".format(num_requests))
         if config:
             for k, v in config.items():
                 c_config[to_std_string(k)] = to_std_string(v)
@@ -438,6 +593,7 @@ cdef class IEPlugin:
             c_config[to_std_string(k)] = to_std_string(v)
         self.impl.setConfig(c_config)
 
+    # TODO: Add export compiled network functionality
 
 cdef class BlobBuffer:
     """Copy-less accessor for Inference Engine Blob"""

inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.cpp

@@ -1,16 +1,6 @@
 // Copyright (C) 2018-2019 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
 //
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//        http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
 
 #include "ie_api_impl.hpp"
 #include "hetero/hetero_plugin_config.hpp"
@@ -45,14 +35,151 @@ std::map<std::string, InferenceEngine::Layout> layout_map = {{"ANY",     Inferen
     }                                               \
 }                                                   \
 
+uint32_t getOptimalNumberOfRequests(const InferenceEngine::IExecutableNetwork::Ptr actual) {
+    try {
+        InferenceEngine::ResponseDesc response;
+        InferenceEngine::Parameter parameter_value;
+        IE_CHECK_CALL(actual->GetMetric(METRIC_KEY(SUPPORTED_METRICS), parameter_value, &response));
+        auto supported_metrics = parameter_value.as<std::vector<std::string>>();
+        std::string key = METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS);
+        if (std::find(supported_metrics.begin(), supported_metrics.end(), key) != supported_metrics.end()) {
+            IE_CHECK_CALL(actual->GetMetric(key, parameter_value, &response));
+            if (parameter_value.is<unsigned int>())
+                return parameter_value.as<unsigned int>();
+            else
+                THROW_IE_EXCEPTION << "Unsupported format for " << key << "!"
+                                  << " Please specify number of infer requests directly!";
+        } else {
+            THROW_IE_EXCEPTION << "Can't load network: " << key << " is not supported!"
+                               << " Please specify number of infer requests directly!";
+        }
+    } catch (const std::exception& ex) {
+        THROW_IE_EXCEPTION << "Can't load network: " << ex.what()
+                           << " Please specify number of infer requests directly!";
+    }
+}
 
-InferenceEnginePython::IENetwork::IENetwork(const std::string &model, const std::string &weights) {
+PyObject* parse_parameter(const InferenceEngine::Parameter & param){
+    // Check for std::string
+    if (param.is<std::string>()){
+        return PyUnicode_FromString(param.as<std::string>().c_str());
+    }
+        // Check for int
+    else if (param.is<int>()) {
+        auto val = param.as<int>();
+        return PyLong_FromLong((long)val);
+    }
+        // Check for float
+    else if (param.is<float>()) {
+        auto val = param.as<float>();
+        return PyFloat_FromDouble((double)val);
+    }
+        // Check for bool
+    else if (param.is<bool>()) {
+        auto val = param.as<bool>();
+        return val ? Py_True : Py_False;
+    }
+        // Check for std::vector<std::string>
+    else if (param.is<std::vector<std::string>>()) {
+        auto val = param.as<std::vector<std::string>>();
+        PyObject *list = PyList_New(0);
+        for (const auto & it : val){
+            PyObject *str_val = PyUnicode_FromString(it.c_str());
+            PyList_Append(list, str_val);
+        }
+        return list;
+    }
+        // Check for std::vector<int>
+    else if (param.is<std::vector<int>>()){
+        auto val = param.as<std::vector<int>>();
+        PyObject *list = PyList_New(0);
+        for (const auto & it : val){
+            PyList_Append(list, PyLong_FromLong(it));
+        }
+        return list;
+    }
+        // Check for std::vector<float>
+    else if (param.is<std::vector<float>>()){
+        auto val = param.as<std::vector<float>>();
+        PyObject *list = PyList_New(0);
+        for (const auto & it : val){
+            PyList_Append(list, PyFloat_FromDouble((double)it));
+        }
+        return list;
+    }
+        // Check for std::tuple<unsigned int, unsigned int>
+    else if (param.is<std::tuple<unsigned int, unsigned int >>()) {
+        auto val = param.as<std::tuple<unsigned int, unsigned int >>();
+        PyObject *tuple = PyTuple_New(2);
+        PyTuple_SetItem(tuple, 0, PyLong_FromUnsignedLong((unsigned long)std::get<0>(val)));
+        PyTuple_SetItem(tuple, 1, PyLong_FromUnsignedLong((unsigned long)std::get<1>(val)));
+        return tuple;
+    }
+        // Check for std::tuple<unsigned int, unsigned int, unsigned int>
+    else if (param.is<std::tuple<unsigned int, unsigned int, unsigned int >>()) {
+        auto val = param.as<std::tuple<unsigned int, unsigned int, unsigned int >>();
+        PyObject *tuple = PyTuple_New(3);
+        PyTuple_SetItem(tuple, 0, PyLong_FromUnsignedLong((unsigned long)std::get<0>(val)));
+        PyTuple_SetItem(tuple, 1, PyLong_FromUnsignedLong((unsigned long)std::get<1>(val)));
+        PyTuple_SetItem(tuple, 2, PyLong_FromUnsignedLong((unsigned long)std::get<2>(val)));
+        return tuple;
+    }
+        // Check for std::map<std::string, std::string>
+    else if (param.is<std::map<std::string, std::string>>()) {
+        auto val = param.as<std::map<std::string, std::string>>();
+        PyObject *dict = PyDict_New();
+        for (const auto &it : val){
+            PyDict_SetItemString(dict, it.first.c_str(), PyUnicode_FromString(it.second.c_str()));
+        }
+        return dict;
+    }
+        // Check for std::map<std::string, int>
+    else if (param.is<std::map<std::string, int>>()) {
+        auto val = param.as<std::map<std::string, int>>();
+        PyObject *dict = PyDict_New();
+        for (const auto &it : val){
+            PyDict_SetItemString(dict, it.first.c_str(), PyLong_FromLong((long)it.second));
+        }
+        return dict;
+    }
+    else {
+        PyErr_SetString(PyExc_TypeError, "Failed to convert parameter to Python representation!");
+        return (PyObject *) NULL;
+    }
+}
+InferenceEnginePython::IENetwork::IENetwork(const std::string &model, const std::string &weights, bool ngraph_compatibility = false) {
+    if (ngraph_compatibility){
+        InferenceEngine::IRReader ir_reader;
+        auto ngraph_function = ir_reader.read(model, weights);
+        actual = InferenceEngine::CNNNetwork(InferenceEngine::convertFunctionToICNNNetwork(ngraph_function));
+    } else {
+        InferenceEngine::CNNNetReader net_reader;
+        net_reader.ReadNetwork(model);
+        net_reader.ReadWeights(weights);
+        actual = net_reader.getNetwork();
+    }
+    name = actual.getName();
+    batch_size = actual.getBatchSize();
+    precision = actual.getPrecision().name();
+}
+
+InferenceEnginePython::IENetwork::IENetwork(const InferenceEngine::CNNNetwork& cnn_network)
+    : actual(cnn_network) {
+    name = actual.getName();
+    batch_size = actual.getBatchSize();
+    precision = actual.getPrecision().name();
+}
+
+void InferenceEnginePython::IENetwork::load_from_buffer(const char *xml, size_t xml_size, uint8_t *bin, size_t bin_size) {
     InferenceEngine::CNNNetReader net_reader;
-    net_reader.ReadNetwork(model);
-    net_reader.ReadWeights(weights);
+    net_reader.ReadNetwork(xml, xml_size);
+    InferenceEngine::TensorDesc tensorDesc(InferenceEngine::Precision::U8, {bin_size}, InferenceEngine::Layout::C);
+    auto weights_blob = InferenceEngine::make_shared_blob<uint8_t>(tensorDesc, bin, bin_size);
+    net_reader.SetWeights(weights_blob);
     name = net_reader.getName();
     actual = net_reader.getNetwork();
     batch_size = actual.getBatchSize();
+    precision = actual.getPrecision().name();
 }
 
 void InferenceEnginePython::IENetwork::serialize(const std::string &path_to_xml, const std::string &path_to_bin) {
@@ -87,7 +214,7 @@ InferenceEnginePython::IENetwork::getLayers() {
             for (auto layer_iter : inputTo) {
                 InferenceEngine::CNNLayerPtr layer_in_data = layer_iter.second;
                 if (!layer_in_data) {
-                    THROW_IE_EXCEPTION << "Layer which takes data " << data->name << " is nullptr";
+                    THROW_IE_EXCEPTION << "Layer which takes data " << data->getName() << " is nullptr";
                 }
                 children.emplace_back(layer_in_data->name);
             }
@@ -150,22 +277,15 @@ const std::map<std::string, InferenceEnginePython::OutputInfo> InferenceEnginePy
 }
 
 void
-InferenceEnginePython::IENetwork::addOutputs(const std::vector<std::string> &out_layers, const std::string &precision) {
-    for (auto &&l : out_layers) {
-        InferenceEngine::OutputsDataMap outputsDataMap = actual.getOutputsInfo();
-        if (outputsDataMap.find(l) != outputsDataMap.end()) {
-            continue;
-        }
-        InferenceEngine::CNNLayerPtr cnnLayer = actual.getLayerByName(l.c_str());
-        std::vector<InferenceEngine::DataPtr> outData = cnnLayer->outData;
-        if (outData.size() != 1) {
-            std::cout << "Layer " << l << " has " << outData.size() << " output blobs and can not be set as output."
-                      << std::endl;
-            continue;
-        }
-        actual.addOutput(l);
-        InferenceEngine::OutputsDataMap outputsDataMapUpd = actual.getOutputsInfo();
-        outputsDataMapUpd[l]->setPrecision(precision_map[precision]);
+InferenceEnginePython::IENetwork::addOutput(const std::string &out_layer, size_t port_id, const std::string &precision) {
+    actual.addOutput(out_layer, port_id);
+    InferenceEngine::OutputsDataMap outputsDataMapUpd = actual.getOutputsInfo();
+    if (outputsDataMapUpd.count(out_layer)) {
+        outputsDataMapUpd[out_layer]->setPrecision(precision_map[precision]);
+    } else if (outputsDataMapUpd.count(out_layer + "." + std::to_string(port_id))){
+        outputsDataMapUpd[out_layer + "." + std::to_string(port_id)]->setPrecision(precision_map[precision]);
+    } else {
+        THROW_IE_EXCEPTION << "Failed to set precision for layer " << out_layer;
     }
 }
 
@@ -192,9 +312,8 @@ const std::map<std::string, std::map<std::string, std::vector<float>>> Inference
     return map;
 }
 
-void
-InferenceEnginePython::IENetwork::setStats(
-        const std::map<std::string, std::map<std::string, std::vector<float>>> &stats) {
+void InferenceEnginePython::IENetwork::setStats(const std::map<std::string, std::map<std::string,
+        std::vector<float>>> &stats) {
     InferenceEngine::ICNNNetworkStats *pstats = nullptr;
     InferenceEngine::ResponseDesc response;
     IE_CHECK_CALL(((InferenceEngine::ICNNNetwork &) actual).getStats(&pstats, &response));
@@ -222,10 +341,11 @@ void InferenceEnginePython::OutputInfo::setPrecision(std::string precision) {
 }
 
 InferenceEnginePython::IEPlugin::IEPlugin(const std::string &device, const std::vector<std::string> &plugin_dirs) {
+    IE_SUPPRESS_DEPRECATED_START
     InferenceEngine::PluginDispatcher dispatcher{plugin_dirs};
     actual = dispatcher.getPluginByDevice(device);
-    const InferenceEngine::Version *pluginVersion;
-    actual->GetVersion(pluginVersion);
+    IE_SUPPRESS_DEPRECATED_END
+    auto pluginVersion = actual.GetVersion();
     version = std::to_string(pluginVersion->apiVersion.major) + ".";
     version += std::to_string(pluginVersion->apiVersion.minor) + ".";
     version += pluginVersion->buildNumber;
@@ -233,17 +353,32 @@ InferenceEnginePython::IEPlugin::IEPlugin(const std::string &device, const std::
 }
 
 void InferenceEnginePython::IEPlugin::setInitialAffinity(const InferenceEnginePython::IENetwork &net) {
-    InferenceEngine::HeteroPluginPtr hetero_plugin(actual);
-    InferenceEngine::ResponseDesc response;
+    InferenceEngine::InferenceEnginePluginPtr hetero_plugin(actual);
+    InferenceEngine::QueryNetworkResult queryRes;
     auto &network = net.actual;
-    IE_CHECK_CALL(hetero_plugin->SetAffinity(network, {}, &response));
+
+    hetero_plugin->QueryNetwork(network, {}, queryRes);
+
+    if (queryRes.rc != InferenceEngine::StatusCode::OK) {
+        THROW_IE_EXCEPTION << queryRes.resp.msg;
+    }
+
+    for (auto && layer : queryRes.supportedLayersMap) {
+        network.getLayerByName(layer.first.c_str())->affinity = layer.second;
+    }
 }
 
 std::set<std::string> InferenceEnginePython::IEPlugin::queryNetwork(const InferenceEnginePython::IENetwork &net) {
     const InferenceEngine::CNNNetwork &network = net.actual;
     InferenceEngine::QueryNetworkResult queryRes;
-    actual->QueryNetwork(network, queryRes);
-    return queryRes.supportedLayers;
+    actual.QueryNetwork(network, {}, queryRes);
+
+    std::set<std::string> supportedLayers;
+    for (auto && layer : queryRes.supportedLayersMap) {
+        supportedLayers.insert(layer.first);
+    }
+
+    return supportedLayers;
 }
 
 
@@ -289,10 +424,9 @@ void InferenceEnginePython::IENetLayer::setPrecision(std::string precision) {
 }
 
 void InferenceEnginePython::IEPlugin::addCpuExtension(const std::string &extension_path) {
-    InferenceEngine::ResponseDesc response;
     auto extension_ptr = InferenceEngine::make_so_pointer<InferenceEngine::IExtension>(extension_path);
     auto extension = std::dynamic_pointer_cast<InferenceEngine::IExtension>(extension_ptr);
-    IE_CHECK_CALL(actual->AddExtension(extension, &response))
+    actual.AddExtension(extension);
 }
 
 std::unique_ptr<InferenceEnginePython::IEExecNetwork>
@@ -302,7 +436,12 @@ InferenceEnginePython::IEPlugin::load(const InferenceEnginePython::IENetwork &ne
     InferenceEngine::ResponseDesc response;
     auto exec_network = InferenceEnginePython::make_unique<InferenceEnginePython::IEExecNetwork>(net.name,
                                                                                                  num_requests);
-    IE_CHECK_CALL(actual->LoadNetwork(exec_network->actual, net.actual, config, &response))
+    exec_network->actual = actual.LoadNetwork(net.actual, config);
+
+    if (0 == num_requests) {
+        num_requests = getOptimalNumberOfRequests(exec_network->actual);
+        exec_network->infer_requests.resize(num_requests);
+    }
 
     for (size_t i = 0; i < num_requests; ++i) {
         InferRequestWrap &infer_request = exec_network->infer_requests[i];
@@ -313,8 +452,7 @@ InferenceEnginePython::IEPlugin::load(const InferenceEnginePython::IENetwork &ne
 }
 
 void InferenceEnginePython::IEPlugin::setConfig(const std::map<std::string, std::string> &config) {
-    InferenceEngine::ResponseDesc response;
-    IE_CHECK_CALL(actual->SetConfig(config, &response))
+    actual.SetConfig(config);
 }
 
 InferenceEnginePython::IEExecNetwork::IEExecNetwork(const std::string &name, size_t num_requests) :
@@ -326,9 +464,29 @@ void InferenceEnginePython::IEExecNetwork::infer() {
     request.infer();
 }
 
+InferenceEnginePython::IENetwork InferenceEnginePython::IEExecNetwork::GetExecGraphInfo() {
+    InferenceEngine::ResponseDesc response;
+    InferenceEngine::ICNNNetwork::Ptr graph;
+    IE_CHECK_CALL(actual->GetExecGraphInfo(graph, &response));
+    return IENetwork(InferenceEngine::CNNNetwork(graph));
+}
 
-void InferenceEnginePython::InferRequestWrap::getBlobPtr(const std::string &blob_name, InferenceEngine::Blob::Ptr &blob_ptr)
-{
+PyObject* InferenceEnginePython::IEExecNetwork::getMetric(const std::string &metric_name) {
+    InferenceEngine::Parameter parameter;
+    InferenceEngine::ResponseDesc response;
+    IE_CHECK_CALL(actual->GetMetric(metric_name, parameter, &response));
+    return parse_parameter(parameter);
+}
+
+PyObject* InferenceEnginePython::IEExecNetwork::getConfig(const std::string &metric_name) {
+    InferenceEngine::Parameter parameter;
+    InferenceEngine::ResponseDesc response;
+    IE_CHECK_CALL(actual->GetMetric(metric_name, parameter, &response));
+    return parse_parameter(parameter);
+}
+
+void InferenceEnginePython::InferRequestWrap::getBlobPtr(const std::string &blob_name,
+                                                         InferenceEngine::Blob::Ptr &blob_ptr) {
     InferenceEngine::ResponseDesc response;
     IE_CHECK_CALL(request_ptr->GetBlob(blob_name.c_str(), blob_ptr, &response));
 }
@@ -339,16 +497,24 @@ void InferenceEnginePython::InferRequestWrap::setBatch(int size) {
     IE_CHECK_CALL(request_ptr->SetBatch(size, &response));
 }
 
-void latency_callback(InferenceEngine::IInferRequest::Ptr request, InferenceEngine::StatusCode code){
+void latency_callback(InferenceEngine::IInferRequest::Ptr request, InferenceEngine::StatusCode code) {
     if (code != InferenceEngine::StatusCode::OK) {
         THROW_IE_EXCEPTION << "Async Infer Request failed with status code " << code;
     }
     InferenceEnginePython::InferRequestWrap *requestWrap;
     InferenceEngine::ResponseDesc dsc;
-    request->GetUserData(reinterpret_cast<void**>(&requestWrap), &dsc);
+    request->GetUserData(reinterpret_cast<void **>(&requestWrap), &dsc);
     auto end_time = Time::now();
     auto execTime = std::chrono::duration_cast<ns>(end_time - requestWrap->start_time);
     requestWrap->exec_time = static_cast<double>(execTime.count()) * 0.000001;
+    if (requestWrap->user_callback) {
+        requestWrap->user_callback(requestWrap->user_data, code);
+    }
+}
+
+void InferenceEnginePython::InferRequestWrap::setCyCallback(cy_callback callback, void *data) {
+    user_callback = callback;
+    user_data = data;
 }
 
 void InferenceEnginePython::InferRequestWrap::infer() {
@@ -413,3 +579,77 @@ std::string InferenceEnginePython::get_version() {
     version_str += version->buildNumber;
     return version_str;
 }
+
+
+InferenceEnginePython::IECore::IECore(const std::string & xmlConfigFile) {
+    actual = InferenceEngine::Core(xmlConfigFile);
+}
+
+std::map<std::string, InferenceEngine::Version> InferenceEnginePython::IECore::getVersions(const std::string &deviceName) {
+    return actual.GetVersions(deviceName);
+}
+
+std::unique_ptr<InferenceEnginePython::IEExecNetwork> InferenceEnginePython::IECore::loadNetwork(IENetwork network,
+        const std::string & deviceName, const std::map<std::string, std::string> & config, int num_requests){
+
+    InferenceEngine::ResponseDesc response;
+    auto exec_network = InferenceEnginePython::make_unique<InferenceEnginePython::IEExecNetwork>(network.name,
+                                                                                                 num_requests);
+    exec_network->actual = actual.LoadNetwork(network.actual, deviceName, config);
+
+    if (0 == num_requests) {
+        num_requests = getOptimalNumberOfRequests(exec_network->actual);
+        exec_network->infer_requests.resize(num_requests);
+    }
+
+    for (size_t i = 0; i < num_requests; ++i) {
+        InferRequestWrap &infer_request = exec_network->infer_requests[i];
+        IE_CHECK_CALL(exec_network->actual->CreateInferRequest(infer_request.request_ptr, &response))
+    }
+
+    return exec_network;
+}
+
+std::map<std::string, std::string> InferenceEnginePython::IECore::queryNetwork(InferenceEnginePython::IENetwork network,
+                                                                  const std::string &deviceName,
+                                                                  const std::map<std::string, std::string> &config) {
+    auto res = actual.QueryNetwork(network.actual, deviceName, config);
+    return res.supportedLayersMap;
+}
+
+void InferenceEnginePython::IECore::setConfig(const std::map<std::string, std::string> &config,
+                                              const std::string &deviceName) {
+    actual.SetConfig(config, deviceName);
+}
+
+void InferenceEnginePython::IECore::registerPlugin(const std::string & pluginName, const std::string &deviceName) {
+    actual.RegisterPlugin(pluginName, deviceName);
+}
+
+void InferenceEnginePython::IECore::unregisterPlugin(const std::string & deviceName){
+    actual.UnregisterPlugin(deviceName);
+}
+
+void InferenceEnginePython::IECore::registerPlugins(const std::string & xmlConfigFile){
+    actual.RegisterPlugins(xmlConfigFile);
+}
+
+void InferenceEnginePython::IECore::addExtension(const std::string & ext_lib_path, const std::string &deviceName) {
+    auto extension_ptr = InferenceEngine::make_so_pointer<InferenceEngine::IExtension>(ext_lib_path);
+    auto extension = std::dynamic_pointer_cast<InferenceEngine::IExtension>(extension_ptr);
+    actual.AddExtension(extension, deviceName);
+}
+
+std::vector<std::string> InferenceEnginePython::IECore::getAvailableDevices() {
+    return actual.GetAvailableDevices();
+}
+
+PyObject* InferenceEnginePython::IECore::getMetric(const std::string &deviceName, const std::string &name) {
+    InferenceEngine::Parameter param = actual.GetMetric(deviceName, name);
+    return parse_parameter(param);
+}
+
+PyObject* InferenceEnginePython::IECore::getConfig(const std::string &deviceName, const std::string &name) {
+    InferenceEngine::Parameter param = actual.GetConfig(deviceName, name);
+    return parse_parameter(param);
+}

inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl.hpp

@@ -1,35 +1,26 @@
 // Copyright (C) 2018-2019 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
 //
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-//        http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
 
 #pragma once
 
-#include <ie_extension.h>
-#include <iterator>
+#include "Python.h"
 
+#include <iterator>
 #include <string>
 #include <utility>
 #include <map>
 #include <vector>
 #include <set>
-
-
 #include <iostream>
 #include <algorithm>
-
 #include <sstream>
 #include <chrono>
+
+#include <ie_extension.h>
 #include "inference_engine.hpp"
+#include "../../../../../src/inference_engine/ie_ir_reader.hpp"
+
 
 typedef std::chrono::high_resolution_clock Time;
 typedef std::chrono::nanoseconds ns;
@@ -90,10 +81,11 @@ struct IENetwork {
     InferenceEngine::CNNNetwork actual;
     std::string name;
     std::size_t batch_size;
+    std::string precision;
 
     void setBatch(const size_t size);
 
-    void addOutputs(const std::vector<std::string> &out_layers, const std::string &precision);
+    void addOutput(const std::string &out_layer, size_t port_id, const std::string &precision);
 
     const std::vector<std::pair<std::string, InferenceEnginePython::IENetLayer>> getLayers();
 
@@ -109,21 +101,33 @@ struct IENetwork {
 
     const std::map<std::string, std::map<std::string, std::vector<float>>> getStats();
 
-    IENetwork(const std::string &model, const std::string &weights);
+    void load_from_buffer(const char* xml, size_t xml_size, uint8_t* bin, size_t bin_size);
+
+    IENetwork(const std::string &model, const std::string &weights, bool ngraph_compatibility);
+
+    IENetwork(const InferenceEngine::CNNNetwork& cnn_network);
 
     IENetwork() = default;
 };
 
 struct InferRequestWrap {
+    using cy_callback = void (*)(void*, int);
+
     InferenceEngine::IInferRequest::Ptr request_ptr;
     Time::time_point start_time;
     double exec_time;
+    cy_callback user_callback;
+    void *user_data;
+    int status;
+
     void infer();
 
     void infer_async();
 
     int  wait(int64_t timeout);
 
+    void setCyCallback(cy_callback callback, void *data);
+
     void getBlobPtr(const std::string &blob_name, InferenceEngine::Blob::Ptr &blob_ptr);
 
     void setBatch(int size);
@@ -139,7 +143,12 @@ struct IEExecNetwork {
 
     IEExecNetwork(const std::string &name, size_t num_requests);
 
+    IENetwork GetExecGraphInfo();
+
     void infer();
+
+    PyObject* getMetric(const std::string & metric_name);
+    PyObject* getConfig(const std::string & metric_name);
 };
 
 
@@ -163,7 +172,25 @@ struct IEPlugin {
 
     std::set<std::string> queryNetwork(const InferenceEnginePython::IENetwork &net);
 
-    InferenceEngine::InferenceEnginePluginPtr actual;
+    InferenceEngine::InferencePlugin actual;
+};
+
+struct IECore {
+    InferenceEngine::Core actual;
+    explicit IECore(const std::string & xmlConfigFile = std::string());
+    std::map<std::string, InferenceEngine::Version> getVersions(const std::string & deviceName);
+    std::unique_ptr<InferenceEnginePython::IEExecNetwork> loadNetwork(IENetwork network, const std::string & deviceName,
+            const std::map<std::string, std::string> & config, int num_requests);
+    std::map<std::string, std::string> queryNetwork(IENetwork network, const std::string & deviceName,
+                                       const std::map<std::string, std::string> & config);
+    void setConfig(const std::map<std::string, std::string> &config, const std::string & deviceName = std::string());
+    void registerPlugin(const std::string & pluginName, const std::string & deviceName);
+    void unregisterPlugin(const std::string & deviceName);
+    void registerPlugins(const std::string & xmlConfigFile);
+    void addExtension(const std::string & ext_lib_path, const std::string & deviceName);
+    std::vector<std::string> getAvailableDevices();
+    PyObject* getMetric(const std::string & deviceName, const std::string & name);
+    PyObject* getConfig(const std::string & deviceName, const std::string & name);
 };
 
 template<class T>

inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api_impl_defs.pxd

@@ -1,12 +1,12 @@
 from libc.stddef cimport size_t
+from libcpp cimport bool
 from libcpp.string cimport string
 from libcpp.vector cimport vector
 from libcpp.map cimport map
 from libcpp.set cimport set
 from libcpp.pair cimport pair
 from libcpp.memory cimport unique_ptr, shared_ptr
-from libc.stdint cimport int64_t
-
+from libc.stdint cimport int64_t, uint8_t
 
 
 cdef extern from "<inference_engine.hpp>" namespace "InferenceEngine":
@@ -24,6 +24,14 @@ cdef extern from "<inference_engine.hpp>" namespace "InferenceEngine":
     cdef cppclass Precision:
         const char*name() const
 
+    cdef struct apiVersion:
+        int minor
+        int major
+
+    cdef cppclass Version:
+        const char *buildNumber
+        const char *description
+        apiVersion apiVersion
 
 cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
     cdef cppclass IENetLayer:
@@ -69,17 +77,21 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
 
     cdef cppclass IEExecNetwork:
         vector[InferRequestWrap] infer_requests
+        IENetwork GetExecGraphInfo() except +
+        object getMetric(const string & metric_name)
+        object getConfig(const string & metric_name)
 
     cdef cppclass IENetwork:
         IENetwork() except +
-        IENetwork(const string &, const string &) except +
+        IENetwork(const string &, const string &, bool ngraph_compatibility) except +
         string name
         size_t batch_size
+        string precision
         map[string, vector[size_t]] inputs
         const vector[pair[string, IENetLayer]] getLayers() except +
         map[string, InputInfo] getInputs() except +
         map[string, OutputInfo] getOutputs() except +
-        void addOutputs(vector[string] &, string &) except +
+        void addOutput(string &, size_t, string &) except +
         void setAffinity(map[string, string] & types_affinity_map, map[string, string] & layers_affinity_map) except +
         void setBatch(size_t size) except +
         void setLayerParams(map[string, map[string, string]] params_map) except +
@@ -87,6 +99,7 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
         void reshape(map[string, vector[size_t]] input_shapes) except +
         void setStats(map[string, map[string, vector[float]]] & stats) except +
         map[string, map[string, vector[float]]] getStats() except +
+        void load_from_buffer(const char*xml, size_t xml_size, uint8_t*bin, size_t bin_size) except +
 
     cdef cppclass IEPlugin:
         IEPlugin() except +
@@ -101,12 +114,30 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
 
     cdef cppclass InferRequestWrap:
         double exec_time;
-        void getBlobPtr(const string &blob_name, Blob.Ptr &blob_ptr) except +
+        void getBlobPtr(const string & blob_name, Blob.Ptr & blob_ptr) except +
         map[string, ProfileInfo] getPerformanceCounts() except +
         void infer() except +
         void infer_async() except +
         int wait(int64_t timeout) except +
         void setBatch(int size) except +
+        void setCyCallback(void (*)(void*, int), void *) except +
+
+    cdef cppclass IECore:
+        IECore() except +
+        IECore(const string & xml_config_file) except +
+        map[string, Version] getVersions(const string & deviceName) except +
+        unique_ptr[IEExecNetwork] loadNetwork(IENetwork network, const string deviceName,
+                                              const map[string, string] & config, int num_requests) except +
+        map[string, string] queryNetwork(IENetwork network, const string deviceName,
+                                         const map[string, string] & config) except +
+        void setConfig(const map[string, string] & config, const string & deviceName) except +
+        void registerPlugin(const string & pluginName, const string & deviceName) except +
+        void unregisterPlugin(const string & deviceName) except +
+        void registerPlugins(const string & xmlConfigFile) except +
+        void addExtension(const string & ext_lib_path, const string & deviceName) except +
+        vector[string] getAvailableDevices() except +
+        object getMetric(const string & deviceName, const string & name) except +
+        object getConfig(const string & deviceName, const string & name) except +
 
     cdef T*get_buffer[T](Blob &)
 

inference-engine/ie_bridges/python/src/openvino/tools/statistics_collector/CMakeLists.txt

@@ -0,0 +1,33 @@
+# If the pyx file is a C++ file, we should specify that here.
+set (CMAKE_INCLUDE_CURRENT_DIR ON)
+set (TARGET_NAME "statistics_collector_api")
+
+set (CMAKE_LIBRARY_OUTPUT_DIRECTORY ${PYTHON_BRIDGE_OUTPUT_DIRECTORY}/tools/statistics_collector)
+set (CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
+
+file(GLOB SOURCE
+        ${CMAKE_CURRENT_SOURCE_DIR}/*.pyx
+        )
+
+set_source_files_properties(${SOURCE} PROPERTIES CYTHON_IS_CXX TRUE
+)
+include_directories (
+    ${CMAKE_SOURCE_DIR}/samples/common
+)
+
+## Compatibility with python 2.7 which has deprecated "register" specifier
+if("${CMAKE_CXX_COMPILER_ID}" STREQUAL "Clang")
+    add_definitions("-Wno-register")
+endif()
+
+cython_add_module (${TARGET_NAME} ${SOURCE})
+
+set_target_properties (${TARGET_NAME} PROPERTIES CXX_STANDARD 11 LINKER_LANGUAGE CXX)
+target_link_libraries (${TARGET_NAME} PRIVATE statistics_collector_s)
+
+# perform copy
+ADD_CUSTOM_COMMAND (TARGET ${TARGET_NAME}
+    POST_BUILD
+    COMMAND ${CMAKE_COMMAND} -E copy ${PYTHON_BRIDGE_SRC_ROOT}/src/openvino/tools/__init__.py ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/../__init__.py
+    COMMAND ${CMAKE_COMMAND} -E copy ${PYTHON_BRIDGE_SRC_ROOT}/src/openvino/tools/statistics_collector/__init__.py ${CMAKE_LIBRARY_OUTPUT_DIRECTORY}/__init__.py
+ )

inference-engine/ie_bridges/python/src/openvino/tools/statistics_collector/__init__.py

@@ -0,0 +1,2 @@
+from .statistics_collector_api import *
+__all__ = ['StatisticsCollector']

inference-engine/ie_bridges/python/src/openvino/tools/statistics_collector/statistics_collector_api.pxd

@@ -0,0 +1,8 @@
+from .cimport statistics_collector_c as C
+from libcpp.string cimport string
+
+
+cdef class StatisticsCollector:
+    cdef C.StatisticsCollector* _impl
+    cdef C.ct_preprocessingOptions ppOptions
+    cpdef void collectStatisticsToIR(self, str outModelName, str output_precision)

inference-engine/ie_bridges/python/src/openvino/tools/statistics_collector/statistics_collector_api.pyx

@@ -0,0 +1,25 @@
+#distutils: language=c++
+from .cimport statistics_collector_c as C
+
+
+cdef class StatisticsCollector:
+    def __cinit__(self,
+    	        deviceName: [str, bytes],
+                custom_cpu_library: [str, bytes],
+                custom_cldnn: [str, bytes],
+                modelFilePath: [str, bytes],
+                imagesPath: [str, bytes],
+                img_number: int,
+                batch: int,
+                progress: [str, bytes]):
+        self.ppOptions._pp_size = 0
+        self.ppOptions._pp_width = 0
+        self.ppOptions._pp_height = 0
+        self._impl = new C.StatisticsCollector(deviceName.encode(), custom_cpu_library.encode(), custom_cldnn.encode(), modelFilePath.encode(), imagesPath.encode(), img_number, batch, self.ppOptions, progress.encode())
+
+    cpdef void collectStatisticsToIR(self, str outModelName, str output_precision):
+        self._impl.collectStatisticsToIR(outModelName.encode(), output_precision.encode())
+
+    def __dealloc__(self):
+        if self._impl is not NULL:
+            del self._impl

inference-engine/ie_bridges/python/src/openvino/tools/statistics_collector/statistics_collector_c.pxd

@@ -0,0 +1,24 @@
+from libc.stddef cimport size_t
+from libcpp.string cimport string
+
+
+cdef extern from "<statistics_processor.hpp>":
+
+    cdef struct ct_preprocessingOptions:
+        string _pp_type
+        size_t _pp_size
+        size_t _pp_width
+        size_t _pp_height
+
+    cdef cppclass StatisticsCollector:
+        StatisticsCollector(const string& deviceName,
+                            const string& custom_cpu_library,
+                            const string& custom_cldnn,
+                            const string& modelFilePath,
+                            const string& imagesPath,
+                            size_t img_number,
+                            size_t batch,
+                            const ct_preprocessingOptions& preprocessingOptions,
+                            const string& progress) except +
+        void collectStatisticsToIR(const string& outModelName, const string& output_precision)
+        ct_preprocessingOptions ppOptions

inference-engine/include/builders/ie_concat_layer.hpp

@@ -74,7 +74,7 @@ public:
     ConcatLayer& setAxis(size_t axis);
 
 private:
-    size_t axis;
+    size_t axis = 1;
 };
 
 }  // namespace Builder

inference-engine/include/builders/ie_deformable_convolution_layer.hpp

@@ -0,0 +1,48 @@
+// Copyright (C) 2019 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+#pragma once
+
+#include <builders/ie_convolution_layer.hpp>
+#include <ie_network.hpp>
+#include <string>
+
+namespace InferenceEngine {
+namespace Builder {
+
+/**
+ * @brief The class represents a builder for Deconvolution layer
+ */
+class INFERENCE_ENGINE_API_CLASS(DeformableConvolutionLayer): public ConvolutionLayer {
+public:
+    /**
+     * @brief The constructor creates a builder with the name
+     * @param name Layer name
+     */
+    explicit DeformableConvolutionLayer(const std::string& name = "");
+    /**
+     * @brief The constructor creates a builder from generic builder
+     * @param layer pointer to generic builder
+     */
+    explicit DeformableConvolutionLayer(const Layer::Ptr& layer);
+    /**
+     * @brief The constructor creates a builder from generic builder
+     * @param layer constant pointer to generic builder
+     */
+    explicit DeformableConvolutionLayer(const Layer::CPtr& layer);
+    /**
+     * @brief Return deformable_group size
+     * @return Deformable group size
+     */
+    size_t getDeformableGroup() const;
+    /**
+     * @brief Sets deformable group size
+     * @param deformableGroup Deformable group
+     * @return reference to layer builder
+     */
+    Builder::DeformableConvolutionLayer& setDeformableGroup(size_t deformableGroup);
+};
+
+}  // namespace Builder
+}  // namespace InferenceEngine

inference-engine/include/builders/ie_eltwise_layer.hpp

@@ -98,7 +98,7 @@ public:
     EltwiseLayer& setScales(const std::vector<float>& scales);
 
 private:
-    EltwiseType type;
+    EltwiseType type = SUM;
 };
 
 }  // namespace Builder

inference-engine/include/builders/ie_grn_layer.hpp

@@ -12,7 +12,7 @@ namespace InferenceEngine {
 namespace Builder {
 
 /**
- * @brief The class represents a builder for ArgMax layer
+ * @brief The class represents a builder for GRN layer
  */
 class INFERENCE_ENGINE_API_CLASS(GRNLayer): public LayerDecorator {
 public:

inference-engine/include/builders/ie_pooling_layer.hpp

@@ -161,8 +161,8 @@ public:
     PoolingLayer& setExcludePad(bool exclude);
 
 private:
-    PoolingType type;
-    RoundingType roundingType;
+    PoolingType type = MAX;
+    RoundingType roundingType = CEIL;
 };
 
 }  // namespace Builder

inference-engine/include/cldnn/cldnn_config.hpp

@@ -56,5 +56,10 @@ DECLARE_CLDNN_CONFIG_KEY(GRAPH_DUMPS_DIR);
 */
 DECLARE_CLDNN_CONFIG_KEY(SOURCES_DUMPS_DIR);
 
+/**
+* @brief This key turns usage of int8 optimizations and qunatized models on.
+*/
+DECLARE_CLDNN_CONFIG_KEY(INT8_ENABLED);
+
 }  // namespace CLDNNConfigParams
 }  // namespace InferenceEngine

inference-engine/include/cpp/ie_cnn_net_reader.h

@@ -17,6 +17,7 @@
 #include "ie_common.h"
 #include "ie_icnn_net_reader.h"
 #include "details/ie_exception_conversion.hpp"
+#include "details/os/os_filesystem.hpp"
 
 namespace InferenceEngine {
 /**
@@ -35,6 +36,16 @@ public:
      */
     CNNNetReader() : actual(shared_from_irelease(InferenceEngine::CreateCNNNetReader())) {}
 
+#ifdef ENABLE_UNICODE_PATH_SUPPORT
+    /**
+     * @brief Resolve wstring path then call orginal ReadNetwork
+     * ICNNNetReader::ReadNetwork
+    */
+    void ReadNetwork(const std::wstring &filepath) {
+        CALL_STATUS_FNC(ReadNetwork, details::wStringtoMBCSstringChar(filepath).c_str());
+    }
+#endif
+
     /**
      * @brief Wraps original method
      * ICNNNetReader::ReadNetwork
@@ -55,15 +66,25 @@ public:
      * @brief Wraps original method
      * ICNNNetReader::SetWeights
      */
-    void SetWeights(const TBlob<uint8_t>::Ptr &weights) const {
+    void SetWeights(const TBlob<uint8_t>::Ptr &weights) {
         CALL_STATUS_FNC(SetWeights, weights);
     }
 
+#ifdef ENABLE_UNICODE_PATH_SUPPORT
+    /**
+    * @brief Resolve wstring path then call orginal ReadWeights
+    * ICNNNetReader::ReadWeights
+    */
+    void ReadWeights(const std::wstring &filepath) {
+        CALL_STATUS_FNC(ReadWeights, details::wStringtoMBCSstringChar(filepath).c_str());
+    }
+#endif
+
     /**
      * @brief Wraps original method
      * ICNNNetReader::ReadWeights
      */
-    void ReadWeights(const std::string &filepath) const {
+    void ReadWeights(const std::string &filepath) {
         CALL_STATUS_FNC(ReadWeights, filepath.c_str());
     }
 

inference-engine/include/cpp/ie_cnn_network.h

@@ -34,10 +34,11 @@ public:
     CNNNetwork() = default;
 
     /**
+     * @deprecated Use CNNNetwork::CNNNetwork(std::shared_ptr<ICNNNetwork>) to construct a network
      * @brief Initialises helper class from externally managed pointer
-     * @deprecated use shared_pointers based version of CNNNetworks constructor
      * @param actual Pointer to the network object
      */
+    INFERENCE_ENGINE_DEPRECATED
     explicit CNNNetwork(ICNNNetwork* actual) : actual(actual) {
         if (actual == nullptr) {
             THROW_IE_EXCEPTION << "CNNNetwork was not initialized.";
@@ -142,11 +143,17 @@ public:
     }
 
     /**
+     * @deprecated No needs to specify target device to the network. Use InferenceEngine::Core with target device directly
      * @brief Sets tha target device
      * @param device Device instance to set
      */
+    #ifndef _WIN32
+    INFERENCE_ENGINE_DEPRECATED
+    #endif
     void setTargetDevice(TargetDevice device) {
+        IE_SUPPRESS_DEPRECATED_START
         actual->setTargetDevice(device);
+        IE_SUPPRESS_DEPRECATED_END
     }
 
     /**
@@ -212,7 +219,7 @@ public:
             if (info) {
                 auto data = info->getInputData();
                 if (data) {
-                    shapes[data->name] = data->getTensorDesc().getDims();
+                    shapes[data->getName()] = data->getTensorDesc().getDims();
                 }
             }
         }

inference-engine/include/cpp/ie_executable_network.hpp

@@ -14,6 +14,7 @@
 #include <memory>
 #include <algorithm>
 #include "ie_iexecutable_network.hpp"
+#include "ie_plugin_ptr.hpp"
 #include "cpp/ie_infer_request.hpp"
 #include "cpp/ie_memory_state.hpp"
 #include "cpp/ie_cnn_network.h"
@@ -26,11 +27,16 @@ namespace InferenceEngine {
  */
 class ExecutableNetwork {
     IExecutableNetwork::Ptr actual;
+    InferenceEnginePluginPtr plg;
 
 public:
     ExecutableNetwork() = default;
+    ~ExecutableNetwork() {
+        actual = nullptr;
+    }
 
-    explicit ExecutableNetwork(IExecutableNetwork::Ptr actual) : actual(actual) {}
+    explicit ExecutableNetwork(IExecutableNetwork::Ptr actual, InferenceEnginePluginPtr plg = {})
+    : actual(actual), plg(plg) {}
 
     /**
      * @brief Wraps original method
@@ -68,7 +74,7 @@ public:
         IInferRequest::Ptr req;
         CALL_STATUS_FNC(CreateInferRequest, req);
         if (req.get() == nullptr) THROW_IE_EXCEPTION << "Internal error: pointer to infer request is null";
-        return InferRequest(req);
+        return InferRequest(req, plg);
     }
 
     /**
@@ -79,7 +85,7 @@ public:
     InferRequest::Ptr CreateInferRequestPtr() {
         IInferRequest::Ptr req;
         CALL_STATUS_FNC(CreateInferRequest, req);
-        return std::make_shared<InferRequest>(req);
+        return std::make_shared<InferRequest>(req, plg);
     }
 
     /**
@@ -139,6 +145,41 @@ public:
         return controller;
     }
 
+    /**
+     * @brief Sets configuration for current executable network
+     * @param config Map of pairs: (config parameter name, config parameter value)
+     * @param resp Pointer to the response message that holds a description of an error if any occurred
+     */
+    void SetConfig(const std::map<std::string, Parameter> &config) {
+        CALL_STATUS_FNC(SetConfig, config);
+    }
+
+    /** @brief Gets configuration dedicated to plugin behaviour
+        * @param name - config key, can be found in ie_plugin_config.hpp
+        * @param options - configuration details for coonfig value
+        * @param result - value of config corresponding to config key
+        * @param resp Pointer to the response message that holds a description of an error if any occurred
+    */
+    Parameter GetConfig(const std::string &name) const {
+        Parameter configValue;
+        CALL_STATUS_FNC(GetConfig, name, configValue);
+        return configValue;
+    }
+
+    /**
+     * @brief Gets general runtime metric for dedicated hardware
+     * @param name  - metric name to request
+     * @param options - configuration details for metric
+     * @param result - metric value corresponding to metric key
+     * @param resp - Pointer to the response message that holds a description of an error if any
+     *             occurred
+     * @return code of the operation. OK if succeeded
+     */
+    Parameter GetMetric(const std::string &name) const {
+        Parameter metricValue;
+        CALL_STATUS_FNC(GetMetric, name, metricValue);
+        return metricValue;
+    }
 
     using Ptr = std::shared_ptr<ExecutableNetwork>;
 };

inference-engine/include/cpp/ie_infer_request.hpp

@@ -13,6 +13,7 @@
 #include <map>
 #include "ie_iinfer_request.hpp"
 #include "details/ie_exception_conversion.hpp"
+#include "ie_plugin_ptr.hpp"
 
 namespace InferenceEngine {
 
@@ -57,6 +58,7 @@ public:
  */
 class InferRequest {
     IInferRequest::Ptr actual;
+    InferenceEnginePluginPtr plg;
     std::shared_ptr<details::ICompletionCallbackWrapper> callback;
 
     static void callWrapper(InferenceEngine::IInferRequest::Ptr request, InferenceEngine::StatusCode code) {
@@ -69,6 +71,10 @@ class InferRequest {
 public:
     InferRequest() = default;
 
+    ~InferRequest() {
+        actual = nullptr;
+    }
+
     /**
      * @brief Sets input/output data to infer
      * @note: Memory allocation does not happen
@@ -147,7 +153,8 @@ public:
      * constructs InferRequest from initialised shared_pointer
      * @param actual
      */
-    explicit InferRequest(IInferRequest::Ptr request) : actual(request) {}
+    explicit InferRequest(IInferRequest::Ptr request, InferenceEnginePluginPtr plg = {})
+    : actual(request), plg(plg) {}
 
     /**
      * @brief Start inference of specified input(s) in asynchronous mode

inference-engine/include/cpp/ie_plugin_cpp.hpp

@@ -44,15 +44,21 @@ public:
     const Version *GetVersion() {
         const Version *versionInfo = nullptr;
         actual->GetVersion(versionInfo);
+        if (versionInfo == nullptr) {
+            THROW_IE_EXCEPTION << "Unknown device is used";
+        }
         return versionInfo;
     }
 
     /**
-     * @brief Wraps original method
-     * IInferencePlugin::LoadNetwork
+     * @deprecated Use InferencePlugin::LoadNetwork(ICNNNetwork &, const std::map<std::string, std::string> &)
+     * @brief Wraps original method IInferencePlugin::LoadNetwork(ICNNNetwork &, ResponseDesc *)
      */
+    INFERENCE_ENGINE_DEPRECATED
     void LoadNetwork(ICNNNetwork &network) {
+        IE_SUPPRESS_DEPRECATED_START
         CALL_STATUS_FNC(LoadNetwork, network);
+        IE_SUPPRESS_DEPRECATED_END
     }
 
     /**
@@ -62,7 +68,7 @@ public:
     ExecutableNetwork LoadNetwork(ICNNNetwork &network, const std::map<std::string, std::string> &config) {
         IExecutableNetwork::Ptr ret;
         CALL_STATUS_FNC(LoadNetwork, ret, network, config);
-        return ExecutableNetwork(ret);
+        return ExecutableNetwork(ret, actual);
     }
 
     /**
@@ -73,25 +79,30 @@ public:
         IExecutableNetwork::Ptr ret;
         CALL_STATUS_FNC(LoadNetwork, ret, network, config);
         if (ret.get() == nullptr) THROW_IE_EXCEPTION << "Internal error: pointer to executable network is null";
-        return ExecutableNetwork(ret);
+        return ExecutableNetwork(ret, actual);
     }
 
     /**
-     * @deprecated Loads IExecutableNetwork to create IInferRequest.
-     * @brief Wraps original method
-     * IInferencePlugin::Infer(const BlobMap&, BlobMap&, ResponseDesc *resp)
+     * @deprecated Use IExecutableNetwork to create IInferRequest.
+     * @brief Wraps original method IInferencePlugin::Infer(const BlobMap&, BlobMap&, ResponseDesc *)
      */
+    INFERENCE_ENGINE_DEPRECATED
     void Infer(const BlobMap &input, BlobMap &result) {
+        IE_SUPPRESS_DEPRECATED_START
         CALL_STATUS_FNC(Infer, input, result);
+        IE_SUPPRESS_DEPRECATED_END
     }
 
     /**
-     * @brief Wraps original method
-     * IInferencePlugin::GetPerformanceCounts
+     * @deprecated Use IInferRequest to get performance counters
+     * @brief Wraps original method IInferencePlugin::GetPerformanceCounts
      */
+    INFERENCE_ENGINE_DEPRECATED
     std::map<std::string, InferenceEngineProfileInfo> GetPerformanceCounts() const {
         std::map<std::string, InferenceEngineProfileInfo> perfMap;
+        IE_SUPPRESS_DEPRECATED_START
         CALL_STATUS_FNC(GetPerformanceCounts, perfMap);
+        IE_SUPPRESS_DEPRECATED_END
         return perfMap;
     }
 
@@ -115,25 +126,25 @@ public:
      * @brief Wraps original method
      * IInferencePlugin::ImportNetwork
     */
-    ExecutableNetwork  ImportNetwork(const std::string &modelFileName, const std::map<std::string, std::string> &config) {
+    ExecutableNetwork ImportNetwork(const std::string &modelFileName, const std::map<std::string, std::string> &config) {
         IExecutableNetwork::Ptr ret;
         CALL_STATUS_FNC(ImportNetwork, ret, modelFileName, config);
-        return ExecutableNetwork(ret);
+        return ExecutableNetwork(ret, actual);
     }
 
     /**
-     * @depricated Use the version with config parameter
+     * @deprecated Use InferencePlugin::QueryNetwork(const ICNNNetwork &, const std::map<std::string, std::string> &, QueryNetworkResult &) const
      * @brief Wraps original method
-     * IInferencePlugin::QueryNetwork
+     * IInferencePlugin::QueryNetwork(const ICNNNetwork&, QueryNetworkResult& ) const
      */
+    INFERENCE_ENGINE_DEPRECATED
     void QueryNetwork(const ICNNNetwork &network, QueryNetworkResult &res) const {
-        actual->QueryNetwork(network, res);
-        if (res.rc != OK) THROW_IE_EXCEPTION << res.resp.msg;
+        QueryNetwork(network, { }, res);
     }
 
     /**
      * @brief Wraps original method
-     * IInferencePlugin::QueryNetwork
+     * IInferencePlugin::QueryNetwork(const ICNNNetwork&, const std::map<std::string, std::string> &, QueryNetworkResult&) const
      */
     void QueryNetwork(const ICNNNetwork &network, const std::map<std::string, std::string> &config, QueryNetworkResult &res) const {
         actual->QueryNetwork(network, config, res);
@@ -141,6 +152,7 @@ public:
     }
 
     /**
+     * @brief Converts InferenceEngine to InferenceEnginePluginPtr pointer
      * @brief Returns wrapped object
      */
     operator InferenceEngine::InferenceEnginePluginPtr() {
@@ -148,11 +160,15 @@ public:
     }
 
     /**
-    * @return wrapped Hetero object if underlined object is HeteroPlugin instance, nullptr otherwise
-    */
+     * @deprecated Deprecated since HeteroPluginPtr is deprecated
+     * @brief Converts InferenceEngine to HeteroPluginPtr pointer
+     * @return wrapped Hetero object if underlined object is HeteroPlugin instance, nullptr otherwise
+     */
+    IE_SUPPRESS_DEPRECATED_START
     operator InferenceEngine::HeteroPluginPtr() {
         return actual;
     }
+    IE_SUPPRESS_DEPRECATED_END
 
     /**
      * @brief Shared pointer on InferencePlugin object

inference-engine/include/details/ie_exception.hpp

@@ -35,11 +35,11 @@
 class NullStream {
  public :
     template <class T>
-    NullStream & operator << (const T &obj) noexcept {
+    NullStream & operator << (const T &) noexcept {
         return *this;
     }
 
-    NullStream &  operator<< (std::ostream & (*manip)(std::ostream &)) noexcept {
+    NullStream &  operator<< (std::ostream & (*)(std::ostream &)) noexcept {
         return *this;
     }
 };

inference-engine/include/details/ie_inetwork_iterator.hpp

@@ -23,8 +23,9 @@ namespace details {
 template<class NT, class LT>
 class INetworkIterator: public std::iterator<std::input_iterator_tag, std::shared_ptr<LT>> {
 public:
-    explicit INetworkIterator(NT * network, bool toEnd = false): network(network), currentIdx(0) {
-        if (!network || toEnd)
+    explicit INetworkIterator(NT * network, bool toEnd): network(network), currentIdx(0) {}
+    explicit INetworkIterator(NT * network): network(network), currentIdx(0) {
+        if (!network)
             return;
         const auto& inputs = network->getInputs();
 

inference-engine/include/details/ie_so_pointer.hpp

@@ -74,6 +74,7 @@ class SOCreatorTrait {};
 */
 template <class T, class Loader = SharedObjectLoader>
 class SOPointer {
+IE_SUPPRESS_DEPRECATED_START
     template <class U, class W> friend class SOPointer;
 public:
     /**
@@ -91,6 +92,18 @@ public:
             SymbolLoader<Loader>(_so_loader).template instantiateSymbol<T>(SOCreatorTrait<T>::name))) {
     }
 
+    /**
+    * @brief Constructs an object with existing reference
+    * @param _pointedObj_ Existing reference to wrap
+    */
+    explicit SOPointer(T * _pointedObj_)
+        : _so_loader()
+        , _pointedObj(_pointedObj_) {
+        if (_pointedObj == nullptr) {
+            THROW_IE_EXCEPTION << "Cannot create SOPointer<T, Loader> from nullptr";
+        }
+    }
+
     /**
     * @brief The copy-like constructor, can create So Pointer that dereferenced into child type if T is derived of U
     * @param that copied SOPointer object
@@ -99,6 +112,9 @@ public:
     SOPointer(const SOPointer<U, W> & that) :
         _so_loader(std::dynamic_pointer_cast<Loader>(that._so_loader)),
         _pointedObj(std::dynamic_pointer_cast<T>(that._pointedObj)) {
+        if (_pointedObj == nullptr) {
+            THROW_IE_EXCEPTION << "Cannot create object from SOPointer<U, W> reference";
+        }
     }
 
     /**
@@ -149,6 +165,7 @@ protected:
      * @brief Gets a smart pointer to the custom object
      */
     std::shared_ptr<T> _pointedObj;
+IE_SUPPRESS_DEPRECATED_END
 };
 
 }  // namespace details

inference-engine/include/details/os/os_filesystem.hpp

@@ -0,0 +1,39 @@
+// Copyright (C) 2018-2019 Intel Corporation
+// SPDX-License-Identifier: Apache-2.0
+//
+
+/**
+ * @brief This is a header file with functions related to filesystem operations.
+ * @file os_filesystem.h
+ */
+#pragma once
+
+#ifdef ENABLE_UNICODE_PATH_SUPPORT
+#include <string>
+#include <codecvt>
+#include <locale>
+
+namespace InferenceEngine {
+namespace details {
+
+/**
+* @brief Conversion from wide character string to a single-byte chain.
+*/
+inline const std::string wStringtoMBCSstringChar(const std::wstring& wstr) {
+    std::wstring_convert<std::codecvt_utf8<wchar_t>> wstring_decoder;
+    return wstring_decoder.to_bytes(wstr);
+}
+
+/**
+* @brief Conversion from single-byte chain to wide character string.
+*/
+inline const std::wstring multiByteCharToWString(const char* str) {
+    std::wstring_convert<std::codecvt_utf8 <wchar_t>> wstring_encoder;
+    std::wstring result = wstring_encoder.from_bytes(str);
+    return result;
+}
+
+}  // namespace details
+}  // namespace InferenceEngine
+
+#endif

inference-engine/include/details/os/win_shared_object_loader.h

@@ -30,7 +30,7 @@ class SharedObjectLoader {
 private:
     HMODULE shared_object;
 
- public:
+public:
     /**
      * @brief Loads a library with the name specified. The library is loaded according to the
      *        WinAPI LoadLibrary rules
@@ -38,6 +38,20 @@ private:
      */
     explicit SharedObjectLoader(LPCTSTR pluginName) {
         char cwd[1024];
+        // Exclude current directory from DLL search path process wise.
+        // If application specific path was configured before then
+        // current directory is alread excluded.
+        // GetDLLDirectory does not distinguish if aplication specific
+        // path was set to "" or NULL so reset it to "" to keep
+        // aplication safe.
+        if (GetDllDirectory(0, NULL) <= 1) {
+            SetDllDirectory(
+#if defined UNICODE
+                L"");
+#else
+                "");
+#endif
+        }
         shared_object = LoadLibrary(pluginName);
         if (!shared_object) {
             THROW_IE_EXCEPTION << "Cannot load library '"

inference-engine/include/gna/gna_config.hpp

@@ -53,6 +53,7 @@ DECLARE_GNA_CONFIG_VALUE(AUTO);
 DECLARE_GNA_CONFIG_VALUE(HW);
 DECLARE_GNA_CONFIG_VALUE(SW);
 DECLARE_GNA_CONFIG_VALUE(SW_EXACT);
+DECLARE_GNA_CONFIG_VALUE(SW_FP32);
 
 /**
 * @brief if enabled produced minimum memory footprint for loaded network in GNA memory, default value is YES

inference-engine/include/hetero/hetero_plugin_config.hpp

@@ -12,6 +12,8 @@
 #pragma once
 
 #include <string>
+#include <vector>
+
 #include "ie_plugin_config.hpp"
 
 namespace InferenceEngine {
@@ -28,6 +30,12 @@ namespace HeteroConfigParams {
  * This option should be used with values: CONFIG_VALUE(NO) (default) or CONFIG_VALUE(YES)
  */
 DECLARE_HETERO_CONFIG_KEY(DUMP_GRAPH_DOT);
+
+/**
+ * @deprecated Use DLIA_CONFIG_KEY(DUMP_SUPPORTED_LAYERS_INFORMATION) FPGA configuration boolean key instead
+ * @brief The bool key to define whether information messages with a reason are printed in case the layer is unsupported by DLA
+ */
+INFERENCE_ENGINE_DEPRECATED
 DECLARE_HETERO_CONFIG_KEY(DUMP_DLA_MESSAGES);
 
 }  // namespace HeteroConfigParams