IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Added template for core dev documentation (#13091)

Browse Source 
* Added template for core dev documentation

* Update test_coverage.md

* Update coding_style.md

* Update build.md

* Create debug_capabilities.md

* Update index.md

* Update debug_capabilities.md

* Update index.md

* Update README.md

* Create shape_propagation

* Update and rename shape_propagation to shape_propagation.md

* Update shape_propagation.md

* Create operation_enabling_flow.md

* Update index.md

* Update README.md

* Update operation_enabling_flow.md
This commit is contained in:
Ilya Churaev 2022-09-19 15:57:01 +04:00 committed by GitHub
parent c66c7064e0
commit dcca08763b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 1113 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
CONTRIBUTING.md
View File

@ -35,7 +35,7 @@ Following these guidelines will increase the likelihood of your pull request bei
- One PR – one issue.
- Build perfectly on your local system.
- Choose the right base branch [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches).
- Follow the [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines) for your code.
- Follow the [Coding Style Guide](./docs/dev/coding_style.md) for your code.
- Update documentation using [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation) if needed.
- Cover your changes with test.
- Add license at the top of new files [C++ example](https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/classification_sample_async/main.cpp#L1-L2), [Python example](https://github.com/openvinotoolkit/openvino/blob/master/samples/python/hello_classification/hello_classification.py#L3-L4).
@ -52,4 +52,4 @@ Your pull request will be automatically tested by OpenVINO™'s precommit (testi
## Merging PR
As soon as the reviewer is fine with the pull request and precommit shows "green" status, the "Approved" review status is put, which signals OpenVINO™ maintainers that they can merge your pull request.
As soon as the reviewer is fine with the pull request and precommit shows "green" status, the "Approved" review status is put, which signals OpenVINO™ maintainers that they can merge your pull request.

12
README.md
View File

@ -144,7 +144,7 @@ The latest documentation for OpenVINO™ Toolkit is available [here](https://doc
### Developer documentation
[Developer documentation](#todo-add) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
[Developer documentation](./docs/dev/index.md) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
## Tutorials
@ -161,11 +161,11 @@ The list of OpenVINO tutorials:
## System requirements
The full information about system requirements depends on platform and available in section `System requirement` on dedicated pages:
- [Linux](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_linux.html)
- [Windows](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_windows.html)
- [macOS](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_macos.html)
- [Raspbian](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_raspbian.html)
The full information about system requirements depends on platform is available on dedicated pages:
- [Linux](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_linux.html#system-requirements)
- [Windows](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_windows.html#system-requirements)
- [macOS](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_macos.html#system-requirements)
- [Raspbian](https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_raspbian.html#system-requirements)
## How to build

411
docs/dev/build.md Normal file
View File

@ -0,0 +1,411 @@
# How to build OpenVINO
The guide provides the basic information about the process of building OpenVINO.
```mermaid
gantt
%% Use a hack for centry as a persantage
dateFormat YYYY
axisFormat %y
todayMarker off
title OpenVINO getting started pipeline
Setup environment :env, 2000, 1716w
Build openvino :crit, build, after env, 1716w
Run tests :active, run, after build, 1716w
```
## Setup environment
This charpter describe how to prepare your machine for OpenVINO development.
### Software requirements
For the OpenVINO build next tools are required:
<details><summary>Windows</summary>
<p>
- [CMake]\* 3.14 or higher
- Microsoft\* Visual Studio 2019, version 16.8 or later
- (Optional) Intel® Graphics Driver for Windows* (30.0) [driver package](https://www.intel.com/content/www/us/en/download/19344/intel-graphics-windows-dch-drivers.html).
- Python 3.6 or higher for OpenVINO Runtime Python API
- [Git for Windows*](https://gitforwindows.org/)
</p>
</details>
<details><summary>Linux</summary>
<p>
- [CMake] 3.13 or higher
- GCC 7.5 or higher to build OpenVINO Runtime
- Python 3.6 or higher for OpenVINO Runtime Python API
- (Optional) [Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.41.14441] to enable inference on Intel integrated GPUs.
<p>
</details>
<details><summary>Mac</summary>
<p>
- [brew](https://brew.sh) package manager to install additional dependencies. Use [install brew](https://brew.sh) guide to achieve this.
- [CMake]\* 3.13 or higher
```sh
% brew install cmake
```
- Clang\* compiler, git and other command line tools from Xcode\* 10.1 or higher:
```sh
% xcode-select --install
```
- The step to install python and python libraries is different depending on host architecture:
- **x86_64** Python\* 3.6 or higher for the OpenVINO Runtime Python API, Development tools (Model Optimizer, POT and others):
```sh
% # let's have a look what python versions are available in brew
% brew search python
% # select preferred version of python based on available ones, e.g. 3.10
% brew install python@3.10
```
- **arm64** Select universal2 installer from [Python releases] download page and install `python-3.X.Y-macos11.pkg` image. This allows to have universal python libraries and build x86_64 OpenVINO Python API and Development tools.
- Additional `pip` dependencies to build OpenVINO Runtime Python API, Development tools (Model Optimizer, POT and others):
```sh
% # update pip and setuptools to newer versions
% python3 -m pip install -U pip setuptools
% python3 -m pip install cython
```
In order to build OpenVINO Python API and Development tools as wheel packages, additionally install requirements (after OpenVINO repo clone):
```sh
% python3 -m pip install -r <openvino source tree>/src/bindings/python/wheel/requirements-dev.txt
```
- (native compilation only) libusb library for MYRIAD device and `pkg-config` which is used to find `libusb` files:
```sh
% brew install pkg-config libusb
```
- (Optional; native compilation only) Latest version of TBB library. By default, OpenVINO downloads prebuilt version of TBB 2020.4 library, but if you want to use latest (add `-DENABLE_SYSTEM_TBB=ON` additionally to cmake configuration step):
```sh
% brew install tbb
```
</p>
</details>
### Clone OpenVINO Repository
Clone OpenVINO repository and init submodules:
```sh
git clone https://github.com/openvinotoolkit/openvino.git
cd openvino
git submodule update --init --recursive
```
<details><summary>(Optional) For users in China, clone submodules via gitee mirrors</summary>
<p>
```sh
chmod +x scripts/submodule_update_with_gitee.sh
./scripts/submodule_update_with_gitee.sh
```
<p>
</details>
Congratulate! Now you are ready to build the OpenVINO.
## How to build
<details><summary>Windows</summary>
<p>
> **NOTE**: By default, the build enables the OpenVINO Runtime GPU plugin to infer models on your Intel® Processor Graphics. This requires you to download and install the Intel® Graphics Driver for Windows (26.20) [driver package](https://www.intel.com/content/www/us/en/download/19344/intel-graphics-windows-dch-drivers.html) before running the build. If you don't want to use the GPU plugin, use the `-DENABLE_INTEL_GPU=OFF` CMake build option and skip the installation of the Intel® Graphics Driver.
1. Create build directory:
```sh
mkdir build && cd build
```
2. In the `build` directory, run `cmake` to fetch project dependencies and
generate a Visual Studio solution.
For Microsoft\* Visual Studio 2019 x64 architecture:
```sh
cmake -G "Visual Studio 16 2019" -A x64 -DCMAKE_BUILD_TYPE=Release ..
```
For Microsoft\* Visual Studio 2019 ARM architecture:
```sh
cmake -G "Visual Studio 16 2019" -A ARM -DCMAKE_BUILD_TYPE=Release ..
```
For Microsoft\* Visual Studio 2019 ARM64 architecture:
```sh
cmake -G "Visual Studio 16 2019" -A ARM64 -DCMAKE_BUILD_TYPE=Release ..
```
3. Build generated solution in Visual Studio or run
`cmake --build . --config Release --verbose -j8` to build from the command line. Note that this process may take some time.
4. Before running the samples, add paths to the Threading Building Blocks (TBB) and OpenCV binaries used for
the build to the `%PATH%` environment variable. By default, TBB binaries are
downloaded by the CMake-based script to the `<openvino_repo>/temp/tbb/bin`
folder, OpenCV binaries to the `<openvino_repo>/temp/opencv_4.5.0/opencv/bin`
folder.
### Additional Build Options
- Internal JIT GEMM implementation is used by default.
- Threading Building Blocks (TBB) is used by default. To build Inference
Engine with OpenMP threading, set the `-DTHREADING=OMP` option.
- Required versions of TBB and OpenCV packages are downloaded automatically by
the CMake-based script. If you want to use the automatically-downloaded
packages but you have already installed TBB or OpenCV packages configured in
your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR`
environment variables before running the `cmake` command; otherwise they won't
be downloaded and the build may fail if incompatible versions were installed.
- If the CMake-based build script can not find and download the OpenCV package
that is supported on your platform, or if you want to use a custom build of
the OpenCV library, refer to the
[Use Custom OpenCV Builds](https://github.com/openvinotoolkit/openvino/wiki/CMakeOptionsForCustomCompilation#Building-with-custom-OpenCV)
section for details.
- To switch off/on the CPU and GPU plugins, use the `cmake` options
`-DENABLE_INTEL_CPU=ON/OFF` and `-DENABLE_INTEL_GPU=ON/OFF` respectively.
- To build the OpenVINO Runtime Python API:
1. First, install all additional packages (e.g., cython and opencv) listed in the
`src\bindings\python\src\compatibility\openvino\requirements-dev.txt` file:
```sh
pip install -r requirements-dev.txt
```
2. Second, enable the `-DENABLE_PYTHON=ON` in the CMake (Step #4) option above. To
specify an exact Python version, use the following options:
```sh
-DPYTHON_EXECUTABLE="C:\Program Files\Python37\python.exe" ^
-DPYTHON_LIBRARY="C:\Program Files\Python37\libs\python37.lib" ^
-DPYTHON_INCLUDE_DIR="C:\Program Files\Python37\include"
```
- OpenVINO runtime compilation options:
`-DENABLE_OV_ONNX_FRONTEND=ON` enables the building of the ONNX importer.
### Building OpenVINO with Ninja* Build System
```sh
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvars64.bat"
cmake -G Ninja -Wno-dev -DCMAKE_BUILD_TYPE=Release ..
cmake --build . --config Release
```
<p>
</details>
<details><summary>Linux</summary>
<p>
1. Install build dependencies using the `install_build_dependencies.sh` script in the
project root folder.
```sh
chmod +x install_build_dependencies.sh
```
```sh
./install_build_dependencies.sh
```
> **NOTE**: By default, the build enables the OpenVINO Runtime GPU plugin to infer models on your Intel® Processor Graphics. This requires you to [Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.41.14441] before running the build. If you don't want to use the GPU plugin, use the `-DENABLE_INTEL_GPU=OFF` CMake build option and skip the installation of the Intel® Graphics Compute Runtime for OpenCL™ Driver.
2. Create a build folder:
```sh
mkdir build && cd build
```
3. OpenVINO Runtime uses a CMake-based build system. In the created `build`
directory, run `cmake` to fetch project dependencies and create Unix
makefiles, then run `make` to build the project:
```sh
cmake -DCMAKE_BUILD_TYPE=Release ..
make --jobs=$(nproc --all)
```
The process may take some time to finish.
### Additional Build Options
You can use the following additional build options:
- Threading Building Blocks (TBB) is used by default. To build the OpenVINO
Runtime with OpenMP\* threading set the `-DTHREADING=OMP` option.
- For IA32 operation systems, use [ia32.linux.toolchain.cmake](https://github.com/openvinotoolkit/openvino/blob/master/cmake/toolchains/ia32.linux.toolchain.cmake) CMake toolchain file:
```sh
cmake -DCMAKE_TOOLCHAIN_FILE=<openvino_repo>/cmake/toolchains/ia32.linux.toolchain.cmake ..
```
- Required versions of TBB and OpenCV packages are downloaded automatically by
the CMake-based script. If you want to use the automatically downloaded
packages but you have already installed TBB or OpenCV packages configured in
your environment, you may need to clean the `TBBROOT` and `OpenCV_DIR`
environment variables before running the `cmake` command, otherwise they
will not be downloaded and the build may fail if incompatible versions were
installed.
- If the CMake-based build script can not find and download the OpenCV package
that is supported on your platform, or if you want to use a custom build of
the OpenCV library, see how to [Use Custom OpenCV Builds](https://github.com/openvinotoolkit/openvino/wiki/CMakeOptionsForCustomCompilation#Building-with-custom-OpenCV).
- To build the OpenVINO Runtime Python API:
1. Install all additional packages (e.g., cython and opencv) listed in the
`/src/bindings/python/src/compatibility/openvino/requirements-dev.txt` file:
```sh
pip install -r requirements-dev.txt
```
2. Enable the `-DENABLE_PYTHON=ON` option in the CMake step above (Step 4). To specify an exact Python version, use the following
options:
```
-DPYTHON_EXECUTABLE=`which python3.7` \
-DPYTHON_LIBRARY=/usr/lib/x86_64-linux-gnu/libpython3.7m.so \
-DPYTHON_INCLUDE_DIR=/usr/include/python3.7
```
3. To build a wheel package (.whl), enable the `-DENABLE_WHEEL=ON` option in the CMake step above (Step 4):
4. After the build process finishes, export the newly built Python libraries to the user environment variables:
```
export PYTHONPATH=PYTHONPATH:<openvino_repo>/bin/intel64/Release/python_api/python3.7
export LD_LIBRARY_PATH=LD_LIBRARY_PATH:<openvino_repo>/bin/intel64/Release
```
or install the wheel with pip:
```
pip install <openvino_repo>/build/wheel/openvino-2022.2.0-000-cp37-cp37-manylinux_2_35_x86_64.whl
```
- To switch the CPU and GPU plugins off/on, use the `cmake` options
`-DENABLE_INTEL_CPU=ON/OFF` and `-DENABLE_INTEL_GPU=ON/OFF` respectively.
- OpenVINO runtime compilation options:
`-DENABLE_OV_ONNX_FRONTEND=ON` enables the building of the ONNX importer.
<p>
</details>
<details><summary>Mac</summary>
<p>
This guide shows how to build OpenVINO Runtime for later inference on Apple with:
<details><summary>Intel CPU</summary>
<p>
This can be done using two ways:
- Compile on Intel CPU host using native compilation. Note, that [Build steps](#build-steps) show this scenario.
- Cross-compile on OSX Apple Silicon.
1. Create a build folder:
```sh
mkdir build && cd build
```
2. (CMake configure) OpenVINO project uses a CMake-based build system. In the created `build` directory, run `cmake` to fetch project dependencies and create build rules:
```sh
cmake -DCMAKE_BUILD_TYPE=Release ..
```
> **Note:** By default OpenVINO CMake scripts try to introspect the system and enable all possible functionality based on that. You can look at the CMake output and see warnings, which show that some functionality is turned off and the corresponding reason, guiding what to do to install additionally to enable unavailable functionality. Additionally, you can change CMake options to enable / disable some functionality, add / remove compilation flags, provide custom version of dependencies like TBB, PugiXML, OpenCV, Protobuf. Please, read [CMake options for custom compilation](CMakeOptionsForCustomCompilation) for this information.
3. (CMake build) Build OpenVINO project:
```sh
cmake --build . --config Release --jobs=$(nproc --all)
```
All built binaries are located in `<openvino_source_dir>/bin/intel64/Release/` and wheel packages are located in `<openvino_build_dir>/wheels`.
4. (Optional install) Once you have built OpenVINO, you can install artifacts to a preferred location:
```sh
cmake -DCMAKE_INSTALL_PREFIX=<installation location> -P cmake_install.cmake
```
### Cross-compilation
Since OSX version 11.x and Xcode version 12.2, the Apple development tools allows to compile arm64 code on x86 hosts and vice-versa. Based on this, OpenVINO can be compiled even on Apple Silicon machines, then such artifacts can be run on both Intel CPU hosts and Apple Silicon hosts (using [Rosetta]). For this, try to compile OpenVINO using the steps above, but adding `-DCMAKE_OSX_ARCHITECTURES=x86_64 -DENABLE_INTEL_MYRIAD=OFF` on cmake configure stage. But, **don't enable any system library usage explicitly** via CMake options, because they have `arm64` architecture, e.g.:
```sh
file /opt/homebrew/Cellar/tbb/2021.5.0_2/lib/libtbb.12.5.dylib
/opt/homebrew/Cellar/tbb/2021.5.0_2/lib/libtbb.12.5.dylib: Mach-O 64-bit dynamically linked shared library arm64
```
If you will see the errors like below:
```sh
ld: warning: ignoring file /opt/homebrew/lib/libopencv_imgproc.4.6.0.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
"cv::Mat::Mat(cv::Size_<int>, int, void*, unsigned long)", referenced from:
_image_resize in opencv_c_wrapper.cpp.o
_image_save in opencv_c_wrapper.cpp.o
....
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
Disable its usage in cmake or totally remove such library from the system (e.g. `brew uninstall opencv`), because it's pure arm64 and cannot be used to compile x86_64 binaries.
> **Note:** using such way OpenVINO Intel CPU plugin can be cross-compiled, because MYRIAD plugin cannot be linked against `arm64` version of `libusb`
Or you have to explicitly find / compile x86_64 (or even `universal2`) dependencies by yourself and pass it to OpenVINO cmake scripts. E.g. compile oneTBB using additional option `-DCMAKE_OSX_ARCHITECTURES="x86_64;arm64"`, install and then set `export TBBROOT=<universal oneTBB install root>` which will be used by OpenVINO.
<p>
</details>
<details><summary>ARM</summary>
<p>
There are two options how to use OpenVINO on Apple Silicon:
- (Native) Compile OpenVINO for arm64 architecture with extra module [OpenVINO ARM plugin](https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/arm_plugin) location in [OpenVINO Contrib](https://github.com/openvinotoolkit/openvino_contrib). Note, build steps will cover this as a default scenario.
- (Rosetta) Compile Intel CPU plugin `x86_64` architecture and run under [Rosetta].
1. Clone contrib repo:
```sh
cd ..
git clone https://github.com/openvinotoolkit/openvino_contrib.git
cd openvino_contrib
git submodule update --init --recursive
cd ../openvino
```
2. Create a build folder:
```sh
mkdir build && cd build
```
3. (CMake configure) OpenVINO project uses a CMake-based build system. In the created `build` directory, run `cmake` to fetch project dependencies and create build rules:
```sh
cmake -DCMAKE_BUILD_TYPE=Release -DIE_EXTRA_MODULES=../openvino_contrib/modules/arm_plugin ..
```
> **Note:** By default OpenVINO CMake scripts try to introspect the system and enable all possible functionality based on that. You can look at the CMake output and see warnings, which show that some functionality is turned off and the corresponding reason, guiding what to do to install additionally to enable unavailable functionality. Additionally, you can change CMake options to enable / disable some functionality, add / remove compilation flags, provide custom version of dependencies like TBB, PugiXML, OpenCV, Protobuf. Please, read [CMake options for custom compilation](CMakeOptionsForCustomCompilation) for this information.
4. (CMake build) Build OpenVINO project:
```sh
cmake --build . --config Release --jobs=$(nproc --all)
```
All built binaries are located in `<openvino_source_dir>/bin/<arm64 | intel64>/Release/` and wheel packages are located in `<openvino_build_dir>/wheels`.
5. (Optional install) Once you have built OpenVINO, you can install artifacts to a preferred location:
```sh
cmake -DCMAKE_INSTALL_PREFIX=<installation location> -P cmake_install.cmake
```
### Building x86_64 binaries
Since OSX version 11.x and Xcode version 12.2, the Apple development tools allows to compile arm64 code on x86 hosts and vice-versa. Based on this, OpenVINO can be compiled as x86_64 binary, then run on Apple Silicon hosts using [Rosetta]. For this, first of all Rosetta must be installed:
```sh
softwareupdate --install-rosetta
```
Then try to compile OpenVINO using the steps above, but adding `-DCMAKE_OSX_ARCHITECTURES=x86_64` on cmake configure stage. But, **don't enable any system library usage explicitly** via CMake options, because they have `arm64` architecture, e.g.:
```sh
file /opt/homebrew/Cellar/tbb/2021.5.0_2/lib/libtbb.12.5.dylib
/opt/homebrew/Cellar/tbb/2021.5.0_2/lib/libtbb.12.5.dylib: Mach-O 64-bit dynamically linked shared library arm64
```
The same for other external dependencies like `libusb`. If you want to enable extra functionality like enable MYRIAD plugin build, you need to provide either x86_64 or universal2 `libusb` library. All other steps are the same as for usual compilation: build, install.
> **Note:** since you are building with `universal2` python libraries, wheel package is created with name `openvino-2022.3.0-000-cp39-cp39-macosx_12_0_universal2.whl` and have proper `universal2` tags, so can *potentially* be used on both Apple Silicon and Intel CPU.
<p>
</details>
<p>
</details>
## Run application
[CMake]:https://cmake.org/download/
[Install Intel® Graphics Compute Runtime for OpenCL™ Driver package 19.41.14441]:https://github.com/intel/compute-runtime/releases/tag/19.41.14441
[Rosetta]:https://support.apple.com/en-us/HT211861

22
docs/dev/coding_style.md Normal file
View File

@ -0,0 +1,22 @@
# OpenVINO Coding Guidelines
## Coding style
Majority of OpenVINO components use `clang-format-9` for code style check.
The code style is based on Google Code style with some differences. All differences are described in the configuration file:
https://github.com/openvinotoolkit/openvino/blob/69f709028a5f8da596d1d0df9a0101e517c35708/src/.clang-format#L1-L28
To fix code style on your local machine, you need to have installed `clang-format-9` tool and be sure that CMake option `ENABLE_CLANG_FORMAT` is enabled.
If all dependencies are resolved, `clang_format_fix_all` target can be used to fix all code style issues.
## Naming style
OpenVINO has a strict rules for naming style in public API. All classes should be started from capital letter, methods and functions are named in `snake_case` style.
To check the naming style `ncc` tool is integrated inside the OpenVINO. The detailed information about naming style can be found in the [configuration file](../../cmake/developer_package/ncc_naming_style/openvino.style).
In order to activate this tool you need to have `clang` on the local machine and enabled CMake option `ENABLE_NCC_STYLE`.
After that `ncc_all` target can be used to check the naming style.
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

75
docs/dev/conditional_compilation.md Normal file
View File

@ -0,0 +1,75 @@
# OpenVINO Conditional Compilation
## Contents
- [Introduction](#introduction)
- [Building for different models](#building-for-different-models)
- [Building for devices with different ISA](#building-for-devices-with-different-isa)
- [Limitations](#limitations)
## Introduction
Conditional compilation can significantly reduce OpenVINO™ binaries size by excluding unnecessary components for particular models inference.
The following components can be excluded from the build:
* Openrations and graph transformations in OpenVINO and plugins
* jit kernels in a CPU plugin
* arbitrary code that is not used for particular model inference
However, conditional compilation has a significant drawback - **the resulting OpenVINO Runtime will work only with a limited set of models and devices.**
### Setup environment
To take advantage of conditional compilation, install the following tools:
* [Python](https://www.python.org)
## Building for different models
Conditional compilation has two stages:
* Collecting information about code usage
* Building the result binaries without unused components or parts
```mermaid
flowchart LR
subgraph collect[Collect statistic]
cmake_collect[mkdir build\ncd build\ncmake -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT ..]
build_collect[cmake --build .\ncmake --build . --target sea_itt_lib]
run_collect["python ../thirdparty/itt_collector/runtool/sea_runtool.py --bindir ${OPENVINO_LIBRARY_DIR} \ \n-o ${MY_MODEL_RESULT} ! ./benchmark_app -niter 1 -nireq 1 -m ${MY_MODEL}.xml\n..."]
cmake_collect-->build_collect
build_collect-->run_collect
run_collect-->run_collect
end
subgraph result[Build cutted OpenVINO]
cmake_res["cmake -DSELECTIVE_BUILD=ON \ \n-DSELECTIVE_BUILD_STAT=${ABSOLUTE_PATH_TO_STATISTICS_FILES}/*.csv \ \n -DENABLE_PROFILING_ITT=OFF .."]
build_res[cmake --build .]
cmake_res-->build_res
end
collect --> result
```
To apply conditional compilation, follow the steps below:
1. Perform code usage analysis:
1. Run the CMake tool with the following options: `-DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT`.
2. Select several models to be used in a specific application or target device.
3. Use the `sea_itt_lib` target to build the ITT collector.
4. Run the target application under the ITT collector for code usage analysis for each model. Statistics are generated in the `.csv` format.
`python thirdparty/itt_collector/runtool/sea_runtool.py --bindir ${OPENVINO_LIBRARY_DIR} -o ${MY_MODEL_RESULT} ! ./benchmark_app -niter 1 -nireq 1 -m ${MY_MODEL}.xml`
2. Build the resulting binaries:
1. Run the CMake tool with the following options: `-DSELECTIVE_BUILD=ON -DSELECTIVE_BUILD_STAT=${ABSOLUTE_PATH_TO_STATISTICS_FILES}/*.csv -DENABLE_PROFILING_ITT=OFF`
2. `cmake --build <cmake_build_directory>`
The `-niter 1 -nireq 1` flags are highly recommended for the benchmark_app. Otherwise, the trace files will be very large.
If you are using an application other than benchmark_app, remember to limit the number of inference requests and iterations.
## Building for devices with different ISA
Building for devices with different ISA is quite similar to building for different models (see the previous chapter).
The differences are only in the code usage analysis step. The analysis step should be performed on target devices and all CSV files with statistics should be copied to the build machine. These files will be used for the final build.
## Limitations
* Currently, Ninja build system is not supported for the conditional compilation build.
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

10
docs/dev/debug_capabilities.md Normal file
View File

@ -0,0 +1,10 @@
# OpenVINO Debug Capabilities
OpenVINO components provides different debug capabilities, to get more infromation please read:
* [OpenVINO Model Debug Capabilities](https://docs.openvino.ai/latest/openvino_docs_OV_UG_Model_Representation.html#model-debug-capabilities)
* [OpenVINO Pass Manager Debug Capabilities](#todo)
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

17
docs/dev/get_started.md Normal file
View File

@ -0,0 +1,17 @@
# Get started
Welcome to the OpenVINO Developer guides. These documents teach you how to build OpenVINO, provide the general ideas about OpenVINO architecture and help to start contribution to OpenVINO.
If you're brand new to OpenVINO and want to jump into code, start with the [Build OpenVINO guilde](./build.md).
And check out these other resources to get more infromation about OpenVINO:
* [OpenVINO official user documentation](https://docs.openvino.ai/)
* [OpenVINO repository structure](./intex.md#openvino-repository-structure)
* [OpenVINO core components](../../src/README.md)
* [OpenVINO contribution guideline](../../CONTRIBUTING.md)
* [OpenVINO conditional compilation](./conditional_compilation.md)
* [OpenVINO Tutorials](../../README.md#tutorials)
## See also
* [OpenVINO Developer documentation](./index.md)
* [OpenVINO Samples](../../samples)

106
docs/dev/index.md Normal file
View File

@ -0,0 +1,106 @@
# OpenVINO™ Developer Documentation
Welcome to the OpenVINO™ Toolkit Developer Documentation. This documentation helps to deeper understand the ideology of OpenVINO, provides the complete information about ideas which are applied inside.
The OpenVINO™ Toolkit provides a functionality to load model from different frameworks and run it on different accelerators.
```mermaid
flowchart LR
subgraph models [Models]
ir[("IR (*.xml)")]
onnx[("ONNX (*.onnx)")]
paddle[("PaddlePaddle (*.pdmodel)")]
tf[("Tensorflow (*.pb)")]
style ir fill:#427cb0
style onnx fill:#427cb0
style paddle fill:#427cb0
style tf fill:#427cb0
style models fill:#aeb6ba
click onnx "https://github.com/onnx/onnx"
end
mo{{Model Optimizer}}
style mo fill:#6c9f7f
onnx--convert--->mo
paddle--convert--->mo
tf--convert--->mo
mo--->ir
subgraph plugins [OV Plugins]
auto(["AUTO"])
cpu(["Intel_CPU"])
gpu(["Intel_GPU"])
end
subgraph frontends [OV Frontends]
ir_fe["IR Frontend"]
onnx_fe["ONNX Frontend"]
paddle_fe["Paddle Frontend"]
end
openvino(openvino library)
ir--Read ir---ir_fe
onnx--Read onnx--- onnx_fe
paddle--Read paddle---paddle_fe
ir_fe--->openvino
onnx_fe--->openvino
paddle_fe--->openvino
openvino--infer--->cpu
openvino--infer--->gpu
openvino--infer--->auto
```
## [Get started](./get_started.md)
* [Build OpenVINO](./build.md)
* How to:
* [Add new operation](../../src/core/docs/operation_enabling_flow.md)
* [Add new transformation](#todo)
* [Get code coverage report](./test_coverage.md)
* [OpenVINO contribution guideline](../../CONTRIBUTING.md)
* [OpenVINO Debug Capabilities](./debug_capabilities.md)
## OpenVINO repository structure
The trend in the organization of structure repository is the idea that components should contain all dependencies inside (like thirdparty, testts, documentation and etc.). Below the abstract component structure is presented with comments and marks for optional folders.
```
ov_component/ // Component folder
cmake/ // (optional) CMake scripts which are related only for this component
dev_api/ // (optional) Developer API is used in case if component wants to provide some API for internal developers
docs/ // (optional) Contains detailed component documentation
include/ // (optional) Public component API
src/ // Sources of the component
tests/ // Tests on the component
thirdparty/ // (optional) Third-party dependencies
CMakeLists.txt // Main CMake script
README.md // (optional) Endtry point for the developer documentation
```
The OpenVINO Repository compositions from next components, click on the component name to get more information about it:
<pre>
<code>
<a href="../../README.md">openvino/</a> // OpenVINO Repository
.ci/ // CI settings for Azure
.github/ // Github actions and PR templates
cmake/ // Global CMake scripts
docs/ // OpenVINO documentation
licensing/ // Licenses
samples/ // OpenVINO samples
scripts/ // Helper scripts
<a href="../../src/README.md">src/</a> // Folder with core OpenVINO components
tests/ // Infrastructure tests, which validate full pipelines
thirdparty/ // Common third-party dependencies
tools/ // OpenVINO tools
</code>
</pre>
## Features
* [Conditional Compilation](./conditional_compilation.md)
## See also
* [OpenVINO™ README](../../README.md)

66
docs/dev/test_coverage.md Normal file
View File

@ -0,0 +1,66 @@
# OpenVINO coverage report build
The coverage report is generated using Lcov tool and based on profile data generated by GCC.
The generated reports are in HTML form and located in `<openvino_build>/coverage`. The reports are generated for the following components:
1. `inference` - Inference component is a part of OpenVINO Runtime library.
1. `core` - Core component is a part of OpenVINO Runtime library.
1. `low_precision_transformations` - Low Precision transformations is a part of OpenVINO Runtime library.
1. `openvino_transformations` - Common transformations is a part of OpenVINO Runtime library.
1. `inference_engine_legacy` - legacy OpenVINO library
1. `preprocessing` - OpenVINO G-API based preprocessing plugin.
1. `snippets` - OpenVINO snippets.
1. OpenVINO open-sources plugins:
- `hetero_plugin` - Heterogeneous plugin.
- `multi_plugin` - Multi plugin.
- `auto_plugin` - Auto plugin.
- `template_plugin` - Template plugin.
- `intel_gpu_plugin` - GPU plugin.
- `intel_cpu_plugin` - CPU plugin.
- `intel_gna_plugin` - GNA plugin.
## Build with profiling data support
To build coverage report, compile OpenVINO with an additional CMake option `-DENABLE_COVERAGE=ON`:
```bash
$ cmake -DENABLE_COVERAGE=ON .
```
And build OpenVINO as usual.
## Generate coverage report
In order to generate coverage reports, first of all, the tests must be run. Depending on how many tests are run, the better covegare percentage can be achieved. E.g. for `openvino` component, `InferenceEngineUnitTests`, `ieUnitTests`, `ieFuncTests` must be run as well as plugin tests.
```bash
$ ctest -V -L IE
```
After sufficient number of tests are executed, the coverage numbers can be calculated. In order to do this, run:
```bash
$ make ov_coverage
```
The following tree of reports are generated:
```bash
$ find coverage -maxdepth 2 -name index.html
coverage/core/index.html
coverage/transformations/index.html
coverage/paddle_frontend/index.html
coverage/tf_frontend/index.html
coverage/openvino/index.html
coverage/onnx_frontend/index.html
coverage/legacy/index.html
coverage/ir_frontend/index.html
coverage/low_precision_transformations/index.html
coverage/template_plugin/index.html
coverage/inference/index.html
coverage/frontend_common/index.html
```
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

72
src/README.md Normal file
View File

@ -0,0 +1,72 @@
# OpenVINO™ Core Components
This section provides references and information about OpenVINO core components.
```
bindings/ // OpenVINO bindings
cmake/ // Common cmake scripts
common/ // Common components
core/ // OpenVINO core component provides model representation, operations and other core functionality
frontends/ // OpenVINO frontends
inference/ // Provides API for model inference
plugins/ // OpenVINO plugins
tests/ // A backed of tests binaries for core and plugins
tests_deprecated/ // Deprecated tests
```
## OpenVINO Runtime library
OpenVINO Runtime is a common OpenVINO library which provides functionality for Neural Network inference. The library includes next parts:
```mermaid
flowchart LR
subgraph openvino [openvino library]
core
inference
transformations[Common transformations]
lp_transformations[LP transformations]
frontend_common
style frontend_common fill:#7f9dc0,stroke:#333,stroke-width:4px
style transformations fill:#3d85c6,stroke:#333,stroke-width:4px
style lp_transformations fill:#0b5394,stroke:#333,stroke-width:4px
style core fill:#679f58,stroke:#333,stroke-width:4px
style inference fill:#d7a203,stroke:#333,stroke-width:4px
end
```
* [core](./core/README.md) is responsible for model representation, contains a set of supported OpenVINO operations and base API for model modification.
* [inference](./inference) provides the API for model inference on different accelerators.
* Transformations:
* [common transformations](../../src/common/transformations) - a set of common transformations which are used for model optimization
* [low precision transformations](../../src/common/low_precision_transformations) - a set of transformations which are needed to optimize quantized models
* **frontend_common** provides frontend common API which allows to support frontends for different frameworks.
## OpenVINO Frontends
OpenVINO Frontends allow to convert model from framework to OpenVINO representation.
* [ir](./frontends/ir)
* [onnx](./frontends/onnx)
## OpenVINO Plugins
Plugins provide a support of hardware device
* [intel_cpu](./plugins/intel_cpu)
## OpenVINO Bindings
OpenVINO provides bindings for several languages:
* [c](./bindings/c)
* [python](./bindings/python)
## Core developer topics
* [OpenVINO architecture](./docs/architecture.md)
* [Plugin Development](https://docs.openvino.ai/latest/openvino_docs_ie_plugin_dg_overview.html)
* [Thread safety](#todo)
* [Performance](#todo)
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

38
src/core/README.md Normal file
View File

@ -0,0 +1,38 @@
# OpenVINO™ Core
OpenVINO Core is a part of OpenVINO Runtime library.
The component is responsible for:
* Model representation - component provides classes for manipulation with models inside the OpenVINO Runtime. For more information please read [Model representation in OpenVINO Runtime User Guide](https://docs.openvino.ai/latest/openvino_docs_OV_UG_Model_Representation.html)
* Operation representation - contains all from the box supported OpenVINO operations and opsets. For more information read [Operations enabling flow guide](./docs/operation_enabling_flow.md).
* Model modification - component provides base classes which allow to develop transformation passes for model modification. For more information read [Transformation enabling flow guide](#todo).
OpenVINO Core supports [conditional compilation feature](../../docs/dev/conditional_compilation.md) and uses [the common coding style rules](../../docs/dev/coding_style.md).
## Key person
People from the [openvino-ngraph-maintainers](https://github.com/orgs/openvinotoolkit/teams/openvino-ngraph-maintainers) allows to approve and merge PRs to the core component. These guys can help in case of any questions about core component.
## Components
OpenVINO Core has the next structure:
* [builders](./builder) is obsolete component which provides helper methods for operation creations. Please don't use this API, and use public OpenVINO API instead.
* [dev_api](./dev_api) contains developer API. In order to use this API, you need to link your component against `openvino::runtime::dev`.
* [docs](./docs) contains developer documentation pages for the component.
* [include](./include) contains public API. Detailed information about provided API can be found [here](#TODO).
* [reference](./reference) is a library which provides reference implementations for all supported operations. Operations with evaluate method use these implementations inside.
* [shape_inference](./shape_inference) library contains implementation of shape inference for OpenVINO operations.
* [src](./src) folder contains sources of the core component.
* [tests](./tests) contains tests for OpenVINO Core components. More information about OpenVINO Core tests can be found [here](./docs/tests.md).
## Debug capabilities
OpenVINO Core contains a set of different debug capabilities which allow to make developer live a little bit easier.
Different debug capabilities supports to collect information about object statuses in the process of OpenVINO Runtime execution.
* OpenVINO Model debug capabilities are described in the [OpenVINO Model User Guide](https://docs.openvino.ai/latest/openvino_docs_OV_UG_Model_Representation.html#model-debug-capabilities).
## Tutorials
TODO
## See also
* [OpenVINO™ README](../../README.md)
* [Developer documentation](../../docs/dev/index.md)

37
src/core/docs/operation_enabling_flow.md Normal file
View File

@ -0,0 +1,37 @@
# OpenVINO Operation Enabling Flow
1. Implement operation "shell" in the `src/core/[src|include]/op/`:
* Implement constructor(s)
* Implement `validate_and_infer_types` method which should support dynamic input tensor(s) (with partially dynamic shapes). For more information read [OpenVINO Shape propagation guide](./shape_propagation.md)
* Implement `visit_attributes` method
* Implement `clone_with_new_inputs` method. The generated operation version must be explicitly specified and be equal to the operation version being added
* In `*.hpp` file add:
```cpp
OPENVINO_OP("<Operation_name>", "opset_name", <Parent_op> /* Not needed if operation is inherited from ov::Op */);
```
If operation is inherited from Operation with `BWDCMP_RTTI_DECLARATION` please add next declaration to header file:
```cpp
BWDCMP_RTTI_DECLARATION;
```
* In `*.cpp` if operation is inherited from Operation with `BWDCMP_RTTI_DECLARATION` please add:
```cpp
BWDCMP_RTTI_DEFINITION(op::vX::<Operation>);
```
* To support conditional compilation add following for each Operation method in `*.cpp` file:
```cpp
NGRAPH_OP_SCOPE(<operation_version>_<operation_name>_<method_name>);
```
* Add shape infer unit-tests to the `src/core/tests/type_prop/`
2. Add operation to the dedicated opset file `src/core/include/openvino/opsets/opsetX_tbl.hpp`
3. Implement `evaluate` method for the operation (reference implementation) in the `openvino/core/reference/[src|include]/ngraph/runtime/reference/`. Reference implementation can be called from Template plugin or from OpenVINO (`src/core/src/op/your_operation_name.cpp/hpp`).
To not increase the binary size of openvino lib it should be placed in Template plugin unless you are directly asked to put it in the OpenVINO core. While adding reference implementation the following points should be considered:
* The method should avoid using the template parameters if possible. However, for the small operations like activation functions it is acceptable.
* The method should be instantiated for practical data types only.
* If the method can be implemented without defining strict data types (for example, data movement operations like Concat or Split) then it should be implemented in a type-agnostic way.
## See also
* [OpenVINO™ Core README](../README.md)
* [OpenVINO™ README](../../../README.md)
* [Developer documentation](../../../docs/dev/index.md)

114
src/core/docs/shape_propagation.md Normal file
View File

@ -0,0 +1,114 @@
# Shape and value propagation in ov::Model
`ov::Model` represents model with operations and their relations describing how data would flow through the function during the inference.
Data size estimations for output tensors of each operation is required for efficient model execution.
Such an estimation depends on the data type and shape of intermediate tensors of the model.
For OpenVINO supported operations you can find type and shape propagation rules in the documentation of the opset of your choice
Shape propagation is a process of output shape calculation for each operation in the graph.
It starts from all the `Parameter` operations in the `ov::Model` and continues to calculate output shapes for all the operations of the `ov::Model`
in the topological order.
Method `ov::Op::validate_and_infer_types` is responsible for operation validation, shape and type propagation.
## OpenVINO shapes representation
Shapes could be represented as `ov::Shape` or as `ov::PartialShape` in OpenVINO.
`ov::Shape` could be used in case rank and all the shape dimensions are static.
`ov::PartialShape` could be used for both static and dynamic shape representations.
Depending on the amount of known information about the rank and shape dimensions we can formulate shapes in different ways.
Here are some examples of shape creation:
1. Shape with undefined rank:
```cpp
ov::PartialShape::dynamic();
```
2. Shape with defined rank and fully undefined dimensions:
```cpp
ov::PartialShape({1, 3, Dimension::dynamic(), Dimension::dynamic()}); // rank == 4, two static dimensions and two fully dynamic dimension
ov::PartialShape({Dimension::dynamic(), Dimension::dynamic()}); // rank == 2, all dimensions are fully dynamic
ov::PartialShape::dynamic(5); // rank == 5, all dimensions are fully dynamic
```
3. Shape with defined rank and undefined dimensions with specified range:
```cpp
ov::PartialShape({{1, 8}, 3, 400, 400}); // rank == 4, first dimension is dynamic and can be any number from 1 to 8 inclusive, all the other dimensions are static
ov::PartialShape({{2, -1}, 3, 400, 400}); // rank == 4, first dimension is dynamic and can be any number larger or equal 2, all the other dimensions are static
ov::PartialShape({{-1, 8}, 3, 400, 400}); // rank == 4, first dimension is dynamic and will not be larger than 8, all the other dimensions are static
```
4. Shape with defined rank and fully defined dimensions:
```cpp
ov::PartialShape({1, 3, 400, 400}); // rank == 4
ov::Shape({1, 3, 400, 400}); // rank == 4
ov::PartialShape({5}); // rank == 1, one-dimensional tensor with five values in it
ov::PartialShape({}); // rank == 0, scalar -- zero-dimensional tensor with single value in it
ov::PartialShape({1}); // rank == 1, one-dimensional tensor with single value in it
ov::PartialShape({1, 3, 0, 0}); // rank == 4, four-dimensional tensor with no value in it empty tensor
```
`ov::PartialShape` stores `ov::Rank` for the shape rank. It could be fully undefined or static.
`ov::PartialShape` stores shape values as a vector of `ov::Dimension` for the case if `ov::Rank` is static.
`ov::Dimension` is represented by an interval which is a pair of values -- maximum and minimum value for the dimension, for both static and dynamic cases.
They are equal for static dimensions and are set to different values for range and fully dynamic dimensions.
1. dynamic ov::Dimension:
```cpp
ov::Dimension::dynamic();
ov::Dimension(-1); // special value for Dimension
ov::Dimension{0, MAX_INT};
```
2. interval ov::Dimension:
```cpp
ov::Dimension{1, 10};
ov::Dimension{0, MAX_INT};
```
3. static ov::Dimension
```cpp
ov::Dimension{3};
ov::Dimension{3, 3};
```
We allow creation of negative dimension on `ov::Interval`, `ov::Dimension`, `ov::PartialShape` level due to historical reasons,
but there is no point in it in terms of shape propagation.
> NOTE: Dimension(-1) is equal to Dimension::dynamic(). There is no way to create static Dimension with value -1
## Shape propagation
The goal of shape propagation is to keep and pull all the known information about output shapes dimensions from the operation semantics.
During the shape propagation we may need input ranks/shapes for the operation or input values.
It is easy to lose detailed shape information. To provide practical guidance we have collected popular mistakes and the ways to avoid them:
Shape calculation DOs and DON`Ts:
1. Using `ov::Shape` class to work with shape instead of `ov::PartialShape` lead to losing detailed shape information
2. Calling `ov::PartialShape::to_shape()` method which converts `ov::PartialShape` object to `ov::Shape` object may result in exceptions thrown while running the code for dynamic `ov::Model`.
To avoid that introduce a check if the shape is dynamic before doing the conversion.
```
ov::PartialShape shape = ...;
if (shape.is_static())
```
In order to make the code work for dynamic shapes too express all the needed shape calculations using capabilities of `ov::PartialShape`, `ov::Rank` and `ov::Dimension` classes.
3. To get the rank of the shape instead of `ov::Shape::size()` method do the following:
```
ov::PartialShape shape = ...;
ov::Rank = rank = shape.rank();
if (rank.is_static())
auto rank_value = rank.get_length();
...
```
4. Mathematical operators are overloaded for the `ov::Dimension` class and you can perform all kinds of manipulations on them.
It would result in correct calculations for both static and dynamic `ov::Dimension`.
For example, multiplication of the first two dimensions of the shape is easy as:
```
ov::PartialShape shape = ...;
ov::Rank = rank = shape.rank();
if (rank.is_static() && rank.get_length() > 2)
ov::Dimension product_of_first_and_second_dims = shape[0] * shape[1];
```
## See also
* [OpenVINO™ Core README](../README.md)
* [OpenVINO™ README](../../../README.md)
* [Developer documentation](../../../docs/dev/index.md)

20
src/core/docs/tests.md Normal file
View File

@ -0,0 +1,20 @@
# OpenVINO Core Tests
OpenVINO Core is covered by tests from the binary `ov_core_unit_tests`. This binary can be built by the target with the same name.
OpenVINO Core [tests](../tests/) have next structure:
* `conditional_compilation` - tests cover conditional compilation feature
* `frontend` - PaddlePaddle frontend tests (should be removed in the nearest future)
* `models` - test models
* `op_eval` - tests cover evaluate methods
* `pass` - tests covers common transformations
* `type_prop` - type and shape propagation tests
* `visitors` - tests covers visitor API for all supported operations
These tests can be disabled by CMake option `ENABLE_OV_CORE_UNIT_TESTS`, this option by default has the same value as `ENABLE_TESTS` option.
## See also
* [OpenVINO™ Core README](../README.md)
* [OpenVINO™ README](../../../README.md)
* [Developer documentation](../../../docs/dev/index.md)
* [Test coverage measurement](../../../docs/dev/test_coverage.md)

117
src/docs/architecture.md Normal file
View File

@ -0,0 +1,117 @@
# OpenVINO Architecture
This guide encompasses existed architectural ideas and guidelines for architecture development.
## Main architectual design concepts
OpenVINO development is based on next principals:
* Performance and Scalability - OpenVINO should provide the optimal performance in different customer cases
* Availability and Resilience - OpenVINO is high quality product and we should demonstrate this high level in different cases
* Security - OpenVINO is applied in different applications, it is why we have a strict validation pipelines which allow to find vulnerabilities on the development stage.
Each OpenVINO component is projected with using DOTADIW (Do One Thing And Do It Well) approach, it means that OpenVINO is highly granularity product and we are avoiding mixing of responsibilities for different components.
### OpenVINO Component responsibilities
```mermaid
flowchart TB
subgraph tools [Tools]
mo{{Model Optimizer}}
pot{{PoT}}
style mo fill:#6c9f7f
style pot fill:#6c9f7f
end
subgraph tutorials [Tutorials]
samples[Samples]
demos[Demos]
notebooks[Notebooks]
end
subgraph api [API]
cpp[C++]
subgraph bindings [Bindings]
c_api[C]
python_api[Python]
end
end
subgraph plugins [OV Plugins]
auto(["AUTO"])
cpu(["Intel_CPU"])
gpu(["Intel_GPU"])
end
subgraph frontends [OV Frontends]
direction TB
ir_fe["IR Frontend"]
onnx_fe["ONNX Frontend"]
paddle_fe["Paddle Frontend"]
end
openvino(openvino library)
frontends--Read model--->openvino
openvino--API--->api
openvino--infer--->plugins
```
All OpenVINO components can be logicaly divided to several groups:
* **OpenVINO library** is a core library which provides the OpenVINO API.
* **Bindings** are libraries which provide OpenVINO API for different languages.
* **Frontends** are libraries which converts model from framework representation to OpenVINO format.
* **Plugins** are components which allow to run models on different accelerators.
* **Tools** a set of different components which provide additional opportunities for OpenVINO.
* **Tutorials** different applications which show how user can work with OpenVINO.
### OpenVINO Inference pipeline
OpenVINO inference pipeline has several objects which are responsible for different stages of model inference.
* `Core` object is a global context for Model inference, allows to create the unique environment for inference pipeline.
* `read_model()` finds the right frontend and converts Framework model to OpenVINO representation.
* `compile_model()` loads hardware plugin and compiles model on the device.
* `CompiledModel` represents the hardware specified model, this model can contain additional plugin optimization and on this stage model already was compiled for the specific device.
* `InferRequest` provides an interface for running inference.
```mermaid
flowchart TB
subgraph core [ov::Core core]
read_model["core.read_model()"]
compile_model["core.compile_model()"]
end
fw_model[(Framework model)]
ov_model[ov::Model]
subgraph compiled_models [Compiled models]
cpu_model["CPU"]
gpu_model["GPU"]
auto_model["AUTP"]
end
subgraph infer_reguests [InferRequests]
cpu_req["CPU"]
gpu_req["GPU"]
auto_req["AUTP"]
end
result[Results]
fw_model--->read_model
read_model--->ov_model
ov_model-->compile_model
fw_model--->compile_model
compile_model--"compile_model(CPU)"--->cpu_model
compile_model--"compile_model(GPU)"--->gpu_model
compile_model--"compile_model(AUTO)"--->auto_model
cpu_model--"create_infer_request()"--->cpu_req
gpu_model--"create_infer_request()"--->gpu_req
auto_model--"create_infer_request()"--->auto_req
cpu_req--"infer()"-->result
gpu_req--"infer()"-->result
auto_req--"infer()"-->result
```
## See also
* [OpenVINO Developer documentation](./index.md)