From 790f74c01cc51817e0838c857d666a12b2218cbe Mon Sep 17 00:00:00 2001 From: Ilya Churaev Date: Thu, 16 Mar 2023 12:39:06 +0400 Subject: [PATCH] Update building and plugin testing docs (#16333) * Update building and plugin testing docs * Fixed typo --- docs/IE_PLUGIN_DG/Building.md | 40 +++--------------------------- docs/IE_PLUGIN_DG/PluginTesting.md | 28 ++++++++------------- docs/IE_PLUGIN_DG/layout.xml | 10 ++++---- 3 files changed, 20 insertions(+), 58 deletions(-) diff --git a/docs/IE_PLUGIN_DG/Building.md b/docs/IE_PLUGIN_DG/Building.md index 66256694b1d..ebd594291fd 100644 --- a/docs/IE_PLUGIN_DG/Building.md +++ b/docs/IE_PLUGIN_DG/Building.md @@ -18,20 +18,12 @@ Once the commands above are executed, the OpenVINO Developer Package is generate - `OpenVINODeveloperPackageConfig-version.cmake` - a file with a package version. - `targets_developer.cmake` - an automatically generated file which contains all targets exported from the OpenVINO build tree. This file is included by `OpenVINODeveloperPackageConfig.cmake` to import the following targets: - Libraries for plugin development: - * `openvino::ngraph` - shared OpenVINO library - * `openvino::openvino_gapi_preproc` - shared library with OpenVINO preprocessing plugin - * `openvino::core::dev` - interface library with OpenVINO Core development headers - * `openvino::runtime::dev` - interface library with OpenVINO Plugin API headers + * `openvino::runtime` - shared OpenVINO library + * `openvino::runtime::dev` - interface library with OpenVINO Developer API * `openvino::pugixml` - static Pugixml library * `openvino::xbyak` - interface library with Xbyak headers * `openvino::itt` - static library with tools for performance measurement using Intel ITT - Libraries for tests development: - * `IE::gtest`, `IE::gtest_main`, `IE::gmock` - Google Tests framework libraries - * `IE::commonTestUtils` - static library with common tests utilities - * `IE::funcTestUtils` - static library with functional tests utilities - * `IE::unitTestUtils` - static library with unit tests utilities - * `IE::ngraphFunctions` - static library with the set of `ngraph::Function` builders - * `IE::funcSharedTests` - static library with common functional tests * `openvino::gtest`, `openvino::gtest_main`, `openvino::gmock` - Google Tests framework libraries * `openvino::commonTestUtils` - static library with common tests utilities * `openvino::funcTestUtils` - static library with functional tests utilities @@ -39,7 +31,7 @@ Once the commands above are executed, the OpenVINO Developer Package is generate * `openvino::ngraphFunctions` - static library with the set of `ov::Model` builders * `openvino::funcSharedTests` - static library with common functional tests -> **NOTE**: it's enough just to run `cmake --build . --target ie_dev_targets` command to build only targets from the +> **NOTE**: it's enough just to run `cmake --build . --target ov_dev_targets` command to build only targets from the > OpenVINO Developer package. Build Plugin using OpenVINO Developer Package @@ -61,31 +53,7 @@ A common plugin consists of the following components: To build a plugin and its tests, run the following CMake scripts: - Root `CMakeLists.txt`, which finds the OpenVINO Developer Package using the `find_package` CMake command and adds the `src` and `tests` subdirectories with plugin sources and their tests respectively: - -```cmake -cmake_minimum_required(VERSION 3.13) - -project(OpenVINOTemplatePlugin) - -set(TEMPLATE_PLUGIN_SOURCE_DIR ${OpenVINOTemplatePlugin_SOURCE_DIR}) - -find_package(OpenVINODeveloperPackage REQUIRED) - -if(CMAKE_COMPILER_IS_GNUCXX) - ov_add_compiler_flags(-Wall) -endif() - -add_subdirectory(src) - -if(ENABLE_TESTS) - include(CTest) - enable_testing() - - if(ENABLE_FUNCTIONAL_TESTS) - add_subdirectory(tests/functional) - endif() -endif() -``` +@snippet template/CMakeLists.txt cmake:main > **NOTE**: The default values of the `ENABLE_TESTS`, `ENABLE_FUNCTIONAL_TESTS` options are shared via the OpenVINO Developer Package and they are the same as for the main OpenVINO build tree. You can override them during plugin build using the command below: ```bash $ cmake -DENABLE_FUNCTIONAL_TESTS=OFF -DOpenVINODeveloperPackage_DIR=../openvino-release-build ../template-plugin diff --git a/docs/IE_PLUGIN_DG/PluginTesting.md b/docs/IE_PLUGIN_DG/PluginTesting.md index 930de0a4209..f8458ae5171 100644 --- a/docs/IE_PLUGIN_DG/PluginTesting.md +++ b/docs/IE_PLUGIN_DG/PluginTesting.md @@ -1,17 +1,16 @@ # Plugin Testing {#openvino_docs_ie_plugin_dg_plugin_testing} -Inference Engine (IE) tests infrastructure provides a predefined set of functional tests and utilities. They are used to verify a plugin using the Inference Engine public API. +OpenVINO tests infrastructure provides a predefined set of functional tests and utilities. They are used to verify a plugin using the OpenVINO public API. All the tests are written in the [Google Test C++ framework](https://github.com/google/googletest). -Inference Engine Plugin tests are included in the `IE::funcSharedTests` CMake target which is built within the OpenVINO repository +OpenVINO Plugin tests are included in the `openvino::funcSharedTests` CMake target which is built within the OpenVINO repository (see [Build Plugin Using CMake](@ref openvino_docs_ie_plugin_dg_plugin_build) guide). This library contains tests definitions (the tests bodies) which can be parametrized and instantiated in plugins depending on whether a plugin supports a particular feature, specific sets of parameters for test on supported operation set and so on. -Test definitions are split into tests class declaration (see `inference_engine/tests/functional/plugin/shared/include`) and tests class implementation (see `inference_engine/tests/functional/plugin/shared/src`) and include the following scopes of plugin conformance tests: +Test definitions are split into tests class declaration (see `src/tests/functional/plugin/shared/include`) and tests class implementation (see `src/tests/functional/plugin/shared/src`) and include the following scopes of plugin conformance tests: -1. **Behavior tests** (`behavior` sub-folder), which are a separate test group to check that a plugin satisfies basic Inference -Engine concepts: plugin creation, multiple executable networks support, multiple synchronous and asynchronous inference requests support, and so on. See the next section with details how to instantiate the tests definition class with plugin-specific parameters. +1. **Behavior tests** (`behavior` sub-folder), which are a separate test group to check that a plugin satisfies basic OpenVINO concepts: plugin creation, multiple executable networks support, multiple synchronous and asynchronous inference requests support, and so on. See the next section with details how to instantiate the tests definition class with plugin-specific parameters. -2. **Single layer tests** (`single_layer_tests` sub-folder). This groups of tests checks that a particular single layer can be inferenced on a device. An example of test instantiation based on test definition from `IE::funcSharedTests` library: +2. **Single layer tests** (`single_layer_tests` sub-folder). This groups of tests checks that a particular single layer can be inferenced on a device. An example of test instantiation based on test definition from `openvino::funcSharedTests` library: - From the declaration of convolution test class we can see that it's a parametrized GoogleTest based class with the `convLayerTestParamsSet` tuple of parameters: @snippet single_layer/convolution.hpp test_convolution:definition @@ -23,29 +22,24 @@ Engine concepts: plugin creation, multiple executable networks support, multiple @snippet single_layer_tests/convolution.cpp test_convolution:instantiate 3. **Sub-graph tests** (`subgraph_tests` sub-folder). This group of tests is designed to tests small patterns or combination of layers. E.g. when a particular topology is being enabled in a plugin e.g. TF ResNet-50, there is no need to add the whole topology to test tests. In opposite way, a particular repetitive subgraph or pattern can be extracted from `ResNet-50` and added to the tests. The instantiation of the sub-graph tests is done in the same way as for single layer tests. -> **Note**, such sub-graphs or patterns for sub-graph tests should be added to `IE::ngraphFunctions` library first (this library is a pre-defined set of small `ov::Model`) and re-used in sub-graph tests after. +> **Note**, such sub-graphs or patterns for sub-graph tests should be added to `openvino::ngraphFunctions` library first (this library is a pre-defined set of small `ov::Model`) and re-used in sub-graph tests after. -4. **HETERO tests** (`subgraph_tests` sub-folder) contains tests for `HETERO` scenario (manual or automatic affinities settings, tests for `QueryNetwork`). +4. **HETERO tests** (`subgraph_tests` sub-folder) contains tests for `HETERO` scenario (manual or automatic affinities settings, tests for `query_model`). 5. **Other tests**, which contain tests for other scenarios and has the following types of tests: - Tests for execution graph - Etc. -To use these tests for your own plugin development, link the `IE::funcSharedTests` library to your test binary and instantiate required test cases with desired parameters values. +To use these tests for your own plugin development, link the `openvino::funcSharedTests` library to your test binary and instantiate required test cases with desired parameters values. > **NOTE**: A plugin may contain its own tests for use cases that are specific to hardware or need to be extensively tested. To build test binaries together with other build artifacts, use the `make all` command. For details, see [Build Plugin Using CMake*](@ref openvino_docs_ie_plugin_dg_plugin_build). -### How to Extend Inference Engine Plugin Tests +### How to Extend OpenVINO Plugin Tests -Inference Engine Plugin tests are open for contribution. -Add common test case definitions applicable for all plugins to the `IE::funcSharedTests` target within the OpenVINO repository. Then, any other plugin supporting corresponding functionality can instantiate the new test. - -All Inference Engine per-layer tests check test layers functionality. They are developed using ov::Model. -as input graphs used by tests. In this case, to test a new layer with layer tests, extend -the `IE::ngraphFunctions` library, which is also included in the Inference Engine Developer package, with a new model. -including the corresponding operation. +OpenVINO Plugin tests are open for contribution. +Add common test case definitions applicable for all plugins to the `openvino::funcSharedTests` target within the OpenVINO repository. Then, any other plugin supporting corresponding functionality can instantiate the new test. > **NOTE**: When implementing a new subgraph test, add new single-layer tests for each operation of the subgraph if such test does not exist. diff --git a/docs/IE_PLUGIN_DG/layout.xml b/docs/IE_PLUGIN_DG/layout.xml index 6ba8aeb8750..1f4120d195f 100644 --- a/docs/IE_PLUGIN_DG/layout.xml +++ b/docs/IE_PLUGIN_DG/layout.xml @@ -2,7 +2,7 @@ - + @@ -89,10 +89,10 @@ - - - - + + + +