openvino/docs/dev/test_coverage.md

OpenVINO coverage report build

The coverage report is generated using Lcov tool and based on profile data generated by GCC. The generated reports are in HTML form and located in <openvino_build>/coverage. The reports are generated for the following components:

1. inference - Inference component is a part of OpenVINO Runtime library.
2. core - Core component is a part of OpenVINO Runtime library.
3. low_precision_transformations - Low Precision transformations is a part of OpenVINO Runtime library.
4. openvino_transformations - Common transformations is a part of OpenVINO Runtime library.
5. inference_engine_legacy - legacy OpenVINO library
6. preprocessing - OpenVINO G-API based preprocessing plugin.
7. snippets - OpenVINO snippets.
8. OpenVINO open-sources plugins:
   • hetero_plugin - Heterogeneous plugin.
   • multi_plugin - Multi plugin.
   • auto_plugin - Auto plugin.
   • template_plugin - Template plugin.
   • intel_gpu_plugin - GPU plugin.
   • intel_cpu_plugin - CPU plugin.
   • intel_gna_plugin - GNA plugin.

Build with profiling data support

To build coverage report, compile OpenVINO with an additional CMake option -DENABLE_COVERAGE=ON:

$ cmake -DENABLE_COVERAGE=ON .

And build OpenVINO as usual.

Generate coverage report

In order to generate coverage reports, first of all, the tests must be run. Depending on how many tests are run, the better covegare percentage can be achieved. E.g. for openvino component, InferenceEngineUnitTests, ieUnitTests, ieFuncTests must be run as well as plugin tests.

$ ctest -V

OpenVINO components define several common groups which allow to run tests for separate component (with ctest argiment -L GROUP_NAME):

• OV - core OpenVINO tests
• IR_FE - IR frontend tests
• ONNX_FE - ONNX frontend tests
• PADDLE_FE - Paddle frontend tests
• TF_FE - TensorFlow frontend tests
• CPU - CPU plugin tests
• GPU - GPU plugin tests
• GNA - GNA plugin tests

After sufficient number of tests are executed, the coverage numbers can be calculated. In order to do this, run:

$ make ov_coverage

The following tree of reports are generated:

$ find coverage -maxdepth 2 -name index.html
coverage/core/index.html
coverage/transformations/index.html
coverage/paddle_frontend/index.html
coverage/tf_frontend/index.html
coverage/openvino/index.html
coverage/onnx_frontend/index.html
coverage/legacy/index.html
coverage/ir_frontend/index.html
coverage/low_precision_transformations/index.html
coverage/template_plugin/index.html
coverage/inference/index.html
coverage/frontend_common/index.html

See also

• OpenVINO™ README
• Developer documentation