IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[IE TESTS] Update conformance Readme file (#5374)

Browse Source 
* [IE TESTS] Update conformance Readme file

* [IE TESTS] Update conformance Readme file
This commit is contained in:
Irina Efode 2021-04-26 11:17:31 +03:00 committed by GitHub
parent e53249ddf2
commit d2868c9873
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 16 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
inference-engine/tests/functional/plugin/conformance/test_runner/README.md
View File

@ -20,8 +20,12 @@ The target is able to take the following command-line arguments:
* `-h` prints target command-line options with description.
* `--device` specifies target device.
* `--input_folders` specifies folders with IRs to run. The separator is `,`.
* `--plugin_lib_name` is name of plugin library. The example is MKLDNNPlugin. Use only with unregistered in IE Core devices.
* `--disable_test_config` allows to ignore all skipped tests with the exception of `DISABLED_` prefix using.
* `--extend_report` allows not to re-write device results to the report (add results of this run to the existing).
* `--extend_report` allows not to re-write device results to the report (add results of this run to the existing). Mutually exclusive with --report_unique_name.
* `--report_unique_name` allows to save report with unique name (report_pid_timestamp.xml). Mutually exclusive with --extend_report.
* `--save_report_timeout` allows to try to save report in cycle using timeout (in seconds).
* `--output_folder` Paths to the output folder to save report.
* All `gtest` command-line parameters
The result of execution is `report.xml` file. It demonstrates tests statistic like pass rate, passed, crashed, skipped and failed tests per operation for
@ -34,9 +38,19 @@ devices.
>
> The example of usage is:
> ```
> python3 gtest_parallel.py /opt/repo/openvino/bin/intel64/Debug/conformanceTests -d . --gtest_filter=*1613473581844763495*:*roi_align*:*PSROIPooling*:*Add*:*BinaryConv* -- --input_folders=/opt/repo/roi_align,/opt/repo/omz/out --device=CPU
> python3 gtest_parallel.py /path/to/openvino/bin/intel64/Debug/conformanceTests -d .
> --gtest_filter=*Add*:*BinaryConv* -- --input_folders=/path/to/ir_1,/path/to/ir_2 --device=CPU
> --report_unique_name --output_folder=/path/to/temp_output_report_folder
> ```
> All arguments after `--` symbol is forwarding to `conformanceTests` target.
>
> After using `--report_unique_name` argument please run
> [the merge xml script](./../../../../ie_test_utils/functional_test_utils/layer_tests_summary/merge_xmls.py)
> to aggregate the results to one report.
> The example of usage is:
> ```
> python3 merge_xmls.py --input_folders=/path/to/temp_output_report_folder --output_folder=/path/to/output_report_folder --output_filename=report_aggregated
> ```
## How to build operation coverage report
Run [the script](./../../../../ie_test_utils/functional_test_utils/layer_tests_summary/summarize.py) to generate `html` report.