[BENCHMARK_APP/PYTHON/CPP] Align benchmark_app output across languages (#12814)

* [PYTHON] Pipeline transfer

* [PYTHON] Align python benchmark

* [PYTHON] Align last step

* [PYTHON] Fix innacuracies of the last step - median

* [PYTHON/CPP] Add Core::get_version method to python API, refactor Ben benchmark to print version with this func

* [PYTHON] Remove get_version_string from API

* [PYTHON/CPP] Align output for model input/output info

* [PYTHON/CPP] Step 4,6 alignment of outputs, step 8 dumps all info stored in config parameters

* [CPP] Fix a bug causing nstreams parameter to never be set to AUTO in CPP benchmark_app

* [CPP] Fix clang format errors

* [CPP] Modify print order and data output for 8th step

* [PYTHON] Add verification checks from C++, modify set_thoughtput_streams to match documentation

* [CPP] Revert changes to C++ benchmark_app

* [CPP] Remove additional spacebar

* Update submodules versions on remote

* Update module from master branch

* Redownload submodules from master and override changes from commit

* [PYTHON] Remove unneccesary parse_status from validation function

* [PYTHON] Check for HINT in map, fix circular import

* [PYTHON] Remove artifacts from commit, fix args.perf_hint set to '' instead to 'none'

* [PYTHON] Reverse changes to perf hint, add key in map check, fix validation function throwing error on set hint

* [PYTHON] Fix linter

* [PYTHON] Remove linter spacebar

* [CPP] Fix wait_all exception throw

* [CPP/PYTHON] Clean artifacts and unwanted changes from work process

* [PYTHON] Fix artifacts from merge, clean submodule update

* [C++ CPU] Fix device name string by removing padding NULL characters from the back

* [CPP] Fix ba infer_request_wrap in other throw-catch clauses

* [PYTHON/CPP] Fix missing latencies in final step for shape group, fix minor misaligned messages, add missing report parameter create infer requests time

* [CPP] Clang fix formatting

* [CPP] Reverse clang fix format on plugin.cpp

* [PYTHON/CPP] Fix C++ progressbar printing endl when disabled, fix rounding in python creating infer request message

* [CPP] Fix foramtiing error

* [PYTHON/C++] Refactor network to model based on naming conventions, provide fresh README output example

* [PYTHON/C++] Add example output to C++ README, remove unnecessary device loop

* [BENCHMARK_APP/C++] Fix artifact from refactoring, remove try-catch clause

* Update samples/cpp/benchmark_app/benchmark_app.hpp

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>

* Update samples/cpp/benchmark_app/main.cpp

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>

* Update tools/benchmark_tool/openvino/tools/benchmark/main.py

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>

* Update samples/cpp/benchmark_app/main.cpp

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>

* [CPP] Fix clang errors

* [CPP/PLUGIN Reverse modification to extract to separate task

* Update tools/benchmark_tool/openvino/tools/benchmark/main.py

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>

* Update tools/benchmark_tool/openvino/tools/benchmark/parameters.py

Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>

* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py

Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>

* Update tools/benchmark_tool/openvino/tools/benchmark/main.py

Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>

* [PYTHON/C++/BENCHMARK_APP] Fix language inconsistencies, remove unnecessary checks

* Update pyopenvino.cpp

* [CPP/BENCHMARK_APP] Remove unnecessary try-catch, fix linter errors

* [PYTHON/CPP/BENCHMARK_APP] Revert changes to Core, align version prints usin only provided methods

* [DOCS/BENCHMARK_APP] Update README with proper model examples

* Update README.md

Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
This commit is contained in:
authored and GitHub committed 2022-11-14 14:10:36 +01:00
1 parent cda2df8736
commit d951623027
18 files changed
+461 -331

No files matched your search

+11 -12
View File
@@ -29,7 +29,7 @@ static const char help_message[] = "Print a usage message";
/// @brief message for images argument
static const char input_message[] =
"Optional. Path to a folder with images and/or binaries or to specific image or binary file.\n"
" In case of dynamic shapes networks with several inputs provide the same number"
" In case of dynamic shapes models with several inputs provide the same number"
" of files for each input (except cases with single file for any input):"
"\"input1:1.jpg input2:1.bin\", \"input1:1.bin,2.bin input2:3.bin input3:4.bin,5.bin \"."
" Also you can pass specific keys for inputs: \"random\" - for fillling input with random data,"
@@ -45,7 +45,7 @@ static const char model_message[] =
/// @brief message for performance hint
static const char hint_message[] =
"Optional. Performance hint allows the OpenVINO device to select the right network-specific settings.\n"
"Optional. Performance hint allows the OpenVINO device to select the right model-specific settings.\n"
" 'throughput' or 'tput': device performance mode will be set to THROUGHPUT.\n"
" 'cumulative_throughput' or 'ctput': device performance mode will be set to "
"CUMULATIVE_THROUGHPUT.\n"
@@ -90,7 +90,7 @@ static const char infer_num_streams_message[] =
"automatic selection "
"usually provides a reasonable performance, it still may be non - optimal for some cases, "
"especially for "
"very small networks. See sample's README for more details. "
"very small models. See sample's README for more details. "
"Also, using nstreams>1 is inherently throughput-oriented option, "
"while for the best-latency estimations the number of streams should be set to 1.";
@@ -138,7 +138,7 @@ static const char report_type_message[] =
"Optional. Enable collecting statistics report. \"no_counters\" report contains "
"configuration options specified, resulting FPS and latency. \"average_counters\" "
"report extends \"no_counters\" report and additionally includes average PM "
"counters values for each layer from the network. \"detailed_counters\" report "
"counters values for each layer from the model. \"detailed_counters\" report "
"extends \"average_counters\" report and additionally includes per-layer PM "
"counters and latency for each executed infer request.";
@@ -188,22 +188,22 @@ static const char dump_config_message[] =
"Optional. Path to JSON file to dump IE parameters, which were set by application.";
static const char shape_message[] =
"Optional. Set shape for network input. For example, \"input1[1,3,224,224],input2[1,4]\" or \"[1,3,224,224]\""
"Optional. Set shape for model input. For example, \"input1[1,3,224,224],input2[1,4]\" or \"[1,3,224,224]\""
" in case of one input size. This parameter affect model input shape and can be dynamic."
" For dynamic dimensions use symbol `?` or '-1'. Ex. [?,3,?,?]."
" For bounded dimensions specify range 'min..max'. Ex. [1..10,3,?,?].";
static const char data_shape_message[] =
"Required for networks with dynamic shapes. Set shape for input blobs."
"Required for models with dynamic shapes. Set shape for input blobs."
" In case of one input size: \"[1,3,224,224]\" or \"input1[1,3,224,224],input2[1,4]\"."
" In case of several input sizes provide the same number for each input (except cases with single shape for any "
"input):"
" \"[1,3,128,128][3,3,128,128][1,3,320,320]\", \"input1[1,1,128,128][1,1,256,256],input2[80,1]\""
" or \"input1[1,192][1,384],input2[1,192][1,384],input3[1,192][1,384],input4[1,192][1,384]\"."
" If network shapes are all static specifying the option will cause an exception.";
" If model shapes are all static specifying the option will cause an exception.";
static const char layout_message[] =
"Optional. Prompts how network layouts should be treated by application. "
"Optional. Prompts how model layouts should be treated by application. "
"For example, \"input1[NCHW],input2[NC]\" or \"[NCHW]\" in case of one input size.";
// @brief message for enabling caching
@@ -211,16 +211,15 @@ static const char cache_dir_message[] = "Optional. Enables caching of loaded mod
"List of devices which support caching is shown at the end of this message.";
// @brief message for single load network
static const char load_from_file_message[] = "Optional. Loads model from file directly without ReadNetwork."
static const char load_from_file_message[] = "Optional. Loads model from file directly without read_model."
" All CNNNetwork options (like re-shape) will be ignored";
// @brief message for inference_precision
static const char inference_precision_message[] = "Optional. Inference precision";
static constexpr char inputs_precision_message[] = "Optional. Specifies precision for all input layers of the network.";
static constexpr char inputs_precision_message[] = "Optional. Specifies precision for all input layers of the model.";
static constexpr char outputs_precision_message[] =
"Optional. Specifies precision for all output layers of the network.";
static constexpr char outputs_precision_message[] = "Optional. Specifies precision for all output layers of the model.";
static constexpr char iop_message[] =
"Optional. Specifies precision for input and output layers by name.\n"