* [Python] Align pc output in Python benchmark, update sorted output to new headers, print floats with 6-digit precision, fix date format printing to excel-friendly
* [PYTHON/C++] Change precision of raports and pc counters report, simplify code
* [C++] Align C++ benchamrk_app report to Python's by setting the precision value to 3
* [C++] Fix CsvDumper precision
* [C++/PYTHON] Align pcand pcsort outputs, merge C++ printSort methods into one
* Update samples/cpp/common/utils/include/samples/common.hpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/common/utils/include/samples/common.hpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++] Display a warning if Power Mode is not 'Max Performance' - rebase from master
* [C++] Prevent std::max from colliding with windows.h max macro
* [PYTHON/C++] Update README docs to reflect possible inaccuracies from optimization settings
* [C++] Cleanup windows optimizations
* [Tools] Avoid use of NumPy deprecated types
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert "[Tools] Avoid use of NumPy deprecated types"
This reverts commit 21ffc167d1.
* Move to 1.24 Numpy
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* replace benchmark_app imean and iscale parameters
Tickets 80777 and 80900
* Warn about difference with imean and iscale, note that in help
* Clang format
* Clang format
* mean->scale
* Keep mean scale empty if not set
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* merge master
* reset original format
* remove AUTO
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* delete condition
* delete condition in py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* sync benchmark Python version with benchmark C++ version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Enable dump/load to support ov::device::properties parsing in benchmark python version.
* Enable benchmark python version to parse configuration file with device properties setting for AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update usage of benchmark dumping/loading configuration in README and help message.
* Update comment.
* Update comments of the code.
* Update format of '-load_config' help message.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* docs: revise example and update outputs
* docs: revise example and update outputs
* docs: typo fix
* docs: rectify changes
* docs: add <a> tag
* Update README.md
* docs: change example to use asl-recognition model
* docs: fix <a> tag
* docs: update sample outputs
* docs: change example to use asl-recognition model
* docs: revise example to use asl-recognition model
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* [Python API][Tools] Top up upper bound version for dependencies: NumPy, TensorFlow, NetworkX
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove upper-bound for TensorFlow and NumPy
* Revert "Remove upper-bound for TensorFlow and NumPy"
This reverts commit 662085df2e.
* Remove upper-bound for NumPy for default installation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* sync benchmark Python version with benchmark C++ version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] Pipeline transfer
* [PYTHON] Align python benchmark
* [PYTHON] Align last step
* [PYTHON] Fix innacuracies of the last step - median
* [PYTHON/CPP] Add Core::get_version method to python API, refactor Ben benchmark to print version with this func
* [PYTHON] Remove get_version_string from API
* [PYTHON/CPP] Align output for model input/output info
* [PYTHON/CPP] Step 4,6 alignment of outputs, step 8 dumps all info stored in config parameters
* [CPP] Fix a bug causing nstreams parameter to never be set to AUTO in CPP benchmark_app
* [CPP] Fix clang format errors
* [CPP] Modify print order and data output for 8th step
* [PYTHON] Add verification checks from C++, modify set_thoughtput_streams to match documentation
* [CPP] Revert changes to C++ benchmark_app
* [CPP] Remove additional spacebar
* Update submodules versions on remote
* Update module from master branch
* Redownload submodules from master and override changes from commit
* [PYTHON] Remove unneccesary parse_status from validation function
* [PYTHON] Check for HINT in map, fix circular import
* [PYTHON] Remove artifacts from commit, fix args.perf_hint set to '' instead to 'none'
* [PYTHON] Reverse changes to perf hint, add key in map check, fix validation function throwing error on set hint
* [PYTHON] Fix linter
* [PYTHON] Remove linter spacebar
* [CPP] Fix wait_all exception throw
* [CPP/PYTHON] Clean artifacts and unwanted changes from work process
* [PYTHON] Fix artifacts from merge, clean submodule update
* [C++ CPU] Fix device name string by removing padding NULL characters from the back
* [CPP] Fix ba infer_request_wrap in other throw-catch clauses
* [PYTHON/CPP] Fix missing latencies in final step for shape group, fix minor misaligned messages, add missing report parameter create infer requests time
* [CPP] Clang fix formatting
* [CPP] Reverse clang fix format on plugin.cpp
* [PYTHON/CPP] Fix C++ progressbar printing endl when disabled, fix rounding in python creating infer request message
* [CPP] Fix foramtiing error
* [PYTHON/C++] Refactor network to model based on naming conventions, provide fresh README output example
* [PYTHON/C++] Add example output to C++ README, remove unnecessary device loop
* [BENCHMARK_APP/C++] Fix artifact from refactoring, remove try-catch clause
* Update samples/cpp/benchmark_app/benchmark_app.hpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* [CPP] Fix clang errors
* [CPP/PLUGIN Reverse modification to extract to separate task
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/parameters.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON/C++/BENCHMARK_APP] Fix language inconsistencies, remove unnecessary checks
* Update pyopenvino.cpp
* [CPP/BENCHMARK_APP] Remove unnecessary try-catch, fix linter errors
* [PYTHON/CPP/BENCHMARK_APP] Revert changes to Core, align version prints usin only provided methods
* [DOCS/BENCHMARK_APP] Update README with proper model examples
* Update README.md
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] CV2 module required only when loading images
* [PYTHON] Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* [PYTHON] Make CV import not global
* [PYTHON] Update format of inputs_filling.py
* [PYTHON] Fix missing newline
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* [BENCHMARK_APP/PYTHON] Introduce exception when module is installed but fails to import
* [SAMPLES/PYTHON] Sporadic access violation fix through checking whether module was cached
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Remove py3.6 constraints
* Cleanup out of date python versions
* Bump pybind11 to v2.10.1 release tag
* [PyOV] Fix wheel requirements for new package versions
Co-authored-by: jiwaszki <jan.iwaszkiewicz@intel.com>
* Benchmark_app python and c/c++ script add op profiling feature
* Fix compile bug about benchmark c/c++ op profiling
* Fix Compile Bug
* Fix issue in PR#12350 : 1. report_type info update; 2. priority of -pc and -pcsort; 3.align format between c++ and python; 4.check 'proportion' for 0 and print N/A; 5.detect the confusing print info
* Fix 8/17 review suggestion
* Fix 8/23 suggestion
* Fix the clang-format issue
* Fix the win-cc issue
* Fix win-cc issue about conversion from 'double' to 'float', possible loss of data
* Push PR by ODT group account
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Fiona Zhao <fiona.zhao@intel.com>
* Set 'UNDEFINED' as the performancehint default value. Update benchmark app to pass performance hint with none value to MULTI/AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Using 'UNDEFINED' as the default value of hint in AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Adding a flag instead of adding a default value of hint to check if user configures the performance hint to AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update benchmark python version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Fix benchmark_app commandline parsing
Object 'str' does not have a contains() method. Replaced by
"searchstring in object" query.
* Fix perf counter output and use consistent units
When ProfilingInfo was bound by pybind11, those entries turned into
Python timedelta objects. This caused two problems:
1. There was a division by 1000 to go to milliseconds. This caused a lot
of precision lost, because `timedelta(microseconds=2300)/1000.0 =
timedelta(2)` and `timedelta(microseconds=33) = timedelta(0)`.
2. When converting those timedelta objects to str, the output is in the
form of (HH:MM:SS.XXXXXX). This is not very useful microsecond based
performance counters.
This change simply reverts everything to printing plain microsecond
based integers.
* Align counter output to ms between Python/C++
* When set batch_size, disable auto batching
* auto-batch only supprt auto plugin
* Set property allow_auto_batching for devices
* cancel static
* Clean Code
* Add test case for testing allow_auto_batching
* Fix the issue after merging latest master branch code
* Add allow_auto_batching in python benchmark_app
* Fix the quotes issue
* When load_network is set to allow-auto-batching, there is no need to read the value of global
* Fix the clang-format
* Change the location
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* [MO] Relax MO upper-bound requirements for TensorFlow and NumPy
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Just debug numpy version
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Pin upper-bounds for NumPy and TensorFlow modules in all reqs files
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update submodule dependency for open_model_zoo
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Install numpy module first
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy version in POT setup.py
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Extend telemetry tests with a set of possible solutions for events
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy module version for layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* revert test code
* revert test code
* enable brgconv f32
* use config to enable brgconv f32
* when brg disabled not init bin-postops
* change prop name for extensive
* use more general field
* fix review comments.
Sebastian Golebiewski
Ilya Naumov
Ilya LavrenovandYuan Xu
Ilya Churaev
Tomasz Dołbniak
Roman Kazantsev
Wang, YangandZlobin Vladimir
EvanandYuan Xu
Nadezhda Ageeva
yanlan song
Xiake Sun
Przemyslaw Wysockiandjiwaszki
Alexey LebedevandAnastasia Kuporosova
Nico Galoppo
Piotr Krzemiński
Ekaterina Aidova
Vladimir Paramuzov
Chenhu Wang
Luo Cheng
guozhong wang
Alexey Lebedev
Sergey Lyubimtsev