* 1. refine the logic to ov::device::properties setting.
2. the config overrides will be performed if same config setting is came from CMD line.-a
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update configuration sample file within README.md.
* Update.
* Update.
* 1. Update configuration example file within REAMDME.md for Python version.
2. implement the config DEVICE_PROPERTIES value convertation between the string type and dictionary of Python type.
3. Update the configuration file loading and dumping logic.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update.
* Update.
* Update.
* Update.
* 1. Enable configs to be interchangeable between C++ and Python.
2. Update perf_count showing logic.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert the logic of showing show performance counters.
* Update help msg for loading config option.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* py/benchmark_app: fix -hint
Don't warn about values which are explicitly set in -hint.
That aligns C++ and Python implementations.
Ticket 106544
* Remove extra throw
* Fix code style
* Check if the device is supported when AUTO retrieves the device list based on the ov::device::priorities.
* Update the logic to handle the situation like -d AUTO:-CPU to benchmark APP.
* Remove MYRIAD and add NVIDIA for AUTO supported devices.
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* Update ov::hint::performance_hint UNDEFINED value from empty string to "UNDEFINED".
* Update benchmark Python version.
* Update.
* Update.
* Update.
* Update the description about hint setting within benchmark APP README and help message.
* Disable set_property() to support ov::device::properties setting.
* Update benchmark APP to set device properties through compile_model() instead of through set_property().
* Update.
* Update.
* Update some test case including ov::device::properties setting via core.ser_property().
* Since core.set_property() didn't support ov::device::properties setting, just remove the test case to check compile_model() works well if setting ov::device::properties via core.set_property() first.
* Update CompileModel in test name to CompiledModel
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Add corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update.
* Remove the changes of this commit as this modification has nothing to do
with this PR.
This reverts commit 4f04b9f085.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* benchmark_app: factor out advanced options
* cpp/benchmark_app: add usage word
* move api to advanced, group b/shape/data_shape/layout into Input shapes, factor out Statistics dumping options
* Factor out Device-specific performance options
* Factor out Preprocessing options
* Minor regroup
* [C++/BENCHMARK_APP] Introduce Numpy array loading for C++ benchmark app
* [DOCS/BENCHMARK_APP] Update docs to reflect changes, update list of available extensions from OpenCV, align help messages
* Update inputs_filling.cpp
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/benchmark_app/inputs_filling.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++/Python] Implement quality-of-life improvements from PR comments
* [C++] Fix compilation errors, fix linter output
* [C++/PYTHON] Apply requested changes
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/benchmark_app/utils.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] Separate loading of numpy arrays similar to images
* [PYTHON] Remove unnecessary 'Prepare xxx file' print
* Update README again because IF OPENCV.. dissapeared for some reason
* Update second README with missing IF OPENCV..
* [C++] Remove unnecessary vector print function
* [C++ Add Numpy processing function - TODO link it to the tensor filling
* Reverse OneDnn plugin modification
* [C++] Numpy array loading for C++
* [C++] Add (almost) all missing types of data
* Reverse submodule modifications
* [C++/PYTHON] Fix compilation errors, clean code
* [C++] Modify supported extensions, add numpy checking to utils, add numpy to get_image_info method
* Update samples/cpp/benchmark_app/inputs_filling.cpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++] Fix utils header file to reflect unordered set change
* [PYTHON/C++] Fix compilation errors in C++ code, fix Python dynamic shapes numpy loading
* [C++] Fix explicit instantiation of NumpyArray reader
* [C++] Clang format, minor syntax fixes
* [PYTHON/C++] Remove unnecessary data types, introduce a new approach to cast data of different types from format_rt_reader, remove uppercase types from Python precision parameters
* [PYTHON] Update README to reflect new precision settings
* [PYTHON] Fix README, fix clang format
* [C++] Clean headers
* [C++] Fix uninitialized variable error
* [C++/PYTHON] Fixed choices in Python benchmark, fixed types in C++ benchmark
* [C++] Fixed ov::float16conversion, fixed Python types map - removed redundancies
* [C++] Add back boolean support
* [C++] Fix compilation errors
---------
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [Python] Align pc output in Python benchmark, update sorted output to new headers, print floats with 6-digit precision, fix date format printing to excel-friendly
* [PYTHON/C++] Change precision of raports and pc counters report, simplify code
* [C++] Align C++ benchamrk_app report to Python's by setting the precision value to 3
* [C++] Fix CsvDumper precision
* [C++/PYTHON] Align pcand pcsort outputs, merge C++ printSort methods into one
* Update samples/cpp/common/utils/include/samples/common.hpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update samples/cpp/common/utils/include/samples/common.hpp
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [C++] Display a warning if Power Mode is not 'Max Performance' - rebase from master
* [C++] Prevent std::max from colliding with windows.h max macro
* [PYTHON/C++] Update README docs to reflect possible inaccuracies from optimization settings
* [C++] Cleanup windows optimizations
* [Tools] Avoid use of NumPy deprecated types
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert "[Tools] Avoid use of NumPy deprecated types"
This reverts commit 21ffc167d1.
* Move to 1.24 Numpy
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* replace benchmark_app imean and iscale parameters
Tickets 80777 and 80900
* Warn about difference with imean and iscale, note that in help
* Clang format
* Clang format
* mean->scale
* Keep mean scale empty if not set
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* merge master
* reset original format
* remove AUTO
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* delete condition
* delete condition in py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* sync benchmark Python version with benchmark C++ version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Enable dump/load to support ov::device::properties parsing in benchmark python version.
* Enable benchmark python version to parse configuration file with device properties setting for AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update usage of benchmark dumping/loading configuration in README and help message.
* Update comment.
* Update comments of the code.
* Update format of '-load_config' help message.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* docs: revise example and update outputs
* docs: revise example and update outputs
* docs: typo fix
* docs: rectify changes
* docs: add <a> tag
* Update README.md
* docs: change example to use asl-recognition model
* docs: fix <a> tag
* docs: update sample outputs
* docs: change example to use asl-recognition model
* docs: revise example to use asl-recognition model
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* [Python API][Tools] Top up upper bound version for dependencies: NumPy, TensorFlow, NetworkX
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove upper-bound for TensorFlow and NumPy
* Revert "Remove upper-bound for TensorFlow and NumPy"
This reverts commit 662085df2e.
* Remove upper-bound for NumPy for default installation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* sync benchmark Python version with benchmark C++ version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] Pipeline transfer
* [PYTHON] Align python benchmark
* [PYTHON] Align last step
* [PYTHON] Fix innacuracies of the last step - median
* [PYTHON/CPP] Add Core::get_version method to python API, refactor Ben benchmark to print version with this func
* [PYTHON] Remove get_version_string from API
* [PYTHON/CPP] Align output for model input/output info
* [PYTHON/CPP] Step 4,6 alignment of outputs, step 8 dumps all info stored in config parameters
* [CPP] Fix a bug causing nstreams parameter to never be set to AUTO in CPP benchmark_app
* [CPP] Fix clang format errors
* [CPP] Modify print order and data output for 8th step
* [PYTHON] Add verification checks from C++, modify set_thoughtput_streams to match documentation
* [CPP] Revert changes to C++ benchmark_app
* [CPP] Remove additional spacebar
* Update submodules versions on remote
* Update module from master branch
* Redownload submodules from master and override changes from commit
* [PYTHON] Remove unneccesary parse_status from validation function
* [PYTHON] Check for HINT in map, fix circular import
* [PYTHON] Remove artifacts from commit, fix args.perf_hint set to '' instead to 'none'
* [PYTHON] Reverse changes to perf hint, add key in map check, fix validation function throwing error on set hint
* [PYTHON] Fix linter
* [PYTHON] Remove linter spacebar
* [CPP] Fix wait_all exception throw
* [CPP/PYTHON] Clean artifacts and unwanted changes from work process
* [PYTHON] Fix artifacts from merge, clean submodule update
* [C++ CPU] Fix device name string by removing padding NULL characters from the back
* [CPP] Fix ba infer_request_wrap in other throw-catch clauses
* [PYTHON/CPP] Fix missing latencies in final step for shape group, fix minor misaligned messages, add missing report parameter create infer requests time
* [CPP] Clang fix formatting
* [CPP] Reverse clang fix format on plugin.cpp
* [PYTHON/CPP] Fix C++ progressbar printing endl when disabled, fix rounding in python creating infer request message
* [CPP] Fix foramtiing error
* [PYTHON/C++] Refactor network to model based on naming conventions, provide fresh README output example
* [PYTHON/C++] Add example output to C++ README, remove unnecessary device loop
* [BENCHMARK_APP/C++] Fix artifact from refactoring, remove try-catch clause
* Update samples/cpp/benchmark_app/benchmark_app.hpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update samples/cpp/benchmark_app/main.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* [CPP] Fix clang errors
* [CPP/PLUGIN Reverse modification to extract to separate task
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/parameters.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/utils.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* Update tools/benchmark_tool/openvino/tools/benchmark/main.py
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON/C++/BENCHMARK_APP] Fix language inconsistencies, remove unnecessary checks
* Update pyopenvino.cpp
* [CPP/BENCHMARK_APP] Remove unnecessary try-catch, fix linter errors
* [PYTHON/CPP/BENCHMARK_APP] Revert changes to Core, align version prints usin only provided methods
* [DOCS/BENCHMARK_APP] Update README with proper model examples
* Update README.md
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
* [PYTHON] CV2 module required only when loading images
* [PYTHON] Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* [PYTHON] Make CV import not global
* [PYTHON] Update format of inputs_filling.py
* [PYTHON] Fix missing newline
* Update tools/benchmark_tool/openvino/tools/benchmark/utils/inputs_filling.py
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* [BENCHMARK_APP/PYTHON] Introduce exception when module is installed but fails to import
* [SAMPLES/PYTHON] Sporadic access violation fix through checking whether module was cached
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Remove py3.6 constraints
* Cleanup out of date python versions
* Bump pybind11 to v2.10.1 release tag
* [PyOV] Fix wheel requirements for new package versions
Co-authored-by: jiwaszki <jan.iwaszkiewicz@intel.com>
* Benchmark_app python and c/c++ script add op profiling feature
* Fix compile bug about benchmark c/c++ op profiling
* Fix Compile Bug
* Fix issue in PR#12350 : 1. report_type info update; 2. priority of -pc and -pcsort; 3.align format between c++ and python; 4.check 'proportion' for 0 and print N/A; 5.detect the confusing print info
* Fix 8/17 review suggestion
* Fix 8/23 suggestion
* Fix the clang-format issue
* Fix the win-cc issue
* Fix win-cc issue about conversion from 'double' to 'float', possible loss of data
* Push PR by ODT group account
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Fiona Zhao <fiona.zhao@intel.com>
