* add missed __init__.py files
* Update __init__.py
empty line
* Merge infirence_engine/tools/benchmark_tool with tools/benchmark_tool
* Update MD links
* remove benchmark_tool from package_BOM.txt
* add tools folder to the list of Doxygen files
* fix relative paths
* Update index.md
remove extra line
* Copied PR6744
* Added CMakeLists.txt into gflags
* Return gflags into .gitmodules
* Added CMakeLists.txt into gtest
* fix: remove extra gflags and gtests from .gitmodules
* Change syntax in set gtest_targets
* Make gtest/CMakeLists.txt as in PR6744 for pre-commit experiment
* Update gtest/CMakeLists.txt
* Change version of gtest repo
* Return gtest info into .gitmodules
* Update gflags version to currently used
* Add input image scale flag in benchmark app.
- user set input image scale with -iscale.
input is divided by scale.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Apply image scale, mean parameter in benchmark APP
Means and sacles values per channel
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Fix clang-format
Signed-off-by: hyunback <hyunback.kim@intel.com>
* fix clang-format issue2.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Update benchmark tool to align the format of mean and sacle values with MO arguments.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Remove debug print.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Add information about model preparation
* Add the missed file extension for the onnx model
* [c] Add information about model preparation
* [cpp] Add information about model preparation
* Update the string about model converter
* Specify a model name at steps 1 and 2 of the running example
* Added pugixml as submodule
* CVS-34900: updated pugixml to v1.11.4
* Fixed link with pugixml
* USe pugixml::static
* Try to fix bug
* Removed GITHUB_PULL_REQUEST
* Replaced OpenVINO_MAIN_SOURCE_DIR -> OpenVINO_SOURCE_DIR
* Removed some usages of IE_MAIN_SOURCE_DIR
* Use ngraph target directly
* [GNA] [WIP] Introduce an option to invoke the QoS feature 56759
* Apply remaining revew (typos) for PR 5741
* Introduce HW_WITH_SW_FBACK
* Add unit test for HW_WITH_SW_FBACK
* Enable HW_WITH_SW_FBACK in speech_sample cpp
* Use perf counters to report number of HW delivered frames to the user (eg speech_sample)
* Update speech_sample.hpp
based on review
* Update gna_config.hpp
Describe special perf counter
* lint fix
* Apply review
* Remove special performance counter
* Add GNA frequency for 6/151 CPU family/model
* Update inference-engine/samples/speech_sample/main.cpp
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* Update main.cpp
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* Updated developer package generation
* Try to fix
* Reuse find_package(InferenceEngine) inside DeveloperPackage
* Fixes for ARM
* Fixed double InferenceEngineDeveloperPackage include
* WA for OpenCV
* [Speech sample] Added numpy array support
* Added zlib library submodule to thirdparty
* Added cnpy library original version
* Changed cnpy library: some methods and fixed Klocwork issues
* Change cmakelists and documentation
* Added license information in TPP
* Added support make install to thirdparty components
* Upload python speech sample draft
* Add function doc strings
* Add the ability to save results to a file
* Add a plugin configuration for GNA to get better accuracy
* Add errors calculation (comparison with a reference)
* Fix flake8 issues
* Add ability to run in hetero mode
* Add ability to load and save numpy format files (.npz)
* Add an error for wrong file extensions & update help message
* Add import and export GNA model options
* Add -we option to export embedded gna model
* Add readme
* Add -oname command line option (Layer names for output blobs)
* Add -iname command line option (Layer names for input blobs)
* Add info about -iname option to README.md
* doc: update readme, fix style
* Add a state reset between inferences
* add reset API to speech README
* doc: remove extra output from README
* remove onnx and TODO, format output
* Add an else branch to the if statement that checks a utterance data type
* Add dummy data for inference if a number of vectors < batch size
* Split the sample into separte files
Co-authored-by: Kate Generalova <kate.generalova@intel.com>
* Added note abour custom version of find_dependency
* Warning about improper version in find_package()
* Removed ENABLE_ALTERNATIVE_TEMP
* Respect COMPONENTS in InferenceEngine cmake config
* Added Jira ticket #
* Deprecated ICNNNetwork
* ICNNNetwork deprecation
* Fixed comments
* More suppressions for ICNNNetwork
* Fixed C API compilation
* Deprecated ICNNNetwork methods only
* [IE CLDNN] Fixed cpplint for clDNN with Ninja generator
* Fixed compilation for ApiVersion with clang
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
* Exclude xbyak from install
* Added automatically generated InferenceEngineConfig.cmake
* Reverted a version back
* Fixed issues with target aliases
* Make TBB dependency private
* Made ie_parallel.cmake self-sufficient
* Don't expose ie_paralle.cmake to end users
* Fixed compilation with TBB
* Fixes for TBB
* Fixed vpu_graph_transformer compilation
* Fixed tests compilation
* Added install of ie_parallel.cmake
* Switched ENABLE_ALTERNATIVE_TEMP to OFF. Fixed COMPONENTS for TBB
* Fixed file name in install rules
* Added find_dependency for TBB in ie_parallel.cmake
* WA for cmake bug with PACKAGE_PREFIX_DIR
* Fixed no-deprecation to fix speech-library build
* Reverted version from 2.1.0 to 2.1
* Revert "Reverted version from 2.1.0 to 2.1"
This reverts commit 7cb5d1563c.
* Added versions to cmake
* Added versions to ie_version.hpp
* Returned custom version file back
* Added InferenceEngineConfig-version.cmake to share as well
* Disabled one more GPU test
* Added one more WA for CI
* WA for CI issue for C API
* Added InferenceEngineConfig-version.cmake to share as well
* Added verison parsing from ie_version.hpp
* Revert "[CPU] Add Roll support (#5112)"
This reverts commit 5d8f209df6.
* Revert "[CPU] windows_Interpolate_fused-FQ_nearest-mode_nspc-layout_fix (#5317)"
This reverts commit 0808975a37.
* Revert "[INT8][BF16] INT8 + BF16 feature was enabled (#5059)"
This reverts commit 7d2ec02d65.
* Support for components
* No version for IEDevScripts package
* Removed IE_VS_VER_HAS_VERSION from vs_version.rc.in
* Added compatibility for 2.x old versioning
* Python API for LoadNetwork by model file name
* BenchmarkApp: Add caching and LoadNetworkFromFile support
2 new options are introduced
- cache_dir <dir> - enables models caching
- load_from_file - use new perform "LoadNetwork" by model file name
Using both parameters will achieve maximum performance of read/load network on startup
Tests:
1) Run "benchmark_app -h". Help will display 2 new options. After available devices there will be list of devices with cache support
2) ./benchmark_app -d CPU -i <model.xml> -load_from_file
Verify that some test steps are skipped (related to ReadNetwork, re-shaping etc)
3) Pre-requisite: support of caching shall be enabled for Template plugin
./benchmark_app -d TEMPLATE -i <model.onnx> -load_from_file -cache_dir someDir
Verify that "someDir" is created and generated blob is available
Run again, verify that loading works as well (should be faster as it will not load onnx model)
4) Run same test as (3), but without -load_from_file option. Verify that cache is properly created
For some devices loadNetwork time shall be improved when cache is available
* Removed additional timing prints
* Correction from old code
* Revert "Removed additional timing prints"
Additional change - when .blob is chosen instead of .xml, it takes priority over caching flags
* Removed new time printings
As discussed, these time measurements like 'total first inference time' will be available in 'timeTests' scripts
* Fix clang-format issues
* New flags for GNA trget
* Add new flags to configuration
* Update utest
* Update speech_Sample
* Add unit tests
* Guard for wrong values
* Use gna execution target as consistent device
* Apply review
* Exclude xbyak from install
* Added automatically generated InferenceEngineConfig.cmake
* Reverted a version back
* Fixed issues with target aliases
* Make TBB dependency private
* Made ie_parallel.cmake self-sufficient
* Don't expose ie_paralle.cmake to end users
* Fixed compilation with TBB
* Fixes for TBB
* Fixed vpu_graph_transformer compilation
* Fixed tests compilation
* Added install of ie_parallel.cmake
* Switched ENABLE_ALTERNATIVE_TEMP to OFF. Fixed COMPONENTS for TBB
* Fixed file name in install rules
* Added find_dependency for TBB in ie_parallel.cmake
* WA for cmake bug with PACKAGE_PREFIX_DIR
* Fixed no-deprecation to fix speech-library build
* Reverted version from 2.1.0 to 2.1
* Revert "Reverted version from 2.1.0 to 2.1"
This reverts commit 7cb5d1563c.
* Returned custom version file back
* Added InferenceEngineConfig-version.cmake to share as well
* Disabled one more GPU test
* Added one more WA for CI
* WA for CI issue for C API
* WIP
* Added info on DockerHub CI Framework
* Feature/azaytsev/change layout (#3295)
* Changes according to feedback comments
* Replaced @ref's with html links
* Fixed links, added a title page for installing from repos and images, fixed formatting issues
* Added links
* minor fix
* Added DL Streamer to the list of components installed by default
* Link fixes
* Link fixes
* ovms doc fix (#2988)
* added OpenVINO Model Server
* ovms doc fixes
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* Updated openvino_docs.xml
* Edits to MO
Per findings spreadsheet
* macOS changes
per issue spreadsheet
* Fixes from review spreadsheet
Mostly IE_DG fixes
* Consistency changes
* Make doc fixes from last round of review
* integrate changes from baychub/master
* Update Intro.md
* Update Cutting_Model.md
* Update Cutting_Model.md
* Fixed link to Customize_Model_Optimizer.md
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
Co-authored-by: baychub <cbay@yahoo.com>
* change the deprecated method to the recent
* first ver of the hybrid cores aware CPU streams (+debug info)
* more debug and fixed sum threads
* disabled NUMA pinning to experiment with affinity via OS
* further brushing of stream to core type logic
* hybrid CPU-aware getNumberOfCPUCores
* adding check on the efficiency
* experimental TBB package (that cmake should pull from the internal server)
* iterating over core types in the reversed order (so the big cores are populated first in case user specified less than all #threads)
* adding back the NUMA affinity code-path for the full validation (incl 2 sockets Windows Server)
* cpplint fix and tabbing the #if clauses for the readbility
* pre-production TBB from internal server
* wrapping over #cores/types
* wrapping over #cores/types, ver 2
* wrapping over #streams instead
* disabling warnings as errors for a while (to unlock testing)
* accomodating new TBB layout for dependencies.bat
* next tbb ver (with debug binaries that probably can unlock the commodity builds, without playing product_configs)
* minor brushing for experiments (so that pinning can be disabled)
* minor brushing from code review
* Updating the SHA hash which appeared when rebasing to the master
* WIP refactoring
* Completed refactoring of the "config" phase of the cpu stream executor and on-the-fly streams to core types mapping
* making the benchmark_app aware about new pinning mode
* Brushing a bit (in preparation for the "soft" affinity)
* map to vector to simplify the things
* updated executors comparison
* more fine-grained pinning scheme for the HYBRID (required to allow all cores on 2+8 1+4, and other LITTLE-skewed scenarios)
TODO: seprate little to big ratio for the fp322 and int8 (and pass the fp32Only flag to the MakeDefaultMultiTHreaded)
* separating fp32 and int8 intensive cases for hybrid execution, also leveraging the HT if the #big_cores is small, refactored. also switched to the 2021.2 oneTBB RC package
* code style
* stripped tbb archives from unused folders and files, also has to rename the LICENSE.txt to the LICENSE to match existing OV packaging tools
* assigning nodeId regradless of pinning mode
* tests OpenCV builds with same 2021.2 oneTBB, ubuntu 18/20
* cmake install paths for oneTBB, alos a ie_parallel.cmake warning on older ver of TBB
* Updated latency case desc to cover multi-socket machines
* adding centos8 OCV with oneTBB build
updating TBB drops with hwloc shared libs added.
* enabled internal OCV from THIRD_PARTY_SERVER to test thru CI..
Added Centos7 notbb OCV build (until g-api get ready for onetbb) to unlock the Centos7 CI build
* separate rpath log to respect one-tbb specific paths
* fixed SEQ code-path
* fixed doc misprint
* allowing all cores in 2+8 for int8 as well
* cleaned from debug printfs
* HYBRID_AWARE pinning option for the Python benchmark_app
* OpenVINO Hybrid CPUs support
* Remove custom::task_arena abstraction layout
* Get back to the custom::task_arena interface
* Add windows.h inclusion
* Fix typo in macro name
* Separate TBB and TBBbind packages
* Fix compile-time conditions
* Fix preprocessors conditions
* Fix typo
* Fix linking
* make linking private
* Fix typo
* Fix target_compile_definitions syntax
* Implement CMake install logic, update sha hash for the tbbbind_2_4 package
* Add tbbbind_2_4 required paths to setup_vars
* Update CI paths
* Include ie_parallel.hpp to ie_system_conf.cpp
* Try to update dependencies scripts
* Try to fix dependencies.bat
* Modify dependencies script
* Use static tbbbind_2_4 library
* Remove redundant paths from CI
* Revert "cleaned from debug printfs"
This reverts commit 82c9bd90c5.
# Conflicts:
# inference-engine/src/inference_engine/os/win/win_system_conf.cpp
# inference-engine/src/inference_engine/threading/ie_cpu_streams_executor.cpp
# inference-engine/src/mkldnn_plugin/config.cpp
* Update tbbbind package version
* fixed compilation
* removing the direct tbb::info calls from CPU plugin, to aggregate everything in the single module (that exposes the higher level APIs)
* Update tbbbind package version
(cherry picked from commit f66b8f6aa6)
* compilation fix
* brushing the headers a bit
* Make custom::task_arena inherited from tbb::task_arena
* change to the latest TBB API, and more debug printfs
* code-style
* ARM compilation
* aligned "failed system config" between OV and TBB (by using '-1')
* macos compilation fix
* default arena creation (to make sure all code-path have that fallback)
* Incapsulate all TBB versions related logic inside the custom namespace
* Move custom layer header to internal scope + minor improvements
* with all NUMA/Hybrid checks now consolidated in the custom_arena, cleaning the ugly ifdefs thta we had
* Introduce new ThreadBindingType + fix compilation
* fixing OMP compilation
* OpenVINO Hybrid CPUs support
* Remove custom::task_arena abstraction layout
* Get back to the custom::task_arena interface
* Add windows.h inclusion
* Fix typo in macro name
* Separate TBB and TBBbind packages
* Fix compile-time conditions
* Fix preprocessors conditions
* Fix typo
* Fix linking
* make linking private
* Fix typo
* Fix target_compile_definitions syntax
* Implement CMake install logic, update sha hash for the tbbbind_2_4 package
* Add tbbbind_2_4 required paths to setup_vars
* Update CI paths
* Include ie_parallel.hpp to ie_system_conf.cpp
* Try to update dependencies scripts
* Try to fix dependencies.bat
* Modify dependencies script
* Use static tbbbind_2_4 library
* Remove redundant paths from CI
* Update tbbbind package version
* Make custom::task_arena inherited from tbb::task_arena
* Incapsulate all TBB versions related logic inside the custom namespace
* Move custom layer header to internal scope + minor improvements
* Introduce new ThreadBindingType + fix compilation
* Fix compilation
* Use public tbbbind_2_4 package
* fixed macos build, corrected comments/desc
* reverted to the default binding selection logic ( to preserve the legacy beh)
* OpenVINO Hybrid CPUs support
* Remove custom::task_arena abstraction layout
* Get back to the custom::task_arena interface
* Add windows.h inclusion
* Fix typo in macro name
* Separate TBB and TBBbind packages
* Fix compile-time conditions
* Fix preprocessors conditions
* Fix typo
* Fix linking
* make linking private
* Fix typo
* Fix target_compile_definitions syntax
* Implement CMake install logic, update sha hash for the tbbbind_2_4 package
* Add tbbbind_2_4 required paths to setup_vars
* Update CI paths
* Include ie_parallel.hpp to ie_system_conf.cpp
* Try to update dependencies scripts
* Try to fix dependencies.bat
* Modify dependencies script
* Use static tbbbind_2_4 library
* Remove redundant paths from CI
* Update tbbbind package version
* Make custom::task_arena inherited from tbb::task_arena
* Incapsulate all TBB versions related logic inside the custom namespace
* Move custom layer header to internal scope + minor improvements
* Introduce new ThreadBindingType + fix compilation
* Fix compilation
* Use public tbbbind_2_4 package
* Apply review comments
* Fix compilation without tbbbind_2_4
* Fix compilation with different TBB versions
* code review remarks
* fix for the NONE pinning code-path under HYBRID_AWAR
* whitespace and cleaning the debug printfs (per review)
* code-review comments
* fixed code-style
Co-authored-by: Kochin, Ivan <ivan.kochin@intel.com>
Co-authored-by: Kochin Ivan <kochin.ivan@intel.com>
Mikhail Ryzhov
Shoujiang Ma
Sergey Lyubimtsev
Ilya Churaev
Daria Ilina
Aleksandr Voron
hyunback kim
Vladimir ParamuzovandMikhail Letavin
Anton Romanov
Ilya Lavrenov
Ivan Kochin
Dmitry Pigasin
Krzysztof BrunieckiandMikhail Ryzhov
Mingyu Kim
Mikhail Kozlov
Eunsoo Sheen
Mikhail Nosov
