* AUTO CTPUT single device always goes single device logic
* CreateInferRequest goes single device logic when perf hint is ctput
* Optimize single device logic according to bell opinion
* Add code comments
* merge two isCumulative conditions
* add ctput test case
* Remove the redundant header file of auto_ctput_test.cpp
* Modify the test case according to bell comments
* Create README.md for AUTO plugin
* link to contribuation guidance
* Update README.md according to developer documentation guidance
* Update for comments
* Update components for review comments
* Update folder for review comments
* update for wording
* update for comments
* change AUTO default hint to Latency
* Change the comment tput to latency according to wangyang opinion
* fix testcase for MULTI, the hint default value returned by MULTI is throughput
* Remove the redundant testcase and modify the name of the testcase that returns the default value of hint
* Code optimization according to bell opinion, add comments to testcase
* Correct the comments of testcase
* When user sets num_streams, AUTO/MULTI does not set the default hint to HW plugin
* Fix the problem that smoke_AUTO_MULTI_ReturnDefaultHintTest fails to run
* add num_streams and default hint mock testcase
* add auto default perf hint mock testcase
* temp resolution to support model path for CPU in auto
Signed-off-by: fishbell <bell.song@intel.com>
* disable batch when load through model path
Signed-off-by: fishbell <bell.song@intel.com>
* add mark for future release
Signed-off-by: fishbell <bell.song@intel.com>
* implement step1: donotparse batch config if user not set explictly
Signed-off-by: fishbell <bell.song@intel.com>
* correct typo in case
Signed-off-by: fishbell <bell.song@intel.com>
Signed-off-by: fishbell <bell.song@intel.com>
* change gpunum to 3
* hold threads for GPU for MULTI:GPU,CPU
* need to first check if there is a CPU in the device list
* use getNumberOfCPUCores to get CPU cores
* load GPU first
* assign the correct value to multiSContext->_devicePriorities
* load GPU first and load CPU last and set numa for CPU
* MULTI set CPU affinity to “NUMA” during load network
* Load the CPU last while maintaining the original device priority
* not using vector for CPU
* There is no user setting affinity in MULTI, and NUMA is set for the CPU
* pass key ENABLE_HYPER_THREAD to CPU plugin and merge xiaoxia PR
* set ENABLE_HYPER_THREAD to NO
* modify log
* Modify the code according to xiaoxia and wanglei comments
* Modify the code according to bell comments
* [AUTO] update property setting logic.
* Update core::set_property() logic to only support primary property for AUTO and MULTI.
* Separate AUTO and MULTI supported properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. revert the changes for ie core.
2. Enable AUTO/MULTI only accepting its own properties.
3. Enable AUTO/MULTI accepting device properties passed from loadnetwork().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "[AUTO]Update the incorrect config test for Myriad (#13271)"
This reverts commit 0552d98802.
* MULTI only accepts its own properties that is same as AUTO currently.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test cases for AUTO/MULTI property test.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update to enable MULTI supporting hw primary property setting throw the compile_model().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the added test case for setting secondary property through set_property().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. For AUTO/MULTI, property setting will be passed via core::compile_model() instead of core::set_property().
2. update the logic to infer precision setting that will transform into secondary property setting to each hw device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* set default value for nstreams when -d AUTO/MULTI and no nstreams setting from command line.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update code format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Enable BA to support if -d AUTO:MULTI,xxx/MULTI:AUTO,xxx. while AUTO Plugin need to update the logic of generating supported config list to virtual device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* move device configuration handling outside form function main.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "move device configuration handling outside form function main."
This reverts commit ef77bfc602.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Setting property performance hint to 'THROUGHPUT' and passing to executable network if no setting for of AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update the logic to handle the ov:device::properties in the ie core.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Remove the config filter within AUTO plugin when passing the configs to target hardwre device through AUTO plugin.
2. add test cases to check if the secondary properties have been passed into the target device through virtaul device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add mock tests to check if device properties works throgh the AUTO or MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add mock head file for loadnetwork with device properties testing.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update gtest commit to fix the warning issue appeared in the macro MATCHER_P.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update to the latest cmake for Windows
* Skip new warnings
* Added compiler flags for old cmake
* Disabled some warnings
* Suppress new warnings
* Fixed some errors
* Disable some warning for lpt
* Fix some warnings
* Disable new warnings
* Fix some warnings
* Disabled Was error for arm
* Fixed some errors
* Fixed some new warnings
* Fixed auto plugin
* Fixed Windows warning
* Fixed Windows
* Disable some warnings
* Some changes
* Small test fixes
* Fixed python build
* Added skip warnings for legacy python API
* Set 'UNDEFINED' as the performancehint default value. Update benchmark app to pass performance hint with none value to MULTI/AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Using 'UNDEFINED' as the default value of hint in AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Adding a flag instead of adding a default value of hint to check if user configures the performance hint to AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update benchmark python version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* fix benchmark filed to pass on MULTI device when enable num_streams.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove part of test cases because MULTI plguin will not check if the unspported property is valid now.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove some incorrect config for MULTI test case since MULTI will pass through those unrecognized config without any exception.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support MULTI to set nstreams to multi target devices by using ov::device:properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support AUTO to set nstreams with multi target devices bu using ov:device::properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Add NVIDIA device name along to CUDA device name (temporary solution)
* Disable nvidia_plugin along to cuda_plugin in .ci/azure/ (temporary solution)
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* update logic of setting TPUT by default for MULTI and AUTO.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set config to MULTI instaed of target device when using -d MULTI:target_device in benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update logit to check if perfomance hint is set.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set the correct _so from passthrough executable network when batch plugin is disabel to fix lifecycle coredump.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Same update for AUTO plugin
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* checking config priorities is set by checking if the value of priorities is empty instead of not founding in the config map.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add test case for MULTI without setting device priority.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add value verificatioin for the key ov::device::priorities and corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Retrieve the device name via the DeviceIDParser rather than via the priorities directly.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add logic to support devices like CPU(x),GPU(x).
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add 'mock' as a available device name when checking if ov::device::priorities is correct.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* do not check if priorities is available when value of priorigies is empty.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* First version of Windows installer
* WIndows NSIS installer
* add auto_batch_timeout for MULTI and AUTO
* fix clang-format for ie_core.cpp
* fix coredump
* simplify insert key to deviceConfig logic and parseDeviceNameIntoConfig() check "AUTO" and "AUTO:" only
* check config auto_batch_timeout
* add CleanUpInIECore()
* fix clang-format for ie_core.cpp
* Enable hint to tput if no property is specified for both AUTO device and target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Update logic.
2. Add test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update. Set hints to default for target device if no hints setting for AUTO plugin and no specific properties setting for target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable IE Core filter to promote the secondary properties to first level for hardware device.
2. Enable IE Core filter to pass the secondary properties to AUTO plguin.
3. Enable AUTO Plugin to parse secondary properties to first level and pass them to corresponding target hardware device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable MULTI Plugin to support secondary properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable HETERO Plugin to support secondary priorities.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Catch the EXPECT_CALL with AVAILABLE_DEVICES argument inputting to GetMetric.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert the logic of handling secondary properties for MULTI and HETERO device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the secondary property flattening logic because this logic has been implemented within AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. update flatten logic when secondary properties is specified.
2. add the test case with secondary properties for CPU.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add the test case with secondary properties for GPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add debug message to fix the test case failure issue.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add more debug info.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
1. For IE Core, 1st level property overides the 2nd level property.
2. For AUTO plugin, add available device list to check if the secondary properties is vaild.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add CUDA and ARM.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update device name for ARM Plugin and add device name for HPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* 1. Enable OPTIMIZATION_CAPABILITIES for AUTO plugin.
2. Add corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove EXPORT_IMPORT as Export is not implemented in the AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
