* 1. Correct the device list by priority order from high to low.
2. Remove GNA, CUDA, HPU, HDDL, NVIDIA from device list supported by AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Filter out supported device when not specify the candidate device for AUTO plugin.
* Add Debug MSG
* Update.
* Update AUTO mock test cases.
* Update.
* Update.
* Update code style.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* change AUTO default hint to Latency
* Change the comment tput to latency according to wangyang opinion
* fix testcase for MULTI, the hint default value returned by MULTI is throughput
* Remove the redundant testcase and modify the name of the testcase that returns the default value of hint
* Code optimization according to bell opinion, add comments to testcase
* Correct the comments of testcase
* When user sets num_streams, AUTO/MULTI does not set the default hint to HW plugin
* Fix the problem that smoke_AUTO_MULTI_ReturnDefaultHintTest fails to run
* add num_streams and default hint mock testcase
* add auto default perf hint mock testcase
* temp resolution to support model path for CPU in auto
Signed-off-by: fishbell <bell.song@intel.com>
* disable batch when load through model path
Signed-off-by: fishbell <bell.song@intel.com>
* add mark for future release
Signed-off-by: fishbell <bell.song@intel.com>
* implement step1: donotparse batch config if user not set explictly
Signed-off-by: fishbell <bell.song@intel.com>
* correct typo in case
Signed-off-by: fishbell <bell.song@intel.com>
Signed-off-by: fishbell <bell.song@intel.com>
* change gpunum to 3
* hold threads for GPU for MULTI:GPU,CPU
* need to first check if there is a CPU in the device list
* use getNumberOfCPUCores to get CPU cores
* load GPU first
* assign the correct value to multiSContext->_devicePriorities
* load GPU first and load CPU last and set numa for CPU
* MULTI set CPU affinity to “NUMA” during load network
* Load the CPU last while maintaining the original device priority
* not using vector for CPU
* There is no user setting affinity in MULTI, and NUMA is set for the CPU
* pass key ENABLE_HYPER_THREAD to CPU plugin and merge xiaoxia PR
* set ENABLE_HYPER_THREAD to NO
* modify log
* Modify the code according to xiaoxia and wanglei comments
* Modify the code according to bell comments
* [AUTO] update property setting logic.
* Update core::set_property() logic to only support primary property for AUTO and MULTI.
* Separate AUTO and MULTI supported properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. revert the changes for ie core.
2. Enable AUTO/MULTI only accepting its own properties.
3. Enable AUTO/MULTI accepting device properties passed from loadnetwork().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "[AUTO]Update the incorrect config test for Myriad (#13271)"
This reverts commit 0552d98802.
* MULTI only accepts its own properties that is same as AUTO currently.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test cases for AUTO/MULTI property test.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update to enable MULTI supporting hw primary property setting throw the compile_model().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the added test case for setting secondary property through set_property().
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. For AUTO/MULTI, property setting will be passed via core::compile_model() instead of core::set_property().
2. update the logic to infer precision setting that will transform into secondary property setting to each hw device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* set default value for nstreams when -d AUTO/MULTI and no nstreams setting from command line.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update code format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Enable BA to support if -d AUTO:MULTI,xxx/MULTI:AUTO,xxx. while AUTO Plugin need to update the logic of generating supported config list to virtual device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* move device configuration handling outside form function main.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert "move device configuration handling outside form function main."
This reverts commit ef77bfc602.
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Setting property performance hint to 'THROUGHPUT' and passing to executable network if no setting for of AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update the logic to handle the ov:device::properties in the ie core.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Remove the config filter within AUTO plugin when passing the configs to target hardwre device through AUTO plugin.
2. add test cases to check if the secondary properties have been passed into the target device through virtaul device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add mock tests to check if device properties works throgh the AUTO or MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add mock head file for loadnetwork with device properties testing.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update gtest commit to fix the warning issue appeared in the macro MATCHER_P.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set 'UNDEFINED' as the performancehint default value. Update benchmark app to pass performance hint with none value to MULTI/AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Using 'UNDEFINED' as the default value of hint in AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Adding a flag instead of adding a default value of hint to check if user configures the performance hint to AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update benchmark python version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* fix benchmark filed to pass on MULTI device when enable num_streams.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove part of test cases because MULTI plguin will not check if the unspported property is valid now.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove some incorrect config for MULTI test case since MULTI will pass through those unrecognized config without any exception.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support MULTI to set nstreams to multi target devices by using ov::device:properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support AUTO to set nstreams with multi target devices bu using ov:device::properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* update logic of setting TPUT by default for MULTI and AUTO.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set config to MULTI instaed of target device when using -d MULTI:target_device in benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update logit to check if perfomance hint is set.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set the correct _so from passthrough executable network when batch plugin is disabel to fix lifecycle coredump.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Same update for AUTO plugin
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* checking config priorities is set by checking if the value of priorities is empty instead of not founding in the config map.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add test case for MULTI without setting device priority.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add value verificatioin for the key ov::device::priorities and corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Retrieve the device name via the DeviceIDParser rather than via the priorities directly.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add logic to support devices like CPU(x),GPU(x).
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add 'mock' as a available device name when checking if ov::device::priorities is correct.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* do not check if priorities is available when value of priorigies is empty.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* add auto_batch_timeout for MULTI and AUTO
* fix clang-format for ie_core.cpp
* fix coredump
* simplify insert key to deviceConfig logic and parseDeviceNameIntoConfig() check "AUTO" and "AUTO:" only
* check config auto_batch_timeout
* add CleanUpInIECore()
* fix clang-format for ie_core.cpp
* Enable hint to tput if no property is specified for both AUTO device and target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Update logic.
2. Add test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update. Set hints to default for target device if no hints setting for AUTO plugin and no specific properties setting for target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable IE Core filter to promote the secondary properties to first level for hardware device.
2. Enable IE Core filter to pass the secondary properties to AUTO plguin.
3. Enable AUTO Plugin to parse secondary properties to first level and pass them to corresponding target hardware device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable MULTI Plugin to support secondary properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable HETERO Plugin to support secondary priorities.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Catch the EXPECT_CALL with AVAILABLE_DEVICES argument inputting to GetMetric.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert the logic of handling secondary properties for MULTI and HETERO device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the secondary property flattening logic because this logic has been implemented within AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. update flatten logic when secondary properties is specified.
2. add the test case with secondary properties for CPU.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add the test case with secondary properties for GPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add debug message to fix the test case failure issue.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add more debug info.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
1. For IE Core, 1st level property overides the 2nd level property.
2. For AUTO plugin, add available device list to check if the secondary properties is vaild.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add CUDA and ARM.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update device name for ARM Plugin and add device name for HPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* 1. Enable OPTIMIZATION_CAPABILITIES for AUTO plugin.
2. Add corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove EXPORT_IMPORT as Export is not implemented in the AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* selectdevice returns MULTI:device in cumulative_throughput
* load multi with throughput and disable cpu helper in cumulative
* disable cpu helper in cumulative_throughput
* add cumulative to bechmark_app help message
* modify benchmark_app.hpp clang-format
* Fix batchability check of MAX_BATCH_SIZE
* Applied review comment
* clonenetwork in auto
Signed-off-by: fishbell <bell.song@intel.com>
* clone in correct way
Signed-off-by: fishbell <bell.song@intel.com>
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
* don't check dynamic shape when there is only one device
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* remove redundant if
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* support config key device priority
for example:
if AUTO:CPU,GPU
the priority of CPU will be higher than GPU
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* add test and fix compile and test error
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* add an info for device priority and add lost [AUTOPLUGIN] on log
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* parseMetaDevice return all DEVICE of GPU, when use AUTO:GPU
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix compile issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* modify test and add test case, fix code issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix a bug and mutli with HETERO test failed
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix mock test faild issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix misprint
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* Disable AUTO:MYRIAD case
MYRIAD/CoreThreadingTests.smoke_QueryNetwork/targetDevice=MULTI_config=MULTI_DEVICE_PRIORITIES:MYRIAD_
faild on windows
the error is
myriadFuncTests-0 INFO: [E:] [BSL] found 0 ioexpander device
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* use ov::device::priorities key in this PR
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix a logic bug in key_network_priority after enable device priority
add test case cover it
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
Wang, YangandChen Peter
Tomasz Jankowski
Ilya Churaev
Nadezhda Ageeva
yanlan song
guozhong wang
Ilya Lavrenov
Sergey Shlyapnikov
Vladislav Volkov