* [AUTO] Plugin takes only Intel dGPU as 1st priority
* Update test case
* Simplify the code
* Support more test cases in GetDeviceList API
* Add notIntelGPU to _deviceBlocklist in AUTO plugin
* Restore some code formats
* Update test cases
* Add some logs to GetValidDevice API
* Simplify the code
---------
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
* update auto architecture doc
* update auto architecture doc
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* update for comments
---------
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Remove exclusive_asyc_requests property from AUTO plugin
* Update test case
* Add test case to test incorrect config
* Remove the test case related to exclusive_asyc_requests property of AUTO plugin
* Benchmark_app set ov::hint::allow_auto_batching through compile_model
* Remove the process about allow_auto_batching in set_property of core
* Remove allow_auto_batching and auto_batch_timeout property from AUTO plugin
* Reserve the info logs and add API to check auto_batching
* Update test case, rm AB property test from core config tests
* Update some API in AUTO plugin config
* Remove cache_dir property from AUTO plugin
* Pass the secondary property to hardware plugin
* Update test case
* Update test case, meta plugin will pass the properties to device without checking
* Show the detailed failure message when AUTO load network failed
* Add test case
* Update test case to check multi load network failed
* Update test case based master
* RM _availableDevices hard code from AUTO
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
[MULTI] pass through to AUTO with CTPUT hint
After this change
-- MULTI doesn't support setting infer request via CPU(4),GPU(8).
-- MULTI doesn't support CompiledModel::set_property() and ExecutableNetwork::GetConfig().
* Implement CTPUT in AUTO code logic
* Add logic to handle device loading failure
* add some code comments
* fix warnning conversion from size_t to int
* Updated code according to comments of bell and wanglei
* the preferred device code path need to be updated with ctput also
* add fallback logic for CTPUT
* Modify the code logic according to bell suggestion
* Add prints for debugging bug
* throw exception when no device to run pipline task
* initialize idleWorkerRequest for CTPUT
* fix getting properties
Signed-off-by: fishbell <bell.song@intel.com>
refine
Signed-off-by: fishbell <bell.song@intel.com>
* fix warning
Signed-off-by: fishbell <bell.song@intel.com>
* fix illegal character on windows
Signed-off-by: fishbell <bell.song@intel.com>
* fix illegal character
Signed-off-by: fishbell <bell.song@intel.com>
add missing include
Signed-off-by: fishbell <bell.song@intel.com>
* more code refine
Signed-off-by: fishbell <bell.song@intel.com>
---------
Signed-off-by: fishbell <bell.song@intel.com>
Co-authored-by: fishbell <bell.song@intel.com>
* 1. Add device blacklist for AUTO plugin.
2. Update the logic to parse out the device candidate list from the inputting config MULTI_DEVICE_PRIORITIES.
3. Update the corresponding mock test cases.
4. Ignore the GTEST warning for the test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
* Update.
* Update.
* Add description about blacklist.
* Apply suggestions from code review
Update.
Co-authored-by: yanlan song <bell.song@intel.com>
* Update.
* Apply suggestions from code review
Updated.
Co-authored-by: yanlan song <bell.song@intel.com>
Co-authored-by: River Li <river.li@intel.com>
* Update test case.
* Update test case.
* Update test case.
* Update.
* Update.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: yanlan song <bell.song@intel.com>
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* Runtime fallback to other devices
* Update properties.hpp
* Update infer callback in AUTO
* Avoid some hang cases
* Add test cases for AUTO runtime fallback
* Replace mockExecutor with ImmediateExecutor
* Update the runtime fallback logic
* Update test case and support the case thar infer failed on CPU_HELP
* Update the test to detect whether to throw exception
* fix the error of CTPUT
* Add lock to AUTO executable network GetContext
* Update variable name in selectOtherDevice API
* Simplify variables and add testcase to improve test coverage
* Fix the issues when release CPU_HELP device and clean up the code
* Clean up code
* Small CoreImpl refactoring
* Removed cache_dirhandling from CPU plugin
* clang-format
* Fixed python tests
* Fix
* Fixed bugs in HETERO case
* Fixed clang-format and warnings in auto plugin
* Added import_export as capability for TEMPLATE plugin
* Commented throw exception from loaded_from_cache
* Fixed clang-formatof ro template plugin
* Enable AUTO to support execution mode hint.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add test case.
* Set default value "PERFORMANCE" for ov::hint::execution_mode.
* Update.
* Update.
* Correct default ov::hint::execution_mode value for the default value checking test case.
* Update.
* Delete obsolete config.hpp file.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* AUTO cumulative throughput mode ignore candidate device that fail to load device
* Simplify the judgement logic of whether Auto set to Multi
* Add description about _AutoSetToMulti variable
* Update variable name to _AutoCallMulti
* Refine logic of AUTO execution_devices
* Add loading error massage
* Add test case
* Add filter to execution_devices of MULTI
* Add execution_devices test in load fail sitution
* Simplify the logic of execution_devices
* Update auto_executable_network.cpp
* Update src/plugins/auto/multi_executable_network.cpp
Co-authored-by: yanlan song <bell.song@intel.com>
* Update src/plugins/auto/auto_executable_network.cpp
Co-authored-by: yanlan song <bell.song@intel.com>
* Update test case
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: yanlan song <bell.song@intel.com>
* Check if the device is supported when AUTO retrieves the device list based on the ov::device::priorities.
* Update the logic to handle the situation like -d AUTO:-CPU to benchmark APP.
* Remove MYRIAD and add NVIDIA for AUTO supported devices.
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* find test case for MultiDeviceInferRequest::SetBlob
* improve line coverage of infer_request
* add test cases for queryState and exception test case for perf count
* fix querystate running fail
* add test case to memory_states.cpp
* rename name of test case
* add memory_states.cpp to CMakeLists.txt
* Use _LogTag to judge whether MULTI
* clang-format intel_gna/memory_states.cpp
* Modify the position of the macro ENABLE_INTEL_CPU in the test case
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
* add test case for get perf_hint from GetMetric
* Increase Mock GetMetric test sleep time
* add mock test case for getMetric
* add new test case OVAutoExecutableNetworkTest
* convert ov::Any to ov::hint::Priority
* resolve conflict of get_metric.hpp
* add macro ENABLE_INTEL_CPU for gpu test case and fix cases not getting instantiated for cpu test
* fix the issue of running Mock GetMetric test cases fail
* add perf_hint test cases to properties_tests.cpp
* Modify the logic of judging whether it is a single device in ctput mode
* Enable AUTO compiledModel::get_property supporting its properties only.
* Update.
* Update.
* Update some releated test cases.
* Update.
* Update related test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Provide ENABLE_STARTUP_FALLBACK property to enable/disable CPU as acceleration
* Add more test cases to ENABLE_STARTUP_FALLBACK property
* Remove unnecessary config
* Remove plugin.hpp
* Update test case by PR#14754
* Update copyright
Co-authored-by: River Li <river.li@intel.com>
* Simplify using namespace
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: River Li <river.li@intel.com>
* remove ... in /src/bindings/c and resolve format
* remove ... in auto and resolve format
* remove ... in auto batch and resolve format
* remove ... in hetero and resolve format
* IE exception
* IE exception
* const
* const
* resolve format and ov::Exception
* ov::Exception
* reset IE::Exception
* remove ov::Exception
* InferenceEngine->IE in auto_schedule.cpp
* fix coredump while runing test case smoke_AutoMultiMock_NumStreamsAndDefaultPerfHintToHWTest
* fix coredump issue of running LoadNetworkWithSecondaryConfigsMockTest
* fix commpile fail in CI
* resolve core dump of running exec_network_get_metrics.cpp and disable some test cases for KEEMBAY and MYRIAD in exec_network_get_metrics.cpp
* remove test cases of MYRIAD and resolve core dump of running KEEMBAYin exec_network_get_metrics.cpp
* resolve conflic of common.h
* resolve conflic of common.h
* resolve common.h complie fail
---------
Co-authored-by: guozhong <guozhong.wang@intel.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* Added -Wall for Clang and GCC
* Fixes
* Don't use /J
* Fixed warnings
* Fixed warnings
* More fixes
* Fixed for MSVC
* Fixed more warnings on Windows
* Suppressed some warnings in template plugin
* Update src/tests/functional/plugin/shared/include/behavior/plugin/caching_tests.hpp
* Added suppression for PT FE
* Suppressed warnings in TF FE
* Suppressed warnings on Core unit tests
* Suppress warnings in python
* Suppressed Windows warning for 3rd party modules
* Suppresed one more warning
* 1. Correct the device list by priority order from high to low.
2. Remove GNA, CUDA, HPU, HDDL, NVIDIA from device list supported by AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Filter out supported device when not specify the candidate device for AUTO plugin.
* Add Debug MSG
* Update.
* Update AUTO mock test cases.
* Update.
* Update.
* Update code style.
---------
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* AUTO CTPUT single device always goes single device logic
* CreateInferRequest goes single device logic when perf hint is ctput
* Optimize single device logic according to bell opinion
* Add code comments
* merge two isCumulative conditions
* add ctput test case
* Remove the redundant header file of auto_ctput_test.cpp
* Modify the test case according to bell comments
* Create README.md for AUTO plugin
* link to contribuation guidance
* Update README.md according to developer documentation guidance
* Update for comments
* Update components for review comments
* Update folder for review comments
* update for wording
* update for comments
