[AUTO BATCH PLUGIN] enable api 2.0 for auto batch plugin (#18172)
* [AUTO BATCH PLUGIN] enable API 2.0 for auto batch plugin
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] disenable auto batch plugin unite test for tmp
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] remove test with ov::auto_batch_timeout(-1), cause the variable is unsigned int
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix compiler error caused by std::atomic_uint32_t
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [Remote Context] fix revew comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix compiler warnings
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix compiler warnings
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix test error
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix CI test error in cpu func test case, caused by batched model lost rt info
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix CI build error, caused by unused variable
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] using ov::threading
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] clear code in batched req share buffer with non-batched req
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] clean code & fix format issue
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] clean code & fix format issue
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] add api implementation about get_default_context() & create_context() and remove the test config with AUTO_BATCH_TIMEOUT(-1)
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix gpu test with auto btch failed
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix warning
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix get_default_context() issue
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix using namespace redundancy
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] modify variable naming style
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix CI test error, cause by tensor reference in virtual plugin
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] implement get_profiling()
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] remove get_context() from auto batch compiled model using the interface from parent class
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] implement create_context() & get_default_context for auto batch plugin
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix format issue
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] implement auto batch remote context
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix error after merge with master
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix compiler error caused by update master
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] refact remote context in auto batch plugin
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] add unite test cases for auto batch plugin
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix CI warning caused by unused variable & add unite of remote context
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] add virtual property for get_context() in icompiled_model & implement it in auto batch plugin
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] add ov::loaded_from_cache support
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix error caused by updating with master
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix unite test error
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix conflict
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix error caused by update master
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comments
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
---------
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
Signed-off-by: xuejun <xuejun.zhai@intel.com>
ba76b45194