IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[WA] performance issue for plugin api 2.0 (#18509)

Browse Source 
* [WA] performance issue for plugin api 2.0

* Avoid onnx failure

* Avoid ov_template_func_tests failure

[ RUN      ] smoke_BehaviorTests/OVRemoteTest.inferWithRemoteNoThrow/element_type=f32targetDevice=TEMPLATE__context_parameters=__tensor_parameters=

MEM_USAGE=400336KB
/mnt/data1_1c41/river/openvino_master/openvino/src/tests/functional/plugin/shared/src/behavior/ov_plugin/remote.cpp:107: Failure
Expected: infer_request.set_input_tensor(0, input_remote_tensor) doesn't throw an exception.
  Actual: it throws.
[  FAILED  ] smoke_BehaviorTests/OVRemoteTest.inferWithRemoteNoThrow/element_type=f32targetDevice=TEMPLATE__context_parameters=__tensor_parameters=, where GetParam() = (f32, "TEMPLATE", {}, ({}, {})) (76 ms)
This commit is contained in:
River Li 2023-07-20 12:22:31 +08:00 committed by GitHub
parent b7b740beb5
commit 0e76496acc
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 11 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/inference/dev_api/openvino/runtime/iasync_infer_request.hpp
View File

@ -172,6 +172,10 @@ protected:
* @brief Throws exception if inference request is busy or canceled
*/
void check_state() const;
/**
* @brief Throws exception if inference request is cancelled
*/
void check_cancelled_state() const;
/**
* @brief Performs inference of pipeline in syncronous mode
* @note Used by Infer which ensures thread-safety and calls this method after.

6
src/inference/src/dev/iasync_infer_request.cpp
View File

@ -201,6 +201,12 @@ void ov::IAsyncInferRequest::check_state() const {
}
}
void ov::IAsyncInferRequest::check_cancelled_state() const {
std::lock_guard<std::mutex> lock{m_mutex};
if (m_state == InferState::CANCELLED)
ov::Cancelled::create("Infer Request was canceled");
}
std::vector<ov::ProfilingInfo> ov::IAsyncInferRequest::get_profiling_info() const {
check_state();
return m_sync_request->get_profiling_info();

1
src/inference/src/dev/isync_infer_request.cpp
View File

@ -115,6 +115,7 @@ const std::shared_ptr<const ov::ICompiledModel>& ov::ISyncInferRequest::get_comp
}
ov::ISyncInferRequest::FoundPort ov::ISyncInferRequest::find_port(const ov::Output<const ov::Node>& port) const {
// This function is hotspot, need optimization.
auto check_nodes = [](const ov::Node* node1, const ov::Node* node2) {
return node1 == node2 ||
(node1->get_friendly_name() == node2->get_friendly_name() &&