IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[MULTI]Data affinity remote context and blobs (#3342)

Browse Source 
* zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these

* Optimized Infer Request Scheduling

* remoteblob checks in the conventional SetBlob

* correctly (with status) reporting NOT_IMPLEMENTED

* SetBlob to accomodate for the RemoteBobs

* Tests for remote blobs support via MULTI: creating the shared_test in case the other (closed source) plugins would want to use that (in the private shared_tests instantiations).
Also instantiating the remote blobs tests for the some basic combinations to test the MULTI supports them

* macos compilation (and general plugin platform support) fix

* shuffled files, so that the MULTI tests are now part of the ieFuncTests (and need no separate target). Also brushed the macro that handales the NOT_IMPLEMENTED as bit

* further shuffled files, so that the initial MULTI tests are now part of the IE tests, yet specific instances do need separate targets

* Fixed misprint

* Brushing the code and comments a bit

* further brushing of the ScheduleToWorkerRequest: moving the task execution directly into the loop over devices (avoids pointers and 'else' clause)

* 1) zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these

2) Initial MULTI section in the opt guide (primarily to document a tip on helping the MULTI to keep the zero-copy path)

* [MULTI] remote context support and associated scheduling (respecting the remote data affinity)

* fix CentOS (old) gcc issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81880
since the intriduced therad_local string is template the bug manifests itself (and the string is not allocated/initialized).
the QA is to wrap the std::string into the function

* further fix for the old gcc versions issue, now with non-trivial thread_local destruction sefault: switching from the std::string to the plain const char*

* additional tests for the MULTI and remote blobs (no remote context and multi GPUs cases)

* fix for the tests (that now can check for more specific NotImplemented exeption).
Alos couple of line endings
This commit is contained in:
Maxim Shevtsov
2020-12-09 09:52:19 +03:00
committed by GitHub
parent 8b9feed603
commit d0eef043fd
14 changed files with 312 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
inference-engine/include/cpp/ie_infer_request.hpp
View File

@@ -14,6 +14,7 @@
#include <string>
#include "cpp/ie_memory_state.hpp"
#include "ie_remote_context.hpp"
#include "ie_iinfer_request.hpp"
#include "details/ie_exception_conversion.hpp"
#include "details/ie_so_loader.h"
@@ -123,8 +124,9 @@ public:
CALL_STATUS_FNC(GetBlob, name.c_str(), data);
std::string error = "Internal error: blob with name `" + name + "` is not allocated!";
auto blobPtr = data.get();
const bool remoteBlobPassed = blobPtr->is<RemoteBlob>();
if (blobPtr == nullptr) THROW_IE_EXCEPTION << error;
if (blobPtr->buffer() == nullptr) THROW_IE_EXCEPTION << error;
if (!remoteBlobPassed && blobPtr->buffer() == nullptr) THROW_IE_EXCEPTION << error;
return data;
}