IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[AUTO] Update property setting rules (#13848)

Browse Source 
* [AUTO] update property setting logic.

* Update core::set_property() logic to only support primary property for AUTO and MULTI.
* Separate AUTO and MULTI supported properties.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Add test case.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* 1. revert the changes for ie core.
2. Enable AUTO/MULTI only accepting its own properties.
3. Enable AUTO/MULTI accepting device properties passed from loadnetwork().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Revert "[AUTO]Update the incorrect config test for Myriad (#13271)"

This reverts commit 0552d98802.

* MULTI only accepts its own properties that is same as AUTO currently.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Add test cases for AUTO/MULTI property test.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update to enable MULTI supporting hw primary property setting throw the compile_model().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Remove the added test case for setting secondary property through set_property().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* 1. For AUTO/MULTI, property setting will be passed via core::compile_model() instead of core::set_property().
2. update the logic to infer precision setting that will transform into secondary property setting to each hw device.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* set default value for nstreams when -d AUTO/MULTI and no nstreams setting from command line.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update code format.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Enable BA to support if -d AUTO:MULTI,xxx/MULTI:AUTO,xxx. while AUTO Plugin need to update the logic of generating supported config list to virtual device.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* move device configuration handling outside form function main.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Revert "move device configuration handling outside form function main."

This reverts commit ef77bfc602.

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Setting property performance hint to 'THROUGHPUT' and passing to executable network if no setting for of AUTO/MULTI.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>
This commit is contained in:
Wang, Yang
2022-11-17 16:38:47 +08:00
committed by GitHub
parent 7c4b7355e6
commit 678b90a661
7 changed files with 378 additions and 91 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
samples/cpp/benchmark_app/utils.cpp
View File

@@ -117,8 +117,6 @@ std::vector<std::string> parse_devices(const std::string& device_string) {
auto bracket = comma_separated_devices.find("("); // e.g. in BATCH:GPU(4)
comma_separated_devices = comma_separated_devices.substr(colon + 1, bracket - colon - 1);
}
if ((comma_separated_devices == "MULTI") || (comma_separated_devices == "HETERO"))
return std::vector<std::string>();
auto devices = split(comma_separated_devices, ',');
result.insert(result.end(), devices.begin(), devices.end());
@@ -130,7 +128,7 @@ void parse_value_for_virtual_device(const std::string& device, std::map<std::str
if (item_virtual != values_string.end() && values_string.size() > 1) {
if (device == "MULTI") {
// Remove the element that the key is virtual device MULTI
// e.g. MULTI:xxx,xxx -nstreams 2 will set nstreams 2 to CPU.
// e.g. MULTI:xxx -nstreams 2 will set nstreams 2 to xxx.
values_string.erase(item_virtual);
} else if (device == "AUTO") {
// Just keep the element that the key is virtual device AUTO
@@ -153,13 +151,7 @@ void parse_value_for_virtual_device(const std::string& device, std::map<std::str
if (values_string.find(device) != values_string.end()) {
auto& nstreams = values_string[device];
// Remove the space at the tail.
nstreams.erase(std::find_if(nstreams.rbegin(),
nstreams.rend(),
[](unsigned char ch) {
return !std::isspace(ch);
})
.base(),
nstreams.end());
nstreams.pop_back();
}
return;
}
@@ -173,13 +165,17 @@ std::map<std::string, std::string> parse_value_per_device(const std::vector<std:
auto device_value_vec = split(device_value_string, ':');
if (device_value_vec.size() == 2) {
auto device_name = device_value_vec.at(0);
auto nstreams = device_value_vec.at(1);
auto value = device_value_vec.at(1);
auto it = std::find(devices.begin(), devices.end(), device_name);
if (it != devices.end()) {
result[device_name] = nstreams;
result[device_name] = value;
} else {
throw std::logic_error("Can't set nstreams value " + std::string(nstreams) + " for device '" +
device_name + "'! Incorrect device name!");
std::string devices_list = "";
for (auto& device : devices)
devices_list += device + " ";
devices_list.pop_back();
throw std::logic_error("Failed to set property to '" + device_name +
"' which is not found whthin the target devices list '" + devices_list + "'!");
}
} else if (device_value_vec.size() == 1) {
auto value = device_value_vec.at(0);