IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[AUTO] Update property setting rules (#13848)

Browse Source 
* [AUTO] update property setting logic.

* Update core::set_property() logic to only support primary property for AUTO and MULTI.
* Separate AUTO and MULTI supported properties.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Add test case.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* 1. revert the changes for ie core.
2. Enable AUTO/MULTI only accepting its own properties.
3. Enable AUTO/MULTI accepting device properties passed from loadnetwork().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Revert "[AUTO]Update the incorrect config test for Myriad (#13271)"

This reverts commit 0552d98802.

* MULTI only accepts its own properties that is same as AUTO currently.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Add test cases for AUTO/MULTI property test.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update to enable MULTI supporting hw primary property setting throw the compile_model().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Remove the added test case for setting secondary property through set_property().

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* 1. For AUTO/MULTI, property setting will be passed via core::compile_model() instead of core::set_property().
2. update the logic to infer precision setting that will transform into secondary property setting to each hw device.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* set default value for nstreams when -d AUTO/MULTI and no nstreams setting from command line.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update code format.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Enable BA to support if -d AUTO:MULTI,xxx/MULTI:AUTO,xxx. while AUTO Plugin need to update the logic of generating supported config list to virtual device.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* move device configuration handling outside form function main.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Revert "move device configuration handling outside form function main."

This reverts commit ef77bfc602.

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Setting property performance hint to 'THROUGHPUT' and passing to executable network if no setting for of AUTO/MULTI.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>

* Update.

Signed-off-by: Wang, Yang <yang4.wang@intel.com>
This commit is contained in:
Wang, Yang 2022-11-17 16:38:47 +08:00 committed by GitHub
parent 7c4b7355e6
commit 678b90a661
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 378 additions and 91 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

85
samples/cpp/benchmark_app/main.cpp
View File

@ -187,6 +187,33 @@ ov::hint::PerformanceMode get_performance_hint(const std::string& device, const
return ov_perf_hint;
}
void setDeviceProperty(ov::Core& core,
std::string& device,
ov::AnyMap& device_config,
const std::pair<std::string, ov::Any>& property,
const std::pair<std::string, ov::Any>& config = {}) {
auto supported_properties = core.get_property(device, ov::supported_properties);
auto supported = [&](const std::string& key) {
return std::find(std::begin(supported_properties), std::end(supported_properties), key) !=
std::end(supported_properties);
};
// check if the HW device supported this property
std::pair<std::string, ov::Any> device_property;
if (!config.first.empty() && supported(config.first)) {
device_property = config;
} else if (supported(property.first))
device_property = property;
if (device_property.first.empty())
return;
if (device_config.find(device) == device_config.end()) {
device_config.insert(ov::device::properties(device, device_property));
} else {
auto& properties = device_config[device].as<ov::AnyMap>();
properties.emplace(device_property);
}
}
/**
* @brief The entry point of the benchmark application
*/
@ -319,13 +346,15 @@ int main(int argc, char* argv[]) {
// Remove the hardware devices if AUTO/MULTI appears in the devices list.
if (if_auto || if_multi) {
devices.clear();
std::string virtual_device = if_auto ? "AUTO" : "MULTI";
// Parse out the currect virtual device as the target device.
std::string virtual_device = split(device_name, ':').at(0);
auto iter_virtual = std::find(hardware_devices.begin(), hardware_devices.end(), virtual_device);
hardware_devices.erase(iter_virtual);
devices.push_back(virtual_device);
parse_value_for_virtual_device(virtual_device, device_nstreams);
parse_value_for_virtual_device(virtual_device, device_infer_precision);
}
// Update config per device according to command line parameters
for (auto& device : devices) {
auto& device_config = config[device];
@ -399,7 +428,6 @@ int main(int argc, char* argv[]) {
else {
auto& property = device_config[it.first].as<ov::AnyMap>();
property.emplace(ov::num_streams(std::stoi(it.second)));
device_config.insert(ov::device::properties(it.first, property));
}
}
}
@ -427,6 +455,17 @@ int main(int argc, char* argv[]) {
// Use API 2.0 key for streams
key = ov::num_streams.name();
device_config[key] = ov::streams::AUTO;
} else if (device == "MULTI" || device == "AUTO") {
// Set nstreams to default value auto if no nstreams specified from cmd line.
std::string key = std::string(getDeviceTypeFromName(device) + "_THROUGHPUT_STREAMS");
for (auto& hwdevice : hardware_devices) {
auto value = std::string(getDeviceTypeFromName(hwdevice) + "_THROUGHPUT_AUTO");
setDeviceProperty(core,
hwdevice,
device_config,
ov::num_streams(ov::streams::AUTO),
std::make_pair(key, value));
}
}
}
}
@ -439,14 +478,36 @@ int main(int argc, char* argv[]) {
auto it_device_infer_precision = device_infer_precision.find(device);
if (it_device_infer_precision != device_infer_precision.end()) {
// set to user defined value
if (!supported(ov::hint::inference_precision.name())) {
if (supported(ov::hint::inference_precision.name())) {
device_config.emplace(ov::hint::inference_precision(it_device_infer_precision->second));
} else if (device == "MULTI" || device == "AUTO") {
// check if the element contains the hardware device property
auto value_vec = split(it_device_infer_precision->second, ' ');
if (value_vec.size() == 1) {
auto key = ov::hint::inference_precision.name();
device_config[key] = it_device_infer_precision->second;
} else {
// set device inference_precison properties in the AUTO/MULTI plugin
std::stringstream strm(it_device_infer_precision->second);
std::map<std::string, std::string> devices_property;
ov::util::Read<std::map<std::string, std::string>>{}(strm, devices_property);
for (auto it : devices_property) {
if (device_config.find(it.first) == device_config.end())
device_config.insert(
ov::device::properties(it.first, ov::hint::inference_precision(it.second)));
else {
auto& property = device_config[it.first].as<ov::AnyMap>();
property.emplace(ov::hint::inference_precision(it.second));
}
}
}
} else {
throw std::logic_error("Device " + device + " doesn't support config key '" +
ov::hint::inference_precision.name() + "'! " +
"Please specify -infer_precision for correct devices in format "
"<dev1>:<infer_precision1>,<dev2>:<infer_precision2>" +
" or via configuration file.");
}
device_config.emplace(ov::hint::inference_precision(it_device_infer_precision->second));
}
};
@ -463,17 +524,15 @@ int main(int argc, char* argv[]) {
auto property_name = str == "nthreads" ? ov::inference_num_threads.name() : ov::affinity.name();
auto property = str == "nthreads" ? ov::inference_num_threads(FLAGS_nthreads)
: ov::affinity(fix_pin_option(FLAGS_pin));
if (supported(property_name) || if_multi) {
if (supported(property_name) || device_name == "AUTO") {
// create nthreads/pin primary property for HW device or AUTO if -d is AUTO directly.
device_config.emplace(property);
} else if (if_auto) {
} else if (if_auto || if_multi) {
// Create secondary property of -nthreads/-pin only for CPU if CPU device appears in the devices
// list specified by -d.
for (auto& device : hardware_devices) {
if (device_config.find(device) == device_config.end()) {
device_config.insert(ov::device::properties(device, property));
} else {
auto& properties = device_config[device].as<ov::AnyMap>();
properties.emplace(property);
device_config.insert(ov::device::properties(device, properties));
}
if (device == "CPU")
setDeviceProperty(core, device, device_config, property);
}
}
};

24
samples/cpp/benchmark_app/utils.cpp
View File

@ -117,8 +117,6 @@ std::vector<std::string> parse_devices(const std::string& device_string) {
auto bracket = comma_separated_devices.find("("); // e.g. in BATCH:GPU(4)
comma_separated_devices = comma_separated_devices.substr(colon + 1, bracket - colon - 1);
}
if ((comma_separated_devices == "MULTI") || (comma_separated_devices == "HETERO"))
return std::vector<std::string>();
auto devices = split(comma_separated_devices, ',');
result.insert(result.end(), devices.begin(), devices.end());
@ -130,7 +128,7 @@ void parse_value_for_virtual_device(const std::string& device, std::map<std::str
if (item_virtual != values_string.end() && values_string.size() > 1) {
if (device == "MULTI") {
// Remove the element that the key is virtual device MULTI
// e.g. MULTI:xxx,xxx -nstreams 2 will set nstreams 2 to CPU.
// e.g. MULTI:xxx -nstreams 2 will set nstreams 2 to xxx.
values_string.erase(item_virtual);
} else if (device == "AUTO") {
// Just keep the element that the key is virtual device AUTO
@ -153,13 +151,7 @@ void parse_value_for_virtual_device(const std::string& device, std::map<std::str
if (values_string.find(device) != values_string.end()) {
auto& nstreams = values_string[device];
// Remove the space at the tail.
nstreams.erase(std::find_if(nstreams.rbegin(),
nstreams.rend(),
[](unsigned char ch) {
return !std::isspace(ch);
})
.base(),
nstreams.end());
nstreams.pop_back();
}
return;
}
@ -173,13 +165,17 @@ std::map<std::string, std::string> parse_value_per_device(const std::vector<std:
auto device_value_vec = split(device_value_string, ':');
if (device_value_vec.size() == 2) {
auto device_name = device_value_vec.at(0);
auto nstreams = device_value_vec.at(1);
auto value = device_value_vec.at(1);
auto it = std::find(devices.begin(), devices.end(), device_name);
if (it != devices.end()) {
result[device_name] = nstreams;
result[device_name] = value;
} else {
throw std::logic_error("Can't set nstreams value " + std::string(nstreams) + " for device '" +
device_name + "'! Incorrect device name!");
std::string devices_list = "";
for (auto& device : devices)
devices_list += device + " ";
devices_list.pop_back();
throw std::logic_error("Failed to set property to '" + device_name +
"' which is not found whthin the target devices list '" + devices_list + "'!");
}
} else if (device_value_vec.size() == 1) {
auto value = device_value_vec.at(0);

16
src/plugins/auto/auto_executable_network.cpp
View File

@ -3,6 +3,7 @@
//
///////////////////////////////////////////////////////////////////////////////////////////////////
#include "ie_performance_hints.hpp"
#include "auto_executable_network.hpp"
// ------------------------------AutoExecutableNetwork----------------------------
@ -45,8 +46,19 @@ IE::Parameter AutoExecutableNetwork::GetMetric(const std::string& name) const {
ov::PropertyName{ov::optimal_number_of_infer_requests.name(), ov::PropertyMutability::RO},
ov::PropertyName{ov::hint::model_priority.name(), ov::PropertyMutability::RO},
ov::PropertyName{ov::device::priorities.name(), ov::PropertyMutability::RO},
ov::PropertyName{ov::execution_devices.name(), ov::PropertyMutability::RO}
};
ov::PropertyName{ov::execution_devices.name(), ov::PropertyMutability::RO}};
} else if (name == ov::hint::performance_mode) {
auto value = _autoSContext->_performanceHint;
if (!_autoSContext->_core->isNewAPI())
return value;
if (value == InferenceEngine::PluginConfigParams::THROUGHPUT)
return ov::hint::PerformanceMode::THROUGHPUT;
else if (value == InferenceEngine::PluginConfigParams::LATENCY)
return ov::hint::PerformanceMode::LATENCY;
else if (value == InferenceEngine::PluginConfigParams::CUMULATIVE_THROUGHPUT)
return ov::hint::PerformanceMode::CUMULATIVE_THROUGHPUT;
else
return ov::hint::PerformanceMode::UNDEFINED;
} else if (name == ov::device::priorities) {
auto value = _autoSContext->_config.find(ov::device::priorities.name());
return decltype(ov::device::priorities)::value_type {value->second.as<std::string>()};

61
src/plugins/auto/plugin.cpp
View File

@ -52,22 +52,8 @@ namespace {
}
return METRIC_VALUE(FP32);
}
std::vector<std::string> supported_configKeys = []() -> decltype(PerfHintsConfig::SupportedKeys()) {
auto res = PerfHintsConfig::SupportedKeys();
res.push_back(ov::device::priorities.name());
res.push_back(ov::enable_profiling.name());
res.push_back(PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS);
res.push_back(ov::hint::model_priority.name());
res.push_back(ov::hint::allow_auto_batching.name());
res.push_back(ov::log::level.name());
res.push_back(ov::intel_auto::device_bind_buffer.name());
res.push_back(ov::auto_batch_timeout.name());
return res;
}();
} // namespace
std::mutex MultiDeviceInferencePlugin::_mtx;
std::map<unsigned int, std::list<std::string>> MultiDeviceInferencePlugin::_priorityMap;
@ -281,41 +267,11 @@ MultiDeviceInferencePlugin::MultiDeviceInferencePlugin() {
InferenceEngine::Parameter MultiDeviceInferencePlugin::GetMetric(const std::string& name,
const std::map<std::string, InferenceEngine::Parameter> & options) const {
auto RO_property = [](const std::string& propertyName) {
return ov::PropertyName(propertyName, ov::PropertyMutability::RO);
};
auto RW_property = [](const std::string& propertyName) {
return ov::PropertyName(propertyName, ov::PropertyMutability::RW);
};
if (name == ov::supported_properties) {
std::vector<ov::PropertyName> roProperties {RO_property(ov::supported_properties.name()),
RO_property(ov::device::full_name.name()),
RO_property(ov::device::capabilities.name())
};
// the whole config is RW before network is loaded.
std::vector<ov::PropertyName> rwProperties {RW_property(ov::hint::model_priority.name()),
RW_property(ov::log::level.name()),
RW_property(ov::device::priorities.name()),
RW_property(ov::enable_profiling.name()),
RW_property(ov::hint::allow_auto_batching.name()),
RW_property(ov::auto_batch_timeout.name()),
RW_property(ov::hint::performance_mode.name()),
RW_property(ov::hint::num_requests.name()),
RW_property(ov::intel_auto::device_bind_buffer.name()),
RW_property(ov::cache_dir.name())
};
std::vector<ov::PropertyName> supportedProperties;
supportedProperties.reserve(roProperties.size() + rwProperties.size());
supportedProperties.insert(supportedProperties.end(), roProperties.begin(), roProperties.end());
supportedProperties.insert(supportedProperties.end(), rwProperties.begin(), rwProperties.end());
return supportedProperties;
auto ret = _pluginConfig.supportedProperties(GetName());
return ret;
} else if (name == METRIC_KEY(SUPPORTED_METRICS)) {
std::vector<std::string> metrics;
metrics.push_back(METRIC_KEY(SUPPORTED_METRICS));
metrics.push_back(METRIC_KEY(FULL_DEVICE_NAME));
metrics.push_back(METRIC_KEY(SUPPORTED_CONFIG_KEYS));
metrics.push_back(METRIC_KEY(OPTIMIZATION_CAPABILITIES));
IE_SET_METRIC_RETURN(SUPPORTED_METRICS, metrics);
IE_SET_METRIC_RETURN(SUPPORTED_METRICS, _pluginConfig.supportedMetrics(GetName()));
} else if (name == ov::device::full_name) {
std::string device_name = { GetName() };
return decltype(ov::device::full_name)::value_type {device_name};
@ -334,7 +290,7 @@ InferenceEngine::Parameter MultiDeviceInferencePlugin::GetMetric(const std::stri
}
IE_SET_METRIC_RETURN(OPTIMIZATION_CAPABILITIES, capabilities);
} else if (name == METRIC_KEY(SUPPORTED_CONFIG_KEYS)) {
IE_SET_METRIC_RETURN(SUPPORTED_CONFIG_KEYS, supported_configKeys);
IE_SET_METRIC_RETURN(SUPPORTED_CONFIG_KEYS, _pluginConfig.supportedConfigKeys(GetName()));
} else {
IE_THROW() << "Unsupported metric key: " << name;
}
@ -371,11 +327,14 @@ IExecutableNetworkInternal::Ptr MultiDeviceInferencePlugin::LoadNetworkImpl(cons
_LogTag = GetName();
auto loadConfig = _pluginConfig;
// updateFromMap will check config valid
loadConfig.UpdateFromMap(config, GetName());
loadConfig.UpdateFromMap(config, GetName(), true);
auto fullConfig = loadConfig._keyConfigMap;
// Remove the performance hint if no setting to this property from user.
if (!loadConfig._isSetPerHint)
if (!loadConfig._isSetPerHint) {
fullConfig.erase(PluginConfigParams::KEY_PERFORMANCE_HINT);
// set performance hint to 'THROUGHPUT' model for AutoExecutable Network.
loadConfig._perfHintsConfig.SetConfig(PluginConfigParams::KEY_PERFORMANCE_HINT, PluginConfigParams::THROUGHPUT);
}
if (!loadConfig._isSetCacheDir)
fullConfig.erase(CONFIG_KEY(CACHE_DIR));
// collect the settings that are applicable to the devices we are loading the network to
@ -588,7 +547,7 @@ QueryNetworkResult MultiDeviceInferencePlugin::QueryNetwork(const CNNNetwork&
auto queryconfig = _pluginConfig;
// updateFromMap will check config valid
queryconfig.UpdateFromMap(config, GetName());
queryconfig.UpdateFromMap(config, GetName(), true);
auto fullConfig = queryconfig._keyConfigMap;
auto priorities = fullConfig.find(MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES);
if (priorities->second.empty()) {

77
src/plugins/auto/utils/config.hpp
View File

@ -27,8 +27,69 @@ struct PluginConfig {
_logLevel("LOG_NONE") {
adjustKeyMapValues();
}
std::vector<std::string> supportedConfigKeys(const std::string& pluginName = "AUTO") const {
std::vector<std::string> supported_configKeys = []() -> decltype(PerfHintsConfig::SupportedKeys()) {
auto res = PerfHintsConfig::SupportedKeys();
res.push_back(ov::device::priorities.name());
res.push_back(ov::enable_profiling.name());
res.push_back(PluginConfigParams::KEY_EXCLUSIVE_ASYNC_REQUESTS);
res.push_back(ov::hint::model_priority.name());
res.push_back(ov::hint::allow_auto_batching.name());
res.push_back(ov::log::level.name());
res.push_back(ov::intel_auto::device_bind_buffer.name());
res.push_back(ov::auto_batch_timeout.name());
return res;
}();
auto multi_supported_configKeys = supported_configKeys;
return pluginName == "AUTO" ? supported_configKeys : multi_supported_configKeys;
}
void UpdateFromMap(const std::map<std::string, std::string>& config, const std::string& pluginName) {
std::vector<ov::PropertyName> supportedProperties(const std::string& pluginName = "AUTO") const {
std::vector<ov::PropertyName> supported_properties = []() -> std::vector<ov::PropertyName> {
auto RO_property = [](const std::string& propertyName) {
return ov::PropertyName(propertyName, ov::PropertyMutability::RO);
};
auto RW_property = [](const std::string& propertyName) {
return ov::PropertyName(propertyName, ov::PropertyMutability::RW);
};
std::vector<ov::PropertyName> roProperties{RO_property(ov::supported_properties.name()),
RO_property(ov::device::full_name.name()),
RO_property(ov::device::capabilities.name())};
// the whole config is RW before network is loaded.
std::vector<ov::PropertyName> rwProperties{RW_property(ov::hint::model_priority.name()),
RW_property(ov::log::level.name()),
RW_property(ov::device::priorities.name()),
RW_property(ov::enable_profiling.name()),
RW_property(ov::hint::allow_auto_batching.name()),
RW_property(ov::auto_batch_timeout.name()),
RW_property(ov::hint::performance_mode.name()),
RW_property(ov::hint::num_requests.name()),
RW_property(ov::intel_auto::device_bind_buffer.name()),
RW_property(ov::cache_dir.name())};
std::vector<ov::PropertyName> supportedProperties;
supportedProperties.reserve(roProperties.size() + rwProperties.size());
supportedProperties.insert(supportedProperties.end(), roProperties.begin(), roProperties.end());
supportedProperties.insert(supportedProperties.end(), rwProperties.begin(), rwProperties.end());
return supportedProperties;
}();
auto multi_supported_properties = supported_properties;
return pluginName == "AUTO" ? supported_properties : multi_supported_properties;
}
std::vector<std::string> supportedMetrics(const std::string& pluginName = "AUTO") const {
std::vector<std::string> supported_metrics = []() -> std::vector<std::string> {
std::vector<std::string> metrics;
metrics.push_back(METRIC_KEY(SUPPORTED_METRICS));
metrics.push_back(METRIC_KEY(FULL_DEVICE_NAME));
metrics.push_back(METRIC_KEY(SUPPORTED_CONFIG_KEYS));
metrics.push_back(METRIC_KEY(OPTIMIZATION_CAPABILITIES));
return metrics;
}();
auto multi_supported_metrics = supported_metrics;
return pluginName == "AUTO" ? supported_metrics : multi_supported_metrics;
}
void UpdateFromMap(const std::map<std::string, std::string>& config, const std::string& pluginName, bool supportHWProprety = false) {
const auto perf_hints_configs = PerfHintsConfig::SupportedKeys();
for (auto&& kvp : config) {
if (kvp.first == ov::enable_profiling) {
@ -102,17 +163,25 @@ struct PluginConfig {
} else if (_availableDevices.end() != std::find(_availableDevices.begin(),
_availableDevices.end(),
DeviceIDParser(kvp.first).getDeviceName())) {
// AUTO and MULTI can accept secondary properites on calling both core::comile_model() and
// core::set_property().
_passThroughConfig.emplace(kvp.first, kvp.second);
// Not setting performance mode to 'THROUGHPUT' as the default value if any secondary properties
// appears in the configuration.
_isSetPerHint = true;
} else if (kvp.first.find("AUTO_") == 0) {
_passThroughConfig.emplace(kvp.first, kvp.second);
} else if (kvp.first == ov::cache_dir.name()) {
_cacheDir = kvp.second;
_isSetCacheDir = true;
} else {
if (pluginName.find("AUTO") != std::string::npos)
if (pluginName.find("AUTO") != std::string::npos || !supportHWProprety)
// AUTO and MULTI just only accept its own properites and secondary property when calling
// core::set_property().
IE_THROW(NotFound) << "Unsupported property " << kvp.first;
else
_passThroughConfig.emplace(kvp.first, kvp.second);
// MULTI could accept the HW primary property like {"NUM_STREAMS", "4"}
_passThroughConfig.emplace(kvp.first, kvp.second);
}
}
if (!config.empty())

40
src/tests/functional/plugin/myriad/shared_tests_instances/behavior/plugin/configuration_tests.cpp
View File

@ -767,13 +767,39 @@ INSTANTIATE_TEST_SUITE_P(smoke_BehaviorTests, IncorrectConfigTests,
const std::vector<std::map<std::string, std::string>>& getIncorrectMultiConfigs() {
static const std::vector<std::map<std::string, std::string>> incorrectMultiConfigs = {
{{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::PluginConfigParams::KEY_PERFORMANCE_HINT, "DOESN'T EXIST"}},
{{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::PluginConfigParams::KEY_PERFORMANCE_HINT, InferenceEngine::PluginConfigParams::LATENCY},
{InferenceEngine::PluginConfigParams::KEY_PERFORMANCE_HINT_NUM_REQUESTS, "-1"}},
{{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::PluginConfigParams::KEY_PERF_COUNT, "ON"}}};
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{KEY_LOG_LEVEL, "INCORRECT_LOG_LEVEL"},
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::MYRIAD_PROTOCOL, "BLUETOOTH"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::MYRIAD_ENABLE_HW_ACCELERATION, "ON"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::MYRIAD_ENABLE_RECEIVING_TENSOR_TIME, "ON"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{KEY_PERF_COUNT, "ON"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::MYRIAD_THROUGHPUT_STREAMS, "ONE"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{KEY_EXCLUSIVE_ASYNC_REQUESTS, "ON"}
},
{
{InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, CommonTestUtils::DEVICE_MYRIAD},
{InferenceEngine::MYRIAD_DDR_TYPE, "1GB"}
},
};
return incorrectMultiConfigs;
}

166
src/tests/unit/auto/auto_property_test.cpp Normal file
View File

@ -0,0 +1,166 @@
// Copyright (C) 2018-2022 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <gmock/gmock.h>
#include <gtest/gtest.h>
#include <common_test_utils/test_constants.hpp>
#include <ie_core.hpp>
#include <ie_metric_helpers.hpp>
#include <multi-device/multi_device_config.hpp>
#include <ngraph_functions/subgraph_builders.hpp>
#include "cpp/ie_plugin.hpp"
#include "mock_common.hpp"
#include "openvino/runtime/compiled_model.hpp"
#include "openvino/runtime/core.hpp"
#include "plugin/mock_auto_device_plugin.hpp"
#include "unit_test_utils/mocks/cpp_interfaces/impl/mock_inference_plugin_internal.hpp"
#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_icore.hpp"
#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_iexecutable_network_internal.hpp"
#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_iinference_plugin.hpp"
#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_ivariable_state_internal.hpp"
#include "unit_test_utils/mocks/mock_iinfer_request.hpp"
using ::testing::_;
using ::testing::AllOf;
using ::testing::AnyNumber;
using ::testing::AtLeast;
using ::testing::ContainsRegex;
using ::testing::Eq;
using ::testing::MatcherCast;
using ::testing::Matches;
using ::testing::NiceMock;
using ::testing::Property;
using ::testing::Return;
using ::testing::ReturnRef;
using ::testing::StrEq;
using ::testing::Throw;
using Config = std::map<std::string, std::string>;
using namespace MockMultiDevice;
using ConfigParams = std::tuple<std::string, // target device name
ov::AnyMap // configuration
>;
class SetPropertyThroughAuto : public ::testing::TestWithParam<ConfigParams> {
public:
std::shared_ptr<ov::Core> core;
std::shared_ptr<NiceMock<MockMultiDeviceInferencePlugin>> plugin;
std::shared_ptr<ngraph::Function> actualNetwork;
std::string device;
ov::AnyMap configuration;
void TearDown() override {
core.reset();
plugin.reset();
actualNetwork.reset();
}
void SetUp() override {
std::tie(device, configuration) = this->GetParam();
core = std::make_shared<ov::Core>();
NiceMock<MockMultiDeviceInferencePlugin>* mock_multi = new NiceMock<MockMultiDeviceInferencePlugin>();
plugin.reset(mock_multi);
// Generic network
actualNetwork = ngraph::builder::subgraph::makeSplitConvConcat();
}
};
using SetUnsupportedPropertyTestP = SetPropertyThroughAuto;
using SetSupportedPropertyTestP = SetPropertyThroughAuto;
using LoadNetworkWithSupportedPropertyTestP = SetPropertyThroughAuto;
using LoadNetworkWithUnsupportedPropertyTestP = SetPropertyThroughAuto;
TEST(SetPropertyOverwriteTest, smoke_AUTO_SetPropertyOverwriteTestNoThrow) {
ov::Core ie;
int32_t curValue = -1;
auto actualNetwork = ngraph::builder::subgraph::makeSplitConvConcat();
ASSERT_NO_THROW(ie.set_property(CommonTestUtils::DEVICE_CPU, {ov::num_streams(2)}));
ASSERT_NO_THROW(ie.set_property(CommonTestUtils::DEVICE_AUTO, {ov::device::properties("CPU", ov::num_streams(4))}));
ASSERT_NO_THROW(ie.compile_model(actualNetwork, CommonTestUtils::DEVICE_AUTO, {}));
ASSERT_NO_THROW(curValue = ie.get_property(CommonTestUtils::DEVICE_CPU, ov::num_streams));
EXPECT_EQ(curValue, 4);
}
TEST(SetPropertyOverwriteTest, smoke_MULTI_SetPropertyOverwriteTestNoThrow) {
ov::Core ie;
int32_t curValue = -1;
auto actualNetwork = ngraph::builder::subgraph::makeSplitConvConcat();
ASSERT_NO_THROW(ie.set_property(CommonTestUtils::DEVICE_CPU, {ov::num_streams(2)}));
ASSERT_NO_THROW(
ie.set_property(CommonTestUtils::DEVICE_MULTI, {ov::device::properties("CPU", ov::num_streams(4))}));
ASSERT_NO_THROW(ie.compile_model(actualNetwork, CommonTestUtils::DEVICE_MULTI, {ov::device::priorities("CPU")}));
ASSERT_NO_THROW(curValue = ie.get_property(CommonTestUtils::DEVICE_CPU, ov::num_streams));
EXPECT_EQ(curValue, 4);
}
TEST_P(SetSupportedPropertyTestP, setSupportedPropertyTestNoThrow) {
ASSERT_NO_THROW(core->set_property(device, configuration));
ov::AnyMap config = {};
if (device == "MULTI")
config = {ov::device::priorities("CPU")};
ASSERT_NO_THROW(core->compile_model(actualNetwork, device, config));
}
TEST_P(SetUnsupportedPropertyTestP, setUnsupportedPropertyTestNoThrow) {
ASSERT_NO_THROW(core->set_property(device, configuration));
ov::AnyMap config = {};
if (device == "MULTI")
config = {ov::device::priorities("CPU")};
ASSERT_THROW(core->compile_model(actualNetwork, device, config), ov::Exception);
}
TEST_P(LoadNetworkWithSupportedPropertyTestP, loadNetworkSupportedPropertyTestNoThrow) {
if (device == "MULTI")
ASSERT_NO_THROW(core->set_property(device, {ov::device::priorities("CPU")}));
ASSERT_NO_THROW(core->compile_model(actualNetwork, device, configuration));
}
TEST_P(LoadNetworkWithUnsupportedPropertyTestP, smoke_Auto_Multi_LoadNetworkUnsupportedPropertyTestNoThrow) {
if (device == "MULTI")
ASSERT_NO_THROW(core->set_property(device, {ov::device::priorities("CPU")}));
ASSERT_THROW(core->compile_model(actualNetwork, device, configuration), ov::Exception);
}
const std::vector<ov::AnyMap> configsSupportedPropertyTest = {
{ov::enable_profiling(true)},
{ov::enable_profiling(true), ov::device::properties("CPU", ov::num_streams(4))}};
const std::vector<ov::AnyMap> configsUnsupportedPropertyTest = {
{ov::num_streams(4)},
{ov::num_streams(4), ov::enable_profiling(true)},
{ov::device::properties("INVALID", ov::num_streams(4))}};
const std::vector<ov::AnyMap> configsMultiLoadNetworkSupportedPropertyTest = {
{ov::num_streams(4)},
{ov::enable_profiling(true)},
{ov::num_streams(4), ov::enable_profiling(true)},
{ov::device::properties("CPU", ov::num_streams(4))},
{ov::num_streams(4), ov::device::properties("CPU", ov::num_streams(4))}};
const std::vector<ov::AnyMap> configsAutoLoadNetworkUnsupportedPropertyTest = configsUnsupportedPropertyTest;
INSTANTIATE_TEST_SUITE_P(smoke_Auto_Multi_SetSupportedPropertyTestNoThrow,
SetSupportedPropertyTestP,
::testing::Combine(::testing::Values(CommonTestUtils::DEVICE_AUTO,
CommonTestUtils::DEVICE_MULTI),
::testing::ValuesIn(configsSupportedPropertyTest)),
::testing::PrintToStringParamName());
INSTANTIATE_TEST_SUITE_P(smoke_Auto_Multi_SetUnsupportedPropertyTestNoThrow,
SetUnsupportedPropertyTestP,
::testing::Combine(::testing::Values(CommonTestUtils::DEVICE_AUTO,
CommonTestUtils::DEVICE_MULTI),
::testing::ValuesIn(configsUnsupportedPropertyTest)),
::testing::PrintToStringParamName());
INSTANTIATE_TEST_SUITE_P(smoke_Multi_LoadNetworkSupportedPropertyTestNoThrow,
LoadNetworkWithSupportedPropertyTestP,
::testing::Combine(::testing::Values(CommonTestUtils::DEVICE_MULTI),
::testing::ValuesIn(configsMultiLoadNetworkSupportedPropertyTest)),
::testing::PrintToStringParamName());
INSTANTIATE_TEST_SUITE_P(smoke_Auto_LoadNetworkUnsupportedPropertyTestNoThrow,
LoadNetworkWithUnsupportedPropertyTestP,
::testing::Combine(::testing::Values(CommonTestUtils::DEVICE_AUTO),
::testing::ValuesIn(configsAutoLoadNetworkUnsupportedPropertyTest)),
::testing::PrintToStringParamName());