Remove myriad plugin (#15131)
* Removed Intel MYRIAD plugin * Removed Intel MYIAD from CI files * Removed Intel MYRIAD from cmake folder * Removed MYRIAD, HDDL from samples * Removed MYRIAD, HDDL from scripts folder * Removed MYRIAD from bindings folder (C and Python API) * Removed MYRIAD tests * Removed MYRIAD from tests folder * Removed MYRIAD from tools folder * Removed HDDL (VAD), MYRIAD (NSC2) from documentation * Fixed build for AUTO unit tests * Fixed clang code style * Fixed comments and issues * removed MYRIAD from AUTO tests * Disabled MULTI tests in CI * Update docs/OV_Runtime_UG/auto_device_selection.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/get_started/get_started_demos.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/local-distribution.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
This commit is contained in:
Ilya Lavrenov
GitHub
@@ -25,7 +25,7 @@ int main(int argc, char* argv[]) {
|
||||
return EXIT_FAILURE;
|
||||
}
|
||||
// Optimize for latency. Most of the devices are configured for latency by default,
|
||||
// but there are exceptions like MYRIAD
|
||||
// but there are exceptions like GNA
|
||||
ov::AnyMap latency{{ov::hint::performance_mode.name(), ov::hint::PerformanceMode::LATENCY}};
|
||||
|
||||
// Create ov::Core and use it to compile a model.
|
||||
|
||||
@@ -55,7 +55,7 @@ For more information on performance hints, see the [High-level Performance Hints
|
||||
|
||||
|
||||
### Device
|
||||
To set which device benchmarking runs on, use the `-d <device>` argument. This will tell benchmark_app to run benchmarking on that specific device. The benchmark app supports "CPU", "GPU", and "MYRIAD" (also known as [VPU](../../../docs/OV_Runtime_UG/supported_plugins/VPU.md)) devices. In order to use the GPU or VPU, the system must have the appropriate drivers installed. If no device is specified, benchmark_app will default to using CPU.
|
||||
To set which device benchmarking runs on, use the `-d <device>` argument. This will tell benchmark_app to run benchmarking on that specific device. The benchmark app supports "CPU", "GPU", and "GNA" devices. In order to use the GPU or GNA, the system must have the appropriate drivers installed. If no device is specified, benchmark_app will default to using CPU.
|
||||
|
||||
For example, to run benchmarking on GPU, use:
|
||||
|
||||
@@ -140,7 +140,7 @@ Options:
|
||||
-latency_percentile Optional. Defines the percentile to be reported in latency metric. The valid range is [1, 100]. The default value is 50 (median).
|
||||
|
||||
device-specific performance options:
|
||||
-nstreams <integer> Optional. Number of streams to use for inference on the CPU, GPU or MYRIAD devices (for HETERO and MULTI device cases use format <dev1>:<nstreams1>,<dev2>:<nstreams2> or just <nstreams>). Default value is determined automatically for a device.Please note that although the automatic selection usually provides a reasonable performance, it still may be non - optimal for some cases, especially for very small models. See sample's README for more details. Also, using nstreams>1 is inherently throughput-oriented option, while for the best-latency estimations the number of streams should be set to 1.
|
||||
-nstreams <integer> Optional. Number of streams to use for inference on the CPU or GPU devices (for HETERO and MULTI device cases use format <dev1>:<nstreams1>,<dev2>:<nstreams2> or just <nstreams>). Default value is determined automatically for a device.Please note that although the automatic selection usually provides a reasonable performance, it still may be non - optimal for some cases, especially for very small models. See sample's README for more details. Also, using nstreams>1 is inherently throughput-oriented option, while for the best-latency estimations the number of streams should be set to 1.
|
||||
-nthreads <integer> Optional. Number of threads to use for inference on the CPU (including HETERO and MULTI cases).
|
||||
-pin <string> ("YES"|"CORE") / "HYBRID_AWARE" / ("NO"|"NONE") / "NUMA" Optional. Explicit inference threads binding options (leave empty to let the OpenVINO make a choice):
|
||||
enabling threads->cores pinning("YES", which is already default for any conventional CPU),
|
||||
|
||||
@@ -74,7 +74,7 @@ static const char infer_num_threads_message[] = "Optional. Number of threads to
|
||||
|
||||
/// @brief message for #streams for CPU inference
|
||||
static const char infer_num_streams_message[] =
|
||||
"Optional. Number of streams to use for inference on the CPU, GPU or MYRIAD devices "
|
||||
"Optional. Number of streams to use for inference on the CPU or GPU devices "
|
||||
"(for HETERO and MULTI device cases use format <dev1>:<nstreams1>,<dev2>:<nstreams2> or just "
|
||||
"<nstreams>). "
|
||||
"Default value is determined automatically for a device.Please note that although the "
|
||||
|
||||
@@ -470,26 +470,24 @@ int main(int argc, char* argv[]) {
|
||||
"but it still may be non-optimal for some cases, for more "
|
||||
"information look at README."
|
||||
<< slog::endl;
|
||||
if (device.find("MYRIAD") == std::string::npos) { // MYRIAD sets the default number of
|
||||
// streams implicitly (without _AUTO)
|
||||
if (supported(key)) {
|
||||
device_config[key] = std::string(getDeviceTypeFromName(device) + "_THROUGHPUT_AUTO");
|
||||
} else if (supported(ov::num_streams.name())) {
|
||||
// Use API 2.0 key for streams
|
||||
key = ov::num_streams.name();
|
||||
device_config[key] = ov::streams::AUTO;
|
||||
} else if (device == "MULTI" || device == "AUTO") {
|
||||
// Set nstreams to default value auto if no nstreams specified from cmd line.
|
||||
for (auto& hwdevice : hardware_devices) {
|
||||
std::string key = std::string(getDeviceTypeFromName(hwdevice) + "_THROUGHPUT_STREAMS");
|
||||
auto value = std::string(getDeviceTypeFromName(hwdevice) + "_THROUGHPUT_AUTO");
|
||||
setDeviceProperty(core,
|
||||
hwdevice,
|
||||
device_config,
|
||||
ov::num_streams(ov::streams::AUTO),
|
||||
is_dev_set_property,
|
||||
std::make_pair(key, value));
|
||||
}
|
||||
|
||||
if (supported(key)) {
|
||||
device_config[key] = std::string(getDeviceTypeFromName(device) + "_THROUGHPUT_AUTO");
|
||||
} else if (supported(ov::num_streams.name())) {
|
||||
// Use API 2.0 key for streams
|
||||
key = ov::num_streams.name();
|
||||
device_config[key] = ov::streams::AUTO;
|
||||
} else if (device == "MULTI" || device == "AUTO") {
|
||||
// Set nstreams to default value auto if no nstreams specified from cmd line.
|
||||
for (auto& hwdevice : hardware_devices) {
|
||||
std::string key = std::string(getDeviceTypeFromName(hwdevice) + "_THROUGHPUT_STREAMS");
|
||||
auto value = std::string(getDeviceTypeFromName(hwdevice) + "_THROUGHPUT_AUTO");
|
||||
setDeviceProperty(core,
|
||||
hwdevice,
|
||||
device_config,
|
||||
ov::num_streams(ov::streams::AUTO),
|
||||
is_dev_set_property,
|
||||
std::make_pair(key, value));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -578,9 +576,6 @@ int main(int argc, char* argv[]) {
|
||||
// for CPU and GPU execution, more throughput-oriented execution via streams
|
||||
setThroughputStreams();
|
||||
set_infer_precision();
|
||||
} else if (device.find("MYRIAD") != std::string::npos) {
|
||||
device_config.emplace(ov::log::level(ov::log::Level::WARNING));
|
||||
setThroughputStreams();
|
||||
} else if (device.find("GNA") != std::string::npos) {
|
||||
set_infer_precision();
|
||||
} else if (device.find("AUTO") != std::string::npos) {
|
||||
|
||||
@@ -61,9 +61,6 @@ size_t InputInfo::depth() const {
|
||||
uint32_t device_default_device_duration_in_seconds(const std::string& device) {
|
||||
static const std::map<std::string, uint32_t> deviceDefaultDurationInSeconds{{"CPU", 60},
|
||||
{"GPU", 60},
|
||||
{"VPU", 60},
|
||||
{"MYRIAD", 60},
|
||||
{"HDDL", 60},
|
||||
{"UNKNOWN", 120}};
|
||||
uint32_t duration = 0;
|
||||
for (const auto& deviceDurationInSeconds : deviceDefaultDurationInSeconds) {
|
||||
|
||||
@@ -58,7 +58,6 @@ Several execution modes are supported via the `-d` flag:
|
||||
|
||||
- `CPU` - All calculation are performed on CPU device using CPU Plugin.
|
||||
- `GPU` - All calculation are performed on GPU device using GPU Plugin.
|
||||
- `MYRIAD` - All calculation are performed on Intel® Neural Compute Stick 2 device using VPU MYRIAD Plugin.
|
||||
- `VPUX` - All calculation are performed on VPUX device using VPUX Plugin.
|
||||
- `GNA_AUTO` - GNA hardware is used if available and the driver is installed. Otherwise, the GNA device is emulated in fast-but-not-bit-exact mode.
|
||||
- `GNA_HW` - GNA hardware is used if available and the driver is installed. Otherwise, an error will occur.
|
||||
@@ -103,7 +102,7 @@ Options:
|
||||
-i "<path>" Required. Paths to input file or Layers names with corresponding paths to the input files. Example of usage for single file: <file.ark> or <file.npz>. Example of usage for named layers: <layer1>=<file1.ark>,<layer2>=<file2.ark>.
|
||||
-m "<path>" Required. Path to an .xml file with a trained model (required if -rg is missing).
|
||||
-o "<path>" Optional. Output file name to save scores or Layer names with corresponding files names to save scores. Example of usage for single file: <output.ark> or <output.npz>. Example of usage for named layers: Example of usage for named layers: <layer1:port_num>=<output_file1.ark>,<layer2:port_num>=<output_file2.ark>.
|
||||
-d "<device>" Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, VPUX, GNA_AUTO, GNA_HW, GNA_HW_WITH_SW_FBACK, GNA_SW_FP32, GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g. HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified.
|
||||
-d "<device>" Optional. Specify a target device to infer on. CPU, GPU, VPUX, GNA_AUTO, GNA_HW, GNA_HW_WITH_SW_FBACK, GNA_SW_FP32, GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g. HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified.
|
||||
-pc Optional. Enables per-layer performance report.
|
||||
-q "<mode>" Optional. Input quantization mode: static (default), dynamic, or user (use with -sf).
|
||||
-qb "<integer>" Optional. Weight bits for quantization: 8 or 16 (default)
|
||||
|
||||
@@ -23,7 +23,7 @@ static const char model_message[] = "Required. Path to an .xml file with a train
|
||||
|
||||
/// @brief message for assigning calculation to device
|
||||
static const char target_device_message[] =
|
||||
"Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, VPUX, GNA_AUTO, GNA_HW, "
|
||||
"Optional. Specify a target device to infer on. CPU, GPU, VPUX, GNA_AUTO, GNA_HW, "
|
||||
"GNA_HW_WITH_SW_FBACK, GNA_SW_FP32, "
|
||||
"GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU"
|
||||
" as a secondary (e.g. HETERO:GNA,CPU) are supported. "
|
||||
@@ -272,7 +272,6 @@ bool parse_and_check_command_line(int argc, char* argv[]) {
|
||||
"HETERO:GNA_HW,CPU",
|
||||
"HETERO:GNA_SW_EXACT,CPU",
|
||||
"HETERO:GNA_SW_FP32,CPU",
|
||||
"MYRIAD",
|
||||
"VPUX"};
|
||||
|
||||
if (std::find(supportedDevices.begin(), supportedDevices.end(), FLAGS_d) == supportedDevices.end()) {
|
||||
|
||||
Reference in New Issue
Block a user