IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[IE][TOOLS] Improvements for compile_tool (#2555)

Browse Source 
* Split cmd arguments onto sections (common, MYRIAD, FPGA).
* Add `-iol` cmd argument as `-iop` analogue for layouts.
* Remove limitations over `-ip`, `-op`, `-iop` arguments.
  Now they supports full meaningfull set of Precision/Layout values and will allow to user set any precision for any input/output.
  The applicability for this in under user reponsibility and support under plugin reponsibility.
* Add VPUX default configuration.
* Adjust code style to the project common style.

Co-authored-by: Alexander Novak <sasha-novak@yandex.ru>
This commit is contained in:
Vladislav Vinogradov 2020-10-07 14:51:10 +03:00 committed by GitHub
parent 6f0aaf2bb5
commit e9fde8f497
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 293 additions and 197 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

68
inference-engine/tools/compile_tool/README.md
View File

@ -1,15 +1,13 @@
# Compile Tool {#openvino_inference_engine_tools_compile_tool_README}
The Compile tool is a C++ application that enables you to dump a loaded
executable network blob. The tool is delivered as an executable file that can be
run on both Linux\* and Windows\*. The tool is located in the `<INSTALLROOT>/deployment_tools/inference_engine/lib/intel64/` directory on
Linux and `<INSTALL_DIR\deployment_tools\inference_engine\bin\intel64\Release>`
on Windows.
The Compile tool is a C++ application that enables you to dump a loaded executable network blob.
The tool is delivered as an executable file that can be run on both Linux\* and Windows\*.
The tool is located in the `<INSTALLROOT>/deployment_tools/inference_engine/lib/intel64/` directory on Linux
and `<INSTALL_DIR\deployment_tools\inference_engine\bin\intel64\Release>` on Windows.
The workflow of the Compile tool is as follows:
1. Upon the start, the tool application reads command-line parameters and loads a network to the
Inference Engine device.
1. Upon the start, the tool application reads command-line parameters and loads a network to the Inference Engine device.
2. The application exports a blob with the compiled network and writes it to the output file.
## Run the Compile Tool
@ -19,37 +17,53 @@ Running the application with the `-h` option yields the following usage message:
```sh
./compile_tool -h
Inference Engine:
API version ............ <version>
Build .................. <build>
API version ............ 2.1
Build .................. custom_vv/compile-tool_8b57af00330063c7f302aaac4d41805de21fc54a
Description ....... API
compile_tool [OPTIONS]
[OPTIONS]:
Common options:
-h Optional. Print the usage message.
-m <value> Required. Path to the XML model.
-d <value> Required. Target device name.
-d <value> Required. Specify a target device for which executable network will be compiled.
Use "-d HETERO:<comma-separated_devices_list>" format to specify HETERO plugin.
Use "-d MULTI:<comma-separated_devices_list>" format to specify MULTI plugin.
The application looks for a suitable plugin for the specified device.
-o <value> Optional. Path to the output file. Default value: "<model_xml_file>.blob".
-c <value> Optional. Path to the configuration file. Default value: "config".
-ip <value> Optional. Specifies precision for all input layers of the network. Supported values: FP32, FP16, U8. Default value: FP16.
-op <value> Optional. Specifies precision for all output layers of the network. Supported values: FP32, FP16, U8. Default value: FP16.
-c <value> Optional. Path to the configuration file.
-ip <value> Optional. Specifies precision for all input layers of the network.
-op <value> Optional. Specifies precision for all output layers of the network.
-iop "<value>" Optional. Specifies precision for input and output layers by name.
By default, all inputs and outputs have the FP16 precision.
Available precisions: FP32, FP16, U8.
Example: -iop "input:FP16, output:FP16".
Notice that quotes are required.
Overwrites precision from ip and op options for specified layers.
-il <value> Optional. Specifies layout for all input layers of the network.
-ol <value> Optional. Specifies layout for all input layers of the network.
-iol "<value>" Optional. Specifies layout for input and output layers by name.
Example: -iol "input:NCHW, output:NHWC".
Notice that quotes are required.
Overwrites layout from il and ol options for specified layers.
VPU options:
-VPU_NUMBER_OF_SHAVES <value> Optional. Specifies number of shaves. Should be set with "VPU_NUMBER_OF_CMX_SLICES". Overwrites value from config.
-VPU_NUMBER_OF_CMX_SLICES <value> Optional. Specifies number of CMX slices. Should be set with "VPU_NUMBER_OF_SHAVES". Overwrites value from config.
-VPU_TILING_CMX_LIMIT_KB <value> Optional. Specifies CMX limit for data tiling in kB. Value should be equal or greater than -1, where -1 means default value of limit. Overwrites value from config.
MYRIAD-specific options:
-VPU_NUMBER_OF_SHAVES <value> Optional. Specifies number of shaves.
Should be set with "VPU_NUMBER_OF_CMX_SLICES".
Overwrites value from config.
DLA options:
-DLA_ARCH_NAME <value> Optional. Specify architecture name used to compile executable network for FPGA device.
-VPU_NUMBER_OF_CMX_SLICES <value> Optional. Specifies number of CMX slices.
Should be set with "VPU_NUMBER_OF_SHAVES".
Overwrites value from config.
-VPU_TILING_CMX_LIMIT_KB <value> Optional. Specifies CMX limit for data tiling.
Value should be equal or greater than -1.
Overwrites value from config.
FPGA-specific options:
-DLA_ARCH_NAME <value> Optional. Specify architecture name used to compile executable network for FPGA device.
```
Running the application with the empty list of options yields an error message.
To dump a blob using a trained Faster R-CNN network, use the command below:
To dump a blob using a trained network, use the command below:
```sh
./compile_tool -m <path_to_model>/model_name.xml
@ -64,7 +78,7 @@ To do that, specify the architecture name of the DLA bitstream using the paramet
### Export
To save a blob file from your application, call the `InferenceEngine::ExecutableNetwork::Export()`
To save a blob file from your application, call the `InferenceEngine::ExecutableNetwork::Export()`
method:
```cpp
@ -75,7 +89,7 @@ executableNetwork.Export(file);
### Import
To import a blob with the network into your application, call the
To import a blob with the network into your application, call the
`InferenceEngine::Core::ImportNetwork` method:
Example:
@ -86,5 +100,5 @@ std::ifstream file{"model_name.blob"};
InferenceEngine::ExecutableNetwork = ie.ImportNetwork(file, "MYRIAD", {});
```
> **NOTE**: Prior to the import, models must be converted to the Inference Engine format
> **NOTE**: Prior to the import, models must be converted to the Inference Engine format
> (\*.xml + \*.bin) using the [Model Optimizer tool](https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer).

422
inference-engine/tools/compile_tool/main.cpp
View File

@ -20,61 +20,88 @@
#include <vpu/utils/string.hpp>
#include "samples/common.hpp"
static constexpr char help_message[] = "Optional. Print the usage message.";
static constexpr char model_message[] = "Required. Path to the XML model.";
static constexpr char targetDeviceMessage[] = "Required. Specify a target device for which executable network will be compiled."
"Use \"-d HETERO:<comma-separated_devices_list>\" format to specify HETERO plugin. "
"Use \"-d MULTI:<comma-separated_devices_list>\" format to specify MULTI plugin. "
"The application looks for a suitable plugin for the specified device.";
static constexpr char help_message[] =
"Optional. Print the usage message.";
static constexpr char output_message[] = "Optional. Path to the output file. Default value: \"<model_xml_file>.blob\".";
static constexpr char config_message[] = "Optional. Path to the configuration file. Default value: \"config\".";
static constexpr char number_of_shaves_message[] = "Optional. Specifies number of shaves."
" Should be set with \"VPU_NUMBER_OF_CMX_SLICES\"."
" Overwrites value from config.";
static constexpr char number_of_cmx_slices_message[] = "Optional. Specifies number of CMX slices."
" Should be set with \"VPU_NUMBER_OF_SHAVES\"."
" Overwrites value from config.";
static constexpr char tiling_cmx_limit_message[] = "Optional. Specifies CMX limit for data tiling."
" Value should be equal or greater than -1."
" Overwrites value from config.";
static constexpr char inputs_precision_message[] = "Optional. Specifies precision for all input layers of the network."
" Supported values: FP32, FP16, U8. Default value: FP16.";
static constexpr char outputs_precision_message[] = "Optional. Specifies precision for all output layers of the network."
" Supported values: FP32, FP16, U8. Default value: FP16.";
static constexpr char iop_message[] = "Optional. Specifies precision for input and output layers by name.\n"
" By default, all inputs and outputs have the FP16 precision.\n"
" Available precisions: FP32, FP16, U8.\n"
static constexpr char model_message[] =
"Required. Path to the XML model.";
static constexpr char targetDeviceMessage[] =
"Required. Specify a target device for which executable network will be compiled.\n"
" Use \"-d HETERO:<comma-separated_devices_list>\" format to specify HETERO plugin.\n"
" Use \"-d MULTI:<comma-separated_devices_list>\" format to specify MULTI plugin.\n"
" The application looks for a suitable plugin for the specified device.";
static constexpr char output_message[] =
"Optional. Path to the output file. Default value: \"<model_xml_file>.blob\".";
static constexpr char config_message[] =
"Optional. Path to the configuration file.";
static constexpr char inputs_precision_message[] =
"Optional. Specifies precision for all input layers of the network.";
static constexpr char outputs_precision_message[] =
"Optional. Specifies precision for all output layers of the network.";
static constexpr char iop_message[] =
"Optional. Specifies precision for input and output layers by name.\n"
" Example: -iop \"input:FP16, output:FP16\".\n"
" Notice that quotes are required.\n"
" Overwrites precision from ip and op options for specified layers.";
static constexpr char inputs_layout_message[] = "Optional. Specifies layout for all input layers of the network."
" Supported values: NCHW, NHWC, NC, C.";
static constexpr char outputs_layout_message[] = "Optional. Specifies layout for all input layers of the network."
" Supported values: NCHW, NHWC, NC, C.";
static constexpr char inputs_layout_message[] =
"Optional. Specifies layout for all input layers of the network.";
static constexpr char dla_arch_name[] = "Optional. Specify architecture name used to compile executable network for FPGA device.";
static constexpr char outputs_layout_message[] =
"Optional. Specifies layout for all input layers of the network.";
static constexpr char iol_message[] =
"Optional. Specifies layout for input and output layers by name.\n"
" Example: -iol \"input:NCHW, output:NHWC\".\n"
" Notice that quotes are required.\n"
" Overwrites layout from il and ol options for specified layers.";
// MYRIAD-specific
static constexpr char number_of_shaves_message[] =
"Optional. Specifies number of shaves.\n"
" Should be set with \"VPU_NUMBER_OF_CMX_SLICES\".\n"
" Overwrites value from config.\n";
static constexpr char number_of_cmx_slices_message[] =
"Optional. Specifies number of CMX slices.\n"
" Should be set with \"VPU_NUMBER_OF_SHAVES\".\n"
" Overwrites value from config.";
static constexpr char tiling_cmx_limit_message[] =
"Optional. Specifies CMX limit for data tiling.\n"
" Value should be equal or greater than -1.\n"
" Overwrites value from config.";
// FPGA-specific
static constexpr char dla_arch_name[] =
"Optional. Specify architecture name used to compile executable network for FPGA device.";
DEFINE_bool(h, false, help_message);
DEFINE_string(m, "", model_message);
DEFINE_string(d, "", targetDeviceMessage);
DEFINE_string(o, "", output_message);
DEFINE_string(c, "config", config_message);
DEFINE_string(c, "", config_message);
DEFINE_string(ip, "", inputs_precision_message);
DEFINE_string(op, "", outputs_precision_message);
DEFINE_string(iop, "", iop_message);
DEFINE_string(il, "", inputs_layout_message);
DEFINE_string(ol, "", outputs_layout_message);
DEFINE_string(iol, "", iol_message);
DEFINE_string(VPU_NUMBER_OF_SHAVES, "", number_of_shaves_message);
DEFINE_string(VPU_NUMBER_OF_CMX_SLICES, "", number_of_cmx_slices_message);
DEFINE_string(VPU_TILING_CMX_LIMIT_KB, "", tiling_cmx_limit_message);
DEFINE_string(DLA_ARCH_NAME, "", dla_arch_name);
static void showUsage() {
std::cout << std::endl;
std::cout << "compile_tool [OPTIONS]" << std::endl;
std::cout << "[OPTIONS]:" << std::endl;
std::cout << std::endl;
std::cout << " Common options: " << std::endl;
std::cout << " -h " << help_message << std::endl;
std::cout << " -m <value> " << model_message << std::endl;
std::cout << " -d <value> " << targetDeviceMessage << std::endl;
@ -82,20 +109,22 @@ static void showUsage() {
std::cout << " -c <value> " << config_message << std::endl;
std::cout << " -ip <value> " << inputs_precision_message << std::endl;
std::cout << " -op <value> " << outputs_precision_message << std::endl;
std::cout << " -iop \"<value>\" " << iop_message << std::endl;
std::cout << " -iop \"<value>\" " << iop_message << std::endl;
std::cout << " -il <value> " << inputs_layout_message << std::endl;
std::cout << " -ol <value> " << outputs_layout_message << std::endl;
std::cout << " " << std::endl;
std::cout << " VPU options: " << std::endl;
std::cout << " -iol \"<value>\" " << iol_message << std::endl;
std::cout << std::endl;
std::cout << " MYRIAD-specific options: " << std::endl;
std::cout << " -VPU_NUMBER_OF_SHAVES <value> " << number_of_shaves_message << std::endl;
std::cout << " -VPU_NUMBER_OF_CMX_SLICES <value> " << number_of_cmx_slices_message << std::endl;
std::cout << " -VPU_TILING_CMX_LIMIT_KB <value> " << tiling_cmx_limit_message << std::endl;
std::cout << " DLA options: " << std::endl;
std::cout << std::endl;
std::cout << " FPGA-specific options: " << std::endl;
std::cout << " -DLA_ARCH_NAME <value> " << dla_arch_name << std::endl;
std::cout << std::endl;
}
static bool parseCommandLine(int *argc, char ***argv, InferenceEngine::Core& ie) {
static bool parseCommandLine(int* argc, char*** argv, InferenceEngine::Core& ie) {
gflags::ParseCommandLineNonHelpFlags(argc, argv, true);
if (FLAGS_h) {
@ -126,9 +155,10 @@ static bool parseCommandLine(int *argc, char ***argv, InferenceEngine::Core& ie)
return true;
}
static std::map<std::string, std::string> parseConfig(const std::string& configName, char comment = '#') {
static std::map<std::string, std::string> parseConfigFile(char comment = '#') {
std::map<std::string, std::string> config;
std::ifstream file{configName};
std::ifstream file(FLAGS_c);
if (file.is_open()) {
std::string key, value;
while (file >> key >> value) {
@ -141,10 +171,13 @@ static std::map<std::string, std::string> parseConfig(const std::string& configN
return config;
}
static std::map<std::string, std::string> configure(const std::string &configFile, const std::string &xmlFileName) {
auto config = parseConfig(configFile);
static std::map<std::string, std::string> configure() {
const bool isMYRIAD = FLAGS_d.find("MYRIAD") != std::string::npos;
const bool isFPGA = FLAGS_d.find("FPGA") != std::string::npos;
if (std::string::npos != FLAGS_d.find("MYRIAD")) {
auto config = parseConfigFile();
if (isMYRIAD) {
IE_SUPPRESS_DEPRECATED_START
config[VPU_MYRIAD_CONFIG_KEY(PLATFORM)] = "VPU_MYRIAD_2480";
IE_SUPPRESS_DEPRECATED_END
@ -162,7 +195,7 @@ IE_SUPPRESS_DEPRECATED_END
}
}
if (std::string::npos != FLAGS_d.find("FPGA")) {
if (isFPGA) {
if (!FLAGS_DLA_ARCH_NAME.empty()) {
config["DLIA_ARCH_NAME"] = FLAGS_DLA_ARCH_NAME;
}
@ -171,94 +204,68 @@ IE_SUPPRESS_DEPRECATED_END
return config;
}
static std::map<std::string, std::string> parsePrecisions(const std::string &iop) {
std::string user_input = iop;
user_input.erase(std::remove_if(user_input.begin(), user_input.end(), ::isspace), user_input.end());
static std::map<std::string, std::string> parseArgMap(std::string argMap) {
argMap.erase(std::remove_if(argMap.begin(), argMap.end(), ::isspace), argMap.end());
std::vector<std::string> inputs;
vpu::splitStringList(user_input, inputs, ',');
std::vector<std::string> pairs;
vpu::splitStringList(argMap, pairs, ',');
std::map<std::string, std::string> precisions;
for (auto &&input : inputs) {
std::vector<std::string> precision;
vpu::splitStringList(input, precision, ':');
if (precision.size() != 2) {
throw std::invalid_argument("Invalid precision " + input + ". Expected layer_name : precision_value");
std::map<std::string, std::string> parsedMap;
for (auto&& pair : pairs) {
std::vector<std::string> keyValue;
vpu::splitStringList(pair, keyValue, ':');
if (keyValue.size() != 2) {
throw std::invalid_argument("Invalid key/value pair " + pair + ". Expected <layer_name>:<value>");
}
precisions[precision[0]] = precision[1];
parsedMap[keyValue[0]] = keyValue[1];
}
return precisions;
return parsedMap;
}
using supported_precisions_t = std::unordered_map<std::string, InferenceEngine::Precision>;
using supported_layouts_t = std::unordered_map<std::string, InferenceEngine::Layout>;
using matchLayoutToDims_t = std::unordered_map<size_t, size_t>;
static InferenceEngine::Layout getLayout(const std::string &value,
const supported_layouts_t &supported_layouts) {
std::string upper_value = value;
std::transform(value.begin(), value.end(), upper_value.begin(), ::toupper);
auto layout = supported_layouts.find(upper_value);
static InferenceEngine::Layout getLayout(std::string value,
const supported_layouts_t& supported_layouts) {
std::transform(value.begin(), value.end(), value.begin(), ::toupper);
const auto layout = supported_layouts.find(value);
if (layout == supported_layouts.end()) {
throw std::logic_error("\"" + value + "\"" + " is not a valid layout.");
throw std::logic_error("\"" + value + "\"" + " is not a valid layout");
}
return layout->second;
}
static InferenceEngine::Precision getPrecision(const std::string &value,
const supported_precisions_t &supported_precisions,
const std::string& error_report = std::string()) {
std::string upper_value = value;
std::transform(value.begin(), value.end(), upper_value.begin(), ::toupper);
auto precision = supported_precisions.find(upper_value);
if (precision == supported_precisions.end()) {
std::string report = error_report.empty() ? ("") : (" " + error_report);
throw std::logic_error("\"" + value + "\"" + " is not a valid precision" + report);
}
return precision->second;
}
static InferenceEngine::Precision getInputPrecision(const std::string &value) {
static const supported_precisions_t supported_precisions = {
{ "FP32", InferenceEngine::Precision::FP32 },
{ "FP16", InferenceEngine::Precision::FP16 },
{ "U8", InferenceEngine::Precision::U8 }
};
return getPrecision(value, supported_precisions, " for input layer");
}
static InferenceEngine::Precision getOutputPrecision(const std::string &value) {
static const supported_precisions_t supported_precisions = {
{ "FP32", InferenceEngine::Precision::FP32 },
{ "FP16", InferenceEngine::Precision::FP16 }
};
return getPrecision(value, supported_precisions, " for output layer");
}
static InferenceEngine::Layout getLayout(const std::string &value) {
static InferenceEngine::Layout getLayout(const std::string& value) {
static const supported_layouts_t supported_layouts = {
{ "NCDHW", InferenceEngine::Layout::NCDHW },
{ "NDHWC", InferenceEngine::Layout::NDHWC },
{ "NCHW", InferenceEngine::Layout::NCHW },
{ "NHWC", InferenceEngine::Layout::NHWC },
{ "CHW", InferenceEngine::Layout::CHW },
{ "NC", InferenceEngine::Layout::NC },
{ "C", InferenceEngine::Layout::C }
{ "C", InferenceEngine::Layout::C },
};
return getLayout(value, supported_layouts);
}
static bool isMatchLayoutToDims(const InferenceEngine::Layout& layout, const size_t dimension) {
static bool isMatchLayoutToDims(InferenceEngine::Layout layout, size_t dimension) {
static const matchLayoutToDims_t matchLayoutToDims = {
{static_cast<size_t>(InferenceEngine::Layout::NCHW), 4 },
{static_cast<size_t>(InferenceEngine::Layout::NHWC), 4 },
{static_cast<size_t>(InferenceEngine::Layout::CHW), 3 },
{static_cast<size_t>(InferenceEngine::Layout::NC), 2 },
{static_cast<size_t>(InferenceEngine::Layout::C), 1 }};
{static_cast<size_t>(InferenceEngine::Layout::NCDHW), 5 },
{static_cast<size_t>(InferenceEngine::Layout::NDHWC), 5 },
{static_cast<size_t>(InferenceEngine::Layout::NCHW), 4 },
{static_cast<size_t>(InferenceEngine::Layout::NHWC), 4 },
{static_cast<size_t>(InferenceEngine::Layout::CHW), 3 },
{static_cast<size_t>(InferenceEngine::Layout::NC), 2 },
{static_cast<size_t>(InferenceEngine::Layout::C), 1 }
};
auto dims = matchLayoutToDims.find(static_cast<size_t>(layout));
const auto dims = matchLayoutToDims.find(static_cast<size_t>(layout));
if (dims == matchLayoutToDims.end()) {
throw std::logic_error("Layout is not valid.");
}
@ -266,6 +273,37 @@ static bool isMatchLayoutToDims(const InferenceEngine::Layout& layout, const siz
return dimension == dims->second;
}
static InferenceEngine::Precision getPrecision(std::string value,
const supported_precisions_t& supported_precisions) {
std::transform(value.begin(), value.end(), value.begin(), ::toupper);
const auto precision = supported_precisions.find(value);
if (precision == supported_precisions.end()) {
throw std::logic_error("\"" + value + "\"" + " is not a valid precision");
}
return precision->second;
}
static InferenceEngine::Precision getPrecision(const std::string& value) {
static const supported_precisions_t supported_precisions = {
{ "FP32", InferenceEngine::Precision::FP32 },
{ "FP16", InferenceEngine::Precision::FP16 },
{ "BF16", InferenceEngine::Precision::BF16 },
{ "U64", InferenceEngine::Precision::U64 },
{ "I64", InferenceEngine::Precision::I64 },
{ "U32", InferenceEngine::Precision::U32 },
{ "I32", InferenceEngine::Precision::I32 },
{ "U16", InferenceEngine::Precision::U16 },
{ "I16", InferenceEngine::Precision::I16 },
{ "U8", InferenceEngine::Precision::U8 },
{ "I8", InferenceEngine::Precision::I8 },
{ "BOOL", InferenceEngine::Precision::BOOL },
};
return getPrecision(value, supported_precisions);
}
bool isFP16(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::FP16;
}
@ -274,118 +312,149 @@ bool isFP32(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::FP32;
}
bool isU8(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::U8;
}
bool isFloat(InferenceEngine::Precision precision) {
return isFP16(precision) || isFP32(precision);
}
static void setPrecisions(const InferenceEngine::CNNNetwork &network, const std::string &iop) {
auto user_precisions_map = parsePrecisions(iop);
static void setPrecisions(const InferenceEngine::CNNNetwork& network) {
const auto user_precisions_map = parseArgMap(FLAGS_iop);
auto inputs = network.getInputsInfo();
auto outputs = network.getOutputsInfo();
for (auto &&item : user_precisions_map) {
std::string layer_name = item.first;
std::string user_precision = item.second;
for (auto&& item : user_precisions_map) {
const auto& layer_name = item.first;
const auto& user_precision = item.second;
auto input = inputs.find(layer_name);
auto output = outputs.find(layer_name);
const auto input = inputs.find(layer_name);
const auto output = outputs.find(layer_name);
if (input != inputs.end()) {
const auto input_precision = input->second->getPrecision();
if ((isFloat(input_precision) && isFloat(getInputPrecision(user_precision))) ||
(isFloat(input_precision) && isU8(getInputPrecision(user_precision)))) {
input->second->setPrecision(getInputPrecision(user_precision));
}
input->second->setPrecision(getPrecision(user_precision));
} else if (output != outputs.end()) {
const auto output_precision = output->second->getPrecision();
if (isFloat(output_precision) && isFloat(getOutputPrecision(user_precision))) {
output->second->setPrecision(getOutputPrecision(user_precision));
}
output->second->setPrecision(getPrecision(user_precision));
} else {
throw std::logic_error(layer_name + " is not an input neither output");
}
}
}
static void setDefaultIOPrecisions(InferenceEngine::CNNNetwork &network) {
if (std::string::npos != FLAGS_d.find("MYRIAD")) {
static void setDefaultIO(InferenceEngine::CNNNetwork& network) {
const bool isMYRIAD = FLAGS_d.find("MYRIAD") != std::string::npos;
const bool isVPUX = FLAGS_d.find("VPUX") != std::string::npos;
if (isMYRIAD) {
const InferenceEngine::Precision fp16 = InferenceEngine::Precision::FP16;
for (auto &&layer : network.getInputsInfo()) {
if (isFP32(layer.second->getPrecision())) {
for (auto&& layer : network.getInputsInfo()) {
if (isFloat(layer.second->getPrecision())) {
layer.second->setPrecision(fp16);
}
}
for (auto &&layer : network.getOutputsInfo()) {
if (isFP32(layer.second->getPrecision())) {
for (auto&& layer : network.getOutputsInfo()) {
if (isFloat(layer.second->getPrecision())) {
layer.second->setPrecision(fp16);
}
}
}
}
static void processPrecisions(InferenceEngine::CNNNetwork &network,
const std::string &inputs_precision, const std::string &outputs_precision,
const std::string &iop) {
if (!inputs_precision.empty()) {
auto precision = getInputPrecision(inputs_precision);
for (auto &&layer : network.getInputsInfo()) {
const auto layerPrecision = layer.second->getPrecision();
if ((isFloat(layerPrecision) && isFloat(precision)) ||
(isFloat(layerPrecision) && isU8(precision))) {
layer.second->setPrecision(precision);
}
if (isVPUX) {
const InferenceEngine::Precision u8 = InferenceEngine::Precision::U8;
const InferenceEngine::Precision fp32 = InferenceEngine::Precision::FP32;
for (auto&& layer : network.getInputsInfo()) {
layer.second->setPrecision(u8);
}
}
if (!outputs_precision.empty()) {
auto precision = getOutputPrecision(outputs_precision);
for (auto &&layer : network.getOutputsInfo()) {
const auto layerPrecision = layer.second->getPrecision();
if (isFloat(layerPrecision) && isFloat(precision)) {
layer.second->setPrecision(precision);
}
for (auto&& layer : network.getOutputsInfo()) {
layer.second->setPrecision(fp32);
}
}
if (!iop.empty()) {
setPrecisions(network, iop);
}
}
static void processLayout(InferenceEngine::CNNNetwork &network,
const std::string &inputs_layout, const std::string &outputs_layout) {
if (!inputs_layout.empty()) {
auto layout = getLayout(inputs_layout);
for (auto &&layer : network.getInputsInfo()) {
static void processPrecisions(InferenceEngine::CNNNetwork& network) {
if (!FLAGS_ip.empty()) {
const auto user_precision = getPrecision(FLAGS_ip);
for (auto&& layer : network.getInputsInfo()) {
layer.second->setPrecision(user_precision);
}
}
if (!FLAGS_op.empty()) {
auto user_precision = getPrecision(FLAGS_op);
for (auto&& layer : network.getOutputsInfo()) {
layer.second->setPrecision(user_precision);
}
}
if (!FLAGS_iop.empty()) {
setPrecisions(network);
}
}
static void setLayouts(const InferenceEngine::CNNNetwork& network) {
const auto user_layouts_map = parseArgMap(FLAGS_iol);
auto inputs = network.getInputsInfo();
auto outputs = network.getOutputsInfo();
for (auto&& item : user_layouts_map) {
const auto& layer_name = item.first;
const auto& user_layout = getLayout(item.second);
const auto input = inputs.find(layer_name);
const auto output = outputs.find(layer_name);
if (input != inputs.end()) {
if (!isMatchLayoutToDims(user_layout, input->second->getTensorDesc().getDims().size())) {
throw std::logic_error(item.second + " layout is not applicable to " + layer_name);
}
input->second->setLayout(user_layout);
} else if (output != outputs.end()) {
if (!isMatchLayoutToDims(user_layout, output->second->getTensorDesc().getDims().size())) {
throw std::logic_error(item.second + " layout is not applicable to " + layer_name);
}
output->second->setLayout(user_layout);
} else {
throw std::logic_error(layer_name + " is not an input neither output");
}
}
}
static void processLayout(InferenceEngine::CNNNetwork& network) {
if (!FLAGS_il.empty()) {
const auto layout = getLayout(FLAGS_il);
for (auto&& layer : network.getInputsInfo()) {
if (isMatchLayoutToDims(layout, layer.second->getTensorDesc().getDims().size())) {
layer.second->setLayout(layout);
}
}
}
if (!outputs_layout.empty()) {
auto layout = getLayout(outputs_layout);
for (auto &&layer : network.getOutputsInfo()) {
if (!FLAGS_ol.empty()) {
const auto layout = getLayout(FLAGS_ol);
for (auto&& layer : network.getOutputsInfo()) {
if (isMatchLayoutToDims(layout, layer.second->getTensorDesc().getDims().size())) {
layer.second->setLayout(layout);
}
}
}
if (!FLAGS_iol.empty()) {
setLayouts(network);
}
}
std::string getFileNameFromPath(const std::string& path,
#if defined(_WIN32)
const std::string sep = "\\") {
const std::string& sep = "\\") {
#else
const std::string sep = "/") {
const std::string& sep = "/") {
#endif
auto pos = path.rfind(sep);
const auto pos = path.rfind(sep);
if (std::string::npos == pos) {
return path;
} else {
@ -395,10 +464,12 @@ std::string getFileNameFromPath(const std::string& path,
using TimeDiff = std::chrono::milliseconds;
int main(int argc, char *argv[]) {
int main(int argc, char* argv[]) {
TimeDiff loadNetworkTimeElapsed {0};
try {
std::cout << "Inference Engine: " << InferenceEngine::GetInferenceEngineVersion() << std::endl;
std::cout << std::endl;
InferenceEngine::Core ie;
@ -408,26 +479,37 @@ int main(int argc, char *argv[]) {
auto network = ie.ReadNetwork(FLAGS_m);
setDefaultIOPrecisions(network);
processPrecisions(network, FLAGS_ip, FLAGS_op, FLAGS_iop);
processLayout(network, FLAGS_il, FLAGS_ol);
setDefaultIO(network);
processPrecisions(network);
processLayout(network);
std::cout << "Network inputs:" << std::endl;
for (auto&& layer : network.getInputsInfo()) {
std::cout << " " << layer.first << " : " << layer.second->getPrecision() << " / " << layer.second->getLayout() << std::endl;
}
std::cout << "Network outputs:" << std::endl;
for (auto&& layer : network.getOutputsInfo()) {
std::cout << " " << layer.first << " : " << layer.second->getPrecision() << " / " << layer.second->getLayout() << std::endl;
}
std::cout << std::endl;
auto timeBeforeLoadNetwork = std::chrono::steady_clock::now();
auto executableNetwork = ie.LoadNetwork(network, FLAGS_d, configure(FLAGS_c, FLAGS_m));
auto executableNetwork = ie.LoadNetwork(network, FLAGS_d, configure());
loadNetworkTimeElapsed = std::chrono::duration_cast<TimeDiff>(std::chrono::steady_clock::now() - timeBeforeLoadNetwork);
std::string outputName = FLAGS_o;
if (outputName.empty()) {
outputName = getFileNameFromPath(fileNameNoExt(FLAGS_m)) + ".blob";
}
std::ofstream outputFile{outputName};
if (!outputFile) {
if (!outputFile.is_open()) {
std::cout << "Output file " << outputName << " can't be opened for writing" << std::endl;
return EXIT_FAILURE;
} else {
executableNetwork.Export(outputFile);
}
} catch (const std::exception &error) {
} catch (const std::exception& error) {
std::cerr << error.what() << std::endl;
return EXIT_FAILURE;
} catch (...) {