IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[VPU] Removes MYRIAD specific tools (#6908)

Browse Source
This commit is contained in:
Polina Brzezinskaya
2021-08-09 20:35:03 +03:00
committed by GitHub
parent eadeae6c47
commit 0c8a5d5279
12 changed files with 1 additions and 1574 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
inference-engine/tools/CMakeLists.txt
View File

@@ -2,5 +2,4 @@
# SPDX-License-Identifier: Apache-2.0
#
add_subdirectory(vpu)
add_subdirectory(compile_tool)

8
inference-engine/tools/vpu/CMakeLists.txt
View File

@@ -1,8 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_MYRIAD)
add_subdirectory(vpu_compile)
add_subdirectory(vpu_perfcheck)
endif()

39
inference-engine/tools/vpu/vpu_compile/CMakeLists.txt
View File

@@ -1,39 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(TARGET_NAME myriad_compile)
file(GLOB SRCS
${CMAKE_CURRENT_SOURCE_DIR}/*.cpp
)
add_executable(${TARGET_NAME} ${SRCS})
if (CMAKE_COMPILER_IS_GNUCXX)
target_compile_options(${TARGET_NAME} PRIVATE -Wall)
endif()
target_link_libraries(${TARGET_NAME} PRIVATE
inference_engine
vpu_graph_transformer
gflags
ie_samples_utils
)
add_dependencies(${TARGET_NAME} myriadPlugin)
set_target_properties(${TARGET_NAME} PROPERTIES
COMPILE_PDB_NAME ${TARGET_NAME}
FOLDER tools
)
add_cpplint_target(${TARGET_NAME}_cpplint FOR_TARGETS ${TARGET_NAME})
# install
ie_cpack_add_component(myriad_dev DEPENDS myriad)
install(TARGETS ${TARGET_NAME}
RUNTIME DESTINATION ${IE_CPACK_RUNTIME_PATH}
COMPONENT myriad_dev)

82
inference-engine/tools/vpu/vpu_compile/README.md
View File

@@ -1,82 +0,0 @@
# myriad_compile tool {#openvino_inference_engine_tools_vpu_vpu_compile_README}
This topic demonstrates how to run the `myriad_compile` tool application, which intended to dump blob for `vpu` plugins of Inference Engine by configuration options.
## How It Works
Upon the start-up, the tool application reads command line parameters and loads a network to the Inference Engine plugin.
Then application exports blob and writes it to the output file.
## Running
Running the application with the <code>-h</code> option yields the following usage message:
```sh
./myriad_compile -h
Inference Engine:
API version ............ <version>
Build .................. <build>
myriad_compile [OPTIONS]
[OPTIONS]:
-h Optional. Print a usage message.
-m <value> Required. Path to xml model.
-pp <value> Optional. Path to a plugin folder.
-o <value> Optional. Path to the output file. Default value: "<model_xml_file>.blob".
-c <value> Optional. Path to the configuration file. Default value: "config".
-ip <value> Optional. Specifies precision for all input layers of network. Supported values: FP32, FP16, U8. Default value: FP16.
-op <value> Optional. Specifies precision for all output layers of network. Supported values: FP32, FP16, U8. Default value: FP16.
-iop "<value>" Optional. Specifies precision for input/output layers by name.
By default all inputs and outputs have FP16 precision.
Available precisions: FP32, FP16, U8.
Example: -iop "input:FP16, output:FP16".
Notice that quotes are required.
Overwrites precision from ip and op options for specified layers.
-VPU_NUMBER_OF_SHAVES <value> Optional. Specifies number of shaves. Should be set with "VPU_NUMBER_OF_CMX_SLICES". Overwrites value from config.
-VPU_NUMBER_OF_CMX_SLICES <value> Optional. Specifies number of CMX slices. Should be set with "VPU_NUMBER_OF_SHAVES". Overwrites value from config.
-VPU_TILING_CMX_LIMIT_KB <value> Optional. Specifies CMX limit for data tiling in kB. Value should be equal or greater than -1, where -1 means default value of limit. Overwrites value from config.
```
Running the application with the empty list of options yields an error message.
You can use the following command to dump blob using a trained Faster R-CNN network:
```sh
./myriad_compile -m <path_to_model>/model_name.xml
```
## Import and Export functionality
#### Export
You can save a blob file from your application.
To do this, you should call the `Export()` method on the `ExecutableNetwork` object.
`Export()` has the following argument:
* Name of output blob [IN]
Example:
```sh
InferenceEngine::Core core;
InferenceEngine::ExecutableNetwork executableNetwork = core.LoadNetwork(network);
executableNetwork.Export("model_name.blob");
```
#### Import
You can upload blob with network into your application.
To do this, you should call the `ImportNetwork()` method on the `Core` object.
`ImportNetwork()` has the following arguments:
* Path to blob [IN]
* Config options [IN]
And returns `ExecutableNetwork` object
Example:
```sh
std::string modelFilename ("model_name.blob");
InferenceEngine::Core core;
InferenceEngine::ExecutableNetwork importedNetwork = core.ImportNetwork(modelFilename);
```
> **NOTE**: Models should be first converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer).

271
inference-engine/tools/vpu/vpu_compile/main.cpp
View File

@@ -1,271 +0,0 @@
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <cstdlib>
#include <iostream>
#include <fstream>
#include <algorithm>
#include <unordered_map>
#include <map>
#include <vector>
#include <string>
#include <gflags/gflags.h>
#include "inference_engine.hpp"
#include <vpu/private_plugin_config.hpp>
#include "samples/common.hpp"
#include <vpu/utils/string.hpp>
#include "vpu_tools_common.hpp"
static constexpr char help_message[] = "Optional. Print a usage message.";
static constexpr char model_message[] = "Required. Path to xml model.";
static constexpr char plugin_path_message[] = "Optional. Path to a plugin folder.";
static constexpr char output_message[] = "Optional. Path to the output file. Default value: \"<model_xml_file>.blob\".";
static constexpr char config_message[] = "Optional. Path to the configuration file. Default value: \"config\".";
static constexpr char number_of_shaves_message[] = "Optional. Specifies number of shaves."
" Should be set with \"VPU_NUMBER_OF_CMX_SLICES\"."
" Overwrites value from config.";
static constexpr char number_of_cmx_slices_message[] = "Optional. Specifies number of CMX slices."
" Should be set with \"VPU_NUMBER_OF_SHAVES\"."
" Overwrites value from config.";
static constexpr char tiling_cmx_limit_message[] = "Optional. Specifies CMX limit for data tiling."
" Value should be equal or greater than -1."
" Overwrites value from config.";
static constexpr char inputs_precision_message[] = "Optional. Specifies precision for all input layers of network."
" Supported values: FP32, FP16, U8. Default value: FP16.";
static constexpr char outputs_precision_message[] = "Optional. Specifies precision for all output layers of network."
" Supported values: FP32, FP16, U8. Default value: FP16.";
static constexpr char iop_message[] = "Optional. Specifies precision for input/output layers by name.\n"
" By default all inputs and outputs have FP16 precision.\n"
" Available precisions: FP32, FP16, U8.\n"
" Example: -iop \"input:FP16, output:FP16\".\n"
" Notice that quotes are required.\n"
" Overwrites precision from ip and op options for specified layers.";
DEFINE_bool(h, false, help_message);
DEFINE_string(m, "", model_message);
DEFINE_string(pp, "", plugin_path_message);
DEFINE_string(o, "", output_message);
DEFINE_string(c, "config", config_message);
DEFINE_string(ip, "", inputs_precision_message);
DEFINE_string(op, "", outputs_precision_message);
DEFINE_string(iop, "", iop_message);
DEFINE_string(VPU_NUMBER_OF_SHAVES, "", number_of_shaves_message);
DEFINE_string(VPU_NUMBER_OF_CMX_SLICES, "", number_of_cmx_slices_message);
DEFINE_string(VPU_TILING_CMX_LIMIT_KB, "", tiling_cmx_limit_message);
static void showUsage() {
std::cout << std::endl;
std::cout << "myriad_compile [OPTIONS]" << std::endl;
std::cout << "[OPTIONS]:" << std::endl;
std::cout << " -h " << help_message << std::endl;
std::cout << " -m <value> " << model_message << std::endl;
std::cout << " -pp <value> " << plugin_path_message << std::endl;
std::cout << " -o <value> " << output_message << std::endl;
std::cout << " -c <value> " << config_message << std::endl;
std::cout << " -ip <value> " << inputs_precision_message << std::endl;
std::cout << " -op <value> " << outputs_precision_message << std::endl;
std::cout << " -iop \"<value>\" " << iop_message << std::endl;
std::cout << " -VPU_NUMBER_OF_SHAVES <value> " << number_of_shaves_message << std::endl;
std::cout << " -VPU_NUMBER_OF_CMX_SLICES <value> " << number_of_cmx_slices_message << std::endl;
std::cout << " -VPU_TILING_CMX_LIMIT_KB <value> " << tiling_cmx_limit_message << std::endl;
std::cout << std::endl;
}
static bool parseCommandLine(int *argc, char ***argv) {
gflags::ParseCommandLineNonHelpFlags(argc, argv, true);
if (FLAGS_h) {
showUsage();
return false;
}
if (FLAGS_m.empty()) {
throw std::invalid_argument("Path to model xml file is required");
}
if (1 < *argc) {
std::stringstream message;
message << "Unknown arguments: ";
for (auto arg = 1; arg < *argc; arg++) {
message << argv[arg];
if (arg < *argc) {
message << " ";
}
}
throw std::invalid_argument(message.str());
}
return true;
}
static std::map<std::string, std::string> configure(const std::string &configFile, const std::string &xmlFileName) {
auto config = parseConfig(configFile);
IE_SUPPRESS_DEPRECATED_START
config[VPU_MYRIAD_CONFIG_KEY(PLATFORM)] = "VPU_MYRIAD_2480";
IE_SUPPRESS_DEPRECATED_END
if (!FLAGS_VPU_NUMBER_OF_SHAVES.empty()) {
config[InferenceEngine::MYRIAD_NUMBER_OF_SHAVES] = FLAGS_VPU_NUMBER_OF_SHAVES;
}
if (!FLAGS_VPU_NUMBER_OF_CMX_SLICES.empty()) {
config[InferenceEngine::MYRIAD_NUMBER_OF_CMX_SLICES] = FLAGS_VPU_NUMBER_OF_CMX_SLICES;
}
if (!FLAGS_VPU_TILING_CMX_LIMIT_KB.empty()) {
config[InferenceEngine::MYRIAD_TILING_CMX_LIMIT_KB] = FLAGS_VPU_TILING_CMX_LIMIT_KB;
}
return config;
}
static std::map<std::string, std::string> parsePrecisions(const std::string &iop) {
std::string user_input = iop;
user_input.erase(std::remove_if(user_input.begin(), user_input.end(), ::isspace), user_input.end());
std::vector<std::string> inputs;
vpu::splitStringList(user_input, inputs, ',');
std::map<std::string, std::string> precisions;
for (auto &&input : inputs) {
std::vector<std::string> precision;
vpu::splitStringList(input, precision, ':');
if (precision.size() != 2) {
throw std::invalid_argument("Invalid precision " + input + ". Expected layer_name : precision_value");
}
precisions[precision[0]] = precision[1];
}
return precisions;
}
using supported_precisions_t = std::unordered_map<std::string, InferenceEngine::Precision>;
static InferenceEngine::Precision getPrecision(const std::string &value,
const supported_precisions_t &supported_precisions,
const std::string& error_report = std::string()) {
std::string upper_value = value;
std::transform(value.begin(), value.end(), upper_value.begin(), ::toupper);
auto precision = supported_precisions.find(upper_value);
if (precision == supported_precisions.end()) {
std::string report = error_report.empty() ? ("") : (" " + error_report);
throw std::logic_error("\"" + value + "\"" + " is not a valid precision" + report);
}
return precision->second;
}
static InferenceEngine::Precision getInputPrecision(const std::string &value) {
static const supported_precisions_t supported_precisions = {
{ "FP32", InferenceEngine::Precision::FP32 },
{ "FP16", InferenceEngine::Precision::FP16 },
{ "U8", InferenceEngine::Precision::U8 }
};
return getPrecision(value, supported_precisions, "for input layer");
}
static InferenceEngine::Precision getOutputPrecision(const std::string &value) {
static const supported_precisions_t supported_precisions = {
{ "FP32", InferenceEngine::Precision::FP32 },
{ "FP16", InferenceEngine::Precision::FP16 }
};
return getPrecision(value, supported_precisions, "for output layer");
}
void setPrecisions(const InferenceEngine::CNNNetwork &network, const std::string &iop) {
auto user_precisions_map = parsePrecisions(iop);
auto inputs = network.getInputsInfo();
auto outputs = network.getOutputsInfo();
for (auto &&item : user_precisions_map) {
std::string layer_name = item.first;
std::string user_precision = item.second;
auto input = inputs.find(layer_name);
auto output = outputs.find(layer_name);
if (input != inputs.end()) {
const auto input_precision = input->second->getPrecision();
if ((isFloat(input_precision) && isFloat(getInputPrecision(user_precision))) ||
(isFP16(input_precision) && isU8(getInputPrecision(user_precision)))) {
input->second->setPrecision(getInputPrecision(user_precision));
}
} else if (output != outputs.end()) {
auto output_precision = output->second->getPrecision();
if (isFloat(output_precision) && isFloat(getOutputPrecision(user_precision))) {
output->second->setPrecision(getOutputPrecision(user_precision));
}
} else {
throw std::logic_error(layer_name + " is not an input neither output");
}
}
}
static void processPrecisions(InferenceEngine::CNNNetwork &network,
const std::string &inputs_precision, const std::string &outputs_precision,
const std::string &iop) {
setPrecisions(network);
if (!inputs_precision.empty()) {
auto precision = getInputPrecision(inputs_precision);
for (auto &&layer : network.getInputsInfo()) {
const auto layerPrecision = layer.second->getPrecision();
if ((isFloat(layerPrecision) && isFloat(precision)) ||
(isFP16(layerPrecision) && isU8(precision))) {
layer.second->setPrecision(precision);
}
}
}
if (!outputs_precision.empty()) {
auto precision = getOutputPrecision(outputs_precision);
for (auto &&layer : network.getOutputsInfo()) {
const auto layerPrecision = layer.second->getPrecision();
if (isFloat(layerPrecision) && isFloat(precision)) {
layer.second->setPrecision(precision);
}
}
}
if (!iop.empty()) {
setPrecisions(network, iop);
}
}
int main(int argc, char *argv[]) {
try {
std::cout << "Inference Engine: " << InferenceEngine::GetInferenceEngineVersion() << std::endl;
if (!parseCommandLine(&argc, &argv)) {
return EXIT_SUCCESS;
}
auto network = readNetwork(FLAGS_m);
processPrecisions(network, FLAGS_ip, FLAGS_op, FLAGS_iop);
InferenceEngine::Core ie;
auto executableNetwork = ie.LoadNetwork(network, "MYRIAD", configure(FLAGS_c, FLAGS_m));
std::string outputName = FLAGS_o;
if (outputName.empty()) {
outputName = fileNameNoExt(FLAGS_m) + ".blob";
}
executableNetwork.Export(outputName);
} catch (const std::exception &error) {
std::cerr << error.what() << std::endl;
return EXIT_FAILURE;
} catch (...) {
std::cerr << "Unknown/internal exception happened." << std::endl;
return EXIT_FAILURE;
}
std::cout << "Done" << std::endl;
return EXIT_SUCCESS;
}

299
inference-engine/tools/vpu/vpu_compile/vpu_tools_common.cpp
View File

@@ -1,299 +0,0 @@
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
/* on windows min and max already defined that makes using numeric_limits impossible */
#ifdef _WIN32
# define NOMINMAX
#endif
#include <sys/stat.h>
#include <algorithm>
#include <map>
#include <unordered_map>
#include <utility>
#include <vector>
#include <string>
#include <limits>
#include "vpu_tools_common.hpp"
#include <vpu/utils/string.hpp>
#include <samples/os/windows/w_dirent.h>
#include <samples/common.hpp>
#include "precision_utils.h"
InferenceEngine::CNNNetwork readNetwork(const std::string &xmlFileName) {
return InferenceEngine::Core().ReadNetwork(xmlFileName);
}
bool isFP16(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::FP16;
}
bool isFP32(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::FP32;
}
bool isU8(InferenceEngine::Precision precision) {
return precision == InferenceEngine::Precision::U8;
}
bool isFloat(InferenceEngine::Precision precision) {
return isFP16(precision) || isFP32(precision);
}
void setPrecisions(const InferenceEngine::CNNNetwork &network) {
for (auto &&layer : network.getInputsInfo()) {
if (isFP32(layer.second->getPrecision())) {
layer.second->setPrecision(InferenceEngine::Precision::FP16);
}
}
for (auto &&layer : network.getOutputsInfo()) {
if (isFP32(layer.second->getPrecision())) {
layer.second->setPrecision(InferenceEngine::Precision::FP16);
}
}
}
BitMap::BitMap(const std::string &filename) {
BmpHeader header;
BmpInfoHeader infoHeader;
std::ifstream input(filename, std::ios::binary);
if (!input) {
return;
}
input.read(reinterpret_cast<char *>(&header.type), 2);
if (header.type != 'M'*256+'B') {
std::cerr << "[BMP] file is not bmp type\n";
return;
}
input.read(reinterpret_cast<char *>(&header.size), 4);
input.read(reinterpret_cast<char *>(&header.reserved), 4);
input.read(reinterpret_cast<char *>(&header.offset), 4);
input.read(reinterpret_cast<char *>(&infoHeader), sizeof(BmpInfoHeader));
bool rowsReversed = infoHeader.height < 0;
_width = static_cast<std::size_t>(infoHeader.width);
_height = static_cast<std::size_t>(std::abs(infoHeader.height));
if (infoHeader.bits != 24) {
std::cerr << "[BMP] 24bpp only supported. But input has:" << infoHeader.bits << "\n";
return;
}
if (infoHeader.compression != 0) {
std::cerr << "[BMP] compression not supported\n";
}
auto padSize = _width & 3;
char pad[3];
size_t size = _width * _height * 3;
_data.reset(new unsigned char[size], std::default_delete<unsigned char[]>());
input.seekg(header.offset, std::ios::beg);
// reading by rows in invert vertically
for (uint32_t i = 0; i < _height; i++) {
uint32_t storeAt = rowsReversed ? i : (uint32_t)_height - 1 - i;
input.read(reinterpret_cast<char *>(_data.get()) + _width * 3 * storeAt, _width * 3);
input.read(pad, padSize);
}
}
void loadImage(const std::string &imageFilename, InferenceEngine::Blob::Ptr &blob) {
InferenceEngine::TensorDesc tensDesc = blob->getTensorDesc();
if (tensDesc.getPrecision() != InferenceEngine::Precision::FP16) {
throw std::invalid_argument("Input must have FP16 precision");
}
BitMap reader(imageFilename);
const auto dims = tensDesc.getDims();
auto numBlobChannels = dims[1];
size_t batch = dims[0];
size_t w = dims[3];
size_t h = dims[2];
size_t img_w = reader.width();
size_t img_h = reader.height();
size_t numImageChannels = reader.size() / (reader.width() * reader.height());
if (numBlobChannels != numImageChannels && numBlobChannels != 1) {
throw std::invalid_argument("Input channels mismatch: image channels " + std::to_string(numImageChannels) +
", network channels " + std::to_string(numBlobChannels) +
", expecting count of image channels are equal to count if network channels"
"or count of network channels are equal to 1");
}
int16_t *blobDataPtr = std::dynamic_pointer_cast<InferenceEngine::TBlob<int16_t>>(blob)->data();
auto nPixels = w * h;
unsigned char *RGB8 = reader.getData().get();
float xscale = 1.0f * img_w / w;
float yscale = 1.0f * img_h / h;
for (std::size_t n = 0; n != batch; n++) {
for (std::size_t i = 0; i < h; ++i) {
int y = static_cast<int>(std::floor((i + 0.5f) * yscale));
for (std::size_t j = 0; j < w; ++j) {
int x = static_cast<int>(std::floor((j + 0.5f) * xscale));
for (std::size_t k = 0; k < numBlobChannels; k++) {
float src = 1.0f * RGB8[(y * img_w + x) * numImageChannels + k];
if (tensDesc.getLayout() == InferenceEngine::NHWC) {
blobDataPtr[n * h * w * numBlobChannels + (i * w + j) * numBlobChannels + k] =
InferenceEngine::PrecisionUtils::f32tof16(src);
} else {
blobDataPtr[n * h * w * numBlobChannels + (i * w + j) + k * nPixels] =
InferenceEngine::PrecisionUtils::f32tof16(src);
}
}
}
}
}
}
void printPerformanceCounts(const std::map<std::string, InferenceEngine::InferenceEngineProfileInfo>& perfMap, const std::string report) {
std::vector<std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo>> perfVec(perfMap.begin(),
perfMap.end());
std::sort(perfVec.begin(), perfVec.end(),
[=](const std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo> &pair1,
const std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo> &pair2) -> bool {
return pair1.second.execution_index < pair2.second.execution_index;
});
size_t maxLayerName = 0u, maxExecType = 0u;
for (auto &&entry : perfVec) {
maxLayerName = std::max(maxLayerName, entry.first.length());
maxExecType = std::max(maxExecType, std::strlen(entry.second.exec_type));
}
size_t indexWidth = 7, nameWidth = maxLayerName + 5, typeWidth = maxExecType + 5, timeWidth = 10;
size_t totalWidth = indexWidth + nameWidth + typeWidth + timeWidth;
std::cout << std::endl << "Detailed " << report << " Profile" << std::endl;
for (size_t i = 0; i < totalWidth; i++)
std::cout << "=";
std::cout << std::endl;
std::cout << std::setw(static_cast<int>(indexWidth)) << std::left << "Index"
<< std::setw(static_cast<int>(nameWidth)) << std::left << "Name"
<< std::setw(static_cast<int>(typeWidth)) << std::left << "Type"
<< std::setw(static_cast<int>(timeWidth)) << std::right << "Time (ms)"
<< std::endl;
for (size_t i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
long long totalTime = 0;
for (const auto& p : perfVec) {
const auto& stageName = p.first;
const auto& info = p.second;
if (info.status == InferenceEngine::InferenceEngineProfileInfo::EXECUTED) {
std::cout << std::setw(static_cast<int>(indexWidth)) << std::left << info.execution_index
<< std::setw(static_cast<int>(nameWidth)) << std::left << stageName
<< std::setw(static_cast<int>(typeWidth)) << std::left << info.exec_type
<< std::setw(static_cast<int>(timeWidth)) << std::right << info.realTime_uSec / 1000.0
<< std::endl;
totalTime += info.realTime_uSec;
}
}
for (std::size_t i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
std::cout << std::setw(static_cast<int>(totalWidth / 2)) << std::right << "Total inference time:"
<< std::setw(static_cast<int>(totalWidth / 2 + 1)) << std::right << totalTime / 1000.0
<< std::endl;
for (std::size_t i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
}
std::vector<std::string> extractFilesByExtension(const std::string& directory, const std::string& extension) {
return extractFilesByExtension(directory, extension, std::numeric_limits<std::size_t>::max());
}
std::vector<std::string> extractFilesByExtension(const std::string& directory, const std::string& extension,
std::size_t max_size) {
if (max_size == 0) {
return {};
}
std::vector<std::string> files;
DIR* dir = opendir(directory.c_str());
if (!dir) {
throw std::invalid_argument("Can not open " + directory);
}
auto getExtension = [](const std::string& name) {
auto extensionPosition = name.rfind('.', name.size());
return extensionPosition == std::string::npos ? "" : name.substr(extensionPosition + 1, name.size() - 1);
};
dirent* ent = nullptr;
while ((ent = readdir(dir)) && files.size() < max_size) {
std::string file_name = ent->d_name;
if (getExtension(file_name) != extension) {
continue;
}
std::string full_file_name = directory + "/" + file_name;
struct stat st = {};
if (stat(full_file_name.c_str(), &st) != 0) {
continue;
}
bool is_directory = (st.st_mode & S_IFDIR) != 0;
if (is_directory) {
continue;
}
files.emplace_back(full_file_name);
}
closedir(dir);
return files;
}
void loadBinaryTensor(const std::string &binaryFileName, InferenceEngine::Blob::Ptr& blob) {
InferenceEngine::TensorDesc tensDesc = blob->getTensorDesc();
if (tensDesc.getPrecision() != InferenceEngine::Precision::FP16) {
throw std::invalid_argument("Input must have FP16 precision");
}
std::ifstream binaryFile(binaryFileName, std::ios_base::binary | std::ios_base::ate);
if (!binaryFile) {
throw std::invalid_argument("Can not open \"" + binaryFileName + "\"");
}
auto fileSize = static_cast<std::size_t>(binaryFile.tellg());
binaryFile.seekg(0, std::ios_base::beg);
if (!binaryFile.good()) {
throw std::invalid_argument("Can not read \"" + binaryFileName + "\"");
}
auto expected_size = blob->byteSize();
if (fileSize != expected_size) {
throw std::invalid_argument("File \"" + binaryFileName + "\" contains " + std::to_string(fileSize) + " bytes "
"but network expects " + std::to_string(expected_size));
}
/* try to read 32 bits data */
std::int16_t *blobDataPtr = std::dynamic_pointer_cast<InferenceEngine::TBlob<std::int16_t>>(blob)->data();
for (std::size_t i = 0; i < blob->size(); i++) {
float tmp = 0.f;
binaryFile.read(reinterpret_cast<char *>(&tmp), sizeof(float));
blobDataPtr[i] = InferenceEngine::PrecisionUtils::f32tof16(tmp);
}
}

71
inference-engine/tools/vpu/vpu_compile/vpu_tools_common.hpp
View File

@@ -1,71 +0,0 @@
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <string>
#include <map>
#include <memory>
#include <vector>
#include "inference_engine.hpp"
InferenceEngine::CNNNetwork readNetwork(const std::string &xmlFileName);
bool isFP16(InferenceEngine::Precision precision);
bool isFP32(InferenceEngine::Precision precision);
bool isU8(InferenceEngine::Precision precision);
bool isFloat(InferenceEngine::Precision precision);
/* Set FP32 to FP16, all others without changes */
void setPrecisions(const InferenceEngine::CNNNetwork &network);
class BitMap {
private:
typedef struct {
unsigned short type; /* Magic identifier */
unsigned int size; /* File size in bytes */
unsigned int reserved;
unsigned int offset; /* Offset to image data, bytes */
} BmpHeader;
typedef struct {
unsigned int size; /* Header size in bytes */
int width, height; /* Width and height of image */
unsigned short planes; /* Number of colour planes */
unsigned short bits; /* Bits per pixel */
unsigned int compression; /* Compression type */
unsigned int imagesize; /* Image size in bytes */
int xresolution, yresolution; /* Pixels per meter */
unsigned int ncolours; /* Number of colours */
unsigned int importantcolours; /* Important colours */
} BmpInfoHeader;
public:
explicit BitMap(const std::string &filename);
~BitMap() = default;
size_t _height = 0;
size_t _width = 0;
std::shared_ptr<unsigned char> _data;
public:
size_t size() const { return _width * _height * 3; }
size_t width() const { return _width; }
size_t height() const { return _height; }
std::shared_ptr<unsigned char> getData() {
return _data;
}
};
void loadImage(const std::string &imageFilename, InferenceEngine::Blob::Ptr &blob);
void printPerformanceCounts(const std::map<std::string, InferenceEngine::InferenceEngineProfileInfo>& perfMap, const std::string report = "per_layer");
std::vector<std::string> extractFilesByExtension(const std::string& directory, const std::string& extension);
std::vector<std::string> extractFilesByExtension(const std::string& directory, const std::string& extension,
std::size_t max_size);
void loadBinaryTensor(const std::string &binaryFileName, InferenceEngine::Blob::Ptr& blob);

47
inference-engine/tools/vpu/vpu_perfcheck/CMakeLists.txt
View File

@@ -1,47 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
function(add_perfcheck_target TARGET_NAME PLUGIN_NAME)
file(GLOB SOURCES *.cpp)
add_executable(${TARGET_NAME} ${SOURCES})
# TODO: enable some day and fix all warnings
# if(CMAKE_COMPILER_IS_GNUCXX)
# target_compile_options(${TARGET_NAME}
# PRIVATE
# "-Wall")
# endif()
target_include_directories(${TARGET_NAME}
SYSTEM PRIVATE
$<TARGET_PROPERTY:vpu_graph_transformer,INTERFACE_INCLUDE_DIRECTORIES>)
target_link_libraries(${TARGET_NAME}
PRIVATE
inference_engine
inference_engine_plugin_api
format_reader
Threads::Threads
ie_samples_utils)
add_dependencies(${TARGET_NAME}
${PLUGIN_NAME} ${ARGN})
set_target_properties(${TARGET_NAME} PROPERTIES
COMPILE_PDB_NAME ${TARGET_NAME}
FOLDER tools)
add_cpplint_target(${TARGET_NAME}_cpplint FOR_TARGETS ${TARGET_NAME})
endfunction()
if(ENABLE_MYRIAD)
add_perfcheck_target(myriad_perfcheck myriadPlugin)
ie_cpack_add_component(myriad_tools DEPENDS myriad)
install(TARGETS myriad_perfcheck
RUNTIME DESTINATION ${IE_CPACK_RUNTIME_PATH}
COMPONENT myriad_tools)
endif()

749
inference-engine/tools/vpu/vpu_perfcheck/main.cpp
View File

@@ -1,749 +0,0 @@
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#if defined(_WIN32)
#define NOMINMAX
#endif
#if (defined(_WIN32) || defined(_WIN64))
#define WIN32_LEAN_AND_MEAN
#else
#include <pthread.h>
#endif
#include <cmath>
#include <chrono>
#include <string>
#include <vector>
#include <iostream>
#include <fstream>
#include <functional>
#include <map>
#include <algorithm>
#include <utility>
#include <iomanip>
#include <atomic>
#include <memory>
#include <mutex>
#include <condition_variable>
#include <stdio.h>
#include <ios>
#include <sys/stat.h>
#include <samples/os/windows/w_dirent.h>
#include <inference_engine.hpp>
#include <precision_utils.h>
#include <samples/common.hpp>
#include <vpu/vpu_config.hpp>
static char* m_exename = nullptr;
#if defined(_WIN32) || defined(__APPLE__) || defined(ANDROID)
typedef std::chrono::time_point<std::chrono::steady_clock> time_point;
#else
typedef std::chrono::time_point<std::chrono::system_clock> time_point;
#endif
typedef std::chrono::high_resolution_clock Time;
typedef std::chrono::duration<double, std::ratio<1, 1000>> ms;
typedef std::chrono::duration<float> fsec;
#define TIMEDIFF(start, end) ((std::chrono::duration_cast<ms>((end) - (start))).count())
class BitMap {
private:
struct BmpHeader {
unsigned short type = 0u; /* Magic identifier */
unsigned int size = 0u; /* File size in bytes */
unsigned int reserved = 0u;
unsigned int offset = 0u; /* Offset to image data, bytes */
};
struct BmpInfoHeader {
unsigned int size = 0u; /* Header size in bytes */
int width = 0, height = 0; /* Width and height of image */
unsigned short planes = 0u; /* Number of colour planes */
unsigned short bits = 0u; /* Bits per pixel */
unsigned int compression = 0u; /* Compression type */
unsigned int imagesize = 0u; /* Image size in bytes */
int xresolution = 0, yresolution = 0; /* Pixels per meter */
unsigned int ncolours = 0u; /* Number of colours */
unsigned int importantcolours = 0u; /* Important colours */
};
public:
explicit BitMap(const std::string &filename) {
BmpHeader header;
BmpInfoHeader infoHeader;
std::ifstream input(filename, std::ios::binary);
if (!input) {
return;
}
input.read(reinterpret_cast<char *>(&header.type), 2);
if (header.type != 'M'*256+'B') {
std::cerr << "[BMP] file is not bmp type\n";
return;
}
input.read(reinterpret_cast<char *>(&header.size), 4);
input.read(reinterpret_cast<char *>(&header.reserved), 4);
input.read(reinterpret_cast<char *>(&header.offset), 4);
input.read(reinterpret_cast<char *>(&infoHeader), sizeof(BmpInfoHeader));
bool rowsReversed = infoHeader.height < 0;
_width = infoHeader.width;
_height = abs(infoHeader.height);
if (infoHeader.bits != 24) {
std::cerr << "[BMP] 24bpp only supported. But input has:" << infoHeader.bits << "\n";
return;
}
if (infoHeader.compression != 0) {
std::cerr << "[BMP] compression not supported\n";
}
int padSize = _width & 3;
char pad[3];
size_t size = _width * _height * 3;
_data.reset(new unsigned char[size], std::default_delete<unsigned char[]>());
input.seekg(header.offset, std::ios::beg);
// reading by rows in invert vertically
for (uint32_t i = 0; i < _height; i++) {
uint32_t storeAt = rowsReversed ? i : (uint32_t)_height - 1 - i;
input.read(reinterpret_cast<char *>(_data.get()) + _width * 3 * storeAt, _width * 3);
input.read(pad, padSize);
}
}
~BitMap() = default;
size_t _height = 0;
size_t _width = 0;
std::shared_ptr<unsigned char> _data;
public:
size_t size() const { return _width * _height * 3; }
size_t width() const { return _width; }
size_t height() const { return _height; }
std::shared_ptr<unsigned char> getData() {
return _data;
}
};
static bool loadImage(const std::string &imageFilename, InferenceEngine::Blob::Ptr &blob);
static bool loadVideo(const std::vector<std::string> &imagesFolder, InferenceEngine::Blob::Ptr &blob);
static bool loadBinaryTensor(const std::string &binaryFilename, InferenceEngine::Blob::Ptr &blob);
static void setConfig(std::map<std::string, std::string>& config,
const std::string& file_config_cl) {
config[CONFIG_KEY(LOG_LEVEL)] = CONFIG_VALUE(LOG_WARNING);
config[InferenceEngine::MYRIAD_ENABLE_RECEIVING_TENSOR_TIME] = CONFIG_VALUE(YES);
config[InferenceEngine::MYRIAD_CUSTOM_LAYERS] = file_config_cl;
}
static void printPerformanceCounts(const std::map<std::string, InferenceEngine::InferenceEngineProfileInfo>& perfMap) {
std::vector<std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo>> perfVec(perfMap.begin(),
perfMap.end());
std::sort(perfVec.begin(), perfVec.end(),
[=](const std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo> &pair1,
const std::pair<std::string, InferenceEngine::InferenceEngineProfileInfo> &pair2) -> bool {
return pair1.second.execution_index < pair2.second.execution_index;
});
size_t maxLayerName = 0u, maxExecType = 0u;
for (auto & it : perfVec) {
maxLayerName = std::max(maxLayerName, it.first.length());
maxExecType = std::max(maxExecType, std::strlen(it.second.exec_type));
}
size_t indexWidth = 7, nameWidth = maxLayerName + 5, typeWidth = maxExecType + 5, timeWidth = 10;
size_t totalWidth = indexWidth + nameWidth + typeWidth + timeWidth;
std::cout << std::endl << "Detailed Per Stage Profile" << std::endl;
for (size_t i = 0; i < totalWidth; i++)
std::cout << "=";
std::cout << std::endl;
std::cout << std::setw(indexWidth) << std::left << "Index"
<< std::setw(nameWidth) << std::left << "Name"
<< std::setw(typeWidth) << std::left << "Type"
<< std::setw(timeWidth) << std::right << "Time (ms)"
<< std::endl;
for (size_t i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
long long totalTime = 0;
for (const auto& p : perfVec) {
const auto& stageName = p.first;
const auto& info = p.second;
if (info.status == InferenceEngine::InferenceEngineProfileInfo::EXECUTED) {
std::cout << std::setw(indexWidth) << std::left << info.execution_index
<< std::setw(nameWidth) << std::left << stageName
<< std::setw(typeWidth) << std::left << info.exec_type
<< std::setw(timeWidth) << std::right << info.realTime_uSec / 1000.0
<< std::endl;
totalTime += info.realTime_uSec;
}
}
for (int i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
std::cout << std::setw(totalWidth / 2) << std::right << "Total inference time:"
<< std::setw(totalWidth / 2 + 1) << std::right << totalTime / 1000.0
<< std::endl;
for (int i = 0; i < totalWidth; i++)
std::cout << "-";
std::cout << std::endl;
}
static std::string getAppRealName(const char* name) {
std::string filename(name);
size_t splitpos = filename.find_last_of('\\');
if (std::string::npos == splitpos) {
splitpos = filename.find_last_of('/');
if (std::string::npos == splitpos) {
return filename;
}
}
return filename.substr(splitpos + 1);
}
static void print_usage() {
std::cout << "Usage:" << std::endl << getAppRealName(m_exename) << " <model_path> <img_dir_path> [number of iterations >= 1000]"
<< " [batch >= 1, default=1] [num_networks, default=1] [config_file_custom_layer, default='']" << std::endl;
}
static void getBMPFiles(std::vector<std::string> &out, const std::string &directory) {
const std::string ext = ".bmp";
DIR *dir;
dirent *ent;
dir = opendir(directory.c_str());
if (!dir)
return;
while ((ent = readdir(dir)) != nullptr) {
const std::string file_name = ent->d_name;
const std::string full_file_name = directory + "/" + file_name;
if ((file_name.length() >= ext.length())
&& (0 == file_name.compare(file_name.length() - ext.length(), ext.length(), ext))) {
// proceed
} else {
continue;
}
struct stat st;
if (stat(full_file_name.c_str(), &st) == -1)
continue;
const bool is_directory = (st.st_mode & S_IFDIR) != 0;
if (is_directory)
continue;
out.push_back(full_file_name);
}
closedir(dir);
}
static void getBINFiles(std::vector<std::string> &out, const std::string &directory) {
const std::string ext = ".bin";
DIR *dir;
dirent *ent;
dir = opendir(directory.c_str());
if (!dir)
return;
while ((ent = readdir(dir)) != nullptr) {
const std::string file_name = ent->d_name;
const std::string full_file_name = directory + "/" + file_name;
if ((file_name.length() >= ext.length())
&& (0 == file_name.compare(file_name.length() - ext.length(), ext.length(), ext))) {
// proceed
} else {
continue;
}
struct stat st;
if (stat(full_file_name.c_str(), &st) == -1)
continue;
const bool is_directory = (st.st_mode & S_IFDIR) != 0;
if (is_directory)
continue;
out.push_back(full_file_name);
}
closedir(dir);
}
int num_requests = 4;
#define MIN_ITER 1000
#define USE_CALLBACK
int niter;
std::atomic<int> iterations_to_run;
std::mutex done_mutex;
std::condition_variable alldone;
int reallydone = 0;
std::vector<time_point> iter_start;
std::vector<time_point> iter_end;
std::vector<double> iter_time;
const int profile = 0;
std::map<std::string, InferenceEngine::InferenceEngineProfileInfo> perfMap;
int process(const std::string& modelFileName, const std::string& inputsDir,
std::string& file_config_cl, int nBatch, int num_networks) {
InferenceEngine::Core ie;
niter /= nBatch;
num_requests = num_requests * num_networks;
// add some more requests. they'll be excluded on performance measurement
niter += 2 * 2 * num_requests;
#if !(defined(_WIN32) || defined(_WIN64))
if (pthread_setname_np(
#ifndef __APPLE__
pthread_self(),
#endif
"MainThread") != 0) {
perror("Setting name for main thread failed");
}
#endif
#ifdef USE_KMB_PLUGIN
std::string deivceName = "KMB";
#else
std::string deviceName = "MYRIAD";
#endif
const auto pluginVersion = ie.GetVersions(deviceName);
std::cout << "InferenceEngine: " << std::endl;
std::cout << pluginVersion << std::endl << std::endl;
std::ifstream file(file_config_cl);
if (!file.is_open()) {
file_config_cl.clear();
}
std::vector<std::string> pictures;
getBMPFiles(pictures, inputsDir);
int numPictures = pictures.size();
std::vector<std::string> binaries;
getBINFiles(binaries, inputsDir);
int numBinaries = binaries.size();
if (pictures.empty() && binaries.empty()) {
std::cout << inputsDir << " directory doesn't contain input files" << std::endl;
return 1;
}
InferenceEngine::CNNNetwork cnnNetwork = ie.ReadNetwork(modelFileName);
if (nBatch != 1) {
std::cout << "Setting batch to : "<< nBatch << "\n";
cnnNetwork.setBatchSize(nBatch);
}
InferenceEngine::InputsDataMap networkInputs;
networkInputs = cnnNetwork.getInputsInfo();
InferenceEngine::OutputsDataMap networkOutputs;
networkOutputs = cnnNetwork.getOutputsInfo();
for (auto &input : networkInputs) {
const auto inputPrecision = input.second->getPrecision();
if (inputPrecision == InferenceEngine::Precision::FP32 ||
inputPrecision == InferenceEngine::Precision::U8) {
input.second->setPrecision(InferenceEngine::Precision::FP16);
}
}
for (auto &output : networkOutputs) {
const auto outputPrecision = output.second->getPrecision();
if (outputPrecision == InferenceEngine::Precision::FP32) {
output.second->setPrecision(InferenceEngine::Precision::FP16);
}
}
std::vector<InferenceEngine::ExecutableNetwork> exeNetwork(num_networks);
std::map<std::string, std::string> networkConfig;
setConfig(networkConfig, file_config_cl);
for (int n = 0; n < num_networks; ++n) {
if (num_networks > 1)
printf("Load network %d...\n", n);
else
printf("Load network... \n");
fflush(stdout);
exeNetwork[n] = ie.LoadNetwork(cnnNetwork, deviceName, networkConfig);
}
std::vector<InferenceEngine::InferRequest> request(num_requests);
iter_start.resize(niter);
iter_end.resize(niter);
iter_time.resize(niter);
iterations_to_run = niter - num_requests;
for (int r = 0, idxPic = 0; r < num_requests; ++r) {
int n = r % num_networks;
request[r] = exeNetwork[n].CreateInferRequest();
for (auto &input : networkInputs) {
auto inputBlob = request[r].GetBlob(input.first);
const auto& dims = inputBlob->getTensorDesc().getDims();
auto layout = inputBlob->getTensorDesc().getLayout();
// number of channels is 3 for Image, dims order is always NCHW
const bool isImage = ((layout == InferenceEngine::NHWC || layout == InferenceEngine::NCHW) && dims[1] == 3);
const bool isVideo = (inputBlob->getTensorDesc().getDims().size() == 5);
if (isImage && (numPictures > 0)) {
if (!loadImage(pictures[(idxPic++) % numPictures], inputBlob))
return 1;
} else if (isVideo && (numPictures > 0)) {
if (!loadVideo(pictures, inputBlob))
return 1;
} else if (numBinaries > 0) {
if (!loadBinaryTensor(binaries[(idxPic++) % numBinaries], inputBlob))
return 1;
} else {
std::cout << inputsDir << " directory doesn't contain correct input files" << std::endl;
return 1;
}
}
request[r].SetCompletionCallback<std::function<void(InferenceEngine::InferRequest, InferenceEngine::StatusCode)>>(
[](InferenceEngine::InferRequest request, InferenceEngine::StatusCode code) -> void {
if (code != InferenceEngine::OK) {
std::cout << "Infer failed: " << code << std::endl;
exit(1);
}
int iter = --iterations_to_run;
int reqIdx = (niter - iter - 1) - num_requests;
iter_end[reqIdx] = Time::now();
if (profile && (reqIdx == niter / 2)) {
perfMap = request.GetPerformanceCounts();
}
if (iter >= 0) {
iter_start[reqIdx + (num_requests)] = Time::now();
request.StartAsync();
}
iter_time[reqIdx] = TIMEDIFF(iter_start[reqIdx], iter_end[reqIdx]);
// printf("request#%d %fms\n", reqIdx, iter_time[reqIdx]);
if (iter == -num_requests) {
reallydone = 1;
alldone.notify_all();
}
});
}
printf("Inference started. Running %d iterations...\n", niter - 2 * 2 * num_requests);
fflush(stdout);
for (int r = 0; r < num_requests; ++r) {
iter_start[r] = Time::now();
request[r].StartAsync();
}
{
std::unique_lock<std::mutex> lock(done_mutex);
alldone.wait(lock, [&](){return reallydone;});
}
// check 10 time intervals to get min/max fps values
const int fps_checks = 10;
// exclude (2 * num_requests) first and last iterations
int num_exclude = 2 * num_requests;
time_point cstart = iter_end[num_exclude - 1];
time_point cend = iter_end[niter - num_exclude - 1];
double totalTime = (std::chrono::duration_cast<ms>(cend - cstart)).count();
std::cout << std::endl << "Total time: " << (totalTime) << " ms" << std::endl;
std::cout << "Average fps on " << (niter - 2 * num_exclude) << " iterations"
<< (nBatch == 1 ? ": " : (" of " + std::to_string(nBatch) + " frames: "))
<< static_cast<double>(niter - 2 * num_exclude) * 1000.0 * nBatch / (totalTime) << " fps" << std::endl;
double check_time = totalTime / fps_checks;
double min_fps = 100000;
double max_fps = -100000;
int citer = num_exclude;
for (int f = 0; f < fps_checks; ++f) {
int fiter = 0;
auto fend = (f < fps_checks - 1) ? cstart + std::chrono::microseconds((unsigned int)(check_time * 1000)) : cend;
while ((citer + fiter < niter - num_exclude) && iter_end[citer + fiter] <= fend) {
fiter++;
}
double ffps = 1000 * fiter * nBatch / (check_time);
min_fps = std::min(min_fps, ffps);
max_fps = std::max(max_fps, ffps);
citer += fiter;
cstart = fend;
}
std::cout << "Min fps: " << min_fps << std::endl;
std::cout << "Max fps: " << max_fps << std::endl;
if (profile) {
printPerformanceCounts(perfMap);
}
return 0;
}
int main(int argc, char *argv[]) {
niter = MIN_ITER;
int num_networks = 1;
int nBatch = 1;
std::string file_config_cl;
m_exename = argv[0];
if (argc < 3) {
print_usage();
return 0;
}
auto parse = [](const std::string& src) {
try {
return std::stol(src, nullptr, 0);
} catch (const std::invalid_argument& exception) {
std::cout << "Cannot perform conversion for " << src << ": " << exception.what() << std::endl;
print_usage();
std::abort();
} catch (const std::out_of_range& exception) {
std::cout << src << " is out of range: " << exception.what() << std::endl;
print_usage();
std::abort();
} catch (...) {
std::cout << "Unexpected exception" << std::endl;
print_usage();
std::abort();
}
};
if (argc > 3) {
niter = static_cast<int>(parse(argv[3]));
}
if (argc > 4) {
nBatch = static_cast<int>(parse(argv[4]));
}
if (argc > 5) {
num_networks = static_cast<int>(parse(argv[5]));
}
if (argc > 6) {
file_config_cl = std::string(argv[6]);
}
if (niter < MIN_ITER) {
print_usage();
return 0;
}
if (num_networks < 1 || num_networks > 16) {
print_usage();
return 0;
}
if (nBatch < 1) {
print_usage();
return 0;
}
try {
std::string modelFileName(argv[1]);
std::string inputsDir(argv[2]);
return process(modelFileName, inputsDir, file_config_cl, nBatch, num_networks);
}
catch (const std::exception& ex) {
std::cout << ex.what();
}
return -1;
}
static bool loadImage(const std::string &imageFilename, InferenceEngine::Blob::Ptr &blob) {
InferenceEngine::TensorDesc tensDesc = blob->getTensorDesc();
const InferenceEngine::Layout layout = tensDesc.getLayout();
if (tensDesc.getPrecision() != InferenceEngine::Precision::FP16) {
std::cout << "loadImage error: Input must have FP16 precision" << std::endl;
return false;
}
if (layout != InferenceEngine::NHWC && layout != InferenceEngine::NCHW) {
std::cout << "loadImage error: Input must have NCHW or NHWC layout" << std::endl;
return false;
}
BitMap reader(imageFilename);
const auto dims = tensDesc.getDims();
const size_t N = dims[0];
const size_t C = dims[1];
const size_t H = dims[2];
const size_t W = dims[3];
const size_t img_w = reader.width();
const size_t img_h = reader.height();
const auto strides = tensDesc.getBlockingDesc().getStrides();
const auto strideN = strides[0];
const auto strideC = layout == InferenceEngine::NHWC ? strides[3] : strides[1];
const auto strideH = layout == InferenceEngine::NHWC ? strides[1] : strides[2];
const auto strideW = layout == InferenceEngine::NHWC ? strides[2] : strides[3];
const size_t numImageChannels = reader.size() / (reader.width() * reader.height());
if (C != numImageChannels && C != 1) {
std::cout << "loadImage error: Input channels mismatch: image channels " << numImageChannels << ", "
<< "network channels " << C << ", expecting count of image channels are equal "
<< "to count if network channels or count of network channels are equal to 1" << std::endl;
return false;
}
int16_t* blobDataPtr = std::dynamic_pointer_cast<InferenceEngine::TBlob<int16_t>>(blob)->data();
const unsigned char* RGB8 = reader.getData().get();
const float xScale = 1.0f * img_w / W;
const float yScale = 1.0f * img_h / H;
for (int n = 0; n != N; n++) {
for (int h = 0; h < H; ++h) {
int y = static_cast<int>(std::floor((h + 0.5f) * yScale));
for (int w = 0; w < W; ++w) {
int x = static_cast<int>(std::floor((w + 0.5f) * xScale));
for (int c = 0; c < C; c++) {
blobDataPtr[n * strideN + c * strideC + h * strideH + w * strideW] =
InferenceEngine::PrecisionUtils::f32tof16(1.0 * RGB8[(y * img_w + x) * numImageChannels + c]);
}
}
}
}
return true;
}
static bool loadVideo(const std::vector<std::string> &imagesFolder, InferenceEngine::Blob::Ptr &blob) {
InferenceEngine::TensorDesc tensDesc = blob->getTensorDesc();
const InferenceEngine::Layout layout = tensDesc.getLayout();
if (tensDesc.getPrecision() != InferenceEngine::Precision::FP16) {
std::cout << "loadVideo error: Input must have FP16 precision" << std::endl;
return false;
}
if (layout != InferenceEngine::NDHWC && layout != InferenceEngine::NCDHW) {
std::cout << "loadVideo error: Input must have NCDHW or NDHWC layout" << std::endl;
return false;
}
const auto dims = tensDesc.getDims();
const size_t N = dims[0];
const size_t C = dims[1];
const size_t D = dims[2];
const size_t H = dims[3];
const size_t W = dims[4];
const auto numUsedImages = std::min(D, imagesFolder.size());
const auto strides = tensDesc.getBlockingDesc().getStrides();
const auto strideN = strides[0];
const auto strideC = layout == InferenceEngine::NDHWC ? strides[4] : strides[1];
const auto strideD = layout == InferenceEngine::NDHWC ? strides[1] : strides[2];
const auto strideH = layout == InferenceEngine::NDHWC ? strides[2] : strides[3];
const auto strideW = layout == InferenceEngine::NDHWC ? strides[3] : strides[4];
auto d = 0;
int16_t* blobDataPtr = std::dynamic_pointer_cast<InferenceEngine::TBlob<int16_t>>(blob)->data();
for ( ; d < numUsedImages; d++) {
BitMap reader(imagesFolder[d]);
const size_t img_w = reader.width();
const size_t img_h = reader.height();
const size_t numImageChannels = reader.size() / (reader.width() * reader.height());
if (C != numImageChannels && C != 1) {
std::cout << "loadVideo error: Input channels mismatch: image channels " << numImageChannels << ", "
<< "network channels " << C << ", expecting count of image channels are equal "
<< "to count if network channels or count of network channels are equal to 1" << std::endl;
return false;
}
const unsigned char* RGB8 = reader.getData().get();
const float xScale = 1.0f * img_w / W;
const float yScale = 1.0f * img_h / H;
for (int n = 0; n != N; n++) {
for (int h = 0; h < H; ++h) {
int y = static_cast<int>(std::floor((h + 0.5f) * yScale));
for (int w = 0; w < W; ++w) {
int x = static_cast<int>(std::floor((w + 0.5f) * xScale));
for (int c = 0; c < C; c++) {
blobDataPtr[n * strideN + c * strideC + d * strideD + h * strideH + w * strideW] =
InferenceEngine::PrecisionUtils::f32tof16(1.0 * RGB8[(y * img_w + x) * numImageChannels + c]);
}
}
}
}
}
for (; d < D; d++)
for (auto n = 0; n != N; n++)
for (auto c = 0; c < C; c++)
for (auto k = 0; k < strideD; k++) {
blobDataPtr[n * strideN + c * strideC + (d) * strideD + k] =
blobDataPtr[n * strideN + c * strideC + (d - 1) * strideD + k];
}
return true;
}
bool loadBinaryTensor(const std::string &binaryFilename, InferenceEngine::Blob::Ptr &blob) {
InferenceEngine::TensorDesc tensDesc = blob->getTensorDesc();
if (tensDesc.getPrecision() != InferenceEngine::Precision::FP16) {
std::cout << "loadBinaryTensor error: Input must have FP16 precision" << std::endl;
return false;
}
std::ifstream binaryFile(binaryFilename, std::ios_base::binary | std::ios_base::ate);
if (!binaryFile) {
std::cout << "loadBinaryTensor error: While opening a file an error is encountered" << std::endl;
return false;
}
int fileSize = binaryFile.tellg();
binaryFile.seekg(0, std::ios_base::beg);
size_t count = blob->size();
if (fileSize != count * sizeof(float)) {
std::cout << "loadBinaryTensor error: File contains insufficient items" << std::endl;
return false;
}
if (binaryFile.good()) {
int16_t *blobDataPtr = std::dynamic_pointer_cast<InferenceEngine::TBlob<int16_t>>(blob)->data();
for (size_t i = 0; i < count; i++) {
float tmp = 0.f;
binaryFile.read(reinterpret_cast<char *>(&tmp), sizeof(float));
blobDataPtr[i] = InferenceEngine::PrecisionUtils::f32tof16(tmp);
}
} else {
std::cout << "loadBinaryTensor error: While reading a file an error is encountered" << std::endl;
return false;
}
return true;
}

2
scripts/deployment_manager/configs/darwin.json
View File

@@ -46,8 +46,6 @@
"dependencies" : ["ie_core"],
"files": [
"deployment_tools/inference_engine/lib/intel64/libmyriadPlugin.so",
"deployment_tools/inference_engine/lib/intel64/myriad_compile",
"deployment_tools/inference_engine/lib/intel64/myriad_perfcheck",
"deployment_tools/inference_engine/lib/intel64/libinference_engine_legacy.dylib",
"deployment_tools/inference_engine/lib/intel64/usb-ma2x8x.mvcmd",
"deployment_tools/inference_engine/lib/intel64/pcie-ma2x8x.mvcmd"

2
scripts/deployment_manager/configs/linux.json
View File

@@ -66,8 +66,6 @@
"deployment_tools/inference_engine/lib/intel64/usb-ma2x8x.mvcmd",
"deployment_tools/inference_engine/lib/intel64/pcie-ma2x8x.mvcmd",
"deployment_tools/inference_engine/lib/intel64/libmyriadPlugin.so",
"deployment_tools/inference_engine/lib/intel64/myriad_compile",
"deployment_tools/inference_engine/lib/intel64/myriad_perfcheck",
"deployment_tools/inference_engine/lib/intel64/vpu_custom_kernels",
"deployment_tools/inference_engine/lib/intel64/libinference_engine_legacy.so",
"install_dependencies/install_NCS_udev_rules.sh"

4
scripts/deployment_manager/configs/windows.json
View File

@@ -61,9 +61,7 @@
"deployment_tools/inference_engine/bin/intel64/Release/usb-ma2x8x.mvcmd",
"deployment_tools/inference_engine/bin/intel64/Release/pcie-ma2x8x.elf",
"deployment_tools/inference_engine/bin/intel64/Release/myriadPlugin.dll",
"deployment_tools/inference_engine/bin/intel64/Release/inference_engine_legacy.dll",
"deployment_tools/inference_engine/bin/intel64/Release/myriad_compile.exe",
"deployment_tools/inference_engine/bin/intel64/Release/myriad_perfcheck.exe"
"deployment_tools/inference_engine/bin/intel64/Release/inference_engine_legacy.dll"
]
},
"gna": {