IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

CVS-44774: Fixed preprocessing for template plugin (#4118)

Browse Source 
* Fixed preprocessing for template plugin

* Added more tests instances

* Split common transformation to smaller ones which can be used by plugins

* Moved preprocessing transformation to Plugin API

* Added PreprocessConversionTest tests

* Disabled tests on GPU: CVS-51764

* Disabled some tests on VPU and TEMPLATE

* Support for input layout conversions in TEMPLATE plugin

* Improvements in Template Plugin

* Fixed compilation

* Fixes

* Disables some tests

* Fixed compilation on Windows

* Fixed docs
This commit is contained in:
Ilya Lavrenov 2021-04-30 10:47:29 +03:00 committed by GitHub
parent ff9e67e732
commit 8b1b900591
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
23 changed files with 589 additions and 52 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
docs/nGraph_DG/nGraphTransformation.md
View File

@ -61,9 +61,9 @@ nGraph has three main transformation types:
Template for FunctionPass transformation class
@snippet src/template_function_transformation.hpp function_pass:template_transformation_hpp
@snippet src/transformations/template_function_transformation.hpp function_pass:template_transformation_hpp
@snippet src/template_function_transformation.cpp function_pass:template_transformation_cpp
@snippet src/transformations/template_function_transformation.cpp function_pass:template_transformation_cpp
Using `ngraph::FunctionPass`, you need to override the `run_on_function` method where you will write the transformation code.
Return value is `true` if the original function has changed during transformation (new operation was added, or operations replacement was made, or node attributes were changed); otherwise, it is `false`.
@ -75,9 +75,9 @@ Also `ngraph::FunctionPass` based transformations can be executed via `pass::Man
`ngraph::pass::MatcherPass` is used for pattern-based transformations.
Template for MatcherPass transformation class
@snippet src/template_pattern_transformation.hpp graph_rewrite:template_transformation_hpp
@snippet src/transformations/template_pattern_transformation.hpp graph_rewrite:template_transformation_hpp
@snippet src/template_pattern_transformation.cpp graph_rewrite:template_transformation_cpp
@snippet src/transformations/template_pattern_transformation.cpp graph_rewrite:template_transformation_cpp
To use `ngraph::pass::MatcherPass`, you need to complete these steps:
1. Create a pattern
@ -113,7 +113,7 @@ That means that matcher passes registered in `pass::GraphRewrite` will be applie
The example below shows how single MatcherPass can fuse sequence of operations using the `register_new_node` method.
@snippet src/template_pattern_transformation.cpp matcher_pass:relu_fusion
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:relu_fusion
> **NOTE**: If you register multiple nodes, please add them in topological order. We do not topologically sort these nodes as it is a time-consuming operation.
@ -128,11 +128,11 @@ register_matcher(m, callback);
### Execute MatcherPass
MatcherPass has multiple ways to be executed:
* Run on a single node - it can be useful if you want to run MatcherPass inside another transformation.
@snippet src/template_pattern_transformation.cpp matcher_pass:run_on_node
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:run_on_node
* Run on `ngraph::Function` using GraphRewrite - this approach gives ability to run MatcherPass on whole `ngraph::Function`. Moreover, multiple MatcherPass transformation can be registered in a single GraphRewite to be executed in a single graph traversal.
@snippet src/template_pattern_transformation.cpp matcher_pass:graph_rewrite
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:graph_rewrite
* Run on `ngraph::Function` using `pass::Manager` - this approach helps you to register MatcherPass for execution on `ngraph::Function` as another transformation types.
@snippet src/template_pattern_transformation.cpp matcher_pass:manager
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:manager
### ngraph::pass::GraphRewrite <a name="graph_rewrite_pass"></a>
@ -140,7 +140,7 @@ MatcherPass has multiple ways to be executed:
GraphRewrite pass serves for running multiple matcher passes on `ngraph::Function` in a single graph traversal.
Example:
@snippet src/template_pattern_transformation.cpp matcher_pass:graph_rewrite
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:graph_rewrite
In addition, GraphRewrite handles nodes that were registered by MatcherPasses during their execution. This nodes will be added to the beginning of the sequence with nodes for pattern matching.
@ -352,7 +352,7 @@ Manual constant folding is more preferable than `ngraph::pass::ConstantFolding()
Below you can find an example of manual constant folding:
@snippet src/template_pattern_transformation.cpp manual_constant_folding
@snippet src/transformations/template_pattern_transformation.cpp manual_constant_folding
## Common mistakes in transformations <a name="common_mistakes"></a>
@ -373,11 +373,11 @@ In addition, `ngraph::pass::Manager` has extended debug capabilities (find more
The example below shows basic usage of `ngraph::pass::Manager`
@snippet src/template_pattern_transformation.cpp matcher_pass:manager3
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:manager3
Another example shows how multiple matcher passes can be united into single GraphRewrite.
@snippet src/template_pattern_transformation.cpp matcher_pass:manager2
@snippet src/transformations/template_pattern_transformation.cpp matcher_pass:manager2
> **Note:** nGraph used to have the `pass::PassConfig` class for transformation pipeline manipulation.
This mechanism is now obsolete and the `pass::PassConfig` class will be removed in future release.

2
docs/template_plugin/src/CMakeLists.txt
View File

@ -5,7 +5,7 @@
# [cmake:plugin]
set(TARGET_NAME "templatePlugin")
file(GLOB SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
file(GLOB_RECURSE SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp)
file(GLOB_RECURSE HEADERS ${CMAKE_CURRENT_SOURCE_DIR}/*.hpp)
# adds a shared library with plugin

2
docs/template_plugin/src/template_config.cpp
View File

@ -28,7 +28,7 @@ Configuration::Configuration(const ConfigMap& config, const Configuration & defa
} else if (CONFIG_KEY(DEVICE_ID) == key) {
deviceId = std::stoi(value);
if (deviceId > 0) {
IE_THROW() << "Device ID " << deviceId << " is not supported";
IE_THROW(NotImplemented) << "Device ID " << deviceId << " is not supported";
}
} else if (CONFIG_KEY(PERF_COUNT) == key) {
perfCount = (CONFIG_VALUE(YES) == value);

26
docs/template_plugin/src/template_executable_network.cpp
View File

@ -16,6 +16,8 @@ using namespace TemplatePlugin;
// ! [executable_network:ctor_cnnnetwork]
TemplatePlugin::ExecutableNetwork::ExecutableNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap& inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap,
const Configuration& cfg,
const Plugin::Ptr& plugin) :
InferenceEngine::ExecutableNetworkThreadSafeDefault(nullptr, nullptr), // Disable default threads creation
@ -25,14 +27,14 @@ TemplatePlugin::ExecutableNetwork::ExecutableNetwork(const std::shared_ptr<const
// you should select proper device based on KEY_DEVICE_ID or automatic behavior
// In this case, _waitExecutor should also be created per device.
try {
CompileNetwork(function);
CompileNetwork(function, inputInfoMap, outputsInfoMap);
InitExecutor(); // creates thread-based executor using for async requests
} catch (const InferenceEngine::Exception&) {
throw;
} catch (const std::exception & e) {
IE_THROW() << "Standard exception from compilation library: " << e.what();
IE_THROW(Unexpected) << "Standard exception from compilation library: " << e.what();
} catch (...) {
IE_THROW() << "Generic exception is thrown";
IE_THROW(Unexpected) << "Generic exception is thrown";
}
}
// ! [executable_network:ctor_cnnnetwork]
@ -64,6 +66,8 @@ TemplatePlugin::ExecutableNetwork::ExecutableNetwork(std::istream & model,
// TODO: implement Import / Export of configuration options and merge with `cfg`
// TODO: implement Import / Export of network precisions, layouts, preprocessing info
InferenceEngine::InputsDataMap inputInfoMap;
InferenceEngine::OutputsDataMap outputInfoMap;
auto cnnnetwork = _plugin->GetCore()->ReadNetwork(xmlString, std::move(dataBlob));
@ -72,27 +76,31 @@ TemplatePlugin::ExecutableNetwork::ExecutableNetwork(std::istream & model,
SetPointerToPlugin(_plugin->shared_from_this());
try {
CompileNetwork(cnnnetwork.getFunction());
CompileNetwork(cnnnetwork.getFunction(), inputInfoMap, outputInfoMap);
InitExecutor(); // creates thread-based executor using for async requests
} catch (const InferenceEngine::Exception&) {
throw;
} catch (const std::exception & e) {
IE_THROW() << "Standard exception from compilation library: " << e.what();
IE_THROW(Unexpected) << "Standard exception from compilation library: " << e.what();
} catch (...) {
IE_THROW() << "Generic exception is thrown";
IE_THROW(Unexpected) << "Generic exception is thrown";
}
}
// ! [executable_network:ctor_import_stream]
// ! [executable_network:map_graph]
// forward declaration
std::shared_ptr<ngraph::Function> TransformNetwork(const std::shared_ptr<const ngraph::Function>& function);
std::shared_ptr<ngraph::Function> TransformNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap & inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap);
void TemplatePlugin::ExecutableNetwork::CompileNetwork(const std::shared_ptr<const ngraph::Function>& function) {
void TemplatePlugin::ExecutableNetwork::CompileNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap & inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap) {
// TODO: perform actual graph compilation / mapping to backend graph representation / kernels
// apply plugins transformations
_function = TransformNetwork(function);
_function = TransformNetwork(function, inputInfoMap, outputsInfoMap);
// Generate backend specific blob mappings. For example Inference Engine uses not ngraph::Result nodes friendly name
// as inference request output names but the name of the layer before.

8
docs/template_plugin/src/template_executable_network.hpp
View File

@ -25,6 +25,8 @@ class Plugin;
class ExecutableNetwork : public InferenceEngine::ExecutableNetworkThreadSafeDefault {
public:
ExecutableNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap& inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap,
const Configuration& cfg,
const std::shared_ptr<Plugin>& plugin);
@ -38,7 +40,7 @@ public:
void ExportImpl(std::ostream& model) override;
InferenceEngine::IInferRequestInternal::Ptr CreateInferRequestImpl(InferenceEngine::InputsDataMap networkInputs,
InferenceEngine::OutputsDataMap networkOutputs) override;
InferenceEngine::OutputsDataMap networkOutputs) override;
InferenceEngine::IInferRequestInternal::Ptr CreateInferRequest() override;
InferenceEngine::Parameter GetMetric(const std::string &name) const override;
InferenceEngine::Parameter GetConfig(const std::string &name) const override;
@ -46,7 +48,9 @@ public:
private:
friend class TemplateInferRequest;
void CompileNetwork(const std::shared_ptr<const ngraph::Function>& function);
void CompileNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap& inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap);
void InitExecutor();
std::atomic<std::size_t> _requestId = {0};

25
docs/template_plugin/src/template_infer_request.cpp
View File

@ -61,7 +61,8 @@ template<typename BlobDataMap, typename GetNetworkPrecisionF>
static void AllocateImpl(const BlobDataMap& userDataMap,
BlobMap& userBlobMap,
BlobMap& deviceBlobMap,
GetNetworkPrecisionF&& GetNetworkPrecision) {
GetNetworkPrecisionF&& GetNetworkPrecision,
bool isInputBlob = true) {
for (auto&& userData : userDataMap) {
auto& dims = userData.second->getTensorDesc().getDims();
const auto devicePrecision = Precision::FP32;
@ -77,7 +78,7 @@ static void AllocateImpl(const BlobDataMap& userDataMap,
case Precision::FP32 : {
userBlob = InferenceEngine::make_shared_blob<float>({userPrecision, dims, userLayout});
} break;
default: IE_THROW() << "Template Plugin: Unsupported Input/Output Precision";
default: IE_THROW(NotImplemented) << "Template Plugin: Unsupported Input/Output Precision";
}
userBlob->allocate();
userBlobMap[userData.first] = userBlob;
@ -92,12 +93,16 @@ static void AllocateImpl(const BlobDataMap& userDataMap,
deviceBlob = InferenceEngine::make_shared_blob<float>({devicePrecision, dims, deviceLayout});
}
} break;
default: IE_THROW() << "Template Plugin: Unsupported network Input/Output Presision";
default: IE_THROW(NotImplemented) << "Template Plugin: Unsupported network Input/Output Presision";
}
// preprocessing converts user input blob to desired device input blob automatically
// NOTE: this is not supported for output user blobs yet
if (userBlob != deviceBlob) {
deviceBlob->allocate();
if (isInputBlob) {
// preprocessing converts user input blob to desired device input blob automatically
deviceBlob->allocate();
} else {
// NOTE: this is not supported for output user blobs yet
IE_THROW(NotImplemented) << "Template Plugin: does not support setPrecision, setLayout for outputs";
}
}
deviceBlobMap[userData.first] = deviceBlob;
}
@ -111,7 +116,7 @@ void TemplateInferRequest::allocateBlobs() {
auto&& results = _executableNetwork->_function->get_results();
AllocateImpl(_networkOutputs, _outputs, _networkOutputBlobs, [&] (const std::string& blobName) {
return results.at(_executableNetwork->_outputIndex.at(blobName))->get_element_type();
});
}, false);
}
// ! [infer_request:infer_impl]
@ -140,7 +145,7 @@ static void blobCopy(const Blob::Ptr& src, const Blob::Ptr& dst) {
blobCopy<std::uint8_t, float>(src, dst);
} break;
default : {
IE_THROW() << "Unsupported precision conversion from "
IE_THROW(NotImplemented) << "Unsupported precision conversion from "
<< src->getTensorDesc().getPrecision() <<" to " << dst->getTensorDesc().getPrecision();
}
}
@ -152,13 +157,13 @@ static void blobCopy(const Blob::Ptr& src, const Blob::Ptr& dst) {
blobCopy<float, std::uint8_t>(src, dst);
} break;
default : {
IE_THROW() << "Unsupported precision conversion from "
IE_THROW(NotImplemented) << "Unsupported precision conversion from "
<< src->getTensorDesc().getPrecision() <<" to " << dst->getTensorDesc().getPrecision();
}
}
} break;
default : {
IE_THROW() << "Unsupported precision conversion from " << src->getTensorDesc().getPrecision();
IE_THROW(NotImplemented) << "Unsupported precision conversion from " << src->getTensorDesc().getPrecision();
}
}
}

20
docs/template_plugin/src/template_plugin.cpp
View File

@ -22,7 +22,8 @@
#include "template_plugin.hpp"
#include "template_executable_network.hpp"
#include "template_infer_request.hpp"
#include "template_pattern_transformation.hpp"
#include "transformations/template_pattern_transformation.hpp"
#include "transformations/preprocessing/preprocessing.hpp"
using namespace TemplatePlugin;
@ -52,12 +53,17 @@ Plugin::~Plugin() {
// ! [plugin:transform_network]
std::shared_ptr<ngraph::Function> TransformNetwork(const std::shared_ptr<const ngraph::Function>& function) {
std::shared_ptr<ngraph::Function> TransformNetwork(const std::shared_ptr<const ngraph::Function>& function,
const InferenceEngine::InputsDataMap & inputInfoMap,
const InferenceEngine::OutputsDataMap& outputsInfoMap) {
// 1. Copy ngraph::Function first to apply some transformations which modify original ngraph::Function
auto transformedNetwork = ngraph::clone_function(*function);
// 2. Perform common optimizations and device-specific transformations
ngraph::pass::Manager passManager;
// Example: register transformation to convert preprocessing information to graph nodes
passManager.register_pass<ngraph::pass::AddPreprocessing>(inputInfoMap);
// TODO: add post-processing based on outputsInfoMap
// Example: register CommonOptimizations transformation from transformations library
passManager.register_pass<ngraph::pass::CommonOptimizations>();
// Template plugin handles only FP32 networks
@ -81,8 +87,12 @@ InferenceEngine::ExecutableNetworkInternal::Ptr Plugin::LoadExeNetworkImpl(const
const ConfigMap &config) {
OV_ITT_SCOPED_TASK(itt::domains::TemplatePlugin, "Plugin::LoadExeNetworkImpl");
InferenceEngine::InputsDataMap networkInputs = network.getInputsInfo();
InferenceEngine::OutputsDataMap networkOutputs = network.getOutputsInfo();
auto fullConfig = Configuration{ config, _cfg };
return std::make_shared<ExecutableNetwork>(network.getFunction(), fullConfig,
return std::make_shared<ExecutableNetwork>(network.getFunction(),
networkInputs, networkOutputs, fullConfig,
std::static_pointer_cast<Plugin>(shared_from_this()));
}
// ! [plugin:load_exe_network_impl]
@ -114,7 +124,7 @@ InferenceEngine::QueryNetworkResult Plugin::QueryNetwork(const InferenceEngine::
}
// 2. It is needed to apply all transformations as it is done in LoadExeNetworkImpl
auto transformedFunction = TransformNetwork(function);
auto transformedFunction = TransformNetwork(function, network.getInputsInfo(), network.getOutputsInfo());
// 3. The same input node can be transformed into supported and unsupported backend node
// So we need store as supported either unsupported node sets
@ -246,7 +256,7 @@ InferenceEngine::Parameter Plugin::GetMetric(const std::string& name, const std:
using uint = unsigned int;
IE_SET_METRIC_RETURN(RANGE_FOR_ASYNC_INFER_REQUESTS, std::make_tuple(uint{1}, uint{1}, uint{1}));
} else {
IE_THROW() << "Unsupported device metric: " << name;
IE_THROW(NotFound) << "Unsupported device metric: " << name;
}
}
// ! [plugin:get_metric]

48
docs/template_plugin/src/transformations/preprocessing/mean_image_or_value.cpp Normal file
View File

@ -0,0 +1,48 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <ngraph/opsets/opset3.hpp>
#include <ngraph/pass/manager.hpp>
#include <ngraph/pattern/op/wrap_type.hpp>
#include "transformations/preprocessing/mean_image_or_value.hpp"
using namespace ngraph;
NGRAPH_RTTI_DEFINITION(ngraph::pass::AddMeanSubtract, "AddMeanSubtract", 0);
ngraph::pass::AddMeanSubtract::AddMeanSubtract(const MeanMap & inputInfoMap) {
// RUN_ON_FUNCTION_SCOPE(AddMeanSubtract);
auto param = ngraph::pattern::wrap_type<ngraph::opset3::Parameter>();
ngraph::matcher_pass_callback callback = [=] (pattern::Matcher& m) {
auto param = std::dynamic_pointer_cast<ngraph::opset3::Parameter>(m.get_match_root());
if (!param) {
return false;
}
auto it = inputInfoMap.find(param->get_friendly_name());
if (it == inputInfoMap.end()) {
return false;
}
auto mean_const = it->second;
NGRAPH_CHECK(mean_const->get_element_type() == ngraph::element::f32,
"Mean for ", param->get_friendly_name(), " must have f32 type");
auto copy_param = param->clone_with_new_inputs({});
auto sub = std::make_shared<ngraph::opset3::Subtract>(copy_param, mean_const);
ngraph::replace_node(param, sub);
sub->set_argument(0, param);
// Return true as the root node was changed
return true;
};
// Register pattern with Parameter operation as a pattern root node
auto m = std::make_shared<ngraph::pattern::Matcher>(param, "AddMeanSubtract");
// Register Matcher
register_matcher(m, callback);
}

33
docs/template_plugin/src/transformations/preprocessing/mean_image_or_value.hpp Normal file
View File

@ -0,0 +1,33 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <map>
#include <string>
#include <ngraph/op/constant.hpp>
#include <ngraph/pass/graph_rewrite.hpp>
#include "transformations_visibility.hpp"
namespace ngraph {
namespace pass {
class AddMeanSubtract;
} // namespace pass
} // namespace ngraph
/**
* @ingroup ie_transformation_common_api
* @brief Add `meanValue` or `meanImage` preprocessing to input nodes
*/
class ngraph::pass::AddMeanSubtract : public ngraph::pass::MatcherPass {
public:
using MeanMap = std::map<std::string, std::shared_ptr<ngraph::op::v0::Constant>>;
NGRAPH_RTTI_DECLARATION;
explicit AddMeanSubtract(const MeanMap & inputInfoMap);
};

101
docs/template_plugin/src/transformations/preprocessing/preprocessing.cpp Normal file
View File

@ -0,0 +1,101 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <ngraph/pass/manager.hpp>
#include <ngraph/opsets/opset3.hpp>
#include "transformations/preprocessing/mean_image_or_value.hpp"
#include "transformations/preprocessing/std_scale.hpp"
#include "transformations/preprocessing/preprocessing.hpp"
NGRAPH_RTTI_DEFINITION(ngraph::pass::AddPreprocessing, "AddPreprocessing", 0);
ngraph::pass::AddPreprocessing::AddPreprocessing(const InferenceEngine::InputsDataMap & inputInfoMap)
: m_inputInfoMap(inputInfoMap) { }
bool ngraph::pass::AddPreprocessing::run_on_function(std::shared_ptr<ngraph::Function> f) {
ngraph::pass::AddMeanSubtract::MeanMap meanMap;
ngraph::pass::AddStdScale::ScaleMap scaleMap;
for (const auto & it : m_inputInfoMap) {
bool has_scales = false, has_mean_values = false, has_mean_image = false;
const InferenceEngine::PreProcessInfo & pInfo = it.second->getPreProcess();
const auto & inputDims = it.second->getTensorDesc().getDims();
const size_t cn = pInfo.getNumberOfChannels();
std::vector<float> meanValues(cn), stdScales(cn);
InferenceEngine::Blob::Ptr meanImage = nullptr;
for (size_t c = 0; c < cn; ++c) {
if ((stdScales[c] = pInfo[c]->stdScale) != 1.0f) {
has_scales = true;
}
if ((meanValues[c] = pInfo[c]->meanValue) != 0.0f) {
has_mean_values = true;
}
if (pInfo[c]->meanData != nullptr) {
has_mean_image = true;
if (c == 0) {
meanImage = pInfo[c]->meanData;
NGRAPH_CHECK(meanImage->getTensorDesc().getPrecision() == InferenceEngine::Precision::FP32,
"Only InferenceEngine::Precision::FP32 precision is supported for PreProcessChannel::meanData");
} else {
NGRAPH_CHECK(meanImage->getTensorDesc() == pInfo[c]->meanData->getTensorDesc(),
"TensorDesc for PreProcessChannel::meanData must be equal");
}
}
}
// no preprocessing for current input
if (!has_mean_values && !has_scales && !has_mean_image) {
continue;
}
NGRAPH_CHECK(!(has_mean_image && has_scales),
"Only PreProcessChannel::meanData or PreProcessChannel::meanValue can be set.");
if (has_scales) {
ngraph::Shape shape(inputDims.size(), 1);
shape[1] = stdScales.size(); // C
scaleMap[it.first] = ngraph::opset3::Constant::create(ngraph::element::f32, shape, stdScales);
}
if (has_mean_values) {
ngraph::Shape shape(inputDims.size(), 1);
shape[1] = meanValues.size(); // C
meanMap[it.first] = ngraph::opset3::Constant::create(ngraph::element::f32, shape, meanValues);
} else if (has_mean_image) {
ngraph::Shape shape = { cn };
auto dims = meanImage->getTensorDesc().getDims();
std::copy(dims.begin(), dims.end(), std::back_inserter(shape));
std::vector<float> meanImageData(ngraph::shape_size(shape));
for (size_t c = 0, i = 0; c < cn; ++c) {
auto lm = pInfo[c]->meanData->buffer();
const float *data = lm.as<const float *>();
std::memcpy(&meanImageData[i], data, meanImage->byteSize());
i += meanImage->size();
}
meanMap[it.first] = ngraph::opset3::Constant::create(ngraph::element::f32,
shape, meanImageData);
}
}
ngraph::pass::Manager manager(get_pass_config());
auto preproc = manager.register_pass<ngraph::pass::GraphRewrite>();
if (!scaleMap.empty()) {
preproc->add_matcher<ngraph::pass::AddStdScale>(scaleMap);
}
if (!meanMap.empty()) {
preproc->add_matcher<ngraph::pass::AddMeanSubtract>(meanMap);
}
manager.run_passes(f);
return false;
}

35
docs/template_plugin/src/transformations/preprocessing/preprocessing.hpp Normal file
View File

@ -0,0 +1,35 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <ngraph/pass/pass.hpp>
#include "ie_input_info.hpp"
namespace ngraph {
namespace pass {
class AddPreprocessing;
} // namespace pass
} // namespace ngraph
/**
* @brief Converts the following preprocessing information to ngraph operations:
* - InferenceEngine::PreProcessInfo->PreProcessChannel::meanData -> Subtract
* - InferenceEngine::PreProcessInfo->PreProcessChannel::meanValue -> Subtract
* - InferenceEngine::PreProcessInfo->PreProcessChannel::stdScale -> Multiply
*
* The order of operations is the following:
* (x - mean) * stdScale
*/
class ngraph::pass::AddPreprocessing : public ngraph::pass::FunctionPass {
const InferenceEngine::InputsDataMap & m_inputInfoMap;
public:
NGRAPH_RTTI_DECLARATION;
explicit AddPreprocessing(const InferenceEngine::InputsDataMap & inputInfoMap);
bool run_on_function(std::shared_ptr<ngraph::Function> f) override;
};

48
docs/template_plugin/src/transformations/preprocessing/std_scale.cpp Normal file
View File

@ -0,0 +1,48 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#include <ngraph/opsets/opset3.hpp>
#include <ngraph/pass/manager.hpp>
#include <ngraph/pattern/op/wrap_type.hpp>
#include "transformations/preprocessing/std_scale.hpp"
using namespace ngraph;
NGRAPH_RTTI_DEFINITION(ngraph::pass::AddStdScale, "AddStdScale", 0);
ngraph::pass::AddStdScale::AddStdScale(const ScaleMap& inputInfoMap) {
// RUN_ON_FUNCTION_SCOPE(AddStdScale);
auto param = ngraph::pattern::wrap_type<ngraph::opset3::Parameter>();
ngraph::matcher_pass_callback callback = [=] (pattern::Matcher& m) {
auto param = std::dynamic_pointer_cast<ngraph::opset3::Parameter>(m.get_match_root());
if (!param) {
return false;
}
auto it = inputInfoMap.find(param->get_friendly_name());
if (it == inputInfoMap.end()) {
return false;
}
auto scale_const = it->second;
NGRAPH_CHECK(scale_const->get_element_type() == ngraph::element::f32,
"Scale for ", param->get_friendly_name(), " must have f32 type");
auto copy_param = param->clone_with_new_inputs({});
auto mul = std::make_shared<ngraph::opset3::Multiply>(copy_param, it->second);
ngraph::replace_node(param, mul);
mul->set_argument(0, param);
// Return true as the root node was changed
return true;
};
// Register pattern with Parameter operation as a pattern root node
auto m = std::make_shared<ngraph::pattern::Matcher>(param, "AddStdScale");
// Register Matcher
register_matcher(m, callback);
}

33
docs/template_plugin/src/transformations/preprocessing/std_scale.hpp Normal file
View File

@ -0,0 +1,33 @@
// Copyright (C) 2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <map>
#include <string>
#include <ngraph/op/constant.hpp>
#include <ngraph/pass/graph_rewrite.hpp>
#include "transformations_visibility.hpp"
namespace ngraph {
namespace pass {
class AddStdScale;
} // namespace pass
} // namespace ngraph
/**
* @ingroup ie_transformation_common_api
* @brief Add `stdScale` preprocessing to input nodes
*/
class ngraph::pass::AddStdScale : public ngraph::pass::MatcherPass {
public:
using ScaleMap = std::map<std::string, std::shared_ptr<ngraph::op::v0::Constant>>;
NGRAPH_RTTI_DECLARATION;
explicit AddStdScale(const ScaleMap& inputInfoMap);
};

0
docs/template_plugin/src/template_function_transformation.cpp → docs/template_plugin/src/transformations/template_function_transformation.cpp
View File

0
docs/template_plugin/src/template_function_transformation.hpp → docs/template_plugin/src/transformations/template_function_transformation.hpp
View File

4
docs/template_plugin/src/template_pattern_transformation.cpp → docs/template_plugin/src/transformations/template_pattern_transformation.cpp
View File

@ -2,8 +2,8 @@
// SPDX-License-Identifier: Apache-2.0
//
#include "template_pattern_transformation.hpp"
#include "template_function_transformation.hpp"
#include "transformations/template_pattern_transformation.hpp"
#include "transformations/template_function_transformation.hpp"
#include <ngraph/opsets/opset3.hpp>
#include <ngraph/pass/manager.hpp>

2
docs/template_plugin/src/template_pattern_transformation.hpp → docs/template_plugin/src/transformations/template_pattern_transformation.hpp
View File

@ -16,7 +16,7 @@ class ReluReluFusionMatcher;
} // namespace ngraph
// ! [graph_rewrite:template_transformation_hpp]
// template_pattern_transformation.hpp
// transformations/template_pattern_transformation.hpp
/**
* @ingroup ie_transformation_common_api
* @brief Add transformation description.

4
docs/template_plugin/tests/functional/shared_tests_instances/behavior/preprocessing.cpp
View File

@ -19,7 +19,7 @@ const std::vector<std::map<std::string, std::string>> configs = {
{}
};
INSTANTIATE_TEST_CASE_P(PreprocessingPrecisionConvertTestsViaSetInput, PreprocessingPrecisionConvertTest,
INSTANTIATE_TEST_CASE_P(smoke_PreprocessingPrecisionConvertTestsViaSetInput, PreprocessingPrecisionConvertTest,
::testing::Combine(
::testing::ValuesIn(inputPrecisions),
::testing::Values(4), // Number of input tensor channels
@ -28,7 +28,7 @@ INSTANTIATE_TEST_CASE_P(PreprocessingPrecisionConvertTestsViaSetInput, Preproces
::testing::ValuesIn(configs)),
PreprocessingPrecisionConvertTest::getTestCaseName);
INSTANTIATE_TEST_CASE_P(PreprocessingPrecisionConvertTestsViaGetBlob, PreprocessingPrecisionConvertTest,
INSTANTIATE_TEST_CASE_P(smoke_PreprocessingPrecisionConvertTestsViaGetBlob, PreprocessingPrecisionConvertTest,
::testing::Combine(
::testing::ValuesIn(inputPrecisions),
::testing::Values(4), // Number of input tensor channels (blob_copy only supports 4d and 5d tensors)

23
docs/template_plugin/tests/functional/shared_tests_instances/behavior/set_preprocess.cpp
View File

@ -19,6 +19,15 @@ const std::vector<std::map<std::string, std::string>> configs = {
{}
};
const std::vector<std::map<std::string, std::string>> multiConfigs = {
{{ InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES,
CommonTestUtils::DEVICE_TEMPLATE }}
};
const std::vector<std::map<std::string, std::string>> heteroConfigs = {
{{ "TARGET_FALLBACK", CommonTestUtils::DEVICE_TEMPLATE }}
};
INSTANTIATE_TEST_CASE_P(smoke_BehaviorTests, PreprocessTest,
::testing::Combine(
::testing::ValuesIn(netPrecisions),
@ -26,6 +35,20 @@ INSTANTIATE_TEST_CASE_P(smoke_BehaviorTests, PreprocessTest,
::testing::ValuesIn(configs)),
PreprocessTest::getTestCaseName);
INSTANTIATE_TEST_CASE_P(smoke_Multi_BehaviorTests, PreprocessTest,
::testing::Combine(
::testing::ValuesIn(netPrecisions),
::testing::Values(CommonTestUtils::DEVICE_MULTI),
::testing::ValuesIn(multiConfigs)),
PreprocessTest::getTestCaseName);
INSTANTIATE_TEST_CASE_P(smoke_Hetero_BehaviorTests, PreprocessTest,
::testing::Combine(
::testing::ValuesIn(netPrecisions),
::testing::Values(CommonTestUtils::DEVICE_HETERO),
::testing::ValuesIn(heteroConfigs)),
PreprocessTest::getTestCaseName);
const std::vector<InferenceEngine::Precision> ioPrecisions = {
InferenceEngine::Precision::FP32,
InferenceEngine::Precision::U8

6
docs/template_plugin/tests/functional/skip_tests_config.cpp
View File

@ -12,10 +12,10 @@ std::vector<std::string> disabledTestPatterns() {
".*ExclusiveAsyncRequests.*",
".*reusableCPUStreamsExecutor.*",
R"(.*SplitLayerTest.*numSplits\=30.*)",
// CVS-44774
".*PreprocessTest.*",
// CVS-51758
".*PreprocessConversionTest.*oPRC=U8.*",
".*PreprocessConversionTest.*oLT=NHWC.*"
".*PreprocessConversionTest.*oLT=NHWC.*",
".*PreprocessingPrecisionConvertTestsViaSetInput.*SetInput.*",
".*PreprocessingPrecisionConvertTestsViaGetBlob.*GetBlob.*",
};
}

183
docs/template_plugin/tests/functional/transformations/preprocessing.cpp Normal file
View File

@ -0,0 +1,183 @@
// // Copyright (C) 2021 Intel Corporation
// // SPDX-License-Identifier: Apache-2.0
// //
// #include <gtest/gtest.h>
// #include <string>
// #include <memory>
// #include <map>
// #include <ngraph/function.hpp>
// #include <ngraph/opsets/opset5.hpp>
// #include <ngraph/pass/manager.hpp>
// #include <transformations/init_node_info.hpp>
// #include <transformations/preprocessing/std_scale.hpp>
// #include <transformations/preprocessing/mean_image_or_value.hpp>
// #include "common_test_utils/ngraph_test_utils.hpp"
// using namespace testing;
// using namespace ngraph;
// TEST(TransformationTests, Preprocessing_AddStdScale) {
// std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
// const Shape data_shape{1, 3, 14, 14};
// const Shape scale_shape{3, 1, 1};
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto relu = std::make_shared<opset5::Relu>(data);
// f = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// auto scales = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// pass::Manager m;
// m.register_pass<pass::InitNodeInfo>();
// m.register_pass<pass::AddStdScale>(pass::AddStdScale::ScaleMap{ { data->get_friendly_name(), scales } });
// m.run_passes(f);
// }
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto scales = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// auto mul = std::make_shared<opset5::Multiply>(data, scales);
// auto relu = std::make_shared<opset5::Relu>(mul);
// f_ref = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// }
// auto res = compare_functions(f, f_ref);
// ASSERT_TRUE(res.first) << res.second;
// }
// TEST(TransformationTests, Preprocessing_AddMeanValue) {
// std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
// const Shape data_shape{1, 3, 14, 14};
// const Shape mean_shape{3, 1, 1};
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto relu = std::make_shared<opset5::Relu>(data);
// f = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// pass::Manager m;
// m.register_pass<pass::InitNodeInfo>();
// m.register_pass<pass::AddMeanSubtract>(pass::AddMeanSubtract::MeanMap{ { data->get_friendly_name(), meanValues } });
// m.run_passes(f);
// }
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto sub = std::make_shared<opset5::Subtract>(data, meanValues);
// auto relu = std::make_shared<opset5::Relu>(sub);
// f_ref = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// }
// auto res = compare_functions(f, f_ref);
// ASSERT_TRUE(res.first) << res.second;
// }
// TEST(TransformationTests, Preprocessing_AddMeanImage) {
// std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
// const Shape data_shape{1, 3, 14, 14};
// const Shape mean_shape{3, 14, 14};
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto relu = std::make_shared<opset5::Relu>(data);
// f = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// pass::Manager m;
// m.register_pass<pass::InitNodeInfo>();
// m.register_pass<pass::AddMeanSubtract>(pass::AddMeanSubtract::MeanMap{ { data->get_friendly_name(), meanValues } });
// m.run_passes(f);
// }
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto sub = std::make_shared<opset5::Subtract>(data, meanValues);
// auto relu = std::make_shared<opset5::Relu>(sub);
// f_ref = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// }
// auto res = compare_functions(f, f_ref);
// ASSERT_TRUE(res.first) << res.second;
// }
// TEST(TransformationTests, Preprocessing_AddMeanImageAndScale) {
// std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
// const Shape data_shape{1, 3, 14, 14};
// const Shape mean_shape{3, 14, 14};
// const Shape scale_shape{3, 1, 1};
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto relu = std::make_shared<opset5::Relu>(data);
// f = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto scaleValues = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// pass::Manager m;
// m.register_pass<pass::InitNodeInfo>();
// m.register_pass<pass::AddStdScale>(pass::AddStdScale::ScaleMap{ { data->get_friendly_name(), scaleValues } });
// m.register_pass<pass::AddMeanSubtract>(pass::AddMeanSubtract::MeanMap{ { data->get_friendly_name(), meanValues } });
// m.run_passes(f);
// }
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto scaleValues = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// auto sub = std::make_shared<opset5::Subtract>(data, meanValues);
// auto mul = std::make_shared<opset5::Multiply>(sub, scaleValues);
// auto relu = std::make_shared<opset5::Relu>(mul);
// f_ref = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// }
// auto res = compare_functions(f, f_ref);
// ASSERT_TRUE(res.first) << res.second;
// }
// TEST(TransformationTests, Preprocessing_AddMeanValueAndScale) {
// std::shared_ptr<Function> f(nullptr), f_ref(nullptr);
// const Shape data_shape{1, 3, 14, 14};
// const Shape mean_shape{3, 1, 1};
// const Shape scale_shape{3, 1, 1};
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto relu = std::make_shared<opset5::Relu>(data);
// f = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto scaleValues = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// pass::Manager m;
// m.register_pass<pass::InitNodeInfo>();
// m.register_pass<pass::AddStdScale>(pass::AddStdScale::ScaleMap{ { data->get_friendly_name(), scaleValues } });
// m.register_pass<pass::AddMeanSubtract>(pass::AddMeanSubtract::MeanMap{ { data->get_friendly_name(), meanValues } });
// m.run_passes(f);
// }
// {
// auto data = std::make_shared<opset5::Parameter>(element::f32, data_shape);
// auto meanValues = opset5::Constant::create(element::f32, mean_shape,
// std::vector<float>(shape_size(mean_shape), 2.0f));
// auto scaleValues = opset5::Constant::create(element::f32, scale_shape,
// std::vector<float>(shape_size(scale_shape), 2.0f));
// auto sub = std::make_shared<opset5::Subtract>(data, meanValues);
// auto mul = std::make_shared<opset5::Multiply>(sub, meanValues);
// auto relu = std::make_shared<opset5::Relu>(mul);
// f_ref = std::make_shared<Function>(NodeVector{relu}, ParameterVector{data});
// }
// auto res = compare_functions(f, f_ref);
// ASSERT_TRUE(res.first) << res.second;
// }

3
inference-engine/src/plugin_api/precision_utils.h
View File

@ -28,6 +28,9 @@ namespace InferenceEngine {
* @{
* @defgroup ie_dev_api_plugin_api Plugin base classes
* @brief A set of base and helper classes to implement a plugin class
*
* @defgroup ie_dev_api_preproc_api Preprocessing API
* @brief A set transformations to convert InferenceEngine::PreProcessInfo to ngraph operations
*
* @defgroup ie_dev_api_exec_network_api Executable Network base classes
* @brief A set of base and helper classes to implement an executable network class

11
inference-engine/tests/functional/plugin/shared/include/behavior/set_preprocess.hpp
View File

@ -124,8 +124,9 @@ TEST_P(PreprocessTest, SetMeanImagePreProcessGetBlob) {
auto outMem = outBlob->cbuffer();
const auto* outData = outMem.as<const float*>();
ASSERT_EQ(inBlob->size(), outBlob->size());
for (size_t i = 0; i < inBlob->size(); i++)
for (size_t i = 0; i < inBlob->size(); i++) {
ASSERT_EQ(inData[i] + inData[i], outData[i]);
}
}
}
@ -255,8 +256,9 @@ TEST_P(PreprocessTest, SetMeanValuePreProcessGetBlob) {
auto outMem = outBlob->cbuffer();
const auto* outData = outMem.as<const float*>();
ASSERT_EQ(inBlob->size(), outBlob->size());
for (size_t i = 0; i < inBlob->size(); i++)
ASSERT_EQ(inData[i]+5, outData[i]);
for (size_t i = 0; i < inBlob->size(); i++) {
ASSERT_EQ(inData[i] + 5, outData[i]);
}
}
}
@ -511,8 +513,9 @@ TEST_P(PreprocessTest, SetScalePreProcessGetBlob) {
auto outMem = outBlob->cbuffer();
const auto* outData = outMem.as<const float*>();
ASSERT_EQ(inBlob->size(), outBlob->size());
for (size_t i = 0; i < inBlob->size(); i++)
for (size_t i = 0; i < inBlob->size(); i++) {
ASSERT_EQ(inData[i]*2, outData[i]);
}
}
}