diff --git a/.ci/azure/linux.yml b/.ci/azure/linux.yml index 22673819b1c..4ed0a79a285 100644 --- a/.ci/azure/linux.yml +++ b/.ci/azure/linux.yml @@ -88,7 +88,7 @@ jobs: rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR) sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR) sudo mkdir -p $(SHARE_DIR) - sudo apt --assume-yes install nfs-common + sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys mkdir -p $(CCACHE_DIR) displayName: 'Make dir' diff --git a/docs/nGraph_DG/nGraphTransformation.md b/docs/nGraph_DG/nGraphTransformation.md index 03777180ad8..524dbf59c6e 100644 --- a/docs/nGraph_DG/nGraphTransformation.md +++ b/docs/nGraph_DG/nGraphTransformation.md @@ -47,7 +47,7 @@ For examples of how to build an nGraph function, see the [Build nGraph Function] ## Transformations types -nGraph has three main transformation types: +nGraph has three main transformation types: * `ngraph::pass::FunctionPass` - straightforward way to work with `ngraph::Function` directly * `ngraph::pass::MatcherPass` - pattern-based transformation approach @@ -81,7 +81,7 @@ Template for MatcherPass transformation class To use `ngraph::pass::MatcherPass`, you need to complete these steps: 1. Create a pattern -2. Implement a callback +2. Implement a callback 3. Register the pattern and Matcher 4. Execute MatcherPass @@ -90,7 +90,7 @@ So let's go through each of these steps. ### Create a pattern Pattern is a single root `ngraph::Function`. But the only difference is that you do not need to create a function object, you just need to create and connect opset or special pattern operations. Then you need to take the last created operation and put it as a root of the pattern. This root node will be used as a root node in pattern matching. -> **NOTE**: Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching. +> **NOTE**: Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching. @snippet example_ngraph_utils.cpp pattern:simple_example @@ -105,7 +105,7 @@ Callback is an action applied to every pattern entrance. In general, callback is The example above shows the callback structure and how Matcher can be used for accessing nodes detected by pattern. Callback return value is `true` if root node was replaced and another pattern cannot be applied to the same root node; otherwise, it is `false`. -> **NOTE**: It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching. +> **NOTE**: It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching. MatcherPass also provides functionality that allows reporting of the newly created nodes that can be used in additional pattern matching. If MatcherPass was registered in `pass::Manager` or `pass::GraphRewrite`, these registered nodes will be added for additional pattern matching. @@ -144,7 +144,7 @@ Example: In addition, GraphRewrite handles nodes that were registered by MatcherPasses during their execution. This nodes will be added to the beginning of the sequence with nodes for pattern matching. -> **NOTE**: when using `pass::Manager` temporary GraphRewrite is used to execute single MatcherPass. +> **NOTE**: when using `pass::Manager` temporary GraphRewrite is used to execute single MatcherPass. GraphRewrite has two algorithms for MatcherPasses execution. First algorithm is straightforward. It applies each MatcherPass in registration order to current node. @@ -153,7 +153,7 @@ GraphRewrite has two algorithms for MatcherPasses execution. First algorithm is But it is not really efficient when you have a lot of registered passes. So first of all GraphRewrite checks that all MatcherPass patterns has type-based root node (it means that type of this node is not hidden into predicate). And then creates map from registered MatcherPasses. That helps to avoid additional cost of applying each MatcherPass for each node. -![graph_rewrite_efficient_search] +![graph_rewrite_efficient_search] > **NOTE**: GraphRewrite execution algorithm cannot be set manually and depends only on root nodes registered inside MatcherPasses. @@ -161,7 +161,7 @@ And then creates map from registered MatcherPasses. That helps to avoid addition Sometimes patterns cannot be expressed via regular nGraph operations or it is too complicated. For example, if you want to detect Convolution->Add sub-graph without specifying particular input type for Convolution operation or you want to create a pattern where some of operations can have different types. -And for these cases nGraph provides additional helpers to construct patterns for GraphRewrite transformations. +And for these cases nGraph provides additional helpers to construct patterns for GraphRewrite transformations. There are two main helpers: 1. `ngraph::pattern::any_input` - helps to express inputs if their types are undefined. @@ -172,7 +172,7 @@ Let's go through the example to have better understanding of how it works: > **NOTE**: Node attributes do not participate in pattern matching and are needed only for operations creation. Only operation types participate in pattern matching. The example below shows basic usage of `pattern::any_input`. -Here we construct Multiply pattern with arbitrary first input and Constant as a second input. +Here we construct Multiply pattern with arbitrary first input and Constant as a second input. Also as Multiply is commutative operation, it does not matter in which order we set inputs (any_input/Constant or Constant/any_input) because both cases will be matched. @snippet example_ngraph_utils.cpp pattern:label_example @@ -196,7 +196,7 @@ In this chapter we will review nGraph API that allows us to manipulate with `ngr First of all let's talk about `ngraph::Node` input/output ports. Each nGraph operation has input and output ports except cases when operation has `Result`, `Parameter`, or `Constant` type. Every port belongs to its node, so using a port we can access parent node, get shape and type for particular input/output, get all consumers in case of output port, and get producer node in case of input port. -With output port we can set inputs for newly created operations. +With output port we can set inputs for newly created operations. Lets look at the code example. @@ -208,8 +208,8 @@ std::shared_ptr neg_const = opset1::Constant::create(sub->get_input_elemen Output data = node->input_value(0); auto neg = std::make_shared(data, neg_const); ``` -In this example, the `opset3::Multiply` operation takes `Output` and `std::shared_ptr` as inputs. But the constructor takes both as `Output`. -In this case, `std::shared_ptr` will be automatically converted to `Output` if node has exactly one output port; otherwise, conversion raises an exception. +In this example, the `opset3::Multiply` operation takes `Output` and `std::shared_ptr` as inputs. But the constructor takes both as `Output`. +In this case, `std::shared_ptr` will be automatically converted to `Output` if node has exactly one output port; otherwise, conversion raises an exception. ### ngraph::Node replacement @@ -251,9 +251,9 @@ To eliminate operation, nGraph has special method that considers all limitations @snippet example_ngraph_utils.cpp ngraph:eliminate_node `replace_output_update_name` in case of successful replacement it automatically preserves friendly name and runtime info. - -## Transformation conditional compilation + +## Transformation conditional compilation Transformation library has two internal macros to support conditional compilation feature. @@ -272,14 +272,14 @@ Use the latest version of OpSet in your transformation. An exception is op_conve ###2. Dynamic Shape and Rank -nGraph has two types for shape representation: +nGraph has two types for shape representation: `ngraph::Shape` - represents static shape. `ngraph::PartialShape` - represents dynamic shape. It means that rank or some of dimensions are dynamic (undefined). `ngraph::PartialShape` can be converted to `ngraph::Shape` using the `get_shape()` method if all dimensions are static; otherwise, conversion raises an exception. @snippet example_ngraph_utils.cpp ngraph:shape -But in most cases before getting static shape using `get_shape()` method, you need to check that shape is static. +But in most cases before getting static shape using `get_shape()` method, you need to check that shape is static. Also if your transformation requires only input shape rank or particular dimension value, please do not use the `get_shape()` method. See the example below demonstrating how to avoid using `get_shape()` @@ -289,7 +289,7 @@ Not using `get_shape()` method makes your transformation more flexible and appli ###3. Friendly Names -Each `ngraph::Node` has a unique name (used for nGraph internals) and a friendly name. In transformations we care only about friendly name because it represents the name from intermediate representation (IR). +Each `ngraph::Node` has a unique name (used for nGraph internals) and a friendly name. In transformations we care only about friendly name because it represents the name from intermediate representation (IR). Also friendly name is used as output tensor name (until we do not have other way to represent output tensor name) and user code that requests intermediate outputs based on these names. To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below. @@ -306,7 +306,7 @@ In more advanced cases, when replaced operation has several outputs and we add a ###4. Runtime Info -Runtime info is a map `std::map>` located inside `ngraph::Node` class. It represents additional attributes in `ngraph::Node`. +Runtime info is a map `std::map` located inside `ngraph::Node` class. It represents additional attributes in `ngraph::Node`. These attributes can be set by users or by plugins and when executing transformation that changes `ngraph::Function` we need to preserve these attributes as they will not be automatically propagated. In most cases, transformations have the following types: 1:1 (replace node with another node), 1:N (replace node with a sub-graph), N:1 (fuse sub-graph into a single node), N:M (any other transformation). Currently, there is no mechanism that automatically detects transformation types, so we need to propagate this runtime information manually. See the examples below. @@ -331,7 +331,7 @@ ngraph::copy_runtime_info({conv, bias}, {conv_ie}); ngraph::copy_runtime_info({a, b, c}, {e, f}); ``` -When transformation has multiple fusions or decompositions, `ngraph::copy_runtime_info` must be called multiple times for each case. +When transformation has multiple fusions or decompositions, `ngraph::copy_runtime_info` must be called multiple times for each case. > **Note**: copy_runtime_info removes rt_info from destination nodes. If you want to keep it, you need to specify them in source nodes like this: copy_runtime_info({a, b, c}, {a, b}) @@ -341,12 +341,12 @@ If your transformation inserts constant sub-graphs that need to be folded, do no The example below shows how constant subgraph can be constructed. ```cpp -// After ConstantFolding pass Power will be replaced with Constant +// After ConstantFolding pass Power will be replaced with Constant auto pow = std::make_shared( opset3::Constant::create(element::f32, Shape{1}, {2}) opset3::Constant::create(element::f32, Shape{1}, {3})); auto mul = std::make_shared(input /* not constant input */, pow); -``` +``` Manual constant folding is more preferable than `ngraph::pass::ConstantFolding()` because it is much faster. @@ -358,18 +358,18 @@ Below you can find an example of manual constant folding: In transformation development process: -* Do not use deprecated nGraph API. Deprecated methods has the `NGRAPH_DEPRECATED` macros in its definition. +* Do not use deprecated nGraph API. Deprecated methods has the `NGRAPH_DEPRECATED` macros in its definition. * Do not pass `shared_ptr` as an input for other node if type of node is unknown or it has multiple outputs. Use explicit output port. * If you replace node with another node that produces different shape, remember that new shape will not be propagated until the first `validate_nodes_and_infer_types` call for `ngraph::Function`. If you are using `pass::Manager`, it will automatically call this method after each transformation execution. * Do not forget to call the `ngraph::ConstantFolding` pass if your transformation creates constant subgraphs. * Use latest OpSet if you are not developing downgrade transformation pass. -* When developing a callback for `ngraph::pass::MatcherPass`, do not change nodes that come after the root node in topological order. +* When developing a callback for `ngraph::pass::MatcherPass`, do not change nodes that come after the root node in topological order. ## Using pass manager `ngraph::pass::Manager` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations. It can register and apply any [transformation types](#transformations_types) on function. -In addition, `ngraph::pass::Manager` has extended debug capabilities (find more information in the [how to debug transformations](#how_to_debug_transformations) section). +In addition, `ngraph::pass::Manager` has extended debug capabilities (find more information in the [how to debug transformations](#how_to_debug_transformations) section). The example below shows basic usage of `ngraph::pass::Manager` diff --git a/docs/snippets/InferenceEngine_Caching3.cpp b/docs/snippets/InferenceEngine_Caching3.cpp index 282d07b1dc9..db6cd89e5c6 100644 --- a/docs/snippets/InferenceEngine_Caching3.cpp +++ b/docs/snippets/InferenceEngine_Caching3.cpp @@ -14,7 +14,7 @@ using namespace InferenceEngine; auto it = std::find(keys.begin(), keys.end(), METRIC_KEY(IMPORT_EXPORT_SUPPORT)); // If metric 'IMPORT_EXPORT_SUPPORT' exists, check it's value - bool cachingSupported = (it != keys.end()) && ie.GetMetric(deviceName, METRIC_KEY(IMPORT_EXPORT_SUPPORT)).as(); + auto cachingSupported = (it != keys.end()) && ie.GetMetric(deviceName, METRIC_KEY(IMPORT_EXPORT_SUPPORT)).as(); //! [part3] return 0; } diff --git a/docs/template_plugin/tests/functional/op_reference/detection_output.cpp b/docs/template_plugin/tests/functional/op_reference/detection_output.cpp index fa05c887089..cc0a6a2a7f4 100644 --- a/docs/template_plugin/tests/functional/op_reference/detection_output.cpp +++ b/docs/template_plugin/tests/functional/op_reference/detection_output.cpp @@ -45,21 +45,21 @@ struct DetectionOutputParams { refData(CreateTensor(iType, oValues)), testcaseName(test_name) { attrs.num_classes = num_classes; - attrs.background_label_id = background_label_id; - attrs.top_k = top_k; - attrs.variance_encoded_in_target = variance_encoded_in_target; - attrs.keep_top_k = keep_top_k; - attrs.code_type = code_type; - attrs.share_location = share_location; - attrs.nms_threshold = nms_threshold; - attrs.confidence_threshold = confidence_threshold; - attrs.clip_after_nms = clip_after_nms; - attrs.clip_before_nms = clip_before_nms; - attrs.decrease_label_id = decrease_label_id; - attrs.normalized = normalized; - attrs.input_height = input_height; - attrs.input_width = input_width; - attrs.objectness_score = objectness_score; + attrs_v8.background_label_id = attrs.background_label_id = background_label_id; + attrs_v8.top_k = attrs.top_k = top_k; + attrs_v8.variance_encoded_in_target = attrs.variance_encoded_in_target = variance_encoded_in_target; + attrs_v8.keep_top_k = attrs.keep_top_k = keep_top_k; + attrs_v8.code_type = attrs.code_type = code_type; + attrs_v8.share_location = attrs.share_location = share_location; + attrs_v8.nms_threshold = attrs.nms_threshold = nms_threshold; + attrs_v8.confidence_threshold = attrs.confidence_threshold = confidence_threshold; + attrs_v8.clip_after_nms = attrs.clip_after_nms = clip_after_nms; + attrs_v8.clip_before_nms = attrs.clip_before_nms = clip_before_nms; + attrs_v8.decrease_label_id = attrs.decrease_label_id = decrease_label_id; + attrs_v8.normalized = attrs.normalized = normalized; + attrs_v8.input_height = attrs.input_height = input_height; + attrs_v8.input_width = attrs.input_width = input_width; + attrs_v8.objectness_score = attrs.objectness_score = objectness_score; size_t num_loc_classes = attrs.share_location ? 1 : attrs.num_classes; size_t prior_box_size = attrs.normalized ? 4 : 5; @@ -107,21 +107,21 @@ template auxConfData(CreateTensor(iType, auxConfValues)), testcaseName(test_name) { attrs.num_classes = num_classes; - attrs.background_label_id = background_label_id; - attrs.top_k = top_k; - attrs.variance_encoded_in_target = variance_encoded_in_target; - attrs.keep_top_k = keep_top_k; - attrs.code_type = code_type; - attrs.share_location = share_location; - attrs.nms_threshold = nms_threshold; - attrs.confidence_threshold = confidence_threshold; - attrs.clip_after_nms = clip_after_nms; - attrs.clip_before_nms = clip_before_nms; - attrs.decrease_label_id = decrease_label_id; - attrs.normalized = normalized; - attrs.input_height = input_height; - attrs.input_width = input_width; - attrs.objectness_score = objectness_score; + attrs_v8.background_label_id = attrs.background_label_id = background_label_id; + attrs_v8.top_k = attrs.top_k = top_k; + attrs_v8.variance_encoded_in_target = attrs.variance_encoded_in_target = variance_encoded_in_target; + attrs_v8.keep_top_k = attrs.keep_top_k = keep_top_k; + attrs_v8.code_type = attrs.code_type = code_type; + attrs_v8.share_location = attrs.share_location = share_location; + attrs_v8.nms_threshold = attrs.nms_threshold = nms_threshold; + attrs_v8.confidence_threshold = attrs.confidence_threshold = confidence_threshold; + attrs_v8.clip_after_nms = attrs.clip_after_nms = clip_after_nms; + attrs_v8.clip_before_nms = attrs.clip_before_nms = clip_before_nms; + attrs_v8.decrease_label_id = attrs.decrease_label_id = decrease_label_id; + attrs_v8.normalized = attrs.normalized = normalized; + attrs_v8.input_height = attrs.input_height = input_height; + attrs_v8.input_width = attrs.input_width = input_width; + attrs_v8.objectness_score = attrs.objectness_score = objectness_score; size_t num_loc_classes = attrs.share_location ? 1 : attrs.num_classes; size_t prior_box_size = attrs.normalized ? 4 : 5; @@ -135,6 +135,7 @@ template } ov::op::v0::DetectionOutput::Attributes attrs; + ov::op::v8::DetectionOutput::Attributes attrs_v8; ov::PartialShape locShape; ov::PartialShape confShape; ov::PartialShape priorBoxesShape; @@ -194,10 +195,61 @@ private: } }; +class ReferenceDetectionOutputV8LayerTest : public testing::TestWithParam, + public CommonReferenceTest { +public: + void SetUp() override { + auto params = GetParam(); + function = CreateFunction(params); + if ((params.auxLocShape.size() != 0) && (params.auxConfShape.size() != 0)) + inputData = {params.locData, params.confData, params.priorBoxesData, params.auxConfData, params.auxLocData}; + else + inputData = {params.locData, params.confData, params.priorBoxesData}; + refOutData = {params.refData}; + } + static std::string getTestCaseName(const testing::TestParamInfo& obj) { + auto param = obj.param; + std::ostringstream result; + result << "locShape=" << param.locShape << "_"; + result << "confShape=" << param.confShape << "_"; + result << "priorBoxesShape=" << param.priorBoxesShape << "_"; + if ((param.auxLocShape.size() != 0) && (param.auxConfShape.size() != 0)) { + result << "auxLocShape=" << param.locShape << "_"; + result << "auxConfShape=" << param.confShape << "_"; + } + result << "iType=" << param.inType; + if (param.testcaseName != "") + result << "_" << param.testcaseName; + return result.str(); + } + +private: + static std::shared_ptr CreateFunction(const DetectionOutputParams& params) { + const auto loc = std::make_shared(params.inType, params.locShape); + const auto conf = std::make_shared(params.inType, params.confShape); + const auto priorBoxes = std::make_shared(params.inType, params.priorBoxesShape); + if ((params.auxLocShape.size() != 0) && (params.auxConfShape.size() != 0)) { + const auto auxConf = std::make_shared(params.inType, params.auxConfShape); + const auto auxLoc = std::make_shared(params.inType, params.auxLocShape); + const auto DetectionOutput = + std::make_shared(loc, conf, priorBoxes, auxConf, auxLoc, params.attrs_v8); + return std::make_shared(NodeVector{DetectionOutput}, + ParameterVector{loc, conf, priorBoxes, auxConf, auxLoc}); + } else { + const auto DetectionOutput = std::make_shared(loc, conf, priorBoxes, params.attrs_v8); + return std::make_shared(NodeVector{DetectionOutput}, ParameterVector{loc, conf, priorBoxes}); + } + } +}; + TEST_P(ReferenceDetectionOutputLayerTest, CompareWithRefs) { Exec(); } +TEST_P(ReferenceDetectionOutputV8LayerTest, CompareWithRefs) { + Exec(); +} + template std::vector generateDetectionOutputFloatParams() { using T = typename element_type_traits::value_type; @@ -517,4 +569,9 @@ std::vector generateDetectionOutputCombinedParams() { INSTANTIATE_TEST_SUITE_P(smoke_DetectionOutput_With_Hardcoded_Refs, ReferenceDetectionOutputLayerTest, testing::ValuesIn(generateDetectionOutputCombinedParams()), ReferenceDetectionOutputLayerTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_DetectionOutput_With_Hardcoded_Refs, + ReferenceDetectionOutputV8LayerTest, + testing::ValuesIn(generateDetectionOutputCombinedParams()), + ReferenceDetectionOutputV8LayerTest::getTestCaseName); + } // namespace \ No newline at end of file diff --git a/docs/template_plugin/tests/functional/shared_tests_instances/behavior/plugin/core_integration.cpp b/docs/template_plugin/tests/functional/shared_tests_instances/behavior/plugin/core_integration.cpp index 78cf8bfaecf..83701ec9030 100644 --- a/docs/template_plugin/tests/functional/shared_tests_instances/behavior/plugin/core_integration.cpp +++ b/docs/template_plugin/tests/functional/shared_tests_instances/behavior/plugin/core_integration.cpp @@ -130,10 +130,10 @@ TEST_F(IEClassGetConfigTestTEMPLATE, smoke_GetConfigNoThrow) { std::string defaultDeviceID = ie.GetConfig(deviceName, CONFIG_KEY(DEVICE_ID)); std::cout << CONFIG_KEY(DEVICE_ID) << " : " << defaultDeviceID << std::endl; } else if (CONFIG_KEY(PERF_COUNT) == confKey) { - bool defaultPerfCount = ie.GetConfig(deviceName, CONFIG_KEY(PERF_COUNT)).as(); + auto defaultPerfCount = ie.GetConfig(deviceName, CONFIG_KEY(PERF_COUNT)).as(); std::cout << CONFIG_KEY(PERF_COUNT) << " : " << defaultPerfCount << std::endl; } else if (CONFIG_KEY(EXCLUSIVE_ASYNC_REQUESTS) == confKey) { - bool defaultExclusive = ie.GetConfig(deviceName, CONFIG_KEY(EXCLUSIVE_ASYNC_REQUESTS)).as(); + auto defaultExclusive = ie.GetConfig(deviceName, CONFIG_KEY(EXCLUSIVE_ASYNC_REQUESTS)).as(); std::cout << CONFIG_KEY(EXCLUSIVE_ASYNC_REQUESTS) << " : " << defaultExclusive << std::endl; } } diff --git a/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.cpp b/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.cpp index d4daf57fd38..a08c640e4c1 100644 --- a/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.cpp +++ b/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.cpp @@ -94,12 +94,12 @@ MKLDNNDescriptor::operator std::shared_ptr() { return typeDesc->getPtr(); } -MKLDNNDescriptor::MKLDNNDescriptor(std::shared_ptr desc) { - this->desc.reset(new DescFwdImpl(desc)); +MKLDNNDescriptor::MKLDNNDescriptor(std::shared_ptr desc) { + this->desc.reset(new DescFwdImpl(desc)); } -MKLDNNDescriptor::operator std::shared_ptr() { - auto typeDesc = std::dynamic_pointer_cast>(desc); +MKLDNNDescriptor::operator std::shared_ptr() { + auto typeDesc = std::dynamic_pointer_cast>(desc); if (typeDesc == nullptr) { IE_THROW() << "Cannot cast descriptor!"; } diff --git a/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.h b/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.h index d02f9c3da70..d85d447ca6c 100644 --- a/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.h +++ b/inference-engine/src/mkldnn_plugin/mkldnn_descriptor.h @@ -28,8 +28,8 @@ public: explicit MKLDNNDescriptor(std::shared_ptr desc); operator std::shared_ptr(); - explicit MKLDNNDescriptor(std::shared_ptr desc); - operator std::shared_ptr(); + explicit MKLDNNDescriptor(std::shared_ptr desc); + operator std::shared_ptr(); explicit MKLDNNDescriptor(std::shared_ptr desc); operator std::shared_ptr(); diff --git a/inference-engine/src/mkldnn_plugin/mkldnn_graph_dumper.cpp b/inference-engine/src/mkldnn_plugin/mkldnn_graph_dumper.cpp index 4ffa1845f6c..aae2495bb2a 100644 --- a/inference-engine/src/mkldnn_plugin/mkldnn_graph_dumper.cpp +++ b/inference-engine/src/mkldnn_plugin/mkldnn_graph_dumper.cpp @@ -186,7 +186,7 @@ std::shared_ptr dump_graph_as_ie_ngraph_net(const MKLDNNGraph } for (auto && kvp : meta_data) - return_node->get_rt_info()[kvp.first] = std::make_shared<::ngraph::VariantWrapper>(kvp.second); + return_node->get_rt_info()[kvp.first] = std::make_shared<::ov::RuntimeAttributeWrapper>(kvp.second); return_node->set_friendly_name(node->getName()); return return_node; diff --git a/inference-engine/src/mkldnn_plugin/mkldnn_infer_request.cpp b/inference-engine/src/mkldnn_plugin/mkldnn_infer_request.cpp index 72efcfcfe37..ccd9c96e675 100644 --- a/inference-engine/src/mkldnn_plugin/mkldnn_infer_request.cpp +++ b/inference-engine/src/mkldnn_plugin/mkldnn_infer_request.cpp @@ -526,9 +526,12 @@ void MKLDNNPlugin::MKLDNNInferRequest::changeDefaultPtr() { break; } - if (child->getType() == Concatenation && dynamic_cast(child.get())->isOptimized()) { - canBeInPlace = false; - break; + if (child->getType() == Concatenation) { + auto concat = dynamic_cast(child.get()); + if (concat && concat->isOptimized()) { + canBeInPlace = false; + break; + } } // Cannot be in-place before split because split is using different ptrs without offsets diff --git a/inference-engine/src/mkldnn_plugin/mkldnn_node.cpp b/inference-engine/src/mkldnn_plugin/mkldnn_node.cpp index 11fe2d4006b..03b2156e183 100644 --- a/inference-engine/src/mkldnn_plugin/mkldnn_node.cpp +++ b/inference-engine/src/mkldnn_plugin/mkldnn_node.cpp @@ -137,7 +137,7 @@ MKLDNNNode::MKLDNNNode(const std::shared_ptr& op, const mkldnn::en } if (op != nullptr) { - std::string inputMemoryFormats = ngraph::getMLKDNNInputMemoryFormats(op); + std::string inputMemoryFormats = ngraph::getMKLDNNInputMemoryFormats(op); if (!inputMemoryFormats.empty()) { std::istringstream stream(inputMemoryFormats); std::string str; @@ -148,7 +148,7 @@ MKLDNNNode::MKLDNNNode(const std::shared_ptr& op, const mkldnn::en } } - std::string outputMemoryFormats = ngraph::getMLKDNNOutputMemoryFormats(op); + std::string outputMemoryFormats = ngraph::getMKLDNNOutputMemoryFormats(op); if (!outputMemoryFormats.empty()) { std::istringstream stream(outputMemoryFormats); std::string str; @@ -162,7 +162,7 @@ MKLDNNNode::MKLDNNNode(const std::shared_ptr& op, const mkldnn::en const auto it = rtInfo.find("enforceBF16evenForGraphTail"); if (it != rtInfo.end()) { - if (const auto value = std::dynamic_pointer_cast>(it->second)) + if (const auto value = std::dynamic_pointer_cast>(it->second)) enforceBF16evenForGraphTail = value->get(); } } diff --git a/inference-engine/src/mkldnn_plugin/ngraph_transformations/move_eltwise_up_data_movement.cpp b/inference-engine/src/mkldnn_plugin/ngraph_transformations/move_eltwise_up_data_movement.cpp index 948a33d3951..ec76db9361f 100644 --- a/inference-engine/src/mkldnn_plugin/ngraph_transformations/move_eltwise_up_data_movement.cpp +++ b/inference-engine/src/mkldnn_plugin/ngraph_transformations/move_eltwise_up_data_movement.cpp @@ -85,7 +85,7 @@ MKLDNNPlugin::MoveEltwiseUpThroughDataMov::MoveEltwiseUpThroughDataMov() { } // eltwise constant shape should match new input shape - if (is_binary_op && current->get_output_shape(0).size() != eltwise->get_input_shape(1).size()) { + if (is_binary_op && current->get_output_partial_shape(0).rank().get_length() != eltwise->get_input_partial_shape(1).rank().get_length()) { auto old_eltwise_const = std::dynamic_pointer_cast(eltwise->get_input_node_shared_ptr(1)); auto new_constant = std::make_shared(*old_eltwise_const.get(), ngraph::Shape{}); ngraph::replace_node(old_eltwise_const, new_constant); diff --git a/inference-engine/src/mkldnn_plugin/ngraph_transformations/rnn_sequences_optimization.cpp b/inference-engine/src/mkldnn_plugin/ngraph_transformations/rnn_sequences_optimization.cpp index 0b81fdcc81e..8a6c9fbbe8e 100644 --- a/inference-engine/src/mkldnn_plugin/ngraph_transformations/rnn_sequences_optimization.cpp +++ b/inference-engine/src/mkldnn_plugin/ngraph_transformations/rnn_sequences_optimization.cpp @@ -67,7 +67,7 @@ namespace { ngraph::replace_node(transposeAfter, {reshape2->output(0)}); } - sequenceOp->get_rt_info()["seqAxis"] = std::make_shared>(seqAxis); + sequenceOp->get_rt_info()["seqAxis"] = std::make_shared>(seqAxis); return true; } diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_embedding_bag_offset_sum_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_embedding_bag_offset_sum_node.h index a7b3cac7a2e..146003c0b41 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_embedding_bag_offset_sum_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_embedding_bag_offset_sum_node.h @@ -39,8 +39,8 @@ private: const int* offsetsData_ = nullptr; const int* defaultIndices_ = nullptr; - size_t _indicesLen; - size_t _offsetsLen; + size_t _indicesLen = 0; + size_t _offsetsLen = 0; }; } // namespace MKLDNNPlugin diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.cpp index 708c6d91921..96d8c48be47 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.cpp @@ -215,12 +215,16 @@ static void nms_cf(const float* conf_data, detections = (post_nms_topn == -1 ? detections : (std::min)(post_nms_topn, detections)); } +bool MKLDNNExperimentalDetectronDetectionOutputNode::needShapeInfer() const { + return false; +} + +bool MKLDNNExperimentalDetectronDetectionOutputNode::needPrepareParams() const { + return false; +} + bool MKLDNNExperimentalDetectronDetectionOutputNode::isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept { try { - if (isDynamicNgraphNode(op)) { - errorMessage = "Doesn't support op with dynamic shapes"; - return false; - } const auto doOp = ngraph::as_type_ptr(op); if (!doOp) { errorMessage = "Node is not an instance of the ExperimentalDetectronDetectionOutput from the operations set v6."; @@ -268,6 +272,12 @@ void MKLDNNExperimentalDetectronDetectionOutputNode::initSupportedPrimitiveDescr impl_desc_type::ref_any); } +void MKLDNNExperimentalDetectronDetectionOutputNode::createPrimitive() { + if (inputShapesDefined()) { + updateLastInputDims(); + } +} + void MKLDNNExperimentalDetectronDetectionOutputNode::execute(mkldnn::stream strm) { const int rois_num = getParentEdgeAt(INPUT_ROIS)->getMemory().getStaticDims()[0]; assert(classes_num_ == static_cast(getParentEdgeAt(INPUT_SCORES)->getMemory().getStaticDims()[1])); diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.h index aac589b058f..3c73bd036bb 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_detection_output_node.h @@ -15,10 +15,13 @@ public: void getSupportedDescriptors() override {}; void initSupportedPrimitiveDescriptors() override; - void createPrimitive() override {}; + void createPrimitive() override; void execute(mkldnn::stream strm) override; bool created() const override; + bool needShapeInfer() const override; + bool needPrepareParams() const override; + void executeDynamicImpl(mkldnn::stream strm) override { execute(strm); } static bool isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept; private: diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.cpp index 977493ed5be..fc36c163484 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.cpp @@ -275,10 +275,6 @@ void fill_output_blobs(const float* proposals, const int* roi_indices, bool MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::isSupportedOperation (const std::shared_ptr& op, std::string& errorMessage) noexcept { try { - if (isDynamicNgraphNode(op)) { - errorMessage = "Doesn't support op with dynamic shapes"; - return false; - } const auto proposalOp = ngraph::as_type_ptr(op); if (!proposalOp) { errorMessage = "Node is not an instance of the Proposal from the operations set v0."; @@ -324,6 +320,12 @@ void MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::initSupportedP impl_desc_type::ref_any); } +void MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::createPrimitive() { + if (inputShapesDefined()) { + updateLastInputDims(); + } +} + void MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::execute(mkldnn::stream strm) { try { if (inputShapes.size() != 4 || outputShapes.size() != 2) { @@ -431,4 +433,12 @@ bool MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::created() cons return getType() == ExperimentalDetectronGenerateProposalsSingleImage; } +bool MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::needShapeInfer() const { + return false; +} + +bool MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode::needPrepareParams() const { + return false; +} + REG_MKLDNN_PRIM_FOR(MKLDNNExperimentalDetectronGenerateProposalsSingleImageNode, ExperimentalDetectronGenerateProposalsSingleImage) diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.h index 3caf61e168b..a18f41e5a94 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_generate_proposals_single_image_node.h @@ -16,10 +16,13 @@ public: void getSupportedDescriptors() override {}; void initSupportedPrimitiveDescriptors() override; - void createPrimitive() override {}; + void createPrimitive() override; void execute(mkldnn::stream strm) override; bool created() const override; + bool needShapeInfer() const override; + bool needPrepareParams() const override; + void executeDynamicImpl(mkldnn::stream strm) override { execute(strm); } static bool isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept; private: diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.cpp index 10359d50949..a30031d9b84 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.cpp @@ -14,10 +14,6 @@ using namespace InferenceEngine; bool MKLDNNExperimentalDetectronPriorGridGeneratorNode::isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept { try { - if (isDynamicNgraphNode(op)) { - errorMessage = "Doesn't support op with dynamic shapes"; - return false; - } const auto priorGridGen = std::dynamic_pointer_cast(op); if (!priorGridGen) { errorMessage = "Only opset6 ExperimentalDetectronPriorGridGenerator operation is supported"; @@ -42,11 +38,6 @@ MKLDNNExperimentalDetectronPriorGridGeneratorNode::MKLDNNExperimentalDetectronPr if (getOriginalInputsNumber() != 3 || getOriginalOutputsNumber() != 1) IE_THROW() << errorPrefix << " has incorrect number of input/output edges!"; - if (op->get_input_shape(INPUT_PRIORS).size() != 2 || - op->get_input_shape(INPUT_FEATUREMAP).size() != 4 || - op->get_input_shape(INPUT_IMAGE).size() != 4) - IE_THROW() << errorPrefix << " has unsupported input shape"; - const auto &attr = priorGridGen->get_attrs(); grid_w_ = attr.w; grid_h_ = attr.h; @@ -65,6 +56,12 @@ void MKLDNNExperimentalDetectronPriorGridGeneratorNode::initSupportedPrimitiveDe impl_desc_type::ref_any); } +void MKLDNNExperimentalDetectronPriorGridGeneratorNode::createPrimitive() { + if (inputShapesDefined()) { + updateLastInputDims(); + } +} + void MKLDNNExperimentalDetectronPriorGridGeneratorNode::execute(mkldnn::stream strm) { const int num_priors_ = getParentEdgeAt(INPUT_PRIORS)->getMemory().getStaticDims()[0]; assert(getParentEdgeAt(INPUT_PRIORS)->getMemory().getStaticDims()[1] == 4); @@ -95,4 +92,8 @@ bool MKLDNNExperimentalDetectronPriorGridGeneratorNode::created() const { return getType() == ExperimentalDetectronPriorGridGenerator; } +bool MKLDNNExperimentalDetectronPriorGridGeneratorNode::needPrepareParams() const { + return false; +} + REG_MKLDNN_PRIM_FOR(MKLDNNExperimentalDetectronPriorGridGeneratorNode, ExperimentalDetectronPriorGridGenerator) diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.h index 2f7e224e63c..c908add3223 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_experimental_detectron_priorgridgenerator_node.h @@ -15,10 +15,12 @@ public: void getSupportedDescriptors() override {}; void initSupportedPrimitiveDescriptors() override; - void createPrimitive() override {}; + void createPrimitive() override; void execute(mkldnn::stream strm) override; bool created() const override; + bool needPrepareParams() const override; + void executeDynamicImpl(mkldnn::stream strm) override { execute(strm); } static bool isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept; private: diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_generic_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_generic_node.cpp index 45ca5d7cf8e..930a73bc4ec 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_generic_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_generic_node.cpp @@ -163,25 +163,10 @@ void MKLDNNGenericNode::execLayer() { // TODO: use ngraph-based extension mechnism if needed to recompute shape isDynBatch = false; - // TODO: uncomment after using ngraph-based extension mechnism - // if (isDynBatch) { - // for (size_t i = 0; i < inputs.size(); i++) { - // auto td = inputs[i]->getTensorDesc(); - // td.setDims(inputDescs[i].getDims()); - // inputs[i] = make_blob_with_precision(td, getParentEdgeAt(i)->getMemory().GetData()); - // } - // } std::vector outputs; for (size_t i = 0; i < outputShapes.size(); i++) { - if (isDynBatch) { - auto out_edge = getChildEdgesAtPort(i)[0]; - auto td = MemoryDescUtils::convertToTensorDesc(out_edge->getMemory().getDesc()); - td.setDims(execOutputShapes[i]); - outputs.push_back(make_blob_with_precision(td, out_edge->getMemory().GetData())); - } else { - outputs.push_back(MemoryDescUtils::interpretAsBlob(getChildEdgesAtPort(i)[0]->getMemory())); - } + outputs.push_back(MemoryDescUtils::interpretAsBlob(getChildEdgesAtPort(i)[0]->getMemory())); } InferenceEngine::ResponseDesc resp; InferenceEngine::StatusCode rc = impls[0]->execute(inputs, outputs, &resp); diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_matrix_nms_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_matrix_nms_node.h index c1f272bd2b2..338247cf103 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_matrix_nms_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_matrix_nms_node.h @@ -48,10 +48,10 @@ private: static const size_t NMS_SELECTED_INDICES = 1; static const size_t NMS_VALID_OUTPUTS = 2; - size_t m_numBatches; - size_t m_numBoxes; - size_t m_numClasses; - size_t m_maxBoxesPerBatch; + size_t m_numBatches = 0; + size_t m_numBoxes = 0; + size_t m_numClasses = 0; + size_t m_maxBoxesPerBatch = 0; MatrixNmsSortResultType m_sortResultType; bool m_sortResultAcrossBatch; diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.cpp index 32b0fd6aaee..379253233ee 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.cpp @@ -20,10 +20,15 @@ using namespace mkldnn; using namespace MKLDNNPlugin; using namespace InferenceEngine; -bool MKLDNNPoolingNode::isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept { +bool MKLDNNPoolingNode::isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept { try { - if (!ngraph::as_type_ptr(op) && !ngraph::as_type_ptr(op)) { - errorMessage = "Only opset1 MaxPool and AvgPool operations are supported"; + if (ov::is_type(op)) { + if (!op->get_output_target_inputs(1).empty()) { + errorMessage = "MaxPool from opset8 is supported only with one output"; + return false; + } + } else if (!ov::is_type(op) && !ov::is_type(op)) { + errorMessage = "MaxPool and AvgPool from opset1 and MaxPool from opset8 are supported"; return false; } } catch (...) { @@ -32,48 +37,52 @@ bool MKLDNNPoolingNode::isSupportedOperation(const std::shared_ptr& op, const mkldnn::engine& eng, MKLDNNWeightsSharing::Ptr &cache) +MKLDNNPoolingNode::MKLDNNPoolingNode(const std::shared_ptr& op, const mkldnn::engine& eng, MKLDNNWeightsSharing::Ptr &cache) : MKLDNNNode(op, eng, cache) { std::string errorMessage; if (!isSupportedOperation(op, errorMessage)) { IE_THROW(NotImplemented) << errorMessage; } - auto maxPoolOp = ngraph::as_type_ptr(op); - auto avgPoolOp = ngraph::as_type_ptr(op); - if (maxPoolOp) { + auto get_attributes = [](std::vector& internal_attribute, const std::vector external_attribute) { + for (size_t i = 0; i < external_attribute.size(); i++) { + internal_attribute.push_back(static_cast(external_attribute[i])); + } + }; + + if (auto maxPoolOp_v8 = ov::as_type_ptr(op)) { + isMaxPool8 = true; algorithm = PoolingMax; exclude_pad = false; - for (int i = 0; i < maxPoolOp->get_strides().size(); i++) { - stride.push_back(static_cast(maxPoolOp->get_strides()[i])); - } - for (int i = 0; i < maxPoolOp->get_kernel().size(); i++) { - kernel.push_back(static_cast(maxPoolOp->get_kernel()[i])); - } - for (int i = 0; i < maxPoolOp->get_pads_begin().size(); i++) { - data_pad_begin.push_back(static_cast(maxPoolOp->get_pads_begin()[i])); - } - for (int i = 0; i < maxPoolOp->get_pads_end().size(); i++) { - data_pad_end.push_back(static_cast(maxPoolOp->get_pads_end()[i])); - } - auto_pad = (maxPoolOp->get_auto_pad() == ov::op::PadType::SAME_LOWER || maxPoolOp->get_auto_pad() == ov::op::PadType::SAME_UPPER); - } else if (avgPoolOp) { + get_attributes(dilation, maxPoolOp_v8->get_dilations()); + get_attributes(stride, maxPoolOp_v8->get_strides()); + get_attributes(kernel, maxPoolOp_v8->get_kernel()); + get_attributes(data_pad_begin, maxPoolOp_v8->get_pads_begin()); + get_attributes(data_pad_end, maxPoolOp_v8->get_pads_end()); + + auto_pad = (maxPoolOp_v8->get_auto_pad() == ov::op::PadType::SAME_LOWER || maxPoolOp_v8->get_auto_pad() == ov::op::PadType::SAME_UPPER); + } else if (auto maxPoolOp_v1 = ov::as_type_ptr(op)) { + algorithm = PoolingMax; + exclude_pad = false; + + get_attributes(stride, maxPoolOp_v1->get_strides()); + get_attributes(kernel, maxPoolOp_v1->get_kernel()); + get_attributes(data_pad_begin, maxPoolOp_v1->get_pads_begin()); + get_attributes(data_pad_end, maxPoolOp_v1->get_pads_end()); + dilation.resize(kernel.size(), 1); + + auto_pad = (maxPoolOp_v1->get_auto_pad() == ov::op::PadType::SAME_LOWER || maxPoolOp_v1->get_auto_pad() == ov::op::PadType::SAME_UPPER); + } else if (auto avgPoolOp = ov::as_type_ptr(op)) { algorithm = PoolingAvg; exclude_pad = avgPoolOp->get_exclude_pad(); - for (int i = 0; i < avgPoolOp->get_strides().size(); i++) { - stride.push_back(static_cast(avgPoolOp->get_strides()[i])); - } - for (int i = 0; i < avgPoolOp->get_kernel().size(); i++) { - kernel.push_back(static_cast(avgPoolOp->get_kernel()[i])); - } - for (int i = 0; i < avgPoolOp->get_pads_begin().size(); i++) { - data_pad_begin.push_back(static_cast(avgPoolOp->get_pads_begin()[i])); - } - for (int i = 0; i < avgPoolOp->get_pads_end().size(); i++) { - data_pad_end.push_back(static_cast(avgPoolOp->get_pads_end()[i])); - } + get_attributes(stride, avgPoolOp->get_strides()); + get_attributes(kernel, avgPoolOp->get_kernel()); + get_attributes(data_pad_begin, avgPoolOp->get_pads_begin()); + get_attributes(data_pad_end, avgPoolOp->get_pads_end()); + dilation.resize(kernel.size(), 1); + auto_pad = (avgPoolOp->get_auto_pad() == ov::op::PadType::SAME_LOWER || avgPoolOp->get_auto_pad() == ov::op::PadType::SAME_UPPER); } } @@ -94,20 +103,23 @@ std::vector MKLDNNPoolingNode::getAvailableFormatsForDims(co return {memory::format_tag::any}; } -void MKLDNNPoolingNode::initEffectivePad(const Shape &inShape, const Shape &outShape) { +void MKLDNNPoolingNode::initEffectiveAttributes(const Shape &inShape, const Shape &outShape) { effective_pad_begin = data_pad_begin; effective_pad_end.resize(data_pad_end.size()); + effective_dilation.resize(dilation.size(), 0); const auto &inDims = inShape.getStaticDims(); const auto &outDims = outShape.getStaticDims(); for (int i = 0; i < effective_pad_end.size(); i++) { int krn = kernel[i]; + int dil = dilation[i]; int src = inDims[2 + i]; int dst = outDims[2 + i]; - int calc_dst = (src - krn + data_pad_begin[i]) / stride[i] + 1; + int calc_dst = (src - (1 + (krn - 1) * dil) + data_pad_begin[i]) / stride[i] + 1; effective_pad_end[i] = (dst - calc_dst) * stride[i]; + effective_dilation[i] = dil - 1; } } @@ -120,8 +132,8 @@ void MKLDNNPoolingNode::getSupportedDescriptors() { if (getChildEdges().empty()) IE_THROW() << "Incorrect number of output edges for layer " << getName(); - inputPrecision = getOriginalInputPrecisionAtPort(0); - outputPrecision = getOriginalOutputPrecisionAtPort(0); + InferenceEngine::Precision inputPrecision = getOriginalInputPrecisionAtPort(0); + InferenceEngine::Precision outputPrecision = getOriginalOutputPrecisionAtPort(0); // WA: LPT transformation has WA which allows average pooling has I8/U8 output precision instead of FP32, // so we explicitly set output precision as FP32 @@ -151,8 +163,8 @@ void MKLDNNPoolingNode::getSupportedDescriptors() { if ((inputRank < 3) || (inputRank > 5)) IE_THROW() << "Pooling layer. Unsupported mode. Only 3D, 4D and 5D blobs are supported as input."; - initEffectivePad(MemoryDescUtils::makeDummyShape(parentShape), - MemoryDescUtils::makeDummyShape(childShape)); + initEffectiveAttributes(MemoryDescUtils::makeDummyShape(parentShape), + MemoryDescUtils::makeDummyShape(childShape)); if (inputPrecision == Precision::I8 || inputPrecision == Precision::U8) { // We have to extend i8i8_pooling_fwd_t from oneDNN to support BF16 output data type @@ -185,7 +197,7 @@ void MKLDNNPoolingNode::getSupportedDescriptors() { } } -std::pair, std::vector> MKLDNNPoolingNode::getPaddingFromNode(std::shared_ptr node) const { +std::pair, std::vector> MKLDNNPoolingNode::getPaddingFromNode(std::shared_ptr node) const { const auto convertPadding = [](const VectorDims &newPads) { std::vector pads(newPads.size()); for (int i = 0; i < newPads.size(); i++) { @@ -195,12 +207,16 @@ std::pair, std::vector> MKLDNNPoolingNode::get }; VectorDims padsBegin, padsEnd; - if (getAlgorithm() == PoolingMax) { - const auto pool = ngraph::as_type_ptr(opToShapeInfer); + if (isMaxPool8) { + const auto pool = ov::as_type_ptr(opToShapeInfer); + padsBegin = pool->get_pads_begin(); + padsEnd = pool->get_pads_end(); + } else if (getAlgorithm() == PoolingMax) { + const auto pool = ov::as_type_ptr(opToShapeInfer); padsBegin = pool->get_pads_begin(); padsEnd = pool->get_pads_end(); } else if (getAlgorithm() == PoolingAvg) { - const auto pool = ngraph::as_type_ptr(opToShapeInfer); + const auto pool = ov::as_type_ptr(opToShapeInfer); padsBegin = pool->get_pads_begin(); padsEnd = pool->get_pads_end(); } @@ -231,15 +247,15 @@ void MKLDNNPoolingNode::prepareParams() { if (auto_pad) { std::tie(data_pad_begin, data_pad_end) = getPaddingFromNode(opToShapeInfer); } - initEffectivePad(inDesc->getShape(), outDesc->getShape()); + initEffectiveAttributes(inDesc->getShape(), outDesc->getShape()); } mkldnn::algorithm alg = getPoolingAlgorithm(); MKLDNNDescriptor desc{createDescriptorInternal(in_candidate, out_candidate, alg)}; - pooling_forward::primitive_desc prim_desc; + pooling_v2_forward::primitive_desc prim_desc; primitive_desc_iterator itpd = desc.createPrimitiveDescriptorIterator(getEngine(), *attr); - while (static_cast(itpd)) { + while (static_cast(itpd)) { impl_desc_type impl_type = parse_impl_name(itpd.impl_info_str()); if (impl_type == selected_pd->getImplementationType()) { @@ -250,7 +266,7 @@ void MKLDNNPoolingNode::prepareParams() { IE_THROW() << "Primitive descriptor was not found for node " << getName() << "."; } - prim.reset(new pooling_forward(prim_desc)); + prim.reset(new pooling_v2_forward(prim_desc)); auto src = getParentEdgesAtPort(0)[0]->getMemoryPtr()->GetPrimitive(); auto dst = getChildEdgesAtPort(0)[0]->getMemoryPtr()->GetPrimitive(); @@ -296,9 +312,9 @@ mkldnn::algorithm MKLDNNPoolingNode::getPoolingAlgorithm() const { } } -std::shared_ptr MKLDNNPoolingNode::createDescriptorInternal(const mkldnn::memory::desc& in_candidate, - const mkldnn::memory::desc& out_candidate, - const mkldnn::algorithm alg) const { +std::shared_ptr MKLDNNPoolingNode::createDescriptorInternal(const mkldnn::memory::desc& in_candidate, + const mkldnn::memory::desc& out_candidate, + const mkldnn::algorithm alg) const { if (alg == mkldnn::algorithm::undef) { IE_THROW() << "Unsupported pooling type"; } @@ -306,13 +322,14 @@ std::shared_ptr MKLDNNPoolingNode::createDescriptorIntern auto convert = [] (std::vector orig_dims) { return memory::dims(orig_dims.begin(), orig_dims.end()); }; - std::shared_ptr desc_ptr( - new pooling_forward::desc(prop_kind::forward_scoring, alg, - in_candidate, out_candidate, - convert(stride), - convert(kernel), - convert(effective_pad_begin), - convert(effective_pad_end))); + std::shared_ptr desc_ptr( + new pooling_v2_forward::desc(prop_kind::forward_scoring, alg, + in_candidate, out_candidate, + convert(stride), + convert(kernel), + convert(effective_dilation), + convert(effective_pad_begin), + convert(effective_pad_end))); if (alg == mkldnn::algorithm::pooling_avg_include_padding) { // In case of AVG including paddings the norm coeff should be calculated @@ -343,14 +360,12 @@ void MKLDNNPoolingNode::createDescriptor(const std::vector &input if (auto_pad) { std::tie(data_pad_begin, data_pad_end) = getPaddingFromNode(opToShapeInfer); } - initEffectivePad(inDesc->getShape(), outDesc->getShape()); + initEffectiveAttributes(inDesc->getShape(), outDesc->getShape()); } auto dnnlOutDesc = MemoryDescUtils::convertToDnnlBlockedMemoryDesc(*outDesc); auto out_candidate = dnnlOutDesc.getDnnlDesc(); - mkldnn::algorithm alg = getPoolingAlgorithm(); - auto desc_ptr = createDescriptorInternal(in_candidate, out_candidate, alg); - + auto desc_ptr = createDescriptorInternal(in_candidate, out_candidate, getPoolingAlgorithm()); descs.emplace_back(desc_ptr); } @@ -383,6 +398,18 @@ void MKLDNNPoolingNode::initSupportedPrimitiveDescriptors() { config.outConfs.push_back(dataConfig); } + + // CPU plugin doesn't support second output of MaxPool-8, but anyway we should have out config for second port as stub + if (isMaxPool8) { + auto& creatorsMap = BlockedDescCreator::getCommonCreators(); + PortConfig dataConfig; + dataConfig.inPlace = -1; + dataConfig.constant = false; + dataConfig.desc = creatorsMap.at(LayoutType::ncsp)->createSharedDesc(config.outConfs.front().desc->getPrecision(), getOutputShapeAtPort(1)); + + config.outConfs.push_back(dataConfig); + } + impl_desc_type impl_type = parse_impl_name(itpd.impl_info_str()); supportedPrimitiveDescriptors.emplace_back(config, impl_type); @@ -434,6 +461,18 @@ void MKLDNNPoolingNode::initDescriptor(const NodeConfig& config) { dataConfig.desc = getDstMemDesc(itpd, i); cfg.outConfs.push_back(dataConfig); } + + // CPU plugin doesn't support second output of MaxPool-8, but anyway we should have out config for second port as stub + if (isMaxPool8) { + auto& creatorsMap = BlockedDescCreator::getCommonCreators(); + PortConfig dataConfig; + dataConfig.inPlace = -1; + dataConfig.constant = false; + dataConfig.desc = creatorsMap.at(LayoutType::ncsp)->createSharedDesc(cfg.outConfs.front().desc->getPrecision(), getOutputShapeAtPort(1)); + + cfg.outConfs.push_back(dataConfig); + } + impl_desc_type impl_type = parse_impl_name(itpd.impl_info_str()); if (selected_count == selectedPrimitiveDescriptorIndex) { if (impl_type != selectedPD->getImplementationType()) { diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.h index 1d91199f95a..f3a6fc781cc 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_pooling_node.h @@ -14,7 +14,7 @@ namespace MKLDNNPlugin { class MKLDNNPoolingNode : public MKLDNNNode { public: - MKLDNNPoolingNode(const std::shared_ptr& op, const mkldnn::engine& eng, MKLDNNWeightsSharing::Ptr &cache); + MKLDNNPoolingNode(const std::shared_ptr& op, const mkldnn::engine& eng, MKLDNNWeightsSharing::Ptr &cache); void createDescriptor(const std::vector& inputDesc, const std::vector& outputDesc) override; @@ -31,7 +31,7 @@ public: void prepareParams() override; void executeDynamicImpl(mkldnn::stream strm) override { execute(strm); } - static bool isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept; + static bool isSupportedOperation(const std::shared_ptr& op, std::string& errorMessage) noexcept; protected: AttrPtr initPrimitiveAttr() const override; @@ -39,17 +39,19 @@ protected: private: void setPostOps(mkldnn::primitive_attr &attr, bool initWeights = false) const; - std::pair, std::vector> getPaddingFromNode(std::shared_ptr node) const; - void initEffectivePad(const Shape &inDims, const Shape &outDims); + std::pair, std::vector> getPaddingFromNode(std::shared_ptr node) const; + void initEffectiveAttributes(const Shape &inDims, const Shape &outDims); mkldnn::algorithm getPoolingAlgorithm() const; - std::shared_ptr createDescriptorInternal(const mkldnn::memory::desc& in_candidate, - const mkldnn::memory::desc& out_candidate, - const mkldnn::algorithm alg) const; + std::shared_ptr createDescriptorInternal(const mkldnn::memory::desc& in_candidate, + const mkldnn::memory::desc& out_candidate, + const mkldnn::algorithm alg) const; AttrPtr pAttr; + bool isMaxPool8 = false; bool auto_pad = false; bool exclude_pad = false; + std::vector dilation; std::vector stride; std::vector kernel; @@ -59,15 +61,16 @@ private: std::vector effective_pad_begin; std::vector effective_pad_end; + /// Effective dilation. Used to define correct dilation for OneDNN. + /// For OneDNN default dilation is vector of zero + std::vector effective_dilation; + /// Effective pad value. Describe how much zero element added to input /// data tensor. May be less than "Effective padding" values. /// If pooling window is out of this padding, the region of averaging /// is decreased. std::vector data_pad_begin; std::vector data_pad_end; - - InferenceEngine::Precision inputPrecision = InferenceEngine::Precision::FP32; - InferenceEngine::Precision outputPrecision = InferenceEngine::Precision::FP32; }; } // namespace MKLDNNPlugin diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_reduce_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_reduce_node.cpp index 3ecc41eee7e..4ccaf709471 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_reduce_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_reduce_node.cpp @@ -138,7 +138,7 @@ private: using Vmm = typename conditional3::type; size_t vlen = cpu_isa_traits::vlen; - bool planar_layout; + bool planar_layout = false; Xbyak::Address table_val(int index) { return ptr[reg_table + index * vlen]; } @@ -1136,7 +1136,7 @@ private: using Vmm = typename conditional3::type; size_t vlen = cpu_isa_traits::vlen; - bool planar_layout; + bool planar_layout = false; Xbyak::Reg64 reg_dst = r8; Xbyak::Reg64 reg_work_amount = r9; diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_rnn.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_rnn.cpp index 7c3e3eb2c20..76e20434d1e 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_rnn.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_rnn.cpp @@ -367,7 +367,7 @@ void MKLDNNRNN::initSeq(const std::shared_ptr& op) { const auto rtInfo = op->get_rt_info(); if (rtInfo.count("seqAxis")) { - nativeOrder = std::dynamic_pointer_cast>(rtInfo.at("seqAxis"))->get() == 0; + nativeOrder = std::dynamic_pointer_cast>(rtInfo.at("seqAxis"))->get() == 0; } out_data_dims.erase(out_data_dims.begin() + 1); diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_scatter_update_node.cpp b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_scatter_update_node.cpp index 566fe7fb00b..412d3853f1a 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_scatter_update_node.cpp +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_scatter_update_node.cpp @@ -114,13 +114,16 @@ void MKLDNNScatterUpdateNode::initSupportedPrimitiveDescriptors() { << "which should be smaller than or equal to input tensor rank"; } - SizeVector expectUpdateShape = {}; size_t tupleRank = indicesRank - 1; + SizeVector expectUpdateShape(tupleRank + srcRank - k, 0); + int updateAxisIter = 0; for (size_t ri = 0; ri < tupleRank; ri++) { - expectUpdateShape.push_back(indicesDim[ri]); + expectUpdateShape[updateAxisIter] = indicesDim[ri]; + updateAxisIter++; } for (size_t rd = k; rd < srcRank; rd++) { - expectUpdateShape.push_back(srcDataDim[rd]); + expectUpdateShape[updateAxisIter] = srcDataDim[rd]; + updateAxisIter++; } if (expectUpdateShape.size() != updateRank) { IE_THROW() << errorPrefix << " do not have matched tensor rank relationship for input, indices and update"; @@ -315,13 +318,16 @@ void MKLDNNScatterUpdateNode::execute(mkldnn::stream strm) { SizeVector updateDim = getParentEdgeAt(UPDATE_ID)->getMemory().getStaticDims(); size_t indicesRank = indicesDim.size(); size_t updateRank = updateDim.size(); - SizeVector expectUpdateShape = {}; + SizeVector expectUpdateShape(srcRank + indicesRank - 1, 0); + int axisIter = 0; for (size_t rs = 0; rs < srcRank; rs++) { if (rs != axis) { - expectUpdateShape.push_back(srcDataDim[rs]); + expectUpdateShape[axisIter] = srcDataDim[rs]; + axisIter++; } else { for (size_t ri = 0; ri < indicesRank; ri++) { - expectUpdateShape.push_back(indicesDim[ri]); + expectUpdateShape[axisIter] = indicesDim[ri]; + axisIter++; } } } diff --git a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_topk_node.h b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_topk_node.h index 83773eb7eec..bd2a72824cc 100644 --- a/inference-engine/src/mkldnn_plugin/nodes/mkldnn_topk_node.h +++ b/inference-engine/src/mkldnn_plugin/nodes/mkldnn_topk_node.h @@ -92,7 +92,8 @@ private: bool sort_value = false; bool mode_max = true; - int dim, before_num; + int dim = 0; + int before_num = 0; std::string errorPrefix; diff --git a/inference-engine/src/mkldnn_plugin/utils/ngraph_utils.hpp b/inference-engine/src/mkldnn_plugin/utils/ngraph_utils.hpp index 08ae121a7db..fe27efb885d 100644 --- a/inference-engine/src/mkldnn_plugin/utils/ngraph_utils.hpp +++ b/inference-engine/src/mkldnn_plugin/utils/ngraph_utils.hpp @@ -13,7 +13,7 @@ namespace MKLDNNPlugin { inline std::string getRTInfoValue(const std::map& rtInfo, std::string paramName) { auto it = rtInfo.find(paramName); if (it != rtInfo.end()) { - auto value = std::dynamic_pointer_cast>(it->second); + auto value = std::dynamic_pointer_cast>(it->second); return value->get(); } else { return ""; @@ -23,10 +23,13 @@ inline std::string getRTInfoValue(const std::map& rtInfo, inline std::string getPrimitivesPriorityValue(const std::shared_ptr &node) { const auto &rtInfo = node->get_rt_info(); - if (!rtInfo.count(ov::PrimitivesPriority::get_type_info_static())) return ""; + auto it_info = rtInfo.find(ov::PrimitivesPriority::get_type_info_static()); - const auto &attr = rtInfo.at(ov::PrimitivesPriority::get_type_info_static()); - return ngraph::as_type_ptr(attr)->get(); + if (it_info == rtInfo.end()) { + return {}; + } + + return it_info->second.as().value; } template diff --git a/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.cpp b/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.cpp index 5dca9699645..9217816bca7 100644 --- a/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.cpp +++ b/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.cpp @@ -11,25 +11,25 @@ using namespace ngraph; using namespace ov; -MLKDNNInputMemoryFormats::~MLKDNNInputMemoryFormats() = default; +MKLDNNInputMemoryFormats::~MKLDNNInputMemoryFormats() = default; -std::string ngraph::getMLKDNNInputMemoryFormats(const std::shared_ptr& node) { - auto it_info = node->get_rt_info().find(MLKDNNInputMemoryFormatsAttr); +std::string ngraph::getMKLDNNInputMemoryFormats(const std::shared_ptr& node) { + auto it_info = node->get_rt_info().find(MKLDNNInputMemoryFormats::get_type_info_static()); if (it_info != node->get_rt_info().end()) { - if (auto ptr = it_info->second.as>()) { - return ptr->getMemoryFormats(); + if (it_info->second.is()) { + return it_info->second.as().getMemoryFormats(); } } return {}; } -MLKDNNOutputMemoryFormats::~MLKDNNOutputMemoryFormats() = default; +MKLDNNOutputMemoryFormats::~MKLDNNOutputMemoryFormats() = default; -std::string ngraph::getMLKDNNOutputMemoryFormats(const std::shared_ptr& node) { - auto it_info = node->get_rt_info().find(MLKDNNOutputMemoryFormatsAttr); +std::string ngraph::getMKLDNNOutputMemoryFormats(const std::shared_ptr& node) { + auto it_info = node->get_rt_info().find(MKLDNNOutputMemoryFormats::get_type_info_static()); if (it_info != node->get_rt_info().end()) { - if (auto ptr = it_info->second.as>()) { - return ptr->getMemoryFormats(); + if (it_info->second.is()) { + return it_info->second.as().getMemoryFormats(); } } return {}; diff --git a/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.hpp b/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.hpp index c2e7498bb58..ea5611558b6 100644 --- a/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.hpp +++ b/inference-engine/src/mkldnn_plugin/utils/rt_info/memory_formats_attribute.hpp @@ -12,30 +12,28 @@ namespace ngraph { -constexpr const char *MLKDNNInputMemoryFormatsAttr = "MLKDNNInputMemoryFormats"; -constexpr const char *MLKDNNOutputMemoryFormatsAttr = "MLKDNNOutputMemoryFormats"; +constexpr const char *MKLDNNInputMemoryFormatsAttr = "MKLDNNInputMemoryFormats"; +constexpr const char *MKLDNNOutputMemoryFormatsAttr = "MKLDNNOutputMemoryFormats"; template -class MLKDNNMemoryFormats : public Variant { +class MKLDNNMemoryFormats : public ov::RuntimeAttribute { protected: std::string memory_format; public: - MLKDNNMemoryFormats() = default; - explicit MLKDNNMemoryFormats(const std::string &_memory_format) : memory_format(_memory_format) {} + MKLDNNMemoryFormats() = default; + explicit MKLDNNMemoryFormats(const std::string &_memory_format) : memory_format(_memory_format) {} std::string getMemoryFormats() const { return memory_format; } - ov::Any merge(const ngraph::NodeVector & nodes) override { + ov::Any merge(const ngraph::NodeVector & nodes) const override { std::set unique_mem_format; for (auto &node : nodes) { - auto it_info = node->get_rt_info().find(MemoryFormat::get_type_info_static().name); + auto it_info = node->get_rt_info().find(MemoryFormat::get_type_info_static()); if (it_info != node->get_rt_info().end()) { - if (auto ptr = it_info->second.template as>()) { - std::string mem_format = ptr->getMemoryFormats(); - if (!mem_format.empty()) { - unique_mem_format.insert(mem_format); - } + std::string mem_format = it_info->second.template as().getMemoryFormats(); + if (!mem_format.empty()) { + unique_mem_format.insert(mem_format); } } } @@ -50,28 +48,28 @@ public: if (unique_mem_format.size() == 1) { final_mem_format = *unique_mem_format.begin(); } - return std::make_shared(final_mem_format); + return MemoryFormat{final_mem_format}; } }; -class MLKDNNInputMemoryFormats : public MLKDNNMemoryFormats { +class MKLDNNInputMemoryFormats : public MKLDNNMemoryFormats { public: - OPENVINO_RTTI(MLKDNNInputMemoryFormatsAttr); - MLKDNNInputMemoryFormats() = default; - explicit MLKDNNInputMemoryFormats(const std::string &_memory_format) : MLKDNNMemoryFormats(_memory_format) {} - ~MLKDNNInputMemoryFormats() override; + OPENVINO_RTTI(MKLDNNInputMemoryFormatsAttr); + MKLDNNInputMemoryFormats() = default; + explicit MKLDNNInputMemoryFormats(const std::string &_memory_format) : MKLDNNMemoryFormats(_memory_format) {} + ~MKLDNNInputMemoryFormats() override; }; -std::string getMLKDNNInputMemoryFormats(const std::shared_ptr& node); +std::string getMKLDNNInputMemoryFormats(const std::shared_ptr& node); -class MLKDNNOutputMemoryFormats : public MLKDNNMemoryFormats { +class MKLDNNOutputMemoryFormats : public MKLDNNMemoryFormats { public: - OPENVINO_RTTI(MLKDNNOutputMemoryFormatsAttr); - MLKDNNOutputMemoryFormats() = default; - explicit MLKDNNOutputMemoryFormats(const std::string &_memory_format) : MLKDNNMemoryFormats(_memory_format) {} - ~MLKDNNOutputMemoryFormats() override; + OPENVINO_RTTI(MKLDNNOutputMemoryFormatsAttr); + MKLDNNOutputMemoryFormats() = default; + explicit MKLDNNOutputMemoryFormats(const std::string &_memory_format) : MKLDNNMemoryFormats(_memory_format) {} + ~MKLDNNOutputMemoryFormats() override; }; -std::string getMLKDNNOutputMemoryFormats(const std::shared_ptr& node); +std::string getMKLDNNOutputMemoryFormats(const std::shared_ptr& node); } // namespace ngraph diff --git a/inference-engine/src/offline_transformations/include/mask_attribute.hpp b/inference-engine/src/offline_transformations/include/mask_attribute.hpp index 4c74a64c074..0d57808949b 100644 --- a/inference-engine/src/offline_transformations/include/mask_attribute.hpp +++ b/inference-engine/src/offline_transformations/include/mask_attribute.hpp @@ -16,7 +16,6 @@ #include #include -#include namespace ngraph { @@ -28,6 +27,11 @@ namespace ngraph { class Mask : public std::vector>, public std::enable_shared_from_this { public: + static const ::ov::DiscreteTypeInfo& get_type_info_static() { + static const ::ov::DiscreteTypeInfo type_info{"Mask", 0, "0"}; + return type_info; + } + using Ptr = std::shared_ptr; Mask() = default; @@ -180,6 +184,7 @@ public: item.clear(); } } + private: bool m_is_shape_like{false}; @@ -199,22 +204,3 @@ Mask::Ptr getMask(const Output & output); void setMask(Output output, const Mask::Ptr & mask); } // namespace ngraph - -namespace ov { - -extern template class VariantImpl; - -template<> -class VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("VariantWrapper"); - BWDCMP_RTTI_DECLARATION; - - static std::shared_ptr> create(const value_type & value) { - return std::make_shared>(value); - } - - explicit VariantWrapper(const value_type &value) : VariantImpl(value) {} -}; - -} // namespace ov diff --git a/inference-engine/src/offline_transformations/src/pruning/mask_attribute.cpp b/inference-engine/src/offline_transformations/src/pruning/mask_attribute.cpp index 9ee1c023137..90f79e049f9 100644 --- a/inference-engine/src/offline_transformations/src/pruning/mask_attribute.cpp +++ b/inference-engine/src/offline_transformations/src/pruning/mask_attribute.cpp @@ -14,28 +14,22 @@ namespace ngraph { Mask::Ptr getMask(const Output & output) { auto &rtInfo = output.get_rt_info(); - using MaskWrapper = VariantWrapper; + if (!rtInfo.count(Mask::get_type_info_static())) return nullptr; - if (!rtInfo.count(MaskWrapper::get_type_info_static().name)) return nullptr; - - const auto &attr = rtInfo.at(MaskWrapper::get_type_info_static().name); - return ov::as_type_ptr(attr)->get(); + const auto &attr = rtInfo.at(Mask::get_type_info_static()); + return attr.as(); } Mask::Ptr getMask(const Output & output) { auto &rtInfo = output.get_rt_info(); - using MaskWrapper = VariantWrapper; - - if (!rtInfo.count(MaskWrapper::get_type_info_static().name)) return nullptr; - - const auto &attr = rtInfo.at(MaskWrapper::get_type_info_static().name); - return ov::as_type_ptr(attr)->get(); + if (!rtInfo.count(Mask::get_type_info_static())) return nullptr; + const auto &attr = rtInfo.at(Mask::get_type_info_static()); + return attr.as(); } void setMask(Output output, const Mask::Ptr & mask) { auto &rtInfo = output.get_rt_info(); - using MaskWrapper = VariantWrapper; - rtInfo[MaskWrapper::get_type_info_static().name] = MaskWrapper::create(mask); + rtInfo[Mask::get_type_info_static()] = mask; } std::ostream & operator<< (std::ostream & out, const Mask & mask) { @@ -54,11 +48,3 @@ std::ostream & operator<< (std::ostream & out, const Mask & mask) { } } // namespace ngraph - -namespace ov { - -template class ngraph::VariantImpl; - -BWDCMP_RTTI_DEFINITION(VariantWrapper); - -} // namespace ov diff --git a/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_deserialization.cpp b/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_deserialization.cpp index 2ba736b3090..c8c14c7b7d7 100644 --- a/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_deserialization.cpp +++ b/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_deserialization.cpp @@ -40,7 +40,7 @@ protected: ov::frontend::FrontEnd::Ptr FE; ov::frontend::InputModel::Ptr inputModel; - ov::VariantVector params{ov::make_variant(&modelStream)}; + ov::RuntimeAttributeVector params{&modelStream}; FE = manager.load_by_model(params); if (FE) @@ -119,7 +119,7 @@ TEST_F(RTInfoDeserialization, NodeV10) { ASSERT_NE(nullptr, f); auto check_rt_info = [](const RTMap& info) { - const std::string& key = VariantWrapper::get_type_info_static(); + const std::string& key = ngraph::FusedNames::get_type_info_static(); EXPECT_FALSE(info.count(key)); const std::string& key_old_api_order = ov::OldApiMapOrder::get_type_info_static(); @@ -278,7 +278,7 @@ TEST_F(RTInfoDeserialization, InputAndOutputV10) { ASSERT_NE(nullptr, f); auto check_rt_info = [](const RTMap& info) { - const std::string& key = VariantWrapper::get_type_info_static(); + const std::string& key = ngraph::FusedNames::get_type_info_static(); ASSERT_FALSE(info.count(key)); }; @@ -421,27 +421,22 @@ TEST_F(RTInfoDeserialization, NodeV11) { ASSERT_NE(nullptr, f); auto check_fused_names = [](const RTMap& info, const std::string& names) { - const std::string& key = VariantWrapper::get_type_info_static(); + const std::string& key = ngraph::FusedNames::get_type_info_static(); ASSERT_TRUE(info.count(key)); - auto fused_names_attr = std::dynamic_pointer_cast>(info.at(key)); - ASSERT_TRUE(fused_names_attr); - EXPECT_EQ(fused_names_attr->get().getNames(), names); + auto fused_names_attr = info.at(key).as(); + EXPECT_EQ(fused_names_attr.getNames(), names); }; auto check_old_api_map_order = [](const RTMap & info, const std::vector & order) { const std::string & old_api_map_key = ov::OldApiMapOrder::get_type_info_static(); ASSERT_TRUE(info.count(old_api_map_key)); - auto old_api_map_attr = std::dynamic_pointer_cast(info.at(old_api_map_key)); - ASSERT_TRUE(old_api_map_attr); - auto old_api_map_attr_val = old_api_map_attr->get(); + auto old_api_map_attr_val = info.at(old_api_map_key).as().value; EXPECT_EQ(old_api_map_attr_val, order); }; auto check_old_api_map_type = [](const RTMap & info, const ngraph::element::Type& type) { const std::string & old_api_map_key = ov::OldApiMapElementType::get_type_info_static(); ASSERT_TRUE(info.count(old_api_map_key)); - auto old_api_map_attr = std::dynamic_pointer_cast(info.at(old_api_map_key)); - ASSERT_TRUE(old_api_map_attr); - auto old_api_map_attr_val = old_api_map_attr->get(); + auto old_api_map_attr_val = info.at(old_api_map_key).as().value; EXPECT_EQ(old_api_map_attr_val, type); }; @@ -501,8 +496,7 @@ TEST_F(RTInfoDeserialization, NodeV11) { auto round = std::make_shared(convert_param, ngraph::opset8::Round::RoundMode::HALF_TO_EVEN); // TODO: runtime information should migrate as well? - round->get_rt_info()[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames("Round1,Round2")); + round->get_rt_info()[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames("Round1,Round2"); // TODO: No guarantee that exactly 'convert, then transpose' will be added by implicit post-processing auto constant_result = std::make_shared(ngraph::element::u64, @@ -722,20 +716,20 @@ TEST_F(RTInfoDeserialization, InputAndOutputV11) { check_version(f, 11); auto check_fused_names = [](const RTMap& info, const std::string& names) { - const std::string& key = VariantWrapper::get_type_info_static(); + const std::string& key = ngraph::FusedNames::get_type_info_static(); ASSERT_TRUE(info.count(key)); - auto fused_names_attr = std::dynamic_pointer_cast>(info.at(key)); - ASSERT_TRUE(fused_names_attr); - ASSERT_EQ(fused_names_attr->get().getNames(), names); + auto fused_names_attr = info.at(key).as(); + ASSERT_EQ(fused_names_attr.getNames(), names); }; auto param = f->get_parameters()[0]; check_fused_names(param->output(0).get_rt_info(), "test1,test2"); EXPECT_EQ(param->get_layout(), "NCHW"); - auto var0 = std::dynamic_pointer_cast( - f->input(0).get_rt_info()[ov::preprocess::TensorInfoMemoryType::get_type_info_static()]); - EXPECT_EQ(var0->get(), "test_memory_type"); + auto var0 = f->input(0).get_rt_info() + .at(ov::preprocess::TensorInfoMemoryType::get_type_info_static()) + .as().value; + EXPECT_EQ(var0, "test_memory_type"); auto result = f->get_result(); check_fused_names(result->input(0).get_rt_info(), "test5,test6"); diff --git a/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_serialization.cpp b/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_serialization.cpp index 8f32c89d1ec..e3cf7db8ae9 100644 --- a/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_serialization.cpp +++ b/inference-engine/tests/functional/inference_engine/ir_serialization/rt_info_serialization.cpp @@ -34,7 +34,7 @@ protected: ov::frontend::FrontEnd::Ptr FE; ov::frontend::InputModel::Ptr inputModel; - ov::VariantVector params{ov::make_variant(model_path), ov::make_variant(weights_path)}; + ov::RuntimeAttributeVector params{model_path, weights_path}; FE = manager.load_by_model(params); if (FE) @@ -51,16 +51,12 @@ private: }; TEST_F(RTInfoSerializationTest, all_attributes_latest) { - auto init_info = [](RTMap& info) { - info[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames("add")); - info[ov::PrimitivesPriority::get_type_info_static()] = - std::make_shared("priority"); - info[ov::OldApiMapOrder::get_type_info_static()] = - std::make_shared(std::vector{0, 2, 3, 1}); - info[ov::OldApiMapElementType::get_type_info_static()] = std::make_shared( - ngraph::element::Type_t::f32); - info[ov::Decompression::get_type_info_static()] = std::make_shared(); + auto init_info = [](RTMap & info) { + info[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames("add"); + info[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority("priority"); + info[ov::OldApiMapOrder::get_type_info_static()] = ov::OldApiMapOrder(std::vector{0, 2, 3, 1}); + info[ov::OldApiMapElementType::get_type_info_static()] = ov::OldApiMapElementType(ngraph::element::Type_t::f32); + info[ov::Decompression::get_type_info_static()] = ov::Decompression{}; }; std::shared_ptr function; @@ -85,36 +81,29 @@ TEST_F(RTInfoSerializationTest, all_attributes_latest) { ASSERT_NE(nullptr, f); auto check_info = [](const RTMap & info) { - const std::string & key = VariantWrapper::get_type_info_static(); + const std::string & key = ngraph::FusedNames::get_type_info_static(); ASSERT_TRUE(info.count(key)); - auto fused_names_attr = std::dynamic_pointer_cast>(info.at(key)); - ASSERT_TRUE(fused_names_attr); - ASSERT_EQ(fused_names_attr->get().getNames(), "add"); + auto fused_names_attr = info.at(key).as(); + ASSERT_EQ(fused_names_attr.getNames(), "add"); const std::string & pkey = ov::PrimitivesPriority::get_type_info_static(); ASSERT_TRUE(info.count(pkey)); - auto primitives_priority_attr = std::dynamic_pointer_cast(info.at(pkey)); - ASSERT_TRUE(primitives_priority_attr); - ASSERT_EQ(primitives_priority_attr->get(), "priority"); + auto primitives_priority_attr = info.at(pkey).as().value; + ASSERT_EQ(primitives_priority_attr, "priority"); const std::string & old_api_map_key_order = ov::OldApiMapOrder::get_type_info_static(); ASSERT_TRUE(info.count(old_api_map_key_order)); - auto old_api_map_attr = std::dynamic_pointer_cast(info.at(old_api_map_key_order)); - ASSERT_TRUE(old_api_map_attr); - auto old_api_map_attr_val = old_api_map_attr->get(); + auto old_api_map_attr_val = info.at(old_api_map_key_order).as().value; ASSERT_EQ(old_api_map_attr_val, std::vector({0, 2, 3, 1})); const std::string & old_api_map_key = ov::OldApiMapElementType::get_type_info_static(); ASSERT_TRUE(info.count(old_api_map_key)); - auto old_api_map_type = std::dynamic_pointer_cast(info.at(old_api_map_key)); - ASSERT_TRUE(old_api_map_type); - auto old_api_map_type_val = old_api_map_type->get(); + auto old_api_map_type_val = info.at(old_api_map_key).as().value; ASSERT_EQ(old_api_map_type_val, ngraph::element::Type_t::f32); const std::string& dkey = ov::Decompression::get_type_info_static(); ASSERT_TRUE(info.count(dkey)); - auto decompression_attr = std::dynamic_pointer_cast(info.at(dkey)); - ASSERT_TRUE(decompression_attr); + ASSERT_NO_THROW(info.at(dkey).as()); }; auto add = f->get_results()[0]->get_input_node_ptr(0); @@ -128,10 +117,8 @@ TEST_F(RTInfoSerializationTest, all_attributes_latest) { TEST_F(RTInfoSerializationTest, all_attributes_v10) { auto init_info = [](RTMap & info) { - info[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames("add")); - info[ov::PrimitivesPriority::get_type_info_static()] = - std::make_shared("priority"); + info[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames("add"); + info["PrimitivesPriority"] = ov::PrimitivesPriority("priority"); }; std::shared_ptr function; @@ -154,7 +141,7 @@ TEST_F(RTInfoSerializationTest, all_attributes_v10) { ASSERT_NE(nullptr, f); auto check_info = [](const RTMap & info) { - const std::string & key = VariantWrapper::get_type_info_static(); + const std::string & key = ngraph::FusedNames::get_type_info_static(); ASSERT_FALSE(info.count(key)); }; @@ -168,10 +155,8 @@ TEST_F(RTInfoSerializationTest, all_attributes_v10) { TEST_F(RTInfoSerializationTest, all_attributes_v11) { auto init_info = [](RTMap & info) { - info[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames("add")); - info[ov::PrimitivesPriority::get_type_info_static()] = - std::make_shared("priority"); + info[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames("add"); + info[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority("priority"); }; std::shared_ptr function; @@ -199,24 +184,23 @@ TEST_F(RTInfoSerializationTest, all_attributes_v11) { ASSERT_NE(nullptr, f); auto check_info = [](const RTMap & info) { - const std::string & key = VariantWrapper::get_type_info_static(); + const std::string & key = ngraph::FusedNames::get_type_info_static(); ASSERT_TRUE(info.count(key)); - auto fused_names_attr = std::dynamic_pointer_cast>(info.at(key)); - ASSERT_TRUE(fused_names_attr); - ASSERT_EQ(fused_names_attr->get().getNames(), "add"); + auto fused_names_attr = info.at(key).as(); + ASSERT_EQ(fused_names_attr.getNames(), "add"); const std::string & pkey = ov::PrimitivesPriority::get_type_info_static(); ASSERT_TRUE(info.count(pkey)); - auto primitives_priority_attr = std::dynamic_pointer_cast(info.at(pkey)); - ASSERT_TRUE(primitives_priority_attr); - ASSERT_EQ(primitives_priority_attr->get(), "priority"); + auto primitives_priority_attr = info.at(pkey).as().value; + ASSERT_EQ(primitives_priority_attr, "priority"); }; auto add = f->get_results()[0]->get_input_node_ptr(0); EXPECT_EQ(f->get_parameters()[0]->get_layout(), "NCHW"); - auto var0 = std::dynamic_pointer_cast( - f->input(0).get_rt_info()[ov::preprocess::TensorInfoMemoryType::get_type_info_static()]); - EXPECT_EQ(var0->get(), "test_memory_type"); + auto var0 = f->input(0).get_rt_info() + .at(ov::preprocess::TensorInfoMemoryType::get_type_info_static()) + .as().value; + EXPECT_EQ(var0, "test_memory_type"); EXPECT_EQ(f->get_results()[0]->get_layout(), "????"); check_info(add->get_rt_info()); check_info(add->input(0).get_rt_info()); diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_transformation.cpp index 9073e67100a..7b2480a42a9 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_transformation.cpp @@ -199,9 +199,9 @@ public: testValues.result.convert2, testValues.result.dequantization2, { - make_shared_attribute_ptr(true), - make_shared_attribute_ptr(interval, 256), - make_shared_attribute_ptr(false) + PrecisionPreservedAttribute(true), + IntervalsAlignmentAttribute(interval, 256), + QuantizationAlignmentAttribute(false) }, testValues.result.precisionAfterOperation, testValues.result.dequantizationAfter, @@ -235,13 +235,13 @@ TEST_P(ConcatTransformation, CompareFunctions) { ConcatTransformationTestValues testValues = std::get<2>(GetParam()); const auto actualFakeQuantizes = LayerTransformation::get(actualFunction); if (testValues.axis == 1) { - ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame>(actualFakeQuantizes)) << + ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame(actualFakeQuantizes)) << "PrecisionsAttribute are not the same"; if (testValues.checkIntervalsAlignmentAttributes) { auto operations = LayerTransformation::get(actualFunction); operations.insert(operations.end(), actualFakeQuantizes.begin(), actualFakeQuantizes.end()); - ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame>(operations)) << + ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame(operations)) << "IntervalsAlignmentAttribute are not the same"; } } @@ -275,13 +275,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2550.f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {0.1f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -304,7 +304,7 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {2.55f}, ngraph::element::f32, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -331,7 +331,7 @@ const std::vector testValues = { {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {2.55f}, ngraph::element::f32, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -352,13 +352,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -390,13 +390,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -428,13 +428,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -462,13 +462,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -496,13 +496,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -526,13 +526,13 @@ const std::vector testValues = { { { 256ul, {{1}, {1}, {}, {}}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {{1}, {1}, {}, {}}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -556,13 +556,13 @@ const std::vector testValues = { { { 256ul, {{1, 1, 1, 1}, {1, 1, 1, 1}, {}, {}}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {{1, 1, 1, 1}, {1, 1, 1, 1}, {}, {}}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -586,13 +586,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {1.275f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -616,13 +616,13 @@ const std::vector testValues = { { { 256ul, {{1}, {1}, {}, {}}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {{1}, {1}, {}, {}}, {0.f}, {1.275f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -657,7 +657,7 @@ const std::vector testValues = { {{1, 3, 1, 1}, {1, 3, 1, 1}, {}, {}}, {0.f, 0.f, 0.f}, {2.55f, 2.55f, 2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -666,7 +666,7 @@ const std::vector testValues = { {{1, 3, 1, 1}, {1, 3, 1, 1}, {}, {}}, {0.f, 0.f, 0.f}, {1.275f, 1.275f, 1.275f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -692,13 +692,13 @@ const std::vector testValues = { { { 256ul, {}, {-1.28f}, {1.27f}, {-128.f}, {127.f}, ngraph::element::i8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {-1.28f}, {1.27f}, {-128.f}, {127.f}, ngraph::element::i8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -722,13 +722,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {-1.28f}, {1.27f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, @@ -752,13 +752,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {-1.28f}, {1.27f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, @@ -896,13 +896,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -940,13 +940,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {-1.28f}, {1.27f}, {-1.28f}, {1.27f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul) } }, {}, {}, @@ -974,13 +974,13 @@ const std::vector testValues = { { { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {1.275f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, @@ -1012,13 +1012,13 @@ const std::vector testValues = { { { 256ul, {}, {1.275f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, {}, { 256ul, {}, {0.f}, {2.55f}, {0.f}, {255.f}, ngraph::element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{0.f, 2.55f}, 256ul) } }, {}, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_neighbors_transformation_with_convolution.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_neighbors_transformation_with_convolution.cpp index 1b22f085f23..0cb4cd1fafe 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_neighbors_transformation_with_convolution.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_neighbors_transformation_with_convolution.cpp @@ -160,16 +160,16 @@ TEST_P(ConcatWithNeighborsWithConvolutionTransformation, CompareFunctions) { auto actualFakeQuantizes = LayerTransformation::get(actualFunction); ASSERT_EQ(3ul, actualFakeQuantizes.size()) << "unexpected FakeQuantize operations count " << actualFakeQuantizes.size(); - ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame>(actualFakeQuantizes)) << + ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame(actualFakeQuantizes)) << "PrecisionsAttribute shared values are not the same"; auto actualConcatOperations = LayerTransformation::get(actualFunction); ASSERT_EQ(2ul, actualConcatOperations.size()) << "unexpected concat operations"; - ASSERT_NE(nullptr, ngraph::pass::low_precision::getAttribute>(actualConcatOperations[0])); - ASSERT_NE(nullptr, ngraph::pass::low_precision::getAttribute>(actualConcatOperations[1])); + ASSERT_FALSE(ngraph::pass::low_precision::getAttribute(actualConcatOperations[0]).empty()); + ASSERT_FALSE(ngraph::pass::low_precision::getAttribute(actualConcatOperations[1]).empty()); actualConcatOperations.insert(actualConcatOperations.end(), actualFakeQuantizes.begin(), actualFakeQuantizes.end()); - ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame>(actualConcatOperations)) << + ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame(actualConcatOperations)) << "IntervalsAlignmentAttribute shared values are not the same"; auto convolutions = LayerTransformation::get(actualFunction); @@ -177,8 +177,8 @@ TEST_P(ConcatWithNeighborsWithConvolutionTransformation, CompareFunctions) { ASSERT_EQ(2ul, convolutions[0]->input(0).get_rt_info().size()) << "unexpected input 0 attributes count: LowPrecision::PerTensorQuantization & LowPrecision::Precisions"; ASSERT_EQ(1ul, convolutions[0]->input(1).get_rt_info().size()) << "unexpected input 1 attributes count"; - auto a1 = std::dynamic_pointer_cast>>(convolutions[0]->input(1).get_rt_info().begin()->second); - ASSERT_EQ(element::i8, *a1->get().get()->sharedValue->precisions.begin()); + auto& a1 = convolutions[0]->input(1).get_rt_info().begin()->second.as(); + ASSERT_EQ(element::i8, a1.value().front()); } const std::vector precisions = { @@ -204,15 +204,15 @@ const std::vector testValues = { { { 256ul, ngraph::Shape({}), {-1.28f / 3.f}, {1.27f / 3.f}, {0.f}, {255.f}, element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } }, { 256ul, ngraph::Shape({}), {-1.28f / 2.f}, {1.27f / 2.f}, {64.f}, {192.f}, element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } }, { 256ul, ngraph::Shape({}), {-1.28f}, {1.27f}, {0.f}, {255.f}, element::u8, - { make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } + { IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 1.27f}, 256ul) } }, ngraph::element::u8, {{}, {}, {}}, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_not_quantized_parent_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_not_quantized_parent_transformation.cpp index b34480ad5a4..b9d82357805 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_not_quantized_parent_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/concat_with_not_quantized_parent_transformation.cpp @@ -218,9 +218,9 @@ public: testValues.result.dequantization2, true, { - make_shared_attribute_ptr(true), - make_shared_attribute_ptr(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul), - make_shared_attribute_ptr(false) + PrecisionPreservedAttribute(true), + IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval{-1.28f, 2.55f}, 256ul), + QuantizationAlignmentAttribute(false) }, testValues.result.precisionAfterOperation, testValues.result.dequantizationAfter, @@ -258,14 +258,14 @@ TEST_P(ConcatWithNotQuantizedParentTransformation, CompareFunctions) { break; } } - ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame>(actualFakeQuantizes)) << + ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame(actualFakeQuantizes)) << "PrecisionsAttribute are not the same"; ConcatWithNotQuantizedParentTransformationTestValues testValues = std::get<2>(GetParam()); if (testValues.checkIntervalsAlignmentAttributes) { auto operations = LayerTransformation::get(actualFunction); operations.insert(operations.end(), actualFakeQuantizes.begin(), actualFakeQuantizes.end()); - ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame>(operations)) << + ASSERT_TRUE(checkIfAttributesSharedValuesAreTheSame(operations)) << "IntervalsAlignmentAttribute are not the same"; } } diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/convert_subtract_constant_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/convert_subtract_constant_transformation.cpp index e910cbf6b8e..a952e3ef173 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/convert_subtract_constant_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/convert_subtract_constant_transformation.cpp @@ -169,7 +169,8 @@ const std::vector testValues = }, { { ngraph::element::f32, false }, - { {127.f}, element::f32, {}, false, 1ul, element::i8, true, {}, { "disabled_constant_folding_0" } }, + { {127.f}, element::f32, {}, false, 1ul, element::i8, true, {}, + { ov::pass::DisableConstantFolding::get_type_info_static() } }, { {0.03f}, element::f32, {}, false } }, { std::vector{ 2.f }, ngraph::element::i8}, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_backprop_data_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_backprop_data_transformation.cpp index a2fc719a092..4ddef412cab 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_backprop_data_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_backprop_data_transformation.cpp @@ -284,7 +284,8 @@ const std::vector testValues = { ngraph::element::u8, {{}, { { 128.f }, ngraph::element::f32, {}, false }, {}}, - {{}, { { 2.f }, ngraph::element::f32, {1, 2, 1, 1}, true, 1ul, element::i8, false, { "disabled_constant_folding_0" } }, {}}, + {{}, { { 2.f }, ngraph::element::f32, {1, 2, 1, 1}, true, 1ul, element::i8, false, + { ov::pass::DisableConstantFolding::get_type_info_static() } }, {}}, {{}, {}, {{ 0.0002f }, ngraph::element::f32, {}}}, op::Constant::create(ngraph::element::i8, ngraph::Shape{}, std::vector{ 2.f }), true diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_qdq_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_qdq_transformation.cpp index fb54eb596cb..69aec47af04 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_qdq_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/convolution_qdq_transformation.cpp @@ -174,7 +174,8 @@ const std::vector testValues = { }, { {}, - { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, { "disabled_constant_folding_0" } }, + { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, + { ov::pass::DisableConstantFolding::get_type_info_static() } }, {} }, { std::vector{ 100.f }, ngraph::element::i8}, @@ -349,7 +350,8 @@ const std::vector testValues = { }, { {}, - { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, { "disabled_constant_folding_0" } }, + { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, + { ov::pass::DisableConstantFolding::get_type_info_static() } }, {} }, { std::vector{ 2.f }, ngraph::element::i8}, @@ -417,7 +419,8 @@ const std::vector testValues = { }, { {}, - { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, { "disabled_constant_folding_0" } }, + { { 127.f }, ngraph::element::f32, { 6, 1, 1, 1 }, false, 1ul, element::i8, false, + { ov::pass::DisableConstantFolding::get_type_info_static() } }, {} }, { std::vector{ 2.f }, ngraph::element::i8}, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/disable_convert_on_const_path_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/disable_convert_on_const_path_transformation.cpp index cafdd3a76b8..41dfb4b2020 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/disable_convert_on_const_path_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/disable_convert_on_const_path_transformation.cpp @@ -145,12 +145,14 @@ const std::vector testValues = { ngraph::element::u8, { {ngraph::element::f32}, - { {128.f}, element::f32, {}, false, 1ul, element::u8, true, {}, { "disabled_constant_folding_0" } }, + { {128.f}, element::f32, {}, false, 1ul, element::u8, true, {}, + {ov::pass::DisableConstantFolding::get_type_info_static() } }, { {0.02f}, element::f32, {}, false } }, { { ngraph::element::f32, false }, - { {128.f}, element::f32, {}, false, 1ul, element::i8, true, {}, { "disabled_constant_folding_0" } }, + { {128.f}, element::f32, {}, false, 1ul, element::i8, true, {}, + {ov::pass::DisableConstantFolding::get_type_info_static() } }, { {0.03f}, element::f32, {}, false } }, { std::vector{ 1.f }, ngraph::element::f32}, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/fake_quantize_with_dq_not_optimal_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/fake_quantize_with_dq_not_optimal_transformation.cpp index 2dddf5e7dcc..1cdcd700944 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/fake_quantize_with_dq_not_optimal_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/fake_quantize_with_dq_not_optimal_transformation.cpp @@ -198,7 +198,9 @@ const std::vector fakeQuanti {}, { {}, - { std::vector(64, 127.f), ngraph::element::f32, {64, 1, 1, 1}, false, 1ul, ngraph::element::i8, false, {"disabled_constant_folding_0"}}, + { std::vector(64, 127.f), ngraph::element::f32, + {64, 1, 1, 1}, false, 1ul, ngraph::element::i8, false, + {ov::pass::DisableConstantFolding::get_type_info_static()}}, {} }, { diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/group_convolution_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/group_convolution_transformation.cpp index 6ce4e41ef28..aa65dd44312 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/group_convolution_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/group_convolution_transformation.cpp @@ -473,7 +473,7 @@ const std::vector testValuesGroupConv = { 1, ngraph::element::i8, false, - {"disabled_constant_folding_0"} + {ov::pass::DisableConstantFolding::get_type_info_static()} }, {} }, diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/layer_transformation.hpp b/inference-engine/tests/functional/inference_engine/lp_transformations/layer_transformation.hpp index b90717de7ac..708b5d0c2ca 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/layer_transformation.hpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/layer_transformation.hpp @@ -83,10 +83,9 @@ public: for (size_t nodeIndex = 0ul; nodeIndex < nodes.size(); nodeIndex++) { auto& rt = nodes[nodeIndex]->get_rt_info(); for (auto& it : rt) { - auto reference = std::dynamic_pointer_cast>>(it.second); - assert(reference != nullptr); - if ((reference->get()->sharedValue->combinedInterval.low != intervalLow) && - (reference->get()->sharedValue->combinedInterval.high != intervalHigh)) { + auto& reference = it.second.as(); + if ((reference.value().combinedInterval.low != intervalLow) && + (reference.value().combinedInterval.high != intervalHigh)) { return false; } } @@ -96,10 +95,10 @@ public: } static bool compare( - const std::shared_ptr& value1, - const std::shared_ptr& value2) { - if ((value1->sharedValue->combinedInterval.low != value2->sharedValue->combinedInterval.low) || - (value1->sharedValue->combinedInterval.high != value2->sharedValue->combinedInterval.high)) { + const IntervalsAlignmentAttribute& value1, + const IntervalsAlignmentAttribute& value2) { + if ((value1.value().combinedInterval.low != value2.value().combinedInterval.low) || + (value1.value().combinedInterval.high != value2.value().combinedInterval.high)) { return false; } return true; @@ -124,10 +123,8 @@ public: return false; } - auto reference = std::dynamic_pointer_cast>(referenceIt->second); - auto actual = std::dynamic_pointer_cast>(actualIt.second); - if ((actual != nullptr) && (reference != nullptr)) { - if (!compare(reference->get(), actual->get())) { + if (!referenceIt->second.empty() && !actualIt.second.empty()) { + if (!compare(referenceIt->second.template as(), actualIt.second.template as())) { return false; } } @@ -139,24 +136,22 @@ public: template static bool checkIfOutputAttributesAreTheSame(const NodeVector& nodes) { - void* first = nullptr; + ov::Any first; for (auto node : nodes) { for (auto output : node->outputs()) { auto& rt = output.get_rt_info(); - const std::string& name = VariantWrapper::type_info.name; + const std::string& name = Attribute::get_type_info_static(); auto it = rt.find(name); if (it == rt.end()) { return false; } auto value = it->second; - OPENVINO_SUPPRESS_DEPRECATED_START - if (first == nullptr) { - first = value.get(); - } else if (value.get() != first) { + if (first.empty()) { + first = value; + } else if (value.template as().attribute != first.template as().attribute) { return false; } - OPENVINO_SUPPRESS_DEPRECATED_END } } return true; @@ -164,15 +159,15 @@ public: template static bool checkIfOutputAttributesSharedValuesAreTheSame(const NodeVector& nodes) { - std::shared_ptr first = nullptr; + ov::Any first; for (auto node : nodes) { for (auto output : node->outputs()) { auto value = ngraph::pass::low_precision::getAttributeFromOutput(output); - if (first == nullptr) { + if (first.empty()) { first = value; } else { - const auto sharedValue1 = std::dynamic_pointer_cast>(value)->get()->sharedValue; - const auto sharedValue2 = std::dynamic_pointer_cast>(first)->get()->sharedValue; + const auto sharedValue1 = value.template as().attribute->sharedValue; + const auto sharedValue2 = first.template as().attribute->sharedValue; if (sharedValue1 != sharedValue2) { return false; } @@ -184,18 +179,18 @@ public: template static bool checkIfAttributesSharedValuesAreTheSame(const NodeVector& nodes) { - std::shared_ptr first = nullptr; + ov::Any first; for (auto node : nodes) { auto value = ngraph::pass::low_precision::getAttribute(node); - if (value == nullptr) { + if (value.empty()) { return false; } - if (first == nullptr) { + if (first.empty()) { first = value; } else { - const auto sharedValue1 = std::dynamic_pointer_cast>(value)->get()->sharedValue; - const auto sharedValue2 = std::dynamic_pointer_cast>(first)->get()->sharedValue; + const auto sharedValue1 = value.template as().attribute->sharedValue; + const auto sharedValue2 = first.template as().attribute->sharedValue; if (sharedValue1 != sharedValue2) { return false; } @@ -206,16 +201,16 @@ public: template static bool checkIfAttributesAreTheSame(const NodeVector& nodes) { - Variant* first = nullptr; + ov::Any first; for (auto node : nodes) { auto value = ngraph::pass::low_precision::getAttribute(node); - if (value == nullptr) { + if (value.empty()) { return false; } - if (first == nullptr) { - first = value.get(); - } else if (value.get() != first) { + if (first.empty()) { + first = value; + } else if (value.template as().attribute != first.template as().attribute) { return false; } } diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/lpt_avoid_shapeof_propagation_test.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/lpt_avoid_shapeof_propagation_test.cpp index a49749405fd..6cfe443b5e5 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/lpt_avoid_shapeof_propagation_test.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/lpt_avoid_shapeof_propagation_test.cpp @@ -227,9 +227,7 @@ TEST(LPT, AvoidDequantizationToShapeOfPropagationFakeQuantizeDecompositionTransf auto shapeOf = std::make_shared(fakeQuantize); auto& outInfo = fakeQuantize->output(0).get_rt_info(); - auto attribute = ngraph::pass::low_precision::make_shared_attribute(element::TypeVector{ element::u8, element::i8 }); - auto attributeWrapper = std::make_shared>>(attribute); - outInfo.emplace(ngraph::VariantWrapper>::type_info.name, attributeWrapper); + outInfo.emplace(PrecisionsAttribute::get_type_info_static(), PrecisionsAttribute(element::TypeVector{ element::u8, element::i8 })); auto result1 = std::make_shared(fakeQuantize); auto result2 = std::make_shared(shapeOf); diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/markup_avg_pool_precisions_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/markup_avg_pool_precisions_transformation.cpp index ce38a8a4dba..dd17e8e12f9 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/markup_avg_pool_precisions_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/markup_avg_pool_precisions_transformation.cpp @@ -140,14 +140,14 @@ TEST_P(MarkupAvgPoolPrecisionsTransformation, CompareFunctions) { ASSERT_EQ(1ul, avgPoolOperations.size()) << "unexpected avgPoolOperations size: " << avgPoolOperations.size(); { - auto avgPoolPrecisioinPreservedAttribute = ngraph::pass::low_precision::getAttribute( + auto avgPoolPrecisioinPreservedAttribute = ngraph::pass::low_precision::getAttribute( *avgPoolOperations.begin()); - ASSERT_NE(nullptr, avgPoolPrecisioinPreservedAttribute); - ASSERT_EQ(true, avgPoolPrecisioinPreservedAttribute->get()->sharedValue->value); + ASSERT_FALSE(avgPoolPrecisioinPreservedAttribute.empty()); + ASSERT_EQ(true, avgPoolPrecisioinPreservedAttribute.as().value()); } const auto precisionPreserved = LayerTransformation::get(actualFunction); - ASSERT_TRUE(checkIfAttributesAreTheSame>(precisionPreserved)) << + ASSERT_TRUE(checkIfAttributesAreTheSame(precisionPreserved)) << "AvgPoolPrecisionPreservedAttribute are not the same"; //auto res = compare_functions(referenceFunction, actualFunction, true, true); diff --git a/inference-engine/tests/functional/inference_engine/lp_transformations/move_fake_quantize_transformation.cpp b/inference-engine/tests/functional/inference_engine/lp_transformations/move_fake_quantize_transformation.cpp index 5a24340e230..6f3e9806f21 100644 --- a/inference-engine/tests/functional/inference_engine/lp_transformations/move_fake_quantize_transformation.cpp +++ b/inference-engine/tests/functional/inference_engine/lp_transformations/move_fake_quantize_transformation.cpp @@ -162,9 +162,9 @@ public: testValues.actual.convertAfter, testValues.actual.dequantizationAfter, { - ngraph::builder::subgraph::make_shared_attribute_ptr(true), - ngraph::builder::subgraph::make_shared_attribute_ptr(interval, 256), - ngraph::builder::subgraph::make_shared_attribute_ptr(false) + PrecisionPreservedAttribute(true), + IntervalsAlignmentAttribute(interval, 256), + QuantizationAlignmentAttribute(false) }, ngraph::element::undefined, {}, @@ -209,9 +209,9 @@ public: testValues.result.convertAfter, testValues.result.dequantizationAfter, { - ngraph::builder::subgraph::make_shared_attribute_ptr(true), - ngraph::builder::subgraph::make_shared_attribute_ptr(interval, 256), - ngraph::builder::subgraph::make_shared_attribute_ptr(false) + PrecisionPreservedAttribute(true), + IntervalsAlignmentAttribute(interval, 256), + QuantizationAlignmentAttribute(false) }, testValues.result.precisionAfterOperation, {}, @@ -242,7 +242,7 @@ TEST_P(MoveFakeQuantizeTransformation, CompareFunctions) { ASSERT_TRUE(LayerTransformation::allNamesAreUnique(actualFunction)) << "Not all names are unique"; const auto actualFakeQuantizes = LayerTransformation::get(actualFunction); - ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame>(actualFakeQuantizes)) << + ASSERT_TRUE(checkIfOutputAttributesSharedValuesAreTheSame(actualFakeQuantizes)) << "PrecisionsAttribute are not the same"; } diff --git a/inference-engine/tests/functional/inference_engine/snippets/registers.cpp b/inference-engine/tests/functional/inference_engine/snippets/registers.cpp index 54ce843308a..99dc5c6675d 100644 --- a/inference-engine/tests/functional/inference_engine/snippets/registers.cpp +++ b/inference-engine/tests/functional/inference_engine/snippets/registers.cpp @@ -9,7 +9,6 @@ #include #include -#include #include #include @@ -53,7 +52,7 @@ TEST(TransformationTests, AssignRegisters) { auto it_rinfo = rt.find("reginfo"); if (it_rinfo != rt.end()) { - auto reginfo = ov::as_type_ptr>>(it_rinfo->second)->get(); + auto reginfo = it_rinfo->second.as>(); auto reg = reginfo[0]; ASSERT_TRUE(ref_registers[op->get_friendly_name()] == reg); total_ops++; @@ -127,7 +126,7 @@ TEST(TransformationTests, AssignRegisters2) { auto& rt = op->get_rt_info(); auto it_rinfo = rt.find("reginfo"); if (it_rinfo != rt.end()) { - auto reginfo = ov::as_type_ptr>>(it_rinfo->second)->get(); + auto reginfo = it_rinfo->second.as>(); auto reg = reginfo[0]; ASSERT_TRUE(ref_registers[op->get_friendly_name()] == reg); total_ops++; diff --git a/inference-engine/tests/functional/inference_engine/transformations/detection_output_downgrade_test.cpp b/inference-engine/tests/functional/inference_engine/transformations/detection_output_downgrade_test.cpp new file mode 100644 index 00000000000..55c40cd9541 --- /dev/null +++ b/inference-engine/tests/functional/inference_engine/transformations/detection_output_downgrade_test.cpp @@ -0,0 +1,189 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "transformations/op_conversions/detection_output_downgrade.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "common_test_utils/ngraph_test_utils.hpp" +#include "transformations/init_node_info.hpp" + +using namespace ngraph; +using namespace testing; + +namespace { +void create_attributes_vectors(std::vector& attrs_v1_vector, + std::vector& attrs_v8_vector) { + // initialize attributes affecting shape inference + // others remain by default + for (int keep_top_k : {10, -1}) { + for (int top_k : {5, -1}) { + for (bool variance_encoded_in_target : {true, false}) { + for (bool share_location : {true, false}) { + for (bool normalized : {true, false}) { + opset1::DetectionOutput::Attributes attrs_v1; + opset8::DetectionOutput::Attributes attrs_v8; + attrs_v1.top_k = attrs_v8.top_k = top_k; + attrs_v1.keep_top_k = attrs_v8.keep_top_k = {keep_top_k}; + attrs_v1.variance_encoded_in_target = attrs_v8.variance_encoded_in_target = + variance_encoded_in_target; + attrs_v1.share_location = attrs_v8.share_location = share_location; + attrs_v1.normalized = attrs_v8.normalized = normalized; + attrs_v1_vector.push_back(attrs_v1); + attrs_v8_vector.push_back(attrs_v8); + } + } + } + } + } +} +} // namespace + +TEST(TransformationTests, DetectionOutput8ToDetectionOutput1) { + std::vector attrs_v1_vector; + std::vector attrs_v8_vector; + Dimension N = 5; + Dimension num_prior_boxes = 100; + Dimension priors_batch_size = N; + Dimension num_classes = 23; + + create_attributes_vectors(attrs_v1_vector, attrs_v8_vector); + ASSERT_TRUE(attrs_v1_vector.size() == attrs_v8_vector.size()) << "Sizes of attribute test vectors must be equal"; + for (size_t ind = 0; ind < attrs_v1_vector.size(); ++ind) { + std::shared_ptr f(nullptr), f_ref(nullptr); + // this case covers deducing a number of classes value + // since this value is not saved in attributes + opset8::DetectionOutput::Attributes attributes_v8 = attrs_v8_vector[ind]; + opset1::DetectionOutput::Attributes attributes_v1 = attrs_v1_vector[ind]; + if (num_classes.is_static()) { + attributes_v1.num_classes = num_classes.get_length(); + } + + Dimension num_loc_classes = attributes_v8.share_location ? 1 : num_classes; + Dimension prior_box_size = attributes_v8.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, num_prior_boxes * num_loc_classes * 4}; + PartialShape class_preds_shape = {N, num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attributes_v8.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + + auto detection_output_v8 = + std::make_shared(box_logits, class_preds, proposals, attributes_v8); + + f = std::make_shared(ngraph::NodeVector{detection_output_v8}, + ngraph::ParameterVector{box_logits, class_preds, proposals}); + + ngraph::pass::Manager manager; + manager.register_pass(); + manager.run_passes(f); + } + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + + auto detection_output_v1 = + std::make_shared(box_logits, class_preds, proposals, attributes_v1); + + f_ref = std::make_shared(ngraph::NodeVector{detection_output_v1}, + ngraph::ParameterVector{box_logits, class_preds, proposals}); + } + auto res = compare_functions(f, f_ref); + ASSERT_TRUE(res.first) << res.second; + } +} + +TEST(TransformationTests, DetectionOutput8ToDetectionOutput1FiveArguments) { + // In this case num_classes attribute value is deduced using inputs shapes + std::vector attrs_v1_vector; + std::vector attrs_v8_vector; + Dimension N = 5; + Dimension num_prior_boxes = 15; + Dimension priors_batch_size = N; + Dimension num_classes = 23; + + create_attributes_vectors(attrs_v1_vector, attrs_v8_vector); + ASSERT_TRUE(attrs_v1_vector.size() == attrs_v8_vector.size()) << "Sizes of attribute test vectors must be equal"; + for (size_t ind = 0; ind < attrs_v1_vector.size(); ++ind) { + std::shared_ptr f(nullptr), f_ref(nullptr); + opset8::DetectionOutput::Attributes attributes_v8 = attrs_v8_vector[ind]; + opset1::DetectionOutput::Attributes attributes_v1 = attrs_v1_vector[ind]; + if (num_classes.is_static()) { + attributes_v1.num_classes = num_classes.get_length(); + } + + Dimension num_loc_classes = attributes_v8.share_location ? 1 : num_classes; + Dimension prior_box_size = attributes_v8.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, num_prior_boxes * num_loc_classes * 4}; + PartialShape class_preds_shape = {N, num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attributes_v8.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + PartialShape ad_class_preds_shape = {N, num_prior_boxes * 2}; + PartialShape ad_box_preds_shape = {N, num_prior_boxes * num_loc_classes * 4}; + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + auto ad_class_preds = + std::make_shared(ngraph::element::f32, ad_class_preds_shape); + auto ad_box_preds = std::make_shared(ngraph::element::f32, ad_box_preds_shape); + + auto detection_output_v8 = std::make_shared(box_logits, + class_preds, + proposals, + ad_class_preds, + ad_box_preds, + attributes_v8); + + f = std::make_shared( + ngraph::NodeVector{detection_output_v8}, + ngraph::ParameterVector{box_logits, class_preds, proposals, ad_class_preds, ad_box_preds}); + + ngraph::pass::Manager manager; + manager.register_pass(); + manager.run_passes(f); + } + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + auto ad_class_preds = + std::make_shared(ngraph::element::f32, ad_class_preds_shape); + auto ad_box_preds = std::make_shared(ngraph::element::f32, ad_box_preds_shape); + + auto detection_output_v1 = std::make_shared(box_logits, + class_preds, + proposals, + ad_class_preds, + ad_box_preds, + attributes_v1); + + f_ref = std::make_shared( + ngraph::NodeVector{detection_output_v1}, + ngraph::ParameterVector{box_logits, class_preds, proposals, ad_class_preds, ad_box_preds}); + } + auto res = compare_functions(f, f_ref); + ASSERT_TRUE(res.first) << res.second; + } +} diff --git a/inference-engine/tests/functional/inference_engine/transformations/detection_output_upgrade_test.cpp b/inference-engine/tests/functional/inference_engine/transformations/detection_output_upgrade_test.cpp new file mode 100644 index 00000000000..c20411dc48b --- /dev/null +++ b/inference-engine/tests/functional/inference_engine/transformations/detection_output_upgrade_test.cpp @@ -0,0 +1,189 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "transformations/op_conversions/detection_output_upgrade.hpp" + +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "common_test_utils/ngraph_test_utils.hpp" +#include "transformations/init_node_info.hpp" + +using namespace ngraph; +using namespace testing; + +namespace { +void create_attributes_vectors(std::vector& attrs_v1_vector, + std::vector& attrs_v8_vector) { + // initialize attributes affecting shape inference + // others remain by default + for (int keep_top_k : {10, -1}) { + for (int top_k : {5, -1}) { + for (bool variance_encoded_in_target : {true, false}) { + for (bool share_location : {true, false}) { + for (bool normalized : {true, false}) { + opset1::DetectionOutput::Attributes attrs_v1; + opset8::DetectionOutput::Attributes attrs_v8; + attrs_v1.top_k = attrs_v8.top_k = top_k; + attrs_v1.keep_top_k = attrs_v8.keep_top_k = {keep_top_k}; + attrs_v1.variance_encoded_in_target = attrs_v8.variance_encoded_in_target = + variance_encoded_in_target; + attrs_v1.share_location = attrs_v8.share_location = share_location; + attrs_v1.normalized = attrs_v8.normalized = normalized; + attrs_v1_vector.push_back(attrs_v1); + attrs_v8_vector.push_back(attrs_v8); + } + } + } + } + } +} +} // namespace + +TEST(TransformationTests, DetectionOutput1ToDetectionOutput8) { + std::vector attrs_v1_vector; + std::vector attrs_v8_vector; + Dimension N = 5; + Dimension num_prior_boxes = 100; + Dimension priors_batch_size = N; + Dimension num_classes = 23; + + create_attributes_vectors(attrs_v1_vector, attrs_v8_vector); + ASSERT_TRUE(attrs_v1_vector.size() == attrs_v8_vector.size()) << "Sizes of attribute test vectors must be equal"; + for (size_t ind = 0; ind < attrs_v1_vector.size(); ++ind) { + std::shared_ptr f(nullptr), f_ref(nullptr); + // this case covers deducing a number of classes value + // since this value is not saved in attributes + opset8::DetectionOutput::Attributes attributes_v8 = attrs_v8_vector[ind]; + opset1::DetectionOutput::Attributes attributes_v1 = attrs_v1_vector[ind]; + if (num_classes.is_static()) { + attributes_v1.num_classes = num_classes.get_length(); + } + + Dimension num_loc_classes = attributes_v8.share_location ? 1 : num_classes; + Dimension prior_box_size = attributes_v8.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, num_prior_boxes * num_loc_classes * 4}; + PartialShape class_preds_shape = {N, num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attributes_v8.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + + auto detection_output_v1 = + std::make_shared(box_logits, class_preds, proposals, attributes_v1); + + f = std::make_shared(ngraph::NodeVector{detection_output_v1}, + ngraph::ParameterVector{box_logits, class_preds, proposals}); + + ngraph::pass::Manager manager; + manager.register_pass(); + manager.run_passes(f); + } + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + + auto detection_output_v8 = + std::make_shared(box_logits, class_preds, proposals, attributes_v8); + + f_ref = std::make_shared(ngraph::NodeVector{detection_output_v8}, + ngraph::ParameterVector{box_logits, class_preds, proposals}); + } + auto res = compare_functions(f, f_ref); + ASSERT_TRUE(res.first) << res.second; + } +} + +TEST(TransformationTests, DetectionOutput1ToDetectionOutput8FiveArguments) { + // In this case num_classes attribute value is deduced using inputs shapes + std::vector attrs_v1_vector; + std::vector attrs_v8_vector; + Dimension N = 5; + Dimension num_prior_boxes = 15; + Dimension priors_batch_size = N; + Dimension num_classes = 23; + + create_attributes_vectors(attrs_v1_vector, attrs_v8_vector); + ASSERT_TRUE(attrs_v1_vector.size() == attrs_v8_vector.size()) << "Sizes of attribute test vectors must be equal"; + for (size_t ind = 0; ind < attrs_v1_vector.size(); ++ind) { + std::shared_ptr f(nullptr), f_ref(nullptr); + opset8::DetectionOutput::Attributes attributes_v8 = attrs_v8_vector[ind]; + opset1::DetectionOutput::Attributes attributes_v1 = attrs_v1_vector[ind]; + if (num_classes.is_static()) { + attributes_v1.num_classes = num_classes.get_length(); + } + + Dimension num_loc_classes = attributes_v8.share_location ? 1 : num_classes; + Dimension prior_box_size = attributes_v8.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, num_prior_boxes * num_loc_classes * 4}; + PartialShape class_preds_shape = {N, num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attributes_v8.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + PartialShape ad_class_preds_shape = {N, num_prior_boxes * 2}; + PartialShape ad_box_preds_shape = {N, num_prior_boxes * num_loc_classes * 4}; + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + auto ad_class_preds = + std::make_shared(ngraph::element::f32, ad_class_preds_shape); + auto ad_box_preds = std::make_shared(ngraph::element::f32, ad_box_preds_shape); + + auto detection_output_v1 = std::make_shared(box_logits, + class_preds, + proposals, + ad_class_preds, + ad_box_preds, + attributes_v1); + + f = std::make_shared( + ngraph::NodeVector{detection_output_v1}, + ngraph::ParameterVector{box_logits, class_preds, proposals, ad_class_preds, ad_box_preds}); + + ngraph::pass::Manager manager; + manager.register_pass(); + manager.run_passes(f); + } + + { + auto box_logits = std::make_shared(ngraph::element::f32, box_logits_shape); + auto class_preds = std::make_shared(ngraph::element::f32, class_preds_shape); + auto proposals = std::make_shared(ngraph::element::f32, proposals_shape); + auto ad_class_preds = + std::make_shared(ngraph::element::f32, ad_class_preds_shape); + auto ad_box_preds = std::make_shared(ngraph::element::f32, ad_box_preds_shape); + + auto detection_output_v8 = std::make_shared(box_logits, + class_preds, + proposals, + ad_class_preds, + ad_box_preds, + attributes_v8); + + f_ref = std::make_shared( + ngraph::NodeVector{detection_output_v8}, + ngraph::ParameterVector{box_logits, class_preds, proposals, ad_class_preds, ad_box_preds}); + } + auto res = compare_functions(f, f_ref); + ASSERT_TRUE(res.first) << res.second; + } +} diff --git a/inference-engine/tests/functional/inference_engine/transformations/nearest_neighbor_upsampling_fusion_test.cpp b/inference-engine/tests/functional/inference_engine/transformations/nearest_neighbor_upsampling_fusion_test.cpp new file mode 100644 index 00000000000..2016d8218cb --- /dev/null +++ b/inference-engine/tests/functional/inference_engine/transformations/nearest_neighbor_upsampling_fusion_test.cpp @@ -0,0 +1,158 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include + +#include +#include + +#include +#include +#include +#include +#include +#include +#include + +#include "common_test_utils/ngraph_test_utils.hpp" + +using namespace testing; + +TEST_F(TransformationTestsF, NearestNeighborUpsamplingFusionSpatial2D1) { + ngraph::Shape input_shape { 1, 120, 150, 32 }; + size_t input_rank = input_shape.size(); + std::vector new_spatial_shape { 240, 450 }; + std::vector scales_as_floats { 2.0f, 3.0f }; + std::vector constants_for_concat_1 { 1, 120, 1, 150, 1, 32 }; + std::vector constants_for_concat_2 { 1, 240, 450, 32 }; + ngraph::Shape mul_const_shape {1, 1, 2, 1, 3, 1}; + std::vector mul_const_value {1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f}; + { + auto input = std::make_shared(ngraph::element::f32, input_shape); + auto shape_node = std::make_shared(input); + + auto sslice_begin = ngraph::opset8::Constant::create(ngraph::element::i64, {1}, std::vector{0}); + auto sslice_end = ngraph::opset8::Constant::create(ngraph::element::i64, {1}, std::vector{1}); + std::vector begin_mask = {0}; + std::vector end_mask = {0}; + auto strided_slice_node = std::make_shared(shape_node, sslice_begin, sslice_end, begin_mask, end_mask); + + ngraph::OutputVector concat_1_inputs_vec(2 + 2 * (input_rank - 2)); + concat_1_inputs_vec[0] = strided_slice_node; + for (size_t i = 1; i < 2 + 2 * (input_rank - 2); ++i) { + const auto unsqueezed_const = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{constants_for_concat_1[i]}); + const auto unsqueeze_axis = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{0}); + const auto current_unsqueeze = std::make_shared(unsqueezed_const, unsqueeze_axis); + concat_1_inputs_vec[i] = current_unsqueeze; + } + auto concat_1 = std::make_shared(concat_1_inputs_vec, 0); + + auto reshape_1 = std::make_shared(input, concat_1, true); + + ngraph::OutputVector concat_2_inputs_vec(input_rank); + concat_2_inputs_vec[0] = strided_slice_node; + for (size_t i = 1; i < input_rank; ++i) { + const auto unsqueezed_const = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{constants_for_concat_2[i]}); + const auto unsqueeze_axis = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{0}); + const auto current_unsqueeze = std::make_shared(unsqueezed_const, unsqueeze_axis); + concat_2_inputs_vec[i] = current_unsqueeze; + } + auto concat_2 = std::make_shared(concat_2_inputs_vec, 0); + + const auto mul_const = ngraph::opset8::Constant::create(ngraph::element::f32, mul_const_shape, mul_const_value); + const auto mul = std::make_shared(reshape_1, mul_const); + + auto reshape_2 = std::make_shared(mul, concat_2, true); + function = std::make_shared(ngraph::NodeVector{ reshape_2 }, ngraph::ParameterVector{ input }); + manager.register_pass(); + } + { + ngraph::opset8::Interpolate::InterpolateAttrs attrs; + + attrs.mode = ngraph::opset8::Interpolate::InterpolateMode::NEAREST; + attrs.shape_calculation_mode = ngraph::opset8::Interpolate::ShapeCalcMode::SCALES; + attrs.nearest_mode = ngraph::opset8::Interpolate::NearestMode::ROUND_PREFER_FLOOR; + attrs.pads_begin = std::vector{0}; + attrs.pads_end = std::vector{0}; + attrs.antialias = false; + attrs.coordinate_transformation_mode = ngraph::opset8::Interpolate::CoordinateTransformMode::HALF_PIXEL; + attrs.cube_coeff = -0.75f; + + auto input = std::make_shared(ngraph::element::f32, input_shape); + auto sizes_node = ngraph::opset8::Constant::create(ngraph::element::i64, {new_spatial_shape.size()}, new_spatial_shape); + auto scales_node = ngraph::opset8::Constant::create(ngraph::element::f32, {scales_as_floats.size()}, scales_as_floats); + auto axes_node = ngraph::opset8::Constant::create(ngraph::element::i64, {2}, std::vector{1, 2}); + auto interpolate = std::make_shared(input, sizes_node, scales_node, axes_node, attrs); + function_ref = std::make_shared(ngraph::NodeVector{ interpolate }, ngraph::ParameterVector{ input }); + } +} + +TEST_F(TransformationTestsF, NearestNeighborUpsamplingFusionSpatial3D1) { + ngraph::Shape input_shape { 1, 130, 120, 85, 3 }; + size_t input_rank = input_shape.size(); + std::vector new_spatial_shape { 260, 360, 340 }; + std::vector scales_as_floats { 2.0f, 3.0, 4.0f }; + std::vector constants_for_concat_1 { 1, 130, 1, 120, 1, 85, 1, 3 }; + std::vector constants_for_concat_2 { 1, 260, 360, 340, 3 }; + ngraph::Shape mul_const_shape {1, 1, 2, 1, 3, 1, 4, 1}; + std::vector mul_const_value(24, 1.0f); + { + auto input = std::make_shared(ngraph::element::f32, input_shape); + auto shape_node = std::make_shared(input); + + auto sslice_begin = ngraph::opset8::Constant::create(ngraph::element::i64, {1}, std::vector{0}); + auto sslice_end = ngraph::opset8::Constant::create(ngraph::element::i64, {1}, std::vector{1}); + std::vector begin_mask = {0}; + std::vector end_mask = {0}; + auto strided_slice_node = std::make_shared(shape_node, sslice_begin, sslice_end, begin_mask, end_mask); + + ngraph::OutputVector concat_1_inputs_vec(2 + 2 * (input_rank - 2)); + concat_1_inputs_vec[0] = strided_slice_node; + for (size_t i = 1; i < 2 + 2 * (input_rank - 2); ++i) { + const auto unsqueezed_const = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{constants_for_concat_1[i]}); + const auto unsqueeze_axis = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{0}); + const auto current_unsqueeze = std::make_shared(unsqueezed_const, unsqueeze_axis); + concat_1_inputs_vec[i] = current_unsqueeze; + } + auto concat_1 = std::make_shared(concat_1_inputs_vec, 0); + + auto reshape_1 = std::make_shared(input, concat_1, true); + + ngraph::OutputVector concat_2_inputs_vec(input_rank); + concat_2_inputs_vec[0] = strided_slice_node; + for (size_t i = 1; i < input_rank; ++i) { + const auto unsqueezed_const = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{constants_for_concat_2[i]}); + const auto unsqueeze_axis = ngraph::opset8::Constant::create(ngraph::element::i64, {}, std::vector{0}); + const auto current_unsqueeze = std::make_shared(unsqueezed_const, unsqueeze_axis); + concat_2_inputs_vec[i] = current_unsqueeze; + } + auto concat_2 = std::make_shared(concat_2_inputs_vec, 0); + + const auto mul_const = ngraph::opset8::Constant::create(ngraph::element::f32, mul_const_shape, mul_const_value); + const auto mul = std::make_shared(reshape_1, mul_const); + + auto reshape_2 = std::make_shared(mul, concat_2, true); + function = std::make_shared(ngraph::NodeVector{ reshape_2 }, ngraph::ParameterVector{ input }); + manager.register_pass(); + } + { + ngraph::opset8::Interpolate::InterpolateAttrs attrs; + + attrs.mode = ngraph::opset8::Interpolate::InterpolateMode::NEAREST; + attrs.shape_calculation_mode = ngraph::opset8::Interpolate::ShapeCalcMode::SCALES; + attrs.nearest_mode = ngraph::opset8::Interpolate::NearestMode::ROUND_PREFER_FLOOR; + attrs.pads_begin = std::vector{0}; + attrs.pads_end = std::vector{0}; + attrs.antialias = false; + attrs.coordinate_transformation_mode = ngraph::opset8::Interpolate::CoordinateTransformMode::HALF_PIXEL; + attrs.cube_coeff = -0.75f; + + auto input = std::make_shared(ngraph::element::f32, input_shape); + auto sizes_node = ngraph::opset8::Constant::create(ngraph::element::i64, {new_spatial_shape.size()}, new_spatial_shape); + auto scales_node = ngraph::opset8::Constant::create(ngraph::element::f32, {scales_as_floats.size()}, scales_as_floats); + auto axes_node = ngraph::opset8::Constant::create(ngraph::element::i64, {3}, std::vector{1, 2, 3}); + auto interpolate = std::make_shared(input, sizes_node, scales_node, axes_node, attrs); + function_ref = std::make_shared(ngraph::NodeVector{ interpolate }, ngraph::ParameterVector{ input }); + } +} diff --git a/inference-engine/tests/functional/inference_engine/transformations/primitives_priority_test.cpp b/inference-engine/tests/functional/inference_engine/transformations/primitives_priority_test.cpp index 9df0ba186fc..e87c1669eef 100644 --- a/inference-engine/tests/functional/inference_engine/transformations/primitives_priority_test.cpp +++ b/inference-engine/tests/functional/inference_engine/transformations/primitives_priority_test.cpp @@ -50,7 +50,7 @@ TEST(TransformationTests, ConvBiasFusion) { for (auto & op : nGraph->get_ops()) { if (auto conv = std::dynamic_pointer_cast(op)) { auto & rtInfo = conv->get_rt_info(); - rtInfo["PrimitivesPriority"] = std::make_shared>("test"); + rtInfo[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority("test"); pp[op->get_friendly_name()] = "test"; } } diff --git a/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_detection_output.cpp b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_detection_output.cpp new file mode 100644 index 00000000000..b421540d689 --- /dev/null +++ b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_detection_output.cpp @@ -0,0 +1,69 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "single_layer_tests/experimental_detectron_detection_output.hpp" + +using namespace ov::test; +using namespace ov::test::subgraph; + +namespace { + +const std::vector score_threshold = { 0.01000000074505806f }; + +const std::vector nms_threshold = { 0.2f }; + +//// specifies maximal delta of logarithms for width and height +const std::vector max_delta_log_wh = { 2.0f }; + +// specifies number of detected classes +const std::vector num_classes = { 2 }; + +// specifies maximal number of detections per class +const std::vector post_nms_count = { 500 }; + +// specifies maximual number of detections per image +const std::vector max_detections_per_image = { 5 }; + +// a flag specifies whether to delete background classes or not +// `true` means background classes should be deleted, +// `false` means background classes shouldn't be deleted. +const std::vector class_agnostic_box_regression = { true }; + +// specifies deltas of weights +const std::vector> deltas_weights = { {10.0f, 10.0f, 5.0f, 5.0f} }; + +const std::vector> inputShapes = { + // inputRois / inputDeltas / inputScores / inputImInfos + static_shapes_to_test_representation({{16, 4}, {16, 8}, {16, 2}, {1, 3}}), + { + {{-1, -1}, {{16, 4}, {16, 4}}}, + {{-1, -1}, {{16, 8}, {16, 8}}}, + {{-1, -1}, {{16, 2}, {16, 2}}}, + {{-1, -1}, {{1, 3}, {1, 3}}} + }, + { + {{{16, 32}, {4, 8}}, {{16, 4}, {16, 4}}}, + {{{16, 32}, {8, 16}}, {{16, 8}, {16, 8}}}, + {{{16, 32}, {2, 4}}, {{16, 2}, {16, 2}}}, + {{{1, 2}, {3, 6}}, {{1, 3}, {1, 3}}} + } +}; + +INSTANTIATE_TEST_SUITE_P(smoke_ExperimentalDetectronDetectionOutput, ExperimentalDetectronDetectionOutputLayerTest, + ::testing::Combine( + ::testing::ValuesIn(inputShapes), + ::testing::ValuesIn(score_threshold), + ::testing::ValuesIn(nms_threshold), + ::testing::ValuesIn(max_delta_log_wh), + ::testing::ValuesIn(num_classes), + ::testing::ValuesIn(post_nms_count), + ::testing::ValuesIn(max_detections_per_image), + ::testing::ValuesIn(class_agnostic_box_regression), + ::testing::ValuesIn(deltas_weights), + ::testing::Values(ov::element::Type_t::f32), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ExperimentalDetectronDetectionOutputLayerTest::getTestCaseName); + +} // namespace diff --git a/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_generate_proposals_single_image.cpp b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_generate_proposals_single_image.cpp new file mode 100644 index 00000000000..d4c704eda49 --- /dev/null +++ b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_generate_proposals_single_image.cpp @@ -0,0 +1,128 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "single_layer_tests/experimental_detectron_generate_proposals_single_image.hpp" +#include "common_test_utils/data_utils.hpp" + +using namespace ov::test; +using namespace ov::test::subgraph; + +namespace { + +const std::vector min_size = { 0 }; +const std::vector nms_threshold = { 0.699999988079071 }; +const std::vector post_nms_count = { 6 }; +const std::vector pre_nms_count = { 1000 }; + +const std::vector>> inputTensors = { + { + "empty", + { + // 3 + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{3}, {1.0f, 1.0f, 1.0f}), + // 36 x 4 = 144 + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{36, 4}, { + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f}), + // 12 x 2 x 6 = 144 + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{12, 2, 6}, { + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f}), + // {3 x 2 x 6} = 36 + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{3, 2, 6}, { + 5.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 4.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 8.0f, 1.0f}) + } + }, + { + "filled", + { + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{3}, {150.0, 150.0, 1.0}), + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{36, 4}, { + 12.0, 68.0, 102.0, 123.0, 46.0, 80.0, 79.0, 128.0, 33.0, 71.0, 127.0, 86.0, 33.0, 56.0, 150.0, 73.0, + 5.0, 41.0, 93.0, 150.0, 74.0, 66.0, 106.0, 115.0, 17.0, 37.0, 87.0, 150.0, 31.0, 27.0, 150.0, 39.0, + 29.0, 23.0, 112.0, 123.0, 41.0, 37.0, 103.0, 150.0, 8.0, 46.0, 98.0, 111.0, 7.0, 69.0, 114.0, 150.0, + 70.0, 21.0, 150.0, 125.0, 54.0, 19.0, 132.0, 68.0, 62.0, 8.0, 150.0, 101.0, 57.0, 81.0, 150.0, 97.0, + 79.0, 29.0, 109.0, 130.0, 12.0, 63.0, 100.0, 150.0, 17.0, 33.0, 113.0, 150.0, 90.0, 78.0, 150.0, 111.0, + 47.0, 68.0, 150.0, 71.0, 66.0, 103.0, 111.0, 150.0, 4.0, 17.0, 112.0, 94.0, 12.0, 8.0, 119.0, 98.0, + 54.0, 56.0, 120.0, 150.0, 56.0, 29.0, 150.0, 31.0, 42.0, 3.0, 139.0, 92.0, 41.0, 65.0, 150.0, 130.0, + 49.0, 13.0, 143.0, 30.0, 40.0, 60.0, 150.0, 150.0, 23.0, 73.0, 24.0, 115.0, 56.0, 84.0, 107.0, 108.0, + 63.0, 8.0, 142.0, 125.0, 78.0, 37.0, 93.0, 144.0, 40.0, 34.0, 150.0, 46.0, 30.0, 21.0, 150.0, 120.0}), + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{12, 2, 6}, { + 9.062256, 10.883133, 9.8441105, 12.694285, 0.41781136, 8.749107, 14.990341, 6.587644, 1.4206103, + 13.299262, 12.432549, 2.736371, 0.22732796, 6.3361835, 12.268727, 2.1009045, 4.771589, 2.5131326, + 5.610736, 9.3604145, 4.27379, 8.317948, 0.60510135, 6.7446275, 1.0207708, 1.1352817, 1.5785321, + 1.718335, 1.8093798, 0.99247587, 1.3233583, 1.7432803, 1.8534478, 1.2593061, 1.7394226, 1.7686696, + 1.647999, 1.7611449, 1.3119122, 0.03007332, 1.1106564, 0.55669737, 0.2546148, 1.9181818, 0.7134989, + 2.0407224, 1.7211134, 1.8565536, 14.562747, 2.8786168, 0.5927796, 0.2064463, 7.6794515, 8.672126, + 10.139171, 8.002429, 7.002932, 12.6314945, 10.550842, 0.15784842, 0.3194304, 10.752157, 3.709805, + 11.628928, 0.7136225, 14.619964, 15.177284, 2.2824087, 15.381494, 0.16618137, 7.507227, 11.173228, + 0.4923559, 1.8227729, 1.4749299, 1.7833921, 1.2363617, -0.23659119, 1.5737582, 1.779316, 1.9828427, + 1.0482665, 1.4900246, 1.3563544, 1.5341306, 0.7634312, 4.6216766e-05, 1.6161222, 1.7512476, 1.9363779, + 0.9195784, 1.4906164, -0.03244795, 0.681073, 0.6192401, 1.8033613, 14.146055, 3.4043705, 15.292292, + 3.5295358, 11.138999, 9.952057, 5.633434, 12.114562, 9.427372, 12.384038, 9.583308, 8.427233, + 15.293704, 3.288159, 11.64898, 9.350885, 2.0037227, 13.523184, 4.4176426, 6.1057625, 14.400079, + 8.248259, 11.815807, 15.713364, 1.0023532, 1.3203261, 1.7100681, 0.7407832, 1.09448, 1.7188418, + 1.4412547, 1.4862992, 0.74790007, 0.31571656, 0.6398838, 2.0236106, 1.1869069, 1.7265586, 1.2624544, + 0.09934269, 1.3508598, 0.85212964, -0.38968498, 1.7059708, 1.6533034, 1.7400402, 1.8123854, -0.43063712}), + CommonTestUtils::create_tensor(ov::element::f32, ov::Shape{3, 2, 6}, { + 0.7719922, 0.35906568, 0.29054508, 0.18124384, 0.5604661, 0.84750974, 0.98948747, 0.009793862, 0.7184191, + 0.5560748, 0.6952493, 0.6732593, 0.3306898, 0.6790913, 0.41128764, 0.34593266, 0.94296855, 0.7348507, + 0.24478768, 0.94024557, 0.05405676, 0.06466125, 0.36244348, 0.07942984, 0.10619422, 0.09412837, 0.9053611, + 0.22870538, 0.9237487, 0.20986171, 0.5067282, 0.29709867, 0.53138554, 0.189101, 0.4786443, 0.88421875}), + } + } +}; + +const std::vector> dynamicInputShape = { + // im_info / anchors / deltas / scores + static_shapes_to_test_representation({{3}, {36, 4}, {12, 2, 6}, {3, 2, 6}}), + { + {{-1}, {{3}}}, + {{-1, -1}, {{36, 4}}}, + {{-1, -1, -1}, {{12, 2, 6}}}, + {{-1, -1, -1}, {{3, 2, 6}}} + }, + { + {{{3, 6}}, {{3}}}, + {{{36, 72}, {4, 8}}, {{36, 4}}}, + {{{12, 24}, {2, 4}, {6, 12}}, {{12, 2, 6}}}, + {{{3, 6}, {2, 4}, {6, 12}}, {{3, 2, 6}}} + } +}; + +INSTANTIATE_TEST_SUITE_P( + smoke_ExperimentalDetectronGenerateProposalsSingleImageLayerTest, + ExperimentalDetectronGenerateProposalsSingleImageLayerTest, + ::testing::Combine( + ::testing::ValuesIn(dynamicInputShape), + ::testing::ValuesIn(min_size), + ::testing::ValuesIn(nms_threshold), + ::testing::ValuesIn(post_nms_count), + ::testing::ValuesIn(pre_nms_count), + ::testing::ValuesIn(inputTensors), + ::testing::Values(ov::element::Type_t::f32), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ExperimentalDetectronGenerateProposalsSingleImageLayerTest::getTestCaseName); +} // namespace diff --git a/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_prior_grid_generator.cpp b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_prior_grid_generator.cpp new file mode 100644 index 00000000000..5e8b13c5466 --- /dev/null +++ b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/experimental_detectron_prior_grid_generator.cpp @@ -0,0 +1,106 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "single_layer_tests/experimental_detectron_prior_grid_generator.hpp" +#include "common_test_utils/data_utils.hpp" + +using namespace ov::test; +using namespace ov::test::subgraph; + +namespace { + +const std::vector params = { + // flatten = true (output tensor is 2D) + { + {true, 0, 0, 4.0f, 4.0f}, + ov::test::static_shapes_to_test_representation({{3, 4}, {1, 16, 4, 5}, {1, 3, 100, 200}}) + }, + // task #72587 + //{ + // {true, 3, 6, 64.0f, 64.0f}, + // ov::test::static_shapes_to_test_representation({{3, 4}, {1, 16, 100, 100}, {1, 3, 100, 200}}) + //}, + { + {true, 0, 0, 4.0f, 4.0f}, + { + // priors + {{-1, -1}, {{3, 4}, {3, 4}}}, + // feature_map + {{-1, -1, -1, -1}, {{1, 16, 4, 5}, {1, 16, 100, 100}}}, + // im_data + {{-1, -1, -1, -1}, {{1, 3, 100, 200}, {1, 3, 100, 200}}} + } + }, + // flatten = false (output tensor is 4D) + { + {false, 0, 0, 8.0f, 8.0f}, + ov::test::static_shapes_to_test_representation({{3, 4}, {1, 16, 3, 7}, {1, 3, 100, 200}}) + }, + // task #72587 + //{ + // {false, 5, 3, 32.0f, 32.0f}, + // ov::test::static_shapes_to_test_representation({{3, 4}, {1, 16, 100, 100}, {1, 3, 100, 200}}) + //}, + { + {false, 0, 0, 8.0f, 8.0f}, + { + // priors + {{-1, -1}, {{3, 4}, {3, 4}}}, + // feature_map + {{-1, -1, -1, -1}, {{1, 16, 3, 7}, {1, 16, 100, 100}}}, + // im_data + {{-1, -1, -1, -1}, {{1, 3, 100, 200}, {1, 3, 100, 200}}} + } + } +}; + +std::vector>> inputTensors = { + { + "test#1", + { + CommonTestUtils::create_tensor( + ov::element::f32, + ov::Shape{3, 4}, + {-24.5, -12.5, 24.5, 12.5, -16.5, -16.5, 16.5, 16.5, -12.5, -24.5, 12.5, 24.5}) + } + }, + { + "test#2", + { + CommonTestUtils::create_tensor( + ov::element::f32, + ov::Shape{3, 4}, + {-44.5, -24.5, 44.5, 24.5, -32.5, -32.5, 32.5, 32.5, -24.5, -44.5, 24.5, 44.5}) + } + }, + { + "test#3", + { + CommonTestUtils::create_tensor( + ov::element::f32, + ov::Shape{3, 4}, + {-364.5, -184.5, 364.5, 184.5, -256.5, -256.5, 256.5, 256.5, -180.5, -360.5, 180.5, 360.5}) + } + }, + { + "test#4", + { + CommonTestUtils::create_tensor( + ov::element::f32, + ov::Shape{3, 4}, + {-180.5, -88.5, 180.5, 88.5, -128.5, -128.5, 128.5, 128.5, -92.5, -184.5, 92.5, 184.5}) + } + } +}; + +INSTANTIATE_TEST_SUITE_P(smoke_ExperimentalDetectronPriorGridGenerator, ExperimentalDetectronPriorGridGeneratorLayerTest, + ::testing::Combine( + ::testing::ValuesIn(params), + ::testing::ValuesIn(inputTensors), + ::testing::Values(ov::element::Type_t::f32), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ExperimentalDetectronPriorGridGeneratorLayerTest::getTestCaseName); + +} // namespace diff --git a/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/pooling.cpp b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/pooling.cpp index a3053fb6809..7b7e79da274 100644 --- a/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/pooling.cpp +++ b/inference-engine/tests/functional/plugin/cpu/shared_tests_instances/single_layer_tests/pooling.cpp @@ -351,8 +351,174 @@ INSTANTIATE_TEST_SUITE_P(smoke_AvgPool_SameLowerPad_CeilRounding_5Dinput, Poolin ::testing::Values(CommonTestUtils::DEVICE_CPU)), PoolingLayerTest::getTestCaseName); +////* ========== Max Pooling V8 ========== */ + +const std::vector> dilation = {{1, 1}, {2, 2}}; +const std::vector> dilation3D = {{1, 1, 1}, {2, 2, 2}}; + +/* ========== Explicit Pad Floor Rounding ========== */ +const auto maxPoolv8_ExplicitPad_FloorRounding_Params = ::testing::Combine( + ::testing::ValuesIn(kernels), + ::testing::ValuesIn(strides), + ::testing::ValuesIn(dilation), + ::testing::ValuesIn(padBegins), + ::testing::ValuesIn(padEnds), + ::testing::Values(ngraph::op::RoundingType::FLOOR), + ::testing::Values(ngraph::op::PadType::EXPLICIT) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolV8_ExplicitPad_FloorRounding, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_ExplicitPad_FloorRounding_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========== Same Upper Pad Floor Rounding ========== */ +const auto maxPoolv8_SameUpperPad_FloorRounding_Params = ::testing::Combine( + ::testing::ValuesIn(kernels), + ::testing::ValuesIn(strides), + ::testing::ValuesIn(dilation), + ::testing::ValuesIn(padBegins), + ::testing::ValuesIn(padEnds), + ::testing::Values(ngraph::op::RoundingType::FLOOR), + ::testing::Values(ngraph::op::PadType::SAME_UPPER) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_SameUpperPad_FloorRounding, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_SameUpperPad_FloorRounding_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========== Same Lower Pad Floor Rounding ========== */ +const auto maxPoolv8_SameLowerPad_FloorRounding_Params = ::testing::Combine( + ::testing::ValuesIn(kernels), + ::testing::ValuesIn(strides), + ::testing::ValuesIn(dilation), + ::testing::ValuesIn(padBegins), + ::testing::ValuesIn(padEnds), + ::testing::Values(ngraph::op::RoundingType::FLOOR), + ::testing::Values(ngraph::op::PadType::SAME_LOWER) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_SameLowerPad_FloorRounding, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_SameLowerPad_FloorRounding_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========= Explicit Pad Floor Rounding 5D input========== */ +const auto maxPoolv8_ExplicitPad_FloorRounding_5Dinput_Params = ::testing::Combine( + ::testing::ValuesIn(kernel3D), + ::testing::ValuesIn(strides3D), + ::testing::Values(dilation3D[0]), + ::testing::ValuesIn(padBegins3D), + ::testing::ValuesIn(padEnds3D), + ::testing::Values(ngraph::op::RoundingType::FLOOR), + ::testing::Values(ngraph::op::PadType::EXPLICIT) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_ExplicitPad_FloorRounding_5Dinput, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_ExplicitPad_FloorRounding_5Dinput_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({32, 32, 2, 2, 2})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========= Same Upper Pad Floor Rounding 5D input========== */ +const auto maxPoolv8_SameUpperPad_FloorRounding_5Dinput_Params = ::testing::Combine( + ::testing::ValuesIn(kernel3D), + ::testing::ValuesIn(strides3D), + ::testing::ValuesIn(dilation3D), + ::testing::ValuesIn(padBegins3D), + ::testing::ValuesIn(padEnds3D), + ::testing::Values(ngraph::op::RoundingType::FLOOR), + ::testing::Values(ngraph::op::PadType::SAME_UPPER) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_SameUpperPad_FloorRounding_5Dinput, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_SameUpperPad_FloorRounding_5Dinput_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({32, 32, 2, 2, 2})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========= Same Lower Pad Ceil Rounding 5D input========== */ +const auto maxPoolv8_SameLowerPad_CeilRounding_5Dinput_Params = ::testing::Combine( + ::testing::ValuesIn(kernel3D), + ::testing::ValuesIn(strides3D), + ::testing::ValuesIn(dilation3D), + ::testing::ValuesIn(padBegins3D), + ::testing::ValuesIn(padEnds3D), + ::testing::Values(ngraph::op::RoundingType::CEIL), + ::testing::Values(ngraph::op::PadType::SAME_LOWER) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_SameLowerPad_CeilRounding_5Dinput, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_SameLowerPad_CeilRounding_5Dinput_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({32, 32, 2, 2, 2})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + +/* ========= Explicit Pad Ceil Rounding ========== */ +const auto maxPoolv8_ExplicitPad_CeilRounding_Params = ::testing::Combine( + ::testing::ValuesIn(kernels), + ::testing::ValuesIn(strides), + ::testing::ValuesIn(dilation), + ::testing::ValuesIn(padBegins), + ::testing::ValuesIn(padEnds), + ::testing::Values(ngraph::op::RoundingType::CEIL), + ::testing::Values(ngraph::op::PadType::EXPLICIT) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolv8_ExplicitPad_CeilRounding, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_ExplicitPad_CeilRounding_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + ////* ========== Avg and Max Polling Cases ========== */ -/* ========== Valid Pad Rounding Not Applicable ========== */ +/* ========== Valid Pad Rounding Not Applicable ========== */ const auto allPools_ValidPad_Params = ::testing::Combine( ::testing::Values(ngraph::helpers::PoolingTypes::MAX, ngraph::helpers::PoolingTypes::AVG), ::testing::ValuesIn(kernels), @@ -366,14 +532,37 @@ const auto allPools_ValidPad_Params = ::testing::Combine( ); INSTANTIATE_TEST_SUITE_P(smoke_MAX_and_AVGPool_ValidPad, PoolingLayerTest, - ::testing::Combine( - allPools_ValidPad_Params, - ::testing::ValuesIn(netPrecisions), - ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), - ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), - ::testing::Values(InferenceEngine::Layout::ANY), - ::testing::Values(InferenceEngine::Layout::ANY), - ::testing::Values(std::vector({1, 3, 30, 30})), - ::testing::Values(CommonTestUtils::DEVICE_CPU)), - PoolingLayerTest::getTestCaseName); + ::testing::Combine( + allPools_ValidPad_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + PoolingLayerTest::getTestCaseName); + +const auto maxPoolv8_ValidPad_Params = ::testing::Combine( + ::testing::ValuesIn(kernels), + ::testing::ValuesIn(strides), + ::testing::ValuesIn(dilation), + ::testing::Values(std::vector({0, 0})), + ::testing::Values(std::vector({0, 0})), + ::testing::Values(ngraph::op::RoundingType::FLOOR), // placeholder value - Rounding Type not applicable for Valid pad type + ::testing::Values(ngraph::op::PadType::VALID) +); + +INSTANTIATE_TEST_SUITE_P(smoke_MAXPoolv8_ValidPad, MaxPoolingV8LayerTest, + ::testing::Combine( + maxPoolv8_ValidPad_Params, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Precision::UNSPECIFIED), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(InferenceEngine::Layout::ANY), + ::testing::Values(std::vector({1, 3, 30, 30})), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + MaxPoolingV8LayerTest::getTestCaseName); + } // namespace diff --git a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/deformable_convolution.cpp b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/deformable_convolution.cpp index 5b7ea755f5b..3371cfc3878 100644 --- a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/deformable_convolution.cpp +++ b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/deformable_convolution.cpp @@ -226,6 +226,20 @@ const std::vector netPrecisions = { InferenceEngine::Precision::FP32 }; +const auto defConvSpecificParams_Smoke = ::testing::Combine( + ::testing::ValuesIn(std::vector { + true, + false + }), // with_bilinear_interpolation_pad + ::testing::ValuesIn(std::vector { + true, + false + }), // with_modulation + ::testing::ValuesIn(std::vector { + OffsetType::REAL_MISC, + }) // offset type +); + const auto defConvSpecificParams = ::testing::Combine( ::testing::ValuesIn(std::vector { true, @@ -251,8 +265,8 @@ std::vector padTypes = { const auto spParams1 = ::testing::Combine( ::testing::Values(1), // batch - ::testing::Values(std::vector({68, 68})), // in. spat. shape - ::testing::Values(std::vector({66, 66})), // off. spat. shape + ::testing::Values(std::vector({34, 34})), // in. spat. shape + ::testing::Values(std::vector({32, 32})), // off. spat. shape ::testing::Values(std::vector({3, 3})), // ker. spat. shape ::testing::ValuesIn(padTypes), // pad. type ::testing::Values(std::vector({0, 0})), // pad. begin @@ -308,6 +322,52 @@ const auto chParamsMulGr = ::testing::Combine( ::testing::ValuesIn(std::vector {3, 7}), // in. ch. per gr. ::testing::ValuesIn(std::vector {3, 7})); // out. ch. per gr. +const auto params1_Smoke = ::testing::Combine( + ::testing::Combine( + spParams1, + chParamsSingleGr, + defConvSpecificParams_Smoke, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ::testing::ValuesIn(filterCPUInfoForDevice())); +const auto params2_Smoke = ::testing::Combine( + ::testing::Combine( + spParams2, + chParamsSingleGr, + defConvSpecificParams_Smoke, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ::testing::ValuesIn(filterCPUInfoForDevice())); +const auto params3_Smoke = ::testing::Combine( + ::testing::Combine( + spParams3, + chParamsSingleGr, + defConvSpecificParams_Smoke, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ::testing::ValuesIn(filterCPUInfoForDevice())); +const auto params4_Smoke = ::testing::Combine( + ::testing::Combine( + spParams4, + chParamsSingleGr, + defConvSpecificParams_Smoke, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ::testing::ValuesIn(filterCPUInfoForDevice())); +const auto params5_Smoke = ::testing::Combine( + ::testing::Combine( + spParams4, + chParamsMulGr, + defConvSpecificParams_Smoke, + ::testing::ValuesIn(netPrecisions), + ::testing::Values(CommonTestUtils::DEVICE_CPU)), + ::testing::ValuesIn(filterCPUInfoForDevice(true))); +INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest1, DefConvLayerCPUTest, params1_Smoke, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest2, DefConvLayerCPUTest, params2_Smoke, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest3, DefConvLayerCPUTest, params3_Smoke, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest4, DefConvLayerCPUTest, params4_Smoke, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest5, DefConvLayerCPUTest, params5_Smoke, DefConvLayerCPUTest::getTestCaseName); + const auto params1 = ::testing::Combine( ::testing::Combine( spParams1, @@ -348,10 +408,11 @@ const auto params5 = ::testing::Combine( ::testing::ValuesIn(netPrecisions), ::testing::Values(CommonTestUtils::DEVICE_CPU)), ::testing::ValuesIn(filterCPUInfoForDevice(true))); -INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest1, DefConvLayerCPUTest, params1, DefConvLayerCPUTest::getTestCaseName); -INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest2, DefConvLayerCPUTest, params2, DefConvLayerCPUTest::getTestCaseName); -INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest3, DefConvLayerCPUTest, params3, DefConvLayerCPUTest::getTestCaseName); -INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest4, DefConvLayerCPUTest, params4, DefConvLayerCPUTest::getTestCaseName); -INSTANTIATE_TEST_SUITE_P(smoke_DefConvLayoutTest5, DefConvLayerCPUTest, params5, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(DefConvLayoutTest1, DefConvLayerCPUTest, params1, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(DefConvLayoutTest2, DefConvLayerCPUTest, params2, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(DefConvLayoutTest3, DefConvLayerCPUTest, params3, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(DefConvLayoutTest4, DefConvLayerCPUTest, params4, DefConvLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(DefConvLayoutTest5, DefConvLayerCPUTest, params5, DefConvLayerCPUTest::getTestCaseName); + } // namespace } // namespace CPULayerTestsDefinitions diff --git a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/interpolate.cpp b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/interpolate.cpp index 7b10c249f1b..c7681435f29 100644 --- a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/interpolate.cpp +++ b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/interpolate.cpp @@ -289,7 +289,12 @@ std::vector filterCPUInfoForDevice() { return resCPUParams; } /* ========== */ -const std::vector coordinateTransformModes = { +const std::vector coordinateTransformModes_Smoke = { + ngraph::op::v4::Interpolate::CoordinateTransformMode::HALF_PIXEL, + ngraph::op::v4::Interpolate::CoordinateTransformMode::ASYMMETRIC, +}; + +const std::vector coordinateTransformModes_Full = { ngraph::op::v4::Interpolate::CoordinateTransformMode::TF_HALF_PIXEL_FOR_NN, ngraph::op::v4::Interpolate::CoordinateTransformMode::PYTORCH_HALF_PIXEL, ngraph::op::v4::Interpolate::CoordinateTransformMode::HALF_PIXEL, @@ -297,12 +302,13 @@ const std::vector coordina ngraph::op::v4::Interpolate::CoordinateTransformMode::ALIGN_CORNERS, }; -const std::vector shapeCalculationMode = { - ngraph::op::v4::Interpolate::ShapeCalcMode::SIZES, - ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, +const std::vector nearestModes_Smoke = { + ngraph::op::v4::Interpolate::NearestMode::SIMPLE, + ngraph::op::v4::Interpolate::NearestMode::ROUND_PREFER_FLOOR, + ngraph::op::v4::Interpolate::NearestMode::FLOOR, }; -const std::vector nearestModes = { +const std::vector nearestModes_Full = { ngraph::op::v4::Interpolate::NearestMode::SIMPLE, ngraph::op::v4::Interpolate::NearestMode::ROUND_PREFER_FLOOR, ngraph::op::v4::Interpolate::NearestMode::FLOOR, @@ -351,7 +357,7 @@ const std::vector> defaultAxes4D = { {0, 1, 2, 3} }; -const std::vector shapeParams4D = { +const std::vector shapeParams4D_Smoke = { ShapeParams{ ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, InputShape{{}, {{1, 11, 4, 4}}}, @@ -366,20 +372,6 @@ const std::vector shapeParams4D = { {{1, 11, 5, 6}}, defaultAxes4D.front() }, - ShapeParams{ - ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, - InputShape{{-1, {2, 20}, -1, -1}, {{1, 11, 4, 4}, {2, 7, 6, 5}}}, - ngraph::helpers::InputLayerType::CONSTANT, - {{1.f, 1.f, 1.25f, 1.5f}}, - defaultAxes4D.front() - }, - ShapeParams{ - ngraph::op::v4::Interpolate::ShapeCalcMode::SIZES, - InputShape{{-1, {2, 20}, -1, -1}, {{1, 11, 4, 4}, {1, 11, 5, 5}}}, - ngraph::helpers::InputLayerType::CONSTANT, - {{1, 11, 5, 6}}, - defaultAxes4D.front() - }, ShapeParams{ ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, InputShape{{-1, {2, 20}, -1, -1}, {{1, 11, 4, 4}, {2, 7, 6, 5}}}, @@ -396,10 +388,36 @@ const std::vector shapeParams4D = { } }; -const auto interpolateCasesNN = ::testing::Combine( +const std::vector shapeParams4D_Full = { + ShapeParams{ + ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, + InputShape{{-1, {2, 20}, -1, -1}, {{1, 11, 4, 4}, {2, 7, 6, 5}}}, + ngraph::helpers::InputLayerType::CONSTANT, + {{1.f, 1.f, 1.25f, 1.5f}}, + defaultAxes4D.front() + }, + ShapeParams{ + ngraph::op::v4::Interpolate::ShapeCalcMode::SIZES, + InputShape{{-1, {2, 20}, -1, -1}, {{1, 11, 4, 4}, {1, 11, 5, 5}}}, + ngraph::helpers::InputLayerType::CONSTANT, + {{1, 11, 5, 6}}, + defaultAxes4D.front() + } +}; + +const auto interpolateCasesNN_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::nearest), - ::testing::ValuesIn(coordinateTransformModes), - ::testing::ValuesIn(nearestModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(nearestModes_Smoke), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(cubeCoefs)); + +const auto interpolateCasesNN_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::nearest), + ::testing::ValuesIn(coordinateTransformModes_Full), + ::testing::ValuesIn(nearestModes_Full), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads4D), ::testing::ValuesIn(pads4D), @@ -407,14 +425,24 @@ const auto interpolateCasesNN = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateNN_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesNN, - ::testing::ValuesIn(shapeParams4D), + interpolateCasesNN_Smoke, + ::testing::ValuesIn(shapeParams4D_Smoke), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice()), ::testing::ValuesIn(interpolateFusingParamsSet), ::testing::ValuesIn(filterAdditionalConfig())), InterpolateLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(InterpolateNN_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesNN_Full, + ::testing::ValuesIn(shapeParams4D_Full), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + const std::vector shapeParams4D_fixed_C = { ShapeParams{ ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, @@ -434,7 +462,7 @@ const std::vector shapeParams4D_fixed_C = { INSTANTIATE_TEST_SUITE_P(smoke_InterpolateNN_Layout_PerChannelFuse_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesNN, + interpolateCasesNN_Smoke, ::testing::ValuesIn(shapeParams4D_fixed_C), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice()), @@ -442,9 +470,28 @@ INSTANTIATE_TEST_SUITE_P(smoke_InterpolateNN_Layout_PerChannelFuse_Test, Interpo ::testing::ValuesIn(filterAdditionalConfig())), InterpolateLayerCPUTest::getTestCaseName); -const auto interpolateCasesLinearOnnx = ::testing::Combine( +INSTANTIATE_TEST_SUITE_P(InterpolateNN_Layout_PerChannelFuse_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesNN_Full, + ::testing::ValuesIn(shapeParams4D_fixed_C), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice()), + ::testing::Values(fusingFakeQuantizePerChannelRelu), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +const auto interpolateCasesLinearOnnx_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear_onnx), - ::testing::ValuesIn(coordinateTransformModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(defNearestModes), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(cubeCoefs)); + +const auto interpolateCasesLinearOnnx_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear_onnx), + ::testing::ValuesIn(coordinateTransformModes_Full), ::testing::ValuesIn(defNearestModes), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads4D), @@ -453,17 +500,36 @@ const auto interpolateCasesLinearOnnx = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateLinearOnnx_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesLinearOnnx, - ::testing::ValuesIn(shapeParams4D), + interpolateCasesLinearOnnx_Smoke, + ::testing::ValuesIn(shapeParams4D_Smoke), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice()), ::testing::ValuesIn(interpolateFusingParamsSet), ::testing::ValuesIn(filterAdditionalConfig())), InterpolateLayerCPUTest::getTestCaseName); -const auto interpolateCasesLinear = ::testing::Combine( +INSTANTIATE_TEST_SUITE_P(InterpolateLinearOnnx_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesLinearOnnx_Full, + ::testing::ValuesIn(shapeParams4D_Full), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +const auto interpolateCasesLinear_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear), - ::testing::ValuesIn(coordinateTransformModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(defNearestModes), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(cubeCoefs)); + +const auto interpolateCasesLinear_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear), + ::testing::ValuesIn(coordinateTransformModes_Full), ::testing::ValuesIn(defNearestModes), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads4D), @@ -472,17 +538,36 @@ const auto interpolateCasesLinear = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateLinear_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesLinear, - ::testing::ValuesIn(shapeParams4D), + interpolateCasesLinear_Smoke, + ::testing::ValuesIn(shapeParams4D_Smoke), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice()), ::testing::ValuesIn(interpolateFusingParamsSet), ::testing::ValuesIn(filterAdditionalConfig())), InterpolateLayerCPUTest::getTestCaseName); -const auto interpolateCasesCubic = ::testing::Combine( +INSTANTIATE_TEST_SUITE_P(InterpolateLinear_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesLinear_Full, + ::testing::ValuesIn(shapeParams4D_Full), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +const auto interpolateCasesCubic_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::cubic), - ::testing::ValuesIn(coordinateTransformModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(defNearestModes), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(pads4D), + ::testing::ValuesIn(cubeCoefs)); + +const auto interpolateCasesCubic_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::cubic), + ::testing::ValuesIn(coordinateTransformModes_Full), ::testing::ValuesIn(defNearestModes), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads4D), @@ -491,8 +576,18 @@ const auto interpolateCasesCubic = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateCubic_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesCubic, - ::testing::ValuesIn(shapeParams4D), + interpolateCasesCubic_Smoke, + ::testing::ValuesIn(shapeParams4D_Smoke), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +INSTANTIATE_TEST_SUITE_P(InterpolateCubic_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesCubic_Full, + ::testing::ValuesIn(shapeParams4D_Full), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice()), ::testing::ValuesIn(interpolateFusingParamsSet), @@ -526,7 +621,7 @@ const std::vector> defaultAxes5D = { {0, 1, 2, 3, 4} }; -const std::vector shapeParams5D = { +const std::vector shapeParams5D_Smoke = { ShapeParams{ ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, InputShape{{}, {{1, 11, 4, 4, 4}}}, @@ -541,20 +636,6 @@ const std::vector shapeParams5D = { {{1, 11, 5, 6, 2}}, defaultAxes5D.front() }, - ShapeParams{ - ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, - InputShape{{-1, {2, 20}, -1, -1, -1}, {{1, 11, 4, 4, 4}, {2, 7, 6, 5, 8}}}, - ngraph::helpers::InputLayerType::CONSTANT, - {{1.f, 1.f, 1.25f, 1.5f, 0.5f}}, - defaultAxes5D.front() - }, - ShapeParams{ - ngraph::op::v4::Interpolate::ShapeCalcMode::SIZES, - InputShape{{-1, {2, 20}, -1, -1, -1}, {{1, 11, 4, 4, 4}, {1, 11, 5, 5, 8}}}, - ngraph::helpers::InputLayerType::CONSTANT, - {{1, 11, 5, 6, 4}}, - defaultAxes5D.front() - }, ShapeParams{ ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, InputShape{{-1, {2, 20}, -1, -1, -1}, {{1, 11, 4, 4, 4}, {2, 7, 6, 5, 8}}}, @@ -571,10 +652,35 @@ const std::vector shapeParams5D = { }, }; -const auto interpolateCasesLinearOnnx5D = ::testing::Combine( +const std::vector shapeParams5D_Full = { + ShapeParams{ + ngraph::op::v4::Interpolate::ShapeCalcMode::SCALES, + InputShape{{-1, {2, 20}, -1, -1, -1}, {{1, 11, 4, 4, 4}, {2, 7, 6, 5, 8}}}, + ngraph::helpers::InputLayerType::CONSTANT, + {{1.f, 1.f, 1.25f, 1.5f, 0.5f}}, + defaultAxes5D.front() + }, + ShapeParams{ + ngraph::op::v4::Interpolate::ShapeCalcMode::SIZES, + InputShape{{-1, {2, 20}, -1, -1, -1}, {{1, 11, 4, 4, 4}, {1, 11, 5, 5, 8}}}, + ngraph::helpers::InputLayerType::CONSTANT, + {{1, 11, 5, 6, 4}}, + defaultAxes5D.front() + } +}; + +const auto interpolateCasesLinearOnnx5D_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear_onnx), - ::testing::ValuesIn(coordinateTransformModes), - ::testing::ValuesIn(nearestModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(defNearestModes), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads5D), + ::testing::ValuesIn(pads5D), + ::testing::ValuesIn(cubeCoefs)); +const auto interpolateCasesLinearOnnx5D_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::linear_onnx), + ::testing::ValuesIn(coordinateTransformModes_Full), + ::testing::ValuesIn(defNearestModes), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads5D), ::testing::ValuesIn(pads5D), @@ -582,18 +688,37 @@ const auto interpolateCasesLinearOnnx5D = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateLinearOnnx5D_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesLinearOnnx5D, - ::testing::ValuesIn(shapeParams5D), + interpolateCasesLinearOnnx5D_Smoke, + ::testing::ValuesIn(shapeParams5D_Smoke), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice5D()), ::testing::ValuesIn(interpolateFusingParamsSet), ::testing::ValuesIn(filterAdditionalConfig())), InterpolateLayerCPUTest::getTestCaseName); -const auto interpolateCasesNN5D = ::testing::Combine( +INSTANTIATE_TEST_SUITE_P(InterpolateLinearOnnx5D_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesLinearOnnx5D_Full, + ::testing::ValuesIn(shapeParams5D_Full), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice5D()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +const auto interpolateCasesNN5D_Smoke = ::testing::Combine( ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::nearest), - ::testing::ValuesIn(coordinateTransformModes), - ::testing::ValuesIn(defNearestModes), + ::testing::ValuesIn(coordinateTransformModes_Smoke), + ::testing::ValuesIn(nearestModes_Smoke), + ::testing::ValuesIn(antialias), + ::testing::ValuesIn(pads5D), + ::testing::ValuesIn(pads5D), + ::testing::ValuesIn(cubeCoefs)); + +const auto interpolateCasesNN5D_Full = ::testing::Combine( + ::testing::Values(ngraph::op::v4::Interpolate::InterpolateMode::nearest), + ::testing::ValuesIn(coordinateTransformModes_Full), + ::testing::ValuesIn(nearestModes_Full), ::testing::ValuesIn(antialias), ::testing::ValuesIn(pads5D), ::testing::ValuesIn(pads5D), @@ -601,8 +726,18 @@ const auto interpolateCasesNN5D = ::testing::Combine( INSTANTIATE_TEST_SUITE_P(smoke_InterpolateNN5D_Layout_Test, InterpolateLayerCPUTest, ::testing::Combine( - interpolateCasesNN5D, - ::testing::ValuesIn(shapeParams5D), + interpolateCasesNN5D_Smoke, + ::testing::ValuesIn(shapeParams5D_Smoke), + ::testing::Values(ElementType::f32), + ::testing::ValuesIn(filterCPUInfoForDevice5D()), + ::testing::ValuesIn(interpolateFusingParamsSet), + ::testing::ValuesIn(filterAdditionalConfig())), + InterpolateLayerCPUTest::getTestCaseName); + +INSTANTIATE_TEST_SUITE_P(InterpolateNN5D_Layout_Test, InterpolateLayerCPUTest, + ::testing::Combine( + interpolateCasesNN5D_Full, + ::testing::ValuesIn(shapeParams5D_Full), ::testing::Values(ElementType::f32), ::testing::ValuesIn(filterCPUInfoForDevice5D()), ::testing::ValuesIn(interpolateFusingParamsSet), diff --git a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pad.cpp b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pad.cpp index 1394af29ae6..5c23d6a3ae6 100644 --- a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pad.cpp +++ b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pad.cpp @@ -102,7 +102,7 @@ const std::vector inputPrecisions = { ElementType::i8 }; -const std::vector argPadValue = {0.f, 1.f, 2.5f, -1.f}; +const std::vector argPadValue = {0.f, 2.5f, -1.f}; const std::vector padMode = { ngraph::helpers::PadMode::EDGE, @@ -112,14 +112,23 @@ const std::vector padMode = { /* *======================* Static Shapes Tests 4D *======================* */ -const std::vector> padsBegin4DConstBlocked = {{0, 0, 0, 0}, {0, 0, 1, 3}, {2, 16, 1, 0}, {0, 0, 2, 0}}; -const std::vector> padsEnd4DConstBlocked = {{0, 0, 0, 0}, {0, 0, 2, 1}, {2, 0, 0, 1}, {1, 32, 2, 0}}; +const std::vector> padsBegin4DConstBlocked_Smoke = {{0, 0, 1, 3}, {2, 16, 1, 0}}; +const std::vector> padsEnd4DConstBlocked_Smoke = {{0, 0, 2, 1}, {2, 0, 0, 1}}; -const std::vector> padsBegin4DBlocked = {{0, 0, 0, 0}, {0, 0, 1, 3}, {2, 0, 1, 0}, {0, 0, 2, 0}}; -const std::vector> padsEnd4DBlocked = {{0, 0, 0, 0}, {0, 0, 2, 1}, {2, 0, 0, 1}, {1, 0, 2, 0}}; +const std::vector> padsBegin4DBlocked_Smoke = {{0, 0, 1, 3}, {2, 0, 1, 0}}; +const std::vector> padsEnd4DBlocked_Smoke = {{0, 0, 2, 1}, {2, 0, 0, 1}}; -const std::vector> padsBegin4D = {{0, 0, 0, 0}, {0, 1, 1, 1}, {0, 2, 1, 0}, {0, 0, 0, 1}}; -const std::vector> padsEnd4D = {{0, 0, 0, 0}, {0, 2, 1, 1}, {0, 0, 2, 0}, {1, 1, 0, 0}}; +const std::vector> padsBegin4D_Smoke = {{0, 1, 1, 1}, {0, 2, 1, 0}}; +const std::vector> padsEnd4D_Smoke = {{0, 2, 1, 1}, {0, 0, 2, 0}}; + +const std::vector> padsBegin4DConstBlocked_Full = {{0, 0, 0, 0}, {0, 0, 1, 3}, {2, 16, 1, 0}, {0, 0, 2, 0}}; +const std::vector> padsEnd4DConstBlocked_Full = {{0, 0, 0, 0}, {0, 0, 2, 1}, {2, 0, 0, 1}, {1, 32, 2, 0}}; + +const std::vector> padsBegin4DBlocked_Full = {{0, 0, 0, 0}, {0, 0, 1, 3}, {2, 0, 1, 0}, {0, 0, 2, 0}}; +const std::vector> padsEnd4DBlocked_Full = {{0, 0, 0, 0}, {0, 0, 2, 1}, {2, 0, 0, 1}, {1, 0, 2, 0}}; + +const std::vector> padsBegin4D_Full = {{0, 0, 0, 0}, {0, 1, 1, 1}, {0, 2, 1, 0}, {0, 0, 0, 1}}; +const std::vector> padsEnd4D_Full = {{0, 0, 0, 0}, {0, 2, 1, 1}, {0, 0, 2, 0}, {1, 1, 0, 0}}; const std::vector CPUParams4DBlocked = { cpuParams_nChw16c, @@ -132,8 +141,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4DConstBlocked), - ::testing::ValuesIn(padsEnd4DConstBlocked), + ::testing::ValuesIn(padsBegin4DConstBlocked_Smoke), + ::testing::ValuesIn(padsEnd4DConstBlocked_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams4DBlocked)), @@ -146,8 +155,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4D), - ::testing::ValuesIn(padsEnd4D), + ::testing::ValuesIn(padsBegin4D_Smoke), + ::testing::ValuesIn(padsEnd4D_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::Values(cpuParams_nhwc)), @@ -160,8 +169,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 10, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4DBlocked), - ::testing::ValuesIn(padsEnd4DBlocked), + ::testing::ValuesIn(padsBegin4DBlocked_Smoke), + ::testing::ValuesIn(padsEnd4DBlocked_Smoke), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams4DBlocked)), @@ -174,8 +183,64 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 10, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4DBlocked), - ::testing::ValuesIn(padsEnd4DBlocked), + ::testing::ValuesIn(padsBegin4DBlocked_Smoke), + ::testing::ValuesIn(padsEnd4DBlocked_Smoke), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::Values(cpuParams_nhwc)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad4DConstBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4DConstBlocked_Full), + ::testing::ValuesIn(padsEnd4DConstBlocked_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams4DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad4DConst, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4D_Full), + ::testing::ValuesIn(padsEnd4D_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::Values(cpuParams_nhwc)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad4DBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 10, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4DBlocked_Full), + ::testing::ValuesIn(padsEnd4DBlocked_Full), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams4DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad4D, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 10, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4DBlocked_Full), + ::testing::ValuesIn(padsEnd4DBlocked_Full), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::Values(cpuParams_nhwc)), @@ -208,8 +273,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(inputShapesDynamic4D), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4D), - ::testing::ValuesIn(padsEnd4D), + ::testing::ValuesIn(padsBegin4D_Smoke), + ::testing::ValuesIn(padsEnd4D_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams4DDynamic)), @@ -222,8 +287,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::Values(inputShapesDynamic4D[1]), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4DConstBlocked), - ::testing::ValuesIn(padsEnd4DConstBlocked), + ::testing::ValuesIn(padsBegin4DConstBlocked_Smoke), + ::testing::ValuesIn(padsEnd4DConstBlocked_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams4DBlocked)), @@ -236,8 +301,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(inputShapesDynamic4D), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4D), - ::testing::ValuesIn(padsEnd4D), + ::testing::ValuesIn(padsBegin4D_Smoke), + ::testing::ValuesIn(padsEnd4D_Smoke), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams4DDynamic)), @@ -250,8 +315,64 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::Values(inputShapesDynamic4D[1]), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin4DBlocked), - ::testing::ValuesIn(padsEnd4DBlocked), + ::testing::ValuesIn(padsBegin4DBlocked_Smoke), + ::testing::ValuesIn(padsEnd4DBlocked_Smoke), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams4DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic4DConst, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(inputShapesDynamic4D), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4D_Full), + ::testing::ValuesIn(padsEnd4D_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams4DDynamic)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic4DConstBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::Values(inputShapesDynamic4D[1]), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4DConstBlocked_Full), + ::testing::ValuesIn(padsEnd4DConstBlocked_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams4DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic4D, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(inputShapesDynamic4D), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4D_Full), + ::testing::ValuesIn(padsEnd4D_Full), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams4DDynamic)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic4DBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::Values(inputShapesDynamic4D[1]), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin4DBlocked_Full), + ::testing::ValuesIn(padsEnd4DBlocked_Full), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams4DBlocked)), @@ -262,14 +383,23 @@ INSTANTIATE_TEST_SUITE_P( /* *======================* Static Shapes Tests 5D *======================* */ -const std::vector> padsBegin5DConstBlocked = {{0, 0, 0, 0, 0}, {0, 0, 1, 1, 0}, {2, 32, 1, 1, 0}, {0, 0, 1, 3, 1}, {0, 0, 0, 1, 0}}; -const std::vector> padsEnd5DConstBlocked = {{0, 0, 0, 0, 0}, {1, 16, 1, 1, 0}, {0, 0, 0, 1, 0}, {0, 0, 0, 1, 1}, {0, 0, 1, 0, 1}}; +const std::vector> padsBegin5DConstBlocked_Smoke = {{0, 0, 1, 1, 0}, {2, 32, 1, 1, 0}}; +const std::vector> padsEnd5DConstBlocked_Smoke = {{1, 16, 1, 1, 0}, {0, 0, 0, 1, 0}}; -const std::vector> padsBegin5DBlocked = {{0, 0, 0, 0, 0}, {0, 0, 1, 1, 0}, {2, 0, 1, 1, 0}, {0, 0, 1, 3, 1}, {0, 0, 0, 1, 0}}; -const std::vector> padsEnd5DBlocked = {{0, 0, 0, 0, 0}, {1, 0, 1, 1, 0}, {0, 0, 0, 1, 0}, {0, 0, 0, 1, 1}, {0, 0, 1, 0, 1}}; +const std::vector> padsBegin5DBlocked_Smoke = {{0, 0, 1, 1, 0}, {2, 0, 1, 1, 0}}; +const std::vector> padsEnd5DBlocked_Smoke = {{1, 0, 1, 1, 0}, {0, 0, 0, 1, 0}}; -const std::vector> padsBegin5D = {{0, 0, 0, 0, 0}, {0, 0, 2, 0, 0}, {1, 1, 1, 1, 0}, {2, 0, 1, 0, 1}, {0, 2, 1, 3, 1}}; -const std::vector> padsEnd5D = {{0, 0, 0, 0, 0}, {0, 0, 1, 0, 0}, {1, 0, 1, 1, 2}, {2, 2, 0, 1, 0}, {1, 1, 2, 0, 1}}; +const std::vector> padsBegin5D_Smoke = {{0, 0, 2, 0, 0}, {1, 1, 1, 1, 0}}; +const std::vector> padsEnd5D_Smoke = {{0, 0, 1, 0, 0}, {1, 0, 1, 1, 2}}; + +const std::vector> padsBegin5DConstBlocked_Full = {{0, 0, 0, 0, 0}, {0, 0, 1, 1, 0}, {2, 32, 1, 1, 0}, {0, 0, 1, 3, 1}, {0, 0, 0, 1, 0}}; +const std::vector> padsEnd5DConstBlocked_Full = {{0, 0, 0, 0, 0}, {1, 16, 1, 1, 0}, {0, 0, 0, 1, 0}, {0, 0, 0, 1, 1}, {0, 0, 1, 0, 1}}; + +const std::vector> padsBegin5DBlocked_Full = {{0, 0, 0, 0, 0}, {0, 0, 1, 1, 0}, {2, 0, 1, 1, 0}, {0, 0, 1, 3, 1}, {0, 0, 0, 1, 0}}; +const std::vector> padsEnd5DBlocked_Full = {{0, 0, 0, 0, 0}, {1, 0, 1, 1, 0}, {0, 0, 0, 1, 0}, {0, 0, 0, 1, 1}, {0, 0, 1, 0, 1}}; + +const std::vector> padsBegin5D_Full = {{0, 0, 0, 0, 0}, {0, 0, 2, 0, 0}, {1, 1, 1, 1, 0}, {2, 0, 1, 0, 1}, {0, 2, 1, 3, 1}}; +const std::vector> padsEnd5D_Full = {{0, 0, 0, 0, 0}, {0, 0, 1, 0, 0}, {1, 0, 1, 1, 2}, {2, 2, 0, 1, 0}, {1, 1, 2, 0, 1}}; const std::vector CPUParams5DBlocked = { cpuParams_nCdhw16c, @@ -282,8 +412,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5DConstBlocked), - ::testing::ValuesIn(padsEnd5DConstBlocked), + ::testing::ValuesIn(padsBegin5DConstBlocked_Smoke), + ::testing::ValuesIn(padsEnd5DConstBlocked_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams5DBlocked)), @@ -296,8 +426,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5D), - ::testing::ValuesIn(padsEnd5D), + ::testing::ValuesIn(padsBegin5D_Smoke), + ::testing::ValuesIn(padsEnd5D_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::Values(cpuParams_ndhwc)), @@ -310,8 +440,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5DBlocked), - ::testing::ValuesIn(padsEnd5DBlocked), + ::testing::ValuesIn(padsBegin5DBlocked_Smoke), + ::testing::ValuesIn(padsEnd5DBlocked_Smoke), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams5DBlocked)), @@ -324,8 +454,64 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5D), - ::testing::ValuesIn(padsEnd5D), + ::testing::ValuesIn(padsBegin5D_Smoke), + ::testing::ValuesIn(padsEnd5D_Smoke), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::Values(cpuParams_ndhwc)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad5DConstBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5DConstBlocked_Full), + ::testing::ValuesIn(padsEnd5DConstBlocked_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams5DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad5DConst, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5D_Full), + ::testing::ValuesIn(padsEnd5D_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::Values(cpuParams_ndhwc)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad5DBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5DBlocked_Full), + ::testing::ValuesIn(padsEnd5DBlocked_Full), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams5DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPad5D, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(static_shapes_to_test_representation({{3, 16, 5, 5, 5}})), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5D_Full), + ::testing::ValuesIn(padsEnd5D_Full), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::Values(cpuParams_ndhwc)), @@ -358,8 +544,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(inputShapesDynamic5D), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5D), - ::testing::ValuesIn(padsEnd5D), + ::testing::ValuesIn(padsBegin5D_Smoke), + ::testing::ValuesIn(padsEnd5D_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams5DDynamic)), @@ -372,8 +558,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::Values(inputShapesDynamic5D[1]), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5DConstBlocked), - ::testing::ValuesIn(padsEnd5DConstBlocked), + ::testing::ValuesIn(padsBegin5DConstBlocked_Smoke), + ::testing::ValuesIn(padsEnd5DConstBlocked_Smoke), ::testing::ValuesIn(argPadValue), ::testing::Values(ngraph::helpers::PadMode::CONSTANT), ::testing::ValuesIn(CPUParams5DBlocked)), @@ -386,8 +572,8 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::ValuesIn(inputShapesDynamic5D), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5D), - ::testing::ValuesIn(padsEnd5D), + ::testing::ValuesIn(padsBegin5D_Smoke), + ::testing::ValuesIn(padsEnd5D_Smoke), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams5DDynamic)), @@ -400,8 +586,64 @@ INSTANTIATE_TEST_SUITE_P( ::testing::Combine( ::testing::Values(inputShapesDynamic5D[1]), ::testing::ValuesIn(inputPrecisions), - ::testing::ValuesIn(padsBegin5DBlocked), - ::testing::ValuesIn(padsEnd5DBlocked), + ::testing::ValuesIn(padsBegin5DBlocked_Smoke), + ::testing::ValuesIn(padsEnd5DBlocked_Smoke), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams5DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic5DConst, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(inputShapesDynamic5D), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5D_Full), + ::testing::ValuesIn(padsEnd5D_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams5DDynamic)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic5DConstBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::Values(inputShapesDynamic5D[1]), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5DConstBlocked_Full), + ::testing::ValuesIn(padsEnd5DConstBlocked_Full), + ::testing::ValuesIn(argPadValue), + ::testing::Values(ngraph::helpers::PadMode::CONSTANT), + ::testing::ValuesIn(CPUParams5DBlocked)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic5D, + PadLayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(inputShapesDynamic5D), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5D_Full), + ::testing::ValuesIn(padsEnd5D_Full), + ::testing::Values(0), + ::testing::ValuesIn(padMode), + ::testing::ValuesIn(CPUParams5DDynamic)), + PadLayerCPUTest::getTestCaseName +); + +INSTANTIATE_TEST_SUITE_P( + CPUPadDynamic5DBlocked, + PadLayerCPUTest, + ::testing::Combine( + ::testing::Values(inputShapesDynamic5D[1]), + ::testing::ValuesIn(inputPrecisions), + ::testing::ValuesIn(padsBegin5DBlocked_Full), + ::testing::ValuesIn(padsEnd5DBlocked_Full), ::testing::Values(0), ::testing::ValuesIn(padMode), ::testing::ValuesIn(CPUParams5DBlocked)), diff --git a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pooling.cpp b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pooling.cpp index 7b77296facc..191d35e33ab 100644 --- a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pooling.cpp +++ b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/pooling.cpp @@ -20,6 +20,11 @@ using poolLayerCpuTestParamsSet = std::tuple; +using maxPoolV8LayerCpuTestParamsSet = std::tuple; + class PoolingLayerCPUTest : public testing::WithParamInterface, virtual public SubgraphBaseTest, public CpuTestWithFusing { public: @@ -68,8 +73,6 @@ public: results << CPUTestsBase::getTestCaseName(cpuParams); results << CpuTestWithFusing::getTestCaseName(fusingParams); return results.str(); - - return results.str(); } protected: @@ -98,7 +101,10 @@ protected: if (selectedType.empty()) { selectedType = getPrimitiveType(); } - selectedType = selectedType + "_" + InferenceEngine::details::convertPrecision(inPrc).name(); + if (isInt8) + selectedType = selectedType + "_I8"; + else + selectedType = makeSelectedTypeStr(selectedType, inPrc); init_input_shapes({inputShapes}); @@ -124,11 +130,87 @@ protected: } }; +class MaxPoolingV8LayerCPUTest : public testing::WithParamInterface, + virtual public SubgraphBaseTest, public CPUTestsBase { +public: + static std::string getTestCaseName(const testing::TestParamInfo& obj) { + LayerTestsDefinitions::maxPoolV8SpecificParams basicParamsSet; + InputShape inputShapes; + ElementType inPrc; + CPUSpecificParams cpuParams; + std::tie(basicParamsSet, inputShapes, inPrc, cpuParams) = obj.param; + + std::vector kernel, stride, dilation; + std::vector padBegin, padEnd; + ngraph::op::PadType padType; + ngraph::op::RoundingType roundingType; + std::tie(kernel, stride, dilation, padBegin, padEnd, roundingType, padType) = basicParamsSet; + + std::ostringstream results; + results << "IS=("; + results << CommonTestUtils::partialShape2str({inputShapes.first}) << ")_"; + results << "TS="; + for (const auto& shape : inputShapes.second) { + results << CommonTestUtils::vec2str(shape) << "_"; + } + results << "Prc=" << inPrc << "_"; + results << "MaxPool_"; + results << "K" << CommonTestUtils::vec2str(kernel) << "_"; + results << "S" << CommonTestUtils::vec2str(stride) << "_"; + results << "D" << CommonTestUtils::vec2str(dilation) << "_"; + results << "PB" << CommonTestUtils::vec2str(padBegin) << "_"; + results << "PE" << CommonTestUtils::vec2str(padEnd) << "_"; + results << "Rounding=" << roundingType << "_"; + results << "AutoPad=" << padType << "_"; + + results << CPUTestsBase::getTestCaseName(cpuParams); + return results.str(); + } + +protected: + void SetUp() override { + targetDevice = CommonTestUtils::DEVICE_CPU; + + LayerTestsDefinitions::maxPoolV8SpecificParams basicParamsSet; + InputShape inputShapes; + ElementType inPrc; + CPUSpecificParams cpuParams; + std::tie(basicParamsSet, inputShapes, inPrc, cpuParams) = this->GetParam(); + + std::vector kernel, stride, dilation; + std::vector padBegin, padEnd; + ngraph::op::PadType padType; + ngraph::op::RoundingType roundingType; + std::tie(kernel, stride, dilation, padBegin, padEnd, roundingType, padType) = basicParamsSet; + std::tie(inFmts, outFmts, priority, selectedType) = cpuParams; + if (selectedType.empty()) { + selectedType = getPrimitiveType(); + } + selectedType = makeSelectedTypeStr(selectedType, inPrc); + + init_input_shapes({inputShapes}); + + auto params = ngraph::builder::makeDynamicParams(inPrc, inputDynamicShapes); + std::shared_ptr pooling = ngraph::builder::makeMaxPoolingV8(params[0], stride, dilation, padBegin, padEnd, + kernel, roundingType, padType); + pooling->get_rt_info() = getCPUInfo(); + ngraph::ResultVector results{std::make_shared(pooling->output(0))}; + function = std::make_shared(results, params, "MaxPooling"); + } +}; + TEST_P(PoolingLayerCPUTest, CompareWithRefs) { SKIP_IF_CURRENT_TEST_IS_DISABLED() run(); -// CheckPluginRelatedResults(executableNetwork, "Pooling"); + CheckPluginRelatedResults(executableNetwork, "Pooling"); +} + +TEST_P(MaxPoolingV8LayerCPUTest, CompareWithRefs) { + SKIP_IF_CURRENT_TEST_IS_DISABLED() + + run(); + CheckPluginRelatedResults(executableNetwork, "Pooling"); } namespace { @@ -291,6 +373,20 @@ const std::vector paramsMax4D = { ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, false }, }; +const std::vector paramsMaxV84D = { + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 2}, {2, 2}, {1, 1}, {0, 0}, {0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_LOWER }, +}; + +const std::vector paramsMaxV84D_ref = { + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 2}, {2, 2}, {2, 2}, {0, 0}, {0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_UPPER }, + LayerTestsDefinitions::maxPoolV8SpecificParams{ {4, 2}, {2, 2}, {1, 2}, {0, 0}, {0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT }, + LayerTestsDefinitions::maxPoolV8SpecificParams{ {4, 2}, {2, 1}, {2, 2}, {0, 0}, {0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT }, +}; + const std::vector paramsAvg4D = { LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::AVG, {2, 2}, {2, 2}, {1, 0}, {0, 0}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_LOWER, true }, @@ -321,6 +417,22 @@ INSTANTIATE_TEST_SUITE_P(smoke_MaxPool_CPU_4D, PoolingLayerCPUTest, ::testing::Values(emptyFusingSpec)), PoolingLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolV8_CPU_4D, MaxPoolingV8LayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(paramsMaxV84D), + ::testing::ValuesIn(inputShapes4D), + ::testing::ValuesIn(inpOutPrecision), + ::testing::ValuesIn(filterCPUInfoForDevice(vecCpuConfigs))), + MaxPoolingV8LayerCPUTest::getTestCaseName); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolV8_CPU_4D_ref, MaxPoolingV8LayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(paramsMaxV84D_ref), + ::testing::ValuesIn(inputShapes4D), + ::testing::ValuesIn(inpOutPrecision), + ::testing::Values(ref)), + MaxPoolingV8LayerCPUTest::getTestCaseName); + INSTANTIATE_TEST_SUITE_P(smoke_AvgPool_CPU_4D, PoolingLayerCPUTest, ::testing::Combine( ::testing::ValuesIn(paramsAvg4D), @@ -349,10 +461,24 @@ const std::vector paramsMax5D = { ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_UPPER, false }, LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::MAX, {2, 2, 2}, {1, 1, 1}, {1, 1, 1}, {1, 1, 1}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, false }, - LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::MAX, {2, 3, 4}, {2, 2, 2}, {1, 1, 1}, {1, 2, 3}, + LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::MAX, {3, 3, 3}, {2, 2, 2}, {1, 1, 1}, {1, 1, 1}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, false }, }; +const std::vector paramsMaxV85D = { + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 2, 2}, {1, 1, 1}, {1, 1, 1}, {0, 0, 0}, {0, 0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_LOWER }, +}; + +const std::vector paramsMaxV85D_ref = { + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 2, 2}, {1, 1, 1}, {2, 2, 2}, {0, 0, 0}, {0, 0, 0}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_UPPER }, + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 2, 2}, {1, 1, 1}, {2, 2, 2}, {1, 1, 1}, {1, 1, 1}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT }, + LayerTestsDefinitions::maxPoolV8SpecificParams{ {2, 3, 4}, {2, 2, 2}, {2, 1, 1}, {1, 1, 1}, {1, 2, 2}, + ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT }, +}; + const std::vector paramsAvg5D = { LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::AVG, {2, 2, 2}, {2, 2, 2}, {1, 0, 0}, {0, 0, 0}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::SAME_LOWER, true }, @@ -366,7 +492,7 @@ const std::vector paramsAvg5D = { ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, true }, LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::AVG, {3, 3, 3}, {3, 3, 3}, {1, 1, 1}, {0, 0, 0}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, true }, - LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::AVG, {4, 4, 4}, {4, 4, 4}, {2, 2, 2}, {2, 2, 2}, + LayerTestsDefinitions::poolSpecificParams{ ngraph::helpers::PoolingTypes::AVG, {4, 4, 4}, {2, 2, 2}, {2, 2, 2}, {2, 2, 2}, ngraph::op::RoundingType::CEIL, ngraph::op::PadType::EXPLICIT, true }, }; @@ -385,6 +511,22 @@ INSTANTIATE_TEST_SUITE_P(smoke_MaxPool_CPU_5D, PoolingLayerCPUTest, ::testing::Values(emptyFusingSpec)), PoolingLayerCPUTest::getTestCaseName); +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolV8_CPU_5D, MaxPoolingV8LayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(paramsMaxV85D), + ::testing::ValuesIn(inputShapes5D), + ::testing::ValuesIn(inpOutPrecision), + ::testing::ValuesIn(filterCPUInfoForDevice(vecCpuConfigs))), + MaxPoolingV8LayerCPUTest::getTestCaseName); + +INSTANTIATE_TEST_SUITE_P(smoke_MaxPoolV8_CPU_5D_ref, MaxPoolingV8LayerCPUTest, + ::testing::Combine( + ::testing::ValuesIn(paramsMaxV85D_ref), + ::testing::ValuesIn(inputShapes5D), + ::testing::ValuesIn(inpOutPrecision), + ::testing::Values(ref)), + MaxPoolingV8LayerCPUTest::getTestCaseName); + INSTANTIATE_TEST_SUITE_P(smoke_AvgPool_CPU_5D, PoolingLayerCPUTest, ::testing::Combine( ::testing::ValuesIn(paramsAvg5D), diff --git a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/reduce_ops.cpp b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/reduce_ops.cpp index dbc3e8435c7..18e992edfea 100644 --- a/inference-engine/tests/functional/plugin/cpu/single_layer_tests/reduce_ops.cpp +++ b/inference-engine/tests/functional/plugin/cpu/single_layer_tests/reduce_ops.cpp @@ -258,6 +258,12 @@ const std::vector reductionTypes = { ngraph::helpers::ReductionType::L2, }; +const std::vector reductionTypesFusing = { + ngraph::helpers::ReductionType::Mean, + ngraph::helpers::ReductionType::Max, + ngraph::helpers::ReductionType::L2, +}; + const std::vector reductionLogicalTypes = { ngraph::helpers::ReductionType::LogicalOr, ngraph::helpers::ReductionType::LogicalAnd @@ -315,13 +321,9 @@ std::vector cpuParams_HybridLayout_5D = { const std::vector fusingParamsSet { /* activations */ - fusingRelu, - fusingElu, - fusingTanh, fusingSwish, /* FQ */ - fusingFakeQuantizePerChannel, fusingFakeQuantizePerChannelRelu, fusingFakeQuantizePerTensorRelu, /* another patterns */ @@ -576,7 +578,7 @@ const auto params_OneAxis_fusing = testing::Combine( testing::ValuesIn(axes), testing::ValuesIn(opTypes), testing::Values(true), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), @@ -589,7 +591,7 @@ const auto params_MultiAxis_4D_fusing = testing::Combine( testing::ValuesIn(axesND), testing::Values(CommonTestUtils::OpType::VECTOR), testing::Values(true), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), @@ -602,7 +604,7 @@ const auto params_MultiAxis_5D_fusing = testing::Combine( testing::ValuesIn(axes5D), testing::Values(CommonTestUtils::OpType::VECTOR), testing::Values(true), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), @@ -637,7 +639,7 @@ const auto params_OneAxis_fusing_KeepNoDims = testing::Combine( testing::ValuesIn(axes), testing::ValuesIn(opTypes), testing::Values(false), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), @@ -650,7 +652,7 @@ const auto params_MultiAxis_4D_Hybrid_fusing_KeepNoDims = testing::Combine( testing::ValuesIn(axesNDFusing), testing::Values(CommonTestUtils::OpType::VECTOR), testing::Values(false), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), @@ -663,7 +665,7 @@ const auto params_MultiAxis_5D_Hybrid_fusing_KeepNoDims = testing::Combine( testing::ValuesIn(axes5DFusing), testing::Values(CommonTestUtils::OpType::VECTOR), testing::Values(false), - testing::ValuesIn(reductionTypes), + testing::ValuesIn(reductionTypesFusing), testing::ValuesIn(inpOutPrc), testing::Values(ElementType::undefined), testing::Values(ElementType::undefined), diff --git a/inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.cpp b/inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.cpp index 8fd24783dfa..005f1702029 100644 --- a/inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.cpp +++ b/inference-engine/tests/functional/plugin/cpu/test_utils/cpu_test_utils.cpp @@ -265,18 +265,18 @@ CPUTestsBase::makeCPUInfo(std::vector inFmts, std::vector(fmts2str(inFmts, "cpu:"))}); + cpuInfo.insert({ngraph::MKLDNNInputMemoryFormats::get_type_info_static(), + ngraph::MKLDNNInputMemoryFormats(fmts2str(inFmts, "cpu:"))}); } if (!outFmts.empty()) { - cpuInfo.insert({std::string(ngraph::MLKDNNOutputMemoryFormatsAttr), - std::make_shared(fmts2str(outFmts, "cpu:"))}); + cpuInfo.insert({ngraph::MKLDNNOutputMemoryFormats::get_type_info_static(), + ngraph::MKLDNNOutputMemoryFormats(fmts2str(outFmts, "cpu:"))}); } if (!priority.empty()) { cpuInfo.insert({"PrimitivesPriority", std::make_shared>(impls2str(priority))}); } - cpuInfo.insert({"enforceBF16evenForGraphTail", ov::make_variant(true)}); + cpuInfo.insert({"enforceBF16evenForGraphTail", ov::make_runtime_attribute(true)}); return cpuInfo; } diff --git a/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/experimental_detectron_roifeatureextractor.cpp b/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/experimental_detectron_roifeatureextractor.cpp new file mode 100644 index 00000000000..c7cd090645c --- /dev/null +++ b/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/experimental_detectron_roifeatureextractor.cpp @@ -0,0 +1,37 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "single_layer_tests/experimental_detectron_roifeatureextractor.hpp" + +using namespace ov::test; +using namespace ov::test::subgraph; + +namespace { +const std::vector outputSize = {7, 14}; +const std::vector samplingRatio = {1, 2, 3}; + +const std::vector> pyramidScales = { + {8, 16, 32, 64}, + {4, 8, 16, 32}, + {2, 4, 8, 16} +}; + +const std::vector> staticInputShape = { + static_shapes_to_test_representation({{1000, 4}, {1, 8, 200, 336}, {1, 8, 100, 168}, {1, 8, 50, 84}, {1, 8, 25, 42}}), + static_shapes_to_test_representation({{1000, 4}, {1, 16, 200, 336}, {1, 16, 100, 168}, {1, 16, 50, 84}, {1, 16, 25, 42}}), + static_shapes_to_test_representation({{1200, 4}, {1, 8, 200, 42}, {1, 8, 100, 336}, {1, 8, 50, 168}, {1, 8, 25, 84}}) +}; + +INSTANTIATE_TEST_SUITE_P(smoke_ExperimentalROI_static, ExperimentalDetectronROIFeatureExtractorLayerTest, + ::testing::Combine( + ::testing::ValuesIn(staticInputShape), + ::testing::ValuesIn(outputSize), + ::testing::ValuesIn(samplingRatio), + ::testing::ValuesIn(pyramidScales), + ::testing::Values(false), + ::testing::Values(ov::element::Type_t::f32), + ::testing::Values(CommonTestUtils::DEVICE_GPU)), + ExperimentalDetectronROIFeatureExtractorLayerTest::getTestCaseName); +} // namespace diff --git a/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/roi_align.cpp b/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/roi_align.cpp new file mode 100644 index 00000000000..47ebcf81677 --- /dev/null +++ b/inference-engine/tests/functional/plugin/gpu/shared_tests_instances/single_layer_tests/roi_align.cpp @@ -0,0 +1,62 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#include + +#include "single_layer_tests/roi_align.hpp" +#include "common_test_utils/test_constants.hpp" + +using namespace LayerTestsDefinitions; + + +const std::vector netPRCs = { + InferenceEngine::Precision::FP32, + // There is no possibility to test ROIAlign in fp16 precision, + // because on edge cases where in fp32 version ROI value is + // a little bit smaller than the nearest integer value, + // it would be bigger than the nearest integer in fp16 precision. + // Such behavior leads to completely different results of ROIAlign + // in fp32 and fp16 precisions. + // In real AI applications this problem is solved by precision-aware training. + + // InferenceEngine::Precision::FP16, +}; + +const auto ROIAlignCases_average = + ::testing::Combine( + ::testing::ValuesIn( + std::vector> { + { 3, 8, 16, 16 }, + { 2, 1, 16, 16 }, + { 2, 1, 8, 16 }}), + ::testing::Values(std::vector{ 2, 4 }), + ::testing::Values(2), + ::testing::Values(2), + ::testing::ValuesIn(std::vector { 1, 0.625 }), + ::testing::Values(2), + ::testing::Values("avg"), + ::testing::ValuesIn(netPRCs), + ::testing::Values(CommonTestUtils::DEVICE_GPU) +); + +INSTANTIATE_TEST_SUITE_P(smoke_TestsROIAlign_average, ROIAlignLayerTest, ROIAlignCases_average, ROIAlignLayerTest::getTestCaseName); + +const auto ROIAlignCases_max = + ::testing::Combine( + ::testing::ValuesIn( + std::vector> { + { 2, 8, 20, 20 }, + { 2, 1, 20, 20 }, + { 2, 1, 10, 20 } + }), + ::testing::Values(std::vector{ 2, 4 }), + ::testing::Values(2), + ::testing::Values(2), + ::testing::ValuesIn(std::vector { 1, 0.625 }), + ::testing::Values(2), + ::testing::Values("max"), + ::testing::ValuesIn(netPRCs), + ::testing::Values(CommonTestUtils::DEVICE_GPU) +); + +INSTANTIATE_TEST_SUITE_P(smoke_TestsROIAlign_max, ROIAlignLayerTest, ROIAlignCases_max, ROIAlignLayerTest::getTestCaseName); diff --git a/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_detection_output.hpp b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_detection_output.hpp new file mode 100644 index 00000000000..b019871db11 --- /dev/null +++ b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_detection_output.hpp @@ -0,0 +1,19 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include + +namespace ov { +namespace test { +namespace subgraph { + +TEST_P(ExperimentalDetectronDetectionOutputLayerTest, ExperimentalDetectronDetectionOutputLayerTests) { + run(); +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_generate_proposals_single_image.hpp b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_generate_proposals_single_image.hpp new file mode 100644 index 00000000000..f3805e591e2 --- /dev/null +++ b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_generate_proposals_single_image.hpp @@ -0,0 +1,19 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include + +namespace ov { +namespace test { +namespace subgraph { + +TEST_P(ExperimentalDetectronGenerateProposalsSingleImageLayerTest, ExperimentalDetectronGenerateProposalsSingleImageLayerTests) { + run(); +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_prior_grid_generator.hpp b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_prior_grid_generator.hpp new file mode 100644 index 00000000000..0a2d956aa7d --- /dev/null +++ b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/experimental_detectron_prior_grid_generator.hpp @@ -0,0 +1,19 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include + +namespace ov { +namespace test { +namespace subgraph { + +TEST_P(ExperimentalDetectronPriorGridGeneratorLayerTest, ExperimentalDetectronPriorGridGeneratorLayerTests) { + run(); +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/pooling.hpp b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/pooling.hpp index f8bd0907fa5..165096fd069 100644 --- a/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/pooling.hpp +++ b/inference-engine/tests/functional/plugin/shared/include/single_layer_tests/pooling.hpp @@ -19,4 +19,9 @@ TEST_P(GlobalPoolingLayerTest, CompareWithRefs) { PluginCache::get().reset(); } } + +TEST_P(MaxPoolingV8LayerTest, CompareWithRefs) { + Run(); +} + } // namespace LayerTestsDefinitions diff --git a/inference-engine/tests/functional/plugin/shared/src/behavior/caching/caching_tests.cpp b/inference-engine/tests/functional/plugin/shared/src/behavior/caching/caching_tests.cpp index 734630d61e5..a206e529c1d 100644 --- a/inference-engine/tests/functional/plugin/shared/src/behavior/caching/caching_tests.cpp +++ b/inference-engine/tests/functional/plugin/shared/src/behavior/caching/caching_tests.cpp @@ -120,7 +120,7 @@ bool LoadNetworkCacheTestBase::importExportSupported(InferenceEngine::Core& ie) std::vector supportedMetricKeys = ie.GetMetric(targetDevice, METRIC_KEY(SUPPORTED_METRICS)); auto it = std::find(supportedMetricKeys.begin(), supportedMetricKeys.end(), METRIC_KEY(IMPORT_EXPORT_SUPPORT)); - bool supported = (it != supportedMetricKeys.end()) && + auto supported = (it != supportedMetricKeys.end()) && ie.GetMetric(targetDevice, METRIC_KEY(IMPORT_EXPORT_SUPPORT)).as(); return supported; } diff --git a/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_detection_output.hpp b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_detection_output.hpp new file mode 100644 index 00000000000..a40ce2dccef --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_detection_output.hpp @@ -0,0 +1,39 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "ngraph_functions/utils/ngraph_helpers.hpp" +#include "common_test_utils/common_utils.hpp" +#include "shared_test_classes/base/ov_subgraph.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +typedef std::tuple< + std::vector, // inputShapes + float, // score_threshold + float, // nms_threshol + float, // max_delta_log_wh + int64_t, // num_classes + int64_t, // post_nms_count + size_t, // max_detections_per_image + bool, // class_agnostic_box_regression + std::vector, // deltas_weights + ElementType, // Network precision + std::string // Device name +> ExperimentalDetectronDetectionOutputTestParams; + +class ExperimentalDetectronDetectionOutputLayerTest : + public testing::WithParamInterface, + virtual public SubgraphBaseTest { +protected: + void SetUp() override; + void generate_inputs(const std::vector& targetInputStaticShapes) override; + +public: + static std::string getTestCaseName(const testing::TestParamInfo& obj); +}; +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_generate_proposals_single_image.hpp b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_generate_proposals_single_image.hpp new file mode 100644 index 00000000000..3a07ff08835 --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_generate_proposals_single_image.hpp @@ -0,0 +1,36 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "ngraph_functions/utils/ngraph_helpers.hpp" +#include "common_test_utils/common_utils.hpp" +#include "shared_test_classes/base/ov_subgraph.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +typedef std::tuple< + std::vector, // Input shapes + float, // min_size: minimum box width & height + float, // nms_threshold: specifies NMS threshold + int64_t, // post_nms_count: number of top-n proposals after NMS + int64_t, // pre_nms_count: number of top-n proposals after NMS + std::pair>, // input tensors + ElementType, // Network precision + std::string // Device name>; +> ExperimentalDetectronGenerateProposalsSingleImageTestParams; + +class ExperimentalDetectronGenerateProposalsSingleImageLayerTest : + public testing::WithParamInterface, + virtual public SubgraphBaseTest { +protected: + void SetUp() override; + void generate_inputs(const std::vector& targetInputStaticShapes) override; + +public: + static std::string getTestCaseName(const testing::TestParamInfo& obj); +}; +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_prior_grid_generator.hpp b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_prior_grid_generator.hpp new file mode 100644 index 00000000000..aa818358d75 --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/experimental_detectron_prior_grid_generator.hpp @@ -0,0 +1,38 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "ngraph_functions/utils/ngraph_helpers.hpp" +#include "common_test_utils/common_utils.hpp" +#include "shared_test_classes/base/ov_subgraph.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +class ExperimentalDetectronPriorGridGeneratorTestParam { +public: + ov::op::v6::ExperimentalDetectronPriorGridGenerator::Attributes attributes; + std::vector inputShapes; +}; + +typedef std::tuple< + ExperimentalDetectronPriorGridGeneratorTestParam, + std::pair>, + ElementType, // Network precision + std::string // Device name>; +> ExperimentalDetectronPriorGridGeneratorTestParams; + +class ExperimentalDetectronPriorGridGeneratorLayerTest : + public testing::WithParamInterface, + virtual public SubgraphBaseTest { +protected: + void SetUp() override; + void generate_inputs(const std::vector& targetInputStaticShapes) override; + +public: + static std::string getTestCaseName(const testing::TestParamInfo& obj); +}; +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/pooling.hpp b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/pooling.hpp index 12417959d22..6730e6c2b56 100644 --- a/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/pooling.hpp +++ b/inference-engine/tests/functional/shared_test_classes/include/shared_test_classes/single_layer/pooling.hpp @@ -48,6 +48,27 @@ typedef std::tuple< std::string // Device name > globalPoolLayerTestParamsSet; +typedef std::tuple< + std::vector, // Kernel size + std::vector, // Stride + std::vector, // Dilation + std::vector, // Pad begin + std::vector, // Pad end + ngraph::op::RoundingType, // Rounding type + ngraph::op::PadType // Pad type +> maxPoolV8SpecificParams; + +typedef std::tuple< + maxPoolV8SpecificParams, + InferenceEngine::Precision, // Net precision + InferenceEngine::Precision, // Input precision + InferenceEngine::Precision, // Output precision + InferenceEngine::Layout, // Input layout + InferenceEngine::Layout, // Output layout + std::vector, // Input shape + std::string // Device name +> maxPoolV8LayerTestParamsSet; + class PoolingLayerTest : public testing::WithParamInterface, virtual public LayerTestsUtils::LayerTestsCommon { public: @@ -66,4 +87,13 @@ protected: void SetUp() override; }; +class MaxPoolingV8LayerTest : public testing::WithParamInterface, + virtual public LayerTestsUtils::LayerTestsCommon { +public: + static std::string getTestCaseName(const testing::TestParamInfo& obj); + +protected: + void SetUp() override; +}; + } // namespace LayerTestsDefinitions diff --git a/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_detection_output.cpp b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_detection_output.cpp new file mode 100644 index 00000000000..3c11da92f9c --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_detection_output.cpp @@ -0,0 +1,144 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "shared_test_classes/single_layer/experimental_detectron_detection_output.hpp" +#include "ngraph_functions/builders.hpp" +#include "common_test_utils/data_utils.hpp" +#include "functional_test_utils/ov_tensor_utils.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +namespace { + std::ostream& operator <<(std::ostream& ss, const ngraph::opset6::ExperimentalDetectronDetectionOutput::Attributes& attributes) { + ss << "score_threshold=" << attributes.score_threshold << "_"; + ss << "nms_threshold=" << attributes.nms_threshold << "_"; + ss << "max_delta_log_wh=" << attributes.max_delta_log_wh << "_"; + ss << "num_classes=" << attributes.num_classes << "_"; + ss << "post_nms_count=" << attributes.post_nms_count << "_"; + ss << "max_detections_per_image=" << attributes.max_detections_per_image << "_"; + ss << "class_agnostic_box_regression=" << (attributes.class_agnostic_box_regression ? "true" : "false") << "_"; + ss << "deltas_weights=" << CommonTestUtils::vec2str(attributes.deltas_weights); + return ss; +} +} // namespace + +std::string ExperimentalDetectronDetectionOutputLayerTest::getTestCaseName( + const testing::TestParamInfo& obj) { + std::vector inputShapes; + ngraph::opset6::ExperimentalDetectronDetectionOutput::Attributes attributes; + ElementType netPrecision; + std::string targetName; + std::tie( + inputShapes, + attributes.score_threshold, + attributes.nms_threshold, + attributes.max_delta_log_wh, + attributes.num_classes, + attributes.post_nms_count, + attributes.max_detections_per_image, + attributes.class_agnostic_box_regression, + attributes.deltas_weights, + netPrecision, + targetName) = obj.param; + + std::ostringstream result; + + using ov::test::operator<<; + result << "input_rois=" << inputShapes[0] << "_"; + result << "input_deltas=" << inputShapes[1] << "_"; + result << "input_scores=" << inputShapes[2] << "_"; + result << "input_im_info=" << inputShapes[3] << "_"; + + using ov::test::subgraph::operator<<; + result << "attributes={" << attributes << "}_"; + result << "netPRC=" << netPrecision << "_"; + result << "trgDev=" << targetName; + return result.str(); +} + +void ExperimentalDetectronDetectionOutputLayerTest::SetUp() { + std::vector inputShapes; + ngraph::opset6::ExperimentalDetectronDetectionOutput::Attributes attributes; + + ElementType netPrecision; + std::string targetName; + std::tie( + inputShapes, + attributes.score_threshold, + attributes.nms_threshold, + attributes.max_delta_log_wh, + attributes.num_classes, + attributes.post_nms_count, + attributes.max_detections_per_image, + attributes.class_agnostic_box_regression, + attributes.deltas_weights, + netPrecision, + targetName) = this->GetParam(); + + inType = outType = netPrecision; + targetDevice = targetName; + + init_input_shapes(inputShapes); + + auto params = ngraph::builder::makeDynamicParams(netPrecision, {inputDynamicShapes}); + auto paramsOuts = ngraph::helpers::convert2OutputVector(ngraph::helpers::castOps2Nodes(params)); + auto experimentalDetectron = std::make_shared( + params[0], // input_rois + params[1], // input_deltas + params[2], // input_scores + params[3], // input_im_info + attributes); + function = std::make_shared( + ov::OutputVector{experimentalDetectron->output(0), experimentalDetectron->output(1)}, + "ExperimentalDetectronDetectionOutput"); +} + +void ExperimentalDetectronDetectionOutputLayerTest::generate_inputs(const std::vector& targetInputStaticShapes) { + static const std::vector inputTensors = { + // 16 x 4 = 64 + CommonTestUtils::create_tensor(ov::element::f32, Shape{16, 4}, { + 1.0f, 1.0f, 10.0f, 10.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 4.0f, 1.0f, 8.0f, 5.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f + }), + // 16 x 8 + CommonTestUtils::create_tensor(ov::element::f32, Shape{16, 8}, { + 5.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 4.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 8.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f + }), + // 16 x 2 = 32 + CommonTestUtils::create_tensor(ov::element::f32, Shape{16, 2}, { + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, + 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f, 1.0f + }), + // 1 x 3 = 3 + CommonTestUtils::create_tensor(ov::element::f32, Shape{1, 3}, {1.0f, 1.0f, 1.0f}) + }; + + inputs.clear(); + const auto& funcInputs = function->inputs(); + for (auto i = 0ul; i < funcInputs.size(); ++i) { + if (targetInputStaticShapes[i] != inputTensors[i].get_shape()) { + throw Exception("input shape is different from tensor shape"); + } + + inputs.insert({funcInputs[i].get_node_shared_ptr(), inputTensors[i]}); + } +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_generate_proposals_single_image.cpp b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_generate_proposals_single_image.cpp new file mode 100644 index 00000000000..2c8e0c22e82 --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_generate_proposals_single_image.cpp @@ -0,0 +1,107 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "shared_test_classes/single_layer/experimental_detectron_generate_proposals_single_image.hpp" +#include "ngraph_functions/builders.hpp" +#include "functional_test_utils/ov_tensor_utils.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +namespace { +std::ostream& operator <<( + std::ostream& ss, + const ov::op::v6::ExperimentalDetectronGenerateProposalsSingleImage::Attributes& attributes) { + ss << "score_threshold=" << attributes.min_size << "_"; + ss << "nms_threshold=" << attributes.nms_threshold << "_"; + ss << "max_delta_log_wh=" << attributes.post_nms_count << "_"; + ss << "num_classes=" << attributes.pre_nms_count; + return ss; +} +} // namespace + +std::string ExperimentalDetectronGenerateProposalsSingleImageLayerTest::getTestCaseName( + const testing::TestParamInfo& obj) { + std::vector inputShapes; + ov::op::v6::ExperimentalDetectronGenerateProposalsSingleImage::Attributes attributes; + std::pair> inputTensors; + ElementType netPrecision; + std::string targetName; + std::tie( + inputShapes, + attributes.min_size, + attributes.nms_threshold, + attributes.post_nms_count, + attributes.pre_nms_count, + inputTensors, + netPrecision, + targetName) = obj.param; + + std::ostringstream result; + using ov::test::operator<<; + result << "im_info=" << inputShapes[0] << "_"; + result << "anchors=" << inputShapes[1] << "_"; + result << "deltas=" << inputShapes[2] << "_"; + result << "scores=" << inputShapes[3] << "_"; + + using ov::test::subgraph::operator<<; + result << "attributes={" << attributes << "}_"; + result << "inputTensors=" << inputTensors.first << "_"; + result << "netPRC=" << netPrecision << "_"; + result << "trgDev=" << targetName; + return result.str(); +} + +void ExperimentalDetectronGenerateProposalsSingleImageLayerTest::SetUp() { + std::vector inputShapes; + ov::op::v6::ExperimentalDetectronGenerateProposalsSingleImage::Attributes attributes; + std::pair> inputTensors; + ElementType netPrecision; + std::string targetName; + std::tie( + inputShapes, + attributes.min_size, + attributes.nms_threshold, + attributes.post_nms_count, + attributes.pre_nms_count, + inputTensors, + netPrecision, + targetName) = this->GetParam(); + + inType = outType = netPrecision; + targetDevice = targetName; + + init_input_shapes(inputShapes); + + auto params = ngraph::builder::makeDynamicParams(netPrecision, {inputDynamicShapes}); + auto paramsOuts = ngraph::helpers::convert2OutputVector(ngraph::helpers::castOps2Nodes(params)); + auto experimentalDetectron = std::make_shared( + params[0], // im_info + params[1], // anchors + params[2], // deltas + params[3], // scores + attributes); + function = std::make_shared( + ov::OutputVector{experimentalDetectron->output(0), experimentalDetectron->output(1)}, + "ExperimentalDetectronGenerateProposalsSingleImage"); +} + +void ExperimentalDetectronGenerateProposalsSingleImageLayerTest::generate_inputs(const std::vector& targetInputStaticShapes) { + auto inputTensors = std::get<5>(GetParam()); + + inputs.clear(); + const auto& funcInputs = function->inputs(); + for (auto i = 0ul; i < funcInputs.size(); ++i) { + if (targetInputStaticShapes[i] != inputTensors.second[i].get_shape()) { + throw Exception("input shape is different from tensor shape"); + } + + inputs.insert({funcInputs[i].get_node_shared_ptr(), inputTensors.second[i]}); + } +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_prior_grid_generator.cpp b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_prior_grid_generator.cpp new file mode 100644 index 00000000000..e63579a0633 --- /dev/null +++ b/inference-engine/tests/functional/shared_test_classes/src/single_layer/experimental_detectron_prior_grid_generator.cpp @@ -0,0 +1,100 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "shared_test_classes/single_layer/experimental_detectron_prior_grid_generator.hpp" +#include "ngraph_functions/builders.hpp" +#include "common_test_utils/data_utils.hpp" +#include "functional_test_utils/ov_tensor_utils.hpp" + +namespace ov { +namespace test { +namespace subgraph { + +namespace { +std::ostream& operator <<( + std::ostream& ss, + const ov::op::v6::ExperimentalDetectronPriorGridGenerator::Attributes& attributes) { + ss << "flatten=" << attributes.flatten << "_"; + ss << "h=" << attributes.h << "_"; + ss << "w=" << attributes.w << "_"; + ss << "stride_x=" << attributes.stride_x << "_"; + ss << "stride_y=" << attributes.stride_y; + return ss; +} +} // namespace + +std::string ExperimentalDetectronPriorGridGeneratorLayerTest::getTestCaseName( + const testing::TestParamInfo& obj) { + ExperimentalDetectronPriorGridGeneratorTestParam param; + std::pair> inputTensors; + ElementType netPrecision; + std::string targetName; + std::tie(param, inputTensors, netPrecision, targetName) = obj.param; + + std::ostringstream result; + using ov::test::operator<<; + result << "priors=" << param.inputShapes[0] << "_"; + result << "feature_map=" << param.inputShapes[1] << "_"; + result << "im_data=" << param.inputShapes[2] << "_"; + + using ov::test::subgraph::operator<<; + result << "attributes=" << param.attributes << "_"; + result << "priorValues=" << inputTensors.first << "_"; + result << "netPRC=" << netPrecision << "_"; + result << "trgDev=" << targetName; + return result.str(); +} + +void ExperimentalDetectronPriorGridGeneratorLayerTest::SetUp() { + ExperimentalDetectronPriorGridGeneratorTestParam param; + std::pair> inputTensors; + ElementType netPrecision; + std::string targetName; + std::tie(param, inputTensors, netPrecision, targetName) = this->GetParam(); + + inType = outType = netPrecision; + targetDevice = targetName; + + init_input_shapes(param.inputShapes); + + auto params = ngraph::builder::makeDynamicParams(netPrecision, {inputDynamicShapes}); + auto paramsOuts = ngraph::helpers::convert2OutputVector(ngraph::helpers::castOps2Nodes(params)); + auto experimentalDetectron = std::make_shared( + params[0], // priors + params[1], // feature_map + params[2], // im_data + param.attributes); + function = std::make_shared( + ov::OutputVector{experimentalDetectron->output(0)}, + "ExperimentalDetectronPriorGridGenerator"); +} + +void ExperimentalDetectronPriorGridGeneratorLayerTest::generate_inputs(const std::vector& targetInputStaticShapes) { + auto inputTensors = std::get<1>(GetParam()); + + inputs.clear(); + const auto& funcInputs = function->inputs(); + + auto i = 0ul; + for (; i < inputTensors.second.size(); ++i) { + if (targetInputStaticShapes[i] != inputTensors.second[i].get_shape()) { + throw Exception("input shape is different from tensor shape"); + } + + inputs.insert({funcInputs[i].get_node_shared_ptr(), inputTensors.second[i]}); + } + + for (auto j = i; j < funcInputs.size(); ++j) { + ov::runtime::Tensor inputTensor = CommonTestUtils::create_tensor( + ov::element::f32, + targetInputStaticShapes[j], + std::vector(0.f, shape_size(targetInputStaticShapes[j]))); + + inputs.insert({funcInputs[j].get_node_shared_ptr(), inputTensor}); + } +} + +} // namespace subgraph +} // namespace test +} // namespace ov diff --git a/inference-engine/tests/functional/shared_test_classes/src/single_layer/memory.cpp b/inference-engine/tests/functional/shared_test_classes/src/single_layer/memory.cpp index bf8e79a8e30..5b08a7327b8 100644 --- a/inference-engine/tests/functional/shared_test_classes/src/single_layer/memory.cpp +++ b/inference-engine/tests/functional/shared_test_classes/src/single_layer/memory.cpp @@ -46,9 +46,9 @@ namespace LayerTestsDefinitions { } auto hostTensor = std::make_shared(ngPrc, inputShape); - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value = std::make_shared(hostTensor); - variable_context->get().set_variable_value(function->get_variable_by_id("v0"), variable_value); + variable_context.set_variable_value(function->get_variable_by_id("v0"), variable_value); eval_context["VariableContext"] = variable_context; } diff --git a/inference-engine/tests/functional/shared_test_classes/src/single_layer/pooling.cpp b/inference-engine/tests/functional/shared_test_classes/src/single_layer/pooling.cpp index 276fc2bfb6f..3dc2b2c7a36 100644 --- a/inference-engine/tests/functional/shared_test_classes/src/single_layer/pooling.cpp +++ b/inference-engine/tests/functional/shared_test_classes/src/single_layer/pooling.cpp @@ -94,6 +94,38 @@ std::string GlobalPoolingLayerTest::getTestCaseName(const testing::TestParamInfo return result.str(); } +std::string MaxPoolingV8LayerTest::getTestCaseName(const testing::TestParamInfo& obj) { + maxPoolV8SpecificParams poolParams; + InferenceEngine::Precision netPrecision; + InferenceEngine::Precision inPrc, outPrc; + InferenceEngine::Layout inLayout, outLayout; + std::vector inputShapes; + std::string targetDevice; + std::tie(poolParams, netPrecision, inPrc, outPrc, inLayout, outLayout, inputShapes, targetDevice) = obj.param; + std::vector kernel, stride, dilation; + std::vector padBegin, padEnd; + ngraph::op::PadType padType; + ngraph::op::RoundingType roundingType; + std::tie(kernel, stride, dilation, padBegin, padEnd, roundingType, padType) = poolParams; + + std::ostringstream result; + result << "IS=" << CommonTestUtils::vec2str(inputShapes) << "_"; + result << "K" << CommonTestUtils::vec2str(kernel) << "_"; + result << "S" << CommonTestUtils::vec2str(stride) << "_"; + result << "D" << CommonTestUtils::vec2str(dilation) << "_"; + result << "PB" << CommonTestUtils::vec2str(padBegin) << "_"; + result << "PE" << CommonTestUtils::vec2str(padEnd) << "_"; + result << "Rounding=" << roundingType << "_"; + result << "AutoPad=" << padType << "_"; + result << "netPRC=" << netPrecision.name() << "_"; + result << "inPRC=" << inPrc.name() << "_"; + result << "outPRC=" << outPrc.name() << "_"; + result << "inL=" << inLayout << "_"; + result << "outL=" << outLayout << "_"; + result << "trgDev=" << targetDevice; + return result.str(); +} + void PoolingLayerTest::SetUp() { poolSpecificParams poolParams; std::vector inputShape; @@ -159,4 +191,28 @@ void GlobalPoolingLayerTest::SetUp() { ngraph::ResultVector results{std::make_shared(pooling)}; function = std::make_shared(results, params, "pooling"); } + +void MaxPoolingV8LayerTest::SetUp() { + maxPoolV8SpecificParams poolParams; + std::vector inputShape; + InferenceEngine::Precision netPrecision; + std::tie(poolParams, netPrecision, inPrc, outPrc, inLayout, outLayout, inputShape, targetDevice) = this->GetParam(); + std::vector kernel, stride, dilation; + std::vector padBegin, padEnd; + ngraph::op::PadType padType; + ngraph::op::RoundingType roundingType; + std::tie(kernel, stride, dilation, padBegin, padEnd, roundingType, padType) = poolParams; + + auto ngPrc = FuncTestUtils::PrecisionUtils::convertIE2nGraphPrc(netPrecision); + auto params = ngraph::builder::makeParams(ngPrc, {inputShape}); + auto paramOuts = ngraph::helpers::convert2OutputVector( + ngraph::helpers::castOps2Nodes(params)); + + std::shared_ptr maxPool = ngraph::builder::makeMaxPoolingV8(paramOuts[0], stride, dilation, padBegin, padEnd, + kernel, roundingType, padType); + + ngraph::ResultVector results{std::make_shared(maxPool->output(0))}; + function = std::make_shared(results, params, "MaxPoolV8"); +} + } // namespace LayerTestsDefinitions diff --git a/inference-engine/tests/ie_test_utils/common_test_utils/data_utils.hpp b/inference-engine/tests/ie_test_utils/common_test_utils/data_utils.hpp index ba04710b1d0..7bcee22e8f3 100644 --- a/inference-engine/tests/ie_test_utils/common_test_utils/data_utils.hpp +++ b/inference-engine/tests/ie_test_utils/common_test_utils/data_utils.hpp @@ -418,6 +418,18 @@ void inline fill_data_random(InferenceEngine:: fill_data_random_float(blob, range, start_from, k, seed); } +template +static ov::runtime::Tensor create_tensor( + const ov::element::Type& element_type, + const ov::Shape& shape, + const std::vector& values, + const size_t size = 0) { + const size_t real_size = size ? size : values.size() * sizeof(T) / element_type.size(); + ov::runtime::Tensor tensor { element_type, shape }; + std::memcpy(tensor.data(), values.data(), std::min(real_size * element_type.size(), sizeof(T) * values.size())); + return tensor; +} + template typename std::enable_if::value, T>::type inline ie_abs(const T &val) { diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/builders.hpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/builders.hpp index 9d8183efc0e..efed4204a3d 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/builders.hpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/builders.hpp @@ -94,46 +94,7 @@ std::shared_ptr makeFakeQuantizeTypeRelaxed( const ngraph::element::Type constantPrecision, const FakeQuantizeOnDataWithConstant& fqOnData); -template -void addAttribute(std::vector> nodes, Args&& ... args) { - const auto attribute = std::make_shared>( - QuantizationAlignmentAttribute(std::forward(args)...)); - - for (const auto& node : nodes) { - node->get_rt_info()[ngraph::VariantWrapper::type_info.name] = attribute; - } -} - -template -void addAttribute2(std::vector> nodes, T attribute) { - const std::string typeInfoName = attribute->get_type_info().name; - for (const auto& node : nodes) { - auto& rt = node->get_rt_info(); - rt[typeInfoName] = attribute; - } -} - -template -void addAttribute3(std::vector> nodes, Args&& ... args) { - const auto attribute = std::make_shared<::ngraph::VariantWrapper>(T(std::forward(args)...)); - for (const auto& node : nodes) { - node->get_rt_info()[ngraph::VariantWrapper::type_info.name] = attribute; - } -} - -void addAttributes(std::vector> nodes, std::vector> attributes); - -template -std::shared_ptr make_shared_attribute(Args&& ... args) { - const auto attribute = std::make_shared<::ngraph::VariantWrapper>(T(std::forward(args)...)); - return attribute; -} - -template -std::shared_ptr make_shared_attribute_ptr(Args&& ... args) { - const auto attribute = std::make_shared<::ngraph::VariantWrapper>>(std::make_shared(std::forward(args)...)); - return attribute; -} +void addAttributes(std::vector> nodes, std::vector attributes); std::shared_ptr makeConvolution( const std::shared_ptr& parent, diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/fake_quantize_on_data.hpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/fake_quantize_on_data.hpp index af98d72327d..0ca59f7b5d0 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/fake_quantize_on_data.hpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/common/fake_quantize_on_data.hpp @@ -24,7 +24,7 @@ public: const std::vector& outputLowValues, const std::vector& outputHighValues, const ngraph::element::Type outputPrecision = ngraph::element::undefined, - const std::vector>& attributes = {}); + const std::vector& attributes = {}); virtual ~FakeQuantizeOnData(); @@ -38,7 +38,7 @@ public: std::vector outputLowValues; std::vector outputHighValues; ngraph::element::Type outputPrecision; - std::vector> attributes; + std::vector attributes; }; inline std::ostream& operator<<(std::ostream& os, const std::vector& values) { @@ -71,8 +71,8 @@ public: const std::vector& outputLowValues, const std::vector& outputHighValues, const ngraph::element::Type outputPrecision = ngraph::element::undefined, - const std::vector>& attributes = {}); + const std::vector& attributes = {}); virtual ~FakeQuantizeOnDataWithConstant(); virtual bool empty() const; @@ -84,7 +84,7 @@ public: std::vector outputLowValues; std::vector outputHighValues; ngraph::element::Type outputPrecision; - std::vector> attributes; + std::vector attributes; }; inline std::ostream& operator<<(std::ostream& out, const FakeQuantizeOnDataWithConstant& data) { diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/concat_function.hpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/concat_function.hpp index 0aa75c2cfbc..b723c4cfbe1 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/concat_function.hpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/concat_function.hpp @@ -123,7 +123,7 @@ public: const FakeQuantizeOnDataWithConstant& fakeQuantize2, const DequantizationOperations::Convert& convert2, const DequantizationOperations& dequantization2, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis, @@ -141,7 +141,7 @@ public: const DequantizationOperations::Convert& convert2, const DequantizationOperations& dequantization2, const bool addReshape2, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis, diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/move_fake_quantize_function.hpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/move_fake_quantize_function.hpp index 9640845c8be..bdd2f6fbc8c 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/move_fake_quantize_function.hpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/include/lpt_ngraph_functions/move_fake_quantize_function.hpp @@ -30,7 +30,7 @@ public: const FakeQuantizeOnDataWithConstant& fqOnData3, const DequantizationOperations::Convert& convert3, const DequantizationOperations& dequantization3, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis); diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/builders.cpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/builders.cpp index 0d639b78519..3c11cefbd63 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/builders.cpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/builders.cpp @@ -282,7 +282,9 @@ std::shared_ptr makeFakeQuantize( auto& rt = fq->get_rt_info(); for (auto& attribute : fqOnData.attributes) { - rt[attribute->get_type_info().name] = attribute; + if (attribute.is()) { + rt[attribute.as().get_type_info()] = attribute; + } } return fq; @@ -298,12 +300,12 @@ std::shared_ptr makeFakeQuantizeTypeRelaxed( fqOnData.outputPrecision == ngraph::element::undefined ? constantPrecision : fqOnData.outputPrecision); } -void addAttributes(std::vector> nodes, std::vector> attributes) { +void addAttributes(std::vector> nodes, std::vector attributes) { for (const auto& node : nodes) { for (const auto& attribute : attributes) { - auto& rt = node->get_rt_info(); - const std::string typeInfoName = attribute->get_type_info().name; - rt[typeInfoName] = attribute; + if (attribute.is()) { + node->get_rt_info()[attribute.as().get_type_info()] = attribute; + } } } } diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/fake_quantize_on_data.cpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/fake_quantize_on_data.cpp index 2c4f2468fe4..6affee88336 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/fake_quantize_on_data.cpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/common/fake_quantize_on_data.cpp @@ -19,7 +19,7 @@ FakeQuantizeOnData::FakeQuantizeOnData( const std::vector& outputLowValues, const std::vector& outputHighValues, const ngraph::element::Type outputPrecision, - const std::vector>& attributes) : + const std::vector& attributes) : quantizationLevel(quantizationLevel), constantShape(constantShape), inputLowValues(inputLowValues), @@ -58,7 +58,7 @@ FakeQuantizeOnDataWithConstant::FakeQuantizeOnDataWithConstant( const std::vector& outputLowValues, const std::vector& outputHighValues, const ngraph::element::Type outputPrecision, - const std::vector>& attributes) : + const std::vector& attributes) : quantizationLevel(quantizationLevel), constantShapes(constantShapes), inputLowValues(inputLowValues), diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/concat_function.cpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/concat_function.cpp index 28c04e0ed4d..9e5d914db09 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/concat_function.cpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/concat_function.cpp @@ -874,7 +874,7 @@ std::shared_ptr ConcatFunction::get( const FakeQuantizeOnDataWithConstant& fqOnData2, const DequantizationOperations::Convert& convert2, const DequantizationOperations& dequantization2, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis, @@ -960,7 +960,7 @@ std::shared_ptr ConcatFunction::get( const DequantizationOperations::Convert& convert2, const DequantizationOperations& dequantization2, const bool addReshape2, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis, diff --git a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/move_fake_quantize_function.cpp b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/move_fake_quantize_function.cpp index c53ddd3f6df..6c3ea3a09b9 100644 --- a/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/move_fake_quantize_function.cpp +++ b/inference-engine/tests/ngraph_helpers/lpt_ngraph_functions/src/move_fake_quantize_function.cpp @@ -32,7 +32,7 @@ std::shared_ptr MoveFakeQuantize::get( const FakeQuantizeOnDataWithConstant& fqOnData3, const DequantizationOperations::Convert& convert3, const DequantizationOperations& dequantization3, - const std::vector>& concatAttributes, + const std::vector& concatAttributes, const ngraph::element::Type precisionAfterOperation, const DequantizationOperations& dequantizationAfter, const std::int64_t& axis) { diff --git a/inference-engine/tests/ngraph_helpers/ngraph_functions/include/ngraph_functions/builders.hpp b/inference-engine/tests/ngraph_helpers/ngraph_functions/include/ngraph_functions/builders.hpp index 66ca6fb334a..b790b831c23 100644 --- a/inference-engine/tests/ngraph_helpers/ngraph_functions/include/ngraph_functions/builders.hpp +++ b/inference-engine/tests/ngraph_helpers/ngraph_functions/include/ngraph_functions/builders.hpp @@ -428,6 +428,15 @@ std::shared_ptr makePooling(const ngraph::Output &in, bool excludePad, const ngraph::helpers::PoolingTypes &poolType); +std::shared_ptr makeMaxPoolingV8(const ngraph::Output &in, + const std::vector &strides, + const std::vector &dilation, + const std::vector &padsBegin, + const std::vector &padsEnd, + const std::vector &kernel, + const op::RoundingType &roundingType, + const op::PadType &padType); + std::shared_ptr makeROIPooling(const Output& input, const Output& coords, const Shape& output_size, diff --git a/inference-engine/tests/ngraph_helpers/ngraph_functions/src/pooling.cpp b/inference-engine/tests/ngraph_helpers/ngraph_functions/src/pooling.cpp index b486ab8e1fe..43f26ba0272 100644 --- a/inference-engine/tests/ngraph_helpers/ngraph_functions/src/pooling.cpp +++ b/inference-engine/tests/ngraph_helpers/ngraph_functions/src/pooling.cpp @@ -35,5 +35,18 @@ std::shared_ptr makePooling(const ngraph::Output &in, return pooling; } +std::shared_ptr makeMaxPoolingV8(const ngraph::Output &in, + const std::vector &strides, + const std::vector &dilation, + const std::vector &padsBegin, + const std::vector &padsEnd, + const std::vector &kernel, + const op::RoundingType &roundingType, + const op::PadType &padType) { + std::shared_ptr pooling = std::make_shared(in, strides, dilation, padsBegin, padsEnd, + kernel, roundingType, padType); + return pooling; +} + } // namespace builder } // namespace ngraph diff --git a/inference-engine/tests/unit/auto/auto_select_device_failed_test.cpp b/inference-engine/tests/unit/auto/auto_select_device_failed_test.cpp index 521bd6efe6e..8925c63c301 100644 --- a/inference-engine/tests/unit/auto/auto_select_device_failed_test.cpp +++ b/inference-engine/tests/unit/auto/auto_select_device_failed_test.cpp @@ -17,6 +17,7 @@ #include #include "plugin/mock_auto_device_plugin.hpp" #include "cpp/ie_plugin.hpp" +#include "mock_common.hpp" using ::testing::MatcherCast; using ::testing::AllOf; @@ -33,15 +34,18 @@ using ::testing::AnyNumber; using Config = std::map; using namespace MockMultiDevice; -#define IE_SET_METRIC(key, name, ...) \ - typename ::InferenceEngine::Metrics::MetricType<::InferenceEngine::Metrics::key>::type name = \ - __VA_ARGS__; - using DeviceParams = std::tuple; +enum MODEL { + GENERAL = 0, + LATENCY = 1, + THROUGHPUT = 2, +}; + using ConfigParams = std::tuple< bool, // if can continue to run bool, // if select throw exception + MODEL, // config model general, latency, throughput std::vector, // {device, loadSuccess} unsigned int, // select count unsigned int, // load count @@ -72,7 +76,8 @@ public: std::vector> deviceConfigs; bool continueRun; bool thrExcWheSelect; - std::tie(continueRun, thrExcWheSelect, deviceConfigs, + MODEL configModel; + std::tie(continueRun, thrExcWheSelect, configModel, deviceConfigs, selectCount, loadCount, loadSuccessCount) = obj.param; std::ostringstream result; for (auto& item : deviceConfigs) { @@ -87,6 +92,22 @@ public: } else { result << "select_success_"; } + + switch (configModel) { + case GENERAL: + result << "GENERAL"; + break; + case LATENCY: + result << "LATENCY"; + break; + case THROUGHPUT: + result << "THROUGHPUT"; + break; + default: + LOG_ERROR("should not come here"); + break; + } + result << "select_" << selectCount << "_loadCount_" << loadCount << "_loadSuccessCount_" << loadSuccessCount; return result.str(); @@ -142,7 +163,8 @@ TEST_P(AutoLoadFailedTest, LoadCNNetWork) { std::vector> deviceConfigs; bool continueRun; bool thrExcWheSelect; - std::tie(continueRun, thrExcWheSelect, deviceConfigs, selectCount, + MODEL configModel; + std::tie(continueRun, thrExcWheSelect, configModel, deviceConfigs, selectCount, loadCount, loadSuccessCount) = this->GetParam(); // test auto plugin @@ -163,7 +185,24 @@ TEST_P(AutoLoadFailedTest, LoadCNNetWork) { ::testing::Matcher(_))) .WillByDefault(Throw(InferenceEngine::GeneralError{""})); } - DeviceInformation devInfo = {deviceName, {}, 2, ""}; + DeviceInformation devInfo; + switch (configModel) { + case GENERAL: + devInfo = {deviceName, {}, 2, ""}; + break; + case LATENCY: + devInfo = {deviceName, {{CONFIG_KEY(PERFORMANCE_HINT), + InferenceEngine::PluginConfigParams::LATENCY}}, 2, ""}; + break; + case THROUGHPUT: + devInfo = {deviceName, {{CONFIG_KEY(PERFORMANCE_HINT), + InferenceEngine::PluginConfigParams::THROUGHPUT}}, 2, ""}; + break; + default: + LOG_ERROR("should not come here"); + break; + } + metaDevices.push_back(std::move(devInfo)); // set the return value of SelectDevice // for example if there are three device, if will return GPU on the first call, and then MYRIAD @@ -207,13 +246,13 @@ TEST_P(AutoLoadFailedTest, LoadCNNetWork) { } // the test configure, for example -// ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, +// ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, // DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, // DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 3, 2}, // // every element for ConfigParams -// {continueRun, selectThrowException, deviceLoadsuccessVector, selectCount, loadCount, loadSuccessCount} -// { true, false, 3 device, 2, 3, 2} +// {continueRun, selectThrowException, config model, deviceLoadsuccessVector, selectCount, loadCount, loadSuccessCount} +// { true, false, GENERAL, 3 device, 2, 3, 2} // // there are three devices for loading // CPU load for accelerator success, but GPU will load faild and then select MYRIAD and load again @@ -223,52 +262,62 @@ TEST_P(AutoLoadFailedTest, LoadCNNetWork) { // the inference request num is loadSuccessCount * optimalNum, in this test case optimalNum is 2 // so inference request num is 4 (CPU 2, MYRIAD 2) // -const std::vector testConfigs = {ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, +const std::vector testConfigs = {ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 2, 2}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 3, 2}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 2, 2}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 2, 1}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 2, 1}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 2, 3, 1}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 3, 4, 2}, + ConfigParams {false, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 3, 4, 0}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 2, 2}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 3, 2}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, + DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 2, 1}, + ConfigParams {false, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 2, 3, 0}, + ConfigParams {false, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}}, 1, 1, 0}, + ConfigParams {false, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 1, 0}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}}, 1, 1, 1}, + ConfigParams {true, false, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 1, 1}, + ConfigParams {false, true, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}}, 1, 0, 0}, + ConfigParams {false, true, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 0, 0}, + ConfigParams {true, true, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1}, + ConfigParams {false, true, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, + DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 2, 2, 0}, + ConfigParams {true, true, GENERAL, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1}, + ConfigParams {true, false, LATENCY, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 3, 3, 1}, - ConfigParams {false, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + ConfigParams {true, false, LATENCY, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1}, + ConfigParams {true, false, THROUGHPUT, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, DeviceParams {CommonTestUtils::DEVICE_MYRIAD, false}, - DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 3, 3, 0}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, - DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 2, 2}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, - DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}, - DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 2, 1}, - ConfigParams {false, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, - DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 2, 2, 0}, - ConfigParams {false, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}}, 1, 1, 0}, - ConfigParams {false, false, {DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 1, 1, 0}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}}, 1, 1, 1}, - ConfigParams {true, false, {DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 1, 1}, - ConfigParams {false, true, {DeviceParams {CommonTestUtils::DEVICE_GPU, true}}, 1, 0, 0}, - ConfigParams {false, true, {DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 1, 0, 0}, - ConfigParams {true, true, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, - DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, - DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1}, - ConfigParams {false, true, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, - DeviceParams {CommonTestUtils::DEVICE_MYRIAD, true}, - DeviceParams {CommonTestUtils::DEVICE_CPU, false}}, 2, 2, 0}, - ConfigParams {true, true, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, - DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 2, 1} + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 3, 4, 2}, + ConfigParams {true, false, THROUGHPUT, {DeviceParams {CommonTestUtils::DEVICE_GPU, false}, + DeviceParams {CommonTestUtils::DEVICE_CPU, true}}, 2, 3, 2} }; INSTANTIATE_TEST_SUITE_P(smoke_Auto_BehaviorTests, AutoLoadFailedTest, diff --git a/inference-engine/tests/unit/auto/exec_network_get_metrics.cpp b/inference-engine/tests/unit/auto/exec_network_get_metrics.cpp new file mode 100644 index 00000000000..2916e6d9e1c --- /dev/null +++ b/inference-engine/tests/unit/auto/exec_network_get_metrics.cpp @@ -0,0 +1,269 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include +#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_icore.hpp" +#include "unit_test_utils/mocks/mock_iinfer_request.hpp" +#include "unit_test_utils/mocks/cpp_interfaces/impl/mock_inference_plugin_internal.hpp" +#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_iexecutable_network_internal.hpp" +#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_ivariable_state_internal.hpp" +#include "unit_test_utils/mocks/cpp_interfaces/interface/mock_iinference_plugin.hpp" +#include +#include +#include +#include +#include +#include "plugin/mock_auto_device_plugin.hpp" +#include "cpp/ie_plugin.hpp" +#include +#include +#include "mock_common.hpp" + +using ::testing::MatcherCast; +using ::testing::AllOf; +using ::testing::Throw; +using ::testing::Matches; +using ::testing::_; +using ::testing::StrEq; +using ::testing::Return; +using ::testing::Property; +using ::testing::Eq; +using ::testing::ReturnRef; +using ::testing::AtLeast; +using ::testing::AnyNumber; +using ::testing::InvokeWithoutArgs; +using Config = std::map; +using namespace MockMultiDevice; + +using ConfigParams = std::tuple< + unsigned int, // cpu OPTIMAL_NUMBER_OF_INFER_REQUESTS + int, // cpu infer requet num of customer want + bool, // if cpu sleep, cpu device will load slow + unsigned int, // gpu OPTIMAL_NUMBER_OF_INFER_REQUESTS + int, // gpu infer requet num of customer want + bool, // if gpu sleep, cpu device will load slow + unsigned int // expect OPTIMAL_NUMBER_OF_INFER_REQUESTS + >; +class ExecNetworkGetMetric : public ::testing::TestWithParam { +public: + std::shared_ptr function; + InferenceEngine::CNNNetwork cnnNet; + std::shared_ptr core; + std::shared_ptr plugin; + + //mock cpu exeNetwork + std::shared_ptr cpuMockIExeNet; + ov::runtime::SoPtr cpuMockExeNetwork; + MockIInferencePlugin* cpuMockIPlugin; + InferenceEngine::InferencePlugin cpuMockPlugin; + //mock gpu exeNetwork + std::shared_ptr gpuMockIExeNet; + ov::runtime::SoPtr gpuMockExeNetwork; + MockIInferencePlugin* gpuMockIPlugin; + InferenceEngine::InferencePlugin gpuMockPlugin; + // config for Auto device + std::map config; + std::vector metaDevices; + std::shared_ptr inferReqInternal; + +public: + static std::string getTestCaseName(testing::TestParamInfo obj) { + unsigned int cpuOptimalNum; + int cpuCustomerNum; + unsigned int gpuOptimalNum; + int gpuCustomerNum; + unsigned int expectOptimalNum; + bool cpuSleep; + bool gpuSleep; + std::tie(cpuOptimalNum, cpuCustomerNum, cpuSleep, + gpuOptimalNum, gpuCustomerNum, gpuSleep, expectOptimalNum) = obj.param; + std::ostringstream result; + result << "cpuOptimalNum_" << cpuOptimalNum << "cpuCustomerNum_" << cpuCustomerNum; + result << "gpuOptimalNum_" << gpuOptimalNum << "gpuCustomerNum_" << gpuCustomerNum; + result << "expectOptimalNum_" << expectOptimalNum; + if (cpuSleep) { + result << "_cpuSleep_" << "true"; + } else { + result << "_cpuSleep_" << "false"; + } + + if (gpuSleep) { + result << "_gpuSleep_" << "true"; + } else { + result << "_gpuSleep_" << "false"; + } + + return result.str(); + } + + void TearDown() override { + core.reset(); + plugin.reset(); + cpuMockIExeNet.reset(); + cpuMockExeNetwork = {}; + cpuMockPlugin = {}; + gpuMockIExeNet.reset(); + gpuMockExeNetwork = {}; + gpuMockPlugin = {}; + config.clear(); + metaDevices.clear(); + inferReqInternal.reset(); + } + + void SetUp() override { + // prepare cpuMockExeNetwork + cpuMockIExeNet = std::make_shared(); + auto cpuMockIPluginPtr = std::make_shared(); + ON_CALL(*cpuMockIPluginPtr, LoadNetwork(MatcherCast(_), _)).WillByDefault(Return(cpuMockIExeNet)); + cpuMockPlugin = InferenceEngine::InferencePlugin{{}, cpuMockIPluginPtr}; + // remove annoying ON CALL message + EXPECT_CALL(*cpuMockIPluginPtr, LoadNetwork(MatcherCast(_), _)).Times(1); + cpuMockExeNetwork = cpuMockPlugin.LoadNetwork(CNNNetwork{}, {}); + + // prepare gpuMockExeNetwork + gpuMockIExeNet = std::make_shared(); + auto gpuMockIPluginPtr = std::make_shared(); + ON_CALL(*gpuMockIPluginPtr, LoadNetwork(MatcherCast(_), _)).WillByDefault(Return(gpuMockIExeNet)); + gpuMockPlugin = InferenceEngine::InferencePlugin{{}, gpuMockIPluginPtr}; + // remove annoying ON CALL message + EXPECT_CALL(*gpuMockIPluginPtr, LoadNetwork(MatcherCast(_), _)).Times(1); + gpuMockExeNetwork = gpuMockPlugin.LoadNetwork(CNNNetwork{}, {}); + + // prepare mockicore and cnnNetwork for loading + core = std::shared_ptr(new MockICore()); + auto* origin_plugin = new MockMultiDeviceInferencePlugin(); + plugin = std::shared_ptr(origin_plugin); + function = ngraph::builder::subgraph::makeConvPoolRelu(); + cnnNet = InferenceEngine::CNNNetwork(function); + // replace core with mock Icore + plugin->SetCore(core); + // mock execNetwork can work + inferReqInternal = std::make_shared(); + ON_CALL(*cpuMockIExeNet.get(), CreateInferRequest()).WillByDefault(Return(inferReqInternal)); + ON_CALL(*gpuMockIExeNet.get(), CreateInferRequest()).WillByDefault(Return(inferReqInternal)); + EXPECT_CALL(*inferReqInternal, SetCallback).Times(AtLeast(1)); + IE_SET_METRIC(SUPPORTED_CONFIG_KEYS, supportConfigs, {}); + ON_CALL(*core, GetMetric(_, StrEq(METRIC_KEY(SUPPORTED_CONFIG_KEYS)), _)) + .WillByDefault(RETURN_MOCK_VALUE(supportConfigs)); + EXPECT_CALL(*core, GetMetric(_, StrEq(METRIC_KEY(SUPPORTED_CONFIG_KEYS)), _)).Times(AnyNumber()); + + // test auto plugin + config.insert({CONFIG_KEY_INTERNAL(MULTI_WORK_MODE_AS_AUTO), InferenceEngine::PluginConfigParams::YES}); + config.insert({InferenceEngine::MultiDeviceConfigParams::KEY_MULTI_DEVICE_PRIORITIES, + CommonTestUtils::DEVICE_CPU + std::string(",") + CommonTestUtils::DEVICE_GPU}); + + ON_CALL(*(HLogger), print(_)).WillByDefault([&](std::stringstream& stream) { + std::cout << stream.str() << std::endl; + }); + } +}; + +TEST_P(ExecNetworkGetMetric, OPTIMAL_NUMBER_OF_INFER_REQUESTS) { + unsigned int cpuOptimalNum; + int cpuCustomerNum; + unsigned int gpuOptimalNum; + int gpuCustomerNum; + unsigned int expectOptimalNum; + bool cpuSleep; + bool gpuSleep; + std::tie(cpuOptimalNum, cpuCustomerNum, cpuSleep, + gpuOptimalNum, gpuCustomerNum, gpuSleep, expectOptimalNum) = this->GetParam(); + + metaDevices.push_back({CommonTestUtils::DEVICE_CPU, {}, cpuCustomerNum, ""}); + metaDevices.push_back({CommonTestUtils::DEVICE_GPU, {}, gpuCustomerNum, ""}); + ON_CALL(*plugin, SelectDevice(_, _)).WillByDefault(Return(metaDevices[1])); + ON_CALL(*plugin, ParseMetaDevices(_, _)).WillByDefault(Return(metaDevices)); + EXPECT_CALL(*plugin, ParseMetaDevices(_, _)).Times(1); + EXPECT_CALL(*plugin, SelectDevice(_, _)).Times(1); + + if (cpuSleep) { + ON_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(StrEq(CommonTestUtils::DEVICE_CPU)), + ::testing::Matcher(_))).WillByDefault(InvokeWithoutArgs([this]() { + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + return cpuMockExeNetwork; + })); + } else { + ON_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(StrEq(CommonTestUtils::DEVICE_CPU)), + ::testing::Matcher(_))).WillByDefault(Return(cpuMockExeNetwork)); + } + + if (gpuSleep) { + ON_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(StrEq(CommonTestUtils::DEVICE_GPU)), + ::testing::Matcher(_))).WillByDefault(InvokeWithoutArgs([this]() { + std::this_thread::sleep_for(std::chrono::milliseconds(100)); + return gpuMockExeNetwork; + })); + } else { + ON_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(StrEq(CommonTestUtils::DEVICE_GPU)), + ::testing::Matcher(_))).WillByDefault(Return(gpuMockExeNetwork)); + } + + ON_CALL(*cpuMockIExeNet.get(), GetMetric(StrEq(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)))) + .WillByDefault(RETURN_MOCK_VALUE(cpuOptimalNum)); + ON_CALL(*gpuMockIExeNet.get(), GetMetric(StrEq(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)))) + .WillByDefault(RETURN_MOCK_VALUE(gpuOptimalNum)); + + EXPECT_CALL(*cpuMockIExeNet.get(), GetMetric(StrEq(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)))) + .Times(AtLeast(1)); + + EXPECT_CALL(*gpuMockIExeNet.get(), GetMetric(StrEq(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)))) + .Times(AtLeast(1)); + + EXPECT_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(CommonTestUtils::DEVICE_CPU), + ::testing::Matcher(_))).Times(1); + + EXPECT_CALL(*core, LoadNetwork(::testing::Matcher(_), + ::testing::Matcher(CommonTestUtils::DEVICE_GPU), + ::testing::Matcher(_))).Times(1); + + if (cpuCustomerNum == -1) { + EXPECT_CALL(*cpuMockIExeNet.get(), CreateInferRequest()).Times(cpuOptimalNum); + } else { + EXPECT_CALL(*cpuMockIExeNet.get(), CreateInferRequest()).Times(cpuCustomerNum); + } + + if (gpuCustomerNum == -1) { + EXPECT_CALL(*gpuMockIExeNet.get(), CreateInferRequest()).Times(gpuOptimalNum); + } else { + EXPECT_CALL(*gpuMockIExeNet.get(), CreateInferRequest()).Times(gpuCustomerNum); + } + + auto AutoExecNetwork = plugin->LoadExeNetworkImpl(cnnNet, config); + auto result = AutoExecNetwork->GetMetric(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)).as(); + EXPECT_EQ(result, expectOptimalNum); +} + + +// ConfigParams {unsigned int, int, bool, +// unsigned int, int, bool, unsigned int} +// +// every element for ConfigParams +// {cpuOptimalNum, customer hope for cpu infer requset num, if cpu sleep when load, +// gpuOptimalNum, customer hope for gpu infer requset num, if gpu sleep when load, +// expectOptimalNum of Auto ExecNetwork} +// +const std::vector testConfigs = { + ConfigParams {1, -1, false, 2, -1, true, 8}, + ConfigParams {1, -1, false, 10, -1, true, 8}, + ConfigParams {12, -1, false, 2, -1, true, 12}, + ConfigParams {12, -1, false, 10, -1, true, 12}, + ConfigParams {1, -1, true, 2, -1, false, 8}, + ConfigParams {1, -1, true, 10, -1, false, 10}, + ConfigParams {6, -1, true, 2, -1, false, 8}, + ConfigParams {6, -1, true, 10, -1, false, 10}, + ConfigParams {6, 4, false, 2, 3, true, 8}, + ConfigParams {6, 4, false, 10, 3, true, 8}, + ConfigParams {1, 4, true, 2, 3, false, 8}, + ConfigParams {1, 4, true, 10, 3, false, 10} + }; + +INSTANTIATE_TEST_SUITE_P(smoke_Auto_BehaviorTests, ExecNetworkGetMetric, + ::testing::ValuesIn(testConfigs), + ExecNetworkGetMetric::getTestCaseName); diff --git a/inference-engine/tests/unit/auto/mock_common.cpp b/inference-engine/tests/unit/auto/mock_common.cpp new file mode 100644 index 00000000000..c29608aa8c8 --- /dev/null +++ b/inference-engine/tests/unit/auto/mock_common.cpp @@ -0,0 +1,16 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "mock_common.hpp" + +// getMetric will return a fake ov::Any, gmock will call ostreamer << ov::Any +// it will cause core dump, so add this special implemented +namespace testing { +namespace internal { + template<> + void PrintTo(const ov::Any& a, std::ostream* os) { + *os << "using custom PrintTo ov::Any"; + } +} +} diff --git a/inference-engine/tests/unit/auto/mock_common.hpp b/inference-engine/tests/unit/auto/mock_common.hpp new file mode 100644 index 00000000000..401ebc22d85 --- /dev/null +++ b/inference-engine/tests/unit/auto/mock_common.hpp @@ -0,0 +1,24 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once +#include +#include +#include + +#define IE_SET_METRIC(key, name, ...) \ + typename ::InferenceEngine::Metrics::MetricType<::InferenceEngine::Metrics::key>::type name = \ + __VA_ARGS__; + +#define RETURN_MOCK_VALUE(value) \ + InvokeWithoutArgs([value](){return value;}) + +// getMetric will return a fake ov::Any, gmock will call ostreamer << ov::Any +// it will cause core dump, so add this special implemented +namespace testing { +namespace internal { + template<> + void PrintTo(const ov::Any& a, std::ostream* os); +} +} diff --git a/inference-engine/tests/unit/inference_engine/ie_compilation_context_test.cpp b/inference-engine/tests/unit/inference_engine/ie_compilation_context_test.cpp index 1989dd11bab..d3f49772975 100644 --- a/inference-engine/tests/unit/inference_engine/ie_compilation_context_test.cpp +++ b/inference-engine/tests/unit/inference_engine/ie_compilation_context_test.cpp @@ -134,13 +134,22 @@ static std::shared_ptr create_simple_function() { data->get_output_tensor(0).set_names({"parameter"}); auto mul_constant = ngraph::opset6::Constant::create(ngraph::element::i8, ngraph::Shape{1}, {3}); + mul_constant->set_friendly_name("mul_constant"); + mul_constant->get_output_tensor(0).set_names({"mul_constant"}); auto mul = std::make_shared(data, mul_constant); + mul->set_friendly_name("mul"); + mul->get_output_tensor(0).set_names({"mul"}); auto add_constant = ngraph::opset6::Constant::create(ngraph::element::i8, ngraph::Shape{1}, {2}); + add_constant->set_friendly_name("add_constant"); + add_constant->get_output_tensor(0).set_names({"add_constant"}); auto add = std::make_shared(mul, add_constant); + add->set_friendly_name("add"); + add->get_output_tensor(0).set_names({"add"}); // Create opset3::Result operation auto res = std::make_shared(add); + res->set_friendly_name("res"); // Create nGraph function auto func = std::make_shared(ngraph::ResultVector{res}, ngraph::ParameterVector{data}); @@ -208,7 +217,7 @@ TEST(NetworkContext_CNNNetwork, HashWithPrimitivesPriority) { auto net2 = createNetwork(); auto net3 = createNetwork(); auto & op2 = net2.getFunction()->get_ops().front()->get_rt_info(); - op2["PrimitivesPriority"] = std::make_shared > ("testPriority"); + op2[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority("testPriority"); auto & op3 = net3.getFunction()->get_ops().front()->get_rt_info(); op3["PrimitivesPriority"] = std::make_shared > ("testPriority"); @@ -222,24 +231,20 @@ TEST(NetworkContext_CNNNetwork, HashWithPrimitivesPriority) { TEST(NetworkContext_CNNNetwork, HashWithFusedNames) { auto setFusedEmpty = [&](Node::RTMap& rtInfo) { - rtInfo[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames()); + rtInfo[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames(); }; auto setFused = [&](Node::RTMap& rtInfo, const std::string& name) { - rtInfo[VariantWrapper::get_type_info_static()] = - std::make_shared>(ngraph::FusedNames(name)); + rtInfo[ngraph::FusedNames::get_type_info_static()] = ngraph::FusedNames(name); }; checkCustomRt(setFusedEmpty, setFused); } TEST(NetworkContext_CNNNetwork, HashWithPrimitivesPriorityType) { auto setPrimEmpty = [&](Node::RTMap& rtInfo) { - rtInfo[ov::PrimitivesPriority::get_type_info_static()] = - std::make_shared(""); + rtInfo[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority(""); }; auto setPrim = [&](Node::RTMap& rtInfo, const std::string& name) { - rtInfo[ov::PrimitivesPriority::get_type_info_static()] = - std::make_shared(name); + rtInfo[ov::PrimitivesPriority::get_type_info_static()] = ov::PrimitivesPriority(name); }; checkCustomRt(setPrimEmpty, setPrim); } diff --git a/inference-engine/thirdparty/clDNN/api/cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp b/inference-engine/thirdparty/clDNN/api/cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp new file mode 100644 index 00000000000..c67006df698 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/api/cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp @@ -0,0 +1,56 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +/////////////////////////////////////////////////////////////////////////////////////////////////// +#pragma once +#include "primitive.hpp" +#include + +namespace cldnn { +/// @addtogroup cpp_api C++ API +/// @{ +/// @addtogroup cpp_topology Network Topology +/// @{ +/// @addtogroup cpp_primitives Primitives +/// @{ + +/// @brief experimental detectron ROI feature extractor +struct experimental_detectron_roi_feature_extractor : public primitive_base { + CLDNN_DECLARE_PRIMITIVE(experimental_detectron_roi_feature_extractor) + + /// @brief Constructs experimental_detectron_roi_feature_extractor primitive + /// @param id This primitive id + /// @param inputs Inputs for primitive id (ROIs, {pyramid levels, ...}, second_output) + /// @param output_dim Attribute specifies the width and height of the output tensor + /// @param pyramid_scales Scales of pyramid levels + /// @param sampling_ratio Attribute specifies the number of sampling points per the output value + /// @param aligned Attribute specifies add offset (-0.5) to ROIs sizes or not + experimental_detectron_roi_feature_extractor(const primitive_id& id, + const std::vector& inputs, + int output_dim, + const std::vector& pyramid_scales, + int sampling_ratio, + bool aligned, + const primitive_id& ext_prim_id = "", + const padding& output_padding = padding()) : + primitive_base(id, inputs, ext_prim_id, output_padding), + output_dim(output_dim), + pooled_height(output_dim), + pooled_width(output_dim), + pyramid_scales(pyramid_scales), + sampling_ratio(sampling_ratio), + aligned(aligned) {} + + int output_dim = 0; + int pooled_height = 0; + int pooled_width = 0; + std::vector pyramid_scales; + int sampling_ratio = 0; + bool aligned = false; +}; + +/// @} +/// @} +/// @} +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/api/cldnn/primitives/roi_align.hpp b/inference-engine/thirdparty/clDNN/api/cldnn/primitives/roi_align.hpp new file mode 100644 index 00000000000..f2ca12cb445 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/api/cldnn/primitives/roi_align.hpp @@ -0,0 +1,69 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#pragma once +#include "primitive.hpp" +#include + +namespace cldnn { +/// @addtogroup cpp_api C++ API +/// @{ +/// @addtogroup cpp_topology Network Topology +/// @{ +/// @addtogroup cpp_primitives Primitives +/// @{ + +/// @brief ROIAlign is a pooling layer used over feature maps of +/// non-uniform input sizes and outputs a feature map of a fixed size. +struct roi_align : public primitive_base { + CLDNN_DECLARE_PRIMITIVE(roi_align) + + /// @brief Pooling mode for the @ref roi_align + enum PoolingMode { + Max, + Avg + }; + + /// @brief Constructs roi_align primitive. + /// @param id This primitive id. + /// @param inputs Inputs data primitive ids. + /// @param pooled_h Height of the ROI output feature map. + /// @param pooled_w Width of the ROI output feature map. + /// @param sampling_ratio Number of bins over height and width to use to calculate each output feature map element. + /// @param spatial_scale multiplicative spatial scale factor to translate ROI coordinates + /// from their input spatial scale to the scale used when pooling. + /// @param mode Method to perform pooling to produce output feature map elements. + /// @param shrink_axis_mask Array of bits, that provide shrinks the dimensionality by 1, taking on the value at index begin[i]. + roi_align(const primitive_id& id, + const std::vector& inputs, + int pooled_h, + int pooled_w, + int sampling_ratio, + float spatial_scale, + PoolingMode mode, + const primitive_id& ext_prim_id = "", + const padding& output_padding = padding()) + : primitive_base(id, inputs, ext_prim_id, output_padding), + pooled_h {pooled_h}, + pooled_w {pooled_w}, + sampling_ratio {sampling_ratio}, + spatial_scale {spatial_scale}, + mode {mode} + {} + + /// @brief Height of the ROI output feature map. + int pooled_h; + /// @brief Width of the ROI output feature map. + int pooled_w; + /// @brief Number of bins over height and width to use to calculate each output feature map element. + int sampling_ratio; + /// @brief multiplicative spatial scale factor to translate ROI coordinates + /// from their input spatial scale to the scale used when pooling. + float spatial_scale; + /// @brief Method to perform pooling to produce output feature map elements. + PoolingMode mode; +}; +/// @} +/// @} +/// @} +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/api/cldnn/runtime/tensor.hpp b/inference-engine/thirdparty/clDNN/api/cldnn/runtime/tensor.hpp index 37b6effb9aa..a6756b9303d 100644 --- a/inference-engine/thirdparty/clDNN/api/cldnn/runtime/tensor.hpp +++ b/inference-engine/thirdparty/clDNN/api/cldnn/runtime/tensor.hpp @@ -34,8 +34,6 @@ struct format_traits { size_t feature_num; /// @brief Number of spatial (x,y) dimensions in a format. size_t spatial_num; - /// @brief Number of local (x,y) dimensions in a format. - size_t local_num; /// @brief Number of groups in a format. size_t group_num; /// @brief Dimensions changing order from rare to often. @@ -50,8 +48,6 @@ struct format_traits { static const char* feature_chars() { return "fic"; } /// @brief Characters representing spatial dimensions in an order. static const char* spatial_chars() { return "xyzhsw"; } - /// @brief Characters representing local dimensions in an order. - static const char* local_chars() { return "kl"; } /// @brief Characters representing group dimensions in an order. static const char* group_chars() { return "g"; } /// @brief Checks if @p c represents batch dimension. @@ -60,8 +56,6 @@ struct format_traits { static bool is_feature_char(char c) { return std::string(feature_chars()).find_first_of(c) != std::string::npos; } /// @brief Checks if @p c represents spatial dimension. static bool is_spatial_char(char c) { return std::string(spatial_chars()).find_first_of(c) != std::string::npos; } - /// @brief Checks if @p c represents local dimensions. - static bool is_local_char(char c) { return std::string(local_chars()).find_first_of(c) != std::string::npos; } /// @brief Checks if @p c represents group dimensions. static bool is_group_char(char c) { return std::string(group_chars()).find_first_of(c) != std::string::npos; } }; @@ -235,139 +229,138 @@ struct format { // B - number of Batch dimensions // F - number of Feature dimensions // S - number of Spatial dimensions - // L - number of Local dimensions // G - number of Group dimensions // Order - dims changing order from rare to often // Inner order - dims order for internal storage in _sizes array // Block sizes - vector of pairs of dimension number (by inner order) and block size ordered from rare to often - // Format B F S L G Order Inner order Block sizes - { yxfb, { 1, 1, 2, 0, 0, "yxfb", "bfxy?", {}}}, - { byxf, { 1, 1, 2, 0, 0, "byxf", "bfxy?", {}}}, - { bfyx, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {}}}, - { fyxb, { 1, 1, 2, 0, 0, "fyxb", "bfxy?", {}}}, - { b_fs_yx_fsv16, { 1, 1, 2, 0, 0, "bfyx", "bfxy", {{1, 16}}}}, - { b_fs_yx_fsv32, { 1, 1, 2, 0, 0, "bfyx", "bfxy", {{1, 32}}}}, - { b_fs_zyx_fsv32, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{1, 32}}}}, - { bs_xs_xsv8_bsv8, { 1, 1, 1, 0, 0, "bx", "b?x??", {{2, 8}, {0, 8}}}}, - { bs_xs_xsv8_bsv16, { 1, 1, 1, 0, 0, "bx", "b?x??", {{2, 8}, {0, 16}}}}, - { bs_x_bsv16, { 1, 1, 1, 0, 0, "bx", "b?x??", {{0, 16}}}}, - { winograd_2x3_s1_data, { 1, 1, 2, 0, 0, "bxyf", "bfxy?", {}}}, - { b_fs_yx_fsv4, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{1, 4}}}}, - { bfzyx, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {}}}, - { bfwzyx, { 1, 1, 4, 0, 0, "bfwzyx", "bfxyzw", {}}}, - { fs_b_yx_fsv32, { 1, 1, 2, 0, 0, "fbyx", "bfxy?", {{1, 32}}}}, - { b_fs_yx_32fp, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {}}}, - { b_fs_zyx_fsv16, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{1, 16}}}}, - { bs_fs_zyx_bsv16_fsv16, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{0, 16 }, {1, 16}}}}, - { bs_fs_yx_bsv16_fsv16, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{0, 16 }, {1, 16}}}}, - { bs_fs_yx_bsv4_fsv4, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{0, 4 }, {1, 4}}}}, - { bs_fs_yx_bsv4_fsv2, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{0, 4 }, {1, 2}}}}, - { bs_fs_zyx_bsv4_fsv4, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{0, 4 }, {1, 4}}}}, - { bs_fs_zyx_bsv4_fsv2, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{0, 4 }, {1, 2}}}}, - { bs_fs_zyx_bsv32_fsv32, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{0, 32 }, {1, 32}}}}, - { bs_fs_zyx_bsv32_fsv16, { 1, 1, 3, 0, 0, "bfzyx", "bfxyz", {{0, 32 }, {1, 16}}}}, - { bs_fs_yx_bsv32_fsv32, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{0, 32 }, {1, 32}}}}, - { bs_fs_yx_bsv32_fsv16, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {{0, 32 }, {1, 16}}}}, - { nv12, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {}}}, - { image_2d_rgba, { 1, 1, 2, 0, 0, "bfyx", "bfxy?", {}}}, + // Format B F S G Order Inner order Block sizes + { yxfb, { 1, 1, 2, 0, "yxfb", "bfxy?", {}}}, + { byxf, { 1, 1, 2, 0, "byxf", "bfxy?", {}}}, + { bfyx, { 1, 1, 2, 0, "bfyx", "bfxy?", {}}}, + { fyxb, { 1, 1, 2, 0, "fyxb", "bfxy?", {}}}, + { b_fs_yx_fsv16, { 1, 1, 2, 0, "bfyx", "bfxy", {{1, 16}}}}, + { b_fs_yx_fsv32, { 1, 1, 2, 0, "bfyx", "bfxy", {{1, 32}}}}, + { b_fs_zyx_fsv32, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{1, 32}}}}, + { bs_xs_xsv8_bsv8, { 1, 1, 1, 0, "bx", "b?x??", {{2, 8}, {0, 8}}}}, + { bs_xs_xsv8_bsv16, { 1, 1, 1, 0, "bx", "b?x??", {{2, 8}, {0, 16}}}}, + { bs_x_bsv16, { 1, 1, 1, 0, "bx", "b?x??", {{0, 16}}}}, + { winograd_2x3_s1_data, { 1, 1, 2, 0, "bxyf", "bfxy?", {}}}, + { b_fs_yx_fsv4, { 1, 1, 2, 0, "bfyx", "bfxy?", {{1, 4}}}}, + { bfzyx, { 1, 1, 3, 0, "bfzyx", "bfxyz", {}}}, + { bfwzyx, { 1, 1, 4, 0, "bfwzyx", "bfxyzw", {}}}, + { fs_b_yx_fsv32, { 1, 1, 2, 0, "fbyx", "bfxy?", {{1, 32}}}}, + { b_fs_yx_32fp, { 1, 1, 2, 0, "bfyx", "bfxy?", {}}}, + { b_fs_zyx_fsv16, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{1, 16}}}}, + { bs_fs_zyx_bsv16_fsv16, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{0, 16 }, {1, 16}}}}, + { bs_fs_yx_bsv16_fsv16, { 1, 1, 2, 0, "bfyx", "bfxy?", {{0, 16 }, {1, 16}}}}, + { bs_fs_yx_bsv4_fsv4, { 1, 1, 2, 0, "bfyx", "bfxy?", {{0, 4 }, {1, 4}}}}, + { bs_fs_yx_bsv4_fsv2, { 1, 1, 2, 0, "bfyx", "bfxy?", {{0, 4 }, {1, 2}}}}, + { bs_fs_zyx_bsv4_fsv4, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{0, 4 }, {1, 4}}}}, + { bs_fs_zyx_bsv4_fsv2, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{0, 4 }, {1, 2}}}}, + { bs_fs_zyx_bsv32_fsv32, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{0, 32 }, {1, 32}}}}, + { bs_fs_zyx_bsv32_fsv16, { 1, 1, 3, 0, "bfzyx", "bfxyz", {{0, 32 }, {1, 16}}}}, + { bs_fs_yx_bsv32_fsv32, { 1, 1, 2, 0, "bfyx", "bfxy?", {{0, 32 }, {1, 32}}}}, + { bs_fs_yx_bsv32_fsv16, { 1, 1, 2, 0, "bfyx", "bfxy?", {{0, 32 }, {1, 16}}}}, + { nv12, { 1, 1, 2, 0, "bfyx", "bfxy?", {}}}, + { image_2d_rgba, { 1, 1, 2, 0, "bfyx", "bfxy?", {}}}, - { oiyx, { 1, 1, 2, 0, 0, "oiyx", "oixy", {}}}, - { ioyx, { 1, 1, 2, 0, 0, "ioyx", "oixy", {}}}, - { iyxo, { 1, 1, 2, 0, 0, "iyxo", "oixy", {}}}, - { yxio, { 1, 1, 2, 0, 0, "yxio", "oixy?", {}}}, - { oizyx, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {}}}, - { iozyx, { 1, 1, 3, 0, 0, "iozyx", "oixyz", {}}}, - { os_is_yx_isv16_osv16, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{1, 16}, {0, 16}}}}, - { o_is_yx_isv16, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{1, 16}}}}, - { os_yxi_osv16, { 1, 1, 2, 0, 0, "oyxi", "oixy?", {{0, 16}}}}, - { os_iyx_osv16, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{0, 16}}}}, - { os_iyx_osv32, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{0, 32}}}}, - { os_iyx_osv64, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{0, 64}}}}, - { winograd_2x3_s1_weights, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { winograd_2x3_s1_fused_weights, { 1, 1, 2, 0, 0, "xyio", "oixy?", {}}}, - { winograd_6x3_s1_fused_weights, { 1, 1, 2, 0, 0, "xyio", "oixy?", {}}}, - { image_2d_weights_winograd_6x3_s1_fbxyb, { 1, 1, 2, 0, 0, "xyio", "oixy?", {}}}, - { image_2d_weights_winograd_6x3_s1_xfbyb, { 1, 1, 2, 0, 0, "xyio", "oixy?", {}}}, - { image_2d_weights_c4_fyx_b, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { image_2d_weights_c1_b_fyx, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { lstm_weights_dio, { 1, 1, 2, 0, 0, "oixy", "oixy?", {}}}, - { os_is_yx_isa8_osv8_isv4, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { os_is_yx_isa8_osv16_isv4, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { os_is_yx_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {}}}, - { os_is_yx_osa4_isa8_osv8_isv2, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{0, 32}, {1, 16}}}}, - { os_is_yx_osa4_isa8_osv8_isv4, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{0, 32}, {1, 32}}}}, - { os_is_zyx_osa4_isa8_osv8_isv2, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 32}, {1, 16}}}}, - { os_is_zyx_osa4_isa8_osv8_isv4, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 32}, {1, 32}}}}, - { os_is_yx_osa2_isa8_osv16_isv2, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{0, 32}, {1, 16}}}}, - { os_is_yx_osa2_isa8_osv16_isv4, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{0, 32}, {1, 32}}}}, - { os_is_zyx_isa8_osv8_isv4, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{1, 8}, {0, 8}, {1, 4}}}}, - { os_is_zyx_isa8_osv16_isv4, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 4}}}}, - { os_is_yx_osa4_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, 0, "oiyx", "oixy?", {{0, 32}, {1, 32}}}}, - { os_is_zyx_osa4_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 32}, {1, 32}}}}, - { is_o_yx_isv32, { 1, 1, 2, 0, 0, "oyxi", "oixy?", {{1, 32}}}}, - { is_o32_yx_isv32_swizzled_by_4, { 1, 1, 2, 0, 0, "oyxi", "oixy?", {}}}, - { os_is_y_x8_osv8_isv4, { 1, 1, 2, 0, 0, "oyxi", "oixy?", {}}}, - { os_is_y_x8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, 0, "oyxi", "oixy?", {}}}, - { os_is_yx_osv16_isv4, { 1, 1, 2, 0, 0, "oixy", "oixy?", {{0, 16}, {1, 4}}}}, - { os_is_yx_osv8_isv4, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{1, 4}, {0, 8}}}}, - { os_is_yx_osv8_isv2, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{1, 2}, {0, 8}}}}, - { os_is_zyx_osv16_isv16, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 16}, {1, 16}}}}, - { os_is_yx_osv32_isv4_swizzled_by_2, { 1, 1, 2, 0, 0, "oixy", "oixy?", {{0, 32}, {1, 4}}}}, - { os_is_yx_osv32_isv4, { 1, 1, 2, 0, 0, "oixy", "oixy?", {{0, 32}, {1, 4}}}}, - { os_is_zyx_osv32_isv4, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 32}, {1, 4}}}}, - { os_is_yx_osv32_isv32p, { 1, 1, 1, 0, 0, "oixy", "oixy?", {}}}, - { os_is_zyx_isv16_osv16, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 16}, {1, 16}}}}, - { is_os_zyx_isv16_osv16, { 1, 1, 3, 0, 0, "iozyx", "oixyz", {{1, 16}, {0, 16}}}}, - { is_os_yx_isv16_osv16, { 1, 1, 2, 0, 0, "ioyx", "oixyz", {{1, 16}, {0, 16}}}}, - { os_is_osv32_isv32_swizzled_by_4, { 1, 1, 0, 0, 0, "oixy", "oixy?", {{0, 32}, {1, 32}}}}, - { os_is_zyx_isv8_osv16_isv2, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 2}}}}, - { os_zyxi_osv16, { 1, 1, 3, 0, 0, "ozyxi", "oixyz", {{0, 16}}}}, - { os_is_yx_isv8_osv16_isv2, { 1, 1, 2, 0, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 2}}}}, - { os_is_yx_osv16_isv16, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{1, 16}, {0, 16}}}}, - { os_is_zyx_osv32_isv16, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 32}, {1, 16}}}}, - { os_is_zyx_osv64_isv16, { 1, 1, 3, 0, 0, "oizyx", "oixyz", {{0, 64}, {1, 16}}}}, - { os_iyx_osv32__ai32, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{0, 32}}}}, - { i_yxs_os_yxsv2_osv16, { 1, 1, 2, 0, 0, "iyxo", "oixy", {{0, 16}}}}, - { iy_xs_os_xsv2_osv8__ao32, { 1, 1, 2, 0, 0, "iyxo", "oixy", {{2, 2}, {0, 8}}}}, - { iy_xs_os_xsv2_osv16__ao32, { 1, 1, 2, 0, 0, "iyxo", "oixy", {{2, 2}, {0, 16}}}}, - { os_i_yxs_osv4_yxsv4, { 1, 1, 2, 0, 0, "oiyx", "oixy", {{0, 4}}}}, + { oiyx, { 1, 1, 2, 0, "oiyx", "oixy", {}}}, + { ioyx, { 1, 1, 2, 0, "ioyx", "oixy", {}}}, + { iyxo, { 1, 1, 2, 0, "iyxo", "oixy", {}}}, + { yxio, { 1, 1, 2, 0, "yxio", "oixy?", {}}}, + { oizyx, { 1, 1, 3, 0, "oizyx", "oixyz", {}}}, + { iozyx, { 1, 1, 3, 0, "iozyx", "oixyz", {}}}, + { os_is_yx_isv16_osv16, { 1, 1, 2, 0, "oiyx", "oixy", {{1, 16}, {0, 16}}}}, + { o_is_yx_isv16, { 1, 1, 2, 0, "oiyx", "oixy?", {{1, 16}}}}, + { os_yxi_osv16, { 1, 1, 2, 0, "oyxi", "oixy?", {{0, 16}}}}, + { os_iyx_osv16, { 1, 1, 2, 0, "oiyx", "oixy?", {{0, 16}}}}, + { os_iyx_osv32, { 1, 1, 2, 0, "oiyx", "oixy?", {{0, 32}}}}, + { os_iyx_osv64, { 1, 1, 2, 0, "oiyx", "oixy?", {{0, 64}}}}, + { winograd_2x3_s1_weights, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { winograd_2x3_s1_fused_weights, { 1, 1, 2, 0, "xyio", "oixy?", {}}}, + { winograd_6x3_s1_fused_weights, { 1, 1, 2, 0, "xyio", "oixy?", {}}}, + { image_2d_weights_winograd_6x3_s1_fbxyb, { 1, 1, 2, 0, "xyio", "oixy?", {}}}, + { image_2d_weights_winograd_6x3_s1_xfbyb, { 1, 1, 2, 0, "xyio", "oixy?", {}}}, + { image_2d_weights_c4_fyx_b, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { image_2d_weights_c1_b_fyx, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { lstm_weights_dio, { 1, 1, 2, 0, "oixy", "oixy?", {}}}, + { os_is_yx_isa8_osv8_isv4, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { os_is_yx_isa8_osv16_isv4, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { os_is_yx_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, "oiyx", "oixy?", {}}}, + { os_is_yx_osa4_isa8_osv8_isv2, { 1, 1, 2, 0, "oiyx", "oixy?", {{0, 32}, {1, 16}}}}, + { os_is_yx_osa4_isa8_osv8_isv4, { 1, 1, 2, 0, "oiyx", "oixy", {{0, 32}, {1, 32}}}}, + { os_is_zyx_osa4_isa8_osv8_isv2, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 32}, {1, 16}}}}, + { os_is_zyx_osa4_isa8_osv8_isv4, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 32}, {1, 32}}}}, + { os_is_yx_osa2_isa8_osv16_isv2, { 1, 1, 2, 0, "oiyx", "oixy", {{0, 32}, {1, 16}}}}, + { os_is_yx_osa2_isa8_osv16_isv4, { 1, 1, 2, 0, "oiyx", "oixy", {{0, 32}, {1, 32}}}}, + { os_is_zyx_isa8_osv8_isv4, { 1, 1, 3, 0, "oizyx", "oixyz", {{1, 8}, {0, 8}, {1, 4}}}}, + { os_is_zyx_isa8_osv16_isv4, { 1, 1, 3, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 4}}}}, + { os_is_yx_osa4_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, "oiyx", "oixy?", {{0, 32}, {1, 32}}}}, + { os_is_zyx_osa4_isa8_osv8_isv4_swizzled_by_4, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 32}, {1, 32}}}}, + { is_o_yx_isv32, { 1, 1, 2, 0, "oyxi", "oixy?", {{1, 32}}}}, + { is_o32_yx_isv32_swizzled_by_4, { 1, 1, 2, 0, "oyxi", "oixy?", {}}}, + { os_is_y_x8_osv8_isv4, { 1, 1, 2, 0, "oyxi", "oixy?", {}}}, + { os_is_y_x8_osv8_isv4_swizzled_by_4, { 1, 1, 2, 0, "oyxi", "oixy?", {}}}, + { os_is_yx_osv16_isv4, { 1, 1, 2, 0, "oixy", "oixy?", {{0, 16}, {1, 4}}}}, + { os_is_yx_osv8_isv4, { 1, 1, 2, 0, "oiyx", "oixy", {{1, 4}, {0, 8}}}}, + { os_is_yx_osv8_isv2, { 1, 1, 2, 0, "oiyx", "oixy", {{1, 2}, {0, 8}}}}, + { os_is_zyx_osv16_isv16, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 16}, {1, 16}}}}, + { os_is_yx_osv32_isv4_swizzled_by_2, { 1, 1, 2, 0, "oixy", "oixy?", {{0, 32}, {1, 4}}}}, + { os_is_yx_osv32_isv4, { 1, 1, 2, 0, "oixy", "oixy?", {{0, 32}, {1, 4}}}}, + { os_is_zyx_osv32_isv4, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 32}, {1, 4}}}}, + { os_is_yx_osv32_isv32p, { 1, 1, 1, 0, "oixy", "oixy?", {}}}, + { os_is_zyx_isv16_osv16, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 16}, {1, 16}}}}, + { is_os_zyx_isv16_osv16, { 1, 1, 3, 0, "iozyx", "oixyz", {{1, 16}, {0, 16}}}}, + { is_os_yx_isv16_osv16, { 1, 1, 2, 0, "ioyx", "oixyz", {{1, 16}, {0, 16}}}}, + { os_is_osv32_isv32_swizzled_by_4, { 1, 1, 0, 0, "oixy", "oixy?", {{0, 32}, {1, 32}}}}, + { os_is_zyx_isv8_osv16_isv2, { 1, 1, 3, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 2}}}}, + { os_zyxi_osv16, { 1, 1, 3, 0, "ozyxi", "oixyz", {{0, 16}}}}, + { os_is_yx_isv8_osv16_isv2, { 1, 1, 2, 0, "oizyx", "oixyz", {{1, 8}, {0, 16}, {1, 2}}}}, + { os_is_yx_osv16_isv16, { 1, 1, 2, 0, "oiyx", "oixy", {{1, 16}, {0, 16}}}}, + { os_is_zyx_osv32_isv16, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 32}, {1, 16}}}}, + { os_is_zyx_osv64_isv16, { 1, 1, 3, 0, "oizyx", "oixyz", {{0, 64}, {1, 16}}}}, + { os_iyx_osv32__ai32, { 1, 1, 2, 0, "oiyx", "oixy", {{0, 32}}}}, + { i_yxs_os_yxsv2_osv16, { 1, 1, 2, 0, "iyxo", "oixy", {{0, 16}}}}, + { iy_xs_os_xsv2_osv8__ao32, { 1, 1, 2, 0, "iyxo", "oixy", {{2, 2}, {0, 8}}}}, + { iy_xs_os_xsv2_osv16__ao32, { 1, 1, 2, 0, "iyxo", "oixy", {{2, 2}, {0, 16}}}}, + { os_i_yxs_osv4_yxsv4, { 1, 1, 2, 0, "oiyx", "oixy", {{0, 4}}}}, - { goiyx, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {}}}, - { gioyx, { 1, 1, 2, 0, 1, "gioyx", "oixy????g", {}}}, - { goizyx, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {}}}, - { giozyx, { 1, 1, 3, 0, 1, "giozyx", "oixyz???g", {}}}, - { g_os_iyx_osv16, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 16}}}}, - { g_os_iyx_osv32, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 32}}}}, - { gs_oiyx_gsv16, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{8, 16}}}}, - { gs_oizyx_gsv16, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{8, 16}}}}, - { gs_oiyx_gsv32, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{8, 32}}}}, - { gyxio, { 1, 1, 2, 0, 1, "gyxio", "oixy????g", {}}}, - { g_is_os_zyx_isv16_osv16, { 1, 1, 3, 0, 1, "giozyx", "oixyz???g", {{1, 16}, {0, 16}}}}, - { g_is_os_yx_isv16_osv16, { 1, 1, 2, 0, 1, "gioyx", "oixy????g", {{1, 16}, {0, 16}}}}, - { g_os_is_zyx_isv8_osv16_isv2, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{1, 8}, {0, 16}, {1, 2}}}}, - { g_os_is_yx_isv8_osv16_isv2, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{1, 8}, {0, 16}, {1, 2}}}}, - { g_os_is_zyx_isv16_osv16, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{0, 16}, {1, 16}}}}, - { g_os_is_yx_osv16_isv4, { 1, 1, 2, 0, 1, "goixy", "oixy????g", {{0, 16}, {1, 4}}}}, - { g_os_is_zyx_osv16_isv16, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{0, 16}, {1, 16}}}}, - { g_os_zyx_is_osv16_isv4, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 16}, {1, 4}}}}, - { g_os_zyx_is_osv16_isv16, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 16}, {1, 16}}}}, - { g_os_zyx_is_osv16_isv32, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 16}, {1, 32}}}}, - { g_os_zyx_is_osv32_isv4, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 32}, {1, 4}}}}, - { g_os_zyx_is_osv32_isv16, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 32}, {1, 16}}}}, - { g_os_zyx_is_osv32_isv32, { 1, 1, 3, 0, 1, "gozyxi", "oixyz???g", {{0, 32}, {1, 32}}}}, - { g_os_is_yx_osa4_isa8_osv8_isv4, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 32}, {1, 32}}}}, - { g_os_is_zyx_osa4_isa8_osv8_isv4, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{0, 32}, {1, 32}}}}, - { g_os_is_yx_osa4_isa8_osv8_isv2, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 32}, {1, 16}}}}, - { g_os_is_zyx_osa4_isa8_osv8_isv2, { 1, 1, 3, 0, 1, "goizyx", "oixyz???g", {{0, 32}, {1, 16}}}}, - { g_os_is_yx_osa2_isa8_osv16_isv4, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 32}, {1, 32}}}}, - { g_os_is_yx_osa2_isa8_osv16_isv2, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{0, 32}, {1, 16}}}}, - { gs_oi_yxs_gsv4_yxsv4, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{8, 4}}}}, - { gs_oi_yxs_gsv16_yxsv4, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{8, 16}}}}, - { gs_oi_yxs_gsv32_yxsv4, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{8, 32}}}}, - { g_os_is_yx_isv16_osv16, { 1, 1, 2, 0, 1, "goiyx", "oixy????g", {{1, 16}, {0, 16}}}}, - { gi_yxs_os_yxsv2_osv16, { 1, 1, 2, 0, 1, "giyxo", "oixy????g", {{0, 16}}}}, - { giy_xs_os_xsv2_osv8__ao32, { 1, 1, 2, 0, 1, "giyxo", "oixy????g", {{2, 2}, {0, 8}}}}, - { giy_xs_os_xsv2_osv16__ao32, { 1, 1, 2, 0, 1, "giyxo", "oixy????g", {{2, 2}, {0, 16}}}}, + { goiyx, { 1, 1, 2, 1, "goiyx", "oixy??g", {}}}, + { gioyx, { 1, 1, 2, 1, "gioyx", "oixy??g", {}}}, + { goizyx, { 1, 1, 3, 1, "goizyx", "oixyz?g", {}}}, + { giozyx, { 1, 1, 3, 1, "giozyx", "oixyz?g", {}}}, + { g_os_iyx_osv16, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 16}}}}, + { g_os_iyx_osv32, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 32}}}}, + { gs_oiyx_gsv16, { 1, 1, 2, 1, "goiyx", "oixy??g", {{6, 16}}}}, + { gs_oizyx_gsv16, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{6, 16}}}}, + { gs_oiyx_gsv32, { 1, 1, 2, 1, "goiyx", "oixy??g", {{6, 32}}}}, + { gyxio, { 1, 1, 2, 1, "gyxio", "oixy??g", {}}}, + { g_is_os_zyx_isv16_osv16, { 1, 1, 3, 1, "giozyx", "oixyz?g", {{1, 16}, {0, 16}}}}, + { g_is_os_yx_isv16_osv16, { 1, 1, 2, 1, "gioyx", "oixy??g", {{1, 16}, {0, 16}}}}, + { g_os_is_zyx_isv8_osv16_isv2, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{1, 8}, {0, 16}, {1, 2}}}}, + { g_os_is_yx_isv8_osv16_isv2, { 1, 1, 2, 1, "goiyx", "oixy??g", {{1, 8}, {0, 16}, {1, 2}}}}, + { g_os_is_zyx_isv16_osv16, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{0, 16}, {1, 16}}}}, + { g_os_is_yx_osv16_isv4, { 1, 1, 2, 1, "goixy", "oixy??g", {{0, 16}, {1, 4}}}}, + { g_os_is_zyx_osv16_isv16, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{0, 16}, {1, 16}}}}, + { g_os_zyx_is_osv16_isv4, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 16}, {1, 4}}}}, + { g_os_zyx_is_osv16_isv16, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 16}, {1, 16}}}}, + { g_os_zyx_is_osv16_isv32, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 16}, {1, 32}}}}, + { g_os_zyx_is_osv32_isv4, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 32}, {1, 4}}}}, + { g_os_zyx_is_osv32_isv16, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 32}, {1, 16}}}}, + { g_os_zyx_is_osv32_isv32, { 1, 1, 3, 1, "gozyxi", "oixyz?g", {{0, 32}, {1, 32}}}}, + { g_os_is_yx_osa4_isa8_osv8_isv4, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 32}, {1, 32}}}}, + { g_os_is_zyx_osa4_isa8_osv8_isv4, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{0, 32}, {1, 32}}}}, + { g_os_is_yx_osa4_isa8_osv8_isv2, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 32}, {1, 16}}}}, + { g_os_is_zyx_osa4_isa8_osv8_isv2, { 1, 1, 3, 1, "goizyx", "oixyz?g", {{0, 32}, {1, 16}}}}, + { g_os_is_yx_osa2_isa8_osv16_isv4, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 32}, {1, 32}}}}, + { g_os_is_yx_osa2_isa8_osv16_isv2, { 1, 1, 2, 1, "goiyx", "oixy??g", {{0, 32}, {1, 16}}}}, + { gs_oi_yxs_gsv4_yxsv4, { 1, 1, 2, 1, "goiyx", "oixy??g", {{6, 4}}}}, + { gs_oi_yxs_gsv16_yxsv4, { 1, 1, 2, 1, "goiyx", "oixy??g", {{6, 16}}}}, + { gs_oi_yxs_gsv32_yxsv4, { 1, 1, 2, 1, "goiyx", "oixy??g", {{6, 32}}}}, + { g_os_is_yx_isv16_osv16, { 1, 1, 2, 1, "goiyx", "oixy??g", {{1, 16}, {0, 16}}}}, + { gi_yxs_os_yxsv2_osv16, { 1, 1, 2, 1, "giyxo", "oixy??g", {{0, 16}}}}, + { giy_xs_os_xsv2_osv8__ao32, { 1, 1, 2, 1, "giyxo", "oixy??g", {{2, 2}, {0, 8}}}}, + { giy_xs_os_xsv2_osv16__ao32, { 1, 1, 2, 1, "giyxo", "oixy??g", {{2, 2}, {0, 16}}}}, }; if (traits.find(fmt) == traits.end()) { throw std::runtime_error("[clDNN] Format description is missing in fmt traits"); @@ -381,8 +374,6 @@ struct format { static size_t feature_num(type fmt) { return traits(fmt).feature_num; } /// @brief Returns number of spatial dimensions for a @p format. static size_t spatial_num(type fmt) { return traits(fmt).spatial_num; } - /// @brief Returns number of local dimensions for a @p format. - static size_t local_num(type fmt) { return traits(fmt).local_num; } /// @brief Returns number of group dimensions for a @p format. static size_t group_num(type fmt) { return traits(fmt).group_num; } /// @brief Returns an order of dimensions for a @ format. @@ -442,8 +433,6 @@ struct format { size_t feature_num() const { return traits(value).feature_num; } /// @brief Returns number of spatial dimensions. size_t spatial_num() const { return traits(value).spatial_num; } - /// @brief Returns number of local dimensions. - size_t local_num() const { return traits(value).local_num; } /// @brief Returns number of group dimensions. size_t group_num() const { return traits(value).group_num; } /// @brief Returns an order of dimensions in form of string. @@ -483,9 +472,8 @@ struct format { constexpr int32_t tensor_batch_dim_max = 1; constexpr int32_t tensor_feature_dim_max = 1; constexpr int32_t tensor_spatial_dim_max = 4; -constexpr int32_t tensor_local_dim_max = 2; constexpr int32_t tensor_group_dim_max = 1; -constexpr int32_t tensor_dim_max = tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max + tensor_local_dim_max + tensor_group_dim_max; +constexpr int32_t tensor_dim_max = tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max + tensor_group_dim_max; struct tensor; @@ -496,7 +484,6 @@ enum class dim_vec_kind { batch, feature, spatial, - local, group }; @@ -524,16 +511,10 @@ struct dim_vec_limits { static constexpr int32_t dim_offset = tensor_batch_dim_max + tensor_feature_dim_max; }; -template <> -struct dim_vec_limits { - static constexpr int32_t max_dimentionality = tensor_local_dim_max; - static constexpr int32_t dim_offset = tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max; -}; - template <> struct dim_vec_limits { static constexpr int32_t max_dimentionality = tensor_group_dim_max; - static constexpr int32_t dim_offset = tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max + tensor_local_dim_max; + static constexpr int32_t dim_offset = tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max; }; /// @brief Template class used in tensor constructor using dim_vec_kinds @@ -570,11 +551,6 @@ details::dim_vec_kind_init spatial(InitTys&&... return details::dim_vec_kind_init(std::forward(inits)...); } -template -details::dim_vec_kind_init local(InitTys&&... inits) { - return details::dim_vec_kind_init(std::forward(inits)...); -} - template details::dim_vec_kind_init group(InitTys&&... inits) { return details::dim_vec_kind_init(std::forward(inits)...); @@ -585,7 +561,6 @@ struct tensor { friend class details::dim_vec_kind_init; friend class details::dim_vec_kind_init; friend class details::dim_vec_kind_init; - friend class details::dim_vec_kind_init; friend class details::dim_vec_kind_init; typedef int32_t value_type; ///< Values type stored in tensor. @@ -594,7 +569,6 @@ struct tensor { mutable_array_ref batch; ///< Batch dimensions. mutable_array_ref feature; ///< Feature maps. mutable_array_ref spatial; ///< Spatial dimensions. - mutable_array_ref local; ///< Local dimensions. mutable_array_ref group; ///< Group dimensions. private: @@ -606,8 +580,7 @@ public: batch(_sizes, tensor_batch_dim_max), feature(_sizes + tensor_batch_dim_max, tensor_feature_dim_max), spatial(_sizes + tensor_batch_dim_max + tensor_feature_dim_max, tensor_spatial_dim_max), - local(_sizes + tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max, tensor_local_dim_max), - group(_sizes + tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max + tensor_local_dim_max, tensor_group_dim_max) { + group(_sizes + tensor_batch_dim_max + tensor_feature_dim_max + tensor_spatial_dim_max, tensor_group_dim_max) { std::fill_n(_sizes, tensor_dim_max, default_size); } diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/common/common_types.h b/inference-engine/thirdparty/clDNN/kernel_selector/common/common_types.h index 97b75822af5..c2e01942504 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/common/common_types.h +++ b/inference-engine/thirdparty/clDNN/kernel_selector/common/common_types.h @@ -21,6 +21,7 @@ enum class KernelType { NORMALIZE, POOLING, ROI_POOLING, + ROI_ALIGN, FULLY_CONNECTED, ACTIVATION, SOFT_MAX, @@ -71,7 +72,8 @@ enum class KernelType { EXTRACT_IMAGE_PATCHES, LOOP, NON_MAX_SUPPRESSION, - DETECTION_OUTPUT + DETECTION_OUTPUT, + EXPERIMENTAL_DETECTRON_ROI_FEATURE_EXTRACTOR }; //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.cpp index 4d158d4b3bf..64ee7122afe 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.cpp @@ -45,110 +45,110 @@ DataTensor::DataChannelArray DataTensor::dataChannelArray {{ }}; WeightsTensor::WeightsChannelArray WeightsTensor::weightsChannelArray {{ - // X, Y, Z, IFM, OFM, LX, LY, G - { WeightsLayout::oi, { -1, -1, -1, 0, 1, -1, -1, -1 } }, - { WeightsLayout::io, { -1, -1, -1, 1, 0, -1, -1, -1 } }, - { WeightsLayout::oiyx, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::ioyx, { 0, 1, -1, 3, 2, -1, -1, -1 } }, - { WeightsLayout::oyxi, { 1, 2, -1, 0, 3, -1, -1, -1 } }, - { WeightsLayout::iyxo, { 1, 2, -1, 3, 0, -1, -1, -1 } }, - { WeightsLayout::yxio, { 2, 3, -1, 1, 0, -1, -1, -1 } }, - { WeightsLayout::os_iyx_osv16, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_iyx_osv32, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_iyx_osv32__ai32, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_iyx_osv64, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_iyx_osv16_rotate_180, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::o_is_yx_isv16, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_yxi_osv16, { 1, 2, -1, 0, 3, -1, -1, -1 } }, - { WeightsLayout::os_i_osv8__ai8, { -1, -1, -1, 0, 1, -1, -1, -1 } }, - { WeightsLayout::os_i_osv16__ai8, { -1, -1, -1, 0, 1, -1, -1, -1 } }, - { WeightsLayout::os_i_osv16, { -1, -1, -1, 0, 1, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv16_isv16, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osv16_isv16, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::g_os_is_zyx_osv16_isv16, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::os_is_zyx_osv32_isv16, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osv64_isv16, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::i_yxs_os_yxsv2_osv16, { 1, 2, -1, 3, 0, -1, -1, -1 } }, - { WeightsLayout::iy_xs_os_xsv2_osv16__ao32, { 1, 2, -1, 3, 0, -1, -1, -1 } }, - { WeightsLayout::iy_xs_os_xsv2_osv8__ao32, { 1, 2, -1, 3, 0, -1, -1, -1 } }, - { WeightsLayout::image_2d_weights_c4_fyx_b, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::image_2d_weights_c1_b_fyx, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::winograd_2x3_s1_weights, { 3, 2, -1, 1, 0, -1, -1, -1 } }, - { WeightsLayout::winograd_2x3_s1_fused_weights, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::winograd_6x3_s1_fused_weights, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::image_2d_weights_winograd_6x3_s1_fbxyb, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::image_2d_weights_winograd_6x3_s1_xfbyb, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::dlstm_dir_io, { 1, 0, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_isa8_osv8_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_isa8_osv16_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_isa8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::g_os_is_yx_osa4_isa8_osv8_isv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_is_zyx_osa4_isa8_osv8_isv4, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv2, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv2, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv4, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::g_os_is_yx_osa4_isa8_osv8_isv2, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_is_zyx_osa4_isa8_osv8_isv2, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::os_is_yx_osa2_isa8_osv16_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::g_os_is_yx_osa2_isa8_osv16_isv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::os_is_yx_osa2_isa8_osv16_isv2, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::g_os_is_yx_osa2_isa8_osv16_isv2, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::os_is_zyx_isa8_osv8_isv4, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_isa8_osv16_isv4, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv4_swizzled_by_4, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::is_o_yx_isv32, { 1, 2, -1, 0, 3, -1, -1, -1 } }, - { WeightsLayout::is_o32_yx_isv32_swizzled_by_4, { 1, 2, -1, 0, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_y_x8_osv8_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_y_x8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv8_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv8_isv2, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv16_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv32_isv4_swizzled_by_2, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv32_isv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_osv32_isv4, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::oizyx, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::iozyx, { 0, 1, 2, 4, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_osv32_isv32p, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_isv16_osv16, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_isv16_osv16, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::is_os_zyx_isv16_osv16, { 0, 1, 2, 4, 3, -1, -1, -1 } }, - { WeightsLayout::is_os_yx_isv16_osv16, { 0, 1, -1, 3, 2, -1, -1, -1 } }, - { WeightsLayout::os_is_osv32_isv32_swizzled_by_4, { -1, -1, -1, 0, 1, -1, -1, -1 } }, - { WeightsLayout::os_is_zyx_isv8_osv16_isv2, { 0, 1, 2, 3, 4, -1, -1, -1 } }, - { WeightsLayout::os_is_yx_isv8_osv16_isv2, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::os_zyxi_osv16, { 1, 2, 3, 0, 4, -1, -1, -1 } }, - { WeightsLayout::os_i_yxs_osv4_yxsv4, { 0, 1, -1, 2, 3, -1, -1, -1 } }, - { WeightsLayout::goiyx, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gioyx, { 0, 1, -1, 3, 2, -1, -1, 4 } }, - { WeightsLayout::goizyx, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::giozyx, { 0, 1, 2, 4, 3, -1, -1, 5 } }, - { WeightsLayout::g_os_iyx_osv16, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_iyx_osv32, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gs_oiyx_gsv16, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gs_oizyx_gsv16, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::gs_oiyx_gsv32, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gyxio, { 2, 3, -1, 1, 0, -1, -1, 4 } }, - { WeightsLayout::gi_yxs_os_yxsv2_osv16, { 1, 2, -1, 3, 0, -1, -1, 4 } }, - { WeightsLayout::g_is_os_zyx_isv16_osv16, { 0, 1, 2, 4, 3, -1, -1, 5 } }, - { WeightsLayout::g_is_os_yx_isv16_osv16, { 0, 1, -1, 3, 2, -1, -1, 4 } }, - { WeightsLayout::g_os_is_zyx_isv8_osv16_isv2, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_is_yx_isv8_osv16_isv2, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_is_zyx_isv16_osv16, { 0, 1, 2, 3, 4, -1, -1, 5 } }, - { WeightsLayout::giy_xs_os_xsv2_osv16__ao32, { 1, 2, -1, 3, 0, -1, -1, 4 } }, - { WeightsLayout::giy_xs_os_xsv2_osv8__ao32, { 1, 2, -1, 3, 0, -1, -1, 4 } }, - { WeightsLayout::g_os_is_yx_isv16_osv16, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gs_oi_yxs_gsv4_yxsv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gs_oi_yxs_gsv16_yxsv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::gs_oi_yxs_gsv32_yxsv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_is_yx_osv16_isv4, { 0, 1, -1, 2, 3, -1, -1, 4 } }, - { WeightsLayout::g_os_zyx_is_osv16_isv4, { 1, 2, 3, 0, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_zyx_is_osv16_isv16, { 1, 2, 3, 0, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_zyx_is_osv16_isv32, { 1, 2, 3, 0, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_zyx_is_osv32_isv4, { 1, 2, 3, 0, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_zyx_is_osv32_isv16, { 1, 2, 3, 0, 4, -1, -1, 5 } }, - { WeightsLayout::g_os_zyx_is_osv32_isv32, { 1, 2, 3, 0, 4, -1, -1, 5 } }, + // X, Y, Z, IFM, OFM, G + { WeightsLayout::oi, { -1, -1, -1, 0, 1, -1 } }, + { WeightsLayout::io, { -1, -1, -1, 1, 0, -1 } }, + { WeightsLayout::oiyx, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::ioyx, { 0, 1, -1, 3, 2, -1 } }, + { WeightsLayout::oyxi, { 1, 2, -1, 0, 3, -1 } }, + { WeightsLayout::iyxo, { 1, 2, -1, 3, 0, -1 } }, + { WeightsLayout::yxio, { 2, 3, -1, 1, 0, -1 } }, + { WeightsLayout::os_iyx_osv16, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_iyx_osv32, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_iyx_osv32__ai32, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_iyx_osv64, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_iyx_osv16_rotate_180, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::o_is_yx_isv16, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_yxi_osv16, { 1, 2, -1, 0, 3, -1 } }, + { WeightsLayout::os_i_osv8__ai8, { -1, -1, -1, 0, 1, -1 } }, + { WeightsLayout::os_i_osv16__ai8, { -1, -1, -1, 0, 1, -1 } }, + { WeightsLayout::os_i_osv16, { -1, -1, -1, 0, 1, -1 } }, + { WeightsLayout::os_is_yx_osv16_isv16, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_zyx_osv16_isv16, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::g_os_is_zyx_osv16_isv16, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::os_is_zyx_osv32_isv16, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_zyx_osv64_isv16, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::i_yxs_os_yxsv2_osv16, { 1, 2, -1, 3, 0, -1 } }, + { WeightsLayout::iy_xs_os_xsv2_osv16__ao32, { 1, 2, -1, 3, 0, -1 } }, + { WeightsLayout::iy_xs_os_xsv2_osv8__ao32, { 1, 2, -1, 3, 0, -1 } }, + { WeightsLayout::image_2d_weights_c4_fyx_b, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::image_2d_weights_c1_b_fyx, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::winograd_2x3_s1_weights, { 3, 2, -1, 1, 0, -1 } }, + { WeightsLayout::winograd_2x3_s1_fused_weights, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::winograd_6x3_s1_fused_weights, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::image_2d_weights_winograd_6x3_s1_fbxyb, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::image_2d_weights_winograd_6x3_s1_xfbyb, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::dlstm_dir_io, { 1, 0, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_isa8_osv8_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_isa8_osv16_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_isa8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::g_os_is_yx_osa4_isa8_osv8_isv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_is_zyx_osa4_isa8_osv8_isv4, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv2, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv2, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv4, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::g_os_is_yx_osa4_isa8_osv8_isv2, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_is_zyx_osa4_isa8_osv8_isv2, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::os_is_yx_osa2_isa8_osv16_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::g_os_is_yx_osa2_isa8_osv16_isv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::os_is_yx_osa2_isa8_osv16_isv2, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::g_os_is_yx_osa2_isa8_osv16_isv2, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::os_is_zyx_isa8_osv8_isv4, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_zyx_isa8_osv16_isv4, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_yx_osa4_isa8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_zyx_osa4_isa8_osv8_isv4_swizzled_by_4, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::is_o_yx_isv32, { 1, 2, -1, 0, 3, -1 } }, + { WeightsLayout::is_o32_yx_isv32_swizzled_by_4, { 1, 2, -1, 0, 3, -1 } }, + { WeightsLayout::os_is_y_x8_osv8_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_y_x8_osv8_isv4_swizzled_by_4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osv8_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osv8_isv2, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osv16_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osv32_isv4_swizzled_by_2, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_yx_osv32_isv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_zyx_osv32_isv4, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::oizyx, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::iozyx, { 0, 1, 2, 4, 3, -1 } }, + { WeightsLayout::os_is_yx_osv32_isv32p, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_is_zyx_isv16_osv16, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_yx_isv16_osv16, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::is_os_zyx_isv16_osv16, { 0, 1, 2, 4, 3, -1 } }, + { WeightsLayout::is_os_yx_isv16_osv16, { 0, 1, -1, 3, 2, -1 } }, + { WeightsLayout::os_is_osv32_isv32_swizzled_by_4, { -1, -1, -1, 0, 1, -1 } }, + { WeightsLayout::os_is_zyx_isv8_osv16_isv2, { 0, 1, 2, 3, 4, -1 } }, + { WeightsLayout::os_is_yx_isv8_osv16_isv2, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::os_zyxi_osv16, { 1, 2, 3, 0, 4, -1 } }, + { WeightsLayout::os_i_yxs_osv4_yxsv4, { 0, 1, -1, 2, 3, -1 } }, + { WeightsLayout::goiyx, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gioyx, { 0, 1, -1, 3, 2, 4 } }, + { WeightsLayout::goizyx, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::giozyx, { 0, 1, 2, 4, 3, 5 } }, + { WeightsLayout::g_os_iyx_osv16, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_iyx_osv32, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gs_oiyx_gsv16, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gs_oizyx_gsv16, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::gs_oiyx_gsv32, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gyxio, { 2, 3, -1, 1, 0, 4 } }, + { WeightsLayout::gi_yxs_os_yxsv2_osv16, { 1, 2, -1, 3, 0, 4 } }, + { WeightsLayout::g_is_os_zyx_isv16_osv16, { 0, 1, 2, 4, 3, 5 } }, + { WeightsLayout::g_is_os_yx_isv16_osv16, { 0, 1, -1, 3, 2, 4 } }, + { WeightsLayout::g_os_is_zyx_isv8_osv16_isv2, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::g_os_is_yx_isv8_osv16_isv2, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_is_zyx_isv16_osv16, { 0, 1, 2, 3, 4, 5 } }, + { WeightsLayout::giy_xs_os_xsv2_osv16__ao32, { 1, 2, -1, 3, 0, 4 } }, + { WeightsLayout::giy_xs_os_xsv2_osv8__ao32, { 1, 2, -1, 3, 0, 4 } }, + { WeightsLayout::g_os_is_yx_isv16_osv16, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gs_oi_yxs_gsv4_yxsv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gs_oi_yxs_gsv16_yxsv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::gs_oi_yxs_gsv32_yxsv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_is_yx_osv16_isv4, { 0, 1, -1, 2, 3, 4 } }, + { WeightsLayout::g_os_zyx_is_osv16_isv4, { 1, 2, 3, 0, 4, 5 } }, + { WeightsLayout::g_os_zyx_is_osv16_isv16, { 1, 2, 3, 0, 4, 5 } }, + { WeightsLayout::g_os_zyx_is_osv16_isv32, { 1, 2, 3, 0, 4, 5 } }, + { WeightsLayout::g_os_zyx_is_osv32_isv4, { 1, 2, 3, 0, 4, 5 } }, + { WeightsLayout::g_os_zyx_is_osv32_isv16, { 1, 2, 3, 0, 4, 5 } }, + { WeightsLayout::g_os_zyx_is_osv32_isv32, { 1, 2, 3, 0, 4, 5 } }, }}; NDims DataTensor::GetSimpleDims(const std::vector& d, DataLayout l) { @@ -859,13 +859,6 @@ WeightsTensor WeightsTensor::TransformIgnorePadding(WeightsLayout l, WeightsType vec[Channelndex(l, WeightsChannelName::IFM)] = IFM().v; vec[Channelndex(l, WeightsChannelName::OFM)] = OFM().v; vec[Channelndex(l, WeightsChannelName::Z)] = Z().v; - } else if (src_channels == 6 && dst_channels == 6) { - vec[Channelndex(l, WeightsChannelName::X)] = X().v; - vec[Channelndex(l, WeightsChannelName::Y)] = Y().v; - vec[Channelndex(l, WeightsChannelName::IFM)] = IFM().v; - vec[Channelndex(l, WeightsChannelName::OFM)] = OFM().v; - vec[Channelndex(l, WeightsChannelName::LX)] = LX().v; - vec[Channelndex(l, WeightsChannelName::LY)] = LY().v; } else if (src_channels == 4 && dst_channels == 5) { vec[Channelndex(l, WeightsChannelName::X)] = X().v; vec[Channelndex(l, WeightsChannelName::Y)] = Y().v; diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.h b/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.h index 725da375516..6fa3d013f87 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.h +++ b/inference-engine/thirdparty/clDNN/kernel_selector/common/tensor_type.h @@ -203,7 +203,7 @@ using NDims = std::vector; //////////////////////////////////////////////////////////////////////////////////////////////////////////////////////// enum class DataChannelName { X = 0, Y = 1, Z = 2, W = 3, FEATURE = 4, BATCH = 5, COUNT = 6 }; -enum class WeightsChannelName { X = 0, Y = 1, Z = 2, IFM = 3, OFM = 4, LX = 5, LY = 6, G = 7, COUNT = 8 }; +enum class WeightsChannelName { X = 0, Y = 1, Z = 2, IFM = 3, OFM = 4, G = 5, COUNT = 6 }; inline bool SimpleLayout(WeightsLayout l) { switch (l) { @@ -568,8 +568,6 @@ struct WeightsTensor : TensorBaseT { Dim Z() const { return Extract(layout, WeightsChannelName::Z, dims); } Dim IFM() const { return Extract(layout, WeightsChannelName::IFM, dims); } Dim OFM() const { return Extract(layout, WeightsChannelName::OFM, dims); } - Dim LX() const { return Extract(layout, WeightsChannelName::LX, dims); } - Dim LY() const { return Extract(layout, WeightsChannelName::LY, dims); } Dim G() const { return Extract(layout, WeightsChannelName::G, dims); } static inline Dim Extract(WeightsLayout l, WeightsChannelName channel, const NDims& d) { diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_base.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_base.cpp index 2a760a8b8a3..e810a835807 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_base.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_base.cpp @@ -69,10 +69,6 @@ JitConstants ConvolutionKernelBase::GetJitConstants(const convolution_params& pa mem_consts.AddConstants({MakeJitConstant("SYMMETRIC_QUANTIZATION", 1)}); } - if (params.local_convolution) { - mem_consts.AddConstants({MakeJitConstant("LOCAL_CONVOLUTION", params.local_convolution)}); - } - if (params.deformable_mode) { mem_consts.AddConstants({MakeJitConstant("DEFORMABLE_GROUPS", params.deformable_groups)}); mem_consts.AddConstants({MakeJitConstant("DEFORMABLE_MODE", params.deformable_mode)}); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.cpp index bef3794a657..6fd108fcb79 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.cpp @@ -80,7 +80,7 @@ static std::pair get_bfyx_req_input_block_dims(size_t output_blo return std::make_pair(input_block_array_size, input_block_read_width); } -static void shrink_blocks_to_output_size(size_t output_x, size_t output_y, size_t& block_x, size_t& block_y) { +static void shrink_blocks_to_output_size(size_t output_x, size_t output_y, size_t& block_x, size_t& block_y, size_t sub_group_size) { // how many elements we will compute in each dimension size_t computed_x = Align(output_x, block_x); size_t computed_y = Align(output_y, block_y); @@ -94,8 +94,10 @@ static void shrink_blocks_to_output_size(size_t output_x, size_t output_y, size_ block_x -= unused_x / simds_x; block_y -= unused_y / simds_y; - block_x = Align(block_x, 2); - block_y = Align(block_y, 2); + if (simds_x * simds_y >= sub_group_size) { + block_x = Align(block_x, 2); + block_y = Align(block_y, 2); + } } ConvolutionKernel_bfyx_os_iyx_osv16::AutoTuneOption ConvolutionKernel_bfyx_os_iyx_osv16::GetAutoTuneOptions( @@ -145,7 +147,7 @@ ConvolutionKernel_bfyx_os_iyx_osv16::AutoTuneOption ConvolutionKernel_bfyx_os_iy // if this is not 1x1 batch1 case then shrink filters, other way we're memory bound and it's best to use 16x1 block // sizes if (cp.filterSize.x != 1 || cp.filterSize.y != 1 || cp.output.Batch().v != 1) { - shrink_blocks_to_output_size(cp.output.X().v, cp.output.Y().v, option.blockWidth, option.blockHeight); + shrink_blocks_to_output_size(cp.output.X().v, cp.output.Y().v, option.blockWidth, option.blockHeight, sub_group_size); } return option; } diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.h index aa0f0610626..a62eaeb31ce 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.h +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_bfyx_os_iyx_osv16.h @@ -40,7 +40,12 @@ protected: // Smaller # EU tends to be computation bounds. // In such case, using larger worksize will result in larger computational inefficiency // w.r.t the unalined output feature - return (params.output.Feature().v > 8 || !IsSIMDSizeSupported(params.engineInfo, 8)) ? 16 : 8; + if (params.output.Feature().v > 8 || !IsSIMDSizeSupported(params.engineInfo, 8) + || (params.output.GetDType() == Datatype::F16) && params.output.Feature().v == 8) { + return 16; + } else { + return 8; + } } else { return 16; } diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_ref.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_ref.cpp index 951a67d58c3..ccd3e55611f 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_ref.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_kernel_ref.cpp @@ -41,7 +41,6 @@ ParamsKey ConvolutionKernel_Ref::GetSupportedKey() const { k.EnableSplitSupport(); k.EnableDepthwiseSeparableOpt(); k.DisableTuning(); - k.EnableLocalConvolution(); k.EnableGroupedConvolution(); k.EnableQuantization(QuantizationType::SYMMETRIC); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.cpp index 7989871f0a8..a3d2ce021ca 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.cpp @@ -55,10 +55,6 @@ ParamsKey convolution_params::GetParamsKey() const { k.EnableDepthwiseSeparableOpt(); } - if (local_convolution) { - k.EnableLocalConvolution(); - } - if (groups > 1 && !depthwise_separable_opt) { k.EnableGroupedConvolution(); } diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.h index a12a3654377..f53854587b8 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.h +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/convolution_params.h @@ -25,7 +25,6 @@ struct convolution_params : public weight_bias_zero_point_params { bool depthwise_separable_opt = false; bool transposed = false; QuantizationType quantization = QuantizationType::NONE; - bool local_convolution = false; bool deformable_mode = false; uint32_t groups = 1; uSize kernelSize; diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_conv.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_conv.cpp index 69d96d2e6ec..0ab23004ef0 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_conv.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_conv.cpp @@ -26,7 +26,6 @@ ParamsKey DeformableConvolutionKernel_bfyx_conv::GetSupportedKey() const { k.EnableSplitSupport(); k.EnableDepthwiseSeparableOpt(); k.DisableTuning(); - k.EnableLocalConvolution(); k.EnableGroupedConvolution(); k.EnableDeformableMode(); k.EnableDeformableMask(); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_interp.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_interp.cpp index 54d1c489f5d..877ebe41c2b 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_interp.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_interp.cpp @@ -26,7 +26,6 @@ ParamsKey DeformableConvolutionKernel_bfyx_interp::GetSupportedKey() const { k.EnableSplitSupport(); k.EnableDepthwiseSeparableOpt(); k.DisableTuning(); - k.EnableLocalConvolution(); k.EnableGroupedConvolution(); k.EnableDeformableMode(); k.EnableDeformableMask(); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_ref.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_ref.cpp index 89140c4694f..0a7fde9b0bb 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_ref.cpp +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/convolution/deformable_convolution_kernel_bfyx_ref.cpp @@ -30,7 +30,6 @@ ParamsKey DeformableConvolutionKernel_bfyx_Ref::GetSupportedKey() const { k.EnableSplitSupport(); k.EnableDepthwiseSeparableOpt(); k.DisableTuning(); - k.EnableLocalConvolution(); k.EnableGroupedConvolution(); k.EnableDeformableMode(); k.EnableDeformableMask(); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.cpp new file mode 100644 index 00000000000..ad582e9b67a --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.cpp @@ -0,0 +1,135 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "experimental_detectron_roi_feature_extractor_kernel_ref.h" +#include "kernel_selector_utils.h" +#include +#include + +namespace kernel_selector { +namespace { + ExperimentalDetectronROIFeatureExtractorRef::DispatchData SetDefault(const experimental_detectron_roi_feature_extractor_params& params) { + ExperimentalDetectronROIFeatureExtractorRef::DispatchData dispatch_data; + auto in_layout = params.inputs[0].GetLayout(); + auto out_layout = params.output.GetLayout(); + + std::vector> dims_by_gws = {{ Tensor::DataChannelName::X, Tensor::DataChannelName::Y }, + { Tensor::DataChannelName::FEATURE }, + { Tensor::DataChannelName::BATCH }}; + + dispatch_data.gws = {params.output.X().v * params.output.Y().v, params.output.Feature().v, params.output.Batch().v}; + + dispatch_data.lws = GetOptimalLocalWorkGroupSizes(dispatch_data.gws, params.engineInfo, in_layout, out_layout, dims_by_gws); + + return dispatch_data; + } + + const std::string common_level_name = "level_"; + + std::string GetInputLevelParams(size_t levels_num) { + std::string result = "const __global INPUT1_TYPE* " + common_level_name + "1"; + std::string idx = ""; + for (size_t i = 1; i < levels_num; i++) { + idx = std::to_string(i + 1); + result += ", const __global INPUT" + idx + "_TYPE* " + common_level_name + idx; + } + return result; + } + + const std::string level_ptrs = "level_ptrs"; + + std::string GetDefinedLevelPtrs(size_t levels_num) { + std::string result = "const __global INPUT1_TYPE* " + level_ptrs + "[" + std::to_string(levels_num) + "] = {" + common_level_name + "1"; + for (size_t i = 1; i < levels_num; i++) { + result += ", " + common_level_name + std::to_string(i + 1); + } + result += "}"; + return result; + } + + const std::string spatial_scales = "spatial_scales"; + + std::string GetDefinedSpatialScales(const std::vector& scales, size_t levels_num) { + std::string result = "__constant float " + spatial_scales + "[" + std::to_string(levels_num) + "] = {" + std::to_string(1.0f / scales[0]); + for (size_t i = 1; i < levels_num; i++) { + result += ", " + std::to_string(1.0f / scales[i]); + } + result += "}"; + return result; + } + + const std::string level_sizes = "level_sizes"; + + std::string GetDefinedLevelSizes(size_t levels_num) { + std::string result = "__constant int " + level_sizes + "[" + std::to_string(3 * levels_num) +"] = {INPUT1_SIZE_Y, INPUT1_SIZE_X, INPUT1_OFFSET"; + std::string idx = ""; + for (size_t i = 1; i < levels_num; i++) { + idx = std::to_string(i + 1); + result += " ,INPUT" + idx + "_SIZE_Y, INPUT" + idx + "_SIZE_X, INPUT" + idx + "_OFFSET"; + } + result += "}"; + return result; + } +} // namespace + +JitConstants ExperimentalDetectronROIFeatureExtractorRef::GetJitConstants(const experimental_detectron_roi_feature_extractor_params& params) const { + JitConstants jit = MakeBaseParamsJitConstants(params); + const size_t levels_num = params.number_of_inputs - 1; + + jit.AddConstants({MakeJitConstant("POOLED_HEIGHT", params.pooled_height), + MakeJitConstant("POOLED_WIDTH", params.pooled_width), + MakeJitConstant("SAMPLING_RATIO", params.sampling_ratio), + MakeJitConstant("IS_ALIGNED", params.aligned), + MakeJitConstant("NUM_PYRAMID_LEVELS", levels_num), + MakeJitConstant("INPUT_LEVEL_PARAMS", GetInputLevelParams(levels_num)), + MakeJitConstant("LEVEL_PTRS", level_ptrs), + MakeJitConstant("DEFINE_LEVEL_PTRS", GetDefinedLevelPtrs(levels_num)), + MakeJitConstant("SPATIAL_SCALES", spatial_scales), + MakeJitConstant("DEFINE_SPATIAL_SCALES", GetDefinedSpatialScales(params.pyramid_scales, levels_num)), + MakeJitConstant("LEVEL_SIZES", level_sizes), + MakeJitConstant("DEFINE_LEVEL_SIZES", GetDefinedLevelSizes(levels_num))}); + + return jit; +} + +KernelsData ExperimentalDetectronROIFeatureExtractorRef::GetKernelsData(const Params& params, const optional_params& options) const { + assert(params.GetType() == KernelType::EXPERIMENTAL_DETECTRON_ROI_FEATURE_EXTRACTOR); + const experimental_detectron_roi_feature_extractor_params& org_params = static_cast(params); + + if (!org_params.activations.empty()) { + return {}; + } + + DispatchData dispatch_data = SetDefault(org_params); + KernelData kd = KernelData::Default(params); + + auto cldnn_jit = GetJitConstants(org_params); + auto entry_point = GetEntryPoint(kernelName, org_params.layerID, params, options); + auto jit = CreateJit(kernelName, cldnn_jit, entry_point); + + auto& kernel = kd.kernels[0]; + FillCLKernelData(kernel, dispatch_data, params.engineInfo, kernelName, jit, entry_point, "", false, false, org_params.number_of_inputs); + return {kd}; +} + +KernelsPriority ExperimentalDetectronROIFeatureExtractorRef::GetKernelsPriority(const Params& /*params*/, const optional_params& /*options*/) const { + return FORCE_PRIORITY_9; +} + +ParamsKey ExperimentalDetectronROIFeatureExtractorRef::GetSupportedKey() const { + ParamsKey key; + key.EnableInputDataType(Datatype::F16); + key.EnableInputDataType(Datatype::F32); + key.EnableOutputDataType(Datatype::F16); + key.EnableOutputDataType(Datatype::F32); + key.EnableInputLayout(DataLayout::bfyx); + key.EnableOutputLayout(DataLayout::bfyx); + key.EnableTensorOffset(); + key.EnableTensorPitches(); + key.EnableBatching(); + key.EnableDifferentTypes(); + return key; +} + +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.h new file mode 100644 index 00000000000..1b93cdb0036 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.h @@ -0,0 +1,39 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "kernel_base_opencl.h" + +namespace kernel_selector { +struct experimental_detectron_roi_feature_extractor_params : public base_params { + experimental_detectron_roi_feature_extractor_params() : base_params(KernelType::EXPERIMENTAL_DETECTRON_ROI_FEATURE_EXTRACTOR) {} + + int output_dim = 0; + int pooled_height = 0; + int pooled_width = 0; + std::vector pyramid_scales; + int sampling_ratio = 0; + bool aligned = false; + std::size_t number_of_inputs = 0; +}; + +struct experimental_detectron_roi_feature_extractor_optional_params : public optional_params { + experimental_detectron_roi_feature_extractor_optional_params() : optional_params(KernelType::EXPERIMENTAL_DETECTRON_ROI_FEATURE_EXTRACTOR) {} +}; + +class ExperimentalDetectronROIFeatureExtractorRef : public KernelBaseOpenCL { +public: + ExperimentalDetectronROIFeatureExtractorRef() : KernelBaseOpenCL("experimental_detectron_roi_feature_extractor_ref") {} + ~ExperimentalDetectronROIFeatureExtractorRef() = default; + + using DispatchData = CommonDispatchData; + + KernelsData GetKernelsData(const Params& params, const optional_params& options) const override; + KernelsPriority GetKernelsPriority(const Params& params, const optional_params& options) const override; + ParamsKey GetSupportedKey() const override; +protected: + virtual JitConstants GetJitConstants(const experimental_detectron_roi_feature_extractor_params& params) const; +}; +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.cpp new file mode 100644 index 00000000000..aa3e49be1c6 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.cpp @@ -0,0 +1,23 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "experimental_detectron_roi_feature_extractor_kernel_selector.h" +#include "experimental_detectron_roi_feature_extractor_kernel_ref.h" + +namespace kernel_selector { + +experimental_detectron_roi_feature_extractor_kernel_selector& experimental_detectron_roi_feature_extractor_kernel_selector::Instance() { + static experimental_detectron_roi_feature_extractor_kernel_selector instance_; + return instance_; + } + +experimental_detectron_roi_feature_extractor_kernel_selector::experimental_detectron_roi_feature_extractor_kernel_selector() { + implementations.push_back(std::make_shared()); +} + +KernelsData experimental_detectron_roi_feature_extractor_kernel_selector::GetBestKernels(const Params& params, const optional_params& options) const { + return GetNaiveBestKernel(params, options, KernelType::EXPERIMENTAL_DETECTRON_ROI_FEATURE_EXTRACTOR); +} + +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.h new file mode 100644 index 00000000000..defb4d08e2c --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.h @@ -0,0 +1,19 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "kernel_selector.h" + +namespace kernel_selector { +class experimental_detectron_roi_feature_extractor_kernel_selector : public kernel_selector_base { +public: + static experimental_detectron_roi_feature_extractor_kernel_selector& Instance(); + + experimental_detectron_roi_feature_extractor_kernel_selector(); + ~experimental_detectron_roi_feature_extractor_kernel_selector() = default; + + KernelsData GetBestKernels(const Params& params, const optional_params& options) const override; +}; +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.cpp new file mode 100644 index 00000000000..6ec2f39feb9 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.cpp @@ -0,0 +1,90 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#include "roi_align_kernel_ref.h" +#include + +namespace kernel_selector { + +ParamsKey ROIAlignKernelRef::GetSupportedKey() const { + ParamsKey k; + k.EnableInputDataType(Datatype::F16); + k.EnableInputDataType(Datatype::F32); + k.EnableInputDataType(Datatype::INT32); + k.EnableOutputDataType(Datatype::F16); + k.EnableOutputDataType(Datatype::F32); + k.EnableInputLayout(DataLayout::bfyx); + k.EnableOutputLayout(DataLayout::bfyx); + k.EnableTensorOffset(); + k.EnableTensorPitches(); + k.EnableBatching(); + k.EnableDifferentTypes(); + k.EnablePoolType(PoolType::MAX); + k.EnablePoolType(PoolType::AVG); + return k; +} + +namespace { + +ROIAlignKernelRef::DispatchData SetDefault(const roi_align_params& params) { + ROIAlignKernelRef::DispatchData dispatchData; + // Determine global work sizes. + dispatchData.gws[0] = params.output.LogicalSize(); + dispatchData.gws[1] = 1; + dispatchData.gws[2] = 1; + dispatchData.lws = GetOptimalLocalWorkGroupSizes(dispatchData.gws, params.engineInfo); + + return dispatchData; +} + +} // anonymous namespace + +KernelsData ROIAlignKernelRef::GetKernelsData(const Params ¶ms, const optional_params &options) const { + if (!Validate(params, options)) { + return {}; + } + KernelData kernel_data = KernelData::Default(params); + roi_align_params &new_params = dynamic_cast(*kernel_data.params.get()); + auto dispatch_data = SetDefault(new_params); + auto entry_point = GetEntryPoint(kernelName, new_params.layerID, params, options); + auto roi_align_specific_jit = GetJitConstants(new_params); + auto jit = CreateJit(kernelName, roi_align_specific_jit, entry_point); + FillCLKernelData(kernel_data.kernels[0], dispatch_data, params.engineInfo, + kernelName, jit, entry_point); + kernel_data.kernels[0].params.arguments.push_back({ArgumentDescriptor::Types::INPUT, 1}); + kernel_data.kernels[0].params.arguments.push_back({ArgumentDescriptor::Types::INPUT, 2}); + + return {kernel_data}; +} + +float ROIAlignKernelRef::GetKernelsPriority(const Params ¶ms, const optional_params &options) const { + return FORCE_PRIORITY_1; +} + +bool ROIAlignKernelRef::Validate(const Params& p, const optional_params& o) const { + if (p.GetType() != KernelType::ROI_ALIGN || o.GetType() != KernelType::ROI_ALIGN) { + return false; + } + + const roi_align_params ¶ms = static_cast(p); + if (params.inputs.size() != 3) + return false; + + if (params.output.Dimentions() > 4 || params.inputs[0].Dimentions() > 4 || params.inputs[1].Dimentions() > 2) + return false; + + return true; +} + +JitConstants ROIAlignKernelRef::GetJitConstants(const roi_align_params ¶ms) const { + JitConstants jit = MakeBaseParamsJitConstants(params); + jit.AddConstant(MakeJitConstant("SPATIAL_SCALE", params.spatial_scale)); + jit.AddConstant(MakeJitConstant("SAMPLING_RATIO", params.sampling_ratio)); + if (params.mode == PoolType::MAX) + jit.AddConstant(MakeJitConstant("MAX_POOL", true)); + else if (params.mode == PoolType::AVG) + jit.AddConstant(MakeJitConstant("AVG_POOL", true)); + return jit; +} + +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.h new file mode 100644 index 00000000000..10dc02a8869 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_ref.h @@ -0,0 +1,44 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#pragma once + +#include + +namespace kernel_selector { + +struct roi_align_params : public base_params { + roi_align_params() : base_params{KernelType::ROI_ALIGN} {} + + int sampling_ratio = 0; + float spatial_scale = 1.f; + PoolType mode = PoolType::MAX; + + ParamsKey GetParamsKey() const override { + auto k = base_params::GetParamsKey(); + k.EnablePoolType(mode); + return k; + } +}; + +struct roi_align_optional_params : optional_params { + roi_align_optional_params() : optional_params{KernelType::ROI_ALIGN} {} +}; + +class ROIAlignKernelRef : public KernelBaseOpenCL { +public: + using KernelBaseOpenCL::KernelBaseOpenCL; + + using DispatchData = CommonDispatchData; + + ROIAlignKernelRef() : KernelBaseOpenCL{"roi_align_ref"} {} + KernelsData GetKernelsData(const Params& params, const optional_params& options) const override; + KernelsPriority GetKernelsPriority(const Params& params, const optional_params& options) const override; + ParamsKey GetSupportedKey() const override; + bool Validate(const Params&, const optional_params&) const override; + +protected: + JitConstants GetJitConstants(const roi_align_params& params) const; +}; + +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.cpp b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.cpp new file mode 100644 index 00000000000..ab7a6f13643 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.cpp @@ -0,0 +1,18 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#include "roi_align_kernel_selector.h" +#include "roi_align_kernel_ref.h" + +namespace kernel_selector { + +roi_align_kernel_selector::roi_align_kernel_selector() { + Attach(); +} + +KernelsData roi_align_kernel_selector::GetBestKernels(const Params ¶ms, + const optional_params &options) const { + return GetNaiveBestKernel(params, options, KernelType::ROI_ALIGN); +} + +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.h new file mode 100644 index 00000000000..5a6f69f48bc --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/actual_kernels/roi_align/roi_align_kernel_selector.h @@ -0,0 +1,20 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#pragma once + +#include "kernel_selector.h" + +namespace kernel_selector { +class roi_align_kernel_selector : public kernel_selector_base { +public: + static roi_align_kernel_selector& Instance() { + static roi_align_kernel_selector instance_; + return instance_; + } + + roi_align_kernel_selector(); + + KernelsData GetBestKernels(const Params& params, const optional_params& options) const override; +}; +} // namespace kernel_selector diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/convolution_gpu_ref.cl b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/convolution_gpu_ref.cl index 4b2918e22b7..facf8fa774e 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/convolution_gpu_ref.cl +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/convolution_gpu_ref.cl @@ -93,10 +93,6 @@ KERNEL(kernel_name)( uint filter_idx = GET_FILTER_INDEX_5D(FILTER, g, of, k, l, j, i); #endif -#ifdef LOCAL_CONVOLUTION - filter_idx += FILTER_SIZE_X * FILTER_SIZE_Y * FILTER_SIZE_Z - * (x + OUTPUT_SIZE_X * y + OUTPUT_SIZE_X * OUTPUT_SIZE_Y * z); -#endif ACCUMULATOR_TYPE in = TO_ACCUMULATOR_TYPE(conv_input[input_idx]); #if ASYMMETRIC_DATA_QUANTIZATION in -= TO_ACCUMULATOR_TYPE(activations_zp[g * FILTER_IFM_NUM + k]); diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/deformable_convolution_gpu_bfyx_ref.cl b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/deformable_convolution_gpu_bfyx_ref.cl index 9bbdd1ffb73..128f38518c2 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/deformable_convolution_gpu_bfyx_ref.cl +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/deformable_convolution_gpu_bfyx_ref.cl @@ -81,12 +81,8 @@ KERNEL(deformable_convolution_gpu_bfyx_ref)( uint ifm = ifm_offset + c; uint filter_idx = GET_FILTER_INDEX(FILTER, conv_group, f, ifm, j, i); -#ifdef LOCAL_CONVOLUTION - filter_idx += FILTER_SIZE_X * FILTER_SIZE_Y * (x + OUTPUT_SIZE_X * y); -#endif - int top_y_index = (int)(floor(transformed_y)); - int left_x_index = (int)(floor(transformed_x)); + int left_x_index = (int)(floor(transformed_x)); #if BILINEAR_INTERPOLATION_PAD int bottom_y_index = top_y_index + 1; int right_x_index = left_x_index + 1; diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/experimental_detectron_roi_feature_extractor_ref.cl b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/experimental_detectron_roi_feature_extractor_ref.cl new file mode 100644 index 00000000000..9433bf3e96d --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/experimental_detectron_roi_feature_extractor_ref.cl @@ -0,0 +1,130 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "include/batch_headers/common.cl" +#include "include/batch_headers/data_types.cl" + +DEFINE_SPATIAL_SCALES; +DEFINE_LEVEL_SIZES; + +inline int FUNC(get_pyramid_level_for_roi)(const __global INPUT0_TYPE* current_roi) { + const INPUT0_TYPE canonical_scale = 224.0; + const int canonical_level = 2; + + int result = NUM_PYRAMID_LEVELS; + + const INPUT0_TYPE x0 = current_roi[0]; + const INPUT0_TYPE y0 = current_roi[1]; + const INPUT0_TYPE x1 = current_roi[2]; + const INPUT0_TYPE y1 = current_roi[3]; + + const INPUT0_TYPE area = (x1 - x0) * (y1 - y0); + if (area > 0) { + result = (int)round(canonical_level + log2(sqrt(area) / canonical_scale)); + result = max(0, min(result, NUM_PYRAMID_LEVELS - 1)); + } + return result; +} + +KERNEL(experimental_detectron_roi_feature_extractor_ref)(const __global INPUT0_TYPE* src_rois, + INPUT_LEVEL_PARAMS, + __global OUTPUT_TYPE* dst_data) +{ + const uint oxy = get_global_id(0); + + const uint x = oxy % POOLED_WIDTH; + const uint y = oxy / POOLED_WIDTH; + const uint c = get_global_id(1); + const uint r = get_global_id(2); + + const __global INPUT0_TYPE* current_roi_ptr = &src_rois[r * INPUT0_BATCH_PITCH]; + + const int level = FUNC_CALL(get_pyramid_level_for_roi)(current_roi_ptr); + + DEFINE_LEVEL_PTRS; + const __global INPUT1_TYPE* current_level_ptr = LEVEL_PTRS[level]; + + INPUT0_TYPE offset = IS_ALIGNED ? TO_INPUT0_TYPE(0.5f) : TO_INPUT0_TYPE(0.0); + + INPUT0_TYPE spatial_scale = SPATIAL_SCALES[level]; + INPUT0_TYPE roi_start_w = current_roi_ptr[0] * spatial_scale - offset; + INPUT0_TYPE roi_start_h = current_roi_ptr[1] * spatial_scale - offset; + INPUT0_TYPE roi_end_w = current_roi_ptr[2] * spatial_scale - offset; + INPUT0_TYPE roi_end_h = current_roi_ptr[3] * spatial_scale - offset; + + INPUT0_TYPE roi_width = max(roi_end_w - roi_start_w, TO_INPUT0_TYPE(1.)); + INPUT0_TYPE roi_height = max(roi_end_h - roi_start_h, TO_INPUT0_TYPE(1.)); + + INPUT0_TYPE bin_width = roi_width / TO_INPUT0_TYPE(POOLED_WIDTH); + INPUT0_TYPE bin_height = roi_height / TO_INPUT0_TYPE(POOLED_HEIGHT); + + const uint roi_bin_grid_w = (SAMPLING_RATIO > 0) ? SAMPLING_RATIO : (uint)ceil(roi_width / POOLED_WIDTH); + const uint roi_bin_grid_h = (SAMPLING_RATIO > 0) ? SAMPLING_RATIO : (uint)ceil(roi_height / POOLED_HEIGHT); + + const uint level_h = LEVEL_SIZES[3 * level]; + const uint level_w = LEVEL_SIZES[3 * level + 1]; + const uint level_offset = LEVEL_SIZES[3 * level + 2]; + + INPUT0_TYPE output_val = 0.0; + const __global INPUT1_TYPE* data = current_level_ptr + level_offset + c * level_h * level_w; + + INPUT0_TYPE current_bin_start_h = roi_start_h + y * bin_height; + INPUT0_TYPE current_bin_start_w = roi_start_w + x * bin_width; + for (int iy = 0; iy < roi_bin_grid_h; iy++) { + INPUT0_TYPE yy = current_bin_start_h + TO_INPUT0_TYPE(iy + 0.5f) * bin_height / TO_INPUT0_TYPE(roi_bin_grid_h); + if (yy < -1.0 || yy > level_h) { + continue; + } + if (yy <= 0) { + yy = 0.0f; + } + int y_low = (int)floor(yy); + int y_high = 0; + + if (y_low >= level_h - 1) { + y_high = y_low = level_h - 1; + yy = TO_INPUT0_TYPE(y_low); + } else { + y_high = y_low + 1; + } + + INPUT0_TYPE ly = yy - y_low; + INPUT0_TYPE hy = TO_INPUT0_TYPE(1.0f) - ly; + + for (int ix = 0; ix < roi_bin_grid_w; ix++) { + INPUT0_TYPE xx = current_bin_start_w + TO_INPUT0_TYPE(ix + 0.5f) * bin_width / TO_INPUT0_TYPE(roi_bin_grid_w); + if (xx < -1.0 || xx > level_w) { + continue; + } + if (xx <= 0) { + xx = 0.0f; + } + int x_low = (int)floor(xx); + int x_high = 0; + + if (x_low >= level_w - 1) { + x_high = x_low = level_w - 1; + xx = TO_INPUT0_TYPE(x_low); + } else { + x_high = x_low + 1; + } + + INPUT0_TYPE lx = xx - x_low; + INPUT0_TYPE hx = TO_INPUT0_TYPE(1.0f) - lx; + + INPUT0_TYPE w1 = hy * hx; + INPUT0_TYPE w2 = hy * lx; + INPUT0_TYPE w3 = ly * hx; + INPUT0_TYPE w4 = ly * lx; + + output_val += w1 * data[y_low * level_w + x_low] + + w2 * data[y_low * level_w + x_high] + + w3 * data[y_high * level_w + x_low] + + w4 * data[y_high * level_w + x_high]; + } + } + output_val /= TO_INPUT0_TYPE(roi_bin_grid_h * roi_bin_grid_w); + const uint output_offset = OUTPUT_OFFSET + x * OUTPUT_X_PITCH + y * OUTPUT_Y_PITCH + c * OUTPUT_FEATURE_PITCH + r * OUTPUT_BATCH_PITCH; + dst_data[output_offset] = output_val; +} diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/roi_align_ref.cl b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/roi_align_ref.cl new file mode 100644 index 00000000000..017eaa73b87 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/cl_kernels/roi_align_ref.cl @@ -0,0 +1,114 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "include/batch_headers/common.cl" +#include "include/batch_headers/data_types.cl" + +#define MAX(a,b) ((a) > (b) ? (a) : (b)) +#define NUM_ROIS OUTPUT_BATCH_NUM +#define NUM_CHANNELS INPUT0_FEATURE_NUM +#define POOLED_WIDTH OUTPUT_SIZE_X +#define POOLED_HEIGHT OUTPUT_SIZE_Y + +KERNEL(roi_align_ref) +( + const __global INPUT0_TYPE * src_data, + __global OUTPUT_TYPE * dst_data, + const __global INPUT1_TYPE * src_rois, + const __global INPUT2_TYPE * src_batches +) +{ + const size_t i = get_global_id(0); + + const uint x = i % POOLED_WIDTH; + const uint y = i / POOLED_WIDTH % POOLED_HEIGHT; + const uint c = i / POOLED_WIDTH / POOLED_HEIGHT % NUM_CHANNELS; + const uint r = i / POOLED_WIDTH / POOLED_HEIGHT / NUM_CHANNELS % NUM_ROIS; + + const __global INPUT1_TYPE* roi_ptr = &src_rois[INPUT1_BATCH_PITCH * r]; + + // Get ROI`s corners + const INPUT1_TYPE x1 = *roi_ptr * (INPUT1_TYPE) SPATIAL_SCALE; + const INPUT1_TYPE y1 = roi_ptr[1] * (INPUT1_TYPE) SPATIAL_SCALE; + const INPUT1_TYPE x2 = roi_ptr[2] * (INPUT1_TYPE) SPATIAL_SCALE; + const INPUT1_TYPE y2 = roi_ptr[3] * (INPUT1_TYPE) SPATIAL_SCALE; + + const INPUT1_TYPE roi_width = MAX(x2 - x1, (INPUT1_TYPE) 1.0); + const INPUT1_TYPE roi_height = MAX(y2 - y1, (INPUT1_TYPE) 1.0); + + const INPUT1_TYPE bin_width = roi_width / POOLED_WIDTH; + const INPUT1_TYPE bin_height = roi_height / POOLED_HEIGHT; + + const int sampling_ratio_x = SAMPLING_RATIO == 0 ? (int) ceil(bin_width) : SAMPLING_RATIO; + const int sampling_ratio_y = SAMPLING_RATIO == 0 ? (int) ceil(bin_height) : SAMPLING_RATIO; + + const INPUT1_TYPE sample_distance_x = bin_width / (INPUT1_TYPE) sampling_ratio_x; + const INPUT1_TYPE sample_distance_y = bin_height / (INPUT1_TYPE) sampling_ratio_y; + + const __global INPUT0_TYPE* data = src_data + INPUT0_OFFSET + r*INPUT0_BATCH_PITCH + INPUT0_FEATURE_PITCH*c; + OUTPUT_TYPE pooled_value = 0; + for (unsigned int y_sample_ind = 0; y_sample_ind < sampling_ratio_y; y_sample_ind++) { + INPUT1_TYPE sample_y = y1 + (INPUT1_TYPE) y * bin_height + + sample_distance_y * ((INPUT1_TYPE) y_sample_ind + (INPUT1_TYPE) 0.5f); + for (unsigned int x_sample_ind = 0; x_sample_ind < sampling_ratio_x; x_sample_ind++) { + INPUT1_TYPE sample_x = x1 + (INPUT1_TYPE) x * bin_width + + sample_distance_x * ((INPUT1_TYPE) x_sample_ind + (INPUT1_TYPE) 0.5f); + unsigned int sample_y_low = 0; + unsigned int sample_x_low = 0; + unsigned int sample_y_high = 0; + unsigned int sample_x_high = 0; + INPUT1_TYPE weight_left = (INPUT1_TYPE) 0.f; + INPUT1_TYPE weight_right = (INPUT1_TYPE) 0.f; + INPUT1_TYPE weight_top = (INPUT1_TYPE) 0.f; + INPUT1_TYPE weight_bottom = (INPUT1_TYPE) 0.f; + if (sample_x >= -1.0 || sample_x <= INPUT0_SIZE_X || sample_y >= -1.0 || sample_y <= INPUT0_SIZE_Y) { + sample_x = MAX(sample_x, (INPUT1_TYPE) 0.f); + sample_y = MAX(sample_y, (INPUT1_TYPE) 0.f); + + sample_y_low = (unsigned int) sample_y; + sample_x_low = (unsigned int) sample_x; + + if (sample_y_low >= INPUT0_SIZE_Y - 1) { + sample_y_high = sample_y_low = INPUT0_SIZE_Y - 1; + sample_y = (INPUT1_TYPE) sample_y_low; + } else { + sample_y_high = sample_y_low + 1; + } + + if (sample_x_low >= INPUT0_SIZE_X - 1) { + sample_x_high = sample_x_low = INPUT0_SIZE_X - 1; + sample_x = (INPUT1_TYPE) sample_x_low; + } else { + sample_x_high = sample_x_low + 1; + } + + // weight calculation for bilinear interpolation + weight_top = sample_y - (INPUT1_TYPE) sample_y_low; + weight_left = sample_x - (INPUT1_TYPE) sample_x_low; + weight_bottom = (INPUT1_TYPE) 1.f - weight_top; + weight_right = (INPUT1_TYPE) 1.f - weight_left; + } + const INPUT0_TYPE top_left = data[sample_y_low * INPUT0_Y_PITCH + sample_x_low * INPUT0_X_PITCH]; + const INPUT0_TYPE top_right = data[sample_y_low * INPUT0_Y_PITCH + sample_x_high * INPUT0_X_PITCH]; + const INPUT0_TYPE bottom_left = data[sample_y_high * INPUT0_Y_PITCH + sample_x_low * INPUT0_X_PITCH]; + const INPUT0_TYPE bottom_right = data[sample_y_high * INPUT0_Y_PITCH + sample_x_high * INPUT0_X_PITCH]; + + const INPUT0_TYPE interpolated_value = weight_bottom * weight_right * top_left + + weight_bottom * weight_left * top_right + + weight_top * weight_right * bottom_left + + weight_top * weight_left * bottom_right; +#if MAX_POOL + pooled_value = MAX(pooled_value, interpolated_value); +#elif AVG_POOL + pooled_value += interpolated_value; +#endif + } + } +#if AVG_POOL + pooled_value /= sampling_ratio_x * sampling_ratio_x; +#endif + const uint output_offset = OUTPUT_OFFSET + x*OUTPUT_X_PITCH + y*OUTPUT_Y_PITCH + c*OUTPUT_FEATURE_PITCH + r*OUTPUT_BATCH_PITCH; + dst_data[output_offset] = ACTIVATION((OUTPUT_TYPE)pooled_value, ACTIVATION_PARAMS); +} + diff --git a/inference-engine/thirdparty/clDNN/kernel_selector/core/kernel_selector_params.h b/inference-engine/thirdparty/clDNN/kernel_selector/core/kernel_selector_params.h index 23a3b1167d4..683a8a78da9 100644 --- a/inference-engine/thirdparty/clDNN/kernel_selector/core/kernel_selector_params.h +++ b/inference-engine/thirdparty/clDNN/kernel_selector/core/kernel_selector_params.h @@ -129,7 +129,6 @@ public: uint32_t split : 1; uint32_t dilation : 1; uint32_t depthwise_separable_opt : 1; - uint32_t local : 1; uint32_t grouped : 1; uint32_t deformable : 1; uint32_t bilinear_interpolation_pad : 1; @@ -287,7 +286,6 @@ public: void EnableSplitSupport() { key.restrict.val.dedicated.conv.split = 1; } void EnableDilation() { key.restrict.val.dedicated.conv.dilation = 1; } void EnableDepthwiseSeparableOpt() { key.restrict.val.dedicated.conv.depthwise_separable_opt = 1; } - void EnableLocalConvolution() { key.restrict.val.dedicated.conv.local = 1; } void EnableGroupedConvolution() { key.restrict.val.dedicated.conv.grouped = 1; } void EnableDeformableMode() { key.restrict.val.dedicated.conv.deformable = 1; } void EnableBilinearInterpolationPad() { key.restrict.val.dedicated.conv.bilinear_interpolation_pad = 1; } diff --git a/inference-engine/thirdparty/clDNN/src/experimental_detectron_roi_feature_extractor.cpp b/inference-engine/thirdparty/clDNN/src/experimental_detectron_roi_feature_extractor.cpp new file mode 100644 index 00000000000..e06eb88606c --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/experimental_detectron_roi_feature_extractor.cpp @@ -0,0 +1,67 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "experimental_detectron_roi_feature_extractor_inst.hpp" +#include "primitive_type_base.h" +#include "cldnn/runtime/error_handler.hpp" +#include "json_object.h" +#include + +namespace cldnn { +primitive_type_id experimental_detectron_roi_feature_extractor::type_id() { + static primitive_type_base instance; + return &instance; +} + +size_t experimental_detectron_roi_feature_extractor_inst::inputs_memory_count() const { + return parent::inputs_memory_count() - 1; +} + +memory::ptr experimental_detectron_roi_feature_extractor_inst::second_output_memory() const { + return input_memory_ptr(parent::inputs_memory_count() - 1); +} + +memory::ptr experimental_detectron_roi_feature_extractor_inst::rois_memory() const { + return input_memory_ptr(0); +} + +void experimental_detectron_roi_feature_extractor_inst::copy_rois_input_to_second_output() const { + second_output_memory()->copy_from(get_network().get_stream(), *rois_memory()); +} + +layout experimental_detectron_roi_feature_extractor_inst::calc_output_layout(experimental_detectron_roi_feature_extractor_node const& node) { + assert(static_cast(node.get_primitive()->output_data_type) == false && + "Output data type forcing is not supported for roi_pooling_node!"); + layout rois_layout = node.input(0).get_output_layout(); + layout data_layout = node.input(1).get_output_layout(); + int num_rois = rois_layout.size.batch[0]; + int num_channels = data_layout.size.feature[0]; + auto desc = node.get_primitive(); + + return layout(data_layout.data_type, format::bfyx, {num_rois, num_channels, desc->output_dim, desc->output_dim}); +} + +std::string experimental_detectron_roi_feature_extractor_inst::to_string(experimental_detectron_roi_feature_extractor_node const& node) { + auto desc = node.get_primitive(); + + std::stringstream primitive_description; + + json_composite experimental_detectron_info; + experimental_detectron_info.add("output_size", desc->output_dim); + experimental_detectron_info.add("pooled_height", desc->pooled_height); + experimental_detectron_info.add("pooled_width", desc->pooled_width); + experimental_detectron_info.add("sampling_ratio", desc->sampling_ratio); + for (std::size_t i = 0; i < desc->pyramid_scales.size(); i++) { + experimental_detectron_info.add("pyramid_scales[" + std::to_string(i) + "]", desc->pyramid_scales[i]); + } + experimental_detectron_info.add("aligned", (desc->aligned ? "true" : "false")); + + auto node_info = node.desc_to_json(); + node_info->add("experimental_detectron_info", experimental_detectron_info); + node_info->dump(primitive_description); + + return primitive_description.str(); +} + +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/impls/ocl/convolution.cpp b/inference-engine/thirdparty/clDNN/src/impls/ocl/convolution.cpp index 1ec873768e4..ff96abd67f5 100644 --- a/inference-engine/thirdparty/clDNN/src/impls/ocl/convolution.cpp +++ b/inference-engine/thirdparty/clDNN/src/impls/ocl/convolution.cpp @@ -91,7 +91,6 @@ public: conv_params.transposed = transposed; conv_params.deformable_groups = deformable_groups; - conv_params.local_convolution = weights_size.local[0] > 1 || weights_size.local[1] > 1; conv_params.split = split; conv_params.groups = groups; diff --git a/inference-engine/thirdparty/clDNN/src/impls/ocl/experimental_detectron_roi_feature_extractor.cpp b/inference-engine/thirdparty/clDNN/src/impls/ocl/experimental_detectron_roi_feature_extractor.cpp new file mode 100644 index 00000000000..3a0937fcc6f --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/impls/ocl/experimental_detectron_roi_feature_extractor.cpp @@ -0,0 +1,88 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp" +#include "experimental_detectron_roi_feature_extractor_inst.hpp" +#include "primitive_base.hpp" +#include "impls/implementation_map.hpp" +#include "kernel_selector_helper.h" +#include "edrfe/experimental_detectron_roi_feature_extractor_kernel_selector.h" +#include "edrfe/experimental_detectron_roi_feature_extractor_kernel_ref.h" + +namespace cldnn { +namespace ocl { +struct experimental_detectron_roi_feature_extractor_impl : public typed_primitive_impl_ocl { + using parent = typed_primitive_impl_ocl; + using parent::parent; + + std::unique_ptr clone() const override { + return make_unique(*this); + } + +protected: + kernel_arguments_data get_arguments(experimental_detectron_roi_feature_extractor_inst& instance, int32_t) const override { + kernel_arguments_data args; + + for (std::size_t i = 0; i < instance.inputs_memory_count(); i++) { + args.inputs.push_back(instance.input_memory_ptr(i)); + } + args.output = instance.output_memory_ptr(); + + return args; + } + + event::ptr execute_impl(const std::vector& events, + experimental_detectron_roi_feature_extractor_inst& instance) override { + instance.copy_rois_input_to_second_output(); + return parent::execute_impl(events, instance); + } + +public: + static primitive_impl* create(const experimental_detectron_roi_feature_extractor_node& arg) { + const auto output_layout = arg.get_output_layout(); + const auto padding_filling_value = output_layout.data_padding.filling_value(); + CLDNN_ERROR_NOT_EQUAL(arg.id(), + "experimental_detectron_roi_feature_extractor padding filling value", + padding_filling_value, + "padding mode", + 0.0f, + "Unknown padding mode in experimental_detectron_roi_feature_extractor."); + + auto params = get_default_params(arg); + auto optional_params = get_default_optional_params(arg.get_program()); + + const auto& primitive = arg.get_primitive(); + size_t number_of_inputs = primitive->input_size() - 1; + for (std::size_t i = 1; i < number_of_inputs; i++) { + params.inputs.push_back(convert_data_tensor(arg.input(i).get_output_layout())); + } + + params.output_dim = primitive->output_dim; + params.pooled_height = primitive->pooled_height; + params.pooled_width = primitive->pooled_width; + params.pyramid_scales = primitive->pyramid_scales; + params.sampling_ratio = primitive->sampling_ratio; + params.aligned = primitive->aligned; + params.number_of_inputs = number_of_inputs; + + auto& kernel_selector = kernel_selector::experimental_detectron_roi_feature_extractor_kernel_selector::Instance(); + auto best_kernels = kernel_selector.GetBestKernels(params, optional_params); + + CLDNN_ERROR_BOOL(arg.id(), "best_kernels.empty()", best_kernels.empty(), "Cannot find a proper kernel with this arguments"); + return new experimental_detectron_roi_feature_extractor_impl(arg, best_kernels.front()); + } +}; + +namespace detail { +attach_experimental_detectron_roi_feature_extractor_impl::attach_experimental_detectron_roi_feature_extractor_impl() { + implementation_map::add(impl_types::ocl, + experimental_detectron_roi_feature_extractor_impl::create, { + std::make_tuple(data_types::f16, format::bfyx), + std::make_tuple(data_types::f32, format::bfyx) + }); +} + +} // namespace detail +} // namespace ocl +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/impls/ocl/register.cpp b/inference-engine/thirdparty/clDNN/src/impls/ocl/register.cpp index 1d455cdbc97..7deb302db2a 100644 --- a/inference-engine/thirdparty/clDNN/src/impls/ocl/register.cpp +++ b/inference-engine/thirdparty/clDNN/src/impls/ocl/register.cpp @@ -28,6 +28,7 @@ void register_implementations() { REGISTER_OCL(depth_to_space); REGISTER_OCL(detection_output); REGISTER_OCL(batch_to_space); + REGISTER_OCL(experimental_detectron_roi_feature_extractor); REGISTER_OCL(eltwise); REGISTER_OCL(fully_connected); REGISTER_OCL(gather); @@ -53,6 +54,7 @@ void register_implementations() { REGISTER_OCL(reorg_yolo); REGISTER_OCL(reshape); REGISTER_OCL(reverse_sequence); + REGISTER_OCL(roi_align); REGISTER_OCL(roi_pooling); REGISTER_OCL(scale); REGISTER_OCL(scatter_update); diff --git a/inference-engine/thirdparty/clDNN/src/impls/ocl/register.hpp b/inference-engine/thirdparty/clDNN/src/impls/ocl/register.hpp index 2b1623576e1..bb8a690a1ee 100644 --- a/inference-engine/thirdparty/clDNN/src/impls/ocl/register.hpp +++ b/inference-engine/thirdparty/clDNN/src/impls/ocl/register.hpp @@ -20,6 +20,7 @@ #include "cldnn/primitives/depth_to_space.hpp" #include "cldnn/primitives/detection_output.hpp" #include "cldnn/primitives/eltwise.hpp" +#include "cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp" #include "cldnn/primitives/fully_connected.hpp" #include "cldnn/primitives/gather.hpp" #include "cldnn/primitives/gather_nd.hpp" @@ -44,6 +45,7 @@ #include "cldnn/primitives/reorg_yolo.hpp" #include "cldnn/primitives/reshape.hpp" #include "cldnn/primitives/reverse_sequence.hpp" +#include "cldnn/primitives/roi_align.hpp" #include "cldnn/primitives/roi_pooling.hpp" #include "cldnn/primitives/scale.hpp" #include "cldnn/primitives/scatter_update.hpp" @@ -92,6 +94,7 @@ REGISTER_OCL(deformable_conv); REGISTER_OCL(deformable_interp); REGISTER_OCL(depth_to_space); REGISTER_OCL(detection_output); +REGISTER_OCL(experimental_detectron_roi_feature_extractor); REGISTER_OCL(eltwise); REGISTER_OCL(embed); REGISTER_OCL(fully_connected); @@ -118,6 +121,7 @@ REGISTER_OCL(reorder); REGISTER_OCL(reorg_yolo); REGISTER_OCL(reshape); REGISTER_OCL(reverse_sequence); +REGISTER_OCL(roi_align); REGISTER_OCL(roi_pooling); REGISTER_OCL(scale); REGISTER_OCL(scatter_update); diff --git a/inference-engine/thirdparty/clDNN/src/impls/ocl/roi_align.cpp b/inference-engine/thirdparty/clDNN/src/impls/ocl/roi_align.cpp new file mode 100644 index 00000000000..6312023233a --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/impls/ocl/roi_align.cpp @@ -0,0 +1,103 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#include "roi_align_inst.h" +#include "primitive_base.hpp" +#include "impls/implementation_map.hpp" +#include "cldnn/runtime/error_handler.hpp" +#include "kernel_selector_helper.h" +#include "roi_align/roi_align_kernel_selector.h" +#include "roi_align/roi_align_kernel_ref.h" + +namespace cldnn { +namespace ocl { + +namespace { +kernel_selector::pool_type from(roi_align::PoolingMode mode) { + switch (mode) { + case roi_align::PoolingMode::Max: + return kernel_selector::pool_type::MAX; + default: + case roi_align::PoolingMode::Avg: + return kernel_selector::pool_type::AVG; + } +} +} // namespace + +struct roi_align_impl : typed_primitive_impl_ocl { + using parent = typed_primitive_impl_ocl; + using parent::parent; + + std::unique_ptr clone() const override { + return make_unique(*this); + } + +protected: + kernel_arguments_data get_arguments(typed_primitive_inst& instance, int32_t) const override { + kernel_arguments_data args; + args.inputs = { instance.input_memory_ptr(), instance.rois_memory(), instance.batches_memory() }; + args.output = instance.output_memory_ptr(); + + return args; + } + +public: + static primitive_impl* create(const roi_align_node& arg) { + const auto& input_layout = arg.input().get_output_layout(); + const auto& output_layout = arg.get_output_layout(); + const auto& rois_layout = arg.input(1).get_output_layout(); + const auto& batches_layout = arg.input(2).get_output_layout(); + const auto& primitive = arg.get_primitive(); + + const auto padding_filling_value = output_layout.data_padding.filling_value(); + + CLDNN_ERROR_NOT_EQUAL(arg.id(), + "roi_align padding filling value", + padding_filling_value, + "padding mode", + 0.0f, + "Unknown padding mode in roi_align."); + CLDNN_ERROR_NOT_PROPER_FORMAT(arg.id(), + "Input_layout.format", + input_layout.format.value, + "output_layout.format", + output_layout.format); + + auto roi_align_params = get_default_params(arg); + auto roi_align_optional_params = + get_default_optional_params(arg.get_program()); + + const auto roi_bfyx = convert_data_tensor(rois_layout); + roi_align_params.inputs.push_back(roi_bfyx.FlattenFeatureAndSpatials()); + roi_align_params.inputs.push_back(convert_data_tensor(batches_layout)); + roi_align_params.mode = from(primitive->mode); + roi_align_params.sampling_ratio = primitive->sampling_ratio; + roi_align_params.spatial_scale = primitive->spatial_scale; + + auto& kernel_selector = kernel_selector::roi_align_kernel_selector::Instance(); + auto best_kernels = kernel_selector.GetBestKernels(roi_align_params, roi_align_optional_params); + + CLDNN_ERROR_BOOL(arg.id(), + "Best_kernel.empty()", + best_kernels.empty(), + "Cannot find a proper kernel with this arguments"); + + auto roi_align = new roi_align_impl(arg, best_kernels[0]); + + return roi_align; + } +}; + +namespace detail { + +attach_roi_align_impl::attach_roi_align_impl() { + implementation_map::add(impl_types::ocl, roi_align_impl::create, + { + std::make_tuple(data_types::f16, format::bfyx), + std::make_tuple(data_types::f32, format::bfyx), + }); +} + +} // namespace detail +} // namespace ocl +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/include/experimental_detectron_roi_feature_extractor_inst.hpp b/inference-engine/thirdparty/clDNN/src/include/experimental_detectron_roi_feature_extractor_inst.hpp new file mode 100644 index 00000000000..e2a4c11cebe --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/include/experimental_detectron_roi_feature_extractor_inst.hpp @@ -0,0 +1,40 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +/////////////////////////////////////////////////////////////////////////////////////////////////// + +#pragma once +#include "cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp" +#include "primitive_inst.h" + +namespace cldnn { +template <> +struct typed_program_node : public typed_program_node_base { + using parent = typed_program_node_base; + using parent::parent; + + program_node& input(size_t index = 0) const { return get_dependency(index); } +}; + +using experimental_detectron_roi_feature_extractor_node = typed_program_node; + +template <> +struct typed_primitive_inst : public typed_primitive_inst_base { + using parent = typed_primitive_inst_base; + using parent::parent; + +public: + size_t inputs_memory_count() const; + void copy_rois_input_to_second_output() const; + + static layout calc_output_layout(experimental_detectron_roi_feature_extractor_node const& node); + static std::string to_string(experimental_detectron_roi_feature_extractor_node const& node); + +private: + memory::ptr second_output_memory() const; + memory::ptr rois_memory() const; +}; + +using experimental_detectron_roi_feature_extractor_inst = typed_primitive_inst; +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/include/program_helpers.h b/inference-engine/thirdparty/clDNN/src/include/program_helpers.h index 564e4f15238..26b06ddbcd2 100644 --- a/inference-engine/thirdparty/clDNN/src/include/program_helpers.h +++ b/inference-engine/thirdparty/clDNN/src/include/program_helpers.h @@ -124,6 +124,8 @@ struct program_helpers { } } static layout get_weights_layout(typed_program_node& data_node, int32_t split); + + static bool are_layouts_identical_for_onednn_sum_post_op(layout input_layout, layout output_layout); }; // Base class for performing pattern match style optimizations. diff --git a/inference-engine/thirdparty/clDNN/src/include/roi_align_inst.h b/inference-engine/thirdparty/clDNN/src/include/roi_align_inst.h new file mode 100644 index 00000000000..785387727b1 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/include/roi_align_inst.h @@ -0,0 +1,40 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#pragma once + +#include +#include "primitive_inst.h" +#include + +namespace cldnn { + +template <> +struct typed_program_node : public typed_program_node_base { + using parent = typed_program_node_base; + +public: + using parent::parent; + + program_node& input(std::size_t index = 0) const { return get_dependency(index); } +}; + +using roi_align_node = typed_program_node; + +template <> +class typed_primitive_inst : public typed_primitive_inst_base { + using parent = typed_primitive_inst_base; + +public: + static layout calc_output_layout(roi_align_node const& node); + static std::string to_string(roi_align_node const& node); + +public: + typed_primitive_inst(network& network, roi_align_node const& desc); + memory::ptr rois_memory() const { return dep_memory_ptr(1); } + memory::ptr batches_memory() const { return dep_memory_ptr(2); } +}; + +using roi_align_inst = typed_primitive_inst; + +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/kernel_selector_helper.cpp b/inference-engine/thirdparty/clDNN/src/kernel_selector_helper.cpp index 09c981cf1d3..ec75b2e6047 100644 --- a/inference-engine/thirdparty/clDNN/src/kernel_selector_helper.cpp +++ b/inference-engine/thirdparty/clDNN/src/kernel_selector_helper.cpp @@ -707,8 +707,6 @@ layout from_weights_tensor(const kernel_selector::weights_tensor& l) { size.spatial[0] = static_cast(l.X().v); size.spatial[1] = static_cast(l.Y().v); size.spatial[2] = static_cast(l.Z().v); - size.local[0] = static_cast(l.LX().v); - size.local[1] = static_cast(l.LY().v); return layout(type, format, size); } diff --git a/inference-engine/thirdparty/clDNN/src/layout_optimizer.cpp b/inference-engine/thirdparty/clDNN/src/layout_optimizer.cpp index 461037f4c45..10cc5ddddd1 100644 --- a/inference-engine/thirdparty/clDNN/src/layout_optimizer.cpp +++ b/inference-engine/thirdparty/clDNN/src/layout_optimizer.cpp @@ -4,6 +4,7 @@ #include "layout_optimizer.h" #include "primitive_inst.h" +#include "program_helpers.h" #include "cldnn/runtime/error_handler.hpp" #include "data_inst.h" @@ -192,6 +193,20 @@ bool layout_optimizer::can_fuse_reorder(program_node& prev, program_node& next, return false; }; + // Not to fuse reorder if this removal changes input format of its next node which has reuse in fused_op + if (next.get_preferred_impl_type() == impl_types::onednn) { + for (auto& fused_op : next.get_fused_primitives()) { + if (fused_op.node->is_type() && fused_op.deps.size() == 1) { + auto eltw_in_layout = next.get_dependency(fused_op.dep_start_idx).get_output_layout(); + auto out_layout = next.get_output_layout(); + if (fused_op.node->as().get_primitive()->needs_onednn_sum_post_op(eltw_in_layout) && + program_helpers::are_layouts_identical_for_onednn_sum_post_op(eltw_in_layout, out_layout) && + prev.get_output_layout().format != out_layout.format) + return false; + } + } + } + if (next.is_type()) { // Avoid fusing current reorder to fuse next reorder if (next.get_users().size() == 1 && next.get_users().front()->is_type() && use_onednn_impls) { diff --git a/inference-engine/thirdparty/clDNN/src/network.cpp b/inference-engine/thirdparty/clDNN/src/network.cpp index c6aa1b78fbc..18bf78981cd 100644 --- a/inference-engine/thirdparty/clDNN/src/network.cpp +++ b/inference-engine/thirdparty/clDNN/src/network.cpp @@ -25,6 +25,7 @@ #include "condition_inst.h" #include "loop_inst.h" #include "kernel_selector_helper.h" +#include "program_helpers.h" #include "runtime/cldnn_itt.hpp" #include @@ -505,13 +506,10 @@ void network::allocate_primitives() { if (!fused_op.node->as().get_primitive()->needs_onednn_sum_post_op(eltw_in_layout)) continue; - if (eltw_in_layout.size == out_layout.size && - eltw_in_layout.format == out_layout.format && - eltw_in_layout.data_padding == out_layout.data_padding && - data_type_traits::size_of(eltw_in_layout.data_type) == data_type_traits::size_of(out_layout.data_type)) { - if (eltw_dep > 0) { + if (program_helpers::are_layouts_identical_for_onednn_sum_post_op(eltw_in_layout, out_layout)) { + if (eltw_dep > 0) throw std::runtime_error("Unsupported multiple full size tensors."); - } + eltw_dep = fused_op.dep_start_idx; can_reuse_eltwise_mem = true; } diff --git a/inference-engine/thirdparty/clDNN/src/program.cpp b/inference-engine/thirdparty/clDNN/src/program.cpp index 5e75913d1ee..895b2213ef0 100644 --- a/inference-engine/thirdparty/clDNN/src/program.cpp +++ b/inference-engine/thirdparty/clDNN/src/program.cpp @@ -1159,7 +1159,8 @@ std::string program::get_implementation_info(const primitive_id& id) const { try { const auto& node = get_node(id); auto impl = node.get_selected_impl(); - return impl ? (impl->get_kernel_name() + "__" + dt_to_str(get_inference_precision(node))) : "undef"; + auto kernel_name = impl ? impl->get_kernel_name() : ""; + return !kernel_name.empty() ? (kernel_name + "__" + dt_to_str(get_inference_precision(node))) : "undef"; } catch (...) { } return "undef"; diff --git a/inference-engine/thirdparty/clDNN/src/program_helpers.cpp b/inference-engine/thirdparty/clDNN/src/program_helpers.cpp index db6c0644410..3701f7f17d9 100644 --- a/inference-engine/thirdparty/clDNN/src/program_helpers.cpp +++ b/inference-engine/thirdparty/clDNN/src/program_helpers.cpp @@ -175,4 +175,15 @@ std::pair program_helpers::are_layouts_identical(layout const& l1, l return {false, false}; } + +// check if input and output layouts are identical to reuse memory in fused_ops of onednn +bool program_helpers::are_layouts_identical_for_onednn_sum_post_op(layout input_layout, layout output_layout) { + if (input_layout.size == output_layout.size && input_layout.format == output_layout.format && + input_layout.data_padding == output_layout.data_padding && + data_type_traits::size_of(input_layout.data_type) == data_type_traits::size_of(output_layout.data_type)) + return true; + + return false; +} + } // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/src/roi_align.cpp b/inference-engine/thirdparty/clDNN/src/roi_align.cpp new file mode 100644 index 00000000000..15ce6b803b4 --- /dev/null +++ b/inference-engine/thirdparty/clDNN/src/roi_align.cpp @@ -0,0 +1,46 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include +#include "primitive_type_base.h" +#include +#include + +namespace cldnn { + +primitive_type_id roi_align::type_id() { + static primitive_type_base instance; + return &instance; +} + +roi_align_inst::typed_primitive_inst(network& network, roi_align_node const& node) + : parent(network, node) {} + +layout roi_align_inst::calc_output_layout(roi_align_node const& node) { + auto primitive = node.get_primitive(); + auto input_layout = node.input(0).get_output_layout(); + auto rois_layout = node.input(0).get_output_layout(); + auto num_rois = rois_layout.size.batch[0]; + auto num_channels = input_layout.size.feature[0]; + return layout(input_layout.data_type, format::bfyx, {num_rois, num_channels, primitive->pooled_h, primitive->pooled_w}); +} + +std::string roi_align_inst::to_string(roi_align_node const& node) { + auto node_info = node.desc_to_json(); + json_composite roi_align_info; + roi_align_info.add("input id", node.input().id()); + roi_align_info.add("rois id", node.get_dependency(1).id()); + roi_align_info.add("batches id", node.get_dependency(2).id()); + roi_align_info.add("pooled_h", node.get_primitive()->pooled_h); + roi_align_info.add("pooled_w", node.get_primitive()->pooled_w); + roi_align_info.add("sampling_ratio", node.get_primitive()->sampling_ratio); + roi_align_info.add("spatial_scale", node.get_primitive()->spatial_scale); + roi_align_info.add("mode", node.get_primitive()->mode == roi_align::PoolingMode::Max ? "Max" : "Avg"); + node_info->add("roi_align info", roi_align_info); + std::stringstream primitive_description; + node_info->dump(primitive_description); + return primitive_description.str(); +} + +} // namespace cldnn diff --git a/inference-engine/thirdparty/clDNN/tests/test_cases/experimental_detectron_roi_feature_extractor_gpu_test.cpp b/inference-engine/thirdparty/clDNN/tests/test_cases/experimental_detectron_roi_feature_extractor_gpu_test.cpp new file mode 100644 index 00000000000..5a8a17515ad --- /dev/null +++ b/inference-engine/thirdparty/clDNN/tests/test_cases/experimental_detectron_roi_feature_extractor_gpu_test.cpp @@ -0,0 +1,185 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "test_utils.h" + +#include +#include +#include + +#include +#include + +using namespace cldnn; +using namespace ::tests; + +TEST(experimental_detectron_roi_feature_extractor_gpu_fp32, one_level) { + auto& engine = get_test_engine(); + + const int rois_num = 2; + const int rois_feature_dim = 4; + auto roi_input = engine.allocate_memory({data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + auto level_1 = engine.allocate_memory({data_types::f32, format::bfyx, {1, 2, 3, 2}}); + auto second_output = engine.allocate_memory({ data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + + std::vector rois {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f}; + set_values(roi_input, rois); + set_values(level_1, {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f, 9.0f, 10.0f, 11.0f}); + + const int output_dim = 3; + const std::vector pyramid_scales = {4}; + const int sampling_ratio = 2; + const bool aligned = false; + + topology topology; + topology.add(input_layout("InputRois", roi_input->get_layout())); + topology.add(input_layout("InputLevel1", level_1->get_layout())); + topology.add(mutable_data("second_output", second_output)); + topology.add(experimental_detectron_roi_feature_extractor("edrfe", + {"InputRois", "InputLevel1", "second_output"}, + output_dim, + pyramid_scales, + sampling_ratio, + aligned)); + + network network(engine, topology); + + network.set_input_data("InputRois", roi_input); + network.set_input_data("InputLevel1", level_1); + + auto outputs = network.execute(); + + std::vector expected_out {1.416667f, 1.75f, 2.083333f, 2.416667f, 2.75f, 3.083333f, 3.166667f, 3.5f, 3.833333f, + 7.416667f, 7.75f, 8.083333f, 8.416667f, 8.75f, 9.083334f, 9.166666f, 9.5f, 9.833334f, + 4.166667f, 4.5f, 4.833333f, 4.166667f, 4.5f, 4.833333f, 2.083333f, 2.25f, 2.416667f, + 10.16667f, 10.5f, 10.83333f, 10.16667f, 10.5f, 10.83333f, 5.083333f, 5.25f, 5.416667f}; + + std::vector& expected_second_out = rois; + + auto output = outputs.at("edrfe").get_memory(); + cldnn::mem_lock output_ptr(output, get_test_stream()); + + ASSERT_EQ(expected_out.size(), output_ptr.size()); + for (std::size_t i = 0; i < expected_out.size(); i++) { + EXPECT_FLOAT_EQ(expected_out[i], output_ptr[i]); + } + + cldnn::mem_lock second_output_ptr(second_output, get_test_stream()); + + ASSERT_EQ(expected_second_out.size(), second_output_ptr.size()); + for (std::size_t i = 0; i < expected_second_out.size(); i++) { + EXPECT_FLOAT_EQ(expected_second_out[i], second_output_ptr[i]); + } +} + +TEST(experimental_detectron_roi_feature_extractor_gpu_fp32, two_levels) { + auto& engine = get_test_engine(); + + const int rois_num = 2; + const int rois_feature_dim = 4; + auto roi_input = engine.allocate_memory({data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + auto level_1 = engine.allocate_memory({data_types::f32, format::bfyx, {1, 2, 3, 2}}); + auto level_2 = engine.allocate_memory({data_types::f32, format::bfyx, {1, 2, 3, 2}}); + auto second_output = engine.allocate_memory({ data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + + std::vector rois {0.0f, 56.0f, 112.0f, 168.0f, 4.0f, 5.0f, 6.0f, 7.0f}; + set_values(roi_input, rois); + set_values(level_1, {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f, 9.0f, 10.0f, 11.0f}); + set_values(level_2, {6.0f, 7.0f, 8.0f, 9.0f, 10.0f, 11.0f, 0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f}); + + const int output_dim = 3; + const std::vector pyramid_scales = {4, 224}; + const int sampling_ratio = 2; + const bool aligned = false; + + topology topology; + topology.add(input_layout("InputRois", roi_input->get_layout())); + topology.add(input_layout("InputLevel1", level_1->get_layout())); + topology.add(input_layout("InputLevel2", level_2->get_layout())); + topology.add(mutable_data("second_output", second_output)); + topology.add(experimental_detectron_roi_feature_extractor("edrfe", + {"InputRois", "InputLevel1", "InputLevel2", "second_output"}, + output_dim, + pyramid_scales, + sampling_ratio, + aligned)); + + network network(engine, topology); + + network.set_input_data("InputRois", roi_input); + network.set_input_data("InputLevel1", level_1); + network.set_input_data("InputLevel2", level_2); + + auto outputs = network.execute(); + + std::vector expected_out {7.41662f, 7.7499523f, 8.0832853f, 8.41662f, 8.74995f, 9.0832853f, 9.16664f, 9.49998f, 9.83331f, + 1.4166187f, 1.7499521f, 2.0832853f, 2.4166186f, 2.7499518f, 3.0832853f, 3.1666427f, 3.4999762f, 3.83331f, + 4.166667f, 4.5f, 4.833333f, 4.166667f, 4.5f, 4.833333f, 2.083333f, 2.25f, 2.416667f, + 10.16667f, 10.5f, 10.83333f, 10.16667f, 10.5f, 10.83333f, 5.083333f, 5.25f, 5.416667f}; + + std::vector& expected_second_out = rois; + + auto output = outputs.at("edrfe").get_memory(); + cldnn::mem_lock output_ptr(output, get_test_stream()); + + ASSERT_EQ(expected_out.size(), output_ptr.size()); + for (std::size_t i = 0; i < expected_out.size(); i++) { + EXPECT_FLOAT_EQ(expected_out[i], output_ptr[i]); + } + + cldnn::mem_lock second_output_ptr(second_output, get_test_stream()); + + ASSERT_EQ(expected_second_out.size(), second_output_ptr.size()); + for (std::size_t i = 0; i < expected_second_out.size(); i++) { + EXPECT_FLOAT_EQ(expected_second_out[i], second_output_ptr[i]); + } +} + +TEST(experimental_detectron_roi_feature_extractor_gpu_fp32, second_output) { + auto& engine = get_test_engine(); + + const int rois_num = 2; + const int rois_feature_dim = 4; + auto roi_input = engine.allocate_memory({data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + auto level_1 = engine.allocate_memory({data_types::f32, format::bfyx, {1, 2, 3, 2}}); + auto second_output = engine.allocate_memory({ data_types::f32, format::bfyx, tensor(batch(rois_num), feature(rois_feature_dim))}); + + std::vector rois {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f}; + set_values(roi_input, rois); + set_values(level_1, {0.0f, 1.0f, 2.0f, 3.0f, 4.0f, 5.0f, 6.0f, 7.0f, 8.0f, 9.0f, 10.0f, 11.0f}); + + const int output_dim = 3; + const std::vector pyramid_scales = {4}; + const int sampling_ratio = 2; + const bool aligned = false; + + topology topology; + topology.add(input_layout("InputRois", roi_input->get_layout())); + topology.add(input_layout("InputLevel1", level_1->get_layout())); + topology.add(mutable_data("second_output_w", second_output)); + topology.add(experimental_detectron_roi_feature_extractor("edrfe", + {"InputRois", "InputLevel1", "second_output_w"}, + output_dim, + pyramid_scales, + sampling_ratio, + aligned)); + topology.add(mutable_data("second_output_r", {"edrfe"}, second_output)); + + network network(engine, topology); + + network.set_input_data("InputRois", roi_input); + network.set_input_data("InputLevel1", level_1); + + auto outputs = network.execute(); + + std::vector& expected_second_out = rois; + + auto output = outputs.at("second_output_r").get_memory(); + cldnn::mem_lock second_output_ptr(output, get_test_stream()); + + ASSERT_EQ(expected_second_out.size(), second_output_ptr.size()); + for (std::size_t i = 0; i < expected_second_out.size(); i++) { + EXPECT_FLOAT_EQ(expected_second_out[i], second_output_ptr[i]); + } +} diff --git a/model-optimizer/automation/package_BOM.txt b/model-optimizer/automation/package_BOM.txt index 9bfd6d68f98..1b36cb13f99 100644 --- a/model-optimizer/automation/package_BOM.txt +++ b/model-optimizer/automation/package_BOM.txt @@ -85,7 +85,7 @@ extensions/front/caffe/conv_ext.py extensions/front/caffe/crop_ext.py extensions/front/caffe/ctcgreedydecoder_ext.py extensions/front/caffe/CustomLayersMapping.xml.example -extensions/front/caffe/detection_output.py +extensions/front/caffe/detection_output_ext.py extensions/front/caffe/dropout_ext.py extensions/front/caffe/elementwise_ext.py extensions/front/caffe/eltwise_add_normalize.py @@ -271,8 +271,8 @@ extensions/front/onnx/cumsum_ext.py extensions/front/onnx/deformable_conv_ext.py extensions/front/onnx/depth_to_space_ext.py extensions/front/onnx/dequantize_linear_ext.py -extensions/front/onnx/detection_output.py -extensions/front/onnx/detectionoutput_ext.py +extensions/front/onnx/detection_output_ext.py +extensions/front/onnx/detection_output_onnx_ext.py extensions/front/onnx/dropout_ext.py extensions/front/onnx/einsum_ext.py extensions/front/onnx/elementwise_ext.py @@ -680,8 +680,8 @@ extensions/ops/ctc_loss.py extensions/ops/cumsum.py extensions/ops/depth_to_space.py extensions/ops/dequantize_linear.py +extensions/ops/detection_output_onnx.py extensions/ops/DetectionOutput.py -extensions/ops/detectionoutput_onnx.py extensions/ops/dft.py extensions/ops/einsum.py extensions/ops/elementwise.py @@ -807,6 +807,7 @@ mo/back/__init__.py mo/back/ie_ir_ver_2/__init__.py mo/back/ie_ir_ver_2/emitter.py mo/back/offline_transformations.py +mo/back/preprocessing.py mo/back/replacement.py mo/front/__init__.py mo/front/caffe/__init__.py @@ -837,7 +838,6 @@ mo/front/common/partial_infer/concat.py mo/front/common/partial_infer/crop.py mo/front/common/partial_infer/elemental.py mo/front/common/partial_infer/eltwise.py -mo/front/common/partial_infer/multi_box_detection.py mo/front/common/partial_infer/multi_box_prior.py mo/front/common/partial_infer/roipooling.py mo/front/common/partial_infer/utils.py diff --git a/model-optimizer/extensions/front/caffe/detection_output.py b/model-optimizer/extensions/front/caffe/detection_output_ext.py similarity index 94% rename from model-optimizer/extensions/front/caffe/detection_output.py rename to model-optimizer/extensions/front/caffe/detection_output_ext.py index 1371fe1ffbd..48559c1411d 100644 --- a/model-optimizer/extensions/front/caffe/detection_output.py +++ b/model-optimizer/extensions/front/caffe/detection_output_ext.py @@ -5,7 +5,6 @@ import logging as log from extensions.ops.DetectionOutput import DetectionOutput from mo.front.caffe.collect_attributes import merge_attrs -from mo.front.common.partial_infer.multi_box_detection import multi_box_detection_infer from mo.front.extractor import FrontExtractorOp @@ -85,7 +84,6 @@ class DetectionOutputFrontExtractor(FrontExtractorOp): interp_mode += interp_mode_values[x] attrs = { - 'num_classes': param.num_classes, 'share_location': int(param.share_location), 'background_label_id': param.background_label_id, 'code_type': code_type, @@ -132,9 +130,6 @@ class DetectionOutputFrontExtractor(FrontExtractorOp): mapping_rule = merge_attrs(param, attrs) - # force setting infer function because it doesn't exist in proto so merge_attrs will not set it - mapping_rule.update({'infer': multi_box_detection_infer}) - # update the attributes of the node DetectionOutput.update_node_stat(node, mapping_rule) return cls.enabled diff --git a/model-optimizer/extensions/front/mxnet/multibox_detection_ext.py b/model-optimizer/extensions/front/mxnet/multibox_detection_ext.py index 05a6dd49166..8961c5d2ea8 100644 --- a/model-optimizer/extensions/front/mxnet/multibox_detection_ext.py +++ b/model-optimizer/extensions/front/mxnet/multibox_detection_ext.py @@ -13,30 +13,20 @@ class MultiBoxDetectionOutputExtractor(FrontExtractorOp): @classmethod def extract(cls, node): attrs = get_mxnet_layer_attrs(node.symbol_dict) - # We can not get num_classes attribute from the operation, so it must be set to None. - # In this case num_classes attribute will be defined in the infer function in - # mo/front/common/partial_infer/multi_box_detection.py - num_classes = None top_k = attrs.int("nms_topk", -1) - keep_top_k = top_k - variance_encoded_in_target = 0 - code_type = "caffe.PriorBoxParameter.CENTER_SIZE" - share_location = 1 nms_threshold = attrs.float("nms_threshold", 0.5) confidence_threshold = attrs.float("threshold", 0.01) - background_label_id = 0 clip = 0 if not attrs.bool("clip", True) else 1 node_attrs = { 'type': 'DetectionOutput', 'op': __class__.op, - 'num_classes': num_classes, - 'keep_top_k': keep_top_k, - 'variance_encoded_in_target': variance_encoded_in_target, - 'code_type': code_type, - 'share_location': share_location, + 'keep_top_k': top_k, + 'variance_encoded_in_target': 0, + 'code_type': "caffe.PriorBoxParameter.CENTER_SIZE", + 'share_location': 1, 'confidence_threshold': confidence_threshold, - 'background_label_id': background_label_id, + 'background_label_id': 0, 'nms_threshold': nms_threshold, 'top_k': top_k, 'decrease_label_id': 1, diff --git a/model-optimizer/extensions/front/onnx/detection_output.py b/model-optimizer/extensions/front/onnx/detection_output_ext.py similarity index 100% rename from model-optimizer/extensions/front/onnx/detection_output.py rename to model-optimizer/extensions/front/onnx/detection_output_ext.py diff --git a/model-optimizer/extensions/front/onnx/detectionoutput_ext.py b/model-optimizer/extensions/front/onnx/detection_output_onnx_ext.py similarity index 93% rename from model-optimizer/extensions/front/onnx/detectionoutput_ext.py rename to model-optimizer/extensions/front/onnx/detection_output_onnx_ext.py index b87865012e2..728bb62025a 100644 --- a/model-optimizer/extensions/front/onnx/detectionoutput_ext.py +++ b/model-optimizer/extensions/front/onnx/detection_output_onnx_ext.py @@ -5,7 +5,7 @@ from math import log import numpy as np -from extensions.ops.detectionoutput_onnx import ExperimentalDetectronDetectionOutput +from extensions.ops.detection_output_onnx import ExperimentalDetectronDetectionOutput from mo.front.extractor import FrontExtractorOp from mo.front.onnx.extractors.utils import onnx_attr diff --git a/model-optimizer/extensions/front/onnx/mask_rcnn_conversion.py b/model-optimizer/extensions/front/onnx/mask_rcnn_conversion.py index ce11bb82fde..22c7128f09f 100644 --- a/model-optimizer/extensions/front/onnx/mask_rcnn_conversion.py +++ b/model-optimizer/extensions/front/onnx/mask_rcnn_conversion.py @@ -5,7 +5,7 @@ import numpy as np from extensions.front.onnx.softmaxONNX_to_softmax import SoftmaxONNXFrontReplacer from extensions.ops.Cast import Cast -from extensions.ops.detectionoutput_onnx import ExperimentalDetectronDetectionOutput +from extensions.ops.detection_output_onnx import ExperimentalDetectronDetectionOutput from extensions.ops.parameter import Parameter from extensions.ops.roifeatureextractor_onnx import ExperimentalDetectronROIFeatureExtractor from mo.front.common.partial_infer.utils import int64_array diff --git a/model-optimizer/extensions/front/tf/AutomlEfficientDet.py b/model-optimizer/extensions/front/tf/AutomlEfficientDet.py index 2ac2a166b66..1a75a868c10 100644 --- a/model-optimizer/extensions/front/tf/AutomlEfficientDet.py +++ b/model-optimizer/extensions/front/tf/AutomlEfficientDet.py @@ -115,7 +115,6 @@ class EfficientDet(FrontReplacementFromConfigFileGeneral): detection_output_node = DetectionOutput(graph, dict( name='detections', - num_classes=int(replacement_descriptions['num_classes']), share_location=1, background_label_id=int(replacement_descriptions['num_classes']) + 1, nms_threshold=replacement_descriptions['nms_threshold'], diff --git a/model-optimizer/extensions/front/tf/ObjectDetectionAPI.py b/model-optimizer/extensions/front/tf/ObjectDetectionAPI.py index 6c8937946f7..0fa2a54397a 100644 --- a/model-optimizer/extensions/front/tf/ObjectDetectionAPI.py +++ b/model-optimizer/extensions/front/tf/ObjectDetectionAPI.py @@ -1067,7 +1067,6 @@ class ObjectDetectionAPIDetectionOutputReplacement(FrontReplacementFromConfigFil code_type='caffe.PriorBoxParameter.CENTER_SIZE', pad_mode='caffe.ResizeParameter.CONSTANT', resize_mode='caffe.ResizeParameter.WARP', - num_classes=num_classes + 1, confidence_threshold=_value_or_raise(match, pipeline_config, 'postprocessing_score_threshold'), top_k=_value_or_raise(match, pipeline_config, 'postprocessing_max_detections_per_class'), keep_top_k=_value_or_raise(match, pipeline_config, 'postprocessing_max_total_detections'), @@ -1436,7 +1435,6 @@ class ObjectDetectionAPIProposalReplacement(FrontReplacementFromConfigFileSubGra 'input_width': 1, 'keep_top_k': max_proposals, 'normalized': True, - 'num_classes': 2, 'objectness_score': 0, 'share_location': True, 'top_k': 6000, @@ -1664,7 +1662,6 @@ class ObjectDetectionAPISSDPostprocessorReplacement(FrontReplacementFromConfigFi [reshape_offsets, reshape_conf_node, priors_node], dict(name=detection_output_op.attrs['type'], background_label_id=0 if has_background_class else -1, - num_classes=num_classes, variances_encoded_in_target=False, confidence_threshold=_value_or_raise(match, pipeline_config, 'postprocessing_score_threshold'), top_k=_value_or_raise(match, pipeline_config, 'postprocessing_max_detections_per_class'), diff --git a/model-optimizer/extensions/ops/DetectionOutput.py b/model-optimizer/extensions/ops/DetectionOutput.py index 55bf0bdcebd..a1ad570f4d8 100644 --- a/model-optimizer/extensions/ops/DetectionOutput.py +++ b/model-optimizer/extensions/ops/DetectionOutput.py @@ -3,10 +3,12 @@ import numpy as np -from mo.front.common.partial_infer.multi_box_detection import multi_box_detection_infer +from mo.front.common.partial_infer.utils import is_fully_defined, compatible_dims from mo.front.extractor import bool_to_str -from mo.graph.graph import Graph, Node +from mo.graph.graph import Graph +from mo.graph.graph import Node from mo.ops.op import Op +from mo.utils.error import Error class DetectionOutput(Op): @@ -17,10 +19,10 @@ class DetectionOutput(Op): super().__init__(graph, { 'type': self.op, 'op': self.op, - 'version': 'opset1', + 'version': 'opset8', 'in_ports_count': 3, 'out_ports_count': 1, - 'infer': multi_box_detection_infer, + 'infer': self.infer, 'input_width': 1, 'input_height': 1, 'normalized': True, @@ -33,7 +35,7 @@ class DetectionOutput(Op): }, attrs) def supported_attrs(self): - return [ + supported_attrs = [ 'background_label_id', ('clip_after_nms', lambda node: bool_to_str(node, 'clip_after_nms')), ('clip_before_nms', lambda node: bool_to_str(node, 'clip_before_nms')), @@ -45,13 +47,60 @@ class DetectionOutput(Op): 'keep_top_k', 'nms_threshold', ('normalized', lambda node: bool_to_str(node, 'normalized')), - 'num_classes', ('share_location', lambda node: bool_to_str(node, 'share_location')), 'top_k', ('variance_encoded_in_target', lambda node: bool_to_str(node, 'variance_encoded_in_target')), 'objectness_score', ] + opset = self.get_opset() + if opset == 'opset1': + supported_attrs += ['num_classes'] + return supported_attrs @staticmethod def type_infer(node: Node): node.out_port(0).set_data_type(np.float32) + + @staticmethod + def infer(node: Node): + node_name = node.soft_get('name', node.id) + loc_shape = node.in_port(0).data.get_shape() + conf_shape = node.in_port(1).data.get_shape() + prior_boxes_shape = node.in_port(2).data.get_shape() + + if loc_shape is None or conf_shape is None or prior_boxes_shape is None: + raise Error('Shapes for the Detection Output node "{}" are not defined'.format(node_name)) + + prior_size = 4 + if node.has('normalized') and not node.normalized: + prior_size = 5 + + if is_fully_defined(prior_boxes_shape[-1]) and prior_boxes_shape[-1] % prior_size != 0: + raise Error('Amount of confidences "{}" is not divisible by {} for node "{}"' + ''.format(prior_boxes_shape[-1], prior_size, node_name)) + + num_priors = prior_boxes_shape[-1] // prior_size + if not node.has_valid('keep_top_k') or node.keep_top_k == -1: + node['keep_top_k'] = num_priors + + num_classes = conf_shape[-1] // num_priors + num_loc_classes = num_classes + if node.has_and_set('share_location') and node.share_location: + num_loc_classes = 1 + + if not compatible_dims(num_priors * num_loc_classes * 4, loc_shape[-1]): + raise Error('Locations and prior boxes shapes mismatch: "{}" vs "{}" for node "{}"' + ''.format(loc_shape, prior_boxes_shape, node_name)) + + if not node.variance_encoded_in_target and not compatible_dims(prior_boxes_shape[-2], 2): + raise Error('The "-2" dimension of the prior boxes must be 2 but it is "{}" for node "{}".' + ''.format(prior_boxes_shape[-2], node_name)) + + if is_fully_defined(conf_shape[-1]) and is_fully_defined(num_priors) and conf_shape[-1] % num_priors != 0: + raise Error('Amount of confidences "{}" is not divisible by amount of priors "{}" for node "{}".' + ''.format(conf_shape[-1], num_priors, node_name)) + + node.out_port(0).data.set_shape([1, 1, conf_shape[0] * node.keep_top_k, 7]) + + # the line below is needed for the TF framework so the MO will not change the layout + node.graph.node[node.out_node(0).id]['nchw_layout'] = True diff --git a/model-optimizer/extensions/ops/detectionoutput_onnx.py b/model-optimizer/extensions/ops/detection_output_onnx.py similarity index 100% rename from model-optimizer/extensions/ops/detectionoutput_onnx.py rename to model-optimizer/extensions/ops/detection_output_onnx.py diff --git a/model-optimizer/mo/back/preprocessing.py b/model-optimizer/mo/back/preprocessing.py new file mode 100644 index 00000000000..861da882169 --- /dev/null +++ b/model-optimizer/mo/back/preprocessing.py @@ -0,0 +1,401 @@ +# Copyright (C) 2018-2021 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import argparse +import logging as log + +from mo.utils.error import Error +from mo.utils.utils import refer_to_faq_msg + +import numpy as np + +from openvino.preprocess import PrePostProcessor # pylint: disable=no-name-in-module,import-error +# pylint: disable=no-name-in-module,import-error +from openvino.runtime import Function, Layout, PartialShape, layout_helpers + + +def update_mean_scale_to_dict(input_nodes: list, mean_scale_val, scale): + """ + Internal function. Updates mean/scale values from array to dictionary + :param: input_nodes Inputs of model + :param: mean_scale_val Parsed 'mean_scale_val' object from command line arguments + :param: scale Global scale factor for all inputs from --scale command line arguments + """ + if not isinstance(mean_scale_val, dict): + if len(mean_scale_val) != len(input_nodes): + raise Error('Numbers of inputs and mean/scale values do not match. ' + refer_to_faq_msg(61)) + + data = np.copy(mean_scale_val) + mean_scale_val = {} + for idx, node in enumerate(input_nodes): + names_list = list(node.get_tensor().get_names()) + if not names_list: + continue + node_name = names_list[0] + mean_scale_val.update( + { + node_name: { + 'mean': data[idx][0], + 'scale': data[idx][1] + } + } + ) + + if scale: + for node in input_nodes: + names_list = list(node.get_tensor().get_names()) + if not names_list: + continue + node_name = names_list[0] + old_val = mean_scale_val[node_name] if node_name in mean_scale_val else None + mean_scale_val.update( + { + node_name: { + 'mean': old_val['mean'] if old_val and 'mean' in old_val else None, + 'scale': scale + } + } + ) + return mean_scale_val + + +def check_keys_valid(ov_function: Function, keys: list, search_outputs: bool): + """ + Internal function: checks if keys from cmd line arguments correspond to ov_function's inputs/outputs + Throws if some key is not found + Throws if some different keys point to the same actual input/output + """ + nodes_used = {} + nodes = ov_function.inputs + if search_outputs: + nodes += ov_function.outputs + + for name in keys: + node_found = False + for ov_node in nodes: + if name in ov_node.get_tensor().get_names(): + if ov_node in nodes_used: + raise Error('Key for {} and {} point to same model input/output.' + .format(name, nodes_used[ov_node])) + nodes_used[ov_node] = name + node_found = True + break + + if not node_found: + if not search_outputs: + raise Error('Input with name {} wasn\'t found! {}'.format(name, refer_to_faq_msg(83))) + else: + raise Error('Input/Output with name {} wasn\'t found! {}'.format(name, refer_to_faq_msg(83))) + + +def update_layout_is_input_flag(ov_function: Function, layout_values: dict): + """ + Internal function: updates layout_values with flag whether each layout belongs to input or to output + """ + for name, layout_value in layout_values.items(): + layout_value['is_input'] = False + for ov_input in ov_function.inputs: + if name in ov_input.get_tensor().get_names(): + layout_value['is_input'] = True + break + return layout_values + + +def find_channels_dimension(shape: PartialShape, num_channels: int, name: str, layout_values): + """ + Internal function. Finds dimension index matching with expected channels number + Raises error if there is no candidates or number of candidates is > 1 + :param: shape Parameter's partial shape + :param: num_channels Number of channels to find in shape + :param: name Parameter's name, used for Error-handling purposes + :param: layout_values Existing source/target layout items specified by user + :return: updated layout items with guessed layouts + """ + if shape.rank.is_dynamic: + raise Error('Can\'t determine channels dimension for dynamic shape for parameter {}.' + .format(name)) + + dim_idx_found = -1 + for dim_idx in range(shape.rank.get_length()): + dim = shape.get_dimension(dim_idx) + if dim.is_static and dim.get_length() == num_channels: + if dim_idx_found >= 0: + raise Error('Can\'t determine channels dimension for {}. ' + 'Input shape is {}, needed channels {}. ' + 'Conflicting dimensions: {} and {}. Please specify layout manually.' + .format(name, shape, num_channels, dim_idx_found, dim_idx)) + dim_idx_found = dim_idx + if dim_idx_found < 0: + raise Error('Can\'t determine channels dimension for {}. ' + 'Input shape is {}, needed channels {}' + .format(name, shape, num_channels)) + + # Restrict guessed channels index to particular position depending on tensor shape(3d, 4d, 5d) + if shape.rank.get_length() == 3: + # CHW or HWC, possible channels index is 0 or 2 + if dim_idx_found != 0 and dim_idx_found != 2: + raise Error('Can\'t determine channels dimension for 3D input {} (CHW or HWC) with shape {}. ' + 'Please specify layout containing \'C\' channels manually.'.format(name, shape)) + elif shape.rank.get_length() == 4: + # NCHW or NHWC, possible channels index is 1 or 3 + if dim_idx_found != 1 and dim_idx_found != 3: + raise Error('Can\'t determine channels dimension for 4D input {} (NCHW or NHWC) with shape {}. ' + 'Please specify layout containing \'C\' channels manually.'.format(name, shape)) + elif shape.rank.get_length() == 5: + # NCDHW or NDHWC, possible channels index is 1 or 4 + if dim_idx_found != 1 and dim_idx_found != 4: + raise Error('Can\'t determine channels dimension for 5D input {} (NCDHW or NDHWC) with shape {}. ' + 'Please specify layout containing \'C\' channels manually.'.format(name, shape)) + else: + raise Error('Can\'t determine channels dimension for {}D input {} with shape {}.' + 'Please specify layout containing \'C\' channels manually.' + .format(shape.rank.get_length(), name, shape)) + + layout_str = "?" * shape.rank.get_length() + layout_str = layout_str[:dim_idx_found] + 'C' + layout_str[dim_idx_found+1:] + layout_values[name] = { + 'source_layout': layout_str, + 'target_layout': None, + 'source_guessed': True, + 'is_input': True + } + return layout_values + + +def guess_source_layouts_by_mean_scale(ov_function: Function, layout_values, mean_scale_values: dict): + """ + Internal function. Try to guess source layout for input by its shape and/or framework + :param: ov_function Original model + :param: layout_values Existing source/target layout items specified by user + :param: mean_scale_values Dictionary with mean/scale values defined for each argument + :return: updated layout items with guessed layouts + """ + for ms_name, mean_scale in mean_scale_values.items(): + num_channels_mean = len(mean_scale['mean']) if mean_scale['mean'] is not None else 0 + num_channels_scale = len(mean_scale['scale']) if hasattr(mean_scale['scale'], '__len__') else 0 + + if num_channels_mean > 1 and \ + num_channels_scale > 1 and \ + num_channels_mean is not num_channels_scale: + raise Error('Mean/Scale values for {} have different sizes: {} {}' + .format(ms_name, num_channels_mean, num_channels_scale)) + + need_guess_channels = num_channels_mean > 1 or num_channels_scale > 1 + if not need_guess_channels: # Mean/scale is complex and needs 'channels' specified in layout + continue + + num_channels = num_channels_mean if num_channels_mean > 1 else num_channels_scale + + for i in range(0, len(ov_function.inputs)): + ov_input = ov_function.input(i) + + if not ov_function.get_parameters()[i].layout.empty: + continue + + if ms_name not in ov_input.get_tensor().get_names(): + continue + + layout_item = None + for name in ov_input.get_tensor().get_names(): + if name in layout_values: + layout_item = layout_values[name] + break + + if layout_item is not None: + # User specified some layout, skip guessing + continue + + # Guess layout is applicable only when number of channels is '3' + if num_channels != 3: + raise Error('Can\'t determine channels dimension for {}. ' + 'When number of mean/scale values is {} (not 3), ' + 'please specify layout for input manually'.format(ms_name, num_channels)) + + layout_values = find_channels_dimension(shape=ov_input.get_partial_shape(), + num_channels=num_channels, + name=ms_name, + layout_values=layout_values) + return layout_values + + +def check_suitable_for_reverse(layout: Layout, ov_input): + """ + Internal function. Checks if input with layout is suitable for reversing channels + :param: layout Existing source/target layout items specified by user + :param: ov_input Model's input + :return: True if reverse channels can be applied to input + """ + if not layout_helpers.has_channels(layout): + return False + if ov_input.get_partial_shape().rank.is_dynamic: + return False + + c_idx = layout_helpers.channels_idx(layout) + rank = ov_input.get_partial_shape().rank.get_length() + if c_idx < 0: + c_idx += rank + if c_idx >= rank: + raise Error('Layout {} for input {} is inconsistent with shape {}'.format( + layout, ov_input.get_tensor().get_any_name(), ov_input.get_partial_shape())) + c_num = ov_input.get_partial_shape()[c_idx] + return c_num.is_dynamic or c_num.get_length() == 3 + + +def guess_source_layouts_for_reverse_channels(ov_function: Function, layout_values): + """ + Internal function. Try to guess source layout for input by finding dimension with size=3 (RGB/BGR) + Additionally checks existing layouts and detects suitable inputs for reversing of input channels + :param: ov_function Original model + :param: layout_values Existing source/target layout items specified by user + :return: array with suitable parameters for reversing of input channels + """ + all_params = [] + suitable_params = [] + for i in range(0, len(ov_function.inputs)): + ov_input = ov_function.input(i) + param_info = [ov_input.get_tensor().get_any_name(), ov_input.get_partial_shape()] + all_params.append(param_info) + + if not ov_function.get_parameters()[i].layout.empty: + if check_suitable_for_reverse(ov_function.get_parameters()[i].layout, ov_input): + suitable_params.append(param_info) + continue + + layout_item = None + first_name = ov_input.get_tensor().get_any_name() + for name in ov_input.get_tensor().get_names(): + if name in layout_values: + layout_item = layout_values[name] + break + + if layout_item is not None: + if layout_item.get('target_layout'): + if check_suitable_for_reverse(Layout(layout_item['target_layout']), ov_input): + suitable_params.append(param_info) + elif layout_item.get('source_layout'): + if check_suitable_for_reverse(Layout(layout_item['source_layout']), ov_input): + suitable_params.append(param_info) + continue + + try: + layout_values = find_channels_dimension(shape=ov_input.get_partial_shape(), + num_channels=3, + name=first_name, + layout_values=layout_values) + except Error as e: + log.debug('Reverse input channels guess did not succeed {}'.format(e)) + else: + layout = layout_values[first_name].get('source_layout') + if layout and check_suitable_for_reverse(Layout(layout), ov_input): + suitable_params.append(param_info) + + if len(suitable_params) < len(all_params): + log.error('Network has {} inputs overall, but only {} of them are suitable for input channels reversing.\n' + 'Suitable for input channel reversing inputs are 4-dimensional with 3 channels\nAll inputs: {}\n' + 'Suitable inputs {}'.format(len(all_params), len(suitable_params), all_params, suitable_params), + extra={'is_warning': True}) + return suitable_params + + +def apply_preprocessing(ov_function: Function, argv: argparse.Namespace): + """ + Applies pre-processing of model inputs by adding appropriate operations + On return, 'ov_function' object will be updated + Expected 'argv.mean_scale_values' formats examples: + a) Dict: {'inputName': {'mean': [1., 2., 3.], 'scale': [2., 4., 8.]}} + b) List: list(np.array([(np.array([1., 2., 3.]), np.array([2., 4., 6.])), + (np.array([7., 8., 9.]), np.array([5., 6., 7.]))) + Expected 'argv.layout_values' format examples: + a) Specific layouts for inputs and outputs + { 'input1': { + 'source_layout': 'nchw', + 'target_layout': 'nhwc' + }, + 'output2': { + 'source_layout': 'nhwc' + } + } + b) Layout for single input: {'': {'source_layout': 'nchw'}} + :param: ov_function OV function for applying mean/scale pre-processing + :param: argv Parsed command line arguments + """ + prep = PrePostProcessor(ov_function) + + if 'mean_scale_values' in argv and argv.mean_scale_values: + mean_scale_values = argv.mean_scale_values + else: + mean_scale_values = {} + + mean_scale_values = update_mean_scale_to_dict(input_nodes=ov_function.inputs, + mean_scale_val=mean_scale_values, + scale=argv.scale) + # On return, mean_scale_values is a dictionary with input names as key and mean/scale pair as value + # {'inputName': {'mean': [1., 2., 3.], 'scale': [2.]}} + + layout_values = {} + if 'layout_values' in argv and argv.layout_values: + layout_values = argv.layout_values + + if '' in layout_values: + if len(ov_function.inputs) > 1: + input_names = [list(ov_input.get_tensor().get_names())[0] for ov_input in ov_function.inputs] + raise Error('Layout without name can be specified for models with only one input, ' + 'but provided model has {} inputs: \'{}\'. ' + 'Please specify explicitly input/output name for --layout option' + .format(len(input_names), input_names)) + layout_values = { + list(ov_function.input().get_tensor().get_names())[0]: { + 'source_layout': layout_values[''].get('source_layout'), + 'target_layout': layout_values[''].get('target_layout') + } + } + check_keys_valid(ov_function=ov_function, keys=mean_scale_values.keys(), search_outputs=False) + check_keys_valid(ov_function=ov_function, keys=layout_values.keys(), search_outputs=True) + + layout_values = update_layout_is_input_flag(ov_function, layout_values) + layout_values = guess_source_layouts_by_mean_scale(ov_function, layout_values, mean_scale_values) + need_reverse = 'reverse_input_channels' in argv and argv.reverse_input_channels + suitable_params_ric = [] + if need_reverse: + suitable_params_ric = guess_source_layouts_for_reverse_channels(ov_function=ov_function, + layout_values=layout_values) + + for node_name, layout_value in layout_values.items(): + if layout_value.get('source_layout'): + if layout_value.get('is_input'): + prep.input(node_name).network().set_layout(Layout(layout_value['source_layout'])) + else: + prep.output(node_name).network().set_layout(Layout(layout_value['source_layout'])) + if layout_value.get('target_layout'): + if layout_value.get('is_input'): + prep.input(node_name).tensor().set_layout(Layout(layout_value['target_layout'])) + else: + prep.output(node_name).tensor().set_layout(Layout(layout_value['target_layout'])) + + for node_name, node_mean_scale_values in mean_scale_values.items(): + # Apply mean first, then scale + if node_mean_scale_values['mean'] is not None: + prep.input(node_name).preprocess().mean(node_mean_scale_values['mean']) + if node_mean_scale_values['scale'] is not None: + prep.input(node_name).preprocess().scale(node_mean_scale_values['scale']) + log.debug('Mean/Scale pre-processing applied to {}'.format(node_name)) + + # Apply reverse_input_channels + if need_reverse: + for name, _ in suitable_params_ric: + prep.input(name).preprocess().reverse_channels() + log.debug('reverse_input_channels pre-processing applied to {}'.format(name)) + + # Apply pre-processing builder to a function + ov_function = prep.build() + + # Remove guessed layout values from ov_function (these values shall not be serialized to IR + for node_name, layout_value in layout_values.items(): + if layout_value.get('source_guessed') and \ + not layout_value.get('target_layout'): + # search for parameter object + for idx, ov_input in enumerate(ov_function.inputs): + if node_name in ov_input.get_tensor().get_names(): + log.debug('Clearing guessed layout {} for {}' + .format(layout_value['source_layout'], node_name)) + ov_function.get_parameters()[idx].layout = Layout() diff --git a/model-optimizer/mo/front/common/partial_infer/multi_box_detection.py b/model-optimizer/mo/front/common/partial_infer/multi_box_detection.py deleted file mode 100644 index 67b6f5fdb09..00000000000 --- a/model-optimizer/mo/front/common/partial_infer/multi_box_detection.py +++ /dev/null @@ -1,56 +0,0 @@ -# Copyright (C) 2018-2021 Intel Corporation -# SPDX-License-Identifier: Apache-2.0 - -import logging as log - -from mo.front.common.partial_infer.utils import is_fully_defined, compatible_dims -from mo.graph.graph import Node -from mo.utils.error import Error - - -def multi_box_detection_infer(node: Node): - loc_shape = node.in_node(0).shape - conf_shape = node.in_node(1).shape - prior_boxes_shape = node.in_node(2).shape - node_name = node.soft_get('name', node.id) - - if loc_shape is None or conf_shape is None or prior_boxes_shape is None: - raise Error('Shapes for the Detection Output node "{}" are not defined'.format(node_name)) - - prior_size = 4 - if node.has('normalized') and not node.normalized: - prior_size = 5 - - if is_fully_defined(prior_boxes_shape[-1]) and prior_boxes_shape[-1] % prior_size != 0: - raise Error('Amount of confidences "{}" is not divisible by {} for node "{}"' - ''.format(prior_boxes_shape[-1], prior_size, node_name)) - - num_priors = prior_boxes_shape[-1] // prior_size - if not node.has_valid('keep_top_k') or node.keep_top_k == -1: - node['keep_top_k'] = num_priors - - # do not try to infer number of classes because it is not possible in case when input shapes are partially defined - if not node.has_valid('num_classes'): - node['num_classes'] = conf_shape[-1] // num_priors - log.debug('Inferred amount of classes "{}"'.format(node.num_classes)) - - num_loc_classes = node.num_classes - if node.has_and_set('share_location') and node.share_location: - num_loc_classes = 1 - - if not compatible_dims(num_priors * num_loc_classes * 4, loc_shape[-1]): - raise Error('Locations and prior boxes shapes mismatch: "{}" vs "{}" for node "{}"' - ''.format(loc_shape, prior_boxes_shape, node_name)) - - if not node.variance_encoded_in_target and not compatible_dims(prior_boxes_shape[-2], 2): - raise Error('The "-2" dimension of the prior boxes must be 2 but it is "{}" for node "{}".' - ''.format(prior_boxes_shape[-2], node_name)) - - if is_fully_defined(conf_shape[-1]) and is_fully_defined(num_priors) and conf_shape[-1] % num_priors != 0: - raise Error('Amount of confidences "{}" is not divisible by amount of priors "{}" for node "{}".' - ''.format(conf_shape[-1], num_priors, node_name)) - - node.out_port(0).data.set_shape([1, 1, conf_shape[0] * node.keep_top_k, 7]) - - # the line below is needed for the TF framework so the MO will not change the layout - node.graph.node[node.out_node(0).id]['nchw_layout'] = True diff --git a/model-optimizer/mo/moc_frontend/serialize.py b/model-optimizer/mo/moc_frontend/serialize.py index 75104318e41..f2eedcd5dba 100644 --- a/model-optimizer/mo/moc_frontend/serialize.py +++ b/model-optimizer/mo/moc_frontend/serialize.py @@ -5,6 +5,7 @@ import argparse import os from mo.pipeline.common import get_ir_version from mo.back.ie_ir_ver_2.emitter import append_ir_info +from mo.back.preprocessing import apply_preprocessing from mo.utils.cli_parser import get_meta_info, parse_transform from openvino.runtime import Function # pylint: disable=no-name-in-module,import-error @@ -13,6 +14,10 @@ from openvino.runtime import Function # pylint: disable=no-name-in-modul def moc_emit_ir(ngraph_function: Function, argv: argparse.Namespace): output_dir = argv.output_dir if argv.output_dir != '.' else os.getcwd() + # Apply preprocessing (mean/scale/reverse_channels/convert_layout/etc) + apply_preprocessing(ov_function=ngraph_function, argv=argv) + + # Apply transformations from mo.back.offline_transformations import apply_user_transformations, apply_moc_transformations apply_user_transformations(ngraph_function, parse_transform(argv.transform)) apply_moc_transformations(ngraph_function) diff --git a/model-optimizer/unit_tests/extensions/front/mxnet/multibox_detection_test.py b/model-optimizer/unit_tests/extensions/front/mxnet/multibox_detection_test.py index c65feaeb055..a62893d900a 100644 --- a/model-optimizer/unit_tests/extensions/front/mxnet/multibox_detection_test.py +++ b/model-optimizer/unit_tests/extensions/front/mxnet/multibox_detection_test.py @@ -21,7 +21,6 @@ class TestMultiBoxDetection_Parsing(unittest.TestCase): exp_attrs = { 'type': 'DetectionOutput', - 'num_classes': None, 'keep_top_k': 400, 'variance_encoded_in_target': 0, 'code_type': "caffe.PriorBoxParameter.CENTER_SIZE", @@ -51,7 +50,6 @@ class TestMultiBoxDetection_Parsing(unittest.TestCase): exp_attrs = { 'type': 'DetectionOutput', - 'num_classes': None, 'keep_top_k': -1, 'variance_encoded_in_target': 0, 'code_type': "caffe.PriorBoxParameter.CENTER_SIZE", diff --git a/model-optimizer/unit_tests/extensions/front/onnx/detection_output_test.py b/model-optimizer/unit_tests/extensions/front/onnx/detection_output_test.py index 2a0fe9116d3..b106623c9f8 100644 --- a/model-optimizer/unit_tests/extensions/front/onnx/detection_output_test.py +++ b/model-optimizer/unit_tests/extensions/front/onnx/detection_output_test.py @@ -6,7 +6,7 @@ import unittest import numpy as np import onnx -from extensions.front.onnx.detection_output import DetectionOutputFrontExtractor +from extensions.front.onnx.detection_output_ext import DetectionOutputFrontExtractor from extensions.ops.DetectionOutput import DetectionOutput from mo.ops.op import Op from unit_tests.utils.extractors import PB diff --git a/model-optimizer/unit_tests/mo/back/moc_preprocessing_test_actual.py b/model-optimizer/unit_tests/mo/back/moc_preprocessing_test_actual.py new file mode 100644 index 00000000000..6c22c95537c --- /dev/null +++ b/model-optimizer/unit_tests/mo/back/moc_preprocessing_test_actual.py @@ -0,0 +1,617 @@ +# Copyright (C) 2018-2021 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import unittest +from argparse import Namespace + +from mo.utils.error import Error + +import numpy as np + + +try: + # pylint: disable=no-name-in-module,import-error + from mo.back.preprocessing import apply_preprocessing + + # pylint: disable=no-name-in-module,import-error + import openvino.runtime.opset8 as ops + from openvino.runtime import Function, Layout, PartialShape + +except Exception: + print("No OpenVINO API available," + "ensure to set correct PYTHONPATH when running these tests") + raise + + +def create_function2(shape1=[2, 2], shape2=[2, 2], dtype1=np.float32, dtype2=np.float32): + input1 = ops.parameter(shape1, dtype=dtype1, name="input1") + input1.get_output_tensor(0).set_names({'input1', 'input1a'}) + relu1 = ops.relu(input1) + res1 = ops.result(relu1, "res1") + res1.get_output_tensor(0).set_names({'res1', 'res1a'}) + input2 = ops.parameter(shape2, dtype=dtype2, name="input2") + input2.get_output_tensor(0).set_names({'input2', 'input2a'}) + relu2 = ops.relu(input2) + res2 = ops.result(relu2, "res2") + res2.get_output_tensor(0).set_names({'res2', 'res2a'}) + function = Function(results=[res1, res2], parameters=[input1, input2], name="TestFunction") + return function + + +def create_function1(shape1=[2, 2]): + input1 = ops.parameter(shape1, dtype=np.float32, name="input1") + input1.get_output_tensor(0).set_names({'input1', 'input1a'}) + relu1 = ops.relu(input1) + res1 = ops.result(relu1, "res1") + res1.get_output_tensor(0).set_names({'res1', 'res1a'}) + function = Function(results=[res1], parameters=[input1], name="TestFunction") + return function + + +def process_function(ov_function: Function, argv: Namespace): + apply_preprocessing(ov_function=ov_function, argv=argv) + + +class TestPreprocessingMOC(unittest.TestCase): + def setUp(self): + pass + + def check_scale_constant(self, node, expected, shape=None): + const_node = node.input(1).get_source_output().get_node() + self.assertEqual(const_node.get_type_name(), 'Constant') + if node.get_type_name() == 'Divide': + self.assertTrue(np.allclose(const_node.get_vector(), expected)) + else: + self.assertTrue(np.allclose(const_node.get_vector(), 1. / expected)) + if shape: + assert const_node.shape == PartialShape(shape) + + def check_mean_constant(self, node, expected, shape=None): + const_node = node.input(1).get_source_output().get_node() + self.assertEqual(const_node.get_type_name(), 'Constant') + if node.get_type_name() == 'Subtract': + self.assertTrue(np.allclose(const_node.get_vector(), expected)) + else: + self.assertTrue(np.allclose(const_node.get_vector(), -expected.toList())) + if shape: + self.assertEqual(const_node.shape, PartialShape(shape)) + + def test_scale_single_value(self): + argv = Namespace(mean_scale_values=None, scale=2.0) + function = create_function2() + process_function(ov_function=function, argv=argv) + + for param in function.get_parameters(): + op_node = list(param.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, [2.0]) + + def test_scale_single_value_fp64(self): + argv = Namespace(mean_scale_values=None, scale=2.0) + function = create_function2(dtype1=np.float64) + process_function(ov_function=function, argv=argv) + + for ov_input in function.inputs: + op_node = list(ov_input.get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, [2.0]) + + def test_scale_single_value_fp16(self): + argv = Namespace(mean_scale_values=None, scale=2.0) + function = create_function2(dtype1=np.float16) + process_function(ov_function=function, argv=argv) + + for ov_input in function.inputs: + op_node = list(ov_input.get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + + def test_scale_vector(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([4.]), 'mean': None}}, scale=None) + function = create_function2() + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, [4.0], shape=None) + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + def test_scale_vector3(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([2., 4., 8.]), 'mean': None}}, scale=None) + function = create_function2(shape1=[1, 3, 224, 224]) + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8.], shape=[1, 3, 1, 1]) + + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that guessed layout (?C??) is not appeared in input1 + self.assertEqual(function.get_parameters()[0].layout, Layout()) + + def test_scale_vector4_layout(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([2., 4., 8., 9.]), 'mean': None}}, + layout_values={'input1': {'source_layout': 'nhwc'}}, + scale=None) + function = create_function2(shape1=[1, 3, 3, 4]) # Use layout to determine channels dim + + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8., 9.], shape=[1, 1, 1, 4]) + + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that layout (NHWC) is appeared in input1 + self.assertEqual(function.get_parameters()[0].layout, Layout('nhwc')) + + def test_mean_single(self): + argv = Namespace(mean_scale_values={'input1': {'mean': np.array([4.]), 'scale': None}}, scale=None) + function = create_function2() + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, [4.0], shape=None) + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + def test_mean_single_fp64(self): + argv = Namespace(mean_scale_values={'input1': {'mean': np.array([4.]), 'scale': None}}, scale=None) + function = create_function2(dtype1=np.float64) + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, [4.0], shape=None) + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + def test_mean_single_fp16(self): + argv = Namespace(mean_scale_values={'input1': {'mean': np.array([4.]), 'scale': None}}, scale=None) + function = create_function2(dtype1=np.float16) + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + # Verify that input2 is not affected + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + def test_mean_vector3(self): + argv = Namespace(mean_scale_values={'input2': {'mean': np.array([2., 4., 8.]), 'scale': None}}, scale=None) + function = create_function2(shape2=[1, 3, 224, 224]) + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[2., 4., 8.], shape=[1, 3, 1, 1]) + + # Verify that input1 is not affected + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that guessed layout (?C??) is not appeared in input2 + self.assertEqual(function.get_parameters()[1].layout, Layout()) + + def test_mean_scale(self): + argv = Namespace(mean_scale_values={'input2a': {'mean': np.array([1., 2., 3.]), + 'scale': np.array([2., 4., 8.])}}, + scale=None) + function = create_function2(shape2=[1, 3, 224, 224]) + process_function(ov_function=function, argv=argv) + # Verify that first is 'subtract mean', then 'scale' + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[1., 2., 3.], shape=[1, 3, 1, 1]) + + op_node = list(op_node.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8.], shape=[1, 3, 1, 1]) + + # Verify that input1 is not affected + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that guessed layout (?C??) is not appeared in input2 + self.assertEqual(function.get_parameters()[1].layout, Layout()) + + def test_mean_scale_with_layout(self): + argv = Namespace(mean_scale_values={'input2a': {'mean': np.array([1., 2., 3., 4.]), + 'scale': np.array([2., 4., 8., 9.])}}, + scale=None) + function = create_function2(shape2=[1, 3, 3, 4]) + function.get_parameters()[1].layout = Layout("NHWC") + process_function(ov_function=function, argv=argv) + # Verify that first is 'subtract mean', then 'scale' + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[1., 2., 3., 4.], shape=[1, 1, 1, 4]) + + op_node = list(op_node.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8., 9.], shape=[1, 1, 1, 4]) + + # Verify that input1 is not affected + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that layout presents in function after preprocessing + self.assertEqual(function.get_parameters()[1].layout, Layout("NHWC")) + + def test_mean_scale_with_layout_dynamic(self): + argv = Namespace(mean_scale_values={'input2a': {'mean': np.array([1., 2., 3., 4.]), + 'scale': np.array([2., 4., 8., 9.])}}, + scale=None) + function = create_function2(shape2=[-1, -1, -1, -1]) + function.get_parameters()[1].layout = Layout("NHWC") + process_function(ov_function=function, argv=argv) + # Verify that first is 'subtract mean', then 'scale' + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[1., 2., 3., 4.], shape=[1, 1, 1, 4]) + + op_node = list(op_node.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8., 9.], shape=[1, 1, 1, 4]) + + # Verify that input1 is not affected + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertEqual(op_node.get_type_name(), 'Relu') + + # Verify that layout presents in function after preprocessing + self.assertEqual(function.get_parameters()[1].layout, Layout("NHWC")) + + def test_no_param_name(self): + argv = Namespace(mean_scale_values=list(np.array([(np.array([1., 2., 3.]), np.array([2., 4., 6.])), + (np.array([7., 8., 9.]), None)], + dtype='object')), scale=None) + function = create_function2(shape1=[1, 3, 224, 224], shape2=[1, 224, 224, 3]) + process_function(ov_function=function, argv=argv) + + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[1., 2., 3.], shape=[1, 3, 1, 1]) + + op_node = list(op_node.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 6.], shape=[1, 3, 1, 1]) + + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[7., 8., 9.], shape=[1, 1, 1, 3]) + + # Verify that guessed layouts are not appeared in inputs + self.assertEqual(function.get_parameters()[0].layout, Layout()) + self.assertEqual(function.get_parameters()[1].layout, Layout()) + + def test_no_param_name_single_value(self): + argv = Namespace(mean_scale_values=list(np.array([(np.array([1.]), None), + (np.array([2., 3., 4.]), np.array([5.]))], + dtype='object')), scale=None) + function = create_function2(shape1=[1, 3, 224, 224], shape2=[1, 224, 224, 3]) + process_function(ov_function=function, argv=argv) + + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[1.], shape=None) + + op_node = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Subtract' or op_node.get_type_name() == 'Add') + self.check_mean_constant(op_node, expected=[2., 3., 4.], shape=[1, 1, 1, 3]) + + op_node = list(op_node.output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[5.], shape=None) + + # Two inputs, but 'mean_scale_value' has only one array + def test_error_no_param_name_number_not_match(self): + argv = Namespace(mean_scale_values=[(np.array([2., 3.]), np.array([4.]))], scale=None) + function = create_function2(shape1=[1, 3, 224, 224], shape2=[1, 2, 224, 224]) + with self.assertRaisesRegex(Error, '.*question.*61.*'): + process_function(ov_function=function, argv=argv) + + def test_mean_scale_error_no_node_name_found(self): + argv = Namespace(mean_scale_values={'not_found': {'scale': np.array([1.]), 'mean': np.array([1.])}}, + scale=None) + function = create_function2(shape1=[1, 3, 224, 224], shape2=[1, 2, 224, 224]) + with self.assertRaisesRegex(Error, '.*question.*83.*'): + process_function(ov_function=function, argv=argv) + + def test_layout_error_no_node_name_found(self): + argv = Namespace(layout_values={'not_found': {'source_layout': 'nhwc'}}, + scale=None) + function = create_function2(shape1=[1, 3, 224, 224], shape2=[1, 2, 224, 224]) + with self.assertRaisesRegex(Error, '.*question.*83.*'): + process_function(ov_function=function, argv=argv) + + def test_error_dimension_mismatch(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3., 4.]), 'mean': None}}, + scale=None) + function = create_function2(shape1=[1, 3, 224, 224]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_dimension_not_clear(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.]), 'mean': None}}, + scale=None) + function = create_function2(shape1=[1, 3, 3, 3]) # Not clear to which 3 should scale be applied + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_dimension_mismatch_with_scale(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3., 4.]), + 'mean': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[1, 3, 4, 224]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_guess_c_wrong_position_3d(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.]), + 'mean': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[2, 3, 4]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_guess_c_wrong_position_4d(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.]), + 'mean': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[1, 2, 3, 4]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_guess_c_wrong_position_5d(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.]), + 'mean': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[1, 2, 3, 4, 5]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_guess_c_wrong_position_6d(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.]), + 'mean': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[1, 2, 4, 5, 6, 3]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_2_names_to_same_input(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([1., 2., 3.])}, + 'input1a': {'scale': np.array([1., 2., 3.])}}, + scale=None) + function = create_function2(shape1=[1, 3, 224, 224]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_error_2_names_to_same_input_single_value(self): + argv = Namespace(mean_scale_values={'input1': {'scale': np.array([2.])}, + 'input1a': {'scale': np.array([3.])}}, + scale=None) + function = create_function2(shape1=[1, 3, 224, 224]) + with self.assertRaises(Exception): + process_function(ov_function=function, argv=argv) + + def test_reverse_input_channels(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 224, 224, 3], shape2=[1, 3, 224, 224]) + process_function(ov_function=function, + argv=argv) + # Verify that some operations are inserted. + # In future, consider using mock PrePostProcessor to verify that 'reverse_channels' was called + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() != 'Relu') + + # Verify that guessed layouts are not appeared in input1,input2 + self.assertEqual(function.get_parameters()[0].layout, Layout()) + self.assertEqual(function.get_parameters()[1].layout, Layout()) + + def test_reverse_input_channels_func_layout(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 3, 3, 3], shape2=[1, 3, 3, 3]) + function.get_parameters()[0].layout = Layout("NCHW") + function.get_parameters()[1].layout = Layout("NHWC") + process_function(ov_function=function, + argv=argv) + # Verify that some operations are inserted. + # In future, consider using mock PrePostProcessor to verify that 'reverse_channels' was called + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() != 'Relu') + + # Verify that guessed layouts are not appeared in input1,input2 + self.assertEqual(function.get_parameters()[0].layout, Layout("NCHW")) + self.assertEqual(function.get_parameters()[1].layout, Layout("NHWC")) + + def test_reverse_input_channels_layout(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None, + layout_values={'input1a': { 'source_layout': 'nhwc' }, + 'input2a': { 'source_layout': 'nchw' } + }) + function = create_function2(shape1=[1, 224, 224, 4], shape2=[1, 4, 224, 224]) + process_function(ov_function=function, argv=argv) + # In future, consider using mock PrePostProcessor to verify that 'reverse_channels' was not called + # Verify that reverse_channels are not applied. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_reverse_input_channels_3d(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None, + layout_values=None) + function = create_function2(shape1=[224, 224, 3], shape2=[3, 224, 224]) + process_function(ov_function=function, argv=argv) + # Verify that reverse_channels are applied. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() != 'Relu') + + def test_reverse_input_channels_6d(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None, + layout_values=None) + function = create_function2(shape1=[4, 4, 4, 4, 4, 3], shape2=[4, 3, 4, 4, 4, 4]) + process_function(ov_function=function, argv=argv) + # Verify that reverse_channels are NOT applied. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_reverse_input_channels_dynamic(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None, + layout_values=None) + function = create_function2(shape1=[1, -1, 5, 5], shape2=[-1, -1, -1, -1]) + process_function(ov_function=function, argv=argv) + # Verify that reverse_channels are NOT applied. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_reverse_input_channels_dynamic_layout(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None, + layout_values={'input1a': { 'source_layout': 'nchw' }, + 'input2a': { 'source_layout': 'nhwc' } + }) + function = create_function2(shape1=[1, -1, 5, 5], shape2=[-1, -1, -1, -1]) + process_function(ov_function=function, argv=argv) + # Verify that reverse_channels are applied. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() != 'Relu') + + def test_reverse_input_channels_2_channels(self): + argv = Namespace(reverse_input_channels=True, + mean_scale_values=None, + scale=None) + function = create_function2(shape1=[1, 224, 224, 2], shape2=[1, 3, 224, 224]) + process_function(ov_function=function, argv=argv) + # Verify that some operations are inserted to input2. + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() != 'Relu') + + # Verify that guessed layouts are not appeared in input1,input2 + self.assertEqual(function.get_parameters()[0].layout, Layout()) + self.assertEqual(function.get_parameters()[1].layout, Layout()) + + # When input name for layout is empty for model with one input - it is applied to this input + def test_scale_vector3_layout_empty_input_name(self): + argv = Namespace(mean_scale_values=list(np.array([(None, np.array([2., 4., 8.]))], + dtype='object')), + layout_values={'': {'source_layout': 'nchw'}}, + scale=None) + function = create_function1(shape1=[1, 3, 3, 3]) # Use layout to determine channels dim + + process_function(ov_function=function, argv=argv) + op_node = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node.get_type_name() == 'Divide' or op_node.get_type_name() == 'Multiply') + self.check_scale_constant(op_node, expected=[2., 4., 8.], shape=[1, 3, 1, 1]) + + # Verify that layout (nchw) is appeared in input1 + self.assertEqual(function.get_parameters()[0].layout, Layout('nchw')) + + def test_layout_output(self): + argv = Namespace(mean_scale_values=None, + layout_values={ + 'res1': { + 'source_layout': 'nchw', + 'target_layout': 'nhwc' + }, + 'res2a': { + 'source_layout': 'ncdhw' + } + }, + scale=None) + function = create_function2(shape1=[1, 3, 3, 3], shape2=[1, 3, 3, 3, 3]) + + process_function(ov_function=function, argv=argv) + op_node = function.get_results()[0].input(0).get_source_output().get_node() + self.assertEqual(op_node.get_type_name(), 'Transpose') + + self.assertEqual(function.get_results()[0].layout, Layout('nhwc')) + self.assertEqual(function.get_results()[1].layout, Layout('ncdhw')) + + def test_error_layout_empty_input_name_2_inputs(self): + argv = Namespace(mean_scale_values=None, + layout_values={'': {'source_layout': 'nchw'}}, + scale=None) + function = create_function2(shape1=[1, 3, 3, 3]) + + # Verify user friendly error message contains number of inputs and their names + with self.assertRaisesRegex(Error, '.*2.*inputs.*input1.*input2.*'): + process_function(ov_function=function, argv=argv) + + def test_reverse_channels_bad_layout(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 224, 224, 3], shape2=[1, 4, 224, 224]) + function.get_parameters()[0].layout = Layout("NDHWC") + with self.assertRaisesRegex(Error, '.*input1.*'): + process_function(ov_function=function, argv=argv) + + def test_guess_layout_reverse_channels_dont_apply_to_4(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 224, 224, 3], shape2=[1, 4, 224, 224]) + process_function(ov_function=function, argv=argv) + + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_error_guess_layout_reverse_channels_multi_3(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 224, 224, 3], shape2=[1, 3, 3, 224]) + process_function(ov_function=function, argv=argv) + # Applied to only input1 + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() != 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + + def test_no_guess_layout_reverse_channels_has_layout_no_c(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 224, 224, 3], shape2=[1, 3, 224, 224]) + function.get_parameters()[0].layout = Layout("NHW?") + function.get_parameters()[1].layout = Layout("N?HW") + process_function(ov_function=function, argv=argv) + # Nothing has applied + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_guess_layout_reverse_channels_incorrect_pos(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[1, 4, 224, 224], shape2=[1, 224, 224, 2]) + function.get_parameters()[0].layout = Layout("NCHW") + function.get_parameters()[1].layout = Layout("NHWC") + process_function(ov_function=function, argv=argv) + # Nothing has applied + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') + + def test_no_reverse_channels_even_with_layout(self): + argv = Namespace(reverse_input_channels=True, mean_scale_values=None, scale=None) + function = create_function2(shape1=[3, 4, 224, 224], shape2=[1, 224, 3, 224]) + process_function(ov_function=function, argv=argv) + # Nothing has applied + op_node0 = list(function.get_parameters()[0].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node0.get_type_name() == 'Relu') + op_node1 = list(function.get_parameters()[1].output(0).get_target_inputs())[0].get_node() + self.assertTrue(op_node1.get_type_name() == 'Relu') diff --git a/model-optimizer/unit_tests/mo/front/common/partial_infer/multi_box_detection_test.py b/model-optimizer/unit_tests/mo/front/common/partial_infer/multi_box_detection_test.py deleted file mode 100644 index 1e73f840d39..00000000000 --- a/model-optimizer/unit_tests/mo/front/common/partial_infer/multi_box_detection_test.py +++ /dev/null @@ -1,155 +0,0 @@ -# Copyright (C) 2018-2021 Intel Corporation -# SPDX-License-Identifier: Apache-2.0 - -import unittest - -import numpy as np - -from mo.front.common.partial_infer.multi_box_detection import multi_box_detection_infer -from mo.front.common.partial_infer.utils import shape_array, dynamic_dimension_value, strict_compare_tensors -from mo.graph.graph import Node -from mo.utils.error import Error -from unit_tests.utils.graph import build_graph - -nodes_attributes = {'node_1': {'value': None, 'kind': 'data'}, - 'node_2': {'value': None, 'kind': 'data'}, - 'node_3': {'value': None, 'kind': 'data'}, - 'detection_output_1': {'type': 'DetectionOutput', 'value': None, 'kind': 'op'}, - 'node_4': {'value': None, 'kind': 'data'} - } - - -class TestMultiBoxDetectionInfer(unittest.TestCase): - def test_do_infer_ideal(self): - graph = build_graph(nodes_attributes, - [('node_1', 'detection_output_1'), - ('node_2', 'detection_output_1'), - ('node_3', 'detection_output_1'), - ('detection_output_1', 'node_4')], - {'node_1': {'shape': np.array([1, 34928])}, - 'node_2': {'shape': np.array([1, 183372])}, - 'node_3': {'shape': np.array([1, 2, 34928])}, - 'detection_output_1': {"background_label_id": 0, "clip": 1, - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "confidence_threshold": 0.01, "keep_top_k": 200, - "nms_threshold": 0.5, "num_classes": 21, - "share_location": 1, "top_k": 200, - "variance_encoded_in_target": 0}, - 'node_4': {'shape': np.array([1, 1, 200, 7])}, - }) - - multi_box_detection_node = Node(graph, 'detection_output_1') - print(multi_box_detection_node) - - multi_box_detection_infer(multi_box_detection_node) - exp_shape = np.array([1, 1, 200, 7]) - res_shape = graph.node['node_4']['shape'] - for i in range(0, len(exp_shape)): - self.assertEqual(exp_shape[i], res_shape[i]) - - self.assertEqual(multi_box_detection_node.background_label_id, 0) - self.assertEqual(multi_box_detection_node.clip, 1) - self.assertEqual(multi_box_detection_node.code_type, 'caffe.PriorBoxParameter.CENTER_SIZE') - self.assertEqual(multi_box_detection_node.confidence_threshold, 0.01) - self.assertEqual(multi_box_detection_node.keep_top_k, 200) - self.assertEqual(multi_box_detection_node.nms_threshold, 0.5) - self.assertEqual(multi_box_detection_node.num_classes, 21) - self.assertEqual(multi_box_detection_node.share_location, 1) - self.assertEqual(multi_box_detection_node.top_k, 200) - self.assertEqual(multi_box_detection_node.variance_encoded_in_target, 0) - - def test_do_infer_without_top_k(self): - graph = build_graph(nodes_attributes, - [('node_1', 'detection_output_1'), - ('node_2', 'detection_output_1'), - ('node_3', 'detection_output_1'), - ('detection_output_1', 'node_4')], - {'node_1': {'shape': np.array([1, 34928])}, - 'node_2': {'shape': np.array([1, 183372])}, - 'node_3': {'shape': np.array([1, 2, 34928])}, - 'detection_output_1': {"background_label_id": "0", "clip": "1", - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "confidence_threshold": "0.01", "keep_top_k": -1, - "nms_threshold": "0.5", "num_classes": 21, - "share_location": "1", "top_k": -1, - "variance_encoded_in_target": "0"}, - 'node_4': {'shape': np.array([1, 1, 69856, 7])}, - }) - - multi_box_detection_node = Node(graph, 'detection_output_1') - - multi_box_detection_infer(multi_box_detection_node) - exp_shape = np.array([1, 1, 8732, 7]) - res_shape = graph.node['node_4']['shape'] - for i in range(0, len(exp_shape)): - self.assertEqual(exp_shape[i], res_shape[i]) - - self.assertEqual(multi_box_detection_node.background_label_id, '0') - self.assertEqual(multi_box_detection_node.clip, '1') - self.assertEqual(multi_box_detection_node.code_type, 'caffe.PriorBoxParameter.CENTER_SIZE') - self.assertEqual(multi_box_detection_node.confidence_threshold, '0.01') - self.assertEqual(multi_box_detection_node.keep_top_k, 8732) - self.assertEqual(multi_box_detection_node.nms_threshold, '0.5') - self.assertEqual(multi_box_detection_node.num_classes, 21) - self.assertEqual(multi_box_detection_node.share_location, '1') - self.assertEqual(multi_box_detection_node.top_k, -1) - self.assertEqual(multi_box_detection_node.variance_encoded_in_target, '0') - - def test_do_infer_without_top_k_dynamic_shape(self): - graph = build_graph(nodes_attributes, - [('node_1', 'detection_output_1'), - ('node_2', 'detection_output_1'), - ('node_3', 'detection_output_1'), - ('detection_output_1', 'node_4')], - {'node_1': {'shape': np.array([1, 34928])}, - 'node_2': {'shape': shape_array([dynamic_dimension_value, 183372])}, - 'node_3': {'shape': np.array([1, 2, 34928])}, - 'detection_output_1': {"background_label_id": "0", "clip": "1", - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "confidence_threshold": "0.01", "keep_top_k": -1, - "nms_threshold": "0.5", "num_classes": 21, - "share_location": "1", "top_k": -1, - "variance_encoded_in_target": "0"}, - 'node_4': {'shape': np.array([1, 1, 69856, 7])}, - }) - - multi_box_detection_node = Node(graph, 'detection_output_1') - - multi_box_detection_infer(multi_box_detection_node) - exp_shape = shape_array([1, 1, dynamic_dimension_value, 7]) - res_shape = graph.node['node_4']['shape'] - self.assertTrue(strict_compare_tensors(exp_shape, res_shape)) - - self.assertEqual(multi_box_detection_node.background_label_id, '0') - self.assertEqual(multi_box_detection_node.clip, '1') - self.assertEqual(multi_box_detection_node.code_type, 'caffe.PriorBoxParameter.CENTER_SIZE') - self.assertEqual(multi_box_detection_node.confidence_threshold, '0.01') - self.assertEqual(multi_box_detection_node.keep_top_k, 8732) - self.assertEqual(multi_box_detection_node.nms_threshold, '0.5') - self.assertEqual(multi_box_detection_node.num_classes, 21) - self.assertEqual(multi_box_detection_node.share_location, '1') - self.assertEqual(multi_box_detection_node.top_k, -1) - self.assertEqual(multi_box_detection_node.variance_encoded_in_target, '0') - - def test_do_infer_raise_error(self): - graph = build_graph(nodes_attributes, - [('node_1', 'detection_output_1'), - ('node_2', 'detection_output_1'), - ('node_3', 'detection_output_1'), - ('detection_output_1', 'node_4')], - {'node_1': {'shape': np.array([1, 34928])}, - 'node_2': {'shape': np.array([1, 183372])}, - 'node_3': {'shape': np.array([1, 3, 34928])}, - 'detection_output_1': {"background_label_id": "0", "clip": "1", - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "confidence_threshold": "0.01", "keep_top_k": -1, - "nms_threshold": "0.5", "num_classes": 21, - "share_location": "1", "top_k": -1, - "variance_encoded_in_target": 0}, - 'node_4': {'shape': np.array([1, 1, 69856, 7])}, - }) - - multi_box_detection_node = Node(graph, 'detection_output_1') - - with self.assertRaisesRegex(Error, 'The "-2" dimension of the prior boxes must be 2 but it is "3" for node*'): - multi_box_detection_infer(multi_box_detection_node) diff --git a/model-optimizer/unit_tests/mo/frontend_ngraph_test.py b/model-optimizer/unit_tests/mo/frontend_ngraph_test.py index f440284b8f9..43aa1749483 100644 --- a/model-optimizer/unit_tests/mo/frontend_ngraph_test.py +++ b/model-optimizer/unit_tests/mo/frontend_ngraph_test.py @@ -43,6 +43,15 @@ def test_moc_extractor(): assert not status.returncode +def test_moc_preprocessing(): + setup_env() + args = [sys.executable, '-m', 'pytest', + os.path.join(os.path.dirname(__file__), 'back/moc_preprocessing_test_actual.py'), '-s'] + + status = subprocess.run(args, env=os.environ) + assert not status.returncode + + def test_main_test(): setup_env() args = [sys.executable, '-m', 'pytest', diff --git a/model-optimizer/unit_tests/mock_mo_frontend/mock_mo_ov_frontend/mock_mo_frontend.hpp b/model-optimizer/unit_tests/mock_mo_frontend/mock_mo_ov_frontend/mock_mo_frontend.hpp index 5f000ab5131..7ad504a24a4 100644 --- a/model-optimizer/unit_tests/mock_mo_frontend/mock_mo_ov_frontend/mock_mo_frontend.hpp +++ b/model-optimizer/unit_tests/mock_mo_frontend/mock_mo_ov_frontend/mock_mo_frontend.hpp @@ -414,10 +414,10 @@ class MOCK_API FrontEndMockPy : public FrontEnd public: FrontEndMockPy() {} - std::shared_ptr convert(InputModel::Ptr model) const override + std::shared_ptr convert(InputModel::Ptr model) const override { m_stat.m_convert_model++; - return std::make_shared(NodeVector{}, ParameterVector{}); + return std::make_shared(ov::NodeVector{}, ov::ParameterVector{}); } static FeStat get_stat() { return m_stat; } @@ -425,22 +425,22 @@ public: static void clear_stat() { m_stat = {}; } private: - InputModel::Ptr load_impl(const std::vector>& params) const override + InputModel::Ptr load_impl(const std::vector& params) const override { - if (params.size() > 0 && ov::is_type>(params[0])) + if (params.size() > 0 && params[0].is()) { - auto path = ov::as_type_ptr>(params[0])->get(); + auto path = params[0].as(); m_stat.m_load_paths.push_back(path); } return std::make_shared(); } - bool supported_impl(const std::vector>& params) const override + bool supported_impl(const std::vector& params) const override { m_stat.m_supported++; - if (params.size() > 0 && ov::is_type>(params[0])) + if (params.size() > 0 && params[0].is()) { - auto path = ov::as_type_ptr>(params[0])->get(); + auto path = params[0].as(); if (path.find(".test_mo_mock_mdl") != std::string::npos) { return true; diff --git a/samples/cpp/benchmark_app/inputs_filling.cpp b/samples/cpp/benchmark_app/inputs_filling.cpp index c47b8083490..73bb0e56847 100644 --- a/samples/cpp/benchmark_app/inputs_filling.cpp +++ b/samples/cpp/benchmark_app/inputs_filling.cpp @@ -117,6 +117,20 @@ void fillBlobBinary(Blob::Ptr& inputBlob, const size_t& inputId, const size_t& inputSize) { MemoryBlob::Ptr minput = as(inputBlob); + auto adjBatchSize = batchSize; + + // Check layout + std::stringstream ss; + auto tensorDesc = inputBlob->getTensorDesc(); + ss << tensorDesc.getLayout(); + auto layout = ss.str(); + std::size_t batchIndex = layout.find("N"); + if (batchIndex == std::string::npos) { + adjBatchSize = 1; + } else if (tensorDesc.getDims().at(batchIndex) != batchSize) { + adjBatchSize = tensorDesc.getDims().at(batchIndex); + } + if (!minput) { IE_THROW() << "We expect inputBlob to be inherited from MemoryBlob in " "fillBlobBinary, " @@ -127,7 +141,7 @@ void fillBlobBinary(Blob::Ptr& inputBlob, auto minputHolder = minput->wmap(); auto inputBlobData = minputHolder.as(); - for (size_t i = 0ULL, inputIndex = requestId * batchSize * inputSize + inputId; i < batchSize; + for (size_t i = 0ULL, inputIndex = requestId * adjBatchSize * inputSize + inputId; i < adjBatchSize; i++, inputIndex += inputSize) { inputIndex %= filePaths.size(); @@ -142,7 +156,7 @@ void fillBlobBinary(Blob::Ptr& inputBlob, if (!binaryFile.good()) { IE_THROW() << "Can not read " << filePaths[inputIndex]; } - auto inputSize = inputBlob->size() * sizeof(T) / batchSize; + auto inputSize = inputBlob->size() * sizeof(T) / adjBatchSize; if (fileSize != inputSize) { IE_THROW() << "File " << filePaths[inputIndex] << " contains " << std::to_string(fileSize) << " bytes " diff --git a/src/bindings/python/src/compatibility/ngraph/__init__.py b/src/bindings/python/src/compatibility/ngraph/__init__.py index 6e864d756e3..7360e1a7a9b 100644 --- a/src/bindings/python/src/compatibility/ngraph/__init__.py +++ b/src/bindings/python/src/compatibility/ngraph/__init__.py @@ -88,6 +88,8 @@ from ngraph.opset8 import hswish from ngraph.opset8 import idft from ngraph.opset8 import if_op from ngraph.opset8 import interpolate +from ngraph.opset8 import i420_to_bgr +from ngraph.opset8 import i420_to_rgb from ngraph.opset8 import less from ngraph.opset8 import less_equal from ngraph.opset8 import log @@ -115,6 +117,8 @@ from ngraph.opset8 import non_max_suppression from ngraph.opset8 import non_zero from ngraph.opset8 import normalize_l2 from ngraph.opset8 import not_equal +from ngraph.opset8 import nv12_to_bgr +from ngraph.opset8 import nv12_to_rgb from ngraph.opset8 import one_hot from ngraph.opset8 import pad from ngraph.opset8 import parameter diff --git a/src/bindings/python/src/compatibility/ngraph/opset8/__init__.py b/src/bindings/python/src/compatibility/ngraph/opset8/__init__.py index 2113bc1f283..55b1a5a8777 100644 --- a/src/bindings/python/src/compatibility/ngraph/opset8/__init__.py +++ b/src/bindings/python/src/compatibility/ngraph/opset8/__init__.py @@ -38,7 +38,7 @@ from ngraph.opset3.ops import cum_sum as cumsum from ngraph.opset8.ops import deformable_convolution from ngraph.opset1.ops import deformable_psroi_pooling from ngraph.opset1.ops import depth_to_space -from ngraph.opset1.ops import detection_output +from ngraph.opset8.ops import detection_output from ngraph.opset7.ops import dft from ngraph.opset1.ops import divide from ngraph.opset7.ops import einsum @@ -71,6 +71,8 @@ from ngraph.opset4.ops import hswish from ngraph.opset7.ops import idft from ngraph.opset8.ops import if_op from ngraph.opset1.ops import interpolate +from ngraph.opset8.ops import i420_to_bgr +from ngraph.opset8.ops import i420_to_rgb from ngraph.opset1.ops import less from ngraph.opset1.ops import less_equal from ngraph.opset1.ops import log @@ -98,6 +100,8 @@ from ngraph.opset5.ops import non_max_suppression from ngraph.opset3.ops import non_zero from ngraph.opset1.ops import normalize_l2 from ngraph.opset1.ops import not_equal +from ngraph.opset8.ops import nv12_to_bgr +from ngraph.opset8.ops import nv12_to_rgb from ngraph.opset1.ops import one_hot from ngraph.opset1.ops import pad from ngraph.opset1.ops import parameter diff --git a/src/bindings/python/src/compatibility/ngraph/opset8/ops.py b/src/bindings/python/src/compatibility/ngraph/opset8/ops.py index 0e375b1a953..d0f32933d66 100644 --- a/src/bindings/python/src/compatibility/ngraph/opset8/ops.py +++ b/src/bindings/python/src/compatibility/ngraph/opset8/ops.py @@ -3,40 +3,28 @@ """Factory functions for all ngraph ops.""" from functools import partial -from typing import Callable, Iterable, List, Optional, Set, Union, Tuple +from typing import List, Optional, Tuple import numpy as np -from ngraph.impl import Node, Shape -from ngraph.impl.op import Constant, Parameter +from ngraph.exceptions import UserInputError +from ngraph.impl import Node from ngraph.opset_utils import _get_node_factory -from ngraph.utils.decorators import binary_op, nameable_op, unary_op +from ngraph.utils.decorators import nameable_op from ngraph.utils.input_validation import ( - assert_list_of_ints, check_valid_attributes, is_non_negative_value, is_positive_value, ) -from ngraph.utils.node_factory import NodeFactory from ngraph.utils.tensor_iterator_types import ( GraphBody, - TensorIteratorSliceInputDesc, - TensorIteratorMergedInputDesc, TensorIteratorInvariantInputDesc, TensorIteratorBodyOutputDesc, - TensorIteratorConcatOutputDesc, ) from ngraph.utils.types import ( NodeInput, - NumericData, - NumericType, - ScalarData, TensorShape, as_node, as_nodes, - get_dtype, - get_element_type, - get_element_type_str, - make_constant_node, ) _get_node_factory_opset8 = partial(_get_node_factory, "opset8") @@ -140,18 +128,18 @@ def adaptive_max_pool( @nameable_op def multiclass_nms( - boxes: NodeInput, - scores: NodeInput, - sort_result_type: str = "none", - sort_result_across_batch: bool = False, - output_type: str = "i64", - iou_threshold: float = 0.0, - score_threshold: float = 0.0, - nms_top_k: int = -1, - keep_top_k: int = -1, - background_class: int = -1, - nms_eta: float = 1.0, - normalized: bool = True + boxes: NodeInput, + scores: NodeInput, + sort_result_type: str = "none", + sort_result_across_batch: bool = False, + output_type: str = "i64", + iou_threshold: float = 0.0, + score_threshold: float = 0.0, + nms_top_k: int = -1, + keep_top_k: int = -1, + background_class: int = -1, + nms_eta: float = 1.0, + normalized: bool = True ) -> Node: """Return a node which performs MulticlassNms. @@ -196,19 +184,19 @@ def multiclass_nms( @nameable_op def matrix_nms( - boxes: NodeInput, - scores: NodeInput, - sort_result_type: str = "none", - sort_result_across_batch: bool = False, - output_type: str = "i64", - score_threshold: float = 0.0, - nms_top_k: int = -1, - keep_top_k: int = -1, - background_class: int = -1, - decay_function: str = "linear", - gaussian_sigma: float = 2.0, - post_threshold: float = 0.0, - normalized: bool = True + boxes: NodeInput, + scores: NodeInput, + sort_result_type: str = "none", + sort_result_across_batch: bool = False, + output_type: str = "i64", + score_threshold: float = 0.0, + nms_top_k: int = -1, + keep_top_k: int = -1, + background_class: int = -1, + decay_function: str = "linear", + gaussian_sigma: float = 2.0, + post_threshold: float = 0.0, + normalized: bool = True ) -> Node: """Return a node which performs MatrixNms. @@ -280,17 +268,17 @@ def gather( @nameable_op def max_pool( - data: NodeInput, - strides: List[int], - dilations: List[int], - pads_begin: List[int], - pads_end: List[int], - kernel_shape: TensorShape, - rounding_type: str = "floor", - auto_pad: Optional[str] = None, - index_element_type: Optional[str] = "i64", - axis: Optional[int] = 0, - name: Optional[str] = None, + data: NodeInput, + strides: List[int], + dilations: List[int], + pads_begin: List[int], + pads_end: List[int], + kernel_shape: TensorShape, + rounding_type: str = "floor", + auto_pad: Optional[str] = None, + index_element_type: Optional[str] = "i64", + axis: Optional[int] = 0, + name: Optional[str] = None, ) -> Node: """Perform max pooling operation and return both values and indices of the selected elements. @@ -456,7 +444,7 @@ def gather_nd( def prior_box( - layer_shape: Node, image_shape: NodeInput, attrs: dict, name: Optional[str] = None + layer_shape: Node, image_shape: NodeInput, attrs: dict, name: Optional[str] = None ) -> Node: """Generate prior boxes of specified sizes and aspect ratios across all dimensions. @@ -558,3 +546,234 @@ def prior_box( check_valid_attributes("PriorBox", attrs, requirements) return _get_node_factory_opset8().create("PriorBox", [layer_shape, as_node(image_shape)], attrs) + + +@nameable_op +def i420_to_bgr( + arg: NodeInput, + arg_u: Optional[NodeInput] = None, + arg_v: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs I420toBGR operation. + + @param arg: The node providing single or Y plane data. + @param arg_u: The node providing U plane data. Required for separate planes. + @param arg_v: The node providing V plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing I420toBGR operation. + """ + if arg_u is None and arg_v is None: + inputs = as_nodes(arg) + elif arg_u is not None and arg_v is not None: + inputs = as_nodes(arg, arg_u, arg_v) + else: + raise UserInputError( + "Operation I420toBGR must have one (single plane) or three (separate planes) inputs provided." + ) + + return _get_node_factory_opset8().create("I420toBGR", inputs) + + +@nameable_op +def i420_to_rgb( + arg: NodeInput, + arg_u: Optional[NodeInput] = None, + arg_v: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs I420toRGB operation. + + @param arg: The node providing single or Y plane data. + @param arg_u: The node providing U plane data. Required for separate planes. + @param arg_v: The node providing V plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing I420toRGB operation. + """ + if arg_u is None and arg_v is None: + inputs = as_nodes(arg) + elif arg_u is not None and arg_v is not None: + inputs = as_nodes(arg, arg_u, arg_v) + else: + raise UserInputError( + "Operation I420toRGB must have one (single plane) or three (separate planes) inputs provided." + ) + + return _get_node_factory_opset8().create("I420toRGB", inputs) + + +@nameable_op +def nv12_to_bgr( + arg: NodeInput, + arg_uv: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs NV12toBGR operation. + + @param arg: The node providing single or Y plane data. + @param arg_uv: The node providing UV plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing NV12toBGR operation. + """ + if arg_uv is None: + inputs = as_nodes(arg) + else: + inputs = as_nodes(arg, arg_uv) + + return _get_node_factory_opset8().create("NV12toBGR", inputs) + + +@nameable_op +def nv12_to_rgb( + arg: NodeInput, + arg_uv: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs NV12toRGB operation. + + @param arg: The node providing single or Y plane data. + @param arg_uv: The node providing UV plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing NV12toRGB operation. + """ + if arg_uv is None: + inputs = as_nodes(arg) + else: + inputs = as_nodes(arg, arg_uv) + + return _get_node_factory_opset8().create("NV12toRGB", inputs) + + +@nameable_op +def detection_output( + box_logits: NodeInput, + class_preds: NodeInput, + proposals: NodeInput, + attrs: dict, + aux_class_preds: Optional[NodeInput] = None, + aux_box_preds: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Generate the detection output using information on location and confidence predictions. + + @param box_logits: The 2D input tensor with box logits. + @param class_preds: The 2D input tensor with class predictions. + @param proposals: The 3D input tensor with proposals. + @param attrs: The dictionary containing key, value pairs for attributes. + @param aux_class_preds: The 2D input tensor with additional class predictions information. + @param aux_box_preds: The 2D input tensor with additional box predictions information. + @param name: Optional name for the output node. + @return Node representing DetectionOutput operation. + Available attributes are: + * background_label_id The background label id. + Range of values: integer value + Default value: 0 + Required: no + * top_k Maximum number of results to be kept per batch after NMS step. + Range of values: integer value + Default value: -1 + Required: no + * variance_encoded_in_target The flag that denotes if variance is encoded in target. + Range of values: {False, True} + Default value: False + Required: no + * keep_top_k Maximum number of bounding boxes per batch to be kept after NMS step. + Range of values: integer values + Default value: None + Required: yes + * code_type The type of coding method for bounding boxes. + Range of values: {'caffe.PriorBoxParameter.CENTER_SIZE', + 'caffe.PriorBoxParameter.CORNER'} + Default value: 'caffe.PriorBoxParameter.CORNER' + Required: no + * share_location The flag that denotes if bounding boxes are shared among different + classes. + Range of values: {True, False} + Default value: True + Required: no + * nms_threshold The threshold to be used in the NMS stage. + Range of values: floating point value + Default value: None + Required: yes + * confidence_threshold Specifies the minimum confidence threshold for detection boxes to be + considered. + Range of values: floating point value + Default value: 0 + Required: no + * clip_after_nms The flag that denotes whether to perform clip bounding boxes after + non-maximum suppression or not. + Range of values: {True, False} + Default value: False + Required: no + * clip_before_nms The flag that denotes whether to perform clip bounding boxes before + non-maximum suppression or not. + Range of values: {True, False} + Default value: False + Required: no + * decrease_label_id The flag that denotes how to perform NMS. + Range of values: False - perform NMS like in Caffe*. + True - perform NMS like in MxNet*. + Default value: False + Required: no + * normalized The flag that denotes whether input tensors with boxes are normalized. + Range of values: {True, False} + Default value: False + Required: no + * input_height The input image height. + Range of values: positive integer number + Default value: 1 + Required: no + * input_width The input image width. + Range of values: positive integer number + Default value: 1 + Required: no + * objectness_score The threshold to sort out confidence predictions. + Range of values: non-negative float number + Default value: 0 + Required: no + Example of attribute dictionary: + @code{.py} + # just required ones + attrs = { + 'keep_top_k': [1, 2, 3], + 'nms_threshold': 0.645, + } + attrs = { + 'keep_top_k': [1, 2, 3], + 'nms_threshold': 0.645, + 'normalized': True, + 'clip_before_nms': True, + 'input_height': [32], + 'input_width': [32], + } + @endcode + Optional attributes which are absent from dictionary will be set with corresponding default. + """ + requirements = [ + ("background_label_id", False, np.integer, None), + ("top_k", False, np.integer, None), + ("variance_encoded_in_target", False, np.bool_, None), + ("keep_top_k", True, np.integer, None), + ("code_type", False, np.str_, None), + ("share_location", False, np.bool_, None), + ("nms_threshold", True, np.floating, None), + ("confidence_threshold", False, np.floating, None), + ("clip_after_nms", False, np.bool_, None), + ("clip_before_nms", False, np.bool_, None), + ("decrease_label_id", False, np.bool_, None), + ("normalized", False, np.bool_, None), + ("input_height", False, np.integer, is_positive_value), + ("input_width", False, np.integer, is_positive_value), + ("objectness_score", False, np.floating, is_non_negative_value), + ] + + check_valid_attributes("DetectionOutput", attrs, requirements) + + inputs = [box_logits, class_preds, proposals] + if aux_class_preds is not None: + inputs.append(aux_class_preds) + if aux_box_preds is not None: + inputs.append(aux_box_preds) + inputs = as_nodes(*inputs) + + return _get_node_factory_opset8().create("DetectionOutput", inputs, attrs) diff --git a/src/bindings/python/src/compatibility/pyngraph/rt_map.cpp b/src/bindings/python/src/compatibility/pyngraph/rt_map.cpp index 535594f0564..5bcc81423c3 100644 --- a/src/bindings/python/src/compatibility/pyngraph/rt_map.cpp +++ b/src/bindings/python/src/compatibility/pyngraph/rt_map.cpp @@ -20,7 +20,7 @@ namespace py = pybind11; -using PyRTMap = std::map; +using PyRTMap = ngraph::RTMap; PYBIND11_MAKE_OPAQUE(PyRTMap); @@ -35,4 +35,7 @@ void regclass_pyngraph_PyRTMap(py::module m) { py_map.def("__setitem__", [](PyRTMap& m, const std::string& k, const int64_t v) { m[k] = v; }); + py_map.def("__getitem__", [](PyRTMap& m, const std::string& k) { + return m.at(k).as>(); + }); } diff --git a/src/bindings/python/src/openvino/runtime/__init__.py b/src/bindings/python/src/openvino/runtime/__init__.py index d62a22a9333..4975a289736 100644 --- a/src/bindings/python/src/openvino/runtime/__init__.py +++ b/src/bindings/python/src/openvino/runtime/__init__.py @@ -44,6 +44,7 @@ from openvino.runtime.impl import Function from openvino.runtime.impl import Node from openvino.runtime.impl import PartialShape from openvino.runtime.impl import Layout +from openvino.runtime.impl import layout_helpers from openvino.runtime.ie_api import Core from openvino.runtime.ie_api import ExecutableNetwork diff --git a/src/bindings/python/src/openvino/runtime/ie_api.py b/src/bindings/python/src/openvino/runtime/ie_api.py index 26088c169c4..22cf00c85e7 100644 --- a/src/bindings/python/src/openvino/runtime/ie_api.py +++ b/src/bindings/python/src/openvino/runtime/ie_api.py @@ -44,13 +44,10 @@ def get_input_types(obj: Union[InferRequestBase, ExecutableNetworkBase]) -> dict class InferRequest(InferRequestBase): """InferRequest wrapper.""" - def infer(self, inputs: dict = None) -> List[np.ndarray]: + def infer(self, inputs: dict = None) -> dict: """Infer wrapper for InferRequest.""" inputs = {} if inputs is None else normalize_inputs(inputs, get_input_types(self)) - res = super().infer(inputs) - # Required to return list since np.ndarray forces all of tensors data to match in - # dimensions. This results in errors when running ops like variadic split. - return [copy.deepcopy(tensor.data) for tensor in res] + return super().infer(inputs) def start_async(self, inputs: dict = None, userdata: Any = None) -> None: """Asynchronous infer wrapper for InferRequest.""" @@ -65,13 +62,10 @@ class ExecutableNetwork(ExecutableNetworkBase): """Create new InferRequest object.""" return InferRequest(super().create_infer_request()) - def infer_new_request(self, inputs: dict = None) -> List[np.ndarray]: + def infer_new_request(self, inputs: dict = None) -> dict: """Infer wrapper for ExecutableNetwork.""" inputs = {} if inputs is None else normalize_inputs(inputs, get_input_types(self)) - res = super().infer_new_request(inputs) - # Required to return list since np.ndarray forces all of tensors data to match in - # dimensions. This results in errors when running ops like variadic split. - return [copy.deepcopy(tensor.data) for tensor in res] + return super().infer_new_request(inputs) class AsyncInferQueue(AsyncInferQueueBase): diff --git a/src/bindings/python/src/openvino/runtime/opset8/__init__.py b/src/bindings/python/src/openvino/runtime/opset8/__init__.py index 073e68a0882..7b3352edcfc 100644 --- a/src/bindings/python/src/openvino/runtime/opset8/__init__.py +++ b/src/bindings/python/src/openvino/runtime/opset8/__init__.py @@ -38,7 +38,7 @@ from openvino.runtime.opset3.ops import cum_sum as cumsum from openvino.runtime.opset8.ops import deformable_convolution from openvino.runtime.opset1.ops import deformable_psroi_pooling from openvino.runtime.opset1.ops import depth_to_space -from openvino.runtime.opset1.ops import detection_output +from openvino.runtime.opset8.ops import detection_output from openvino.runtime.opset7.ops import dft from openvino.runtime.opset1.ops import divide from openvino.runtime.opset7.ops import einsum @@ -71,6 +71,8 @@ from openvino.runtime.opset4.ops import hswish from openvino.runtime.opset7.ops import idft from openvino.runtime.opset8.ops import if_op from openvino.runtime.opset1.ops import interpolate +from openvino.runtime.opset8.ops import i420_to_bgr +from openvino.runtime.opset8.ops import i420_to_rgb from openvino.runtime.opset1.ops import less from openvino.runtime.opset1.ops import less_equal from openvino.runtime.opset1.ops import log @@ -98,6 +100,8 @@ from openvino.runtime.opset5.ops import non_max_suppression from openvino.runtime.opset3.ops import non_zero from openvino.runtime.opset1.ops import normalize_l2 from openvino.runtime.opset1.ops import not_equal +from openvino.runtime.opset8.ops import nv12_to_bgr +from openvino.runtime.opset8.ops import nv12_to_rgb from openvino.runtime.opset1.ops import one_hot from openvino.runtime.opset1.ops import pad from openvino.runtime.opset1.ops import parameter diff --git a/src/bindings/python/src/openvino/runtime/opset8/ops.py b/src/bindings/python/src/openvino/runtime/opset8/ops.py index eb126f2aea1..fda7230a825 100644 --- a/src/bindings/python/src/openvino/runtime/opset8/ops.py +++ b/src/bindings/python/src/openvino/runtime/opset8/ops.py @@ -3,40 +3,28 @@ """Factory functions for all openvino ops.""" from functools import partial -from typing import Callable, Iterable, List, Optional, Set, Union, Tuple +from typing import List, Optional, Tuple import numpy as np -from openvino.runtime.impl import Node, Shape -from openvino.runtime.impl.op import Constant, Parameter +from openvino.runtime.exceptions import UserInputError +from openvino.runtime.impl import Node from openvino.runtime.opset_utils import _get_node_factory -from openvino.runtime.utils.decorators import binary_op, nameable_op, unary_op +from openvino.runtime.utils.decorators import nameable_op from openvino.runtime.utils.input_validation import ( - assert_list_of_ints, check_valid_attributes, is_non_negative_value, is_positive_value, ) -from openvino.runtime.utils.node_factory import NodeFactory from openvino.runtime.utils.tensor_iterator_types import ( GraphBody, - TensorIteratorSliceInputDesc, - TensorIteratorMergedInputDesc, TensorIteratorInvariantInputDesc, TensorIteratorBodyOutputDesc, - TensorIteratorConcatOutputDesc, ) from openvino.runtime.utils.types import ( NodeInput, - NumericData, - NumericType, - ScalarData, TensorShape, as_node, as_nodes, - get_dtype, - get_element_type, - get_element_type_str, - make_constant_node, ) _get_node_factory_opset8 = partial(_get_node_factory, "opset8") @@ -140,18 +128,18 @@ def adaptive_max_pool( @nameable_op def multiclass_nms( - boxes: NodeInput, - scores: NodeInput, - sort_result_type: str = "none", - sort_result_across_batch: bool = False, - output_type: str = "i64", - iou_threshold: float = 0.0, - score_threshold: float = 0.0, - nms_top_k: int = -1, - keep_top_k: int = -1, - background_class: int = -1, - nms_eta: float = 1.0, - normalized: bool = True + boxes: NodeInput, + scores: NodeInput, + sort_result_type: str = "none", + sort_result_across_batch: bool = False, + output_type: str = "i64", + iou_threshold: float = 0.0, + score_threshold: float = 0.0, + nms_top_k: int = -1, + keep_top_k: int = -1, + background_class: int = -1, + nms_eta: float = 1.0, + normalized: bool = True ) -> Node: """Return a node which performs MulticlassNms. @@ -196,19 +184,19 @@ def multiclass_nms( @nameable_op def matrix_nms( - boxes: NodeInput, - scores: NodeInput, - sort_result_type: str = "none", - sort_result_across_batch: bool = False, - output_type: str = "i64", - score_threshold: float = 0.0, - nms_top_k: int = -1, - keep_top_k: int = -1, - background_class: int = -1, - decay_function: str = "linear", - gaussian_sigma: float = 2.0, - post_threshold: float = 0.0, - normalized: bool = True + boxes: NodeInput, + scores: NodeInput, + sort_result_type: str = "none", + sort_result_across_batch: bool = False, + output_type: str = "i64", + score_threshold: float = 0.0, + nms_top_k: int = -1, + keep_top_k: int = -1, + background_class: int = -1, + decay_function: str = "linear", + gaussian_sigma: float = 2.0, + post_threshold: float = 0.0, + normalized: bool = True ) -> Node: """Return a node which performs MatrixNms. @@ -280,17 +268,17 @@ def gather( @nameable_op def max_pool( - data: NodeInput, - strides: List[int], - dilations: List[int], - pads_begin: List[int], - pads_end: List[int], - kernel_shape: TensorShape, - rounding_type: str = "floor", - auto_pad: Optional[str] = None, - index_element_type: Optional[str] = "i64", - axis: Optional[int] = 0, - name: Optional[str] = None, + data: NodeInput, + strides: List[int], + dilations: List[int], + pads_begin: List[int], + pads_end: List[int], + kernel_shape: TensorShape, + rounding_type: str = "floor", + auto_pad: Optional[str] = None, + index_element_type: Optional[str] = "i64", + axis: Optional[int] = 0, + name: Optional[str] = None, ) -> Node: """Perform max pooling operation and return both values and indices of the selected elements. @@ -457,7 +445,7 @@ def gather_nd( @nameable_op def prior_box( - layer_shape: Node, image_shape: NodeInput, attrs: dict, name: Optional[str] = None + layer_shape: Node, image_shape: NodeInput, attrs: dict, name: Optional[str] = None ) -> Node: """Generate prior boxes of specified sizes and aspect ratios across all dimensions. @@ -559,3 +547,234 @@ def prior_box( check_valid_attributes("PriorBox", attrs, requirements) return _get_node_factory_opset8().create("PriorBox", [layer_shape, as_node(image_shape)], attrs) + + +@nameable_op +def i420_to_bgr( + arg: NodeInput, + arg_u: Optional[NodeInput] = None, + arg_v: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs I420toBGR operation. + + @param arg: The node providing single or Y plane data. + @param arg_u: The node providing U plane data. Required for separate planes. + @param arg_v: The node providing V plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing I420toBGR operation. + """ + if arg_u is None and arg_v is None: + inputs = as_nodes(arg) + elif arg_u is not None and arg_v is not None: + inputs = as_nodes(arg, arg_u, arg_v) + else: + raise UserInputError( + "Operation I420toBGR must have one (single plane) or three (separate planes) inputs provided." + ) + + return _get_node_factory_opset8().create("I420toBGR", inputs) + + +@nameable_op +def i420_to_rgb( + arg: NodeInput, + arg_u: Optional[NodeInput] = None, + arg_v: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs I420toRGB operation. + + @param arg: The node providing single or Y plane data. + @param arg_u: The node providing U plane data. Required for separate planes. + @param arg_v: The node providing V plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing I420toRGB operation. + """ + if arg_u is None and arg_v is None: + inputs = as_nodes(arg) + elif arg_u is not None and arg_v is not None: + inputs = as_nodes(arg, arg_u, arg_v) + else: + raise UserInputError( + "Operation I420toRGB must have one (single plane) or three (separate planes) inputs provided." + ) + + return _get_node_factory_opset8().create("I420toRGB", inputs) + + +@nameable_op +def nv12_to_bgr( + arg: NodeInput, + arg_uv: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs NV12toBGR operation. + + @param arg: The node providing single or Y plane data. + @param arg_uv: The node providing UV plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing NV12toBGR operation. + """ + if arg_uv is None: + inputs = as_nodes(arg) + else: + inputs = as_nodes(arg, arg_uv) + + return _get_node_factory_opset8().create("NV12toBGR", inputs) + + +@nameable_op +def nv12_to_rgb( + arg: NodeInput, + arg_uv: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Return a node which performs NV12toRGB operation. + + @param arg: The node providing single or Y plane data. + @param arg_uv: The node providing UV plane data. Required for separate planes. + @param name: The optional name for the created output node. + @return The new node performing NV12toRGB operation. + """ + if arg_uv is None: + inputs = as_nodes(arg) + else: + inputs = as_nodes(arg, arg_uv) + + return _get_node_factory_opset8().create("NV12toRGB", inputs) + + +@nameable_op +def detection_output( + box_logits: NodeInput, + class_preds: NodeInput, + proposals: NodeInput, + attrs: dict, + aux_class_preds: Optional[NodeInput] = None, + aux_box_preds: Optional[NodeInput] = None, + name: Optional[str] = None, +) -> Node: + """Generate the detection output using information on location and confidence predictions. + + @param box_logits: The 2D input tensor with box logits. + @param class_preds: The 2D input tensor with class predictions. + @param proposals: The 3D input tensor with proposals. + @param attrs: The dictionary containing key, value pairs for attributes. + @param aux_class_preds: The 2D input tensor with additional class predictions information. + @param aux_box_preds: The 2D input tensor with additional box predictions information. + @param name: Optional name for the output node. + @return Node representing DetectionOutput operation. + Available attributes are: + * background_label_id The background label id. + Range of values: integer value + Default value: 0 + Required: no + * top_k Maximum number of results to be kept per batch after NMS step. + Range of values: integer value + Default value: -1 + Required: no + * variance_encoded_in_target The flag that denotes if variance is encoded in target. + Range of values: {False, True} + Default value: False + Required: no + * keep_top_k Maximum number of bounding boxes per batch to be kept after NMS step. + Range of values: integer values + Default value: None + Required: yes + * code_type The type of coding method for bounding boxes. + Range of values: {'caffe.PriorBoxParameter.CENTER_SIZE', + 'caffe.PriorBoxParameter.CORNER'} + Default value: 'caffe.PriorBoxParameter.CORNER' + Required: no + * share_location The flag that denotes if bounding boxes are shared among different + classes. + Range of values: {True, False} + Default value: True + Required: no + * nms_threshold The threshold to be used in the NMS stage. + Range of values: floating point value + Default value: None + Required: yes + * confidence_threshold Specifies the minimum confidence threshold for detection boxes to be + considered. + Range of values: floating point value + Default value: 0 + Required: no + * clip_after_nms The flag that denotes whether to perform clip bounding boxes after + non-maximum suppression or not. + Range of values: {True, False} + Default value: False + Required: no + * clip_before_nms The flag that denotes whether to perform clip bounding boxes before + non-maximum suppression or not. + Range of values: {True, False} + Default value: False + Required: no + * decrease_label_id The flag that denotes how to perform NMS. + Range of values: False - perform NMS like in Caffe*. + True - perform NMS like in MxNet*. + Default value: False + Required: no + * normalized The flag that denotes whether input tensors with boxes are normalized. + Range of values: {True, False} + Default value: False + Required: no + * input_height The input image height. + Range of values: positive integer number + Default value: 1 + Required: no + * input_width The input image width. + Range of values: positive integer number + Default value: 1 + Required: no + * objectness_score The threshold to sort out confidence predictions. + Range of values: non-negative float number + Default value: 0 + Required: no + Example of attribute dictionary: + @code{.py} + # just required ones + attrs = { + 'keep_top_k': [1, 2, 3], + 'nms_threshold': 0.645, + } + attrs = { + 'keep_top_k': [1, 2, 3], + 'nms_threshold': 0.645, + 'normalized': True, + 'clip_before_nms': True, + 'input_height': [32], + 'input_width': [32], + } + @endcode + Optional attributes which are absent from dictionary will be set with corresponding default. + """ + requirements = [ + ("background_label_id", False, np.integer, None), + ("top_k", False, np.integer, None), + ("variance_encoded_in_target", False, np.bool_, None), + ("keep_top_k", True, np.integer, None), + ("code_type", False, np.str_, None), + ("share_location", False, np.bool_, None), + ("nms_threshold", True, np.floating, None), + ("confidence_threshold", False, np.floating, None), + ("clip_after_nms", False, np.bool_, None), + ("clip_before_nms", False, np.bool_, None), + ("decrease_label_id", False, np.bool_, None), + ("normalized", False, np.bool_, None), + ("input_height", False, np.integer, is_positive_value), + ("input_width", False, np.integer, is_positive_value), + ("objectness_score", False, np.floating, is_non_negative_value), + ] + + check_valid_attributes("DetectionOutput", attrs, requirements) + + inputs = [box_logits, class_preds, proposals] + if aux_class_preds is not None: + inputs.append(aux_class_preds) + if aux_box_preds is not None: + inputs.append(aux_box_preds) + inputs = as_nodes(*inputs) + + return _get_node_factory_opset8().create("DetectionOutput", inputs, attrs) diff --git a/src/bindings/python/src/pyopenvino/core/common.cpp b/src/bindings/python/src/pyopenvino/core/common.cpp index 6ef5774163a..afcbb3d7b2a 100644 --- a/src/bindings/python/src/pyopenvino/core/common.cpp +++ b/src/bindings/python/src/pyopenvino/core/common.cpp @@ -296,4 +296,83 @@ uint32_t get_optimal_number_of_requests(const ov::runtime::ExecutableNetwork& ac } } +py::dict outputs_to_dict(const std::vector>& outputs, ov::runtime::InferRequest& request) { + py::dict res; + for (const auto& out : outputs) { + ov::runtime::Tensor t{request.get_tensor(out)}; + std::cout << t.get_element_type() << " !\n"; + switch (t.get_element_type()) { + case ov::element::Type_t::i8: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::i16: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::i32: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::i64: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::u8: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::u16: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::u32: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::u64: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::bf16: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr.view("int16"); + break; + } + case ov::element::Type_t::f16: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr.view("int16"); + break; + } + case ov::element::Type_t::f32: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::f64: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + case ov::element::Type_t::boolean: { + py::array arr(t.get_shape(), t.data()); + res[py::cast(out)] = arr; + break; + } + default: { + break; + } + } + } + return res; +} + }; // namespace Common diff --git a/src/bindings/python/src/pyopenvino/core/common.hpp b/src/bindings/python/src/pyopenvino/core/common.hpp index 68d28ec6396..7be421b3434 100644 --- a/src/bindings/python/src/pyopenvino/core/common.hpp +++ b/src/bindings/python/src/pyopenvino/core/common.hpp @@ -44,6 +44,8 @@ namespace Common uint32_t get_optimal_number_of_requests(const ov::runtime::ExecutableNetwork& actual); + py::dict outputs_to_dict(const std::vector>& outputs, ov::runtime::InferRequest& request); + // Use only with classes that are not creatable by users on Python's side, because // Objects created in Python that are wrapped with such wrapper will cause memory leaks. template diff --git a/src/bindings/python/src/pyopenvino/core/containers.hpp b/src/bindings/python/src/pyopenvino/core/containers.hpp index e24e7336236..268ce68ff06 100644 --- a/src/bindings/python/src/pyopenvino/core/containers.hpp +++ b/src/bindings/python/src/pyopenvino/core/containers.hpp @@ -17,7 +17,6 @@ namespace py = pybind11; namespace Containers { using TensorIndexMap = std::map; using TensorNameMap = std::map; - using InferResults = std::vector; void regclass_TensorIndexMap(py::module m); void regclass_TensorNameMap(py::module m); diff --git a/src/bindings/python/src/pyopenvino/core/executable_network.cpp b/src/bindings/python/src/pyopenvino/core/executable_network.cpp index 37d6811f38d..116582465fc 100644 --- a/src/bindings/python/src/pyopenvino/core/executable_network.cpp +++ b/src/bindings/python/src/pyopenvino/core/executable_network.cpp @@ -36,12 +36,7 @@ void regclass_ExecutableNetwork(py::module m) { // Update inputs if there are any Common::set_request_tensors(request, inputs); request.infer(); - - Containers::InferResults results; - for (const auto out : self.outputs()) { - results.push_back(request.get_tensor(out)); - } - return results; + return Common::outputs_to_dict(self.outputs(), request); }, py::arg("inputs")); diff --git a/src/bindings/python/src/pyopenvino/core/infer_request.cpp b/src/bindings/python/src/pyopenvino/core/infer_request.cpp index 762a272ae83..14dc2befeef 100644 --- a/src/bindings/python/src/pyopenvino/core/infer_request.cpp +++ b/src/bindings/python/src/pyopenvino/core/infer_request.cpp @@ -65,11 +65,8 @@ void regclass_InferRequest(py::module m) { self._start_time = Time::now(); self._request.infer(); self._end_time = Time::now(); - Containers::InferResults results; - for (auto& out : self._outputs) { - results.push_back(self._request.get_tensor(out)); - } - return results; + + return Common::outputs_to_dict(self._outputs, self._request); }, py::arg("inputs")); @@ -271,4 +268,8 @@ void regclass_InferRequest(py::module m) { cls.def_property_readonly("profiling_info", [](InferRequestWrapper& self) { return self._request.get_profiling_info(); }); + + cls.def_property_readonly("results", [](InferRequestWrapper& self) { + return Common::outputs_to_dict(self._outputs, self._request); + }); } diff --git a/src/bindings/python/src/pyopenvino/graph/layout.cpp b/src/bindings/python/src/pyopenvino/graph/layout.cpp index 8ce4fdf77bf..bbf58a04069 100644 --- a/src/bindings/python/src/pyopenvino/graph/layout.cpp +++ b/src/bindings/python/src/pyopenvino/graph/layout.cpp @@ -39,4 +39,5 @@ void regclass_graph_Layout(py::module m) { layout.def("__str__", [](const ov::Layout& self) { return self.to_string(); }); + layout.def_property_readonly("empty", &ov::Layout::empty); } diff --git a/src/bindings/python/src/pyopenvino/graph/node.cpp b/src/bindings/python/src/pyopenvino/graph/node.cpp index 1552367b965..f556b580843 100644 --- a/src/bindings/python/src/pyopenvino/graph/node.cpp +++ b/src/bindings/python/src/pyopenvino/graph/node.cpp @@ -9,7 +9,7 @@ #include #include "dict_attribute_visitor.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/op/add.hpp" #include "openvino/op/divide.hpp" #include "openvino/op/multiply.hpp" diff --git a/src/bindings/python/src/pyopenvino/graph/rt_map.cpp b/src/bindings/python/src/pyopenvino/graph/rt_map.cpp index 95f1d8f0f42..ec60f8d0f36 100644 --- a/src/bindings/python/src/pyopenvino/graph/rt_map.cpp +++ b/src/bindings/python/src/pyopenvino/graph/rt_map.cpp @@ -10,7 +10,7 @@ #include "dict_attribute_visitor.hpp" #include "openvino/core/node.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/op/add.hpp" #include "openvino/op/divide.hpp" #include "openvino/op/multiply.hpp" diff --git a/src/bindings/python/src/pyopenvino/graph/variant.hpp b/src/bindings/python/src/pyopenvino/graph/variant.hpp index 3dee3c6d6f0..fe84fd63d30 100644 --- a/src/bindings/python/src/pyopenvino/graph/variant.hpp +++ b/src/bindings/python/src/pyopenvino/graph/variant.hpp @@ -10,7 +10,7 @@ #include #include -#include "openvino/core/any.hpp" // ov::Variant +#include "openvino/core/any.hpp" // ov::RuntimeAttribute namespace py = pybind11; diff --git a/src/bindings/python/tests/mock/mock_py_ov_frontend/mock_py_frontend.hpp b/src/bindings/python/tests/mock/mock_py_ov_frontend/mock_py_frontend.hpp index 416787a6a0b..0e45241492c 100644 --- a/src/bindings/python/tests/mock/mock_py_ov_frontend/mock_py_frontend.hpp +++ b/src/bindings/python/tests/mock/mock_py_ov_frontend/mock_py_frontend.hpp @@ -614,16 +614,16 @@ class MOCK_API FrontEndMockPy : public FrontEnd { public: FrontEndMockPy() {} - InputModel::Ptr load_impl(const std::vector>& params) const override { - if (params.size() > 0 && ov::is_type>(params[0])) - m_stat.m_load_paths.push_back(ov::as_type_ptr>(params[0])->get()); + InputModel::Ptr load_impl(const std::vector& params) const override { + if (params.size() > 0 && params[0].is()) + m_stat.m_load_paths.push_back(params[0].as()); return std::make_shared(); } - bool supported_impl(const std::vector>& params) const override { + bool supported_impl(const std::vector& params) const override { m_stat.m_supported++; - if (params.size() > 0 && ov::is_type>(params[0])) { - auto path = ov::as_type_ptr>(params[0])->get(); + if (params.size() > 0 && params[0].is()) { + auto path = params[0].as(); if (path.find(".test_mock_py_mdl") != std::string::npos) { return true; } @@ -631,26 +631,26 @@ public: return false; } - std::shared_ptr convert(InputModel::Ptr model) const override { + std::shared_ptr convert(InputModel::Ptr model) const override { m_stat.m_convert_model++; - return std::make_shared(NodeVector{}, ParameterVector{}); + return std::make_shared(ov::NodeVector{}, ov::ParameterVector{}); } - void convert(std::shared_ptr func) const override { + void convert(std::shared_ptr func) const override { m_stat.m_convert++; } - std::shared_ptr convert_partially(InputModel::Ptr model) const override { + std::shared_ptr convert_partially(InputModel::Ptr model) const override { m_stat.m_convert_partially++; - return std::make_shared(NodeVector{}, ParameterVector{}); + return std::make_shared(ov::NodeVector{}, ov::ParameterVector{}); } - std::shared_ptr decode(InputModel::Ptr model) const override { + std::shared_ptr decode(InputModel::Ptr model) const override { m_stat.m_decode++; - return std::make_shared(NodeVector{}, ParameterVector{}); + return std::make_shared(ov::NodeVector{}, ov::ParameterVector{}); } - void normalize(std::shared_ptr function) const override { + void normalize(std::shared_ptr function) const override { m_stat.m_normalize++; } diff --git a/src/bindings/python/tests/runtime.py b/src/bindings/python/tests/runtime.py index a64425d34bc..d263723bb81 100644 --- a/src/bindings/python/tests/runtime.py +++ b/src/bindings/python/tests/runtime.py @@ -83,12 +83,13 @@ class Computation(object): def convert_buffers(self, source_buffers, target_dtypes): converted_buffers = [] for i in range(len(source_buffers)): + k = list(source_buffers)[i] target_dtype = target_dtypes[i] # custom conversion for bf16 if self.results[i].get_output_element_type(0) == Type.bf16: - converted_buffers.append((source_buffers[i].view(np.uint32) >> 16).astype(np.uint16)) + converted_buffers.append((source_buffers[k].view(np.uint32) >> 16).astype(np.uint16)) else: - converted_buffers.append(source_buffers[i].astype(target_dtype)) + converted_buffers.append(source_buffers[k].astype(target_dtype)) return converted_buffers def __call__(self, *input_values: NumericData) -> List[NumericData]: diff --git a/src/bindings/python/tests/test_inference_engine/test_core.py b/src/bindings/python/tests/test_inference_engine/test_core.py index 095dcb7e88d..7ba09f09b15 100644 --- a/src/bindings/python/tests/test_inference_engine/test_core.py +++ b/src/bindings/python/tests/test_inference_engine/test_core.py @@ -24,7 +24,7 @@ def test_compact_api_xml(): model = compile_model(test_net_xml) assert(isinstance(model, ExecutableNetwork)) results = model.infer_new_request({"data": img}) - assert np.argmax(results) == 2 + assert np.argmax(results[list(results)[0]]) == 2 def test_compact_api_onnx(): @@ -33,7 +33,7 @@ def test_compact_api_onnx(): model = compile_model(test_net_onnx) assert(isinstance(model, ExecutableNetwork)) results = model.infer_new_request({"data": img}) - assert np.argmax(results) == 2 + assert np.argmax(results[list(results)[0]]) == 2 def test_core_class(): @@ -53,8 +53,7 @@ def test_core_class(): input_tensor = Tensor(input_data) results = request.infer({"parameter": input_tensor}) - - assert np.allclose(results, expected_output) + assert np.allclose(results[list(results)[0]], expected_output) def test_compile_model(device): diff --git a/src/bindings/python/tests/test_inference_engine/test_executable_network.py b/src/bindings/python/tests/test_inference_engine/test_executable_network.py index 1396cf4a99d..e958f5b3e8a 100644 --- a/src/bindings/python/tests/test_inference_engine/test_executable_network.py +++ b/src/bindings/python/tests/test_inference_engine/test_executable_network.py @@ -231,7 +231,7 @@ def test_infer_new_request_numpy(device): img = read_image() exec_net = ie.compile_model(func, device) res = exec_net.infer_new_request({"data": img}) - assert np.argmax(res) == 2 + assert np.argmax(res[list(res)[0]]) == 2 def test_infer_new_request_tensor_numpy_copy(device): @@ -242,8 +242,8 @@ def test_infer_new_request_tensor_numpy_copy(device): exec_net = ie.compile_model(func, device) res_tensor = exec_net.infer_new_request({"data": tensor}) res_img = exec_net.infer_new_request({"data": tensor}) - assert np.argmax(res_tensor) == 2 - assert np.argmax(res_tensor) == np.argmax(res_img) + assert np.argmax(res_tensor[list(res_tensor)[0]]) == 2 + assert np.argmax(res_tensor[list(res_tensor)[0]]) == np.argmax(res_img[list(res_img)[0]]) def test_infer_tensor_numpy_shared_memory(device): @@ -255,8 +255,8 @@ def test_infer_tensor_numpy_shared_memory(device): exec_net = ie.compile_model(func, device) res_tensor = exec_net.infer_new_request({"data": tensor}) res_img = exec_net.infer_new_request({"data": tensor}) - assert np.argmax(res_tensor) == 2 - assert np.argmax(res_tensor) == np.argmax(res_img) + assert np.argmax(res_tensor[list(res_tensor)[0]]) == 2 + assert np.argmax(res_tensor[list(res_tensor)[0]]) == np.argmax(res_img[list(res_img)[0]]) def test_infer_new_request_wrong_port_name(device): @@ -292,7 +292,7 @@ def test_infer_numpy_model_from_buffer(device): img = read_image() exec_net = core.compile_model(func, device) res = exec_net.infer_new_request({"data": img}) - assert np.argmax(res) == 2 + assert np.argmax(res[list(res)[0]]) == 2 def test_infer_tensor_model_from_buffer(device): @@ -306,4 +306,4 @@ def test_infer_tensor_model_from_buffer(device): tensor = Tensor(img) exec_net = core.compile_model(func, device) res = exec_net.infer_new_request({"data": tensor}) - assert np.argmax(res) == 2 + assert np.argmax(res[list(res)[0]]) == 2 diff --git a/src/bindings/python/tests/test_inference_engine/test_infer_request.py b/src/bindings/python/tests/test_inference_engine/test_infer_request.py index 9376e9f08b8..1fa7ad1ec14 100644 --- a/src/bindings/python/tests/test_inference_engine/test_infer_request.py +++ b/src/bindings/python/tests/test_inference_engine/test_infer_request.py @@ -61,9 +61,10 @@ def test_tensor_setter(device): assert np.allclose(tensor.data, t1.data, atol=1e-2, rtol=1e-2) res = request1.infer({0: tensor}) - res_1 = np.sort(res[0]) + k = list(res)[0] + res_1 = np.sort(res[k]) t2 = request1.get_tensor("fc_out") - assert np.allclose(t2.data, res[0].data, atol=1e-2, rtol=1e-2) + assert np.allclose(t2.data, res[k].data, atol=1e-2, rtol=1e-2) request = exec_net_2.create_infer_request() res = request.infer({"data": tensor}) @@ -187,7 +188,7 @@ def test_infer_mixed_keys(device): request = model.create_infer_request() res = request.infer({0: tensor2, "data": tensor}) - assert np.argmax(res) == 2 + assert np.argmax(res[list(res)[0]]) == 2 def test_infer_queue(device): @@ -304,11 +305,49 @@ def test_query_state_write_buffer(device, input_shape, data_type, mode): # reset initial state of ReadValue to zero mem_state.reset() res = request.infer({0: np.full(input_shape, 1, dtype=data_type)}) - # always ones expected_res = np.full(input_shape, 1, dtype=data_type) else: res = request.infer({0: np.full(input_shape, 1, dtype=data_type)}) expected_res = np.full(input_shape, i, dtype=data_type) - assert np.allclose(res[0], expected_res, atol=1e-6), \ + + assert np.allclose(res[list(res)[0]], expected_res, atol=1e-6), \ "Expected values: {} \n Actual values: {} \n".format(expected_res, res) + + +def test_get_results(device): + core = Core() + func = core.read_model(test_net_xml, test_net_bin) + core.set_config({"PERF_COUNT": "YES"}, device) + exec_net = core.compile_model(func, device) + img = read_image() + request = exec_net.create_infer_request() + outputs = request.infer({0: img}) + assert np.allclose(list(outputs.values()), list(request.results.values())) + + +def test_results_async_infer(device): + jobs = 8 + num_request = 4 + core = Core() + func = core.read_model(test_net_xml, test_net_bin) + exec_net = core.compile_model(func, device) + infer_queue = AsyncInferQueue(exec_net, num_request) + jobs_done = [{"finished": False, "latency": 0} for _ in range(jobs)] + + def callback(request, job_id): + jobs_done[job_id]["finished"] = True + jobs_done[job_id]["latency"] = request.latency + + img = read_image() + infer_queue.set_callback(callback) + assert infer_queue.is_ready + for i in range(jobs): + infer_queue.start_async({"data": img}, i) + infer_queue.wait_all() + + request = exec_net.create_infer_request() + outputs = request.infer({0: img}) + + for i in range(num_request): + np.allclose(list(outputs.values()), list(infer_queue[i].results.values())) diff --git a/src/bindings/python/tests/test_ngraph/test_create_op.py b/src/bindings/python/tests/test_ngraph/test_create_op.py index 8d5592cb20a..35db15f640e 100644 --- a/src/bindings/python/tests/test_ngraph/test_create_op.py +++ b/src/bindings/python/tests/test_ngraph/test_create_op.py @@ -4,6 +4,7 @@ import numpy as np import pytest from openvino.runtime import PartialShape, Dimension +from openvino.runtime.exceptions import UserInputError import openvino.runtime.opset8 as ov import openvino.runtime.opset1 as ov_opset1 @@ -109,7 +110,7 @@ def test_ctc_greedy_decoder(dtype): (np.float64, np.int64, "i64", "i32", False, False), (np.float64, np.int64, "i32", "i64", False, False), (np.float64, np.int64, "i64", "i64", False, False) - ],) + ], ) def test_ctc_greedy_decoder_seq_len(fp_dtype, int_dtype, int_ci, int_sl, merge_repeated, blank_index): input0_shape = [8, 20, 128] input1_shape = [8] @@ -1092,41 +1093,6 @@ def test_prior_box_clustered(int_dtype, fp_dtype): assert list(node.get_output_shape(0)) == [2, 4332] -@pytest.mark.parametrize( - "int_dtype, fp_dtype", - [ - (np.int8, np.float32), - (np.int16, np.float32), - (np.int32, np.float32), - (np.int64, np.float32), - (np.uint8, np.float32), - (np.uint16, np.float32), - (np.uint32, np.float32), - (np.uint64, np.float32), - (np.int32, np.float16), - (np.int32, np.float64), - ], -) -def test_detection_output(int_dtype, fp_dtype): - attributes = { - "num_classes": int_dtype(85), - "keep_top_k": np.array([64], dtype=int_dtype), - "nms_threshold": fp_dtype(0.645), - } - - box_logits = ov.parameter([4, 8], fp_dtype, "box_logits") - class_preds = ov.parameter([4, 170], fp_dtype, "class_preds") - proposals = ov.parameter([4, 2, 10], fp_dtype, "proposals") - aux_class_preds = ov.parameter([4, 4], fp_dtype, "aux_class_preds") - aux_box_preds = ov.parameter([4, 8], fp_dtype, "aux_box_preds") - - node = ov.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) - - assert node.get_type_name() == "DetectionOutput" - assert node.get_output_size() == 1 - assert list(node.get_output_shape(0)) == [1, 1, 256, 7] - - @pytest.mark.parametrize( "int_dtype, fp_dtype", [ @@ -1951,3 +1917,114 @@ def test_slice(): assert node.get_output_size() == 1 assert node.get_output_element_type(0) == Type.f32 assert tuple(node.get_output_shape(0)) == np.zeros(data_shape)[2:9:2, ::, 0:2:1].shape + + +def test_i420_to_bgr(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ov.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ov.i420_to_bgr(arg_single_plane) + + assert node_single_plane.get_type_name() == "I420toBGR" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (three args) + arg_y = ov.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_u = ov.parameter([1, 240, 320, 1], name="input_u", dtype=np.float32) + arg_v = ov.parameter([1, 240, 320, 1], name="input_v", dtype=np.float32) + + node_separate_planes = ov.i420_to_bgr(arg_y, arg_u, arg_v) + + assert node_separate_planes.get_type_name() == "I420toBGR" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + # Incorrect inputs number + with pytest.raises(UserInputError, match=r".*Operation I420toBGR*."): + node_separate_planes = ov.i420_to_bgr(arg_y, arg_v) + + with pytest.raises(UserInputError, match=r".*Operation I420toBGR*."): + node_separate_planes = ov.i420_to_bgr(arg_single_plane, None, arg_v) + + +def test_i420_to_rgb(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ov.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ov.i420_to_rgb(arg_single_plane) + + assert node_single_plane.get_type_name() == "I420toRGB" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (three args) + arg_y = ov.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_u = ov.parameter([1, 240, 320, 1], name="input_u", dtype=np.float32) + arg_v = ov.parameter([1, 240, 320, 1], name="input_v", dtype=np.float32) + + node_separate_planes = ov.i420_to_rgb(arg_y, arg_u, arg_v) + + assert node_separate_planes.get_type_name() == "I420toRGB" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + with pytest.raises(UserInputError, match=r".*Operation I420toRGB*."): + node_separate_planes = ov.i420_to_rgb(arg_y, arg_v) + + with pytest.raises(UserInputError, match=r".*Operation I420toRGB*."): + node_separate_planes = ov.i420_to_rgb(arg_single_plane, None, arg_v) + + +def test_nv12_to_bgr(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ov.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ov.nv12_to_bgr(arg_single_plane) + + assert node_single_plane.get_type_name() == "NV12toBGR" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (two args) + arg_y = ov.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_uv = ov.parameter([1, 240, 320, 2], name="input_uv", dtype=np.float32) + + node_separate_planes = ov.nv12_to_bgr(arg_y, arg_uv) + + assert node_separate_planes.get_type_name() == "NV12toBGR" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + +def test_nv12_to_rgb(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ov.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ov.nv12_to_rgb(arg_single_plane) + + assert node_single_plane.get_type_name() == "NV12toRGB" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (two args) + arg_y = ov.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_uv = ov.parameter([1, 240, 320, 2], name="input_uv", dtype=np.float32) + + node_separate_planes = ov.nv12_to_rgb(arg_y, arg_uv) + + assert node_separate_planes.get_type_name() == "NV12toRGB" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape diff --git a/src/bindings/python/tests/test_ngraph/test_detection_output.py b/src/bindings/python/tests/test_ngraph/test_detection_output.py new file mode 100644 index 00000000000..5534b2ea6f7 --- /dev/null +++ b/src/bindings/python/tests/test_ngraph/test_detection_output.py @@ -0,0 +1,111 @@ +# Copyright (C) 2018-2021 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import numpy as np +import openvino.runtime.opset8 as ov +import pytest + +np_types = [np.float32, np.int32] +integral_np_types = [ + np.int8, + np.int16, + np.int32, + np.int64, + np.uint8, + np.uint16, + np.uint32, + np.uint64, +] + + +@pytest.mark.parametrize( + "int_dtype, fp_dtype", + [ + (np.int8, np.float32), + (np.int16, np.float32), + (np.int32, np.float32), + (np.int64, np.float32), + (np.uint8, np.float32), + (np.uint16, np.float32), + (np.uint32, np.float32), + (np.uint64, np.float32), + (np.int32, np.float16), + (np.int32, np.float64), + ], +) +def test_detection_output(int_dtype, fp_dtype): + attributes = { + "keep_top_k": np.array([64], dtype=int_dtype), + "nms_threshold": fp_dtype(0.645), + } + + box_logits = ov.parameter([4, 8], fp_dtype, "box_logits") + class_preds = ov.parameter([4, 170], fp_dtype, "class_preds") + proposals = ov.parameter([4, 2, 10], fp_dtype, "proposals") + aux_class_preds = ov.parameter([4, 4], fp_dtype, "aux_class_preds") + aux_box_preds = ov.parameter([4, 8], fp_dtype, "aux_box_preds") + + node = ov.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) + + assert node.get_type_name() == "DetectionOutput" + assert node.get_output_size() == 1 + assert list(node.get_output_shape(0)) == [1, 1, 256, 7] + + +@pytest.mark.parametrize( + "int_dtype, fp_dtype", + [ + (np.int8, np.float32), + (np.int16, np.float32), + (np.int32, np.float32), + (np.int64, np.float32), + (np.uint8, np.float32), + (np.uint16, np.float32), + (np.uint32, np.float32), + (np.uint64, np.float32), + (np.int32, np.float16), + (np.int32, np.float64), + ], +) +def test_dynamic_get_attribute_value(int_dtype, fp_dtype): + attributes = { + "background_label_id": int_dtype(13), + "top_k": int_dtype(16), + "variance_encoded_in_target": True, + "keep_top_k": np.array([64, 32, 16, 8], dtype=int_dtype), + "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", + "share_location": False, + "nms_threshold": fp_dtype(0.645), + "confidence_threshold": fp_dtype(0.111), + "clip_after_nms": True, + "clip_before_nms": False, + "decrease_label_id": True, + "normalized": True, + "input_height": int_dtype(86), + "input_width": int_dtype(79), + "objectness_score": fp_dtype(0.77), + } + + box_logits = ov.parameter([4, 680], fp_dtype, "box_logits") + class_preds = ov.parameter([4, 170], fp_dtype, "class_preds") + proposals = ov.parameter([4, 1, 8], fp_dtype, "proposals") + aux_class_preds = ov.parameter([4, 4], fp_dtype, "aux_class_preds") + aux_box_preds = ov.parameter([4, 680], fp_dtype, "aux_box_preds") + + node = ov.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) + + assert node.get_background_label_id() == int_dtype(13) + assert node.get_top_k() == int_dtype(16) + assert node.get_variance_encoded_in_target() + assert np.all(np.equal(node.get_keep_top_k(), np.array([64, 32, 16, 8], dtype=int_dtype))) + assert node.get_code_type() == "caffe.PriorBoxParameter.CENTER_SIZE" + assert not node.get_share_location() + assert np.isclose(node.get_nms_threshold(), fp_dtype(0.645)) + assert np.isclose(node.get_confidence_threshold(), fp_dtype(0.111)) + assert node.get_clip_after_nms() + assert not node.get_clip_before_nms() + assert node.get_decrease_label_id() + assert node.get_normalized() + assert node.get_input_height() == int_dtype(86) + assert node.get_input_width() == int_dtype(79) + assert np.isclose(node.get_objectness_score(), fp_dtype(0.77)) diff --git a/src/bindings/python/tests/test_ngraph/test_dyn_attributes.py b/src/bindings/python/tests/test_ngraph/test_dyn_attributes.py index 3d692f3b140..3a41281500c 100644 --- a/src/bindings/python/tests/test_ngraph/test_dyn_attributes.py +++ b/src/bindings/python/tests/test_ngraph/test_dyn_attributes.py @@ -2,9 +2,8 @@ # SPDX-License-Identifier: Apache-2.0 import numpy as np -import pytest - import openvino.runtime.opset8 as ov +import pytest @pytest.fixture() @@ -37,68 +36,6 @@ def test_dynamic_attributes_softmax(): assert node.get_axis() == 3 -@pytest.mark.parametrize( - "int_dtype, fp_dtype", - [ - (np.int8, np.float32), - (np.int16, np.float32), - (np.int32, np.float32), - (np.int64, np.float32), - (np.uint8, np.float32), - (np.uint16, np.float32), - (np.uint32, np.float32), - (np.uint64, np.float32), - (np.int32, np.float16), - (np.int32, np.float64), - ], -) -def test_dynamic_get_attribute_value(int_dtype, fp_dtype): - attributes = { - "num_classes": int_dtype(85), - "background_label_id": int_dtype(13), - "top_k": int_dtype(16), - "variance_encoded_in_target": True, - "keep_top_k": np.array([64, 32, 16, 8], dtype=int_dtype), - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "share_location": False, - "nms_threshold": fp_dtype(0.645), - "confidence_threshold": fp_dtype(0.111), - "clip_after_nms": True, - "clip_before_nms": False, - "decrease_label_id": True, - "normalized": True, - "input_height": int_dtype(86), - "input_width": int_dtype(79), - "objectness_score": fp_dtype(0.77), - } - - box_logits = ov.parameter([4, 680], fp_dtype, "box_logits") - class_preds = ov.parameter([4, 170], fp_dtype, "class_preds") - proposals = ov.parameter([4, 1, 8], fp_dtype, "proposals") - aux_class_preds = ov.parameter([4, 4], fp_dtype, "aux_class_preds") - aux_box_preds = ov.parameter([4, 680], fp_dtype, "aux_box_preds") - - node = ov.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) - - assert node.get_num_classes() == int_dtype(85) - assert node.get_background_label_id() == int_dtype(13) - assert node.get_top_k() == int_dtype(16) - assert node.get_variance_encoded_in_target() - assert np.all(np.equal(node.get_keep_top_k(), np.array([64, 32, 16, 8], dtype=int_dtype))) - assert node.get_code_type() == "caffe.PriorBoxParameter.CENTER_SIZE" - assert not node.get_share_location() - assert np.isclose(node.get_nms_threshold(), fp_dtype(0.645)) - assert np.isclose(node.get_confidence_threshold(), fp_dtype(0.111)) - assert node.get_clip_after_nms() - assert not node.get_clip_before_nms() - assert node.get_decrease_label_id() - assert node.get_normalized() - assert node.get_input_height() == int_dtype(86) - assert node.get_input_width() == int_dtype(79) - assert np.isclose(node.get_objectness_score(), fp_dtype(0.77)) - assert node.get_num_classes() == int_dtype(85) - - @pytest.mark.parametrize( "int_dtype, fp_dtype", [ diff --git a/src/bindings/python/tests_compatibility/test_ngraph/test_create_op.py b/src/bindings/python/tests_compatibility/test_ngraph/test_create_op.py index c5d97de1753..ceeb71c126d 100644 --- a/src/bindings/python/tests_compatibility/test_ngraph/test_create_op.py +++ b/src/bindings/python/tests_compatibility/test_ngraph/test_create_op.py @@ -8,6 +8,7 @@ from _pyngraph import PartialShape, Dimension import ngraph as ng import ngraph.opset1 as ng_opset1 import ngraph.opset5 as ng_opset5 +from ngraph.exceptions import UserInputError from ngraph.impl import Type np_types = [np.float32, np.int32] @@ -109,7 +110,7 @@ def test_ctc_greedy_decoder(dtype): (np.float64, np.int64, "i64", "i32", False, False), (np.float64, np.int64, "i32", "i64", False, False), (np.float64, np.int64, "i64", "i64", False, False) - ],) + ], ) def test_ctc_greedy_decoder_seq_len(fp_dtype, int_dtype, int_ci, int_sl, merge_repeated, blank_index): input0_shape = [8, 20, 128] input1_shape = [8] @@ -1092,41 +1093,6 @@ def test_prior_box_clustered(int_dtype, fp_dtype): assert list(node.get_output_shape(0)) == [2, 4332] -@pytest.mark.parametrize( - "int_dtype, fp_dtype", - [ - (np.int8, np.float32), - (np.int16, np.float32), - (np.int32, np.float32), - (np.int64, np.float32), - (np.uint8, np.float32), - (np.uint16, np.float32), - (np.uint32, np.float32), - (np.uint64, np.float32), - (np.int32, np.float16), - (np.int32, np.float64), - ], -) -def test_detection_output(int_dtype, fp_dtype): - attributes = { - "num_classes": int_dtype(85), - "keep_top_k": np.array([64], dtype=int_dtype), - "nms_threshold": fp_dtype(0.645), - } - - box_logits = ng.parameter([4, 8], fp_dtype, "box_logits") - class_preds = ng.parameter([4, 170], fp_dtype, "class_preds") - proposals = ng.parameter([4, 2, 10], fp_dtype, "proposals") - aux_class_preds = ng.parameter([4, 4], fp_dtype, "aux_class_preds") - aux_box_preds = ng.parameter([4, 8], fp_dtype, "aux_box_preds") - - node = ng.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) - - assert node.get_type_name() == "DetectionOutput" - assert node.get_output_size() == 1 - assert list(node.get_output_shape(0)) == [1, 1, 256, 7] - - @pytest.mark.parametrize( "int_dtype, fp_dtype", [ @@ -1951,3 +1917,114 @@ def test_slice(): assert node.get_output_size() == 1 assert node.get_output_element_type(0) == Type.f32 assert tuple(node.get_output_shape(0)) == np.zeros(data_shape)[2:9:2, ::, 0:2:1].shape + + +def test_i420_to_bgr(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ng.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ng.i420_to_bgr(arg_single_plane) + + assert node_single_plane.get_type_name() == "I420toBGR" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (three args) + arg_y = ng.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_u = ng.parameter([1, 240, 320, 1], name="input_u", dtype=np.float32) + arg_v = ng.parameter([1, 240, 320, 1], name="input_v", dtype=np.float32) + + node_separate_planes = ng.i420_to_bgr(arg_y, arg_u, arg_v) + + assert node_separate_planes.get_type_name() == "I420toBGR" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + # Incorrect inputs number + with pytest.raises(UserInputError, match=r".*Operation I420toBGR*."): + node_separate_planes = ng.i420_to_bgr(arg_y, arg_v) + + with pytest.raises(UserInputError, match=r".*Operation I420toBGR*."): + node_separate_planes = ng.i420_to_bgr(arg_single_plane, None, arg_v) + + +def test_i420_to_rgb(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ng.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ng.i420_to_rgb(arg_single_plane) + + assert node_single_plane.get_type_name() == "I420toRGB" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (three args) + arg_y = ng.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_u = ng.parameter([1, 240, 320, 1], name="input_u", dtype=np.float32) + arg_v = ng.parameter([1, 240, 320, 1], name="input_v", dtype=np.float32) + + node_separate_planes = ng.i420_to_rgb(arg_y, arg_u, arg_v) + + assert node_separate_planes.get_type_name() == "I420toRGB" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + with pytest.raises(UserInputError, match=r".*Operation I420toRGB*."): + node_separate_planes = ng.i420_to_rgb(arg_y, arg_v) + + with pytest.raises(UserInputError, match=r".*Operation I420toRGB*."): + node_separate_planes = ng.i420_to_rgb(arg_single_plane, None, arg_v) + + +def test_nv12_to_bgr(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ng.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ng.nv12_to_bgr(arg_single_plane) + + assert node_single_plane.get_type_name() == "NV12toBGR" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (two args) + arg_y = ng.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_uv = ng.parameter([1, 240, 320, 2], name="input_uv", dtype=np.float32) + + node_separate_planes = ng.nv12_to_bgr(arg_y, arg_uv) + + assert node_separate_planes.get_type_name() == "NV12toBGR" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape + + +def test_nv12_to_rgb(): + expected_output_shape = [1, 480, 640, 3] + + # # Single plane (one arg) + arg_single_plane = ng.parameter([1, 720, 640, 1], name="input", dtype=np.float32) + node_single_plane = ng.nv12_to_rgb(arg_single_plane) + + assert node_single_plane.get_type_name() == "NV12toRGB" + assert node_single_plane.get_output_size() == 1 + assert node_single_plane.get_output_element_type(0) == Type.f32 + assert list(node_single_plane.get_output_shape(0)) == expected_output_shape + + # Separate planes (two args) + arg_y = ng.parameter([1, 480, 640, 1], name="input_y", dtype=np.float32) + arg_uv = ng.parameter([1, 240, 320, 2], name="input_uv", dtype=np.float32) + + node_separate_planes = ng.nv12_to_rgb(arg_y, arg_uv) + + assert node_separate_planes.get_type_name() == "NV12toRGB" + assert node_separate_planes.get_output_size() == 1 + assert node_separate_planes.get_output_element_type(0) == Type.f32 + assert list(node_separate_planes.get_output_shape(0)) == expected_output_shape diff --git a/src/bindings/python/tests_compatibility/test_ngraph/test_detection_output.py b/src/bindings/python/tests_compatibility/test_ngraph/test_detection_output.py new file mode 100644 index 00000000000..83082d33d0c --- /dev/null +++ b/src/bindings/python/tests_compatibility/test_ngraph/test_detection_output.py @@ -0,0 +1,111 @@ +# Copyright (C) 2018-2021 Intel Corporation +# SPDX-License-Identifier: Apache-2.0 + +import ngraph as ng +import numpy as np +import pytest + +np_types = [np.float32, np.int32] +integral_np_types = [ + np.int8, + np.int16, + np.int32, + np.int64, + np.uint8, + np.uint16, + np.uint32, + np.uint64, +] + + +@pytest.mark.parametrize( + "int_dtype, fp_dtype", + [ + (np.int8, np.float32), + (np.int16, np.float32), + (np.int32, np.float32), + (np.int64, np.float32), + (np.uint8, np.float32), + (np.uint16, np.float32), + (np.uint32, np.float32), + (np.uint64, np.float32), + (np.int32, np.float16), + (np.int32, np.float64), + ], +) +def test_detection_output(int_dtype, fp_dtype): + attributes = { + "keep_top_k": np.array([64], dtype=int_dtype), + "nms_threshold": fp_dtype(0.645), + } + + box_logits = ng.parameter([4, 8], fp_dtype, "box_logits") + class_preds = ng.parameter([4, 170], fp_dtype, "class_preds") + proposals = ng.parameter([4, 2, 10], fp_dtype, "proposals") + aux_class_preds = ng.parameter([4, 4], fp_dtype, "aux_class_preds") + aux_box_preds = ng.parameter([4, 8], fp_dtype, "aux_box_preds") + + node = ng.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) + + assert node.get_type_name() == "DetectionOutput" + assert node.get_output_size() == 1 + assert list(node.get_output_shape(0)) == [1, 1, 256, 7] + + +@pytest.mark.parametrize( + "int_dtype, fp_dtype", + [ + (np.int8, np.float32), + (np.int16, np.float32), + (np.int32, np.float32), + (np.int64, np.float32), + (np.uint8, np.float32), + (np.uint16, np.float32), + (np.uint32, np.float32), + (np.uint64, np.float32), + (np.int32, np.float16), + (np.int32, np.float64), + ], +) +def test_dynamic_get_attribute_value(int_dtype, fp_dtype): + attributes = { + "background_label_id": int_dtype(13), + "top_k": int_dtype(16), + "variance_encoded_in_target": True, + "keep_top_k": np.array([64, 32, 16, 8], dtype=int_dtype), + "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", + "share_location": False, + "nms_threshold": fp_dtype(0.645), + "confidence_threshold": fp_dtype(0.111), + "clip_after_nms": True, + "clip_before_nms": False, + "decrease_label_id": True, + "normalized": True, + "input_height": int_dtype(86), + "input_width": int_dtype(79), + "objectness_score": fp_dtype(0.77), + } + + box_logits = ng.parameter([4, 680], fp_dtype, "box_logits") + class_preds = ng.parameter([4, 170], fp_dtype, "class_preds") + proposals = ng.parameter([4, 1, 8], fp_dtype, "proposals") + aux_class_preds = ng.parameter([4, 4], fp_dtype, "aux_class_preds") + aux_box_preds = ng.parameter([4, 680], fp_dtype, "aux_box_preds") + + node = ng.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) + + assert node.get_background_label_id() == int_dtype(13) + assert node.get_top_k() == int_dtype(16) + assert node.get_variance_encoded_in_target() + assert np.all(np.equal(node.get_keep_top_k(), np.array([64, 32, 16, 8], dtype=int_dtype))) + assert node.get_code_type() == "caffe.PriorBoxParameter.CENTER_SIZE" + assert not node.get_share_location() + assert np.isclose(node.get_nms_threshold(), fp_dtype(0.645)) + assert np.isclose(node.get_confidence_threshold(), fp_dtype(0.111)) + assert node.get_clip_after_nms() + assert not node.get_clip_before_nms() + assert node.get_decrease_label_id() + assert node.get_normalized() + assert node.get_input_height() == int_dtype(86) + assert node.get_input_width() == int_dtype(79) + assert np.isclose(node.get_objectness_score(), fp_dtype(0.77)) diff --git a/src/bindings/python/tests_compatibility/test_ngraph/test_dyn_attributes.py b/src/bindings/python/tests_compatibility/test_ngraph/test_dyn_attributes.py index a945ec91ef0..34adade060e 100644 --- a/src/bindings/python/tests_compatibility/test_ngraph/test_dyn_attributes.py +++ b/src/bindings/python/tests_compatibility/test_ngraph/test_dyn_attributes.py @@ -1,11 +1,10 @@ # Copyright (C) 2018-2021 Intel Corporation # SPDX-License-Identifier: Apache-2.0 +import ngraph as ng import numpy as np import pytest -import ngraph as ng - @pytest.fixture() def _proposal_node(): @@ -37,68 +36,6 @@ def test_dynamic_attributes_softmax(): assert node.get_axis() == 3 -@pytest.mark.parametrize( - "int_dtype, fp_dtype", - [ - (np.int8, np.float32), - (np.int16, np.float32), - (np.int32, np.float32), - (np.int64, np.float32), - (np.uint8, np.float32), - (np.uint16, np.float32), - (np.uint32, np.float32), - (np.uint64, np.float32), - (np.int32, np.float16), - (np.int32, np.float64), - ], -) -def test_dynamic_get_attribute_value(int_dtype, fp_dtype): - attributes = { - "num_classes": int_dtype(85), - "background_label_id": int_dtype(13), - "top_k": int_dtype(16), - "variance_encoded_in_target": True, - "keep_top_k": np.array([64, 32, 16, 8], dtype=int_dtype), - "code_type": "caffe.PriorBoxParameter.CENTER_SIZE", - "share_location": False, - "nms_threshold": fp_dtype(0.645), - "confidence_threshold": fp_dtype(0.111), - "clip_after_nms": True, - "clip_before_nms": False, - "decrease_label_id": True, - "normalized": True, - "input_height": int_dtype(86), - "input_width": int_dtype(79), - "objectness_score": fp_dtype(0.77), - } - - box_logits = ng.parameter([4, 680], fp_dtype, "box_logits") - class_preds = ng.parameter([4, 170], fp_dtype, "class_preds") - proposals = ng.parameter([4, 1, 8], fp_dtype, "proposals") - aux_class_preds = ng.parameter([4, 4], fp_dtype, "aux_class_preds") - aux_box_preds = ng.parameter([4, 680], fp_dtype, "aux_box_preds") - - node = ng.detection_output(box_logits, class_preds, proposals, attributes, aux_class_preds, aux_box_preds) - - assert node.get_num_classes() == int_dtype(85) - assert node.get_background_label_id() == int_dtype(13) - assert node.get_top_k() == int_dtype(16) - assert node.get_variance_encoded_in_target() - assert np.all(np.equal(node.get_keep_top_k(), np.array([64, 32, 16, 8], dtype=int_dtype))) - assert node.get_code_type() == "caffe.PriorBoxParameter.CENTER_SIZE" - assert not node.get_share_location() - assert np.isclose(node.get_nms_threshold(), fp_dtype(0.645)) - assert np.isclose(node.get_confidence_threshold(), fp_dtype(0.111)) - assert node.get_clip_after_nms() - assert not node.get_clip_before_nms() - assert node.get_decrease_label_id() - assert node.get_normalized() - assert node.get_input_height() == int_dtype(86) - assert node.get_input_width() == int_dtype(79) - assert np.isclose(node.get_objectness_score(), fp_dtype(0.77)) - assert node.get_num_classes() == int_dtype(85) - - @pytest.mark.parametrize( "int_dtype, fp_dtype", [ diff --git a/src/common/legacy/src/convert_function_to_cnn_network.cpp b/src/common/legacy/src/convert_function_to_cnn_network.cpp index ddc4ae231d3..3e05992d0d2 100644 --- a/src/common/legacy/src/convert_function_to_cnn_network.cpp +++ b/src/common/legacy/src/convert_function_to_cnn_network.cpp @@ -1629,7 +1629,7 @@ CNNLayerCreator::CNNLayerCreator(const std::shared_ptr<::ngraph::Node>& node): n << " attribute is set in " << node->get_friendly_name() << " node"; } - auto getStringValue = [] (const std::shared_ptr & variant) { + auto getStringValue = [] (const ov::Any& variant) { auto castedVariant = std::dynamic_pointer_cast>(variant); IE_ASSERT(castedVariant != nullptr); return castedVariant->get(); @@ -1932,13 +1932,13 @@ void convertFunctionToICNNNetwork(const std::shared_ptr type - using VariantString = ::ngraph::VariantWrapper; for (const auto &rt : rt_info) { - if (auto str_attr = std::dynamic_pointer_cast(rt.second)) { + if (rt.second.is()) { + auto str_attr = rt.second.as(); if (details::CaselessEq()(rt.first, "affinity")) { - cnnLayer->affinity = str_attr->get(); + cnnLayer->affinity = str_attr; } else { - cnnLayer->params[rt.first] = str_attr->get(); + cnnLayer->params[rt.first] = str_attr; } } } diff --git a/src/common/low_precision_transformations/include/low_precision/create_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/create_attribute.hpp index 819cd11b430..c7b7c468826 100644 --- a/src/common/low_precision_transformations/include/low_precision/create_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/create_attribute.hpp @@ -47,8 +47,8 @@ public: } { OV_ITT_SCOPE(FIRST_INFERENCE, itt::domains::LPT_LT, "CreateAttribute"); - const auto attribute = ngraph::VariantWrapper::create(op, params); - if (attribute == nullptr) { + const auto attribute = AttributeType::create(op, params); + if (attribute.empty()) { return false; } } diff --git a/src/common/low_precision_transformations/include/low_precision/create_precisions_dependent_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/create_precisions_dependent_attribute.hpp index 4104d646e23..2e05cc85761 100644 --- a/src/common/low_precision_transformations/include/low_precision/create_precisions_dependent_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/create_precisions_dependent_attribute.hpp @@ -45,20 +45,18 @@ public: OV_ITT_SCOPE(FIRST_INFERENCE, itt::domains::LPT_LT, "CreatePrecisionsDependentAttribute"); auto &rt = node->get_rt_info(); - const auto precisionPreservedAttribute = std::make_shared>( - std::make_shared(false)); - rt[ngraph::VariantWrapper::type_info.name] = precisionPreservedAttribute; - const auto &targetSharedValue = precisionPreservedAttribute->get()->sharedValue; + const auto precisionPreservedAttribute = PrecisionPreservedAttribute(false); + rt[PrecisionPreservedAttribute::get_type_info_static()] = precisionPreservedAttribute; + const auto &targetSharedValue = precisionPreservedAttribute.attribute->sharedValue; - const auto attribute = std::make_shared>>( - std::make_shared()); - rt[ngraph::VariantWrapper>::type_info.name] = attribute; + const auto attribute = AttributeType{}; + rt[AttributeType::get_type_info_static()] = attribute; - ngraph::pass::low_precision::NetworkHelper::reassign( + ngraph::pass::low_precision::NetworkHelper::reassign( targetSharedValue, { - std::dynamic_pointer_cast(attribute->get()), - std::dynamic_pointer_cast(precisionPreservedAttribute->get()) + attribute.attribute, + precisionPreservedAttribute.attribute }); } return true; diff --git a/src/common/low_precision_transformations/include/low_precision/network_helper.hpp b/src/common/low_precision_transformations/include/low_precision/network_helper.hpp index 9da92e72631..dc040b7cb2c 100644 --- a/src/common/low_precision_transformations/include/low_precision/network_helper.hpp +++ b/src/common/low_precision_transformations/include/low_precision/network_helper.hpp @@ -218,8 +218,8 @@ public: static void replaceAttributeInNodes( std::shared_ptr f, const std::string& name, - const std::shared_ptr newAttribute, - const std::shared_ptr oldAttribute, + const ov::Any& newAttribute, + const ov::Any& oldAttribute, const std::shared_ptr& initialNode) { std::set> visited; std::deque> nodes; @@ -246,7 +246,7 @@ public: auto it = rt.find(name); if (it != rt.end()) { const auto currentAttribute = it->second; - if (oldAttribute == as_type_ptr(currentAttribute)) { + if (oldAttribute == currentAttribute) { rt[name] = newAttribute; } handleConnectedNodes = true; @@ -296,10 +296,10 @@ public: } } - template + template static void reassign( - const std::shared_ptr& sharedValue, - const std::vector>& attributes) { + const std::shared_ptr& sharedValue, + const std::vector>& attributes) { for (const auto attributeWeakPtr : attributes) { auto attribute = attributeWeakPtr.lock(); if (attribute == nullptr) { @@ -415,53 +415,37 @@ std::shared_ptr fold_reshape(Args&&... args) { } template -std::shared_ptr> getAttribute(const std::shared_ptr& inputNode) { - auto& rt = inputNode->get_rt_info(); - auto it = rt.find(ngraph::VariantWrapper::type_info.name); +ov::Any getAttribute(const std::shared_ptr& node) { + auto& rt = node->get_rt_info(); + auto it = rt.find(T::get_type_info_static()); if (it == rt.end()) { - return nullptr; + return {}; } - - auto attribute = std::dynamic_pointer_cast>(it->second); - assert(attribute != nullptr); - return attribute; + return it->second; } template -std::shared_ptr> getAttribute(const Input& input) { +ov::Any getAttribute(const Input& input) { auto& rt = input.get_rt_info(); - auto it = rt.find(ngraph::VariantWrapper::type_info.name); + auto it = rt.find(T::get_type_info_static()); if (it == rt.end()) { - return nullptr; + return {}; } - - auto attribute = std::dynamic_pointer_cast>(it->second); - assert(attribute != nullptr); - return attribute; + return it->second; } template -std::shared_ptr> getAttributeFromOutput(const Output& output) { +ov::Any getAttributeFromOutput(const Output& output) { auto& rt = output.get_rt_info(); - auto it = rt.find(ngraph::VariantWrapper::type_info.name); + auto it = rt.find(T::get_type_info_static()); if (it == rt.end()) { - return nullptr; + return {}; } - - auto attribute = std::dynamic_pointer_cast>(it->second); - assert(attribute != nullptr); - return attribute; + return it->second; } bool isDisabled(const std::shared_ptr& node); -template -std::shared_ptr make_shared_attribute(Args&& ... args) { - std::shared_ptr attribute = std::make_shared(std::forward(args)...); - attribute->sharedValue->attributes.push_back(attribute); - return attribute; -} - } // namespace low_precision } // namespace pass } // namespace ngraph diff --git a/src/common/low_precision_transformations/include/low_precision/propagate_shared_value.hpp b/src/common/low_precision_transformations/include/low_precision/propagate_shared_value.hpp index 6a83a74089a..0be0cf20fa2 100644 --- a/src/common/low_precision_transformations/include/low_precision/propagate_shared_value.hpp +++ b/src/common/low_precision_transformations/include/low_precision/propagate_shared_value.hpp @@ -39,11 +39,7 @@ public: if (ov::is_type(node)) { assert(node->get_output_size() == 1ul); auto& outputRtInfo = node->output(0).get_rt_info(); - - auto attribute = make_shared_attribute(std::set{element::u8, element::i8}); - - auto attributeWrapper = std::make_shared>>(attribute); - outputRtInfo[ngraph::VariantWrapper>::type_info.name] = attributeWrapper; + outputRtInfo[AttributeType::get_type_info_static()] = AttributeType{std::set{element::u8, element::i8}}; continue; } @@ -52,17 +48,16 @@ public: auto parentNode = input.get_source_output().get_node_shared_ptr(); auto getAttributes = [](const Input& nodeInput) { - const std::string name = ngraph::VariantWrapper>::type_info.name; + const std::string name = PrecisionsAttribute::get_type_info_static(); auto node = nodeInput.get_source_output().get_node_shared_ptr(); - std::vector>>> attributes; + std::vector attributes; if (ov::is_type(node)) { // output auto& rt = nodeInput.get_source_output().get_rt_info(); auto it = rt.find(name); if (it != rt.end()) { - const auto& attribute = std::dynamic_pointer_cast>>(it->second); - attributes.push_back(attribute); + attributes.push_back(it->second); } } @@ -71,14 +66,13 @@ public: auto& nodeRt = input.get_rt_info(); - const std::string name = ngraph::VariantWrapper>::type_info.name; + const std::string name = PrecisionsAttribute::get_type_info_static(); const auto it = nodeRt.find(name); if (it == nodeRt.end()) { continue; } - const auto& attribute = std::dynamic_pointer_cast>>(it->second); - std::vector>>> attributes{ attribute }; + std::vector attributes{ it->second }; auto parentAttributes = getAttributes(input); if (parentAttributes.empty()) { @@ -86,7 +80,7 @@ public: } for (auto& parentAttribute : parentAttributes) { - parentAttribute->merge(attributes); + parentAttribute.as.merge(attributes); } nodeRt[name] = parentAttributes[0]; @@ -100,9 +94,9 @@ public: } private: - std::vector>>> getParentInputRestrictions( + std::vector getParentInputRestrictions( const std::shared_ptr node) { - std::vector>>> parentAttributes; + std::vector parentAttributes; for (size_t index = 0ul; index < node->get_input_size(); index++) { const Input& input = node->input(index); auto inputNode = input.get_source_output().get_node()->shared_from_this(); @@ -110,23 +104,21 @@ private: const auto dequantization = NetworkHelper::getDequantization(node, index); if (!dequantization.empty() && (ov::is_type(dequantization.data.get_node())) && - ov::is_type(dequantization.data.get_node()->get_input_node_ptr(0))) { + ov::is_type(dequantization.data.get_node()->get_input_node_ptr(0))) { inputNode = dequantization.data.get_node()->get_input_node_shared_ptr(0); } if (NetworkHelper::isPrecisionPreserved(inputNode)) { auto& inputRtInfo = inputNode->get_rt_info(); - auto inputAttributeIt = inputRtInfo.find(ngraph::VariantWrapper>::type_info.name); + auto inputAttributeIt = inputRtInfo.find(PrecisionsAttribute::get_type_info_static()); if (inputAttributeIt != inputRtInfo.end()) { - const auto attribute = std::dynamic_pointer_cast>>(inputAttributeIt->second); - parentAttributes.push_back(attribute); + parentAttributes.push_back(inputAttributeIt->second); } } else if (ov::is_type(inputNode)) { const auto& outputPortRtInfo = inputNode->outputs()[0].get_rt_info(); - auto attributeIt = outputPortRtInfo.find(ngraph::VariantWrapper>::type_info.name); + auto attributeIt = outputPortRtInfo.find(PrecisionsAttribute::get_type_info_static()); if (attributeIt != outputPortRtInfo.end()) { - const auto attribute = std::dynamic_pointer_cast>>(attributeIt->second); - parentAttributes.push_back(attribute); + parentAttributes.push_back(attributeIt->second); } } } @@ -145,19 +137,18 @@ private: // merge parent inputs to one current output auto resultAttribute = parentRestrictions[0]; - std::vector>>> toMerge = parentRestrictions; + std::vector toMerge = parentRestrictions; toMerge.erase(toMerge.begin()); - resultAttribute->merge(toMerge); + resultAttribute.as().merge(toMerge); for (size_t index = 1ul; index < parentRestrictions.size(); index++) { - const auto oldAttribute = parentRestrictions[index]->get(); - NetworkHelper::reassign( - resultAttribute->get()->sharedValue, - parentRestrictions[index]->get()->sharedValue->attributes); + NetworkHelper::reassign( + resultAttribute.as().attribute->sharedValue, + parentRestrictions[index].as().attribute->sharedValue->attributes); } auto& rt = node->get_rt_info(); - rt[ngraph::VariantWrapper>::type_info.name] = resultAttribute; + rt[PrecisionsAttribute::get_type_info_static()] = resultAttribute; } } }; diff --git a/src/common/low_precision_transformations/include/low_precision/propagate_through_precision_preserved.hpp b/src/common/low_precision_transformations/include/low_precision/propagate_through_precision_preserved.hpp index bfa0e1fcceb..844e23bfb95 100644 --- a/src/common/low_precision_transformations/include/low_precision/propagate_through_precision_preserved.hpp +++ b/src/common/low_precision_transformations/include/low_precision/propagate_through_precision_preserved.hpp @@ -49,27 +49,27 @@ public: return false; } - auto resultAttribute = parentRestrictions[0]; + auto& resultAttribute = parentRestrictions[0].template as(); - std::vector>>> toMerge = parentRestrictions; - // TODO: LPT: handle pointer on itself in VariantWrapper::merge and remove erase, task #59498 + std::vector toMerge = parentRestrictions; + // TODO: LPT: handle pointer on itself in IntervalsAlignmentAttribute::merge and remove erase, task #59498 toMerge.erase(toMerge.begin()); - resultAttribute->merge(toMerge); + const_cast(resultAttribute).merge(toMerge); for (size_t index = 1ul; index < parentRestrictions.size(); index++) { - const auto attributes = parentRestrictions[index]->get()->sharedValue->attributes; - for (const auto attributeWeakPtr : attributes) { + auto& attributes = parentRestrictions[index].template as().attribute->sharedValue->attributes; + for (auto&& attributeWeakPtr : attributes) { auto attribute = attributeWeakPtr.lock(); if (attribute == nullptr) { continue; } - attribute->sharedValue = resultAttribute->get()->sharedValue; - resultAttribute->get()->sharedValue->attributes.push_back(attribute); + attribute->sharedValue = resultAttribute.attribute->sharedValue; + resultAttribute.attribute->sharedValue->attributes.push_back(attribute); } } auto &rt = node->get_rt_info(); - rt[ngraph::VariantWrapper>::type_info.name] = resultAttribute; + rt[AttributeType::get_type_info_static()] = resultAttribute; } return true; }; @@ -79,20 +79,20 @@ public: } private: - std::shared_ptr>> getSourceOutputAttribute(const Input& input) { + ov::Any getSourceOutputAttribute(const Input& input) { auto input2 = input; auto output = input2.get_source_output(); - std::shared_ptr>> attribute = getAttributeFromOutput>(output); - if (attribute == nullptr) { - attribute = getAttribute>(output.get_node_shared_ptr()); + auto attribute = getAttributeFromOutput(output); + if (attribute.empty()) { + attribute = getAttribute(output.get_node_shared_ptr()); } return attribute; } // TODO: possible duplicate: PropagateToInput::getSourceOutputAttribute - std::vector>>> getParentInputRestrictions( + std::vector getParentInputRestrictions( const std::shared_ptr node) { - std::vector>>> parentAttributes; + std::vector parentAttributes; auto getInput = [](const std::shared_ptr& node, const size_t index) -> Input { const auto dequantization = NetworkHelper::getDequantization(node, index); if (!dequantization.empty() && @@ -108,7 +108,7 @@ private: for (size_t index = 0ul; index < node->get_input_size(); index++) { const Input& input = getInput(node, index); const auto attribute = getSourceOutputAttribute(input); - if (attribute != nullptr) { + if (!attribute.empty()) { parentAttributes.push_back(attribute); } } diff --git a/src/common/low_precision_transformations/include/low_precision/propagate_to_input.hpp b/src/common/low_precision_transformations/include/low_precision/propagate_to_input.hpp index 64268e9270b..42f840c1573 100644 --- a/src/common/low_precision_transformations/include/low_precision/propagate_to_input.hpp +++ b/src/common/low_precision_transformations/include/low_precision/propagate_to_input.hpp @@ -45,18 +45,19 @@ public: continue; } - auto attribute = getAttribute>(input); - if (attribute != nullptr) { - if ((attribute->get()->sharedValue != nullptr) && (attribute->get()->sharedValue->precisions.empty())) { + auto attribute = getAttribute(input); + if (!attribute.empty()) { + if ((attribute.template as().attribute->sharedValue != nullptr) && + (attribute.template as().value().empty())) { return false; } - std::vector>>> attributes = { attribute }; - parentAttribute->merge(attributes); + std::vector attributes = { attribute }; + parentAttribute.template as().merge(attributes); } auto& rt = input.get_rt_info(); - rt[ngraph::VariantWrapper>::type_info.name] = parentAttribute; + rt[AttributeType::get_type_info_static()] = parentAttribute; } } return true; @@ -68,7 +69,7 @@ public: private: // TODO: possible duplicate: PropagateThroughPrecisionPreserved::getParentInputRestrictions - std::shared_ptr>> getSourceOutputAttribute(const Input& input) { + ov::Any getSourceOutputAttribute(const Input& input) { auto getInput = [](const Input& input) { const auto dequantization = NetworkHelper::getDequantization(input.get_node()->shared_from_this(), input.get_index()); if (!dequantization.empty() && @@ -83,20 +84,20 @@ private: auto input2 = getInput(input); auto output = input2.get_source_output(); - std::shared_ptr>> attribute = getAttributeFromOutput>(output); - if (attribute == nullptr) { - attribute = getAttribute>(output.get_node_shared_ptr()); + auto attribute = getAttributeFromOutput(output); + if (attribute.empty()) { + attribute = getAttribute(output.get_node_shared_ptr()); } return attribute; } - std::vector>>> getParentInputRestrictions( + std::vector getParentInputRestrictions( const std::shared_ptr node) { - std::vector>>> parentAttributes; + std::vector parentAttributes; for (size_t index = 0ul; index < node->get_input_size(); index++) { const Input& input = node->input(index); const auto attribute = getSourceOutputAttribute(input); - if (attribute != nullptr) { + if (!attribute.empty()) { parentAttributes.push_back(attribute); } } diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/avg_pool_precision_preserved_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/avg_pool_precision_preserved_attribute.hpp index 1e8794c5d5b..9da87cba5ce 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/avg_pool_precision_preserved_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/avg_pool_precision_preserved_attribute.hpp @@ -14,28 +14,11 @@ namespace ngraph { class LP_TRANSFORMATIONS_API AvgPoolPrecisionPreservedAttribute : public PrecisionPreservedAttribute { -}; - -using AvgPoolPrecisionPreservedAttributePtr = std::shared_ptr; -} // namespace ngraph - -namespace ov { -extern template class LP_TRANSFORMATIONS_API VariantImpl; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper : public VariantImpl { public: - static constexpr VariantTypeInfo type_info{ "LowPrecision::AvgPoolPrecisionPreserved", 0 }; - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - ngraph::AvgPoolPrecisionPreservedAttributePtr get() { return this->m_value; } - - void merge(std::vector>>>& attributes); - std::string to_string() override; + OPENVINO_RTTI("LowPrecision::AvgPoolPrecisionPreserved", "", ov::RuntimeAttribute, 0); + using PrecisionPreservedAttribute::PrecisionPreservedAttribute; + void merge(std::vector& attributes); + std::string to_string() const override; }; -} // namespace ov + +} // namespace ngraph diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/intervals_alignment_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/intervals_alignment_attribute.hpp index 964989a04ec..af8664110d3 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/intervals_alignment_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/intervals_alignment_attribute.hpp @@ -15,16 +15,14 @@ #include "low_precision/lpt_visibility.hpp" namespace ngraph { -class IntervalsAlignmentAttribute; - -class LP_TRANSFORMATIONS_API IntervalsAlignmentSharedValue : public SharedValue { +class LP_TRANSFORMATIONS_API IntervalsAlignmentSharedValue { public: class Interval { public: Interval() = default; Interval(const float low, const float high) : low(low), high(high) {} - float low; - float high; + float low = 0.f; + float high = 0.f; }; IntervalsAlignmentSharedValue() = default; @@ -38,7 +36,7 @@ public: Interval combinedInterval; Interval minInterval; - size_t minLevels; + size_t minLevels = 0; // preferable precisions which are preferred by affected quantization operations to avoid zero points std::set preferablePrecisions; @@ -47,8 +45,9 @@ public: #endif }; -class LP_TRANSFORMATIONS_API IntervalsAlignmentAttribute : public SharedValueAttribute { +class LP_TRANSFORMATIONS_API IntervalsAlignmentAttribute : public SharedAttribute { public: + OPENVINO_RTTI("LowPrecision::IntervalsAlignment", "", ov::RuntimeAttribute, 0); IntervalsAlignmentAttribute() = default; IntervalsAlignmentAttribute(IntervalsAlignmentSharedValue::Interval combinedInterval, size_t levels); IntervalsAlignmentAttribute( @@ -57,36 +56,14 @@ public: const IntervalsAlignmentSharedValue::Interval minInterval, const size_t minLevels); + static ov::Any create( + const std::shared_ptr& node, + const AttributeParameters& params); + void merge(std::vector& attributes); + std::string to_string() const override; + // specify subgraph original levels size_t levels; }; -using IntervalsAlignmentAttributePtr = std::shared_ptr; } // namespace ngraph - -namespace ov { - -extern template class LP_TRANSFORMATIONS_API ngraph::VariantImpl; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper> : - public VariantImpl> { -public: - static constexpr VariantTypeInfo type_info{ "LowPrecision::IntervalsAlignment", 0 }; - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - std::shared_ptr get() const { return this->m_value; } - - static std::shared_ptr>> create( - const std::shared_ptr& node, - const AttributeParameters& params); - void merge(std::vector>>>& attributes); - std::string to_string() override; -}; - -} // namespace ov diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/per_tensor_quantization_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/per_tensor_quantization_attribute.hpp index 84f5216bda7..9a991dbf684 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/per_tensor_quantization_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/per_tensor_quantization_attribute.hpp @@ -14,24 +14,9 @@ #include "attribute_parameters.hpp" namespace ngraph { -class LP_TRANSFORMATIONS_API PerTensorQuantizationAttribute { +class LP_TRANSFORMATIONS_API PerTensorQuantizationAttribute : public ov::RuntimeAttribute { +public: + OPENVINO_RTTI("LowPrecision::PerTensorQuantization", "", ov::RuntimeAttribute, 0); + ~PerTensorQuantizationAttribute(); }; } // namespace ngraph - -namespace ov { - -extern template class LP_TRANSFORMATIONS_API ngraph::VariantImpl; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper : public VariantImpl { -public: - static constexpr VariantTypeInfo type_info { "LowPrecision::PerTensorQuantization", 0 }; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } -}; - -} // namespace ov diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/precision_preserved_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/precision_preserved_attribute.hpp index 08a4e213586..3a82a18d979 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/precision_preserved_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/precision_preserved_attribute.hpp @@ -13,44 +13,14 @@ #include "low_precision/rt_info/shared_value_attribute.hpp" namespace ngraph { - -class LP_TRANSFORMATIONS_API PrecisionPreservedAttribute; - -class LP_TRANSFORMATIONS_API PrecisionPreservedSharedValue : public SharedValue { +class LP_TRANSFORMATIONS_API PrecisionPreservedAttribute : public SharedAttribute { public: - PrecisionPreservedSharedValue() = default; - PrecisionPreservedSharedValue(const bool value) : value(value) {} - bool value; -}; + OPENVINO_RTTI("LowPrecision::PrecisionPreserved", "", ov::RuntimeAttribute, 0); -class LP_TRANSFORMATIONS_API PrecisionPreservedAttribute : public SharedValueAttribute { -public: PrecisionPreservedAttribute() = default; PrecisionPreservedAttribute(const bool value); -}; -using PrecisionPreservedAttributePtr = std::shared_ptr; + std::string to_string() const override; +}; } // namespace ngraph - -namespace ov { - -extern template class LP_TRANSFORMATIONS_API ngraph::VariantImpl; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper : public VariantImpl { -public: - static constexpr VariantTypeInfo type_info{ "LowPrecision::PrecisionPreserved", 0 }; - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - ngraph::PrecisionPreservedAttributePtr get() { return this->m_value; } - - std::string to_string() override; -}; - -} // namespace ov diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/precisions_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/precisions_attribute.hpp index b7057464c4f..14bb7742b96 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/precisions_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/precisions_attribute.hpp @@ -20,49 +20,18 @@ namespace ngraph { -class PrecisionsAttribute; - -class LP_TRANSFORMATIONS_API PrecisionsSharedValue : public SharedValue { -public: - std::vector precisions; -}; - -using PrecisionsAttributePtr = std::shared_ptr; - -class LP_TRANSFORMATIONS_API PrecisionsAttribute : public SharedValueAttribute { +class LP_TRANSFORMATIONS_API PrecisionsAttribute : public SharedAttribute> { public: + OPENVINO_RTTI("LowPrecision::Precisions", "", ov::RuntimeAttribute, 0); static std::vector defaultPrecisions; PrecisionsAttribute(const std::vector& precisions = defaultPrecisions); -}; -} // namespace ngraph -namespace ov { - -extern template class LP_TRANSFORMATIONS_API ngraph::VariantImpl>; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper> : public VariantImpl> { -public: - static constexpr VariantTypeInfo type_info{ "LowPrecision::Precisions", 0 }; - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - ov::Any init(const std::shared_ptr& node) override; - - std::shared_ptr get() { return this->m_value; } - - // create attribute instance for node - static std::shared_ptr>> create( + static ov::Any create( const std::shared_ptr& node, const AttributeParameters& params); // merge attribute instances which can be got from different sources: node, input port or output port - void merge(std::vector>>>& attributes); + void merge(std::vector& attributes); // vizualize shared attributes details in VizualizeTree pass - std::string to_string() override; + std::string to_string() const override; }; - -} // namespace ov +} // namespace ngraph diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/quantization_alignment_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/quantization_alignment_attribute.hpp index 57f43d06cfe..d8a09ccdc54 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/quantization_alignment_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/quantization_alignment_attribute.hpp @@ -18,46 +18,16 @@ #include "attribute_parameters.hpp" namespace ngraph { -class QuantizationAlignmentAttribute; - -class LP_TRANSFORMATIONS_API QuantizationAlignmentSharedValue : public SharedValue { -public: - QuantizationAlignmentSharedValue(const bool value = false) : value(value) {} - bool value; -}; - -class LP_TRANSFORMATIONS_API QuantizationAlignmentAttribute : public SharedValueAttribute{ +class LP_TRANSFORMATIONS_API QuantizationAlignmentAttribute : public SharedAttribute { public: + OPENVINO_RTTI("LowPrecision::QuantizationAlignment", "", ov::RuntimeAttribute, 0); QuantizationAlignmentAttribute(const bool value = false); -}; -using QuantizationAlignmentAttributePtr = std::shared_ptr; -} // namespace ngraph - -namespace ov { - -extern template class LP_TRANSFORMATIONS_API ngraph::VariantImpl; - -template<> -class LP_TRANSFORMATIONS_API VariantWrapper> : - public VariantImpl> { -public: - static constexpr VariantTypeInfo type_info{ "LowPrecision::QuantizationAlignment", 0 }; - - const VariantTypeInfo& get_type_info() const override { - return type_info; - } - - VariantWrapper(const value_type& value) : VariantImpl(value) {} - - ov::Any init(const std::shared_ptr& node) override; - - std::shared_ptr get() { return this->m_value; } - - static std::shared_ptr>> create( + static ov::Any create( const std::shared_ptr& node, const AttributeParameters& params); - void merge(std::vector>>>& attributes); - std::string to_string() override; + void merge(std::vector& attributes); + std::string to_string() const override; }; -} // namespace ov + +} // namespace ngraph diff --git a/src/common/low_precision_transformations/include/low_precision/rt_info/shared_value_attribute.hpp b/src/common/low_precision_transformations/include/low_precision/rt_info/shared_value_attribute.hpp index 706ff46d590..5b922c07b5c 100644 --- a/src/common/low_precision_transformations/include/low_precision/rt_info/shared_value_attribute.hpp +++ b/src/common/low_precision_transformations/include/low_precision/rt_info/shared_value_attribute.hpp @@ -14,46 +14,70 @@ #include #include -template -class LP_TRANSFORMATIONS_API SharedValue; - -template -class LP_TRANSFORMATIONS_API SharedValueAttribute { +template +class LP_TRANSFORMATIONS_API SharedAttribute : public ov::RuntimeAttribute { public: - SharedValueAttribute() : sharedValue(std::make_shared()) {} - virtual ~SharedValueAttribute() = default; - std::shared_ptr sharedValue; - std::string get_string() { - std::stringstream ss; + virtual ~SharedAttribute() = default; + class LP_TRANSFORMATIONS_API SharedValueAttribute : public std::enable_shared_from_this { + public: + struct LP_TRANSFORMATIONS_API SharedValue : public std::enable_shared_from_this { + SharedValue() = default; + SharedValue(const T& value) : value{value} {} + T value = {}; + std::vector> attributes; + }; + SharedValueAttribute() : sharedValue(std::make_shared()) {} - const size_t rawPointer = (size_t)this; - ss << rawPointer << ": "; + SharedValueAttribute(const T& value) : sharedValue{std::make_shared(value)} {} - const size_t sharedValueRawPointer = (size_t)sharedValue.get(); - ss << "sharedValue: " << sharedValueRawPointer; + std::shared_ptr sharedValue; - bool firstAttribute = true; - ss << ", attributes: {"; - for (auto& attributeWeakPtr : sharedValue->attributes) { - auto attribute = attributeWeakPtr.lock(); - if (attribute == nullptr) { - continue; + std::string get_string() { + std::stringstream ss; + + const size_t rawPointer = (size_t)this; + ss << rawPointer << ": "; + + const size_t sharedValueRawPointer = (size_t)sharedValue.get(); + ss << "sharedValue: " << sharedValueRawPointer; + + bool firstAttribute = true; + ss << ", attributes: {"; + for (auto& attributeWeakPtr : sharedValue->attributes) { + auto attribute = attributeWeakPtr.lock(); + if (attribute == nullptr) { + continue; + } + + if (!firstAttribute) { + ss << ", "; + } + ss << (size_t)attribute.get(); + firstAttribute = false; } - - if (!firstAttribute) { - ss << ", "; - } - ss << (size_t)attribute.get(); - firstAttribute = false; + ss << "}, "; + return ss.str(); } - ss << "}, "; - return ss.str(); + }; + + SharedAttribute() : attribute{std::make_shared()} { + attribute->sharedValue->attributes.emplace_back(attribute); + } + SharedAttribute(const T& value) : attribute{std::make_shared(value)} { + attribute->sharedValue->attributes.emplace_back(attribute); + } + + std::shared_ptr attribute; + + const T& value() const { + OPENVINO_ASSERT(attribute != nullptr, "Empty attribute"); + OPENVINO_ASSERT(attribute->sharedValue != nullptr, "Empty shared value"); + return attribute->sharedValue->value; + } + + T& value() { + OPENVINO_ASSERT(attribute != nullptr, "Empty attribute"); + OPENVINO_ASSERT(attribute->sharedValue != nullptr, "Empty shared value"); + return attribute->sharedValue->value; } }; - -template -class LP_TRANSFORMATIONS_API SharedValue { -public: - virtual ~SharedValue() = default; - std::vector> attributes; -}; diff --git a/src/common/low_precision_transformations/include/low_precision/update_shared_precision_preserved.hpp b/src/common/low_precision_transformations/include/low_precision/update_shared_precision_preserved.hpp index 0a17f7328ac..42745f8f793 100644 --- a/src/common/low_precision_transformations/include/low_precision/update_shared_precision_preserved.hpp +++ b/src/common/low_precision_transformations/include/low_precision/update_shared_precision_preserved.hpp @@ -51,11 +51,9 @@ public: // TODO: check if node can be quantized, if not, then doesn't update for (auto input : node->inputs()) { - auto precisionsAttributeWrapper = getAttribute(input); - if (precisionsAttributeWrapper != nullptr) { - const auto precisionsAttribute = precisionsAttributeWrapper->get(); - assert(precisionsAttribute != nullptr); - if (precisionsAttribute->sharedValue->precisions.empty()) { + auto precisionsAttributeWrapper = getAttribute(input); + if (!precisionsAttributeWrapper.empty()) { + if (precisionsAttributeWrapper.as().value().empty()) { return false; } } @@ -63,16 +61,15 @@ public: for (auto input : node->inputs()) { if (needToCheckExpectedAttributeType) { - if (getAttribute(input) == nullptr) { + if (getAttribute(input).empty()) { return false; } } auto parentAttribute = getSourceAttribute(input); - if (parentAttribute == nullptr) { + if (parentAttribute.empty()) { continue; } - - parentAttribute->get()->sharedValue->value = true; + parentAttribute.template as().value() = true; } } @@ -95,11 +92,11 @@ private: return input; } - std::shared_ptr> getSourceAttribute(const Input& input) { + ov::Any getSourceAttribute(const Input& input) { const auto dequantizationInput = getDequantizationInput(input); const auto output = dequantizationInput.get_source_output(); auto attribute = ngraph::pass::low_precision::getAttribute(output.get_node()->shared_from_this()); - if (attribute == nullptr) { + if (attribute.empty()) { attribute = ngraph::pass::low_precision::getAttribute(output.get_node_shared_ptr()); } return attribute; diff --git a/src/common/low_precision_transformations/src/align_quantization_intervals.cpp b/src/common/low_precision_transformations/src/align_quantization_intervals.cpp index 728161d0207..882bc757b61 100644 --- a/src/common/low_precision_transformations/src/align_quantization_intervals.cpp +++ b/src/common/low_precision_transformations/src/align_quantization_intervals.cpp @@ -18,7 +18,7 @@ bool ngraph::pass::low_precision::AlignQuantizationIntervals::run_on_function(st ngraph::pass::Manager manager; manager.set_per_pass_validation(false); std::shared_ptr intervalsAlignment = manager.register_pass(); - intervalsAlignment->add_matcher>(); + intervalsAlignment->add_matcher>(); intervalsAlignment->add_matcher>(); manager.run_passes(f); return false; diff --git a/src/common/low_precision_transformations/src/align_quantization_parameters.cpp b/src/common/low_precision_transformations/src/align_quantization_parameters.cpp index 72d4ed1184c..c33bbc22994 100644 --- a/src/common/low_precision_transformations/src/align_quantization_parameters.cpp +++ b/src/common/low_precision_transformations/src/align_quantization_parameters.cpp @@ -19,9 +19,9 @@ bool ngraph::pass::low_precision::AlignQuantizationParameters::run_on_function(s ngraph::pass::Manager manager; manager.set_per_pass_validation(false); std::shared_ptr propagation = manager.register_pass(); - propagation->add_matcher>(); + propagation->add_matcher>(); propagation->add_matcher>(); - propagation->add_matcher>(); + propagation->add_matcher>(); manager.run_passes(f); return false; } diff --git a/src/common/low_precision_transformations/src/fake_quantize_decomposition.cpp b/src/common/low_precision_transformations/src/fake_quantize_decomposition.cpp index d13d3b54f2e..dafcd031f64 100644 --- a/src/common/low_precision_transformations/src/fake_quantize_decomposition.cpp +++ b/src/common/low_precision_transformations/src/fake_quantize_decomposition.cpp @@ -45,8 +45,8 @@ namespace { // 2. Precisions on port DataPrecision getDataPrecisionByOutputPortAndFakeQuantize(std::shared_ptr layer) { const QuantizationDetails quantizationDetails = QuantizationDetails::getDetails(layer); - auto precisionsAttribute = getAttributeFromOutput>(layer->output(0)); - if (precisionsAttribute == nullptr) { + auto precisionsAttribute = getAttributeFromOutput(layer->output(0)); + if (precisionsAttribute.empty()) { // TODO: explore this case in more details: // 1. we should not be here assert(true); @@ -60,7 +60,7 @@ DataPrecision getDataPrecisionByOutputPortAndFakeQuantize(std::shared_ptrget()->sharedValue->precisions; + const auto& precisions = precisionsAttribute.as().value(); ngraph::element::Type precision; bool hasZeroPoint; @@ -77,7 +77,7 @@ DataPrecision getDataPrecisionByOutputPortAndFakeQuantize(std::shared_ptrget()->sharedValue->precisions = { precision }; + precisionsAttribute.as().value() = { precision }; } else { // use only available precision precision = *precisions.begin(); @@ -100,8 +100,8 @@ DataPrecision getDataPrecisionByOutputPort(std::shared_ptr const std::vector outputLowValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(3))->cast_vector(); const std::vector outputHighValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(4))->cast_vector(); - auto precisionsAttribute = getAttributeFromOutput>(layer->output(0)); - if (precisionsAttribute == nullptr) { + auto precisionsAttribute = getAttributeFromOutput(layer->output(0)); + if (precisionsAttribute.empty()) { // TODO: explore this case in more details: // 1. we should not be here assert(true); @@ -119,7 +119,7 @@ DataPrecision getDataPrecisionByOutputPort(std::shared_ptr precisionDetailsAtOutputIntervals.hasZeroPoint); } - const auto& precisions = precisionsAttribute->get()->sharedValue->precisions; + const auto& precisions = precisionsAttribute.as().value(); std::vector precisionsForLevels{}; switch (levels) { case 65536: @@ -153,7 +153,7 @@ DataPrecision getDataPrecisionByOutputPort(std::shared_ptr } // update shared attribute to affect all operations in subgraph - precisionsAttribute->get()->sharedValue->precisions = { precision }; + precisionsAttribute.as().value() = { precision }; } else { // use only available precision precision = *resultPrecisions.begin(); @@ -175,14 +175,14 @@ DataPrecision getDataPrecisionByOutputPort(std::shared_ptr std::tuple, std::shared_ptr> decomposeFakeQuantize( MatcherPass* matcherPass, std::shared_ptr& layer, - const std::shared_ptr& intervalsAlignment, + const ov::Any& intervalsAlignment, const DataPrecision& dataPrecision, const bool updatePrecisions, const element::Type deqPrecision) { std::shared_ptr dequantize; std::shared_ptr newFQ; - if (intervalsAlignment != nullptr) { + if (!intervalsAlignment.empty()) { OV_ITT_SCOPE(FIRST_INFERENCE, itt::domains::LPT_LT, "decomposeFakeQuantize1"); const std::vector outputLowValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(3))->cast_vector(); const std::vector outputHighValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(4))->cast_vector(); @@ -194,8 +194,8 @@ std::tuple, std::shared_ptr> decomposeFakeQuantize( const size_t levels = NetworkHelper::calculateLevels( dataPrecision.min, dataPrecision.max, - intervalsAlignment->sharedValue->combinedInterval.low, - intervalsAlignment->sharedValue->combinedInterval.high, + intervalsAlignment.as().value().combinedInterval.low, + intervalsAlignment.as().value().combinedInterval.high, outputLowValues[0], outputHighValues[0], dequantizationMul, @@ -286,8 +286,8 @@ bool FakeQuantizeDecompositionTransformation::transform(TransformationContext& c return false; } - auto attribute = getAttributeFromOutput>(layer->output(0)); - if ((attribute == nullptr) || (attribute->get()->sharedValue->precisions.empty())) { + auto attribute = getAttributeFromOutput(layer->output(0)); + if (attribute.empty() || (attribute.as().value().empty())) { return false; } @@ -320,42 +320,33 @@ bool FakeQuantizeDecompositionTransformation::transform(TransformationContext& c DataPrecision dataPrecision = fq_decomposition::getDataPrecisionByOutputPort(layer); - std::shared_ptr precisionsAttribute; + PrecisionsAttribute precisionsAttribute; { // TODO: LPT: return attribute (not wrapper) - auto attributeWrapper = getAttributeFromOutput>(layer->output(0)); - if (attributeWrapper == nullptr) { - THROW_IE_LPT_EXCEPTION(*layer) << "PrecisionAttribute is absent"; - } - precisionsAttribute = attributeWrapper->get(); - if (precisionsAttribute == nullptr) { + auto attributeWrapper = getAttributeFromOutput(layer->output(0)); + if (attributeWrapper.empty()) { THROW_IE_LPT_EXCEPTION(*layer) << "PrecisionAttribute is absent"; } + precisionsAttribute = attributeWrapper.as(); } - std::shared_ptr quantizationAlignment; + ov::Any quantizationAlignment; for (const auto& input : layer->output(0).get_target_inputs()) { - const auto alignmentValueWrapper = low_precision::getAttribute>(input.get_node()->shared_from_this()); - if (alignmentValueWrapper != nullptr) { - quantizationAlignment = alignmentValueWrapper->get(); - if (quantizationAlignment->sharedValue->value) { + quantizationAlignment = low_precision::getAttribute(input.get_node()->shared_from_this()); + if (!quantizationAlignment.empty()) { + if (quantizationAlignment.as().value()) { break; } } } - std::shared_ptr intervalsAlignment; - { - if ((quantizationAlignment != nullptr) && quantizationAlignment->sharedValue->value) { - auto intervalsAlignmentWrapper = low_precision::getAttribute>(layer); - if (intervalsAlignmentWrapper != nullptr) { - intervalsAlignment = intervalsAlignmentWrapper->get(); - } - } + ov::Any intervalsAlignment; + if (!quantizationAlignment.empty() && quantizationAlignment.as().value()) { + intervalsAlignment = low_precision::getAttribute(layer); } // FakeQuantize operations are combined in supported cascade (per tensor quantization) - if ((intervalsAlignment != nullptr) && (intervalsAlignment->sharedValue->minLevels <= 2ul)) { + if (!intervalsAlignment.empty() && (intervalsAlignment.as().value().minLevels <= 2ul)) { return false; } @@ -366,20 +357,20 @@ bool FakeQuantizeDecompositionTransformation::transform(TransformationContext& c const auto levels = layer->get_levels(); const std::vector outputLowValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(3))->cast_vector(); const std::vector outputHighValues = ov::as_type_ptr(layer->get_input_node_shared_ptr(4))->cast_vector(); - if (intervalsAlignment == nullptr) { + if (intervalsAlignment.empty()) { // define precision by FakeQuantize intervals LayerTransformation::PrecisionDetails precisionDetailsAtOutputIntervals = LayerTransformation::getPrecisionDetails( levels, outputLowValues, outputHighValues); const auto foundIt = std::find( - precisionsAttribute->sharedValue->precisions.begin(), - precisionsAttribute->sharedValue->precisions.end(), + precisionsAttribute.value().begin(), + precisionsAttribute.value().end(), precisionDetailsAtOutputIntervals.precision); bool hasZeroPoint; - if (foundIt == precisionsAttribute->sharedValue->precisions.end()) { - precision = *precisionsAttribute->sharedValue->precisions.begin(); + if (foundIt == precisionsAttribute.value().end()) { + precision = *precisionsAttribute.value().begin(); hasZeroPoint = true; } else { precision = precisionDetailsAtOutputIntervals.precision; @@ -393,12 +384,12 @@ bool FakeQuantizeDecompositionTransformation::transform(TransformationContext& c hasZeroPoint); } else { // define precision by attribute - if (intervalsAlignment->sharedValue->preferablePrecisions.empty()) { + if (intervalsAlignment.as().value().preferablePrecisions.empty()) { // TODO: LPT: add user defined preferredPrecision - precision = *precisionsAttribute->sharedValue->precisions.begin(); + precision = *precisionsAttribute.value().begin(); } else { // TODO: LPT: add user defined preferredPrecision - precision = *intervalsAlignment->sharedValue->preferablePrecisions.begin(); + precision = *intervalsAlignment.as().value().preferablePrecisions.begin(); } dataPrecision = DataPrecision( @@ -425,8 +416,8 @@ bool FakeQuantizeDecompositionTransformation::transform(TransformationContext& c updateOutput(context, dequantize, newFakeQuantize); - if (precisionsAttribute->sharedValue->precisions.size() != 1ul) { - precisionsAttribute->sharedValue->precisions = { dataPrecision.precision }; + if (precisionsAttribute.value().size() != 1ul) { + precisionsAttribute.value() = { dataPrecision.precision }; } return true; diff --git a/src/common/low_precision_transformations/src/fuse_multiply_to_fake_quantize.cpp b/src/common/low_precision_transformations/src/fuse_multiply_to_fake_quantize.cpp index 9d48b5a23f3..b48e97f0a90 100644 --- a/src/common/low_precision_transformations/src/fuse_multiply_to_fake_quantize.cpp +++ b/src/common/low_precision_transformations/src/fuse_multiply_to_fake_quantize.cpp @@ -80,9 +80,9 @@ bool FuseMultiplyToFakeQuantizeTransformation::transform(TransformationContext& replace_node(multiply, newFakeQuantize); NetworkHelper::copyInfo(fakeQuantize, newFakeQuantize); - const auto intervalAlignment = getAttribute(fakeQuantize); - if ((intervalAlignment != nullptr) && (intervalAlignment->get()->levels != 0ul)) { - newFakeQuantize->set_levels(intervalAlignment->get()->levels); + const auto intervalAlignment = getAttribute(fakeQuantize); + if (!intervalAlignment.empty() && (intervalAlignment.as().levels != 0ul)) { + newFakeQuantize->set_levels(intervalAlignment.as().levels); } updateOutput(context, newFakeQuantize, multiply); diff --git a/src/common/low_precision_transformations/src/markup_avg_pool_precision_preserved.cpp b/src/common/low_precision_transformations/src/markup_avg_pool_precision_preserved.cpp index 2dc256920c7..fff8b62e533 100644 --- a/src/common/low_precision_transformations/src/markup_avg_pool_precision_preserved.cpp +++ b/src/common/low_precision_transformations/src/markup_avg_pool_precision_preserved.cpp @@ -20,7 +20,7 @@ bool ngraph::pass::low_precision::MarkupAvgPoolPrecisionPreserved::run_on_functi std::shared_ptr markupAvgPoolPrecision = manager.register_pass(); markupAvgPoolPrecision->add_matcher>(); markupAvgPoolPrecision->add_matcher>(); - markupAvgPoolPrecision->add_matcher>(); + markupAvgPoolPrecision->add_matcher>(); manager.run_passes(f); return false; } diff --git a/src/common/low_precision_transformations/src/markup_can_be_quantized.cpp b/src/common/low_precision_transformations/src/markup_can_be_quantized.cpp index 10553e07fd7..64bf123fd95 100644 --- a/src/common/low_precision_transformations/src/markup_can_be_quantized.cpp +++ b/src/common/low_precision_transformations/src/markup_can_be_quantized.cpp @@ -22,13 +22,9 @@ bool ngraph::pass::low_precision::MarkupCanBeQuantized::run_on_function(std::sha auto setEmptyPrecisions = [](const std::shared_ptr& node) { for (auto& input : node->inputs()) { auto& rt = input.get_rt_info(); - - auto attribute = ngraph::pass::low_precision::make_shared_attribute(std::vector()); - auto attributeWrapper = std::make_shared>>(attribute); - rt.emplace( - ngraph::VariantWrapper>::type_info.name, - attributeWrapper); + PrecisionsAttribute::get_type_info_static(), + PrecisionsAttribute(std::vector())); } }; diff --git a/src/common/low_precision_transformations/src/markup_per_tensor_quantization.cpp b/src/common/low_precision_transformations/src/markup_per_tensor_quantization.cpp index 4cd37c94658..d2dfa732f57 100644 --- a/src/common/low_precision_transformations/src/markup_per_tensor_quantization.cpp +++ b/src/common/low_precision_transformations/src/markup_per_tensor_quantization.cpp @@ -33,8 +33,8 @@ bool ngraph::pass::low_precision::MarkupPerTensorQuantization::run_on_function(s auto createAttribute = [](Input& input){ auto &rt = input.get_rt_info(); rt.emplace( - ngraph::VariantWrapper::type_info.name, - std::make_shared<::ngraph::VariantWrapper>(PerTensorQuantizationAttribute())); + PerTensorQuantizationAttribute::get_type_info_static(), + PerTensorQuantizationAttribute()); }; if (restrictedPorts.empty()) { diff --git a/src/common/low_precision_transformations/src/markup_precisions.cpp b/src/common/low_precision_transformations/src/markup_precisions.cpp index 620a229e662..eab0401604d 100644 --- a/src/common/low_precision_transformations/src/markup_precisions.cpp +++ b/src/common/low_precision_transformations/src/markup_precisions.cpp @@ -42,30 +42,21 @@ void setRestriction( // if available precisions for any port is empty then mark all input ports for (auto& input : node->inputs()) { auto& rt = input.get_rt_info(); - - auto attribute = ngraph::pass::low_precision::make_shared_attribute(std::vector()); - auto attributeWrapper = std::make_shared>>(attribute); - rt.emplace( - ngraph::VariantWrapper>::type_info.name, - attributeWrapper); + PrecisionsAttribute::get_type_info_static(), + PrecisionsAttribute(std::vector())); } } else { for (const std::pair>& item : precisionsByPort) { Input input = node->input(item.first); + auto& rt = input.get_rt_info(); - auto precisionsAttribute = ngraph::pass::low_precision::getAttribute>(input); - if ((precisionsAttribute != nullptr) && - (precisionsAttribute->get()->sharedValue != nullptr) && - (precisionsAttribute->get()->sharedValue->precisions.empty())) { + auto precisionsAttribute = ngraph::pass::low_precision::getAttribute(input); + if ((!precisionsAttribute.empty()) && + (precisionsAttribute.as().value().empty())) { return; } - - auto attribute = ngraph::pass::low_precision::make_shared_attribute(item.second); - auto attributeWrapper = std::make_shared>>(attribute); - - auto& rt = input.get_rt_info(); - rt[ngraph::VariantWrapper>::type_info.name] = attributeWrapper; + rt[PrecisionsAttribute::get_type_info_static()] = PrecisionsAttribute(item.second); } } } @@ -93,9 +84,8 @@ bool ngraph::pass::low_precision::MarkupPrecisions::run_on_function(std::shared_ if (precisionPreserved) { auto& rt = node->get_rt_info(); rt.emplace( - ngraph::VariantWrapper::type_info.name, - std::make_shared<::ngraph::VariantWrapper>( - make_shared_attribute(precisionPreserved))); + PrecisionPreservedAttribute::get_type_info_static(), + PrecisionPreservedAttribute(precisionPreserved)); } const auto& typeInfo = node->get_type_info(); diff --git a/src/common/low_precision_transformations/src/mat_mul.cpp b/src/common/low_precision_transformations/src/mat_mul.cpp index 1ced4c45188..1533fdee2ea 100644 --- a/src/common/low_precision_transformations/src/mat_mul.cpp +++ b/src/common/low_precision_transformations/src/mat_mul.cpp @@ -54,10 +54,10 @@ bool MatMulTransformation::transform(TransformationContext &context, ngraph::pat if (fakeQuantize != nullptr) { const QuantizationDetails quantizationDetails = QuantizationDetails::getDetails(fakeQuantize); - const auto precisionsAttribute = getAttributeFromOutput(fakeQuantize); - const auto precisions = precisionsAttribute == nullptr ? + const auto precisionsAttribute = getAttributeFromOutput(fakeQuantize); + const auto precisions = precisionsAttribute.empty() ? PrecisionsAttribute::defaultPrecisions : - precisionsAttribute->get()->sharedValue->precisions; + precisionsAttribute.as().value(); const DataPrecision dataPrecision = getDataPrecision(fakeQuantize, quantizationDetails, precisions); auto tuple = NetworkHelper::decomposeFakeQuantize( @@ -259,10 +259,10 @@ bool MatMulTransformation::canBeTransformed(const TransformationContext& context const QuantizationDetails quantizationDetails = QuantizationDetails::getDetails(fakeQuantize); - const auto precisionsAttribute = getAttribute(matMul->input(1)); - const auto precisions = precisionsAttribute == nullptr ? + const auto precisionsAttribute = getAttribute(matMul->input(1)); + const auto precisions = precisionsAttribute.empty() ? PrecisionsAttribute::defaultPrecisions : - precisionsAttribute->get()->sharedValue->precisions; + precisionsAttribute.as().value(); const DataPrecision dataPrecision = getDataPrecision(fakeQuantize, quantizationDetails, precisions); if (dataPrecision.hasZeroPoint) { diff --git a/src/common/low_precision_transformations/src/multiply_to_group_convolution.cpp b/src/common/low_precision_transformations/src/multiply_to_group_convolution.cpp index 2caf1fa9940..10695f13eb7 100644 --- a/src/common/low_precision_transformations/src/multiply_to_group_convolution.cpp +++ b/src/common/low_precision_transformations/src/multiply_to_group_convolution.cpp @@ -66,10 +66,10 @@ bool MultiplyToGroupConvolutionTransformation::transform(TransformationContext& // if restrictions are absent precisions attribute is used if (weightsPrecision == element::undefined) { - const auto precisionsAttribute = getAttribute(multiply->input(inputIndex == 0ul ? 1ul : 0ul)); + const auto precisionsAttribute = getAttribute(multiply->input(inputIndex == 0ul ? 1ul : 0ul)); const auto precisions = precisionsAttribute == nullptr ? PrecisionsAttribute::defaultPrecisions : - precisionsAttribute->get()->sharedValue->precisions; + precisionsAttribute.as().value(); weightsPrecision = precisions[0]; } } else { diff --git a/src/common/low_precision_transformations/src/network_helper.cpp b/src/common/low_precision_transformations/src/network_helper.cpp index 50d6da8fc1b..f72a2511384 100644 --- a/src/common/low_precision_transformations/src/network_helper.cpp +++ b/src/common/low_precision_transformations/src/network_helper.cpp @@ -668,13 +668,12 @@ std::shared_ptr NetworkHelper::fuseConvert(const std::shar bool NetworkHelper::isPrecisionPreserved(const std::shared_ptr& node) { auto& rt = node->get_rt_info(); - auto it = rt.find(ngraph::VariantWrapper::type_info.name); + auto it = rt.find(PrecisionPreservedAttribute::get_type_info_static()); if (it == rt.end()) { return false; } - auto attribute = std::dynamic_pointer_cast>(it->second); - assert(attribute != nullptr); - return attribute->get()->sharedValue->value; + auto attribute = it->second; + return attribute.as().value(); } size_t NetworkHelper::calculateLevels( @@ -1854,15 +1853,11 @@ bool NetworkHelper::isDQByDynamicDimension(const std::shared_ptr& layer, s bool isDisabled(const std::shared_ptr& node) { for (const auto& input : node->inputs()) { - auto precisionAttribute = getAttribute>(input); - if (precisionAttribute == nullptr) { + auto precisionAttribute = getAttribute(input); + if (precisionAttribute.empty()) { continue; } - - assert(precisionAttribute->get() != nullptr); - assert(precisionAttribute->get()->sharedValue != nullptr); - - const auto& precisionRestrictions = precisionAttribute->get()->sharedValue->precisions; + const auto& precisionRestrictions = precisionAttribute.as().value(); if (precisionRestrictions.empty()) { return true; } diff --git a/src/common/low_precision_transformations/src/propagate_precisions.cpp b/src/common/low_precision_transformations/src/propagate_precisions.cpp index 4b15dd7e7b9..1e671f4e091 100644 --- a/src/common/low_precision_transformations/src/propagate_precisions.cpp +++ b/src/common/low_precision_transformations/src/propagate_precisions.cpp @@ -21,7 +21,7 @@ bool ngraph::pass::low_precision::PropagatePrecisions::run_on_function(std::shar ngraph::pass::Manager manager; manager.set_per_pass_validation(false); std::shared_ptr precisionsPropagation = manager.register_pass(); - precisionsPropagation->add_matcher>(AttributeSource::OutputPort); + precisionsPropagation->add_matcher>(AttributeSource::OutputPort); precisionsPropagation->add_matcher>(); precisionsPropagation->add_matcher>(); manager.run_passes(f); diff --git a/src/common/low_precision_transformations/src/rt_info/avg_pool_precision_preserved_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/avg_pool_precision_preserved_attribute.cpp index aba2aa578e5..dea564d37ca 100644 --- a/src/common/low_precision_transformations/src/rt_info/avg_pool_precision_preserved_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/avg_pool_precision_preserved_attribute.cpp @@ -11,18 +11,13 @@ using namespace ngraph; using namespace ov; -template class ngraph::VariantImpl; -constexpr VariantTypeInfo VariantWrapper::type_info; - -void VariantWrapper::merge( - std::vector>>>& attributes) { +void AvgPoolPrecisionPreservedAttribute::merge(std::vector& attributes) { } -std::string VariantWrapper::to_string() { - auto value = this->m_value; +std::string AvgPoolPrecisionPreservedAttribute::to_string() const { std::stringstream ss; - ss << m_value->get_string(); - ss << "value: " << (value->sharedValue->value ? "true" : "false"); + ss << attribute->get_string(); + ss << "value: " << (value() ? "true" : "false"); return ss.str(); } diff --git a/src/common/low_precision_transformations/src/rt_info/intervals_alignment_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/intervals_alignment_attribute.cpp index 7beb4722459..269c1e82645 100644 --- a/src/common/low_precision_transformations/src/rt_info/intervals_alignment_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/intervals_alignment_attribute.cpp @@ -17,23 +17,21 @@ using namespace ngraph::pass::low_precision; IntervalsAlignmentAttribute::IntervalsAlignmentAttribute( const IntervalsAlignmentSharedValue::Interval combinedInterval, - size_t levels) : levels(levels) { - sharedValue = std::make_shared(combinedInterval, combinedInterval, levels); + size_t levels) : + SharedAttribute(IntervalsAlignmentSharedValue{combinedInterval, combinedInterval, levels}), + levels(levels) { } IntervalsAlignmentAttribute::IntervalsAlignmentAttribute( const IntervalsAlignmentSharedValue::Interval combinedInterval, const size_t levels, const IntervalsAlignmentSharedValue::Interval minInterval, - const size_t minLevels) : levels(levels) { - sharedValue = std::make_shared(combinedInterval, minInterval, minLevels); + const size_t minLevels) : + SharedAttribute(IntervalsAlignmentSharedValue{combinedInterval, minInterval, minLevels}), + levels(levels) { } -template class ngraph::VariantImpl; - -constexpr VariantTypeInfo VariantWrapper::type_info; - -std::shared_ptr>> VariantWrapper::create( +ov::Any IntervalsAlignmentAttribute::create( const std::shared_ptr& node, const AttributeParameters& params) { if (!ov::is_type(node)) { @@ -109,11 +107,10 @@ std::shared_ptr>> Va auto& rtInfo = node->get_rt_info(); const IntervalsAlignmentSharedValue::Interval interval{ lowInterval, highInterval }; - const auto attribute = std::make_shared<::ngraph::VariantWrapper>( - ngraph::pass::low_precision::make_shared_attribute( + auto attribute = IntervalsAlignmentAttribute( interval, - fakeQuantize->get_levels())); - rtInfo[ngraph::VariantWrapper::type_info.name] = attribute; + fakeQuantize->get_levels()); + auto& result = (rtInfo[IntervalsAlignmentAttribute::get_type_info_static()] = attribute); const std::vector outputLowValues = ov::as_type_ptr(fakeQuantize->get_input_node_shared_ptr(3))->cast_vector(); const std::vector outputHighValues = ov::as_type_ptr(fakeQuantize->get_input_node_shared_ptr(4))->cast_vector(); @@ -123,50 +120,49 @@ std::shared_ptr>> Va outputHighValues); if (preferablePrecision.precision != element::undefined) { - attribute->get()->sharedValue->preferablePrecisions.insert(preferablePrecision.precision); + attribute.value().preferablePrecisions.insert(preferablePrecision.precision); } #ifdef LPT_DEBUG - attribute->get()->sharedValue->minLevelsOperation = node->get_friendly_name(); + attribute.value().minLevelsOperation = node->get_friendly_name(); #endif - return attribute; + return result; } } -void VariantWrapper::merge( - std::vector>>>& attributes) { - std::shared_ptr resultAttribute = get(); +void IntervalsAlignmentAttribute::merge( + std::vector& attributes) { for (const auto& attributeWrapper : attributes) { - auto attribute = attributeWrapper->get(); + auto attribute = attributeWrapper.as().attribute; // TODO: LPT: copy/past: merge() - const auto& resultSharedValue = resultAttribute->sharedValue; - const auto& sharedValue = attribute->sharedValue; - if (resultAttribute->levels != attribute->levels) { + auto& resultSharedValue = value(); + auto& sharedValue = attributeWrapper.as().value(); + if (levels != attributeWrapper.as().levels) { // TODO: LPT: not supported right now - resultAttribute->levels = 0ul; - resultSharedValue->minLevels = 0ul; + levels = 0ul; + resultSharedValue.minLevels = 0ul; } - if (resultSharedValue->combinedInterval.low > sharedValue->combinedInterval.low) { - resultSharedValue->combinedInterval.low = sharedValue->combinedInterval.low; + if (resultSharedValue.combinedInterval.low > sharedValue.combinedInterval.low) { + resultSharedValue.combinedInterval.low = sharedValue.combinedInterval.low; } - if (resultSharedValue->combinedInterval.high < sharedValue->combinedInterval.high) { - resultSharedValue->combinedInterval.high = sharedValue->combinedInterval.high; + if (resultSharedValue.combinedInterval.high < sharedValue.combinedInterval.high) { + resultSharedValue.combinedInterval.high = sharedValue.combinedInterval.high; } - assert(!std::isinf(resultSharedValue->combinedInterval.low)); - assert(!std::isinf(resultSharedValue->combinedInterval.high)); + assert(!std::isinf(resultSharedValue.combinedInterval.low)); + assert(!std::isinf(resultSharedValue.combinedInterval.high)); - resultSharedValue->preferablePrecisions.insert(sharedValue->preferablePrecisions.begin(), sharedValue->preferablePrecisions.end()); + resultSharedValue.preferablePrecisions.insert(sharedValue.preferablePrecisions.begin(), sharedValue.preferablePrecisions.end()); - const auto resultSize = std::abs(resultSharedValue->minInterval.high - resultSharedValue->minInterval.low); - const auto size = std::abs(sharedValue->minInterval.high - sharedValue->minInterval.low); + const auto resultSize = std::abs(resultSharedValue.minInterval.high - resultSharedValue.minInterval.low); + const auto size = std::abs(sharedValue.minInterval.high - sharedValue.minInterval.low); if (resultSize > size) { - resultSharedValue->minInterval = sharedValue->minInterval; - if (resultAttribute->levels != 0ul) { + resultSharedValue.minInterval = sharedValue.minInterval; + if (levels != 0ul) { float dequantizationMul; float dequantizationSub; float updatedOutputLowValue; @@ -174,44 +170,44 @@ void VariantWrapper::merge( const size_t minLevels = NetworkHelper::calculateLevels( 0.f, - DataPrecision::getMaxValue(resultAttribute->levels), - resultSharedValue->combinedInterval.low, - resultSharedValue->combinedInterval.high, - resultSharedValue->minInterval.low, - resultSharedValue->minInterval.high, + DataPrecision::getMaxValue(levels), + resultSharedValue.combinedInterval.low, + resultSharedValue.combinedInterval.high, + resultSharedValue.minInterval.low, + resultSharedValue.minInterval.high, dequantizationMul, dequantizationSub, updatedOutputLowValue, updatedOutputHighValue); - resultSharedValue->minLevels = minLevels; + resultSharedValue.minLevels = minLevels; } #ifdef LPT_DEBUG - resultSharedValue->minLevelsOperation = sharedValue->minLevelsOperation; + resultSharedValue.minLevelsOperation = sharedValue.minLevelsOperation; #endif } } } -std::string VariantWrapper::to_string() { +std::string IntervalsAlignmentAttribute::to_string() const { std::stringstream preferablePrecisions; preferablePrecisions << "{"; size_t index = 0; - for (const auto& precision : m_value->sharedValue->preferablePrecisions) { + for (const auto& precision : value().preferablePrecisions) { preferablePrecisions << (index > 0 ? ", " : "") << precision; ++index; } preferablePrecisions << "}"; std::stringstream ss; - ss << m_value->get_string(); - ss << "levels: " + std::to_string(m_value->levels) << ", " << - "combined: { " << m_value->sharedValue->combinedInterval.low << ", " << m_value->sharedValue->combinedInterval.high << " }, " << - "min: { " << m_value->sharedValue->minInterval.low << ", " << m_value->sharedValue->minInterval.high << " }, " - "minLevels: " << m_value->sharedValue->minLevels << + ss << attribute->get_string(); + ss << "levels: " + std::to_string(levels) << ", " << + "combined: { " << value().combinedInterval.low << ", " << value().combinedInterval.high << " }, " << + "min: { " << value().minInterval.low << ", " << value().minInterval.high << " }, " + "minLevels: " << value().minLevels << #ifdef LPT_DEBUG - ", minLevelsOperation: " << m_value->sharedValue->minLevelsOperation << + ", minLevelsOperation: " << value().minLevelsOperation << #endif ", preferablePrecisions: " << preferablePrecisions.str(); return ss.str(); diff --git a/src/common/low_precision_transformations/src/rt_info/per_tensor_quantization_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/per_tensor_quantization_attribute.cpp index 0036e9e378a..78338a05b6d 100644 --- a/src/common/low_precision_transformations/src/rt_info/per_tensor_quantization_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/per_tensor_quantization_attribute.cpp @@ -7,5 +7,4 @@ using namespace ngraph; using namespace ov; -template class ngraph::VariantImpl; -constexpr VariantTypeInfo VariantWrapper::type_info; +PerTensorQuantizationAttribute::~PerTensorQuantizationAttribute() = default; diff --git a/src/common/low_precision_transformations/src/rt_info/precision_preserved_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/precision_preserved_attribute.cpp index c2b94e425b0..ecb9bf9271a 100644 --- a/src/common/low_precision_transformations/src/rt_info/precision_preserved_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/precision_preserved_attribute.cpp @@ -10,18 +10,13 @@ using namespace ngraph; using namespace ov; -PrecisionPreservedAttribute::PrecisionPreservedAttribute(const bool value) { - sharedValue->value = value; +PrecisionPreservedAttribute::PrecisionPreservedAttribute(const bool value) : + SharedAttribute(value) { } -template class ngraph::VariantImpl; - -constexpr VariantTypeInfo VariantWrapper::type_info; - -std::string VariantWrapper::to_string() { - auto& value = this->m_value; +std::string PrecisionPreservedAttribute::to_string() const { std::stringstream ss; - ss << m_value->get_string(); - ss << "value: " << (value->sharedValue->value ? "true" : "false"); + ss << attribute->get_string(); + ss << "value: " << (value() ? "true" : "false"); return ss.str(); } diff --git a/src/common/low_precision_transformations/src/rt_info/precisions_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/precisions_attribute.cpp index f7158632319..1d103efcc3b 100644 --- a/src/common/low_precision_transformations/src/rt_info/precisions_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/precisions_attribute.cpp @@ -19,30 +19,21 @@ using namespace ov; // order defines default precision std::vector PrecisionsAttribute::defaultPrecisions = {ngraph::element::u8, ngraph::element::i8}; -PrecisionsAttribute::PrecisionsAttribute(const std::vector& precisions) { - sharedValue->precisions = precisions; +PrecisionsAttribute::PrecisionsAttribute(const std::vector& precisions) : + SharedAttribute(precisions) { } -template class ngraph::VariantImpl>; - -constexpr VariantTypeInfo VariantWrapper>::type_info; - -std::shared_ptr>> VariantWrapper>::create( +ov::Any PrecisionsAttribute::create( const std::shared_ptr& node, const AttributeParameters& params) { - auto attribute = ngraph::pass::low_precision::make_shared_attribute(); - auto wrapper = std::make_shared>>(attribute); - auto& rt = ov::is_type(node) ? node->output(0).get_rt_info() : node->get_rt_info(); - rt[ngraph::VariantWrapper>::type_info.name] = wrapper; - return wrapper; + return (rt[PrecisionsAttribute::get_type_info_static()] = PrecisionsAttribute()); } -void VariantWrapper>::merge( - std::vector>>>& attributes) { - auto& my = this->get()->sharedValue->precisions; +void PrecisionsAttribute::merge(std::vector& attributes) { + auto& my = value(); for (auto attribute : attributes) { - const auto& attributeValues = attribute->get()->sharedValue->precisions; + const auto& attributeValues = attribute.as().value(); auto it = my.begin(); while (it != my.end()) { if (std::find(attributeValues.begin(), attributeValues.end(), *it) == attributeValues.end()) { @@ -57,22 +48,19 @@ void VariantWrapper>::merge( } } -ov::Any VariantWrapper>::init(const std::shared_ptr& node) { - return {}; -} -std::string VariantWrapper>::to_string() { +std::string PrecisionsAttribute::to_string() const { std::stringstream ss; - ss << m_value->get_string(); + ss << attribute->get_string(); bool firstPrecision = true; ss << "precisions: {"; - for (const auto& value : m_value->sharedValue->precisions) { + for (const auto& type : value()) { if (!firstPrecision) { ss << ", "; } - ss << value; + ss << type; firstPrecision = false; } ss << "}"; diff --git a/src/common/low_precision_transformations/src/rt_info/quantization_alignment_attribute.cpp b/src/common/low_precision_transformations/src/rt_info/quantization_alignment_attribute.cpp index f275f836f6d..d6621a39a77 100644 --- a/src/common/low_precision_transformations/src/rt_info/quantization_alignment_attribute.cpp +++ b/src/common/low_precision_transformations/src/rt_info/quantization_alignment_attribute.cpp @@ -16,27 +16,19 @@ using namespace ov; using namespace ngraph; using namespace ngraph::pass::low_precision; -QuantizationAlignmentAttribute::QuantizationAlignmentAttribute(const bool hasToBeAligned) { - sharedValue = std::make_shared(hasToBeAligned); +QuantizationAlignmentAttribute::QuantizationAlignmentAttribute(const bool hasToBeAligned) : + SharedAttribute(hasToBeAligned) { } -template class ngraph::VariantImpl; - -constexpr VariantTypeInfo VariantWrapper::type_info; - -ov::Any VariantWrapper::init(const std::shared_ptr& node) { - return nullptr; -} - -std::shared_ptr>> VariantWrapper::create( +ov::Any QuantizationAlignmentAttribute::create( const std::shared_ptr& node, const AttributeParameters& params) { - if (getAttribute>(node) != nullptr) { - return nullptr; + if (!getAttribute(node).empty()) { + return {}; } if (!NetworkHelper::isPrecisionPreserved(node)) { - return nullptr; + return {}; } bool leastOneOperationIsFakeQuantize = false; @@ -66,26 +58,22 @@ std::shared_ptr>> if (leastOneOperationIsFakeQuantize && !leastOneOperationIsNotFakeQuantize) { auto& rt = node->get_rt_info(); - const auto attribute = std::make_shared>( - make_shared_attribute()); - rt[ngraph::VariantWrapper::type_info.name] = attribute; - return attribute; + rt[QuantizationAlignmentAttribute::get_type_info_static()] = QuantizationAlignmentAttribute(); + return rt[QuantizationAlignmentAttribute::get_type_info_static()]; } - return nullptr; + return {}; } -void VariantWrapper::merge( - std::vector>>>& attributes) { - auto currentAttributte = get(); - for (const auto& attribute : attributes) { - currentAttributte->sharedValue->value = currentAttributte->sharedValue->value || attribute->get()->sharedValue->value; +void QuantizationAlignmentAttribute::merge(std::vector& attributes) { + for (const auto& other_attribute : attributes) { + value() = value() || other_attribute.as().value(); } } -std::string VariantWrapper::to_string() { +std::string QuantizationAlignmentAttribute::to_string() const { std::stringstream ss; - ss << m_value->get_string(); - ss << "value: " << (m_value->sharedValue->value ? "true" : "false"); + ss << attribute->get_string(); + ss << "value: " << (value() ? "true" : "false"); return ss.str(); } diff --git a/src/common/low_precision_transformations/src/weightable_layer_transformation.cpp b/src/common/low_precision_transformations/src/weightable_layer_transformation.cpp index 0a9a2a86690..c146be66495 100644 --- a/src/common/low_precision_transformations/src/weightable_layer_transformation.cpp +++ b/src/common/low_precision_transformations/src/weightable_layer_transformation.cpp @@ -298,10 +298,10 @@ bool WeightableLayerTransformation::decomposeFakeQuantizeForWeightsPath(const st } const QuantizationDetails quantizationDetails = QuantizationDetails::getDetails(fq); - const auto precisionsAttribute = getAttributeFromOutput(fq); - const auto precisions = precisionsAttribute == nullptr ? + const auto precisionsAttribute = getAttributeFromOutput(fq); + const auto precisions = precisionsAttribute.empty() ? PrecisionsAttribute::defaultPrecisions : - precisionsAttribute->get()->sharedValue->precisions; + precisionsAttribute.as().value(); const DataPrecision dataPrecision = getDataPrecision(fq, quantizationDetails, precisions); @@ -365,10 +365,10 @@ DataPrecision WeightableLayerTransformation::getDataPrecisionOnWeights(const std return DataPrecision(); } - const auto precisionsAttribute = getAttributeFromOutput(fq); - const auto precisions = precisionsAttribute == nullptr ? + const auto precisionsAttribute = getAttributeFromOutput(fq); + const auto precisions = precisionsAttribute.empty() ? PrecisionsAttribute::defaultPrecisions : - precisionsAttribute->get()->sharedValue->precisions; + precisionsAttribute.as().value(); return getDataPrecision(fq, quantizationDetails, precisions); } diff --git a/src/common/snippets/include/snippets/register_info.hpp b/src/common/snippets/include/snippets/register_info.hpp deleted file mode 100644 index 8d0e64ef2b6..00000000000 --- a/src/common/snippets/include/snippets/register_info.hpp +++ /dev/null @@ -1,21 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#pragma once - -#include -#include -#include - -namespace ov { - -template <> -class TRANSFORMATIONS_API VariantWrapper> : public VariantImpl> { -public: - OPENVINO_RTTI("VariantWrapper>"); - BWDCMP_RTTI_DECLARATION; - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -} // namespace ov diff --git a/src/common/snippets/src/generator.cpp b/src/common/snippets/src/generator.cpp index 33b373f2d29..1668d1b474f 100644 --- a/src/common/snippets/src/generator.cpp +++ b/src/common/snippets/src/generator.cpp @@ -3,7 +3,6 @@ // #include "snippets/generator.hpp" -#include "snippets/register_info.hpp" #include "snippets/pass/assign_registers.hpp" #include "snippets/pass/vector_to_scalar.hpp" #include "snippets/pass/insert_load_store.hpp" @@ -17,10 +16,9 @@ auto ngraph::snippets::getRegisters(std::shared_ptr& n) -> ngraph: // ToDo: change to reg_t std::vector rout; - auto rinfo = rt["reginfo"]; - if (!rinfo.empty()) { - auto reginfo = ngraph::as_type_ptr>>(rinfo)->get(); - for (auto reg : reginfo) { + auto it_rt = rt.find("reginfo"); + if (it_rt == rt.end()) { + for (auto reg : it_rt->second.as>()) { rout.push_back(reg); } } @@ -28,10 +26,9 @@ auto ngraph::snippets::getRegisters(std::shared_ptr& n) -> ngraph: std::vector rin; for (auto input : n->inputs()) { auto rt = input.get_source_output().get_node_shared_ptr()->get_rt_info(); - auto rinfo = rt["reginfo"]; - if (!rinfo.empty()) { - auto reginfo = ngraph::as_type_ptr>>(rinfo)->get(); - for (auto reg : reginfo) { + auto it_rt = rt.find("reginfo"); + if (it_rt == rt.end()) { + for (auto reg : it_rt->second.as>()) { rin.push_back(reg); } } diff --git a/src/common/snippets/src/pass/assign_registers.cpp b/src/common/snippets/src/pass/assign_registers.cpp index 7561b2403a0..61ecdf4eba6 100644 --- a/src/common/snippets/src/pass/assign_registers.cpp +++ b/src/common/snippets/src/pass/assign_registers.cpp @@ -7,7 +7,6 @@ #include "remarks.hpp" #include "snippets/pass/assign_registers.hpp" -#include "snippets/register_info.hpp" #include "snippets/snippets_isa.hpp" #include @@ -152,7 +151,7 @@ bool ngraph::snippets::pass::AssignRegisters::run_on_function(std::shared_ptr(n)) { auto ea = reg64_tmp_start+static_cast(f->get_result_index(result) + f->get_parameters().size()); - rt["effectiveAddress"] = std::make_shared>(VariantWrapper(ea)); + rt["effectiveAddress"] = ea; continue; } // store effective address and procced with vector registers @@ -161,10 +160,10 @@ bool ngraph::snippets::pass::AssignRegisters::run_on_function(std::shared_ptr(source)) { auto ea = reg64_tmp_start+static_cast(f->get_parameter_index(param)); - rt["effectiveAddress"] = std::make_shared>(VariantWrapper(ea)); + rt["effectiveAddress"] = ea; } else if (auto constant = ov::as_type_ptr(source)) { auto ea = reg64_tmp_start+static_cast(f->get_parameters().size() + f->get_results().size() + 1 + constantID); - rt["effectiveAddress"] = std::make_shared>(VariantWrapper(ea)); + rt["effectiveAddress"] = ea; constantID++; } else { throw ngraph_error("load/broadcast should follow only Parameter or non-Scalar constant"); @@ -176,7 +175,7 @@ bool ngraph::snippets::pass::AssignRegisters::run_on_function(std::shared_ptr>>(VariantWrapper>(regs)); + rt["reginfo"] = regs; } return false; diff --git a/src/common/snippets/src/register_info.cpp b/src/common/snippets/src/register_info.cpp deleted file mode 100644 index 597ef5e2298..00000000000 --- a/src/common/snippets/src/register_info.cpp +++ /dev/null @@ -1,9 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#include "snippets/register_info.hpp" - -template class ngraph::VariantImpl>; - -BWDCMP_RTTI_DEFINITION(ngraph::VariantWrapper>); diff --git a/src/common/transformations/include/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.hpp b/src/common/transformations/include/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.hpp new file mode 100644 index 00000000000..21850936b0e --- /dev/null +++ b/src/common/transformations/include/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.hpp @@ -0,0 +1,32 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include +#include + +#include + +#include +#include + +namespace ngraph { +namespace pass { + +class TRANSFORMATIONS_API NearestNeighborUpsamplingFusion; + +} // namespace pass +} // namespace ngraph + +/** + * @ingroup ie_transformation_common_api + * @brief NearestNeighborUpsamplingFusion transformation fuses subgraph that uses the simpler operations, as ShapeOf, + * StridedSlice, Concat, Reshape, Mul to calculate Interpolate with mode='nearest'. + */ +class ngraph::pass::NearestNeighborUpsamplingFusion : public ngraph::pass::MatcherPass { +public: + NGRAPH_RTTI_DECLARATION; + NearestNeighborUpsamplingFusion(); +}; diff --git a/src/common/transformations/include/transformations/fix_rt_info.hpp b/src/common/transformations/include/transformations/fix_rt_info.hpp new file mode 100644 index 00000000000..fbc0f4d7df6 --- /dev/null +++ b/src/common/transformations/include/transformations/fix_rt_info.hpp @@ -0,0 +1,44 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +/** + * @brief Defines initialize node runtime information pass + * @file init_node_info.hpp + */ + +#include +#include + +#include + +/** + * @brief ngraph namespace + */ +namespace ngraph { + +/** + * @brief ngraph::pass namespace + */ +namespace pass { + +class NGRAPH_API FixRtInfo; + +} // namespace pass +} // namespace ngraph + +/** + * @ingroup ie_transformation_common_api + * @brief FixRtInfo transformation helps to fix info attributes in a single place. + * User can pass runtime attribute using various types. + * This Pass should generalize them runtime info representation. + * + * Used to extract runtime attributes from shared pointer to `ov::RuntimeAttributeWrapper` to standard or trivial types + */ +class ngraph::pass::FixRtInfo: public ngraph::pass::FunctionPass { +public: + NGRAPH_RTTI_DECLARATION; + bool run_on_function(std::shared_ptr f) override; +}; diff --git a/src/common/transformations/include/transformations/op_conversions/detection_output_downgrade.hpp b/src/common/transformations/include/transformations/op_conversions/detection_output_downgrade.hpp new file mode 100644 index 00000000000..2205bbef275 --- /dev/null +++ b/src/common/transformations/include/transformations/op_conversions/detection_output_downgrade.hpp @@ -0,0 +1,27 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include +#include + +namespace ngraph { +namespace pass { + +class TRANSFORMATIONS_API ConvertDetectionOutput8ToDetectionOutput1; + +} // namespace pass +} // namespace ngraph + +/** + * @ingroup ie_transformation_common_api + * @brief ConvertDetectionOutput8ToDetectionOutput1 converts v8::DetectionOutput + * into v0::DetectionOutput. + */ +class ngraph::pass::ConvertDetectionOutput8ToDetectionOutput1 : public ngraph::pass::MatcherPass { +public: + NGRAPH_RTTI_DECLARATION; + ConvertDetectionOutput8ToDetectionOutput1(); +}; diff --git a/src/common/transformations/include/transformations/op_conversions/detection_output_upgrade.hpp b/src/common/transformations/include/transformations/op_conversions/detection_output_upgrade.hpp new file mode 100644 index 00000000000..68041d7d8fa --- /dev/null +++ b/src/common/transformations/include/transformations/op_conversions/detection_output_upgrade.hpp @@ -0,0 +1,27 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include +#include + +namespace ngraph { +namespace pass { + +class TRANSFORMATIONS_API ConvertDetectionOutput1ToDetectionOutput8; + +} // namespace pass +} // namespace ngraph + +/** + * @ingroup ie_transformation_common_api + * @brief ConvertDetectionOutput1ToDetectionOutput8 converts v0::DetectionOutput + * into v8::DetectionOutput. + */ +class ngraph::pass::ConvertDetectionOutput1ToDetectionOutput8 : public ngraph::pass::MatcherPass { +public: + NGRAPH_RTTI_DECLARATION; + ConvertDetectionOutput1ToDetectionOutput8(); +}; diff --git a/src/common/transformations/include/transformations/rt_info/attributes.hpp b/src/common/transformations/include/transformations/rt_info/attributes.hpp index 11a0d515ccb..424a8104e86 100644 --- a/src/common/transformations/include/transformations/rt_info/attributes.hpp +++ b/src/common/transformations/include/transformations/rt_info/attributes.hpp @@ -10,7 +10,6 @@ #include #include #include -#include #include #include #include @@ -30,19 +29,18 @@ namespace pass { class TRANSFORMATIONS_API Attributes { public: Attributes(); - ~Attributes(); - std::shared_ptr create_by_type_info(const ov::DiscreteTypeInfo& type_info); + Any create_by_type_info(const ov::DiscreteTypeInfo& type_info_name); private: template void register_factory() { - m_factory_registry.emplace(T::get_type_info_static(), [] { - return std::make_shared(); + m_factory_registry.emplace(T::get_type_info_static(), [] () -> Any { + return T{}; }); } - std::unordered_map()>> m_factory_registry; + std::unordered_map> m_factory_registry; }; } // namespace pass } // namespace ov diff --git a/src/common/transformations/include/transformations/rt_info/decompression.hpp b/src/common/transformations/include/transformations/rt_info/decompression.hpp index 18965d1aee2..b68b6b3b345 100644 --- a/src/common/transformations/include/transformations/rt_info/decompression.hpp +++ b/src/common/transformations/include/transformations/rt_info/decompression.hpp @@ -11,7 +11,7 @@ #include #include "openvino/core/node.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "transformations_visibility.hpp" @@ -28,7 +28,7 @@ TRANSFORMATIONS_API bool is_decompression(const std::shared_ptr& node); * @brief Decompression class represents runtime info attribute that marks operation * as used as decompression for Compressed Only format. */ -class TRANSFORMATIONS_API Decompression : public VariantImpl { +class TRANSFORMATIONS_API Decompression : public RuntimeAttribute { public: OPENVINO_RTTI("decompression", "0"); diff --git a/src/common/transformations/include/transformations/rt_info/disable_fp16_compression.hpp b/src/common/transformations/include/transformations/rt_info/disable_fp16_compression.hpp index 00352928ba2..a7bd10d0dcd 100644 --- a/src/common/transformations/include/transformations/rt_info/disable_fp16_compression.hpp +++ b/src/common/transformations/include/transformations/rt_info/disable_fp16_compression.hpp @@ -5,7 +5,7 @@ #pragma once #include "openvino/core/node.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "transformations_visibility.hpp" @@ -22,7 +22,7 @@ TRANSFORMATIONS_API bool fp16_compression_is_disabled(const std::shared_ptr { +class TRANSFORMATIONS_API DisableFP16Compression : public RuntimeAttribute { public: OPENVINO_RTTI("disable_fp16_compression", "0"); diff --git a/src/common/transformations/include/transformations/rt_info/fused_names_attribute.hpp b/src/common/transformations/include/transformations/rt_info/fused_names_attribute.hpp index 98ab5b7ae9f..7dacdc018a7 100644 --- a/src/common/transformations/include/transformations/rt_info/fused_names_attribute.hpp +++ b/src/common/transformations/include/transformations/rt_info/fused_names_attribute.hpp @@ -19,6 +19,7 @@ #include #include #include +#include "openvino/core/runtime_attribute.hpp" namespace ngraph { @@ -28,12 +29,11 @@ namespace ngraph { * @brief FusedName class represents runtime info attribute that stores * all operation names that was fully or partially fused into node */ -class NGRAPH_API FusedNames { -private: +class NGRAPH_API FusedNames : public ov::RuntimeAttribute { std::set fused_names; public: - friend class VariantWrapper; + OPENVINO_RTTI("fused_names", "0"); /** * A default constructor @@ -64,6 +64,14 @@ public: * @return vector if strings */ std::vector getVectorNames() const; + + ov::Any merge(const ngraph::NodeVector& nodes) const override; + + ov::Any init(const std::shared_ptr& node) const override; + + bool visit_attributes(AttributeVisitor& visitor) override; + + std::string to_string() const override; }; /** @@ -82,24 +90,3 @@ NGRAPH_API std::string getFusedNames(const std::shared_ptr & node) NGRAPH_API std::vector getFusedNamesVector(const std::shared_ptr & node); } // namespace ngraph - -namespace ov { - -extern template class NGRAPH_API VariantImpl; - -template<> -class NGRAPH_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("fused_names", "0"); - - VariantWrapper() = default; - - VariantWrapper(const value_type &value) : VariantImpl(value) {} - - Any merge(const ngraph::NodeVector & nodes) override; - - Any init(const std::shared_ptr & node) override; - - bool visit_attributes(AttributeVisitor & visitor) override; -}; -} // namespace ov diff --git a/src/common/transformations/include/transformations/rt_info/nms_selected_indices.hpp b/src/common/transformations/include/transformations/rt_info/nms_selected_indices.hpp index 312d0ee52d6..48e72367a7c 100644 --- a/src/common/transformations/include/transformations/rt_info/nms_selected_indices.hpp +++ b/src/common/transformations/include/transformations/rt_info/nms_selected_indices.hpp @@ -13,6 +13,7 @@ #include #include #include +#include "openvino/core/runtime_attribute.hpp" namespace ov { @@ -20,14 +21,10 @@ TRANSFORMATIONS_API bool has_nms_selected_indices(const Node * node); TRANSFORMATIONS_API void set_nms_selected_indices(Node * node); -class TRANSFORMATIONS_API NmsSelectedIndices : public VariantImpl { +class TRANSFORMATIONS_API NmsSelectedIndices : ov::RuntimeAttribute { public: OPENVINO_RTTI("nms_selected_indices", "0"); - NmsSelectedIndices() = default; - - NmsSelectedIndices(const value_type &value) : VariantImpl(value) {} - bool is_copyable() const override { return false; } }; diff --git a/src/common/transformations/include/transformations/rt_info/old_api_map_element_type_attribute.hpp b/src/common/transformations/include/transformations/rt_info/old_api_map_element_type_attribute.hpp index e28ae834c68..49b6b158917 100644 --- a/src/common/transformations/include/transformations/rt_info/old_api_map_element_type_attribute.hpp +++ b/src/common/transformations/include/transformations/rt_info/old_api_map_element_type_attribute.hpp @@ -28,7 +28,7 @@ namespace ov { * @brief OldApiMapElementType class represents runtime info attribute that stores legacy type * that is required for obtaining IR in old API. */ -class TRANSFORMATIONS_API OldApiMapElementType : public VariantImpl { +class TRANSFORMATIONS_API OldApiMapElementType : public RuntimeAttribute { public: OPENVINO_RTTI("old_api_map_element_type", "0"); @@ -41,13 +41,15 @@ public: * Constructs a new OldApiMapElementType object. * @param[in] value The object that stores values of OldApiMapElementType. */ - OldApiMapElementType(const value_type& value) : VariantImpl(value) {} + OldApiMapElementType(const ngraph::element::Type& value) : value(value) {} bool is_copyable() const override { return false; } bool visit_attributes(AttributeVisitor& visitor) override; + + ngraph::element::Type value; }; inline bool has_old_api_map_element_type(const std::shared_ptr& node) { @@ -57,13 +59,12 @@ inline bool has_old_api_map_element_type(const std::shared_ptr& node) { inline OldApiMapElementType get_old_api_map_element_type(const std::shared_ptr& node) { const auto& rt_map = node->get_rt_info(); - const auto& var = rt_map.at(OldApiMapElementType::get_type_info_static()); - return ngraph::as_type_ptr(var)->get(); + return rt_map.at(OldApiMapElementType::get_type_info_static()).as(); } inline void set_old_api_map_element_type(std::shared_ptr& node, const OldApiMapElementType& old_api_map) { auto& rt_map = node->get_rt_info(); - rt_map[OldApiMapElementType::get_type_info_static()] = std::make_shared(old_api_map); + rt_map[OldApiMapElementType::get_type_info_static()] = old_api_map; } } // namespace ov \ No newline at end of file diff --git a/src/common/transformations/include/transformations/rt_info/old_api_map_order_attribute.hpp b/src/common/transformations/include/transformations/rt_info/old_api_map_order_attribute.hpp index f650ab6f35a..4188003d628 100644 --- a/src/common/transformations/include/transformations/rt_info/old_api_map_order_attribute.hpp +++ b/src/common/transformations/include/transformations/rt_info/old_api_map_order_attribute.hpp @@ -39,7 +39,7 @@ class OldApiMapOrder; * Order of the transpose which should be applied to Result with new API layout to * obtain Result with old API layout. */ -class TRANSFORMATIONS_API OldApiMapOrder : public VariantImpl> { +class TRANSFORMATIONS_API OldApiMapOrder : public RuntimeAttribute { public: OPENVINO_RTTI("old_api_map_order", "0"); @@ -52,13 +52,15 @@ public: * Constructs a new OldApiMapOrder object. * @param[in] value The object that stores values of OldApiMapOrder. */ - OldApiMapOrder(const value_type& value) : VariantImpl(value) {} + OldApiMapOrder(const std::vector& value) : value(value) {} bool is_copyable() const override { return false; } bool visit_attributes(AttributeVisitor& visitor) override; + + std::vector value; }; inline bool has_old_api_map_order(const std::shared_ptr& node) { @@ -68,13 +70,12 @@ inline bool has_old_api_map_order(const std::shared_ptr& node) { inline OldApiMapOrder get_old_api_map_order(const std::shared_ptr& node) { const auto& rt_map = node->get_rt_info(); - const auto& var = rt_map.at(OldApiMapOrder::get_type_info_static()); - return ngraph::as_type_ptr(var)->get(); + return rt_map.at(OldApiMapOrder::get_type_info_static()).as(); } inline void set_old_api_map_order(std::shared_ptr& node, const OldApiMapOrder& old_api_map) { auto& rt_map = node->get_rt_info(); - rt_map[OldApiMapOrder::get_type_info_static()] = std::make_shared(old_api_map); + rt_map[OldApiMapOrder::get_type_info_static()] = old_api_map; } } // namespace ov diff --git a/src/common/transformations/include/transformations/rt_info/primitives_priority_attribute.hpp b/src/common/transformations/include/transformations/rt_info/primitives_priority_attribute.hpp index 49ccfaf674f..3db735d4379 100644 --- a/src/common/transformations/include/transformations/rt_info/primitives_priority_attribute.hpp +++ b/src/common/transformations/include/transformations/rt_info/primitives_priority_attribute.hpp @@ -18,6 +18,8 @@ #include #include +#include "openvino/core/runtime_attribute.hpp" + namespace ov { /** * @ingroup ie_runtime_attr_api @@ -26,20 +28,20 @@ namespace ov { */ NGRAPH_API std::string getPrimitivesPriority(const std::shared_ptr & node); -class NGRAPH_API PrimitivesPriority : public VariantImpl { +class NGRAPH_API PrimitivesPriority : public ov::RuntimeAttribute { public: OPENVINO_RTTI("primitives_priority", "0"); PrimitivesPriority() = default; - ~PrimitivesPriority() override = default; + PrimitivesPriority(const std::string &value) : value(value) {} - PrimitivesPriority(const value_type &value) : VariantImpl(value) {} - - Any merge(const ngraph::NodeVector & nodes) override; - - Any init(const std::shared_ptr & node) override; + Any merge(const ngraph::NodeVector & nodes) const override; bool visit_attributes(AttributeVisitor & visitor) override; + + std::string to_string() const override; + + std::string value; }; } // namespace ov diff --git a/src/common/transformations/include/transformations/rt_info/strides_property.hpp b/src/common/transformations/include/transformations/rt_info/strides_property.hpp index 01c1359f599..956f7e3c7a3 100644 --- a/src/common/transformations/include/transformations/rt_info/strides_property.hpp +++ b/src/common/transformations/include/transformations/rt_info/strides_property.hpp @@ -8,18 +8,17 @@ #include #include #include +#include "openvino/core/runtime_attribute.hpp" namespace ov { -class TRANSFORMATIONS_API StridesPropagation : public VariantImpl { -public: - OPENVINO_RTTI("strides_propagation", "0"); - - StridesPropagation() = default; - - StridesPropagation(const value_type& value) : VariantImpl(value) {} -}; - TRANSFORMATIONS_API bool has_strides_prop(const ngraph::Input& node); TRANSFORMATIONS_API ngraph::Strides get_strides_prop(const ngraph::Input& node); TRANSFORMATIONS_API void insert_strides_prop(ngraph::Input& node, const ngraph::Strides& strides); +class TRANSFORMATIONS_API StridesPropagation : public ov::RuntimeAttribute { +public: + OPENVINO_RTTI("strides_propagation", "0"); + StridesPropagation() = default; + StridesPropagation(const ngraph::Strides& value) : value{value} {} + ngraph::Strides value; +}; } // namespace ov diff --git a/src/common/transformations/src/transformations/common_optimizations/common_optimizations.cpp b/src/common/transformations/src/transformations/common_optimizations/common_optimizations.cpp index ed32b6a1ae5..da8a676f5bd 100644 --- a/src/common/transformations/src/transformations/common_optimizations/common_optimizations.cpp +++ b/src/common/transformations/src/transformations/common_optimizations/common_optimizations.cpp @@ -67,6 +67,7 @@ #include "transformations/op_conversions/convert_broadcast_to_tiles.hpp" #include "transformations/op_conversions/convert_gelu.hpp" #include "transformations/op_conversions/convert_interpolate1_to_interpolate4.hpp" +#include "transformations/op_conversions/detection_output_downgrade.hpp" #include "transformations/op_conversions/batch_norm_decomposition.hpp" #include "transformations/op_conversions/einsum_decomposition.hpp" #include "transformations/op_conversions/gelu7_downgrade.hpp" @@ -176,6 +177,7 @@ bool ngraph::pass::CommonOptimizations::run_on_function(std::shared_ptr(); manager.register_pass(); manager.register_pass(); // not plugins implemented priorbox8 + manager.register_pass(); auto fq_fusions = manager.register_pass(); fq_fusions->add_matcher(); diff --git a/src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp b/src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp index 5ff6b8e2ce9..1d39f6cc321 100644 --- a/src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp +++ b/src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp @@ -55,6 +55,7 @@ #include #include #include +#include NGRAPH_RTTI_DEFINITION(ngraph::pass::MOCTransformations, "MOCTransformations", 0); @@ -144,6 +145,9 @@ bool ngraph::pass::MOCTransformations::run_on_function(std::shared_ptradd_matcher(); common_fusions->add_matcher(); common_fusions->add_matcher(m_use_shapes); + if (m_use_shapes) { + common_fusions->add_matcher(); + } common_fusions->add_matcher(); common_fusions->add_matcher(); common_fusions->add_matcher(); diff --git a/src/common/transformations/src/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.cpp b/src/common/transformations/src/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.cpp new file mode 100644 index 00000000000..ad11f1c28a2 --- /dev/null +++ b/src/common/transformations/src/transformations/common_optimizations/nearest_neighbor_upsampling_fusion.cpp @@ -0,0 +1,336 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "itt.hpp" +#include "transformations/common_optimizations/nearest_neighbor_upsampling_fusion.hpp" +#include "transformations/utils/utils.hpp" + +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +namespace { +using namespace ngraph; + +// In the transformation, a constant for Multiply must have the following shape: +// [1, 1, S_1, 1, S_2, ..., 1, S_i, ..., 1, S_{r - 2}, 1], (1) +// where (r - 2) is number of spatial axes, and each S_i is a scale for the axis i. +// This function returns the vector with elements +// [S_1, S_2, ..., S_i, ..., S_{r - 2}], +// when the shape, 's', has the form (1), and the empty vector otherwise. +std::vector get_scales_from_mul_const_shape(const Shape& s, uint64_t input_rank) { + if (input_rank < 4 || s.size() != 2 * input_rank - 2) return {}; + + ngraph::Shape expected_shape(2 * input_rank - 2, 1); + std::vector scales(input_rank - 2); + for (uint64_t i = 1; i <= input_rank - 2; ++i) { + expected_shape[2 * i] = s[2 * i]; + scales[i - 1] = static_cast(s[2 * i]); + } + + if (s != expected_shape) return {}; + + return scales; +} + +bool check_concat_1(const std::shared_ptr& concat, const Shape& shape) { + size_t rank = shape.size(); + + const auto inputs = concat->input_values(); + size_t num_of_input_values = inputs.size(); + + if (num_of_input_values != 2 * rank - 2) return false; + + std::vector input_constants(num_of_input_values, 1); + for (size_t i = 1; i < num_of_input_values; ++i) { + const auto& current_input = std::dynamic_pointer_cast(inputs[i].get_node_shared_ptr()); + if (!current_input) return false; + + const auto current_input_axis = std::dynamic_pointer_cast(current_input->input_value(1).get_node_shared_ptr()); + if (!current_input_axis || current_input_axis->cast_vector() != std::vector{0}) return false; + + const auto unsqueezed_const = std::dynamic_pointer_cast(current_input->input_value(0).get_node_shared_ptr()); + if (!unsqueezed_const) return false; + + const auto unsqueezed_const_value = unsqueezed_const->cast_vector(); + if (unsqueezed_const_value.size() != 1) return false; + + input_constants[i] = unsqueezed_const_value[0]; + } + + std::vector expected_input_constants(num_of_input_values, 1); + for (size_t i = 1; i <= rank - 2; ++i) { + expected_input_constants[2 * i - 1] = static_cast(shape[i]); + } + expected_input_constants.back() = static_cast(shape.back()); + + if (input_constants != expected_input_constants) return false; + + return true; +} + +// In this transformation 'concat_2' must have r inputs (where r is an output rank of the root of the transformation pattern). +// And (r - 1) inputs must be unsqueezed constants, and the list of these constants is +// [newD_1, newD_2, ..., newD_{r - 2}, C], +// where C is number of channels in the output shape of the root of the transformation pattern. +// +// This function gets a new spatial shape from unsqueezed constants of 'concat_2', that is, the vector with elements +// [newD_1, newD_2, ..., newD_{r - 2}]. +std::vector get_new_spatial_shape_from_concat_2(const std::shared_ptr& concat, const Shape& input_shape) { + size_t rank = input_shape.size(); + + const auto inputs = concat->input_values(); + size_t num_of_input_values = inputs.size(); + + if (num_of_input_values != rank) return {}; + + std::vector input_constants(num_of_input_values - 1, 0); + + for (size_t i = 1; i < num_of_input_values; ++i) { + const auto& current_input = std::dynamic_pointer_cast(inputs[i].get_node_shared_ptr()); + if (!current_input) return {}; + + const auto current_input_axis = std::dynamic_pointer_cast(current_input->input_value(1).get_node_shared_ptr()); + if (!current_input_axis || current_input_axis->cast_vector() != std::vector{0}) return {}; + + const auto unsqueezed_const = std::dynamic_pointer_cast(current_input->input_value(0).get_node_shared_ptr()); + if (!unsqueezed_const) return {}; + + const auto unsqueezed_const_value = unsqueezed_const->cast_vector(); + if (unsqueezed_const_value.size() != 1) return {}; + + input_constants[i - 1] = unsqueezed_const_value[0]; + } + + if (input_constants.back() != static_cast(input_shape.back())) return {}; + + input_constants.pop_back(); + + return input_constants; +} +} // namespace + +NGRAPH_RTTI_DEFINITION(ngraph::pass::NearestNeighborUpsamplingFusion, "NearestNeighborUpsamplingFusion", 0); + +ngraph::pass::NearestNeighborUpsamplingFusion::NearestNeighborUpsamplingFusion() { + MATCHER_SCOPE(NearestNeighborUpsamplingFusion); + // This transformation looks for Interpolate layer implemented using simple operations, namely ShapeOf, StridedSlice, Concat, + // Reshape, Mul, and replaces found pattern with a sequence of Shape, StridedSlice, Const, Mul, Interpolate. + // Found pattern (for 4D case, in a general case the pattern is similar): + // + // |---------| + // | op | + // |---|-----| + // | shape: [N, H, W, C] |-----------| |----------------| + // |------------------------------------>|0 ShapeOf |------->|0 StridedSlice | + // | |-----------| |-------|--------| + // | | + // | | + // | |------------------|---------------------------| + // | | | + // | | | + // | | | + // | | | + // | |---------------| | |---------------| | + // | | Concat | | | Concat | | + // | | (concat_1) | | | (concat_2) | | + // | | | | | | | + // | | 0|<-----------------------| | 0|<----------| + // | | | | | + // | | | | | + // | | | |-------------| |------------| | | |-------------| |--------------| + // | | | | | | Constant | | | | | | Constant | + // | | | | 0|<--| value: H | | | | 0|<--| value: new_H | + // | | | | | |------------| | | | | |--------------| + // | | | | | | | | | + // | | | | Unsqueeze | |------------| | | | Unsqueeze | |------------| + // | | | | | | Constant | | | | | | Constant | + // | | 1|<-----| 1|<--| value: 0 | | 1|<-----| 1|<--| value: 0 | + // | | | |-------------| |------------| | | |-------------| |------------| + // | | | | | + // | | | |-------------| |------------| | | |-------------| |--------------| + // | | | | | | Constant | | | | | | Constant | + // | | | | 0|<--| value: 1 | | | | 0|<--| value: new_W | + // | | | | | |------------| | | | | |--------------| + // | | | | | | | | | + // | | | | Unsqueeze | |------------| | | | Unsqueeze | |------------| + // | | | | | | Constant | | | | | | Constant | + // | | 2|<-----| 1|<--| value: 0 | | 2|<-----| 1|<--| value: 0 | + // | | | |-------------| |------------| | | |-------------| |------------| + // | | | | | + // | | | |-------------| |------------| | | |-------------| |------------| + // | | | | | | Constant | | | | | | Constant | + // | | | | 0|<--| value: W | | | | 0|<--| value: C | + // | | | | | |------------| | | | | |------------| + // | | | | | | | | | + // | | | | Unsqueeze | |------------| | | | Unsqueeze | |------------| + // | | | | | | Constant | | | | | | Constant | + // | | 3|<-----| 1|<--| value: 0 | | 3|<-----| 1|<--| value: 0 | + // | | | |-------------| |------------| |------|--------| |-------------| |------------| + // | | | | + // | | | |-------------| |------------| | + // | | | | | | Constant | | + // | | | | 0|<--| value: 1 | | + // | | | | | |------------| | + // | | | | | | + // | | | | Unsqueeze | |------------| | + // | | | | | | Constant | | + // | | 4|<-----| 1|<--| value: 0 | | + // | | | |-------------| |------------| | + // | | | | + // | | | |-------------| |------------| | + // | | | | | | Constant | | + // | | | | 0|<--| value: C | | + // | | | | | |------------| | + // | | | | | | + // | | | | Unsqueeze | |------------| | + // | | | | | | Constant | | + // | | 5|<-----| 1|<--| value: 0 | | + // | |------|--------| |-------------| |------------| | + // | | | + // | | | + // | |------------| | + // | | | + // | |---------------| | | + // | | Reshape | | | + // |---->|0 (reshape_1) 1|<---| | + // | | | + // |-----|---------| | + // | | + // |----------| | + // | |-------------| |----------------| | + // |--->|0 Mul 1|<----| Const | | + // | (mul) | | (mul_const) | |--------| + // |-------------| |----------------| | + // | | + // | | + // | |---------------------| | + // | | Reshape | | + // |---------------------------------------->|0 (reshape_2) 1|<------| + // | | + // |---------------------| + // + // Here digits 0, 1, ..., are numbers of input ports of nodes. + // + // That is, we found the subgraph of the above mentioned form, where + // 1) an output rank r of 'op' is greater or equal to 4; + // 2) an output shape of 'op' has the form [N, D_1, D_2, ..., D_{r - 2}, C]; + // 3) unsqueezed constants for 'concat_1' are + // D_1 for the input port 1 of 'concat_1' and 1 for the input port 2 of 'concat_1'; + // D_2 for the input port 3 of 'concat_1' and 1 for the input port 4 of 'concat_1'; + // ... + // D_i for the input port 2 * (i - 1) + 1 of 'concat_1' and 1 for the input port 2 * i of 'concat_1'; + // ... + // D_{r - 2} for the input port 2 * ((r - 2) - 1) + 1 of 'concat_1' and 1 for the input port 2 * (r - 2) of 'concat_1'; + // C for the input port 2 * (r - 2) + 1 of 'concat_1'; + // 4) unsqueezed constants for 'concat_2' are + // newD_1 for the input port 1 of 'concat_1'; + // newD_2 for the input port 2 of 'concat_1'; + // ... + // newD_i for the input port i of 'concat_1'; + // ... + // newD_{r - 2} for the input port (r - 2) of 'concat_1'; + // C for the input port (r - 2) + 1 of 'concat_1'; + // 5) the shape of 'mul_const' is [1, 1, S_1, 1, S_2, ..., 1, S_i, ..., 1, S_{r - 2}, 1] where S_i is a scale for the axis i; + // 6) all elements of 'mul_const' are equal to 1.0. + // + // Such subgraph can be replaced by the Interpolate node with + // 1) mode='nearest' and shape_calculation_mode='scales'; + // 2) 'sizes' input as a constant with the value [newD_1, newD_2, ..., newD_i, ..., newD_{r - 2}]; + // 3) 'scales' input as a constant with the value [S_1, S_2, ..., S_i, ..., S_{r - 2}]; + // 4) 'axes' input as a constant with the value [1, 2, ..., r - 2]. + // + // Of course, the replacement shouldn't be done, if all S_i are equal to 1. + auto input = ngraph::pattern::any_input(pattern::has_static_shape()); + auto concat_1 = pattern::wrap_type(); + auto concat_2 = pattern::wrap_type(); + auto reshape_1 = pattern::wrap_type({input, concat_1}); + auto mul_const = pattern::wrap_type(pattern::has_static_shape()); + auto mul = pattern::wrap_type({reshape_1, mul_const}); + auto reshape_2 = pattern::wrap_type({mul, concat_2}); + + ngraph::matcher_pass_callback callback = [=](ngraph::pattern::Matcher& m) { + const auto &pattern_to_output = m.get_pattern_value_map(); + + const auto reshape_2_node = std::dynamic_pointer_cast(pattern_to_output.at(reshape_2).get_node_shared_ptr()); + const auto mul_node = std::dynamic_pointer_cast(pattern_to_output.at(mul).get_node_shared_ptr()); + if (!reshape_2_node || !mul_node) return false; + + const auto mul_const_node = std::dynamic_pointer_cast(pattern_to_output.at(mul_const).get_node_shared_ptr()); + if (!mul_const_node) return false; + + const auto reshape_1_node = std::dynamic_pointer_cast(pattern_to_output.at(reshape_1).get_node_shared_ptr()); + if (!reshape_1_node) return false; + + uint64_t input_rank = static_cast(reshape_1_node->get_input_partial_shape(0).rank().get_length()); + const auto mul_const_shape = mul_const_node->get_output_shape(0); + const auto scales = get_scales_from_mul_const_shape(mul_const_shape, input_rank); + if (scales.empty() || std::all_of(scales.begin(), scales.end(), [](float s) { return s == 1.0f;})) { return false; } + + const auto mul_const_value = mul_const_node->cast_vector(); + if (std::any_of(mul_const_value.begin(), mul_const_value.end(), [](float x){ return x != 1.0f; })) { return false; } + + const auto concat_1_node = std::dynamic_pointer_cast(pattern_to_output.at(concat_1).get_node_shared_ptr()); + if (!concat_1_node) return false; + + const auto input_shape = reshape_1_node->get_input_shape(0); + if (!check_concat_1(concat_1_node, input_shape)) return false; + + const auto concat_2_node = std::dynamic_pointer_cast(pattern_to_output.at(concat_2).get_node_shared_ptr()); + if (!concat_2_node) return false; + + const auto new_spatial_shape = get_new_spatial_shape_from_concat_2(concat_2_node, input_shape); + if (new_spatial_shape.empty()) return false; + + const auto ss_before_concat_1 = std::dynamic_pointer_cast(concat_1_node->input_value(0).get_node_shared_ptr()); + const auto ss_before_concat_2 = std::dynamic_pointer_cast(concat_2_node->input_value(0).get_node_shared_ptr()); + if (!ss_before_concat_1 || !ss_before_concat_2 || ss_before_concat_1.get() != ss_before_concat_2.get()) return false; + + const auto shapeof_node = std::dynamic_pointer_cast(ss_before_concat_1->input_value(0).get_node_shared_ptr()); + if (!shapeof_node) return false; + + const auto before_shapeof = shapeof_node->input_value(0); + const auto before_reshape_1 = reshape_1_node->input_value(0); + if (before_shapeof.get_node() != before_reshape_1.get_node()) return false; + + opset8::Interpolate::InterpolateAttrs attrs; + attrs.mode = opset8::Interpolate::InterpolateMode::NEAREST; + attrs.shape_calculation_mode = opset8::Interpolate::ShapeCalcMode::SCALES; + attrs.nearest_mode = opset8::Interpolate::NearestMode::ROUND_PREFER_FLOOR; + attrs.pads_begin = std::vector{0}; + attrs.pads_end = std::vector{0}; + attrs.antialias = false; + attrs.coordinate_transformation_mode = opset8::Interpolate::CoordinateTransformMode::HALF_PIXEL; + attrs.cube_coeff = -0.75f; + + const auto& input_node = pattern_to_output.at(input); + const auto& type = input_node.get_element_type(); + const auto scales_node = opset8::Constant::create(type, {scales.size()}, scales); + const auto sizes_node = opset8::Constant::create(element::i64, {new_spatial_shape.size()}, new_spatial_shape); + + std::vector axes(input_rank - 2); + std::iota(axes.begin(), axes.end(), static_cast(1)); + const auto axes_node = opset8::Constant::create(element::i64, {axes.size()}, axes); + + auto interpolate = register_new_node(before_shapeof, sizes_node, scales_node, axes_node, attrs); + + interpolate->set_friendly_name(reshape_2_node->get_friendly_name()); + copy_runtime_info({reshape_2_node, mul_node, mul_const_node, concat_1_node, concat_2_node, ss_before_concat_1, shapeof_node}, + {scales_node, sizes_node, axes_node, interpolate}); + replace_node(reshape_2_node, interpolate); + + return true; + }; + + auto m = std::make_shared(reshape_2, matcher_name); + register_matcher(m, callback); +} diff --git a/src/common/transformations/src/transformations/op_conversions/detection_output_downgrade.cpp b/src/common/transformations/src/transformations/op_conversions/detection_output_downgrade.cpp new file mode 100644 index 00000000000..1bb2f6734c1 --- /dev/null +++ b/src/common/transformations/src/transformations/op_conversions/detection_output_downgrade.cpp @@ -0,0 +1,80 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "transformations/op_conversions/detection_output_downgrade.hpp" + +#include +#include +#include +#include +#include + +#include "itt.hpp" + +using namespace std; +using namespace ngraph; +using namespace ngraph::op::util; + +NGRAPH_RTTI_DEFINITION(pass::ConvertDetectionOutput8ToDetectionOutput1, "ConvertDetectionOutput8ToDetectionOutput1", 0); + +pass::ConvertDetectionOutput8ToDetectionOutput1::ConvertDetectionOutput8ToDetectionOutput1() { + MATCHER_SCOPE(ConvertDetectionOutput8ToDetectionOutput1); + + auto detection_output_v8_pattern = pattern::wrap_type(); + + matcher_pass_callback callback = [=](pattern::Matcher& m) { + auto detection_output_v8_node = std::dynamic_pointer_cast(m.get_match_root()); + if (!detection_output_v8_node) + return false; + const auto& attributes_v8 = detection_output_v8_node->get_attrs(); + auto num_classes = detection_output_v8_node->compute_num_classes(attributes_v8); + + // the transformation is applicable only if the number of classes is deduced + if (num_classes.is_dynamic()) + return false; + + opset1::DetectionOutput::Attributes attributes_v1; + attributes_v1.background_label_id = attributes_v8.background_label_id; + attributes_v1.clip_after_nms = attributes_v8.clip_after_nms; + attributes_v1.clip_before_nms = attributes_v8.clip_before_nms; + attributes_v1.code_type = attributes_v8.code_type; + attributes_v1.confidence_threshold = attributes_v8.confidence_threshold; + attributes_v1.decrease_label_id = attributes_v8.decrease_label_id; + attributes_v1.input_height = attributes_v8.input_height; + attributes_v1.input_width = attributes_v8.input_width; + attributes_v1.keep_top_k = attributes_v8.keep_top_k; + attributes_v1.nms_threshold = attributes_v8.nms_threshold; + attributes_v1.normalized = attributes_v8.normalized; + attributes_v1.num_classes = num_classes.get_length(); + attributes_v1.objectness_score = attributes_v8.objectness_score; + attributes_v1.share_location = attributes_v8.share_location; + attributes_v1.top_k = attributes_v8.top_k; + attributes_v1.variance_encoded_in_target = attributes_v8.variance_encoded_in_target; + + std::shared_ptr detection_output_v1_node = nullptr; + if (detection_output_v8_node->get_input_size() == 3) { + detection_output_v1_node = make_shared(detection_output_v8_node->input_value(0), + detection_output_v8_node->input_value(1), + detection_output_v8_node->input_value(2), + attributes_v1); + } else if (detection_output_v8_node->get_input_size() == 5) { + detection_output_v1_node = make_shared(detection_output_v8_node->input_value(0), + detection_output_v8_node->input_value(1), + detection_output_v8_node->input_value(2), + detection_output_v8_node->input_value(3), + detection_output_v8_node->input_value(4), + attributes_v1); + } + if (!detection_output_v1_node) + return false; + + detection_output_v1_node->set_friendly_name(detection_output_v8_node->get_friendly_name()); + ngraph::copy_runtime_info(detection_output_v8_node, detection_output_v1_node); + ngraph::replace_node(detection_output_v8_node, detection_output_v1_node); + return true; + }; + + auto m = make_shared(detection_output_v8_pattern, matcher_name); + register_matcher(m, callback); +} diff --git a/src/common/transformations/src/transformations/op_conversions/detection_output_upgrade.cpp b/src/common/transformations/src/transformations/op_conversions/detection_output_upgrade.cpp new file mode 100644 index 00000000000..d254dc8d4a6 --- /dev/null +++ b/src/common/transformations/src/transformations/op_conversions/detection_output_upgrade.cpp @@ -0,0 +1,76 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "transformations/op_conversions/detection_output_upgrade.hpp" + +#include +#include +#include +#include +#include + +#include "itt.hpp" + +using namespace std; +using namespace ngraph; +using namespace ngraph::op::util; + +NGRAPH_RTTI_DEFINITION(pass::ConvertDetectionOutput1ToDetectionOutput8, "ConvertDetectionOutput1ToDetectionOutput8", 0); + +pass::ConvertDetectionOutput1ToDetectionOutput8::ConvertDetectionOutput1ToDetectionOutput8() { + MATCHER_SCOPE(ConvertDetectionOutput1ToDetectionOutput8); + + auto detection_output_v1_pattern = pattern::wrap_type(); + + matcher_pass_callback callback = [=](pattern::Matcher& m) { + auto detection_output_v1_node = std::dynamic_pointer_cast(m.get_match_root()); + if (!detection_output_v1_node) + return false; + + const auto& attributes_v1 = detection_output_v1_node->get_attrs(); + opset8::DetectionOutput::Attributes attributes_v8; + attributes_v8.background_label_id = attributes_v1.background_label_id; + attributes_v8.clip_after_nms = attributes_v1.clip_after_nms; + attributes_v8.clip_before_nms = attributes_v1.clip_before_nms; + attributes_v8.code_type = attributes_v1.code_type; + attributes_v8.confidence_threshold = attributes_v1.confidence_threshold; + attributes_v8.decrease_label_id = attributes_v1.decrease_label_id; + attributes_v8.input_height = attributes_v1.input_height; + attributes_v8.input_width = attributes_v1.input_width; + attributes_v8.keep_top_k = attributes_v1.keep_top_k; + attributes_v8.nms_threshold = attributes_v1.nms_threshold; + attributes_v8.normalized = attributes_v1.normalized; + attributes_v8.objectness_score = attributes_v1.objectness_score; + attributes_v8.share_location = attributes_v1.share_location; + attributes_v8.top_k = attributes_v1.top_k; + attributes_v8.variance_encoded_in_target = attributes_v1.variance_encoded_in_target; + + std::shared_ptr detection_output_v8_node = nullptr; + if (detection_output_v1_node->get_input_size() == 3) { + auto detection_output_v8_node = + make_shared(detection_output_v1_node->input_value(0), + detection_output_v1_node->input_value(1), + detection_output_v1_node->input_value(2), + attributes_v8); + } else if (detection_output_v1_node->get_input_size() == 5) { + auto detection_output_v8_node = + make_shared(detection_output_v1_node->input_value(0), + detection_output_v1_node->input_value(1), + detection_output_v1_node->input_value(2), + detection_output_v1_node->input_value(3), + detection_output_v1_node->input_value(4), + attributes_v8); + } + if (!detection_output_v8_node) + return false; + + detection_output_v8_node->set_friendly_name(detection_output_v1_node->get_friendly_name()); + ngraph::copy_runtime_info(detection_output_v1_node, detection_output_v8_node); + ngraph::replace_node(detection_output_v1_node, detection_output_v8_node); + return true; + }; + + auto m = make_shared(detection_output_v1_pattern, matcher_name); + register_matcher(m, callback); +} diff --git a/src/common/transformations/src/transformations/rt_info/attributes.cpp b/src/common/transformations/src/transformations/rt_info/attributes.cpp index 641fc08d2be..821c959932f 100644 --- a/src/common/transformations/src/transformations/rt_info/attributes.cpp +++ b/src/common/transformations/src/transformations/rt_info/attributes.cpp @@ -5,20 +5,20 @@ #include "transformations/rt_info/attributes.hpp" ov::pass::Attributes::Attributes() { - register_factory>(); + register_factory(); register_factory(); register_factory(); register_factory(); register_factory(); - register_factory(); register_factory(); register_factory(); register_factory(); register_factory(); register_factory(); + register_factory(); } -std::shared_ptr ov::pass::Attributes::create_by_type_info(const ov::DiscreteTypeInfo& type_info) { +ov::Any ov::pass::Attributes::create_by_type_info(const ov::DiscreteTypeInfo& type_info) { auto it_type = m_factory_registry.find(type_info); if (it_type != m_factory_registry.end()) { return it_type->second(); @@ -26,5 +26,3 @@ std::shared_ptr ov::pass::Attributes::create_by_type_info(const ov: return {}; } } - -ov::pass::Attributes::~Attributes() = default; diff --git a/src/common/transformations/src/transformations/rt_info/decompression.cpp b/src/common/transformations/src/transformations/rt_info/decompression.cpp index 01b45f2bfab..8538c53e23a 100644 --- a/src/common/transformations/src/transformations/rt_info/decompression.cpp +++ b/src/common/transformations/src/transformations/rt_info/decompression.cpp @@ -6,7 +6,7 @@ void ov::mark_as_decompression(const std::shared_ptr& node) { auto& rt_info = node->get_rt_info(); - rt_info[Decompression::get_type_info_static()] = std::make_shared(); + rt_info[Decompression::get_type_info_static()] = Decompression(); } void ov::unmark_as_decompression(const std::shared_ptr& node) { diff --git a/src/common/transformations/src/transformations/rt_info/disable_fp16_compression.cpp b/src/common/transformations/src/transformations/rt_info/disable_fp16_compression.cpp index 79513ddf2b7..286dbdcf7c2 100644 --- a/src/common/transformations/src/transformations/rt_info/disable_fp16_compression.cpp +++ b/src/common/transformations/src/transformations/rt_info/disable_fp16_compression.cpp @@ -6,7 +6,7 @@ void ov::disable_fp16_compression(const std::shared_ptr& node) { auto& rt_info = node->get_rt_info(); - rt_info[DisableFP16Compression::get_type_info_static()] = std::make_shared(); + rt_info[DisableFP16Compression::get_type_info_static()] = DisableFP16Compression{}; } void ov::enable_fp16_compression(const std::shared_ptr& node) { diff --git a/src/common/transformations/src/transformations/rt_info/nms_selected_indices.cpp b/src/common/transformations/src/transformations/rt_info/nms_selected_indices.cpp index 640bb29d08f..a9303ef6bc9 100644 --- a/src/common/transformations/src/transformations/rt_info/nms_selected_indices.cpp +++ b/src/common/transformations/src/transformations/rt_info/nms_selected_indices.cpp @@ -4,14 +4,11 @@ #include "transformations/rt_info/nms_selected_indices.hpp" -template class ov::VariantImpl; void ov::set_nms_selected_indices(Node * node) { - auto & rt_info = node->get_rt_info(); - rt_info[NmsSelectedIndices::get_type_info_static()] = std::make_shared(true); + node->get_rt_info().emplace(NmsSelectedIndices::get_type_info_static(), NmsSelectedIndices{}); } bool ov::has_nms_selected_indices(const Node * node) { - const auto & rt_info = node->get_rt_info(); - return rt_info.count(NmsSelectedIndices::get_type_info_static()); + return node->get_rt_info().count(NmsSelectedIndices::get_type_info_static()); } diff --git a/src/common/transformations/src/transformations/rt_info/old_api_map_element_type_attribute.cpp b/src/common/transformations/src/transformations/rt_info/old_api_map_element_type_attribute.cpp index 3aca75d542d..c4bbebf4972 100644 --- a/src/common/transformations/src/transformations/rt_info/old_api_map_element_type_attribute.cpp +++ b/src/common/transformations/src/transformations/rt_info/old_api_map_element_type_attribute.cpp @@ -7,6 +7,6 @@ using namespace ov; bool OldApiMapElementType::visit_attributes(AttributeVisitor& visitor) { - visitor.on_attribute("value", m_value); + visitor.on_attribute("value", value); return true; } diff --git a/src/common/transformations/src/transformations/rt_info/old_api_map_order_attribute.cpp b/src/common/transformations/src/transformations/rt_info/old_api_map_order_attribute.cpp index 2a9ec1a0659..a700edc1972 100644 --- a/src/common/transformations/src/transformations/rt_info/old_api_map_order_attribute.cpp +++ b/src/common/transformations/src/transformations/rt_info/old_api_map_order_attribute.cpp @@ -7,6 +7,6 @@ using namespace ov; bool OldApiMapOrder::visit_attributes(AttributeVisitor& visitor) { - visitor.on_attribute("value", m_value); + visitor.on_attribute("value", value); return true; } diff --git a/src/common/transformations/src/transformations/rt_info/strides_property.cpp b/src/common/transformations/src/transformations/rt_info/strides_property.cpp index 63ede121c4e..4903cc43070 100644 --- a/src/common/transformations/src/transformations/rt_info/strides_property.cpp +++ b/src/common/transformations/src/transformations/rt_info/strides_property.cpp @@ -5,17 +5,13 @@ #include "transformations/rt_info/strides_property.hpp" bool ov::has_strides_prop(const ngraph::Input& node) { - const auto& rt_map = node.get_rt_info(); - return rt_map.count(StridesPropagation::get_type_info_static()); + return node.get_rt_info().count(StridesPropagation::get_type_info_static()); } ngraph::Strides ov::get_strides_prop(const ngraph::Input& node) { - const auto& rt_map = node.get_rt_info(); - const auto& var = rt_map.at(StridesPropagation::get_type_info_static()); - return ngraph::as_type_ptr(var)->get(); + return node.get_rt_info().at(StridesPropagation::get_type_info_static()).as().value; } void ov::insert_strides_prop(ngraph::Input& node, const ngraph::Strides& strides) { - auto& rt_map = node.get_rt_info(); - rt_map[StridesPropagation::get_type_info_static()] = std::make_shared(strides); + node.get_rt_info().emplace(StridesPropagation::get_type_info_static(), StridesPropagation{strides}); } diff --git a/src/core/CMakeLists.txt b/src/core/CMakeLists.txt index 4cbb338d9e7..48bc4b6d698 100644 --- a/src/core/CMakeLists.txt +++ b/src/core/CMakeLists.txt @@ -93,6 +93,7 @@ file(GLOB_RECURSE smart_reshape_srcs ${CMAKE_CURRENT_SOURCE_DIR}/src/pass/smart_ file(GLOB_RECURSE rt_info_srcs ${CMAKE_CURRENT_SOURCE_DIR}/src/pass/rt_info/*.cpp) set_source_files_properties("${CMAKE_CURRENT_SOURCE_DIR}/src/pass/convert_precision.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/pass/convert_fp32_to_fp16.cpp" + "${CMAKE_CURRENT_SOURCE_DIR}/src/pass/fix_rt_info.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/pass/init_node_info.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/pass/serialize.cpp" "${CMAKE_CURRENT_SOURCE_DIR}/src/op/type_relaxed.cpp" diff --git a/src/core/include/ngraph/op/detection_output.hpp b/src/core/include/ngraph/op/detection_output.hpp index 5400d5b22bf..0bda488d7fc 100644 --- a/src/core/include/ngraph/op/detection_output.hpp +++ b/src/core/include/ngraph/op/detection_output.hpp @@ -15,5 +15,9 @@ namespace v0 { using ov::op::v0::DetectionOutput; } // namespace v0 using v0::DetectionOutput; + +namespace v8 { +using ov::op::v8::DetectionOutput; +} // namespace v8 } // namespace op } // namespace ngraph diff --git a/src/core/include/ngraph/op/util/detection_output_base.hpp b/src/core/include/ngraph/op/util/detection_output_base.hpp new file mode 100644 index 00000000000..79d66ef3857 --- /dev/null +++ b/src/core/include/ngraph/op/util/detection_output_base.hpp @@ -0,0 +1,16 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "ngraph/op/op.hpp" +#include "openvino/op/util/detection_output_base.hpp" + +namespace ngraph { +namespace op { +namespace util { +using ov::op::util::DetectionOutputBase; +} // namespace util +} // namespace op +} // namespace ngraph diff --git a/src/core/include/ngraph/variant.hpp b/src/core/include/ngraph/variant.hpp index a203a35d2df..df8716d69f4 100644 --- a/src/core/include/ngraph/variant.hpp +++ b/src/core/include/ngraph/variant.hpp @@ -8,20 +8,38 @@ #include #include "openvino/core/any.hpp" // used for ov::RTMap -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace ov { class Node; } namespace ngraph { using ov::Node; -using ov::VariantTypeInfo; +using VariantTypeInfo = ov::DiscreteTypeInfo; -using ov::Variant; -using ov::VariantImpl; -using ov::VariantWrapper; +using Variant = ov::RuntimeAttribute; +template +using VariantImpl = ov::RuntimeAttributeImpl; -using ov::make_variant; +template +using VariantWrapper = ov::RuntimeAttributeWrapper; + +template +inline std::shared_ptr make_variant(const T& p) { + return ov::make_runtime_attribute(p); +} + +template +inline std::shared_ptr make_variant(const char (&s)[N]) { + return ov::make_runtime_attribute(s); +} + +#if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) +template +inline std::shared_ptr make_variant(const wchar_t (&s)[N]) { + return ov::make_runtime_attribute(s); +} +#endif using ov::RTMap; } // namespace ngraph diff --git a/src/core/include/openvino/core/any.hpp b/src/core/include/openvino/core/any.hpp index db9ad76ee0c..01010add718 100644 --- a/src/core/include/openvino/core/any.hpp +++ b/src/core/include/openvino/core/any.hpp @@ -11,13 +11,14 @@ #include #include #include +#include #include #include #include #include "openvino/core/attribute_visitor.hpp" #include "openvino/core/except.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace InferenceEngine { class InferencePlugin; @@ -27,7 +28,7 @@ class ExecutableNetwork; namespace ov { class Node; -class Variant; +class RuntimeAttribute; class ParamMap; namespace runtime { @@ -84,6 +85,57 @@ class OPENVINO_API Any { constexpr static const bool value = sizeof(test>(nullptr)) == sizeof(char); }; + template + struct HasBaseMemberType { + template + static auto test(U*) -> decltype(std::is_class::value, std::true_type()) { + return {}; + } + template + static auto test(...) -> std::false_type { + return {}; + } + constexpr static const auto value = std::is_same(nullptr))>::value; + }; + + template + struct TupleToTypeIndex; + + template + struct TupleToTypeIndex> { + static std::vector get() { + return {typeid(Args)...}; + } + }; + + template + struct HasOnAttribute { + template + static auto test(U*) + -> decltype(std::declval().on_attribute(std::declval()), std::true_type()) { + return {}; + } + template + static auto test(...) -> std::false_type { + return {}; + } + constexpr static const auto value = std::is_same(nullptr))>::value; + }; + + template + struct Visitable { + template + static auto test(U*) + -> decltype(std::declval().visit_attributes(std::declval()), std::true_type()) { + return {}; + } + template + static auto test(...) -> std::false_type { + return {}; + } + constexpr static const auto value = std::is_same(nullptr))>::value; + }; + /** * @brief Base API of erased type */ @@ -93,23 +145,24 @@ class OPENVINO_API Any { using Ptr = std::shared_ptr; virtual const std::type_info& type_info() const = 0; + virtual std::vector base_type_info() const = 0; virtual const void* addressof() const = 0; void* addressof() { return const_cast(const_cast(this)->addressof()); } virtual Base::Ptr copy() const = 0; virtual bool equal(const Base& rhs) const = 0; - virtual void print(std::ostream&) const = 0; + virtual void print(std::ostream& os) const = 0; virtual const DiscreteTypeInfo& get_type_info() const = 0; - virtual std::shared_ptr as_variant() const; + virtual std::shared_ptr as_runtime_attribute() const; virtual bool is_copyable() const; virtual Any init(const std::shared_ptr& node); virtual Any merge(const std::vector>& nodes); virtual std::string to_string(); - std::string to_string() const; virtual bool visit_attributes(AttributeVisitor&); bool visit_attributes(AttributeVisitor&) const; + std::string to_string() const; template bool is() const { @@ -135,42 +188,47 @@ class OPENVINO_API Any { template struct Impl; template - struct Impl>::value>::type> + struct Impl>::value>::type> : public Base { const DiscreteTypeInfo& get_type_info() const override { - return static_cast(variant.get())->get_type_info(); + return static_cast(runtime_attribute.get())->get_type_info(); } - std::shared_ptr as_variant() const override { - return std::dynamic_pointer_cast(variant); + std::shared_ptr as_runtime_attribute() const override { + return std::static_pointer_cast(runtime_attribute); } bool is_copyable() const override { - return static_cast(variant.get())->is_copyable(); + return static_cast(runtime_attribute.get())->is_copyable(); } Any init(const std::shared_ptr& node) override { - return static_cast(variant.get())->init(node); + return static_cast(runtime_attribute.get())->init(node); } Any merge(const std::vector>& nodes) override { - return static_cast(variant.get())->merge(nodes); + return static_cast(runtime_attribute.get())->merge(nodes); } std::string to_string() override { - return static_cast(variant.get())->to_string(); - } - bool visit_attributes(AttributeVisitor& visitor) override { - return static_cast(variant.get())->visit_attributes(visitor); + return static_cast(runtime_attribute.get())->to_string(); } - Impl(const T& variant_) : variant{variant_} {} + bool visit_attributes(AttributeVisitor& visitor) override { + return static_cast(runtime_attribute.get())->visit_attributes(visitor); + } + + Impl(const T& runtime_attribute) : runtime_attribute{runtime_attribute} {} const std::type_info& type_info() const override { return typeid(T); } + std::vector base_type_info() const override { + return {typeid(std::shared_ptr)}; + } + const void* addressof() const override { - return std::addressof(variant); + return std::addressof(runtime_attribute); } Base::Ptr copy() const override { - return std::make_shared>(this->variant); + return std::make_shared>(this->runtime_attribute); } template @@ -186,20 +244,20 @@ class OPENVINO_API Any { bool equal(const Base& rhs) const override { if (rhs.is()) { - return equal_impl(this->variant, rhs.as()); + return equal_impl(this->runtime_attribute, rhs.as()); } return false; } void print(std::ostream& os) const override { - os << variant->to_string(); + os << runtime_attribute->to_string(); } - T variant; + T runtime_attribute; }; template - struct Impl>::value>::type> + struct Impl>::value>::type> : public Base { OPENVINO_RTTI(typeid(T).name()); @@ -218,6 +276,21 @@ class OPENVINO_API Any { return std::make_shared>(this->value); } + template + static std::vector base_type_info_impl( + typename std::enable_if::value, std::true_type>::type = {}) { + return TupleToTypeIndex::get(); + } + template + static std::vector base_type_info_impl( + typename std::enable_if::value, std::false_type>::type = {}) { + return {typeid(T)}; + } + + std::vector base_type_info() const override { + return base_type_info_impl(); + } + template static typename std::enable_if::value, bool>::type equal_impl(const U& rhs, const U& lhs) { @@ -251,7 +324,7 @@ class OPENVINO_API Any { T value; }; - friend class ::ov::Variant; + friend class ::ov::RuntimeAttribute; friend class ::ov::ParamMap; friend class ::InferenceEngine::InferencePlugin; friend class ::InferenceEngine::ExecutableNetwork; @@ -264,7 +337,7 @@ class OPENVINO_API Any { void impl_check() const; - mutable Base::Ptr _variant_impl; + mutable Base::Ptr _runtime_attribute_impl; Base::Ptr _impl; @@ -313,6 +386,12 @@ public: return any; } + /** + * Returns type info + * @return type info + */ + const std::type_info& type_info() const; + /** * Checks that any contains a value * @return false if any contains a value else false @@ -325,35 +404,18 @@ public: * @return true if type of value is correct */ template - typename std::enable_if>::value, bool>::type is() const { - if (_impl != nullptr) { - return _impl->type_info() == typeid(decay_t); - } else { - return false; - } - } - - /** - * @brief check the type of value in any - * @tparam T Type of value - * @return true if type of value is correct - */ - template - typename std::enable_if>::value, bool>::type is() const { + bool is() const { if (_impl != nullptr) { if (_impl->type_info() == typeid(decay_t)) { return true; - } else { - auto variant = _impl->as_variant(); - if (variant == nullptr) { - return false; - } else { - return std::dynamic_pointer_cast(variant) != nullptr; + } + for (const auto& type_index : _impl->base_type_info()) { + if (type_index == typeid(decay_t)) { + return true; } } - } else { - return false; } + return false; } /** @@ -362,30 +424,31 @@ public: * @return casted object */ template - typename std::enable_if>::value, T>::type&& as() && { + typename std::enable_if>::value, T>::type&& as() && { if (_impl == nullptr) { - _variant_impl = std::make_shared>>(T{}); - return _variant_impl->as(); + _runtime_attribute_impl = std::make_shared>>(T{}); + return _runtime_attribute_impl->as(); } else { if (_impl->type_info() == typeid(decay_t)) { return std::move(*static_cast*>(_impl->addressof())); } else { - auto variant = _impl->as_variant(); - if (variant == nullptr) { - throw ov::Exception{std::string{"Any does not contains pointer to variant. It contains "} + - _impl->type_info().name()}; + auto runtime_attribute = _impl->as_runtime_attribute(); + if (runtime_attribute == nullptr) { + throw ov::Exception{ + std::string{"Any does not contains pointer to runtime_attribute. It contains "} + + _impl->type_info().name()}; } - auto vptr = std::dynamic_pointer_cast(variant); - if (vptr == nullptr && T::element_type::get_type_info_static() != variant->get_type_info() && - T::element_type::get_type_info_static() != Variant::get_type_info_static()) { - throw ov::Exception{std::string{"Could not cast Any variant to "} + typeid(T).name() + " from " + - _impl->type_info().name() + "; from " + - static_cast(variant->get_type_info()) + " to " + + auto vptr = std::dynamic_pointer_cast(runtime_attribute); + if (vptr == nullptr && T::element_type::get_type_info_static() != runtime_attribute->get_type_info() && + T::element_type::get_type_info_static() != RuntimeAttribute::get_type_info_static()) { + throw ov::Exception{std::string{"Could not cast Any runtime_attribute to "} + typeid(T).name() + + " from " + _impl->type_info().name() + "; from " + + static_cast(runtime_attribute->get_type_info()) + " to " + static_cast(T::element_type::get_type_info_static())}; } - vptr = std::static_pointer_cast(variant); - _variant_impl = std::make_shared>>(vptr); - return _variant_impl->as(); + vptr = std::static_pointer_cast(runtime_attribute); + _runtime_attribute_impl = std::make_shared>>(vptr); + return _runtime_attribute_impl->as(); } } } @@ -396,30 +459,31 @@ public: * @return casted object */ template - typename std::enable_if>::value, T>::type& as() & { + typename std::enable_if>::value, T>::type& as() & { if (_impl == nullptr) { - _variant_impl = std::make_shared>>(T{}); - return _variant_impl->as(); + _runtime_attribute_impl = std::make_shared>>(T{}); + return _runtime_attribute_impl->as(); } else { if (_impl->type_info() == typeid(decay_t)) { return *static_cast*>(_impl->addressof()); } else { - auto variant = _impl->as_variant(); - if (variant == nullptr) { - throw ov::Exception{std::string{"Any does not contains pointer to variant. It contains "} + - _impl->type_info().name()}; + auto runtime_attribute = _impl->as_runtime_attribute(); + if (runtime_attribute == nullptr) { + throw ov::Exception{ + std::string{"Any does not contains pointer to runtime_attribute. It contains "} + + _impl->type_info().name()}; } - auto vptr = std::dynamic_pointer_cast(variant); - if (vptr == nullptr && T::element_type::get_type_info_static() != variant->get_type_info() && - T::element_type::get_type_info_static() != Variant::get_type_info_static()) { - throw ov::Exception{std::string{"Could not cast Any variant to "} + typeid(T).name() + " from " + - _impl->type_info().name() + "; from " + - static_cast(variant->get_type_info()) + " to " + + auto vptr = std::dynamic_pointer_cast(runtime_attribute); + if (vptr == nullptr && T::element_type::get_type_info_static() != runtime_attribute->get_type_info() && + T::element_type::get_type_info_static() != RuntimeAttribute::get_type_info_static()) { + throw ov::Exception{std::string{"Could not cast Any runtime_attribute to "} + typeid(T).name() + + " from " + _impl->type_info().name() + "; from " + + static_cast(runtime_attribute->get_type_info()) + " to " + static_cast(T::element_type::get_type_info_static())}; } - vptr = std::static_pointer_cast(variant); - _variant_impl = std::make_shared>>(vptr); - return _variant_impl->as(); + vptr = std::static_pointer_cast(runtime_attribute); + _runtime_attribute_impl = std::make_shared>>(vptr); + return _runtime_attribute_impl->as(); } } } @@ -430,30 +494,32 @@ public: * @return casted object */ template - const typename std::enable_if>::value, T>::type& as() const& { + const typename std::enable_if>::value, T>::type& as() + const& { if (_impl == nullptr) { - _variant_impl = std::make_shared>>(T{}); - return _variant_impl->as(); + _runtime_attribute_impl = std::make_shared>>(T{}); + return _runtime_attribute_impl->as(); } else { if (_impl->type_info() == typeid(decay_t)) { return *static_cast*>(_impl->addressof()); } else { - auto variant = _impl->as_variant(); - if (variant == nullptr) { - throw ov::Exception{std::string{"Any does not contains pointer to variant. It contains "} + - _impl->type_info().name()}; + auto runtime_attribute = _impl->as_runtime_attribute(); + if (runtime_attribute == nullptr) { + throw ov::Exception{ + std::string{"Any does not contains pointer to runtime_attribute. It contains "} + + _impl->type_info().name()}; } - auto vptr = std::dynamic_pointer_cast(variant); - if (vptr == nullptr && T::element_type::get_type_info_static() != variant->get_type_info() && - T::element_type::get_type_info_static() != Variant::get_type_info_static()) { - throw ov::Exception{std::string{"Could not cast Any variant to "} + typeid(T).name() + " from " + - _impl->type_info().name() + "; from " + - static_cast(variant->get_type_info()) + " to " + + auto vptr = std::dynamic_pointer_cast(runtime_attribute); + if (vptr == nullptr && T::element_type::get_type_info_static() != runtime_attribute->get_type_info() && + T::element_type::get_type_info_static() != RuntimeAttribute::get_type_info_static()) { + throw ov::Exception{std::string{"Could not cast Any runtime_attribute to "} + typeid(T).name() + + " from " + _impl->type_info().name() + "; from " + + static_cast(runtime_attribute->get_type_info()) + " to " + static_cast(T::element_type::get_type_info_static())}; } - vptr = std::static_pointer_cast(variant); - _variant_impl = std::make_shared>>(vptr); - return _variant_impl->as(); + vptr = std::static_pointer_cast(runtime_attribute); + _runtime_attribute_impl = std::make_shared>>(vptr); + return _runtime_attribute_impl->as(); } } } @@ -464,7 +530,7 @@ public: * @return casted object */ template - typename std::enable_if>::value, T>::type&& as() && { + typename std::enable_if>::value, T>::type&& as() && { impl_check(); _impl->type_check(typeid(decay_t)); return std::move(*static_cast*>(_impl->addressof())); @@ -476,10 +542,17 @@ public: * @return casted object */ template - typename std::enable_if>::value, T>::type& as() & { + typename std::enable_if>::value, T>::type& as() & { impl_check(); - _impl->type_check(typeid(decay_t)); - return *static_cast*>(_impl->addressof()); + if (_impl->type_info() == typeid(decay_t)) { + return *static_cast*>(_impl->addressof()); + } + for (const auto& type_index : _impl->base_type_info()) { + if (type_index == typeid(decay_t)) { + return *static_cast*>(_impl->addressof()); + } + } + throw ov::Exception{std::string{"Bad cast from: "} + _impl->type_info().name() + " to: " + typeid(T).name()}; } /** @@ -488,10 +561,18 @@ public: * @return casted object */ template - const typename std::enable_if>::value, T>::type& as() const& { + const typename std::enable_if>::value, T>::type& as() + const& { impl_check(); - _impl->type_check(typeid(decay_t)); - return *static_cast*>(_impl->addressof()); + if (_impl->type_info() == typeid(decay_t)) { + return *static_cast*>(_impl->addressof()); + } + for (const auto& type_index : _impl->base_type_info()) { + if (type_index == typeid(decay_t)) { + return *static_cast*>(_impl->addressof()); + } + } + throw ov::Exception{std::string{"Bad cast from: "} + _impl->type_info().name() + " to: " + typeid(T).name()}; } /** @@ -607,9 +688,9 @@ template <> struct AsTypePtr { template static std::shared_ptr call(const Any& any) { - if (any.is>()) { + try { return any.as>(); - } else { + } catch (...) { return {}; } } @@ -623,9 +704,9 @@ using RTMap = std::map; namespace std { template std::shared_ptr dynamic_pointer_cast(const ::ov::Any& any) { - if (any.is>()) { + try { return any.as>(); - } else { + } catch (...) { return {}; } } diff --git a/src/core/include/openvino/core/core.hpp b/src/core/include/openvino/core/core.hpp index 611af2ee1aa..f444ffd57cb 100644 --- a/src/core/include/openvino/core/core.hpp +++ b/src/core/include/openvino/core/core.hpp @@ -29,11 +29,11 @@ #include "openvino/core/partial_shape.hpp" #include "openvino/core/rank.hpp" #include "openvino/core/rtti.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/core/shape.hpp" #include "openvino/core/strides.hpp" #include "openvino/core/type.hpp" #include "openvino/core/validation_util.hpp" -#include "openvino/core/variant.hpp" #include "openvino/core/version.hpp" #include "openvino/core/visibility.hpp" diff --git a/src/core/include/openvino/core/descriptor/input.hpp b/src/core/include/openvino/core/descriptor/input.hpp index 6081e2c9250..1d5994cf05b 100644 --- a/src/core/include/openvino/core/descriptor/input.hpp +++ b/src/core/include/openvino/core/descriptor/input.hpp @@ -9,7 +9,7 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/descriptor/tensor.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace ov { class Node; diff --git a/src/core/include/openvino/core/descriptor/output.hpp b/src/core/include/openvino/core/descriptor/output.hpp index 30484f42dbb..dab168980f0 100644 --- a/src/core/include/openvino/core/descriptor/output.hpp +++ b/src/core/include/openvino/core/descriptor/output.hpp @@ -14,7 +14,7 @@ #include "openvino/core/descriptor/input.hpp" #include "openvino/core/descriptor/tensor.hpp" #include "openvino/core/node_output.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace ov { // The forward declaration of Node is needed here because Node has a deque of diff --git a/src/core/include/openvino/core/descriptor/tensor.hpp b/src/core/include/openvino/core/descriptor/tensor.hpp index 10d3fa00da1..c7909eee1db 100644 --- a/src/core/include/openvino/core/descriptor/tensor.hpp +++ b/src/core/include/openvino/core/descriptor/tensor.hpp @@ -16,7 +16,6 @@ #include "openvino/core/partial_shape.hpp" #include "openvino/core/shape.hpp" #include "openvino/core/type/element_type.hpp" -#include "openvino/core/variant.hpp" namespace ngraph { namespace runtime { @@ -114,7 +113,7 @@ protected: std::string m_name; std::unordered_set m_names; - std::map m_rt_info; + RTMap m_rt_info; mutable std::atomic_bool m_shape_changed; }; diff --git a/src/core/include/openvino/core/function.hpp b/src/core/include/openvino/core/function.hpp index 9fb61bbc853..6e451845037 100644 --- a/src/core/include/openvino/core/function.hpp +++ b/src/core/include/openvino/core/function.hpp @@ -15,7 +15,6 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/node.hpp" #include "openvino/core/rtti.hpp" -#include "openvino/core/variant.hpp" #include "openvino/op/assign.hpp" #include "openvino/op/parameter.hpp" #include "openvino/op/read_value.hpp" diff --git a/src/core/include/openvino/core/layout.hpp b/src/core/include/openvino/core/layout.hpp index cf358babd4d..ef55e966a6e 100644 --- a/src/core/include/openvino/core/layout.hpp +++ b/src/core/include/openvino/core/layout.hpp @@ -11,7 +11,7 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/partial_shape.hpp" #include "openvino/core/rank.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace ov { @@ -161,15 +161,19 @@ protected: std::string m_dump; }; -class OPENVINO_API LayoutAttribute : public VariantImpl { +class OPENVINO_API LayoutAttribute : public ov::RuntimeAttribute { public: OPENVINO_RTTI("layout", "0"); LayoutAttribute() = default; - explicit LayoutAttribute(const Layout& value) : VariantImpl(value) {} + explicit LayoutAttribute(const Layout& value) : value(value) {} bool visit_attributes(AttributeVisitor& visitor) override; + + std::string to_string() const override; + + Layout value; }; } // namespace ov diff --git a/src/core/include/openvino/core/node.hpp b/src/core/include/openvino/core/node.hpp index 4abdd23b1ea..11cfe250b8f 100644 --- a/src/core/include/openvino/core/node.hpp +++ b/src/core/include/openvino/core/node.hpp @@ -31,9 +31,9 @@ #include "openvino/core/node_output.hpp" #include "openvino/core/node_vector.hpp" #include "openvino/core/rtti.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/core/strides.hpp" #include "openvino/core/type.hpp" -#include "openvino/core/variant.hpp" #include "openvino/op/util/attr_types.hpp" #include "openvino/op/util/variable.hpp" #include "openvino/op/util/variable_value.hpp" diff --git a/src/core/include/openvino/core/node_input.hpp b/src/core/include/openvino/core/node_input.hpp index 6004b34a836..c126f160472 100644 --- a/src/core/include/openvino/core/node_input.hpp +++ b/src/core/include/openvino/core/node_input.hpp @@ -10,9 +10,9 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/descriptor/tensor.hpp" #include "openvino/core/partial_shape.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/core/shape.hpp" #include "openvino/core/type/element_type.hpp" -#include "openvino/core/variant.hpp" namespace ov { class Node; diff --git a/src/core/include/openvino/core/node_output.hpp b/src/core/include/openvino/core/node_output.hpp index 83e89a0a4fd..2ceeba995e3 100644 --- a/src/core/include/openvino/core/node_output.hpp +++ b/src/core/include/openvino/core/node_output.hpp @@ -11,9 +11,9 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/descriptor/tensor.hpp" #include "openvino/core/partial_shape.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/core/shape.hpp" #include "openvino/core/type/element_type.hpp" -#include "openvino/core/variant.hpp" namespace ov { class Node; diff --git a/src/core/include/openvino/core/preprocess/input_tensor_info.hpp b/src/core/include/openvino/core/preprocess/input_tensor_info.hpp index 0f56e973dd2..fc7c25e701a 100644 --- a/src/core/include/openvino/core/preprocess/input_tensor_info.hpp +++ b/src/core/include/openvino/core/preprocess/input_tensor_info.hpp @@ -13,18 +13,20 @@ namespace ov { namespace preprocess { -class OPENVINO_API TensorInfoMemoryType : public VariantImpl { +class OPENVINO_API TensorInfoMemoryType : public RuntimeAttribute { public: OPENVINO_RTTI("memory_type", "0"); TensorInfoMemoryType() = default; - explicit TensorInfoMemoryType(const std::string& value) : VariantImpl(value) {} + explicit TensorInfoMemoryType(const std::string& value) : value(value) {} bool visit_attributes(AttributeVisitor& visitor) override { - visitor.on_attribute("value", m_value); + visitor.on_attribute("value", value); return true; } + + std::string value; }; /// \brief Information about user's input tensor. By default, it will be initialized to same data (type/shape/etc) as diff --git a/src/core/include/openvino/core/runtime_attribute.hpp b/src/core/include/openvino/core/runtime_attribute.hpp new file mode 100644 index 00000000000..d9a9de0047d --- /dev/null +++ b/src/core/include/openvino/core/runtime_attribute.hpp @@ -0,0 +1,112 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include +#include + +#include "openvino/core/core_visibility.hpp" +#include "openvino/core/node_vector.hpp" +#include "openvino/core/rtti.hpp" +#include "openvino/core/type.hpp" + +namespace ov { +class Node; +class AttributeVisitor; +class Any; + +class OPENVINO_API RuntimeAttribute { +public: + static const DiscreteTypeInfo& get_type_info_static() { + static const ::ov::DiscreteTypeInfo type_info{"RuntimeAttribute", 0}; + return type_info; + } + virtual const DiscreteTypeInfo& get_type_info() const { + return get_type_info_static(); + } + using Ptr = std::shared_ptr; + using Base = std::tuple<::ov::RuntimeAttribute>; + virtual ~RuntimeAttribute() = default; + virtual bool is_copyable() const; + virtual Any init(const std::shared_ptr& node) const; + virtual Any merge(const ov::NodeVector& nodes) const; + virtual std::string to_string() const; + virtual bool visit_attributes(AttributeVisitor&); + bool visit_attributes(AttributeVisitor& visitor) const { + return const_cast(this)->visit_attributes(visitor); + } +}; +OPENVINO_API std::ostream& operator<<(std::ostream& os, const RuntimeAttribute& attrubute); + +template +class RuntimeAttributeImpl : public RuntimeAttribute { +public: + OPENVINO_RTTI(typeid(VT).name()); + using value_type = VT; + + RuntimeAttributeImpl() = default; + + RuntimeAttributeImpl(const value_type& value) : m_value(value) {} + + const value_type& get() const { + return m_value; + } + value_type& get() { + return m_value; + } + void set(const value_type& value) { + m_value = value; + } + +protected: + value_type m_value; +}; + +extern template class OPENVINO_API RuntimeAttributeImpl; +extern template class OPENVINO_API RuntimeAttributeImpl; +extern template class OPENVINO_API RuntimeAttributeImpl; + +template +class RuntimeAttributeWrapper {}; + +template <> +class OPENVINO_API RuntimeAttributeWrapper : public RuntimeAttributeImpl { +public: + OPENVINO_RTTI("RuntimeAttributeWrapper"); + RuntimeAttributeWrapper(const value_type& value) : RuntimeAttributeImpl(value) {} + std::string to_string() const override { + return m_value; + } +}; + +template <> +class OPENVINO_API RuntimeAttributeWrapper : public RuntimeAttributeImpl { +public: + OPENVINO_RTTI("RuntimeAttributeWrapper"); + RuntimeAttributeWrapper(const value_type& value) : RuntimeAttributeImpl(value) {} + std::string to_string() const override { + return std::to_string(m_value); + } +}; + +template +inline std::shared_ptr make_runtime_attribute(const T& p) { + return std::static_pointer_cast(std::make_shared>(p)); +} + +template +inline std::shared_ptr make_runtime_attribute(const char (&s)[N]) { + return std::static_pointer_cast(std::make_shared>(s)); +} + +#if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) +template +inline std::shared_ptr make_runtime_attribute(const wchar_t (&s)[N]) { + return std::static_pointer_cast(std::make_shared>(s)); +} +#endif + +using RuntimeAttributeVector = std::vector; +} // namespace ov diff --git a/src/core/include/openvino/core/variant.hpp b/src/core/include/openvino/core/variant.hpp deleted file mode 100644 index 015fb744bf1..00000000000 --- a/src/core/include/openvino/core/variant.hpp +++ /dev/null @@ -1,114 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#pragma once - -#include -#include - -#include "openvino/core/core_visibility.hpp" -#include "openvino/core/node_vector.hpp" -#include "openvino/core/rtti.hpp" -#include "openvino/core/type.hpp" - -namespace ov { -class Node; -class AttributeVisitor; -class Any; -using VariantTypeInfo = DiscreteTypeInfo; - -class OPENVINO_API Variant { -public: - static const ::ov::DiscreteTypeInfo& get_type_info_static() { - static const ::ov::DiscreteTypeInfo type_info{"Variant", 0}; - return type_info; - } - virtual ~Variant(); - virtual const VariantTypeInfo& get_type_info() const { - return get_type_info_static(); - } - virtual bool is_copyable() const; - virtual Any init(const std::shared_ptr& node); - virtual Any merge(const ov::NodeVector& nodes); - virtual std::string to_string() { - return ""; - } - virtual bool visit_attributes(AttributeVisitor&) { - return false; - } - - using type_info_t = DiscreteTypeInfo; -}; - -template -class VariantImpl : public Variant { -public: - OPENVINO_RTTI(typeid(VT).name(), "0"); - using value_type = VT; - - VariantImpl() = default; - - VariantImpl(const value_type& value) : m_value(value) {} - - const value_type& get() const { - return m_value; - } - value_type& get() { - return m_value; - } - void set(const value_type& value) { - m_value = value; - } - -protected: - value_type m_value; -}; - -template <> -class VariantImpl : public Variant { -public: - using value_type = void; - - VariantImpl() = default; -}; - -extern template class OPENVINO_API VariantImpl; -extern template class OPENVINO_API VariantImpl; -extern template class OPENVINO_API VariantImpl; - -template -class VariantWrapper {}; - -template <> -class OPENVINO_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("VariantWrapper"); - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -template <> -class OPENVINO_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("VariantWrapper"); - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -template -inline std::shared_ptr make_variant(const T& p) { - return std::dynamic_pointer_cast>(std::make_shared>(p)); -} - -template -inline std::shared_ptr make_variant(const char (&s)[N]) { - return std::dynamic_pointer_cast>(std::make_shared>(s)); -} - -#if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) -template -inline std::shared_ptr make_variant(const wchar_t (&s)[N]) { - return std::dynamic_pointer_cast>(std::make_shared>(s)); -} -#endif -using VariantVector = std::vector>; -} // namespace ov diff --git a/src/core/include/openvino/op/detection_output.hpp b/src/core/include/openvino/op/detection_output.hpp index ea88903c753..4bc2e542b27 100644 --- a/src/core/include/openvino/op/detection_output.hpp +++ b/src/core/include/openvino/op/detection_output.hpp @@ -5,34 +5,20 @@ #pragma once #include "openvino/op/op.hpp" +#include "openvino/op/util/detection_output_base.hpp" namespace ov { namespace op { namespace v0 { /// \brief Layer which performs non-max suppression to /// generate detection output using location and confidence predictions -class OPENVINO_API DetectionOutput : public Op { +class OPENVINO_API DetectionOutput : public op::util::DetectionOutputBase { public: - struct Attributes { + struct Attributes : public op::util::DetectionOutputBase::AttributesBase { int num_classes; - int background_label_id = 0; - int top_k = -1; - bool variance_encoded_in_target = false; - std::vector keep_top_k; - std::string code_type = std::string{"caffe.PriorBoxParameter.CORNER"}; - bool share_location = true; - float nms_threshold; - float confidence_threshold = 0; - bool clip_after_nms = false; - bool clip_before_nms = false; - bool decrease_label_id = false; - bool normalized = false; - size_t input_height = 1; - size_t input_width = 1; - float objectness_score = 0; }; - OPENVINO_OP("DetectionOutput", "opset1"); + OPENVINO_OP("DetectionOutput", "opset1", op::util::DetectionOutputBase); BWDCMP_RTTI_DECLARATION; DetectionOutput() = default; @@ -75,5 +61,55 @@ private: Attributes m_attrs; }; } // namespace v0 + +namespace v8 { +/// \brief Layer which performs non-max suppression to +/// generate detection output using location and confidence predictions +class OPENVINO_API DetectionOutput : public op::util::DetectionOutputBase { +public: + using Attributes = op::util::DetectionOutputBase::AttributesBase; + + OPENVINO_OP("DetectionOutput", "opset8", op::util::DetectionOutputBase); + + DetectionOutput() = default; + /// \brief Constructs a DetectionOutput operation + /// + /// \param box_logits Box logits + /// \param class_preds Class predictions + /// \param proposals Proposals + /// \param aux_class_preds Auxilary class predictions + /// \param aux_box_preds Auxilary box predictions + /// \param attrs Detection Output attributes + DetectionOutput(const Output& box_logits, + const Output& class_preds, + const Output& proposals, + const Output& aux_class_preds, + const Output& aux_box_preds, + const Attributes& attrs); + + /// \brief Constructs a DetectionOutput operation + /// + /// \param box_logits Box logits + /// \param class_preds Class predictions + /// \param proposals Proposals + /// \param attrs Detection Output attributes + DetectionOutput(const Output& box_logits, + const Output& class_preds, + const Output& proposals, + const Attributes& attrs); + + void validate_and_infer_types() override; + + std::shared_ptr clone_with_new_inputs(const OutputVector& new_args) const override; + + const Attributes& get_attrs() const { + return m_attrs; + } + bool visit_attributes(AttributeVisitor& visitor) override; + +private: + Attributes m_attrs; +}; +} // namespace v8 } // namespace op } // namespace ov diff --git a/src/core/include/openvino/op/util/detection_output_base.hpp b/src/core/include/openvino/op/util/detection_output_base.hpp new file mode 100644 index 00000000000..4a8030ae891 --- /dev/null +++ b/src/core/include/openvino/op/util/detection_output_base.hpp @@ -0,0 +1,45 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#pragma once + +#include "openvino/op/op.hpp" + +namespace ov { +namespace op { +namespace util { +/// \brief DetectionOutputBase basic class for DetectionOutput v0 and v8 +class OPENVINO_API DetectionOutputBase : public Op { +public: + struct AttributesBase { + int background_label_id = 0; + int top_k = -1; + bool variance_encoded_in_target = false; + std::vector keep_top_k; + std::string code_type = std::string{"caffe.PriorBoxParameter.CORNER"}; + bool share_location = true; + float nms_threshold; + float confidence_threshold = 0; + bool clip_after_nms = false; + bool clip_before_nms = false; + bool decrease_label_id = false; + bool normalized = false; + size_t input_height = 1; + size_t input_width = 1; + float objectness_score = 0; + }; + + OPENVINO_OP("DetectionOutputBase", "util"); + DetectionOutputBase() = default; + DetectionOutputBase(const OutputVector& args); + + void validate_and_infer_types_base(const AttributesBase& attrs, Dimension num_classes); + + bool visit_attributes_base(AttributeVisitor& visitor, AttributesBase& attrs); + + Dimension compute_num_classes(const AttributesBase& attrs); +}; +} // namespace util +} // namespace op +} // namespace ov diff --git a/src/core/include/openvino/op/util/precision_sensitive_attribute.hpp b/src/core/include/openvino/op/util/precision_sensitive_attribute.hpp index eb0f7308d4b..267d5ef1f60 100644 --- a/src/core/include/openvino/op/util/precision_sensitive_attribute.hpp +++ b/src/core/include/openvino/op/util/precision_sensitive_attribute.hpp @@ -6,7 +6,7 @@ #include "openvino/core/core_visibility.hpp" #include "openvino/core/node.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" namespace ov { @@ -21,7 +21,7 @@ bool OPENVINO_API is_precision_sensitive(const ov::Input& node_input); * input to an operation as a precision sensitive and disables compression to FP16 * of the subgraph before this input. */ -class OPENVINO_API PrecisionSensitive : public VariantImpl { +class OPENVINO_API PrecisionSensitive : public RuntimeAttribute { public: OPENVINO_RTTI("precision_sensitive", "0"); diff --git a/src/core/include/openvino/op/util/variable_context.hpp b/src/core/include/openvino/op/util/variable_context.hpp index 463e57e2691..03e9935f092 100644 --- a/src/core/include/openvino/op/util/variable_context.hpp +++ b/src/core/include/openvino/op/util/variable_context.hpp @@ -8,7 +8,7 @@ #include #include "openvino/core/node_vector.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/op/util/variable.hpp" #include "openvino/op/util/variable_value.hpp" @@ -73,16 +73,4 @@ private: }; } // namespace util } // namespace op -template <> -class OPENVINO_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("VariantWrapper"); - BWDCMP_RTTI_DECLARATION; - - explicit VariantWrapper(const value_type& value) : VariantImpl(value) {} - -private: - using Variant::init; - using Variant::merge; -}; } // namespace ov diff --git a/src/core/include/openvino/opsets/opset8_tbl.hpp b/src/core/include/openvino/opsets/opset8_tbl.hpp index a11dd507302..4f626f4fb91 100644 --- a/src/core/include/openvino/opsets/opset8_tbl.hpp +++ b/src/core/include/openvino/opsets/opset8_tbl.hpp @@ -31,7 +31,6 @@ _OPENVINO_OP_REG(Cosh, ov::op::v0) _OPENVINO_OP_REG(CumSum, ov::op::v0) _OPENVINO_OP_REG(DeformablePSROIPooling, ov::op::v1) _OPENVINO_OP_REG(DepthToSpace, ov::op::v0) -_OPENVINO_OP_REG(DetectionOutput, ov::op::v0) _OPENVINO_OP_REG(Divide, ov::op::v1) _OPENVINO_OP_REG(Elu, ov::op::v0) _OPENVINO_OP_REG(Erf, ov::op::v0) @@ -177,6 +176,7 @@ _OPENVINO_OP_REG(GatherND, ov::op::v8) _OPENVINO_OP_REG(AdaptiveAvgPool, ov::op::v8) _OPENVINO_OP_REG(AdaptiveMaxPool, ov::op::v8) _OPENVINO_OP_REG(DeformableConvolution, ov::op::v8) +_OPENVINO_OP_REG(DetectionOutput, ov::op::v8) _OPENVINO_OP_REG(I420toBGR, ov::op::v8) _OPENVINO_OP_REG(I420toRGB, ov::op::v8) _OPENVINO_OP_REG(MatrixNms, ov::op::v8) diff --git a/src/core/include/openvino/pass/constant_folding.hpp b/src/core/include/openvino/pass/constant_folding.hpp index 462d9916cb5..9e6128dc453 100644 --- a/src/core/include/openvino/pass/constant_folding.hpp +++ b/src/core/include/openvino/pass/constant_folding.hpp @@ -4,7 +4,7 @@ #pragma once -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "openvino/pass/pass.hpp" namespace ov { @@ -32,14 +32,10 @@ OPENVINO_API void enable_constant_folding(const std::shared_ptr& node); OPENVINO_API bool constant_folding_is_disabled(const std::shared_ptr& node); -class OPENVINO_API DisableConstantFolding : public VariantImpl { +class OPENVINO_API DisableConstantFolding : public ov::RuntimeAttribute { public: - OPENVINO_RTTI("disabled_constant_folding", "0"); - + OPENVINO_RTTI("DisableConstantFolding"); DisableConstantFolding() = default; - - DisableConstantFolding(const value_type& value) : VariantImpl(value) {} - bool is_copyable() const override { return false; } diff --git a/src/core/reference/include/ngraph/runtime/reference/detection_output.hpp b/src/core/reference/include/ngraph/runtime/reference/detection_output.hpp index ec137360c1c..cf27b56f1cf 100644 --- a/src/core/reference/include/ngraph/runtime/reference/detection_output.hpp +++ b/src/core/reference/include/ngraph/runtime/reference/detection_output.hpp @@ -10,6 +10,7 @@ #include #include "ngraph/op/detection_output.hpp" +#include "ngraph/op/util/detection_output_base.hpp" #include "ngraph/shape.hpp" namespace ngraph { @@ -28,7 +29,7 @@ private: }; using LabelBBox = std::map>; - ngraph::op::DetectionOutputAttrs attrs; + ngraph::op::util::DetectionOutputBase::AttributesBase attrs; size_t numImages; size_t priorSize; size_t numPriors; @@ -37,6 +38,7 @@ private: size_t offset; size_t numResults; size_t outTotalSize; + size_t numClasses; void GetLocPredictions(const dataType* locData, std::vector& locations) { locations.resize(numImages); @@ -64,12 +66,12 @@ private: for (size_t i = 0; i < numImages; ++i) { std::map>& labelScores = confPreds[i]; for (size_t p = 0; p < numPriors; ++p) { - int startIdx = p * attrs.num_classes; - for (int c = 0; c < attrs.num_classes; ++c) { + int startIdx = p * numClasses; + for (int c = 0; c < numClasses; ++c) { labelScores[c].push_back(confData[startIdx + c]); } } - confData += numPriors * attrs.num_classes; + confData += numPriors * numClasses; } } @@ -80,18 +82,18 @@ private: for (size_t i = 0; i < numImages; ++i) { std::map>& labelScores = confPreds[i]; for (size_t p = 0; p < numPriors; ++p) { - int startIdx = p * attrs.num_classes; + int startIdx = p * numClasses; if (armConfData[p * 2 + 1] < attrs.objectness_score) { - for (int c = 0; c < attrs.num_classes; ++c) { + for (int c = 0; c < numClasses; ++c) { c == attrs.background_label_id ? labelScores[c].push_back(1) : labelScores[c].push_back(0); } } else { - for (int c = 0; c < attrs.num_classes; ++c) { + for (int c = 0; c < numClasses; ++c) { labelScores[c].push_back(confData[startIdx + c]); } } } - confData += numPriors * attrs.num_classes; + confData += numPriors * numClasses; armConfData += numPriors * 2; } } @@ -369,7 +371,7 @@ private: for (size_t p = 0; p < numPriors; p++) { dataType conf = -1; int id = 0; - for (int c = 1; c < attrs.num_classes; c++) { + for (int c = 1; c < numClasses; c++) { if (attrs.background_label_id > -1 && c == attrs.background_label_id) continue; dataType temp = confScores.at(c)[p]; @@ -425,7 +427,25 @@ public: offset = _attrs.normalized ? 0 : 1; numPriors = priorsShape[2] / priorSize; priorsBatchSize = priorsShape[0]; - numLocClasses = _attrs.share_location ? 1 : static_cast(_attrs.num_classes); + numClasses = _attrs.num_classes; + numLocClasses = _attrs.share_location ? 1 : numClasses; + numResults = outShape[2]; + outTotalSize = shape_size(outShape); + } + + referenceDetectionOutput(const ngraph::op::util::DetectionOutputBase::AttributesBase& _attrs, + const ngraph::Shape& locShape, + const ngraph::Shape& classPredShape, + const ngraph::Shape& priorsShape, + const ngraph::Shape& outShape) + : attrs(_attrs) { + numImages = locShape[0]; + priorSize = _attrs.normalized ? 4 : 5; + offset = _attrs.normalized ? 0 : 1; + numPriors = priorsShape[2] / priorSize; + priorsBatchSize = priorsShape[0]; + numClasses = classPredShape[1] / numPriors; + numLocClasses = _attrs.share_location ? 1 : numClasses; numResults = outShape[2]; outTotalSize = shape_size(outShape); } @@ -469,7 +489,7 @@ public: int numDet = 0; if (!attrs.decrease_label_id) { // Caffe style - for (int c = 0; c < attrs.num_classes; ++c) { + for (int c = 0; c < numClasses; ++c) { if (c == attrs.background_label_id) { continue; } diff --git a/src/core/src/any.cpp b/src/core/src/any.cpp index 2c28f292e04..93036af1ba7 100644 --- a/src/core/src/any.cpp +++ b/src/core/src/any.cpp @@ -10,7 +10,7 @@ void Any::Base::type_check(const std::type_info& type_info_) const { OPENVINO_ASSERT(type_info() == type_info_, "Bad cast from: ", type_info().name(), " To type: ", type_info_.name()); } -std::shared_ptr Any::Base::as_variant() const { +std::shared_ptr Any::Base::as_runtime_attribute() const { return {}; } @@ -50,6 +50,11 @@ void Any::impl_check() const { OPENVINO_ASSERT(_impl != nullptr, "Any was not initialized."); } +const std::type_info& Any::type_info() const { + impl_check(); + return _impl->type_info(); +} + bool Any::empty() const { return _impl == nullptr; } diff --git a/src/core/src/function.cpp b/src/core/src/function.cpp index 88b4355ea6c..3dc78ca07b7 100644 --- a/src/core/src/function.cpp +++ b/src/core/src/function.cpp @@ -513,9 +513,7 @@ bool ov::Function::evaluate(const HostTensorVector& output_tensors, bool ov::Function::evaluate(ov::runtime::TensorVector& output_tensors, const ov::runtime::TensorVector& input_tensors, ov::EvaluationContext evaluation_context) const { - if (evaluation_context.find("VariableContext") == evaluation_context.end()) - evaluation_context["VariableContext"] = - std::make_shared>(ov::op::util::VariableContext()); + evaluation_context.emplace("VariableContext", ov::op::util::VariableContext()); std::map value_map; for (size_t i = 0; i < m_parameters.size(); ++i) { value_map[m_parameters.at(i)->output(0)] = input_tensors.at(i); diff --git a/src/core/src/layout.cpp b/src/core/src/layout.cpp index d583f13750f..afeafa46551 100644 --- a/src/core/src/layout.cpp +++ b/src/core/src/layout.cpp @@ -425,10 +425,14 @@ void AttributeAdapter::set(const std::string& value) { } bool LayoutAttribute::visit_attributes(AttributeVisitor& visitor) { - std::string layout_str = m_value.to_string(); + std::string layout_str = value.to_string(); visitor.on_attribute("layout", layout_str); - m_value = Layout(layout_str); + value = Layout(layout_str); return true; } +std::string LayoutAttribute::to_string() const { + return value.to_string(); +} + } // namespace ov diff --git a/src/core/src/node.cpp b/src/core/src/node.cpp index 259cda314de..edbb0291fcd 100644 --- a/src/core/src/node.cpp +++ b/src/core/src/node.cpp @@ -18,6 +18,7 @@ #include "ngraph/op/result.hpp" #include "ngraph/pattern/matcher.hpp" #include "openvino/core/descriptor/input.hpp" +#include "openvino/pass/constant_folding.hpp" #include "shared_node_info.hpp" using namespace std; @@ -781,7 +782,7 @@ OPENVINO_SUPPRESS_DEPRECATED_END bool ov::Node::constant_fold(OutputVector& output_values, const OutputVector& input_values) { OV_ITT_SCOPED_TASK(ov::itt::domains::nGraph, "Node::constant_fold"); - if (m_rt_info.count("disabled_constant_folding_0")) { + if (m_rt_info.count(ov::pass::DisableConstantFolding::get_type_info_static())) { return false; } diff --git a/src/core/src/op/assign.cpp b/src/core/src/op/assign.cpp index b951c86b186..68d09e7915e 100644 --- a/src/core/src/op/assign.cpp +++ b/src/core/src/op/assign.cpp @@ -100,15 +100,15 @@ bool op::v6::Assign::evaluate(const HostTensorVector& outputs, const auto& found_context = evaluation_context.find("VariableContext"); NODE_VALIDATION_CHECK(this, found_context != evaluation_context.end(), "VariableContext not found."); - auto variable_context = std::dynamic_pointer_cast>(found_context->second); - NODE_VALIDATION_CHECK(this, variable_context != nullptr, "Cannot cast found Context to VariableContext."); - const auto& variable_values = variable_context->get().get_variable_values(); + auto& variable_context = const_cast(found_context->second.as()); + + const auto& variable_values = variable_context.get_variable_values(); // automatically allocate memory if not provided by user if (variable_values.find(m_variable) == variable_values.end()) { auto host_tensor = std::make_shared(m_variable->get_info().data_type, m_variable->get_info().data_shape); - variable_context->get().set_variable_value(m_variable, make_shared(host_tensor)); + variable_context.set_variable_value(m_variable, make_shared(host_tensor)); } const auto var_value = variable_values.find(m_variable)->second; diff --git a/src/core/src/op/detection_output.cpp b/src/core/src/op/detection_output.cpp index f98d7438c66..27936f16bf6 100644 --- a/src/core/src/op/detection_output.cpp +++ b/src/core/src/op/detection_output.cpp @@ -8,15 +8,15 @@ using namespace std; +// ------------------------------ V0 ------------------------------ BWDCMP_RTTI_DEFINITION(ov::op::v0::DetectionOutput); - ov::op::v0::DetectionOutput::DetectionOutput(const Output& box_logits, const Output& class_preds, const Output& proposals, const Output& aux_class_preds, const Output& aux_box_preds, const Attributes& attrs) - : Op({box_logits, class_preds, proposals, aux_class_preds, aux_box_preds}), + : DetectionOutputBase({box_logits, class_preds, proposals, aux_class_preds, aux_box_preds}), m_attrs(attrs) { constructor_validate_and_infer_types(); } @@ -25,7 +25,7 @@ ov::op::v0::DetectionOutput::DetectionOutput(const Output& box_logits, const Output& class_preds, const Output& proposals, const Attributes& attrs) - : Op({box_logits, class_preds, proposals}), + : DetectionOutputBase({box_logits, class_preds, proposals}), m_attrs(attrs) { constructor_validate_and_infer_types(); } @@ -33,180 +33,7 @@ ov::op::v0::DetectionOutput::DetectionOutput(const Output& box_logits, void ov::op::v0::DetectionOutput::validate_and_infer_types() { NGRAPH_OP_SCOPE(v0_DetectionOutput_validate_and_infer_types); NODE_VALIDATION_CHECK(this, m_attrs.num_classes > 0, "Number of classes must be greater than zero"); - - NODE_VALIDATION_CHECK(this, m_attrs.keep_top_k.size() > 0, "keep_top_k attribute must be provided"); - - NODE_VALIDATION_CHECK(this, - m_attrs.code_type == "caffe.PriorBoxParameter.CORNER" || - m_attrs.code_type == "caffe.PriorBoxParameter.CENTER_SIZE", - "code_type must be either \"caffe.PriorBoxParameter.CORNER\" or " - "\"caffe.PriorBoxParameter.CENTER_SIZE\""); - - auto box_logits_et = get_input_element_type(0); - NODE_VALIDATION_CHECK(this, - box_logits_et.is_real(), - "Box logits' data type must be floating point. Got " + box_logits_et.get_type_name()); - auto class_preds_et = get_input_element_type(1); - NODE_VALIDATION_CHECK(this, - class_preds_et == box_logits_et, - "Class predictions' data type must be the same as box logits type (" + - box_logits_et.get_type_name() + "). Got " + class_preds_et.get_type_name()); - auto proposals_et = get_input_element_type(2); - NODE_VALIDATION_CHECK(this, - proposals_et.is_real(), - "Proposals' data type must be floating point. Got " + proposals_et.get_type_name()); - - const ov::PartialShape& box_logits_pshape = get_input_partial_shape(0); - const ov::PartialShape& class_preds_pshape = get_input_partial_shape(1); - const ov::PartialShape& proposals_pshape = get_input_partial_shape(2); - - int num_loc_classes = m_attrs.share_location ? 1 : m_attrs.num_classes; - int prior_box_size = m_attrs.normalized ? 4 : 5; - - Dimension num_images = Dimension::dynamic(); - Dimension num_prior_boxes = Dimension::dynamic(); - if (box_logits_pshape.rank().is_static()) { - NODE_VALIDATION_CHECK( - this, - box_logits_pshape.rank().get_length() == 2, - "Box logits rank must be 2. Got " + std::to_string(box_logits_pshape.rank().get_length())); - num_images = box_logits_pshape[0]; - if (box_logits_pshape[1].is_static()) { - NODE_VALIDATION_CHECK(this, - (box_logits_pshape[1].get_length() % (num_loc_classes * 4)) == 0, - "Box logits' second dimension must be a multiply of num_loc_classes * 4 (" + - std::to_string(num_loc_classes * 4) + "). Current value is: ", - box_logits_pshape[1].get_length(), - "."); - num_prior_boxes = box_logits_pshape[1].get_length() / (num_loc_classes * 4); - } - } - if (class_preds_pshape.rank().is_static()) { - NODE_VALIDATION_CHECK( - this, - class_preds_pshape.rank().get_length() == 2, - "Class predictions rank must be 2. Got " + std::to_string(class_preds_pshape.rank().get_length())); - if (num_images.is_dynamic() && class_preds_pshape[0].is_static()) { - num_images = class_preds_pshape[0]; - } else { - NODE_VALIDATION_CHECK(this, - class_preds_pshape[0].compatible(num_images), - "Class predictions' first dimension is not compatible with batch size."); - } - if (class_preds_pshape[1].is_static()) { - if (num_prior_boxes.is_dynamic()) { - NODE_VALIDATION_CHECK(this, - class_preds_pshape[1].get_length() % m_attrs.num_classes == 0, - "Class predictions' second dimension must be a multiply of num_classes (" + - std::to_string(m_attrs.num_classes) + "). Current value is: ", - class_preds_pshape[1].get_length(), - "."); - num_prior_boxes = class_preds_pshape[1].get_length() / m_attrs.num_classes; - } else { - int num_prior_boxes_val = num_prior_boxes.get_length(); - NODE_VALIDATION_CHECK(this, - class_preds_pshape[1].get_length() == num_prior_boxes_val * m_attrs.num_classes, - "Class predictions' second dimension must be equal to num_prior_boxes * " - "num_classes (" + - std::to_string(num_prior_boxes_val * m_attrs.num_classes) + - "). Current value is: ", - class_preds_pshape[1].get_length(), - "."); - } - } - } - if (proposals_pshape.rank().is_static()) { - NODE_VALIDATION_CHECK(this, - proposals_pshape.rank().get_length() == 3, - "Proposals rank must be 3. Got " + std::to_string(proposals_pshape.rank().get_length())); - if (num_images.is_static() && proposals_pshape[0].is_static()) { - int64_t proposals_1st_dim = proposals_pshape[0].get_length(); - int64_t num_images_val = num_images.get_length(); - NODE_VALIDATION_CHECK(this, - proposals_1st_dim == 1 || proposals_1st_dim == num_images_val, - "Proposals' first dimension is must be equal to either batch size (" + - std::to_string(num_images_val) + - ") or 1. Got: " + std::to_string(proposals_1st_dim) + "."); - } - if (proposals_pshape[1].is_static()) { - size_t proposals_expected_2nd_dim = m_attrs.variance_encoded_in_target ? 1 : 2; - NODE_VALIDATION_CHECK(this, - proposals_pshape[1].compatible(proposals_expected_2nd_dim), - "Proposals' second dimension is mismatched. Current value is: ", - proposals_pshape[1].get_length(), - ", expected: ", - proposals_expected_2nd_dim, - "."); - } - if (proposals_pshape[2].is_static()) { - if (num_prior_boxes.is_dynamic()) { - NODE_VALIDATION_CHECK(this, - proposals_pshape[2].get_length() % prior_box_size == 0, - "Proposals' third dimension must be a multiply of prior_box_size (" + - std::to_string(prior_box_size) + "). Current value is: ", - proposals_pshape[2].get_length(), - "."); - num_prior_boxes = proposals_pshape[2].get_length() / prior_box_size; - } else { - int num_prior_boxes_val = num_prior_boxes.get_length(); - NODE_VALIDATION_CHECK(this, - proposals_pshape[2].get_length() == num_prior_boxes_val * prior_box_size, - "Proposals' third dimension must be equal to num_prior_boxes " - "* prior_box_size (" + - std::to_string(num_prior_boxes_val * prior_box_size) + - "). Current value is: ", - proposals_pshape[2].get_length(), - "."); - } - } - } - - if (get_input_size() > 3) { - auto aux_class_preds_et = get_input_element_type(3); - NODE_VALIDATION_CHECK(this, - aux_class_preds_et == class_preds_et, - "Additional class predictions' data type must be the same as class " - "predictions data type (" + - class_preds_et.get_type_name() + "). Got " + aux_class_preds_et.get_type_name()); - auto aux_box_preds_et = get_input_element_type(4); - NODE_VALIDATION_CHECK(this, - aux_box_preds_et == box_logits_et, - "Additional box predictions' data type must be the same as box logits data type (" + - box_logits_et.get_type_name() + "). Got " + aux_box_preds_et.get_type_name()); - - const ov::PartialShape& aux_class_preds_pshape = get_input_partial_shape(3); - const ov::PartialShape& aux_box_preds_pshape = get_input_partial_shape(4); - if (aux_class_preds_pshape.rank().is_static()) { - NODE_VALIDATION_CHECK(this, - aux_class_preds_pshape[0].compatible(num_images), - "Additional class predictions' first dimension must be " - "compatible with batch size."); - if (num_prior_boxes.is_static()) { - int num_prior_boxes_val = num_prior_boxes.get_length(); - NODE_VALIDATION_CHECK(this, - aux_class_preds_pshape[1].get_length() == num_prior_boxes_val * 2, - "Additional class predictions' second dimension must be equal to " - "num_prior_boxes * 2 (" + - std::to_string(num_prior_boxes_val * 2) + "). Got " + - std::to_string(aux_class_preds_pshape[1].get_length()) + "."); - } - } - NODE_VALIDATION_CHECK(this, - aux_box_preds_pshape.compatible(box_logits_pshape), - "Additional box predictions' shape must be compatible with box logits shape."); - } - - std::vector output_shape{1, 1}; - if (m_attrs.keep_top_k[0] > 0) { - output_shape.push_back(num_images * m_attrs.keep_top_k[0]); - } else if (m_attrs.top_k > 0) { - output_shape.push_back(num_images * m_attrs.top_k * m_attrs.num_classes); - } else { - output_shape.push_back(num_images * num_prior_boxes * m_attrs.num_classes); - } - output_shape.emplace_back(7); - - set_output_type(0, box_logits_et, output_shape); + validate_and_infer_types_base(m_attrs, m_attrs.num_classes); } shared_ptr ov::op::v0::DetectionOutput::clone_with_new_inputs(const OutputVector& new_args) const { @@ -232,20 +59,58 @@ shared_ptr ov::op::v0::DetectionOutput::clone_with_new_inputs(const Ou bool ov::op::v0::DetectionOutput::visit_attributes(AttributeVisitor& visitor) { NGRAPH_OP_SCOPE(v0_DetectionOutput_visit_attributes); visitor.on_attribute("num_classes", m_attrs.num_classes); - visitor.on_attribute("background_label_id", m_attrs.background_label_id); - visitor.on_attribute("top_k", m_attrs.top_k); - visitor.on_attribute("variance_encoded_in_target", m_attrs.variance_encoded_in_target); - visitor.on_attribute("keep_top_k", m_attrs.keep_top_k); - visitor.on_attribute("code_type", m_attrs.code_type); - visitor.on_attribute("share_location", m_attrs.share_location); - visitor.on_attribute("nms_threshold", m_attrs.nms_threshold); - visitor.on_attribute("confidence_threshold", m_attrs.confidence_threshold); - visitor.on_attribute("clip_after_nms", m_attrs.clip_after_nms); - visitor.on_attribute("clip_before_nms", m_attrs.clip_before_nms); - visitor.on_attribute("decrease_label_id", m_attrs.decrease_label_id); - visitor.on_attribute("normalized", m_attrs.normalized); - visitor.on_attribute("input_height", m_attrs.input_height); - visitor.on_attribute("input_width", m_attrs.input_width); - visitor.on_attribute("objectness_score", m_attrs.objectness_score); + visit_attributes_base(visitor, m_attrs); + return true; +} + +// ------------------------------ V8 ------------------------------ +ov::op::v8::DetectionOutput::DetectionOutput(const Output& box_logits, + const Output& class_preds, + const Output& proposals, + const Output& aux_class_preds, + const Output& aux_box_preds, + const Attributes& attrs) + : DetectionOutputBase({box_logits, class_preds, proposals, aux_class_preds, aux_box_preds}), + m_attrs(attrs) { + constructor_validate_and_infer_types(); +} + +ov::op::v8::DetectionOutput::DetectionOutput(const Output& box_logits, + const Output& class_preds, + const Output& proposals, + const Attributes& attrs) + : DetectionOutputBase({box_logits, class_preds, proposals}), + m_attrs(attrs) { + constructor_validate_and_infer_types(); +} + +void ov::op::v8::DetectionOutput::validate_and_infer_types() { + NGRAPH_OP_SCOPE(v0_DetectionOutput_validate_and_infer_types); + validate_and_infer_types_base(m_attrs, Dimension::dynamic()); +} + +shared_ptr ov::op::v8::DetectionOutput::clone_with_new_inputs(const OutputVector& new_args) const { + NGRAPH_OP_SCOPE(v0_DetectionOutput_clone_with_new_inputs); + check_new_args_count(this, new_args); + + auto num_args = new_args.size(); + + NODE_VALIDATION_CHECK(this, num_args == 3 || num_args == 5, "DetectionOutput accepts 3 or 5 inputs."); + + if (num_args == 3) { + return make_shared(new_args.at(0), new_args.at(1), new_args.at(2), m_attrs); + } else { + return make_shared(new_args.at(0), + new_args.at(1), + new_args.at(2), + new_args.at(3), + new_args.at(4), + m_attrs); + } +} + +bool ov::op::v8::DetectionOutput::visit_attributes(AttributeVisitor& visitor) { + NGRAPH_OP_SCOPE(v0_DetectionOutput_visit_attributes); + visit_attributes_base(visitor, m_attrs); return true; } diff --git a/src/core/src/op/parameter.cpp b/src/core/src/op/parameter.cpp index ad74129c7a1..7f00f2ffe7f 100644 --- a/src/core/src/op/parameter.cpp +++ b/src/core/src/op/parameter.cpp @@ -51,22 +51,16 @@ void op::Parameter::set_is_relevant_to_shapes(bool is_relevant) { ov::Layout op::Parameter::get_layout() const { auto it = output(0).get_rt_info().find(ov::LayoutAttribute::get_type_info_static()); if (it == output(0).get_rt_info().end()) { - return ""; + return {}; } - auto layout = std::dynamic_pointer_cast(it->second); - OPENVINO_ASSERT(layout, - "'", - ov::LayoutAttribute::get_type_info_static(), - "' runtime info for parameter is invalid, use set_layout API"); - return layout->get(); + return it->second.as().value; } void op::Parameter::set_layout(const ov::Layout& layout) { if (layout.empty()) { output(0).get_rt_info().erase(ov::LayoutAttribute::get_type_info_static()); } else { - output(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = - std::make_shared(layout); + output(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = ov::LayoutAttribute(layout); } } diff --git a/src/core/src/op/read_value.cpp b/src/core/src/op/read_value.cpp index 1f0bc7fb194..8d677698dd1 100644 --- a/src/core/src/op/read_value.cpp +++ b/src/core/src/op/read_value.cpp @@ -92,9 +92,7 @@ bool op::v6::ReadValue::evaluate(const HostTensorVector& outputs, const auto& found_context = evaluation_context.find("VariableContext"); NODE_VALIDATION_CHECK(this, found_context != evaluation_context.end(), "VariableContext not found."); - auto variable_context = std::dynamic_pointer_cast>(found_context->second); - NODE_VALIDATION_CHECK(this, variable_context != nullptr, "Cannot cast found Context to VariableContext."); - const auto& variable_values = variable_context->get().get_variable_values(); + const auto& variable_values = found_context->second.as().get_variable_values(); const auto& var_value = variable_values.find(m_variable); bool use_context = var_value != variable_values.end() && !var_value->second->get_reset(); diff --git a/src/core/src/op/result.cpp b/src/core/src/op/result.cpp index 09595715ca0..91b0d1d99ce 100644 --- a/src/core/src/op/result.cpp +++ b/src/core/src/op/result.cpp @@ -72,20 +72,14 @@ ov::Layout op::Result::get_layout() const { if (it == input(0).get_rt_info().end()) { return {}; } - auto layout = std::dynamic_pointer_cast(it->second); - OPENVINO_ASSERT(layout, - "'", - ov::LayoutAttribute::get_type_info_static(), - "' runtime info for result is invalid, use set_layout API"); - return layout->get(); + return it->second.as().value; } void op::Result::set_layout(const ov::Layout& layout) { if (layout.empty()) { input(0).get_rt_info().erase(ov::LayoutAttribute::get_type_info_static()); } else { - input(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = - std::make_shared(layout); + input(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = ov::LayoutAttribute{layout}; } } diff --git a/src/core/src/op/shape_of.cpp b/src/core/src/op/shape_of.cpp index 1ed1d31b446..16574b37a55 100644 --- a/src/core/src/op/shape_of.cpp +++ b/src/core/src/op/shape_of.cpp @@ -16,6 +16,7 @@ #include "ngraph/runtime/host_tensor.hpp" #include "ngraph/runtime/reference/shape_of.hpp" #include "ngraph/type/element_type_traits.hpp" +#include "openvino/pass/constant_folding.hpp" using namespace std; using namespace ngraph; @@ -182,7 +183,7 @@ bool op::v3::ShapeOf::evaluate_upper(const HostTensorVector& output_values) cons bool op::v3::ShapeOf::constant_fold(OutputVector& output_values, const OutputVector& input_values) { OV_ITT_SCOPED_TASK(ov::itt::domains::nGraph, "op::v3::ShapeOf::constant_fold"); - if (get_rt_info().count("disabled_constant_folding_0")) + if (get_rt_info().count(ov::pass::DisableConstantFolding::get_type_info_static())) return false; return shape_of::constant_fold_shape_of(this, output_values[0], input_values[0]); } @@ -241,7 +242,7 @@ bool op::v0::ShapeOf::has_evaluate() const { bool op::v0::ShapeOf::constant_fold(OutputVector& output_values, const OutputVector& input_values) { OV_ITT_SCOPED_TASK(ov::itt::domains::nGraph, "op::v0::ShapeOf::constant_fold"); - if (get_rt_info().count("disabled_constant_folding_0")) + if (get_rt_info().count(ov::pass::DisableConstantFolding::get_type_info_static())) return false; return shape_of::constant_fold_shape_of(this, output_values[0], input_values[0]); } diff --git a/src/core/src/op/util/detection_output_base.cpp b/src/core/src/op/util/detection_output_base.cpp new file mode 100644 index 00000000000..c8e2462c012 --- /dev/null +++ b/src/core/src/op/util/detection_output_base.cpp @@ -0,0 +1,340 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "ngraph/op/util/detection_output_base.hpp" + +#include + +#include "ngraph/op/concat.hpp" +#include "ngraph/op/constant.hpp" +#include "ngraph/op/squeeze.hpp" +#include "ngraph/runtime/host_tensor.hpp" +#include "ngraph/shape.hpp" + +using namespace std; +using namespace ov::op::util; + +DetectionOutputBase::DetectionOutputBase(const ov::OutputVector& args) : Op(args) {} + +ov::Dimension DetectionOutputBase::compute_num_classes(const AttributesBase& attrs) { + Dimension num_classes = Dimension::dynamic(); + + NODE_VALIDATION_CHECK(this, + 3 <= get_input_size() && get_input_size() <= 5, + "A number of arguments must be greater than or equal to 3 and less than or equal to 5. Got " + + std::to_string(get_input_size())); + + const ov::PartialShape& box_logits_pshape = get_input_partial_shape(0); + const ov::PartialShape& class_preds_pshape = get_input_partial_shape(1); + const ov::PartialShape& proposals_pshape = get_input_partial_shape(2); + ov::PartialShape ad_class_preds_shape = ov::PartialShape::dynamic(); + ov::PartialShape ad_box_preds_shape = ov::PartialShape::dynamic(); + + if (box_logits_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK( + this, + box_logits_pshape.rank().get_length() == 2, + "Box logits rank must be 2. Got " + std::to_string(box_logits_pshape.rank().get_length())); + } + if (class_preds_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK( + this, + class_preds_pshape.rank().get_length() == 2, + "Class predictions rank must be 2. Got " + std::to_string(class_preds_pshape.rank().get_length())); + } + if (proposals_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK(this, + proposals_pshape.rank().get_length() == 3, + "Proposals rank must be 3. Got " + std::to_string(proposals_pshape.rank().get_length())); + } + if (get_input_size() >= 4) { + ad_class_preds_shape = get_input_partial_shape(3); + if (ad_class_preds_shape.rank().is_static()) { + NODE_VALIDATION_CHECK(this, + ad_class_preds_shape.rank().get_length() == 2, + "Additional class predictions rank must be 2. Got " + + std::to_string(ad_class_preds_shape.rank().get_length())); + } + } + if (get_input_size() == 5) { + ad_box_preds_shape = get_input_partial_shape(4); + if (ad_box_preds_shape.rank().is_static()) { + NODE_VALIDATION_CHECK(this, + ad_box_preds_shape.rank().get_length() == 2, + "Additional box predictions rank must be 2. Got " + + std::to_string(ad_box_preds_shape.rank().get_length())); + } + } + + int prior_box_size = attrs.normalized ? 4 : 5; + Dimension num_prior_boxes = Dimension::dynamic(); + + // try to deduce a number of prior boxes + if (num_prior_boxes.is_dynamic() && proposals_pshape.rank().is_static() && proposals_pshape[2].is_static()) { + NODE_VALIDATION_CHECK(this, + proposals_pshape[2].get_length() % prior_box_size == 0, + "Proposals' third dimension must be a multiply of prior_box_size (" + + std::to_string(prior_box_size) + "). Current value is: ", + proposals_pshape[2].get_length(), + "."); + num_prior_boxes = proposals_pshape[2].get_length() / prior_box_size; + NODE_VALIDATION_CHECK( + this, + num_prior_boxes.get_length() > 0, + "A number of prior boxes must be greater zero. Got: " + std::to_string(num_prior_boxes.get_length())); + } + if (num_prior_boxes.is_dynamic() && ad_class_preds_shape.rank().is_static() && + ad_class_preds_shape[1].is_static()) { + NODE_VALIDATION_CHECK( + this, + ad_class_preds_shape[1].get_length() % 2 == 0, + "Additional class predictions second dimension must be a multiply of 2. Current value is: ", + ad_class_preds_shape[1].get_length(), + "."); + num_prior_boxes = ad_class_preds_shape[1].get_length() / 2; + NODE_VALIDATION_CHECK( + this, + num_prior_boxes.get_length() > 0, + "A number of prior boxes must be greater zero. Got: " + std::to_string(num_prior_boxes.get_length())); + } + + // try to deduce a number of classes + if (num_classes.is_dynamic() && num_prior_boxes.is_static() && class_preds_pshape.rank().is_static() && + class_preds_pshape[1].is_static()) { + NODE_VALIDATION_CHECK(this, + class_preds_pshape[1].get_length() % num_prior_boxes.get_length() == 0, + "Class predictions second dimension must be a multiply of num_prior_boxes (" + + std::to_string(num_prior_boxes.get_length()) + "). Current value is: ", + class_preds_pshape[1].get_length(), + "."); + num_classes = class_preds_pshape[1].get_length() / num_prior_boxes.get_length(); + } + if (num_classes.is_dynamic() && num_prior_boxes.is_static() && box_logits_pshape.rank().is_static() && + box_logits_pshape[1].is_static() && !attrs.share_location) { + NODE_VALIDATION_CHECK(this, + box_logits_pshape[1].get_length() % (num_prior_boxes.get_length() * 4) == 0, + "Box logits second dimension must be a multiply of num_prior_boxes * 4 (" + + std::to_string(num_prior_boxes.get_length() * 4) + "). Current value is: ", + box_logits_pshape[1].get_length(), + "."); + num_classes = box_logits_pshape[1].get_length() / (num_prior_boxes.get_length() * 4); + } + if (num_classes.is_dynamic() && num_prior_boxes.is_static() && ad_box_preds_shape.is_static() && + ad_box_preds_shape[1].is_static() && !attrs.share_location) { + NODE_VALIDATION_CHECK( + this, + ad_box_preds_shape[1].get_length() % (num_prior_boxes.get_length() * 4) == 0, + "Additional box predictions second dimension must be a multiply of num_prior_boxes * 4 (" + + std::to_string(num_prior_boxes.get_length() * 4) + "). Current value is: ", + ad_box_preds_shape[1].get_length(), + "."); + num_classes = ad_box_preds_shape[1].get_length() / (num_prior_boxes.get_length() * 4); + } + + return num_classes; +} + +void DetectionOutputBase::validate_and_infer_types_base(const DetectionOutputBase::AttributesBase& attrs, + ov::Dimension num_classes) { + NODE_VALIDATION_CHECK(this, attrs.keep_top_k.size() > 0, "keep_top_k attribute must be provided"); + + NODE_VALIDATION_CHECK( + this, + attrs.code_type == "caffe.PriorBoxParameter.CORNER" || attrs.code_type == "caffe.PriorBoxParameter.CENTER_SIZE", + "code_type must be either \"caffe.PriorBoxParameter.CORNER\" or " + "\"caffe.PriorBoxParameter.CENTER_SIZE\""); + + auto box_logits_et = get_input_element_type(0); + NODE_VALIDATION_CHECK(this, + box_logits_et.is_real(), + "Box logits' data type must be floating point. Got " + box_logits_et.get_type_name()); + auto class_preds_et = get_input_element_type(1); + NODE_VALIDATION_CHECK(this, + class_preds_et == box_logits_et, + "Class predictions' data type must be the same as box logits type (" + + box_logits_et.get_type_name() + "). Got " + class_preds_et.get_type_name()); + auto proposals_et = get_input_element_type(2); + NODE_VALIDATION_CHECK(this, + proposals_et.is_real(), + "Proposals' data type must be floating point. Got " + proposals_et.get_type_name()); + + const ov::PartialShape& box_logits_pshape = get_input_partial_shape(0); + const ov::PartialShape& class_preds_pshape = get_input_partial_shape(1); + const ov::PartialShape& proposals_pshape = get_input_partial_shape(2); + + // deduce a number of classes for DetectionOutput-8 + if (num_classes.is_dynamic()) { + num_classes = compute_num_classes(attrs); + } + + Dimension num_loc_classes = attrs.share_location ? 1 : num_classes; + int prior_box_size = attrs.normalized ? 4 : 5; + + Dimension num_images = Dimension::dynamic(); + Dimension num_prior_boxes = Dimension::dynamic(); + if (box_logits_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK( + this, + box_logits_pshape.rank().get_length() == 2, + "Box logits rank must be 2. Got " + std::to_string(box_logits_pshape.rank().get_length())); + num_images = box_logits_pshape[0]; + if (box_logits_pshape[1].is_static() && num_loc_classes.is_static()) { + NODE_VALIDATION_CHECK(this, + (box_logits_pshape[1].get_length() % (num_loc_classes.get_length() * 4)) == 0, + "Box logits' second dimension must be a multiply of num_loc_classes * 4 (" + + std::to_string(num_loc_classes.get_length() * 4) + "). Current value is: ", + box_logits_pshape[1].get_length(), + "."); + num_prior_boxes = box_logits_pshape[1].get_length() / (num_loc_classes.get_length() * 4); + } + } + if (class_preds_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK( + this, + class_preds_pshape.rank().get_length() == 2, + "Class predictions rank must be 2. Got " + std::to_string(class_preds_pshape.rank().get_length())); + if (num_images.is_dynamic() && class_preds_pshape[0].is_static()) { + num_images = class_preds_pshape[0]; + } else { + NODE_VALIDATION_CHECK(this, + class_preds_pshape[0].compatible(num_images), + "Class predictions' first dimension is not compatible with batch size."); + } + if (class_preds_pshape[1].is_static()) { + if (num_prior_boxes.is_dynamic() && num_classes.is_static()) { + NODE_VALIDATION_CHECK(this, + class_preds_pshape[1].get_length() % num_classes.get_length() == 0, + "Class predictions' second dimension must be a multiply of num_classes (" + + std::to_string(num_classes.get_length()) + "). Current value is: ", + class_preds_pshape[1].get_length(), + "."); + num_prior_boxes = class_preds_pshape[1].get_length() / num_classes.get_length(); + } else if (num_classes.is_static()) { + int num_prior_boxes_val = num_prior_boxes.get_length(); + NODE_VALIDATION_CHECK( + this, + class_preds_pshape[1].get_length() == num_prior_boxes_val * num_classes.get_length(), + "Class predictions' second dimension must be equal to num_prior_boxes * " + "num_classes (" + + std::to_string(num_prior_boxes_val * num_classes.get_length()) + "). Current value is: ", + class_preds_pshape[1].get_length(), + "."); + } + } + } + if (proposals_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK(this, + proposals_pshape.rank().get_length() == 3, + "Proposals rank must be 3. Got " + std::to_string(proposals_pshape.rank().get_length())); + if (num_images.is_static() && proposals_pshape[0].is_static()) { + int64_t proposals_1st_dim = proposals_pshape[0].get_length(); + int64_t num_images_val = num_images.get_length(); + NODE_VALIDATION_CHECK(this, + proposals_1st_dim == 1 || proposals_1st_dim == num_images_val, + "Proposals' first dimension is must be equal to either batch size (" + + std::to_string(num_images_val) + + ") or 1. Got: " + std::to_string(proposals_1st_dim) + "."); + } + if (proposals_pshape[1].is_static()) { + size_t proposals_expected_2nd_dim = attrs.variance_encoded_in_target ? 1 : 2; + NODE_VALIDATION_CHECK(this, + proposals_pshape[1].compatible(proposals_expected_2nd_dim), + "Proposals' second dimension is mismatched. Current value is: ", + proposals_pshape[1].get_length(), + ", expected: ", + proposals_expected_2nd_dim, + "."); + } + if (proposals_pshape[2].is_static()) { + if (num_prior_boxes.is_dynamic()) { + NODE_VALIDATION_CHECK(this, + proposals_pshape[2].get_length() % prior_box_size == 0, + "Proposals' third dimension must be a multiply of prior_box_size (" + + std::to_string(prior_box_size) + "). Current value is: ", + proposals_pshape[2].get_length(), + "."); + num_prior_boxes = proposals_pshape[2].get_length() / prior_box_size; + } else { + int num_prior_boxes_val = num_prior_boxes.get_length(); + NODE_VALIDATION_CHECK(this, + proposals_pshape[2].get_length() == num_prior_boxes_val * prior_box_size, + "Proposals' third dimension must be equal to num_prior_boxes " + "* prior_box_size (" + + std::to_string(num_prior_boxes_val * prior_box_size) + + "). Current value is: ", + proposals_pshape[2].get_length(), + "."); + } + } + } + + if (get_input_size() > 3) { + auto aux_class_preds_et = get_input_element_type(3); + NODE_VALIDATION_CHECK(this, + aux_class_preds_et == class_preds_et, + "Additional class predictions' data type must be the same as class " + "predictions data type (" + + class_preds_et.get_type_name() + "). Got " + aux_class_preds_et.get_type_name()); + auto aux_box_preds_et = get_input_element_type(4); + NODE_VALIDATION_CHECK(this, + aux_box_preds_et == box_logits_et, + "Additional box predictions' data type must be the same as box logits data type (" + + box_logits_et.get_type_name() + "). Got " + aux_box_preds_et.get_type_name()); + + const ov::PartialShape& aux_class_preds_pshape = get_input_partial_shape(3); + const ov::PartialShape& aux_box_preds_pshape = get_input_partial_shape(4); + if (aux_class_preds_pshape.rank().is_static()) { + NODE_VALIDATION_CHECK(this, + aux_class_preds_pshape[0].compatible(num_images), + "Additional class predictions' first dimension must be " + "compatible with batch size."); + if (num_prior_boxes.is_static()) { + int num_prior_boxes_val = num_prior_boxes.get_length(); + NODE_VALIDATION_CHECK(this, + aux_class_preds_pshape[1].get_length() == num_prior_boxes_val * 2, + "Additional class predictions' second dimension must be equal to " + "num_prior_boxes * 2 (" + + std::to_string(num_prior_boxes_val * 2) + "). Got " + + std::to_string(aux_class_preds_pshape[1].get_length()) + "."); + } + } + NODE_VALIDATION_CHECK(this, + aux_box_preds_pshape.compatible(box_logits_pshape), + "Additional box predictions' shape must be compatible with box logits shape."); + } + + std::vector output_shape{1, 1}; + if (attrs.keep_top_k[0] > 0) { + output_shape.push_back(num_images * attrs.keep_top_k[0]); + } else if (attrs.top_k > 0 && num_classes.is_static()) { + output_shape.push_back(num_images * attrs.top_k * num_classes); + } else if (num_classes.is_static()) { + output_shape.push_back(num_images * num_prior_boxes * num_classes); + } else { + output_shape.push_back(Dimension::dynamic()); + } + output_shape.emplace_back(7); + + set_output_type(0, box_logits_et, output_shape); +} + +bool ov::op::util::DetectionOutputBase::visit_attributes_base(AttributeVisitor& visitor, + DetectionOutputBase::AttributesBase& attrs) { + visitor.on_attribute("background_label_id", attrs.background_label_id); + visitor.on_attribute("top_k", attrs.top_k); + visitor.on_attribute("variance_encoded_in_target", attrs.variance_encoded_in_target); + visitor.on_attribute("keep_top_k", attrs.keep_top_k); + visitor.on_attribute("code_type", attrs.code_type); + visitor.on_attribute("share_location", attrs.share_location); + visitor.on_attribute("nms_threshold", attrs.nms_threshold); + visitor.on_attribute("confidence_threshold", attrs.confidence_threshold); + visitor.on_attribute("clip_after_nms", attrs.clip_after_nms); + visitor.on_attribute("clip_before_nms", attrs.clip_before_nms); + visitor.on_attribute("decrease_label_id", attrs.decrease_label_id); + visitor.on_attribute("normalized", attrs.normalized); + visitor.on_attribute("input_height", attrs.input_height); + visitor.on_attribute("input_width", attrs.input_width); + visitor.on_attribute("objectness_score", attrs.objectness_score); + return true; +} diff --git a/src/core/src/op/util/evaluation_context.cpp b/src/core/src/op/util/evaluation_context.cpp deleted file mode 100644 index e3680bca154..00000000000 --- a/src/core/src/op/util/evaluation_context.cpp +++ /dev/null @@ -1,9 +0,0 @@ -// Copyright (C) 2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#include - -#include "ngraph/op/util/variable_context.hpp" - -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); diff --git a/src/core/src/op/util/precision_sensitive_attribute.cpp b/src/core/src/op/util/precision_sensitive_attribute.cpp index 6591699a341..5664311b7f6 100644 --- a/src/core/src/op/util/precision_sensitive_attribute.cpp +++ b/src/core/src/op/util/precision_sensitive_attribute.cpp @@ -6,7 +6,7 @@ void ov::mark_as_precision_sensitive(ov::Input node_input) { auto& rt_info = node_input.get_rt_info(); - rt_info[PrecisionSensitive::get_type_info_static()] = std::make_shared(); + rt_info[PrecisionSensitive::get_type_info_static()] = PrecisionSensitive{}; } void ov::unmark_as_precision_sensitive(ov::Input node_input) { diff --git a/src/core/src/pass/constant_folding.cpp b/src/core/src/pass/constant_folding.cpp index 5be3c068808..ee25911f6a8 100644 --- a/src/core/src/pass/constant_folding.cpp +++ b/src/core/src/pass/constant_folding.cpp @@ -88,7 +88,7 @@ bool ngraph::pass::ConstantFolding::pre_calculated_values_folding(const std::sha if (status) { for (const auto& node : order) { const auto& rt_info = node->get_rt_info(); - if (rt_info.count("disabled_constant_folding_0")) { + if (rt_info.count(DisableConstantFolding::get_type_info_static())) { status = false; break; } @@ -122,16 +122,13 @@ bool ngraph::pass::ConstantFolding::pre_calculated_values_folding(const std::sha } void ov::pass::disable_constant_folding(const std::shared_ptr& node) { - auto& rt_info = node->get_rt_info(); - rt_info[DisableConstantFolding::get_type_info_static()] = std::make_shared(true); + node->get_rt_info().emplace(DisableConstantFolding::get_type_info_static(), DisableConstantFolding{}); } void ov::pass::enable_constant_folding(const std::shared_ptr& node) { - auto& rt_info = node->get_rt_info(); - rt_info.erase(DisableConstantFolding::get_type_info_static()); + node->get_rt_info().erase(DisableConstantFolding::get_type_info_static()); } bool ov::pass::constant_folding_is_disabled(const std::shared_ptr& node) { - const auto& rt_info = node->get_rt_info(); - return rt_info.count(DisableConstantFolding::get_type_info_static()); + return node->get_rt_info().count(DisableConstantFolding::get_type_info_static()); } diff --git a/src/core/src/pass/fix_rt_info.cpp b/src/core/src/pass/fix_rt_info.cpp new file mode 100644 index 00000000000..87ea78b7967 --- /dev/null +++ b/src/core/src/pass/fix_rt_info.cpp @@ -0,0 +1,58 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "transformations/fix_rt_info.hpp" + +#include +#include +#include +#include +#include + +#include "itt.hpp" +#include "transformations/rt_info/primitives_priority_attribute.hpp" + +NGRAPH_RTTI_DEFINITION(ngraph::pass::FixRtInfo, "FixRtInfo", 0); + +bool ngraph::pass::FixRtInfo::run_on_function(std::shared_ptr f) { + // TODO: enable conditional compile + // RUN_ON_FUNCTION_SCOPE(FixRtInfo); + + for (auto& node : f->get_ops()) { + // Recursively apply transformation for sub-graph based operations + if (auto sub_graph_node = std::dynamic_pointer_cast(node)) { + if (auto sub_graph = sub_graph_node->get_function()) { + run_on_function(sub_graph); + } + } + auto& rt_info = node->get_rt_info(); + { + auto it_info = rt_info.find("PrimitivesPriority"); + if (it_info != rt_info.end()) { + if (it_info->second.is>>()) { + rt_info.emplace(ov::PrimitivesPriority::get_type_info_static(), + ov::PrimitivesPriority{ + it_info->second.as>>()->get()}); + } else if (it_info->second.is()) { + rt_info.emplace(ov::PrimitivesPriority::get_type_info_static(), + it_info->second.as()); + } + if (it_info->second.is()) { + rt_info.emplace(ov::PrimitivesPriority::get_type_info_static(), + ov::PrimitivesPriority{it_info->second.as()}); + } + rt_info.erase(it_info); + } + } + { + auto it_info = rt_info.find("affinity"); + if (it_info != rt_info.end()) { + if (it_info->second.is>>()) { + it_info->second = it_info->second.as>>()->get(); + } + } + } + } + return false; +} diff --git a/src/core/src/pass/init_node_info.cpp b/src/core/src/pass/init_node_info.cpp index a40837fbd6d..e1dedd20b18 100644 --- a/src/core/src/pass/init_node_info.cpp +++ b/src/core/src/pass/init_node_info.cpp @@ -11,6 +11,7 @@ #include #include "itt.hpp" +#include "transformations/fix_rt_info.hpp" #include "transformations/rt_info/fused_names_attribute.hpp" #include "transformations/rt_info/primitives_priority_attribute.hpp" @@ -19,14 +20,6 @@ NGRAPH_RTTI_DEFINITION(ngraph::pass::InitNodeInfo, "InitNodeInfo", 0); bool ngraph::pass::InitNodeInfo::run_on_function(std::shared_ptr f) { // TODO: enable conditional compile // RUN_ON_FUNCTION_SCOPE(InitNodeInfo); - std::vector> attributes{ - std::make_shared>(ngraph::FusedNames())}; - - using VariantCreator = std::function(const std::string&)>; - std::map update_attributes{ - {"PrimitivesPriority", [](const std::string& value) -> std::shared_ptr { - return std::make_shared(value); - }}}; for (auto& node : f->get_ops()) { // Recursively apply transformation for sub-graph based operations @@ -36,27 +29,8 @@ bool ngraph::pass::InitNodeInfo::run_on_function(std::shared_ptrget_rt_info(); - // Default attributes initialization - for (auto& attr : attributes) { - // Skip initialization if attribute has been already set - if (rtInfo.count(attr->get_type_info())) - continue; - auto init_attr = attr->init(node); - if (!init_attr.empty()) { - rtInfo[attr->get_type_info()] = init_attr; - } - } - // Convert manually set attributes to appropriate VariantWrapper class instances - // all manually set attributes must belong to VariantWrapper class - for (auto& attr : update_attributes) { - if (rtInfo.count(attr.first)) { - if (auto variant_string = std::dynamic_pointer_cast>(rtInfo[attr.first])) { - rtInfo.erase(attr.first); - auto res = attr.second(variant_string->get()); - rtInfo[res->get_type_info()] = res; - } - } - } + rtInfo.emplace(FusedNames::get_type_info_static(), FusedNames{node->get_friendly_name()}); } + FixRtInfo{}.run_on_function(f); return false; } diff --git a/src/core/src/pass/low_latency.cpp b/src/core/src/pass/low_latency.cpp index 2bbbaab5d09..6cc86ed0b81 100644 --- a/src/core/src/pass/low_latency.cpp +++ b/src/core/src/pass/low_latency.cpp @@ -44,7 +44,7 @@ ngraph::pass::LowLatency::LowLatency() { } // Mark the TI layer to be unrolled. Enable unconditional ti unrolling for all plugins. auto& rt_info = sub_graph_op->get_rt_info(); - rt_info["UNROLL_TI"] = std::make_shared>(1); + rt_info["UNROLL_TI"] = std::make_shared>(1); int64_t variable_id = 0; std::vector> assigns; diff --git a/src/core/src/pass/rt_info/fused_names_attribute.cpp b/src/core/src/pass/rt_info/fused_names_attribute.cpp index 8a25cec593f..4d426e4349a 100644 --- a/src/core/src/pass/rt_info/fused_names_attribute.cpp +++ b/src/core/src/pass/rt_info/fused_names_attribute.cpp @@ -35,55 +35,52 @@ void FusedNames::fuseWith(const FusedNames& names) { } std::string ngraph::getFusedNames(const std::shared_ptr& node) { - const auto& rtInfo = node->get_rt_info(); - using FusedNamesWrapper = VariantWrapper; - - if (!rtInfo.count(FusedNamesWrapper::get_type_info_static())) - return {}; - - const auto& attr = rtInfo.at(FusedNamesWrapper::get_type_info_static()); - FusedNames fusedNames = ov::as_type_ptr(attr)->get(); - return fusedNames.getNames(); + if (node) { + const auto& rtInfo = node->get_rt_info(); + auto it_info = rtInfo.find(FusedNames::get_type_info_static()); + if (it_info != rtInfo.end()) { + return it_info->second.as().getNames(); + } else { + return {}; + } + } + return {}; } std::vector ngraph::getFusedNamesVector(const std::shared_ptr& node) { - if (!node) - return {}; - - const auto& rtInfo = node->get_rt_info(); - using FusedNamesWrapper = VariantWrapper; - - if (!rtInfo.count(FusedNamesWrapper::get_type_info_static())) - return {}; - - const auto& attr = rtInfo.at(FusedNamesWrapper::get_type_info_static()); - FusedNames fusedNames = ov::as_type_ptr(attr)->get(); - return fusedNames.getVectorNames(); + if (node) { + const auto& rtInfo = node->get_rt_info(); + auto it_info = rtInfo.find(FusedNames::get_type_info_static()); + if (it_info != rtInfo.end()) { + return it_info->second.as().getVectorNames(); + } else { + return {}; + } + } + return {}; } -template class ov::VariantImpl; - -Any VariantWrapper::merge(const ngraph::NodeVector& nodes) { +Any FusedNames::merge(const ngraph::NodeVector& nodes) const { FusedNames mergedNames; for (auto& node : nodes) { const auto& rtInfo = node->get_rt_info(); - - if (!rtInfo.count(VariantWrapper::get_type_info_static())) - continue; - - const auto attr = rtInfo.at(VariantWrapper::get_type_info_static()); - if (auto fusedNames = std::dynamic_pointer_cast>(attr)) { - mergedNames.fuseWith(fusedNames->get()); + auto it_info = rtInfo.find(FusedNames::get_type_info_static()); + if (it_info != rtInfo.end()) { + mergedNames.fuseWith(it_info->second.as()); } } - return std::make_shared>(mergedNames); + return mergedNames; } -Any VariantWrapper::init(const std::shared_ptr& node) { - return std::make_shared>(FusedNames(node->get_friendly_name())); +Any FusedNames::init(const std::shared_ptr& node) const { + return FusedNames{node->get_friendly_name()}; } -bool VariantWrapper::visit_attributes(AttributeVisitor& visitor) { - visitor.on_attribute("value", m_value.fused_names); +bool FusedNames::visit_attributes(AttributeVisitor& visitor) { + visitor.on_attribute("value", fused_names); return true; } + +std::string FusedNames::to_string() const { + return getNames(); +} diff --git a/src/core/src/pass/rt_info/primitives_priority_attribute.cpp b/src/core/src/pass/rt_info/primitives_priority_attribute.cpp index 036580253cc..f1628b531f2 100644 --- a/src/core/src/pass/rt_info/primitives_priority_attribute.cpp +++ b/src/core/src/pass/rt_info/primitives_priority_attribute.cpp @@ -18,16 +18,19 @@ using namespace ov; using namespace ngraph; std::string ov::getPrimitivesPriority(const std::shared_ptr& node) { - const auto& rtInfo = node->get_rt_info(); - - if (!rtInfo.count(PrimitivesPriority::get_type_info_static())) - return ""; - - const auto& attr = rtInfo.at(PrimitivesPriority::get_type_info_static()); - return ov::as_type_ptr(attr)->get(); + if (node) { + const auto& rtInfo = node->get_rt_info(); + auto it_info = rtInfo.find(PrimitivesPriority::get_type_info_static()); + if (it_info != rtInfo.end()) { + return it_info->second.as().value; + } else { + return {}; + } + } + return {}; } -Any PrimitivesPriority::merge(const ngraph::NodeVector& nodes) { +Any PrimitivesPriority::merge(const ngraph::NodeVector& nodes) const { auto isConvolutionBased = [](const std::shared_ptr& node) -> bool { if (std::dynamic_pointer_cast(node) || std::dynamic_pointer_cast(node) || @@ -49,21 +52,21 @@ Any PrimitivesPriority::merge(const ngraph::NodeVector& nodes) { } if (unique_pp.size() > 1) { - throw ngraph_error(std::string(get_type_info()) + " no rule defined for multiple values."); + throw ngraph_error("PrimitivesPriority no rule defined for multiple values."); } std::string final_primitives_priority; if (unique_pp.size() == 1) { final_primitives_priority = *unique_pp.begin(); } - return std::make_shared(final_primitives_priority); -} - -Any PrimitivesPriority::init(const std::shared_ptr& node) { - throw ngraph_error(std::string(get_type_info()) + " has no default initialization."); + return PrimitivesPriority(final_primitives_priority); } bool PrimitivesPriority::visit_attributes(AttributeVisitor& visitor) { - visitor.on_attribute("value", m_value); + visitor.on_attribute("value", value); return true; } + +std::string PrimitivesPriority::to_string() const { + return value; +} diff --git a/src/core/src/pass/serialize.cpp b/src/core/src/pass/serialize.cpp index b1dc8049364..e95b7000702 100644 --- a/src/core/src/pass/serialize.cpp +++ b/src/core/src/pass/serialize.cpp @@ -20,6 +20,7 @@ #include "openvino/pass/constant_folding.hpp" #include "pugixml.hpp" #include "transformations/hash.hpp" +#include "transformations/rt_info/primitives_priority_attribute.hpp" using namespace ngraph; @@ -139,20 +140,14 @@ public: for (const auto& rt_info_name : list_of_names) { const auto& found_rt_info = rt_info.find(rt_info_name); if (found_rt_info != rt_info.end()) { - xml_node_append_attribute(rt_info_name, found_rt_info->second); + std::stringstream strm; + found_rt_info->second.print(strm); + m_xml_node.append_attribute(rt_info_name.c_str()).set_value(strm.str().c_str()); } } } private: - template - void xml_node_append_attribute(const std::string& name, const std::shared_ptr& variant) { - if (auto v = std::dynamic_pointer_cast>(variant)) { - const auto& value = v->get(); - m_xml_node.append_attribute(name.c_str()).set_value(value.c_str()); - } - } - pugi::xml_node& m_xml_node; }; @@ -880,22 +875,23 @@ void ngfunction_2_ir(pugi::xml_node& netXml, // general attributes pugi::xml_node data = layer.append_child("data"); - auto append_runtime_info = [](pugi::xml_node& node, const RTMap& attributes) { + auto append_runtime_info = [](pugi::xml_node& node, RTMap& attributes) { pugi::xml_node rt_node = node.append_child("rt_info"); bool has_attrs = false; - for (const auto& item : attributes) { - auto attribute_node = rt_node.append_child("attribute"); - OPENVINO_SUPPRESS_DEPRECATED_START - attribute_node.append_attribute("name").set_value(item.second->get_type_info().name); - attribute_node.append_attribute("version").set_value( - item.second->get_type_info().get_version().c_str()); - rt_info::RTInfoSerializer serializer(attribute_node); - if (!item.second->visit_attributes(serializer)) { - rt_node.remove_child(attribute_node); - } else { - has_attrs = true; + for (auto& item : attributes) { + if (item.second.is()) { + auto attribute_node = rt_node.append_child("attribute"); + auto& rt_attribute = item.second.as(); + const auto& type_info = rt_attribute.get_type_info(); + attribute_node.append_attribute("name").set_value(type_info.name); + attribute_node.append_attribute("version").set_value(type_info.get_version().c_str()); + rt_info::RTInfoSerializer serializer(attribute_node); + if (!rt_attribute.visit_attributes(serializer)) { + rt_node.remove_child(attribute_node); + } else { + has_attrs = true; + } } - OPENVINO_SUPPRESS_DEPRECATED_END } if (!has_attrs) { node.remove_child(rt_node); @@ -910,7 +906,7 @@ void ngfunction_2_ir(pugi::xml_node& netXml, // if (node->get_input_size() > 0) { pugi::xml_node input = layer.append_child("input"); - for (const auto& i : node->inputs()) { + for (auto& i : node->inputs()) { // WA for LSTMCellv0, peephole input shall not be serialized if (i.get_index() == 6 && dynamic_cast(node)) { port_id++; @@ -940,7 +936,7 @@ void ngfunction_2_ir(pugi::xml_node& netXml, // if ((node->get_output_size() > 0) && !ngraph::op::is_output(node)) { pugi::xml_node output = layer.append_child("output"); - for (const auto& o : node->outputs()) { + for (auto& o : node->outputs()) { pugi::xml_node port = output.append_child("port"); port.append_attribute("id").set_value(port_id++); port.append_attribute("precision").set_value(get_precision_name(o.get_element_type()).c_str()); diff --git a/src/core/src/pass/visualize_tree.cpp b/src/core/src/pass/visualize_tree.cpp index 7f69d491055..2127d7250ec 100644 --- a/src/core/src/pass/visualize_tree.cpp +++ b/src/core/src/pass/visualize_tree.cpp @@ -161,11 +161,17 @@ static std::string get_attribute_values(const std::map& at bool first = true; for (const auto& item : attributes) { ss << (first ? " " : delimiter) << item.first; - OPENVINO_SUPPRESS_DEPRECATED_START - const auto attributeValue = item.second == nullptr ? "[EMPTY]" : item.second->to_string(); - OPENVINO_SUPPRESS_DEPRECATED_END - if (!attributeValue.empty()) - ss << "{" << attributeValue << "}"; + if (item.second.is()) { + ss << "{" << item.second.as().to_string() << "}"; + } else if (!item.second.empty()) { + ss << "{"; + item.second.print(ss); + ss << "}"; + } else { + ss << "{" + << "[EMPTY]" + << "}"; + } first = false; } diff --git a/src/core/src/preprocess/pre_post_process.cpp b/src/core/src/preprocess/pre_post_process.cpp index c27f0e7c813..07bb0cec76c 100644 --- a/src/core/src/preprocess/pre_post_process.cpp +++ b/src/core/src/preprocess/pre_post_process.cpp @@ -442,7 +442,7 @@ std::shared_ptr PrePostProcessor::build() { plane_param->output(0).get_rt_info().erase(TensorInfoMemoryType::get_type_info_static()); } else { plane_param->output(0).get_rt_info()[TensorInfoMemoryType::get_type_info_static()] = - std::make_shared(input->get_tensor_data()->get_memory_type()); + TensorInfoMemoryType(input->get_tensor_data()->get_memory_type()); } } new_params.push_back(plane_param); @@ -578,10 +578,17 @@ std::shared_ptr PrePostProcessor::build() { // Create result auto new_result = std::make_shared(node); new_result->set_friendly_name(result->get_friendly_name()); + node.get_tensor().set_names(start_out_node_names); + + // Preserve runtime info of original result + new_result->get_rt_info() = result->get_rt_info(); + new_result->input(0).get_rt_info() = result->input(0).get_rt_info(); + new_result->output(0).get_rt_info() = result->output(0).get_rt_info(); + + // Update layout if (!context.layout().empty()) { new_result->set_layout(context.layout()); } - node.get_tensor().set_names(start_out_node_names); for (auto& old_result : results) { if (result == old_result) { diff --git a/src/core/src/preprocess/preprocess_steps_impl.cpp b/src/core/src/preprocess/preprocess_steps_impl.cpp index f970c61f2a8..19e42bc45dd 100644 --- a/src/core/src/preprocess/preprocess_steps_impl.cpp +++ b/src/core/src/preprocess/preprocess_steps_impl.cpp @@ -43,9 +43,15 @@ void PreStepsList::add_scale_impl(const std::vector& values) { if (values.size() == 1) { shape = Shape{1}; } else { - shape = construct_mean_scale_shape(nodes[0].get_node_shared_ptr(), values.size(), context); + shape = construct_mean_scale_shape(nodes[0], values.size(), context); } - auto constant = op::v0::Constant::create(element::f32, shape, values); + auto element_type = nodes[0].get_element_type(); + OPENVINO_ASSERT(element_type.is_real(), + "Scale preprocessing can be applied to 'float' inputs. Consider using of " + "'convert_element_type' before scaling. Current type is: ", + element_type); + + auto constant = op::v0::Constant::create(element_type, shape, values); auto new_op = std::make_shared(nodes[0], constant); return std::make_tuple(std::vector>{new_op}, false); @@ -66,7 +72,13 @@ void PreStepsList::add_mean_impl(const std::vector& values) { } else { shape = construct_mean_scale_shape(nodes[0], values.size(), context); } - auto constant = op::v0::Constant::create(element::f32, shape, values); + auto element_type = nodes[0].get_element_type(); + OPENVINO_ASSERT(element_type.is_real(), + "Mean preprocessing can be applied to 'float' inputs. Consider using of 'convert_element_type' " + "before scaling. Current type is: ", + element_type); + + auto constant = op::v0::Constant::create(element_type, shape, values); auto new_op = std::make_shared(nodes[0], constant); return std::make_tuple(std::vector>{new_op}, false); diff --git a/src/core/src/rt_info.cpp b/src/core/src/rt_info.cpp index eaed0197af7..be62a3eeb7e 100644 --- a/src/core/src/rt_info.cpp +++ b/src/core/src/rt_info.cpp @@ -11,10 +11,13 @@ namespace { ngraph::Node::RTMap mergeRuntimeInfo(const ngraph::NodeVector& nodes) { std::unordered_map> attrs; - OPENVINO_SUPPRESS_DEPRECATED_START for (const auto& node : nodes) { for (const auto& item : node->get_rt_info()) { - if (item.second->is_copyable() && item.first != "opset") { + bool copy = item.first != "opset"; + if (item.second.is()) { + copy &= item.second.as().is_copyable(); + } + if (copy) { attrs[item.first].push_back(item.second); } } @@ -26,18 +29,19 @@ ngraph::Node::RTMap mergeRuntimeInfo(const ngraph::NodeVector& nodes) { if (item.second.size() == 1) { merged_attrs[item.first] = attr; } else { - auto merge_attr = attr->merge(nodes); - if (!merge_attr.empty()) { - merged_attrs[item.first] = merge_attr; + if (attr.is()) { + auto merge_attr = attr.as().merge(nodes); + if (!merge_attr.empty()) { + merged_attrs[item.first] = merge_attr; + } } } } - OPENVINO_SUPPRESS_DEPRECATED_END return merged_attrs; } -std::shared_ptr get_opset(const ngraph::Node::RTMap& rt_info) { +ov::Any get_opset(const ngraph::Node::RTMap& rt_info) { auto it = rt_info.find("opset"); if (it != rt_info.end()) { return it->second; @@ -48,7 +52,7 @@ std::shared_ptr get_opset(const ngraph::Node::RTMap& rt_info) { void assign_runtime_info(const ngraph::Node::RTMap& from, ngraph::Node::RTMap& to) { auto opset = get_opset(to); to = from; - if (opset) { + if (!opset.empty()) { to["opset"] = opset; } } @@ -61,14 +65,16 @@ void ngraph::copy_runtime_info(std::shared_ptr from, std::shared_p attrs.clear(); for (const auto& item : from->get_rt_info()) { - OPENVINO_SUPPRESS_DEPRECATED_START - if (item.second->is_copyable() && item.first != "opset") { + bool copy = item.first != "opset"; + if (item.second.is()) { + copy &= item.second.as().is_copyable(); + } + if (copy) { attrs[item.first] = item.second; } - OPENVINO_SUPPRESS_DEPRECATED_END } - if (opset) { + if (!opset.empty()) { attrs["opset"] = opset; } } diff --git a/src/core/src/runtime_attribute.cpp b/src/core/src/runtime_attribute.cpp new file mode 100644 index 00000000000..f639c0c6017 --- /dev/null +++ b/src/core/src/runtime_attribute.cpp @@ -0,0 +1,39 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "openvino/core/runtime_attribute.hpp" + +#include "ngraph/node.hpp" +#include "openvino/core/attribute_visitor.hpp" + +namespace ov { + +std::string RuntimeAttribute::to_string() const { + return {}; +} +bool RuntimeAttribute::visit_attributes(AttributeVisitor&) { + return false; +} + +Any RuntimeAttribute::init(const std::shared_ptr& node) const { + return {}; +} + +Any RuntimeAttribute::merge(const ngraph::NodeVector& nodes) const { + return {}; +} + +bool RuntimeAttribute::is_copyable() const { + return true; +} + +std::ostream& operator<<(std::ostream& os, const RuntimeAttribute& attrubute) { + return os << attrubute.to_string(); +} + +template class ov::RuntimeAttributeImpl; +template class ov::RuntimeAttributeImpl; +template class ov::RuntimeAttributeImpl; + +} // namespace ov \ No newline at end of file diff --git a/src/core/src/shared_node_info.hpp b/src/core/src/shared_node_info.hpp index c14857c0693..eef60f20940 100644 --- a/src/core/src/shared_node_info.hpp +++ b/src/core/src/shared_node_info.hpp @@ -7,7 +7,6 @@ #include #include #include -#include namespace ov { class SharedRTInfo { diff --git a/src/core/src/validation_util.cpp b/src/core/src/validation_util.cpp index 244be50b167..cb62ec2fc44 100644 --- a/src/core/src/validation_util.cpp +++ b/src/core/src/validation_util.cpp @@ -1212,8 +1212,15 @@ void propagate_rt_info(Node* node, const Output& final_port) { auto& rt_info = consumer->get_rt_info(); for (const auto& it : orig_rt_info) { - if (rt_info.find(it.first) == rt_info.end() && it.second->is_copyable()) - rt_info[it.first] = it.second; + if (rt_info.find(it.first) == rt_info.end()) { + bool copy = true; + if (it.second.is()) { + copy = it.second.as().is_copyable(); + } + if (copy) { + rt_info[it.first] = it.second; + } + } } } } diff --git a/src/core/src/variant.cpp b/src/core/src/variant.cpp deleted file mode 100644 index 4e3c73bfcf4..00000000000 --- a/src/core/src/variant.cpp +++ /dev/null @@ -1,29 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#include "ngraph/variant.hpp" - -#include "ngraph/node.hpp" -#include "openvino/core/any.hpp" -#include "openvino/core/attribute_visitor.hpp" - -using namespace ngraph; - -Variant::~Variant() = default; - -ov::Any Variant::init(const std::shared_ptr& node) { - return {}; -} - -ov::Any Variant::merge(const ngraph::NodeVector& nodes) { - return {}; -} - -bool Variant::is_copyable() const { - return true; -} - -template class ngraph::VariantImpl; -template class ngraph::VariantImpl; -template class ngraph::VariantImpl; diff --git a/src/core/tests/any.cpp b/src/core/tests/any.cpp index 47428262111..00747e0b868 100644 --- a/src/core/tests/any.cpp +++ b/src/core/tests/any.cpp @@ -7,12 +7,14 @@ #include #include "gtest/gtest.h" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" using namespace ov; class DestructorTest { public: + // Base member type defined. ov:Any can access all derived class + using Base = std::tuple; DestructorTest() { constructorCount++; } @@ -29,9 +31,22 @@ public: destructorCount++; } + virtual const std::type_info& type_info() const { + return typeid(DestructorTest); + } + static size_t destructorCount; static size_t constructorCount; }; + +class Derived : public DestructorTest { +public: + using DestructorTest::DestructorTest; + const std::type_info& type_info() const override { + return typeid(Derived); + } +}; + size_t DestructorTest::destructorCount = 0; size_t DestructorTest::constructorCount = 0; @@ -392,55 +407,87 @@ TEST_F(AnyTests, PrintToMapOfAnysDoesNothing) { ASSERT_EQ(stream.str(), std::string{}); } -TEST_F(AnyTests, constructFromVariantImpl) { +TEST_F(AnyTests, constructFromRuntimeAttributeImpl) { auto parameter = Any{4}; auto get_impl = [&] { - return std::make_shared>(); + return std::make_shared>(); }; auto other_parameter = Any{get_impl()}; } -TEST_F(AnyTests, dynamicPointerCastToVariant) { - Any p = std::make_shared>("42"); - auto str_variant = std::dynamic_pointer_cast>(p); +TEST_F(AnyTests, dynamicPointerCastToRuntimeAttribute) { + Any p = std::make_shared>("42"); + auto str_variant = std::dynamic_pointer_cast>(p); ASSERT_EQ("42", str_variant->get()); } -TEST_F(AnyTests, asTypePtrToVariant) { - Any p = std::make_shared>("42"); - auto str_variant = ov::as_type_ptr>(p); +TEST_F(AnyTests, asTypePtrToRuntimeAttribute) { + Any p = std::make_shared>("42"); + auto str_variant = ov::as_type_ptr>(p); ASSERT_EQ("42", str_variant->get()); } -TEST_F(AnyTests, castToVariant) { +TEST_F(AnyTests, castToRuntimeAttribute) { { - Any p = std::make_shared>("42"); - std::shared_ptr> str_variant = p; + Any p = std::make_shared>("42"); + std::shared_ptr> str_variant = p; ASSERT_EQ("42", str_variant->get()); } { - Any p = std::make_shared>("42"); - auto f = [](const std::shared_ptr>& str_variant) { - ASSERT_NE(nullptr, str_variant); + Any p = std::make_shared>("42"); + auto f = [](const std::shared_ptr>& str_variant) { ASSERT_EQ("42", str_variant->get()); }; f(p); } { - Any p = std::make_shared>("42"); - auto f = [](std::shared_ptr>& str_variant) { - ASSERT_NE(nullptr, str_variant); + Any p = std::make_shared>("42"); + auto f = [](std::shared_ptr>& str_variant) { ASSERT_EQ("42", str_variant->get()); }; f(p); } { - std::shared_ptr v = std::make_shared>("42"); + std::shared_ptr v = std::make_shared>("42"); Any p = v; - auto f = [](std::shared_ptr>& str_variant) { + auto f = [](std::shared_ptr>& str_variant) { ASSERT_NE(nullptr, str_variant); ASSERT_EQ("42", str_variant->get()); }; f(p); } } + +TEST_F(AnyTests, accessUsingBaseReference) { + ASSERT_EQ(0, DestructorTest::constructorCount); + ASSERT_EQ(0, DestructorTest::destructorCount); + { + using Base = DestructorTest; + auto p = Any::make(); + ASSERT_TRUE(p.is()); + ASSERT_TRUE(p.is()); + ASSERT_NO_THROW(p.as()); + ASSERT_NO_THROW(p.as()); + ASSERT_EQ(typeid(Derived), p.as().type_info()); + } + ASSERT_EQ(1, DestructorTest::constructorCount); + ASSERT_EQ(1, DestructorTest::destructorCount); +} + +struct WrongBase { + // No Base member type defined + // Should be: using Base = std::tuple; +}; + +struct WrongDerived : public WrongBase { + // No Base member type defined + // Should be: using Base = std::tuple; +}; + +TEST_F(AnyTests, accessUsingWrongBaseReference) { + Any p = WrongDerived{}; + ASSERT_TRUE(p.is()); + ASSERT_FALSE(p.is()); + ASSERT_NO_THROW(p.as()); + ASSERT_THROW(p.as(), ov::Exception); +} diff --git a/src/core/tests/constant_folding.cpp b/src/core/tests/constant_folding.cpp index ba6c894b130..3452e4b062b 100644 --- a/src/core/tests/constant_folding.cpp +++ b/src/core/tests/constant_folding.cpp @@ -3177,7 +3177,7 @@ TEST(constant_folding, disable_constant_folding) { auto constant_shape = op::Constant::create(element::i64, Shape{1}, {3}); auto dyn_reshape = make_shared(input, constant_shape, true); auto& rt_info = dyn_reshape->get_rt_info(); - rt_info["disabled_constant_folding_0"]; + rt_info[ov::pass::DisableConstantFolding::get_type_info_static()]; auto f = make_shared(dyn_reshape, ParameterVector{input}); pass::Manager pass_manager; diff --git a/src/core/tests/frontend/shared/include/op_fuzzy.hpp b/src/core/tests/frontend/shared/include/op_fuzzy.hpp index ca345622152..92b3e23add3 100644 --- a/src/core/tests/frontend/shared/include/op_fuzzy.hpp +++ b/src/core/tests/frontend/shared/include/op_fuzzy.hpp @@ -30,5 +30,5 @@ protected: void doLoadFromFile(); - void runConvertedModel(const std::shared_ptr function, const std::string& model_file); + void runConvertedModel(const std::shared_ptr function, const std::string& model_file); }; diff --git a/src/core/tests/frontend/shared/src/load_from.cpp b/src/core/tests/frontend/shared/src/load_from.cpp index bbd371f5592..6050809e7b4 100644 --- a/src/core/tests/frontend/shared/src/load_from.cpp +++ b/src/core/tests/frontend/shared/src/load_from.cpp @@ -6,7 +6,7 @@ #include -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "utils.hpp" using namespace ngraph; @@ -47,8 +47,8 @@ TEST_P(FrontEndLoadFromTest, testLoadFromFilePathWithExplicitVariants) { FrontEnd::Ptr fe; ASSERT_NO_THROW(frontends = m_fem.get_available_front_ends()); - std::vector> variants; - variants.emplace_back(ov::make_variant(model_path)); + std::vector variants; + variants.emplace_back(model_path); ASSERT_NO_THROW(m_frontEnd = m_fem.load_by_model(variants)); ASSERT_NE(m_frontEnd, nullptr); diff --git a/src/core/tests/op.cpp b/src/core/tests/op.cpp index 898e83315b9..c3294ee9244 100644 --- a/src/core/tests/op.cpp +++ b/src/core/tests/op.cpp @@ -62,60 +62,3 @@ TEST(op, opset_multi_thread) { doTest(ngraph::get_opset6); doTest(ngraph::get_opset7); } - -struct Ship { - std::string name; - int16_t x; - int16_t y; -}; - -namespace ov { -template <> -class VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("VariantWrapper"); - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -} // namespace ov - -TEST(op, variant) { - shared_ptr var_std_string = make_variant("My string"); - ASSERT_TRUE((ov::is_type>(var_std_string))); - EXPECT_EQ((ov::as_type_ptr>(var_std_string)->get()), "My string"); - - shared_ptr var_int64_t = make_variant(27); - ASSERT_TRUE((ov::is_type>(var_int64_t))); - EXPECT_FALSE((ov::is_type>(var_int64_t))); - EXPECT_EQ((ov::as_type_ptr>(var_int64_t)->get()), 27); - - shared_ptr var_ship = make_variant(Ship{"Lollipop", 3, 4}); - ASSERT_TRUE((ov::is_type>(var_ship))); - Ship& ship = ov::as_type_ptr>(var_ship)->get(); - EXPECT_EQ(ship.name, "Lollipop"); - EXPECT_EQ(ship.x, 3); - EXPECT_EQ(ship.y, 4); - - auto node = make_shared(element::f32, Shape{1}); - // Check Node RTInfo - node->get_rt_info()["A"] = var_ship; - auto node_var_ship = node->get_rt_info().at("A"); - ASSERT_TRUE((ov::is_type>(node_var_ship))); - Ship& node_ship = ov::as_type_ptr>(node_var_ship)->get(); - EXPECT_EQ(&node_ship, &ship); - - // Check Node Input RTInfo - auto relu = make_shared(node); - relu->input(0).get_rt_info()["A"] = var_ship; - auto node_input_var_ship = node->get_rt_info().at("A"); - ASSERT_TRUE((ov::is_type>(node_input_var_ship))); - Ship& node_input_ship = ov::as_type_ptr>(node_input_var_ship)->get(); - EXPECT_EQ(&node_input_ship, &ship); - - // Check Node Input RTInfo - node->output(0).get_rt_info()["A"] = var_ship; - auto node_output_var_ship = node->get_rt_info().at("A"); - ASSERT_TRUE((ov::is_type>(node_output_var_ship))); - Ship& node_output_ship = ov::as_type_ptr>(node_input_var_ship)->get(); - EXPECT_EQ(&node_output_ship, &ship); -} diff --git a/src/core/tests/op_eval/memory.cpp b/src/core/tests/op_eval/memory.cpp index 68ac1bfd115..e4b7f41fe41 100644 --- a/src/core/tests/op_eval/memory.cpp +++ b/src/core/tests/op_eval/memory.cpp @@ -85,7 +85,7 @@ TEST(op_eval, assign_readvalue_evaluation_context) { HostTensorPtr h_tensor = make_host_tensor(Shape{3}, inputs); VariableContext variable_context; variable_context.set_variable_value(variables[0], std::make_shared(h_tensor)); - eval_context["VariableContext"] = std::make_shared>(variable_context); + eval_context.emplace("VariableContext", variable_context); const int COUNT_RUNS = 10; for (int i = 0; i < COUNT_RUNS; ++i) { @@ -106,10 +106,10 @@ TEST(op_eval, assign_readvalue_add) { // creating context EvaluationContext eval_context; - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value = make_shared(make_host_tensor(Shape{3}, inputs)); - variable_context->get().set_variable_value(variables[0], variable_value); - eval_context["VariableContext"] = variable_context; + variable_context.set_variable_value(variables[0], variable_value); + eval_context.emplace("VariableContext", variable_context); auto result = make_shared(); const int COUNT_RUNS = 10; @@ -134,11 +134,11 @@ TEST(op_eval, assign_readvalue_reset_before_evaluate) { // creating context EvaluationContext eval_context; - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value = make_shared(make_host_tensor(Shape{3}, inputs)); variable_value->set_reset(false); - variable_context->get().set_variable_value(variables[0], variable_value); - eval_context["VariableContext"] = variable_context; + variable_context.set_variable_value(variables[0], variable_value); + eval_context.emplace("VariableContext", variable_context); auto result = make_shared(); const int COUNT_RUNS = 10; @@ -162,10 +162,10 @@ TEST(op_eval, assign_readvalue_add_reset) { // creating a Context EvaluationContext eval_context; - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value = make_shared(make_host_tensor(Shape{3}, inputs)); - variable_context->get().set_variable_value(variables[0], variable_value); - eval_context["VariableContext"] = variable_context; + variable_context.set_variable_value(variables[0], variable_value); + eval_context.emplace("VariableContext", variable_context); auto result = make_shared(); const int COUNT_RUNS = 10; @@ -182,9 +182,7 @@ TEST(op_eval, assign_readvalue_add_reset) { const auto& found_context = eval_context.find("VariableContext"); EXPECT_NE(found_context, eval_context.end()); - auto var_context = std::dynamic_pointer_cast>(found_context->second); - EXPECT_NE(var_context, nullptr); - variable_context->get().reset_variable_context(); + found_context->second.as().reset_variable_context(); for (int i = 0; i < COUNT_RUNS; ++i) { ASSERT_TRUE(fun->evaluate({result}, {make_host_tensor(Shape{3}, inputs)}, eval_context)); @@ -206,10 +204,10 @@ TEST(op_eval, assign_readvalue_add_modify) { // creating context EvaluationContext eval_context; - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value = make_shared(make_host_tensor(Shape{3}, inputs)); - variable_context->get().set_variable_value(variables[0], variable_value); - eval_context["VariableContext"] = variable_context; + variable_context.set_variable_value(variables[0], variable_value); + eval_context.emplace("VariableContext", variable_context); auto result = make_shared(); const int COUNT_RUNS = 10; @@ -225,9 +223,7 @@ TEST(op_eval, assign_readvalue_add_modify) { const auto& found_context = eval_context.find("VariableContext"); EXPECT_NE(found_context, eval_context.end()); - auto var_context = std::dynamic_pointer_cast>(found_context->second); - EXPECT_NE(var_context, nullptr); - const auto& var_value = variable_context->get().get_variable_value(variables[0]); + const auto& var_value = found_context->second.as().get_variable_value(variables[0]); EXPECT_NE(var_value, nullptr); var_value->set_value(make_host_tensor(Shape{3}, {1, 2, 3})); @@ -253,14 +249,14 @@ TEST(op_eval, assign_readvalue_add_modify_multi_variables) { // creating context EvaluationContext eval_context; - auto variable_context = std::make_shared>(VariableContext()); + auto variable_context = VariableContext(); auto variable_value_1 = make_shared(make_host_tensor(Shape{3}, inputs_1)); auto variable_value_2 = make_shared(make_host_tensor(Shape{3}, inputs_2)); variable_value_1->set_reset(false); variable_value_2->set_reset(false); - variable_context->get().set_variable_value(var_1, variable_value_1); - variable_context->get().set_variable_value(var_2, variable_value_2); - eval_context["VariableContext"] = variable_context; + variable_context.set_variable_value(var_1, variable_value_1); + variable_context.set_variable_value(var_2, variable_value_2); + eval_context.emplace("VariableContext", variable_context); auto result = make_shared(); const int COUNT_RUNS = 10; @@ -279,14 +275,13 @@ TEST(op_eval, assign_readvalue_add_modify_multi_variables) { const auto& found_context = eval_context.find("VariableContext"); EXPECT_NE(found_context, eval_context.end()); - auto var_context = std::dynamic_pointer_cast>(found_context->second); - EXPECT_NE(var_context, nullptr); + auto var_context = found_context->second.as(); - auto var_value = variable_context->get().get_variable_value(var_1); + auto var_value = variable_context.get_variable_value(var_1); EXPECT_NE(var_value, nullptr); var_value->set_value(make_host_tensor(Shape{3}, {1, 2, 3})); - auto var_value_2 = variable_context->get().get_variable_value(var_2); + auto var_value_2 = variable_context.get_variable_value(var_2); EXPECT_NE(var_value_2, nullptr); var_value_2->set_reset(true); diff --git a/src/core/tests/pass/serialization/read_ir.hpp b/src/core/tests/pass/serialization/read_ir.hpp index 6e502845d39..e28d39e1e5a 100644 --- a/src/core/tests/pass/serialization/read_ir.hpp +++ b/src/core/tests/pass/serialization/read_ir.hpp @@ -16,9 +16,9 @@ inline std::shared_ptr readModel(const std::string& model_path, co ov::frontend::FrontEnd::Ptr FE; ov::frontend::InputModel::Ptr inputModel; - ov::VariantVector params{ov::make_variant(model_path)}; + ov::RuntimeAttributeVector params{model_path}; if (!weights_path.empty()) - params.emplace_back(ov::make_variant(weights_path)); + params.emplace_back(weights_path); FE = manager.load_by_model(params); if (FE) @@ -37,7 +37,7 @@ inline std::shared_ptr readModel(const std::string& model) { std::istringstream modelStringStream(model); std::istream& modelStream = modelStringStream; - ov::VariantVector params{ov::make_variant(&modelStream)}; + ov::RuntimeAttributeVector params{&modelStream}; FE = manager.load_by_model(params); if (FE) diff --git a/src/core/tests/preprocess.cpp b/src/core/tests/preprocess.cpp index 15222dbd013..9c472af835f 100644 --- a/src/core/tests/preprocess.cpp +++ b/src/core/tests/preprocess.cpp @@ -52,12 +52,20 @@ TEST(pre_post_process, simple_mean_scale) { EXPECT_EQ(f->get_output_element_type(0), element::f32); } -TEST(pre_post_process, simple_mean_scale_getters) { - auto f = create_simple_function(element::f32, Shape{1, 3, 2, 2}); +TEST(pre_post_process, simple_mean_scale_getters_f16) { + auto f = create_simple_function(element::f16, Shape{1, 3, 2, 2}); auto p = PrePostProcessor(f); p.input("tensor_input1").preprocess().mean(1).scale(2); f = p.build(); - EXPECT_EQ(f->get_output_element_type(0), element::f32); + EXPECT_EQ(f->get_output_element_type(0), element::f16); +} + +TEST(pre_post_process, simple_mean_scale_getters_f64) { + auto f = create_simple_function(element::f64, Shape{1, 3, 2, 2}); + auto p = PrePostProcessor(f); + p.input("tensor_input1").preprocess().mean(1).scale(2); + f = p.build(); + EXPECT_EQ(f->get_output_element_type(0), element::f64); } TEST(pre_post_process, convert_element_type_and_scale) { @@ -889,20 +897,20 @@ TEST(pre_post_process, preprocess_reverse_channels_no_c_dim) { TEST(pre_post_process, preprocess_preserve_rt_info) { auto f = create_simple_function(element::f32, Shape{1, 3, 2, 2}); - f->get_parameters()[0]->get_rt_info()["someKey"] = ov::make_variant("someValue"); - f->input().get_rt_info()["someKey_in"] = ov::make_variant("someValue_in"); + f->get_parameters()[0]->get_rt_info()["someKey"] = "someValue"; + f->input().get_rt_info()["someKey_in"] = "someValue_in"; auto p = PrePostProcessor(f); p.input().tensor().set_element_type(element::u8); f = p.build(); EXPECT_EQ(f->input().get_element_type(), element::u8); ASSERT_EQ(f->get_parameters()[0]->get_rt_info().count("someKey"), 1); - auto var0 = std::dynamic_pointer_cast>(f->get_parameters()[0]->get_rt_info()["someKey"]); - EXPECT_EQ(var0->get(), "someValue"); + auto var0 = f->get_parameters()[0]->get_rt_info()["someKey"].as(); + EXPECT_EQ(var0, "someValue"); ASSERT_EQ(f->input().get_rt_info().count("someKey_in"), 1); - auto var0_in = std::dynamic_pointer_cast>(f->input().get_rt_info()["someKey_in"]); - EXPECT_EQ(var0_in->get(), "someValue_in"); + auto var0_in = f->input().get_rt_info()["someKey_in"].as(); + EXPECT_EQ(var0_in, "someValue_in"); } TEST(pre_post_process, preprocess_memory_type) { @@ -911,14 +919,13 @@ TEST(pre_post_process, preprocess_memory_type) { p.input().tensor().set_memory_type("abc"); f = p.build(); ASSERT_EQ(f->input().get_rt_info().count(TensorInfoMemoryType::get_type_info_static()), 1); - auto var0 = std::dynamic_pointer_cast( - f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()]); - EXPECT_EQ(var0->get(), "abc"); + auto var0 = f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()].as().value; + EXPECT_EQ(var0, "abc"); } TEST(pre_post_process, preprocess_memory_type_clear) { auto f = create_simple_function(element::f32, Shape{1, 3, 2, 2}); - f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()] = ov::make_variant("abc"); + f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()] = TensorInfoMemoryType("abc"); auto p = PrePostProcessor(f); p.input().tensor().set_memory_type(""); f = p.build(); @@ -932,9 +939,8 @@ TEST(pre_post_process, preprocess_memory_type_not_cleared) { f = p.build(); ASSERT_EQ(f->input().get_rt_info().count(TensorInfoMemoryType::get_type_info_static()), 1); - auto var0 = std::dynamic_pointer_cast( - f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()]); - EXPECT_EQ(var0->get(), "abc"); + auto var0 = f->input().get_rt_info()[TensorInfoMemoryType::get_type_info_static()].as().value; + EXPECT_EQ(var0, "abc"); } // --- PostProcess - set/convert element type --- @@ -1170,6 +1176,29 @@ TEST(pre_post_process, postprocess_convert_layout_invalid_dims_dyn_shape) { // Postprocessing - other +TEST(pre_post_process, postprocess_preserve_rt_info) { + auto f = create_simple_function(element::f32, Shape{1, 3, 2, 2}); + f->get_results()[0]->get_rt_info()["someKey"] = "someValue"; + f->input().get_rt_info()["someKey_in"] = "someValue_in"; + f->output().get_rt_info()["someKey_out"] = "someValue_out"; + auto p = PrePostProcessor(f); + p.output().tensor().set_element_type(element::u8); + f = p.build(); + EXPECT_EQ(f->output().get_element_type(), element::u8); + + ASSERT_EQ(f->get_results()[0]->get_rt_info().count("someKey"), 1); + auto var0 = f->get_results()[0]->get_rt_info()["someKey"].as(); + EXPECT_EQ(var0, "someValue"); + + ASSERT_EQ(f->input().get_rt_info().count("someKey_in"), 1); + auto var0_in = f->input().get_rt_info()["someKey_in"].as(); + EXPECT_EQ(var0_in, "someValue_in"); + + ASSERT_EQ(f->output().get_rt_info().count("someKey_out"), 1); + auto var0_out = f->output().get_rt_info()["someKey_out"].as(); + EXPECT_EQ(var0_out, "someValue_out"); +} + TEST(pre_post_process, postprocess_custom_step) { auto f = create_simple_function(element::f32, Shape{1, 3, 2, 2}); bool hit = false; diff --git a/src/core/tests/runtime/interpreter/evaluates_map.cpp b/src/core/tests/runtime/interpreter/evaluates_map.cpp index b83769c5757..5fd24338f67 100644 --- a/src/core/tests/runtime/interpreter/evaluates_map.cpp +++ b/src/core/tests/runtime/interpreter/evaluates_map.cpp @@ -1438,6 +1438,37 @@ bool evaluate(const shared_ptr& op, using T = typename element_type_traits::value_type; runtime::reference::referenceDetectionOutput refDetOut(op->get_attrs(), op->get_input_shape(0), + op->get_input_shape(1), + op->get_input_shape(2), + op->get_output_shape(0)); + if (op->get_input_size() == 3) { + refDetOut.run(inputs[0]->get_data_ptr(), + inputs[1]->get_data_ptr(), + inputs[2]->get_data_ptr(), + nullptr, + nullptr, + outputs[0]->get_data_ptr()); + } else if (op->get_input_size() == 5) { + refDetOut.run(inputs[0]->get_data_ptr(), + inputs[1]->get_data_ptr(), + inputs[2]->get_data_ptr(), + inputs[3]->get_data_ptr(), + inputs[4]->get_data_ptr(), + outputs[0]->get_data_ptr()); + } else { + throw ngraph_error("DetectionOutput layer supports only 3 or 5 inputs"); + } + return true; +} + +template +bool evaluate(const shared_ptr& op, + const HostTensorVector& outputs, + const HostTensorVector& inputs) { + using T = typename element_type_traits::value_type; + runtime::reference::referenceDetectionOutput refDetOut(op->get_attrs(), + op->get_input_shape(0), + op->get_input_shape(1), op->get_input_shape(2), op->get_output_shape(0)); if (op->get_input_size() == 3) { diff --git a/src/core/tests/runtime/interpreter/opset_int_tbl.hpp b/src/core/tests/runtime/interpreter/opset_int_tbl.hpp index e100660e9a9..fe1230d79d4 100644 --- a/src/core/tests/runtime/interpreter/opset_int_tbl.hpp +++ b/src/core/tests/runtime/interpreter/opset_int_tbl.hpp @@ -106,6 +106,7 @@ NGRAPH_OP(MulticlassNms, op::v8) NGRAPH_OP(DeformableConvolution, ngraph::op::v8) NGRAPH_OP(If, ngraph::op::v8) NGRAPH_OP(GatherND, op::v8) +NGRAPH_OP(DetectionOutput, op::v8) NGRAPH_OP(Sigmoid, op::v0) NGRAPH_OP(Tanh, op::v0) diff --git a/src/core/tests/type_prop/detection_output.cpp b/src/core/tests/type_prop/detection_output.cpp index c4cbad1d266..ed7b83ae859 100644 --- a/src/core/tests/type_prop/detection_output.cpp +++ b/src/core/tests/type_prop/detection_output.cpp @@ -13,6 +13,7 @@ using namespace std; using namespace ngraph; +// ------------------------------ V0 ------------------------------ std::shared_ptr create_detection_output(const PartialShape& box_logits_shape, const PartialShape& class_preds_shape, const PartialShape& proposals_shape, @@ -656,3 +657,225 @@ TEST(type_prop_layers, detection_output_invalid_aux_box_preds) { } } } + +// ------------------------------ V8 ------------------------------ +namespace { +std::shared_ptr create_detection_output_v8(const PartialShape& box_logits_shape, + const PartialShape& class_preds_shape, + const PartialShape& proposals_shape, + const op::v8::DetectionOutput::Attributes& attrs, + element::Type input_type) { + auto box_logits = make_shared(input_type, box_logits_shape); + auto class_preds = make_shared(input_type, class_preds_shape); + auto proposals = make_shared(input_type, proposals_shape); + return make_shared(box_logits, class_preds, proposals, attrs); +} + +std::shared_ptr create_detection_output2_v8(const PartialShape& box_logits_shape, + const PartialShape& class_preds_shape, + const PartialShape& proposals_shape, + const PartialShape& aux_class_preds_shape, + const PartialShape& aux_box_preds_shape, + const op::v8::DetectionOutput::Attributes& attrs, + element::Type input_type) { + auto box_logits = make_shared(input_type, box_logits_shape); + auto class_preds = make_shared(input_type, class_preds_shape); + auto proposals = make_shared(input_type, proposals_shape); + auto aux_class_preds = make_shared(input_type, aux_class_preds_shape); + auto aux_box_preds = make_shared(input_type, aux_box_preds_shape); + return make_shared(box_logits, + class_preds, + proposals, + aux_class_preds, + aux_box_preds, + attrs); +} + +PartialShape compute_reference_output_shape(const std::vector& keep_top_k, + int top_k, + const Dimension& deduced_N, + const Dimension& deduced_num_classes, + const Dimension& deduced_num_prior_boxes) { + if (keep_top_k.size() > 0 && keep_top_k[0] > 0) { + return PartialShape({1, 1, deduced_N * keep_top_k[0], 7}); + } else if (top_k > 0) { + return PartialShape({1, 1, deduced_N * top_k * deduced_num_classes, 7}); + } else { + return PartialShape({1, 1, deduced_N * deduced_num_classes * deduced_num_prior_boxes, 7}); + } +} + +std::vector create_attributes_vector() { + // initialize attributes affecting shape inference + // others remain by default + std::vector result; + for (int keep_top_k : {10, -1}) { + for (int top_k : {5, -1}) { + for (bool variance_encoded_in_target : {true, false}) { + for (bool share_location : {true, false}) { + for (bool normalized : {true, false}) { + op::v8::DetectionOutput::Attributes attrs; + attrs.top_k = top_k; + attrs.keep_top_k = {keep_top_k}; + attrs.variance_encoded_in_target = variance_encoded_in_target; + attrs.share_location = share_location; + attrs.normalized = normalized; + result.push_back(attrs); + } + } + } + } + } + return result; +} +} // namespace + +TEST(type_prop_layers, detection_outputv8_all_static) { + // this case covers deducing a number of classes value + // since this value is not saved in attributes + op::v8::DetectionOutput::Attributes attrs; + + // initialize attributes affecting shape inference + // others remain by default + Dimension N = 5; + Dimension num_prior_boxes = 100; + Dimension priors_batch_size = N; + Dimension num_classes = 23; + + auto attrs_vector = create_attributes_vector(); + for (const auto& attrs : attrs_vector) { + Dimension num_loc_classes = attrs.share_location ? 1 : num_classes; + Dimension prior_box_size = attrs.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, num_prior_boxes * num_loc_classes * 4}; + PartialShape class_preds_shape = {N, num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attrs.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + PartialShape output_shape_reference = + compute_reference_output_shape(attrs.keep_top_k, attrs.top_k, N, num_classes, num_prior_boxes); + + auto op = create_detection_output_v8(box_logits_shape, class_preds_shape, proposals_shape, attrs, element::f32); + ASSERT_EQ(op->get_output_partial_shape(0), output_shape_reference); + ASSERT_EQ(op->get_element_type(), element::f32); + } +} + +TEST(type_prop_layers, detection_outputv8_dynamic) { + op::v8::DetectionOutput::Attributes attrs; + + // initialize attributes affecting shape inference + // others remain by default + Dimension N = 13; + Dimension num_prior_boxes = 33; + Dimension priors_batch_size = 1; + Dimension num_classes = 10; + + auto attrs_vector = create_attributes_vector(); + for (const auto& attrs : attrs_vector) { + Dimension prior_box_size = attrs.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, Dimension::dynamic()}; + PartialShape class_preds_shape = {Dimension::dynamic(), num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attrs.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + PartialShape output_shape_reference = + compute_reference_output_shape(attrs.keep_top_k, attrs.top_k, N, num_classes, num_prior_boxes); + + auto op = create_detection_output_v8(box_logits_shape, class_preds_shape, proposals_shape, attrs, element::f32); + ASSERT_EQ(op->get_output_partial_shape(0), output_shape_reference); + ASSERT_EQ(op->get_element_type(), element::f32); + } +} + +TEST(type_prop_layers, detection_outputv8_num_classes_not_deduced) { + op::v8::DetectionOutput::Attributes attrs; + + // initialize attributes affecting shape inference + // others remain by default + Dimension N = 13; + Dimension num_prior_boxes = 33; + Dimension priors_batch_size = 1; + + auto attrs_vector = create_attributes_vector(); + for (const auto& attrs : attrs_vector) { + Dimension prior_box_size = attrs.normalized ? 4 : 5; + + PartialShape box_logits_shape = {N, Dimension::dynamic()}; + PartialShape class_preds_shape = {N, Dimension::dynamic()}; + PartialShape proposals_shape = {priors_batch_size, + attrs.variance_encoded_in_target ? 1 : 2, + num_prior_boxes * prior_box_size}; + PartialShape output_shape_reference = + compute_reference_output_shape(attrs.keep_top_k, attrs.top_k, N, Dimension::dynamic(), num_prior_boxes); + + auto op = create_detection_output_v8(box_logits_shape, class_preds_shape, proposals_shape, attrs, element::f32); + ASSERT_EQ(op->get_output_partial_shape(0), output_shape_reference); + ASSERT_EQ(op->get_element_type(), element::f32); + } +} + +TEST(type_prop_layers, detection_outputv8_num_classes_no_deduction) { + // In this case a number of classes and a number of prior boxes are not deduced + op::v8::DetectionOutput::Attributes attrs; + + // initialize attributes affecting shape inference + // others remain by default + Dimension N = 3; + Dimension priors_batch_size = N; + + auto attrs_vector = create_attributes_vector(); + for (const auto& attrs : attrs_vector) { + PartialShape box_logits_shape = {N, Dimension::dynamic()}; + PartialShape class_preds_shape = {N, Dimension::dynamic()}; + PartialShape proposals_shape = {priors_batch_size, + attrs.variance_encoded_in_target ? 1 : 2, + Dimension::dynamic()}; + PartialShape output_shape_reference = compute_reference_output_shape(attrs.keep_top_k, + attrs.top_k, + N, + Dimension::dynamic(), + Dimension::dynamic()); + + auto op = create_detection_output_v8(box_logits_shape, class_preds_shape, proposals_shape, attrs, element::f32); + ASSERT_EQ(op->get_output_partial_shape(0), output_shape_reference); + ASSERT_EQ(op->get_element_type(), element::f32); + } +} + +TEST(type_prop_layers, detection_outputv8_dynamic2) { + // In this case a number of prior boxes is deduced using additional input + // and after that a number of classes is deduced using the second input shape + op::v8::DetectionOutput::Attributes attrs; + + // initialize attributes affecting shape inference + // others remain by default + Dimension N = 13; + Dimension num_prior_boxes = 33; + Dimension priors_batch_size = 1; + Dimension num_classes = 10; + + auto attrs_vector = create_attributes_vector(); + for (const auto& attrs : attrs_vector) { + PartialShape box_logits_shape = {N, Dimension::dynamic()}; + PartialShape class_preds_shape = {Dimension::dynamic(), num_prior_boxes * num_classes}; + PartialShape proposals_shape = {priors_batch_size, + attrs.variance_encoded_in_target ? 1 : 2, + Dimension::dynamic()}; + PartialShape ad_class_preds_shape = {N, num_prior_boxes * 2}; + PartialShape ad_box_preds_shape = {N, Dimension::dynamic()}; + PartialShape output_shape_reference = + compute_reference_output_shape(attrs.keep_top_k, attrs.top_k, N, num_classes, num_prior_boxes); + + auto op = create_detection_output2_v8(box_logits_shape, + class_preds_shape, + proposals_shape, + ad_class_preds_shape, + ad_box_preds_shape, + attrs, + element::f32); + ASSERT_EQ(op->get_output_partial_shape(0), output_shape_reference); + ASSERT_EQ(op->get_element_type(), element::f32); + } +} diff --git a/src/core/tests/type_prop/parameter.cpp b/src/core/tests/type_prop/parameter.cpp index cc9e11a6828..27ef12b1037 100644 --- a/src/core/tests/type_prop/parameter.cpp +++ b/src/core/tests/type_prop/parameter.cpp @@ -47,7 +47,6 @@ TEST(type_prop, param_layout_empty) { TEST(type_prop, param_layout_invalid) { auto a = make_shared(element::f32, PartialShape::dynamic()); - a->get_output_tensor(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = - ov::make_variant("NCHW"); // incorrect way - ASSERT_THROW(a->get_layout(), ov::AssertFailure); + a->get_output_tensor(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = "NCHW"; // incorrect way + ASSERT_THROW(a->get_layout(), ov::Exception); } diff --git a/src/core/tests/type_prop/result.cpp b/src/core/tests/type_prop/result.cpp index 85c343d3592..6966c9f1997 100644 --- a/src/core/tests/type_prop/result.cpp +++ b/src/core/tests/type_prop/result.cpp @@ -48,7 +48,6 @@ TEST(type_prop, result_layout_empty) { TEST(type_prop, result_layout_invalid) { auto a = make_shared(element::f32, PartialShape::dynamic()); auto result = make_shared(a); - result->input(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = - ov::make_variant("NCHW"); // incorrect way - ASSERT_THROW(result->get_layout(), ov::AssertFailure); + result->input(0).get_rt_info()[ov::LayoutAttribute::get_type_info_static()] = "NCHW"; // incorrect way + ASSERT_THROW(result->get_layout(), ov::Exception); } diff --git a/src/core/tests/util/graph_comparator.cpp b/src/core/tests/util/graph_comparator.cpp index c529f2d1d84..8b99adb226a 100644 --- a/src/core/tests/util/graph_comparator.cpp +++ b/src/core/tests/util/graph_comparator.cpp @@ -43,7 +43,6 @@ bool compare_rt_keys(const Node& node1, const Node& node2) { // The "opset" parameter in RT info is optional // and mandatory only for TypeRelaxed operations. // Therefore, we ignore this key when comparing RT keys. - const auto& first_node_rt_info = node1->get_rt_info(); const auto& second_node_rt_info = node2->get_rt_info(); @@ -51,6 +50,10 @@ bool compare_rt_keys(const Node& node1, const Node& node2) { auto second_node_rt_info_it = second_node_rt_info.begin(); while (first_node_rt_info_it != first_node_rt_info.end() && second_node_rt_info_it != second_node_rt_info.end()) { + std::stringstream strm; + first_node_rt_info_it->second.print(strm); + strm << " "; + second_node_rt_info_it->second.print(strm); bool is_continue = false; if (first_node_rt_info_it->first == "opset") { ++first_node_rt_info_it; diff --git a/src/core/tests/visitors/op/detection_output.cpp b/src/core/tests/visitors/op/detection_output.cpp index 488bbf423d3..91c95c2ce21 100644 --- a/src/core/tests/visitors/op/detection_output.cpp +++ b/src/core/tests/visitors/op/detection_output.cpp @@ -3,29 +3,19 @@ // #include "gtest/gtest.h" -#include "ngraph/ngraph.hpp" -#include "ngraph/op/util/attr_types.hpp" -#include "ngraph/opsets/opset1.hpp" -#include "ngraph/opsets/opset3.hpp" -#include "ngraph/opsets/opset4.hpp" -#include "ngraph/opsets/opset5.hpp" +#include "openvino/op/util/attr_types.hpp" +#include "openvino/opsets/opset8.hpp" #include "util/visitor.hpp" using namespace std; using namespace ngraph; +using namespace ov::op; +using namespace ov::op::util; using ngraph::test::NodeBuilder; using ngraph::test::ValueMap; -TEST(attributes, detection_output_op) { - NodeBuilder::get_ops().register_factory(); - const auto box_logits = make_shared(element::f32, Shape{1, 2 * 1 * 4}); - const auto class_preds = make_shared(element::f32, Shape{1, 2 * 32}); - const auto proposals = make_shared(element::f32, Shape{1, 2, 2 * 4}); - const auto aux_class_preds = make_shared(element::f32, Shape{1, 2 * 2}); - const auto aux_box_pred = make_shared(element::f32, Shape{1, 2 * 1 * 4}); - - op::DetectionOutputAttrs attrs; - attrs.num_classes = 32; +namespace { +void initialize_attributes(DetectionOutputBase::AttributesBase& attrs) { attrs.background_label_id = 0; attrs.top_k = 1; attrs.variance_encoded_in_target = false; @@ -41,29 +31,70 @@ TEST(attributes, detection_output_op) { attrs.input_height = 32; attrs.input_width = 32; attrs.objectness_score = 0.73f; +} +void is_equal_attrs(const DetectionOutputBase::AttributesBase& attrs1, + const DetectionOutputBase::AttributesBase& attrs2) { + EXPECT_EQ(attrs1.background_label_id, attrs2.background_label_id); + EXPECT_EQ(attrs1.top_k, attrs2.top_k); + EXPECT_EQ(attrs1.variance_encoded_in_target, attrs2.variance_encoded_in_target); + EXPECT_EQ(attrs1.keep_top_k, attrs2.keep_top_k); + EXPECT_EQ(attrs1.code_type, attrs2.code_type); + EXPECT_EQ(attrs1.share_location, attrs2.share_location); + EXPECT_EQ(attrs1.nms_threshold, attrs2.nms_threshold); + EXPECT_EQ(attrs1.confidence_threshold, attrs2.confidence_threshold); + EXPECT_EQ(attrs1.clip_after_nms, attrs2.clip_after_nms); + EXPECT_EQ(attrs1.clip_before_nms, attrs2.clip_before_nms); + EXPECT_EQ(attrs1.decrease_label_id, attrs2.decrease_label_id); + EXPECT_EQ(attrs1.normalized, attrs2.normalized); + EXPECT_EQ(attrs1.input_height, attrs2.input_height); + EXPECT_EQ(attrs1.input_width, attrs2.input_width); + EXPECT_EQ(attrs1.objectness_score, attrs2.objectness_score); +} +} // namespace + +TEST(attributes, detection_output_op) { + NodeBuilder::get_ops().register_factory(); + const auto box_logits = make_shared(element::f32, Shape{1, 2 * 1 * 4}); + const auto class_preds = make_shared(element::f32, Shape{1, 2 * 32}); + const auto proposals = make_shared(element::f32, Shape{1, 2, 2 * 4}); + const auto aux_class_preds = make_shared(element::f32, Shape{1, 2 * 2}); + const auto aux_box_pred = make_shared(element::f32, Shape{1, 2 * 1 * 4}); + + op::v0::DetectionOutput::Attributes attrs; + initialize_attributes(attrs); + attrs.num_classes = 32; auto detection_output = - make_shared(box_logits, class_preds, proposals, aux_class_preds, aux_box_pred, attrs); + make_shared(box_logits, class_preds, proposals, aux_class_preds, aux_box_pred, attrs); NodeBuilder builder(detection_output); - auto g_detection_output = ov::as_type_ptr(builder.create()); + auto g_detection_output = ov::as_type_ptr(builder.create()); const auto do_attrs = detection_output->get_attrs(); const auto g_do_attrs = g_detection_output->get_attrs(); EXPECT_EQ(g_do_attrs.num_classes, do_attrs.num_classes); - EXPECT_EQ(g_do_attrs.background_label_id, do_attrs.background_label_id); - EXPECT_EQ(g_do_attrs.top_k, do_attrs.top_k); - EXPECT_EQ(g_do_attrs.variance_encoded_in_target, do_attrs.variance_encoded_in_target); - EXPECT_EQ(g_do_attrs.keep_top_k, do_attrs.keep_top_k); - EXPECT_EQ(g_do_attrs.code_type, do_attrs.code_type); - EXPECT_EQ(g_do_attrs.share_location, do_attrs.share_location); - EXPECT_EQ(g_do_attrs.nms_threshold, do_attrs.nms_threshold); - EXPECT_EQ(g_do_attrs.confidence_threshold, do_attrs.confidence_threshold); - EXPECT_EQ(g_do_attrs.clip_after_nms, do_attrs.clip_after_nms); - EXPECT_EQ(g_do_attrs.clip_before_nms, do_attrs.clip_before_nms); - EXPECT_EQ(g_do_attrs.decrease_label_id, do_attrs.decrease_label_id); - EXPECT_EQ(g_do_attrs.normalized, do_attrs.normalized); - EXPECT_EQ(g_do_attrs.input_height, do_attrs.input_height); - EXPECT_EQ(g_do_attrs.input_width, do_attrs.input_width); - EXPECT_EQ(g_do_attrs.objectness_score, do_attrs.objectness_score); + is_equal_attrs(g_do_attrs, do_attrs); +} + +// ------------------------------ V8 ------------------------------ +TEST(attributes, detection_output_v8) { + NodeBuilder::get_ops().register_factory(); + const auto box_logits = make_shared(element::f32, Shape{1, 2 * 1 * 4}); + const auto class_preds = make_shared(element::f32, Shape{1, 2 * 32}); + const auto proposals = make_shared(element::f32, Shape{1, 2, 2 * 4}); + const auto aux_class_preds = make_shared(element::f32, Shape{1, 2 * 2}); + const auto aux_box_pred = make_shared(element::f32, Shape{1, 2 * 1 * 4}); + + op::v8::DetectionOutput::Attributes attrs; + initialize_attributes(attrs); + + auto detection_output = + make_shared(box_logits, class_preds, proposals, aux_class_preds, aux_box_pred, attrs); + NodeBuilder builder(detection_output); + auto g_detection_output = ov::as_type_ptr(builder.create()); + + const auto do_attrs = detection_output->get_attrs(); + const auto g_do_attrs = g_detection_output->get_attrs(); + + is_equal_attrs(g_do_attrs, do_attrs); } diff --git a/src/frontends/common/include/common/frontend.hpp b/src/frontends/common/include/common/frontend.hpp index 4dba69f4c2e..58ad1d66f64 100644 --- a/src/frontends/common/include/common/frontend.hpp +++ b/src/frontends/common/include/common/frontend.hpp @@ -10,10 +10,10 @@ #include "frontend_defs.hpp" #include "input_model.hpp" +#include "openvino/core/any.hpp" #include "openvino/core/extension.hpp" #include "openvino/core/function.hpp" #include "openvino/core/op_extension.hpp" -#include "openvino/core/variant.hpp" namespace ov { namespace frontend { @@ -36,7 +36,7 @@ public: /// \return true if model recognized, false - otherwise. template inline bool supported(const Types&... vars) const { - return supported_impl({ov::make_variant(vars)...}); + return supported_impl({ov::Any(vars)...}); } /// \brief Loads an input model by any specified arguments. Each FrontEnd separately @@ -48,10 +48,10 @@ public: /// \return Loaded input model. template inline InputModel::Ptr load(const Types&... vars) const { - return load_impl({ov::make_variant(vars)...}); + return load_impl({ov::Any{vars}...}); } - inline InputModel::Ptr load(const ov::VariantVector& vars) const { + inline InputModel::Ptr load(const ov::RuntimeAttributeVector& vars) const { return load_impl(vars); } @@ -125,12 +125,12 @@ public: } protected: - virtual bool supported_impl(const std::vector>& variants) const; - virtual InputModel::Ptr load_impl(const std::vector>& variants) const; + virtual bool supported_impl(const std::vector& variants) const; + virtual InputModel::Ptr load_impl(const std::vector& variants) const; }; template <> -inline bool FrontEnd::supported(const std::vector>& variants) const { +inline bool FrontEnd::supported(const std::vector& variants) const { return supported_impl(variants); } diff --git a/src/frontends/common/include/common/parameters.hpp b/src/frontends/common/include/common/parameters.hpp deleted file mode 100644 index f825a63de47..00000000000 --- a/src/frontends/common/include/common/parameters.hpp +++ /dev/null @@ -1,70 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#pragma once - -#include -#include -#include - -#include "frontend_defs.hpp" -#include "ngraph/opsets/opset.hpp" -#include "openvino/core/rtti.hpp" -#include "openvino/core/variant.hpp" -#include "openvino/op/constant.hpp" - -namespace ov { - -template <> -class FRONTEND_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("Variant::std::istream*"); - BWDCMP_RTTI_DECLARATION; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -template <> -class FRONTEND_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("Variant::std::istringstream*"); - BWDCMP_RTTI_DECLARATION; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -#if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) -template <> -class FRONTEND_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("Variant::std::wstring"); - BWDCMP_RTTI_DECLARATION; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; -#endif - -using Weights = std::shared_ptr; - -template <> -class FRONTEND_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("Variant::Weights"); - BWDCMP_RTTI_DECLARATION; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -using Extensions = std::map; - -template <> -class FRONTEND_API VariantWrapper : public VariantImpl { -public: - OPENVINO_RTTI("Variant::Extensions"); - BWDCMP_RTTI_DECLARATION; - - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; - -} // namespace ov diff --git a/src/frontends/common/include/manager.hpp b/src/frontends/common/include/manager.hpp index 5fed6c6d284..b79fc89ba2c 100644 --- a/src/frontends/common/include/manager.hpp +++ b/src/frontends/common/include/manager.hpp @@ -10,8 +10,7 @@ #include "common/frontend.hpp" #include "common/frontend_defs.hpp" -#include "common/parameters.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/any.hpp" namespace ov { namespace frontend { @@ -56,7 +55,7 @@ public: /// if no suitable frontend is found template FrontEnd::Ptr load_by_model(const Types&... vars) { - return load_by_model_impl({make_variant(vars)...}); + return load_by_model_impl({ov::Any{vars}...}); } /// \brief Gets list of registered frontends. Any not loaded frontends will be loaded by this call @@ -73,13 +72,13 @@ public: private: class Impl; - FrontEnd::Ptr load_by_model_impl(const std::vector>& variants); + FrontEnd::Ptr load_by_model_impl(const std::vector& variants); std::unique_ptr m_impl; }; template <> -FRONTEND_API FrontEnd::Ptr FrontEndManager::load_by_model(const std::vector>& variants); +FRONTEND_API FrontEnd::Ptr FrontEndManager::load_by_model(const std::vector& variants); // --------- Plugin exporting information -------------- diff --git a/src/frontends/common/src/frontend_manager.cpp b/src/frontends/common/src/frontend_manager.cpp index 193a151f0dc..48525eca7af 100644 --- a/src/frontends/common/src/frontend_manager.cpp +++ b/src/frontends/common/src/frontend_manager.cpp @@ -73,7 +73,7 @@ public: return names; } - FrontEnd::Ptr load_by_model(const std::vector>& variants) { + FrontEnd::Ptr load_by_model(const std::vector& variants) { std::lock_guard guard(m_loading_mutex); // Step 1: Search from hard-coded prioritized frontends first auto ptr = search_priority(variants); @@ -116,7 +116,7 @@ private: return info.is_file_name_match(names.file_name) || info.get_creator().m_name == names.name; } - FrontEnd::Ptr search_priority(const std::vector>& variants) { + FrontEnd::Ptr search_priority(const std::vector& variants) { // Map between file extension and suitable frontend static const std::map priority_fe_extensions = { {".xml", {"ir", "ir"}}, @@ -138,12 +138,12 @@ private: std::string model_path; const auto& model_variant = variants.at(0); - if (ov::is_type>(model_variant)) { - const auto& tmp_path = ov::as_type_ptr>(model_variant)->get(); + if (model_variant.is()) { + const auto& tmp_path = model_variant.as(); model_path = tmp_path; #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - } else if (ov::is_type>(model_variant)) { - auto wpath = ov::as_type_ptr>(model_variant)->get(); + } else if (model_variant.is()) { + auto wpath = model_variant.as(); model_path = ov::util::wstring_to_string(wpath); #endif } @@ -216,7 +216,7 @@ FrontEnd::Ptr FrontEndManager::load_by_framework(const std::string& framework) { return m_impl->load_by_framework(framework); } -FrontEnd::Ptr FrontEndManager::load_by_model_impl(const std::vector>& variants) { +FrontEnd::Ptr FrontEndManager::load_by_model_impl(const std::vector& variants) { return m_impl->load_by_model(variants); } @@ -229,7 +229,7 @@ void FrontEndManager::register_front_end(const std::string& name, FrontEndFactor } template <> -FrontEnd::Ptr FrontEndManager::load_by_model(const std::vector>& variants) { +FrontEnd::Ptr FrontEndManager::load_by_model(const std::vector& variants) { return load_by_model_impl(variants); } @@ -239,30 +239,30 @@ FrontEnd::FrontEnd() = default; FrontEnd::~FrontEnd() = default; -bool FrontEnd::supported_impl(const std::vector>& variants) const { +bool FrontEnd::supported_impl(const std::vector& variants) const { return false; } -InputModel::Ptr FrontEnd::load_impl(const std::vector>& params) const { +InputModel::Ptr FrontEnd::load_impl(const std::vector& params) const { FRONT_END_NOT_IMPLEMENTED(load_impl); } -std::shared_ptr FrontEnd::convert(InputModel::Ptr model) const { +std::shared_ptr FrontEnd::convert(InputModel::Ptr model) const { FRONT_END_NOT_IMPLEMENTED(convert); } -void FrontEnd::convert(std::shared_ptr) const { +void FrontEnd::convert(std::shared_ptr) const { FRONT_END_NOT_IMPLEMENTED(convert); } -std::shared_ptr FrontEnd::convert_partially(InputModel::Ptr model) const { +std::shared_ptr FrontEnd::convert_partially(InputModel::Ptr model) const { FRONT_END_NOT_IMPLEMENTED(convert_partially); } -std::shared_ptr FrontEnd::decode(InputModel::Ptr model) const { +std::shared_ptr FrontEnd::decode(InputModel::Ptr model) const { FRONT_END_NOT_IMPLEMENTED(decode); } -void FrontEnd::normalize(std::shared_ptr function) const { +void FrontEnd::normalize(std::shared_ptr function) const { FRONT_END_NOT_IMPLEMENTED(normalize); } @@ -370,15 +370,15 @@ void InputModel::extract_subgraph(const std::vector& inputs, const s } // Setting tensor properties -void InputModel::set_partial_shape(Place::Ptr place, const ngraph::PartialShape&) { +void InputModel::set_partial_shape(Place::Ptr place, const PartialShape&) { FRONT_END_NOT_IMPLEMENTED(set_partial_shape); } -ngraph::PartialShape InputModel::get_partial_shape(Place::Ptr place) const { +PartialShape InputModel::get_partial_shape(Place::Ptr place) const { FRONT_END_NOT_IMPLEMENTED(get_partial_shape); } -void InputModel::set_element_type(Place::Ptr place, const ngraph::element::Type&) { +void InputModel::set_element_type(Place::Ptr place, const element::Type&) { FRONT_END_NOT_IMPLEMENTED(set_element_type); } diff --git a/src/frontends/common/src/parameters.cpp b/src/frontends/common/src/parameters.cpp deleted file mode 100644 index 0d7125d6642..00000000000 --- a/src/frontends/common/src/parameters.cpp +++ /dev/null @@ -1,14 +0,0 @@ -// Copyright (C) 2018-2021 Intel Corporation -// SPDX-License-Identifier: Apache-2.0 -// - -#include "common/parameters.hpp" - -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); - -#if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) -BWDCMP_RTTI_DEFINITION(ov::VariantWrapper); -#endif diff --git a/src/frontends/ir/include/ir_frontend/frontend.hpp b/src/frontends/ir/include/ir_frontend/frontend.hpp index f5a85ea86c5..a3f554d843d 100644 --- a/src/frontends/ir/include/ir_frontend/frontend.hpp +++ b/src/frontends/ir/include/ir_frontend/frontend.hpp @@ -6,7 +6,6 @@ #include "common/frontend.hpp" #include "common/telemetry_extension.hpp" -#include "openvino/core/variant.hpp" #include "utility.hpp" namespace ov { @@ -35,12 +34,12 @@ protected: /// \brief Check if FrontEndIR can recognize model from given parts /// \param params Can be path to the model file or std::istream /// \return InputModel::Ptr - bool supported_impl(const std::vector>& variants) const override; + bool supported_impl(const std::vector& variants) const override; /// \brief Reads model from file or std::istream /// \param params Can be path to the model file or std::istream /// \return InputModel::Ptr - InputModel::Ptr load_impl(const std::vector>& params) const override; + InputModel::Ptr load_impl(const std::vector& params) const override; private: std::vector> shared_objects; diff --git a/src/frontends/ir/include/ir_frontend/model.hpp b/src/frontends/ir/include/ir_frontend/model.hpp index 8e92efcef8d..a05d5eecedf 100644 --- a/src/frontends/ir/include/ir_frontend/model.hpp +++ b/src/frontends/ir/include/ir_frontend/model.hpp @@ -9,6 +9,7 @@ #include "ir_frontend/utility.hpp" #include "manager.hpp" +#include "ngraph/runtime/aligned_buffer.hpp" namespace ov { namespace frontend { diff --git a/src/frontends/ir/src/frontend.cpp b/src/frontends/ir/src/frontend.cpp index 413c628d93b..096e84187f4 100644 --- a/src/frontends/ir/src/frontend.cpp +++ b/src/frontends/ir/src/frontend.cpp @@ -11,7 +11,7 @@ #include "ir_frontend/utility.hpp" #include "ngraph/runtime/aligned_buffer.hpp" #include "ngraph/runtime/shared_buffer.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/any.hpp" #include "openvino/util/file_util.hpp" #include "so_extension.hpp" #include "xml_parse_utils.h" @@ -59,7 +59,7 @@ size_t GetIRVersion(std::istream& model) { } // namespace -bool FrontEndIR::supported_impl(const std::vector>& variants) const { +bool FrontEndIR::supported_impl(const std::vector& variants) const { std::ifstream local_model_stream; std::istream* provided_model_stream = nullptr; @@ -68,18 +68,18 @@ bool FrontEndIR::supported_impl(const std::vector>& var } const auto& model_variant = variants[0]; - if (ov::is_type>(model_variant)) { - const auto& path = ov::as_type_ptr>(model_variant)->get(); + if (model_variant.is()) { + const auto& path = model_variant.as(); local_model_stream.open(path, std::ios::in | std::ifstream::binary); #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - } else if (ov::is_type>(model_variant)) { - const auto& path = ov::as_type_ptr>(model_variant)->get(); + } else if (model_variant.is()) { + const auto& path = model_variant.as(); local_model_stream.open(path, std::ios::in | std::ifstream::binary); #endif - } else if (ov::is_type>(model_variant)) { - provided_model_stream = ov::as_type_ptr>(model_variant)->get(); - } else if (ov::is_type>(model_variant)) { - provided_model_stream = ov::as_type_ptr>(model_variant)->get(); + } else if (model_variant.is()) { + provided_model_stream = model_variant.as(); + } else if (model_variant.is()) { + provided_model_stream = model_variant.as(); } if (provided_model_stream && local_model_stream.is_open()) { @@ -111,7 +111,7 @@ void FrontEndIR::add_extension(const ov::Extension::Ptr& ext) { extensions.emplace_back(ext); } -InputModel::Ptr FrontEndIR::load_impl(const std::vector>& variants) const { +InputModel::Ptr FrontEndIR::load_impl(const std::vector& variants) const { std::ifstream local_model_stream; std::istream* provided_model_stream = nullptr; std::shared_ptr weights; @@ -142,10 +142,10 @@ InputModel::Ptr FrontEndIR::load_impl(const std::vector std::string weights_path, model_path; #endif - // Load model const auto& model_variant = variants.at(0); - if (ov::is_type>(model_variant)) { - const auto& tmp_path = ov::as_type_ptr>(model_variant)->get(); + + if (model_variant.is()) { + const auto& tmp_path = model_variant.as(); #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) model_path = ov::util::string_to_wstring(tmp_path.c_str()); #else @@ -153,32 +153,32 @@ InputModel::Ptr FrontEndIR::load_impl(const std::vector #endif local_model_stream.open(model_path, std::ios::in | std::ifstream::binary); #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - } else if (ov::is_type>(model_variant)) { - model_path = ov::as_type_ptr>(model_variant)->get(); + } else if (model_variant.is()) { + model_path = model_variant.as(); local_model_stream.open(model_path, std::ios::in | std::ifstream::binary); #endif - } else if (ov::is_type>(model_variant)) { - provided_model_stream = ov::as_type_ptr>(model_variant)->get(); - } else if (ov::is_type>(model_variant)) { - provided_model_stream = ov::as_type_ptr>(model_variant)->get(); + } else if (model_variant.is()) { + provided_model_stream = model_variant.as(); + } else if (model_variant.is()) { + provided_model_stream = model_variant.as(); } // Check weights and extensions for (size_t variant_id = 1; variant_id < variants.size(); ++variant_id) { const auto& variant = variants.at(variant_id); - if (ov::is_type>(variant)) { - const auto& tmp_path = ov::as_type_ptr>(variant)->get(); + if (variant.is()) { + const auto& tmp_path = variant.as(); #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) weights_path = ov::util::string_to_wstring(tmp_path.c_str()); #else weights_path = tmp_path; #endif #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - } else if (ov::is_type>(variant)) { - weights_path = ov::as_type_ptr>(variant)->get(); + } else if (variant.is()) { + weights_path = variant.as(); #endif - } else if (ov::is_type>>(variant)) { - weights = ov::as_type_ptr>>(variant)->get(); + } else if (variant.is>()) { + weights = variant.as>(); } } diff --git a/src/frontends/ir/src/ir_deserializer.cpp b/src/frontends/ir/src/ir_deserializer.cpp index c19c1e4d98c..b4d1c687260 100644 --- a/src/frontends/ir/src/ir_deserializer.cpp +++ b/src/frontends/ir/src/ir_deserializer.cpp @@ -601,10 +601,11 @@ GenericLayerParams XmlDeserializer::parseGenericParams(const pugi::xml_node& nod return params; } -std::shared_ptr XmlDeserializer::createNode(const std::vector>& inputs, - const pugi::xml_node& node, - const ov::Weights& weights, - const GenericLayerParams& params) { +std::shared_ptr XmlDeserializer::createNode( + const std::vector>& inputs, + const pugi::xml_node& node, + const std::shared_ptr& weights, + const GenericLayerParams& params) { // Check that inputs are correctly defined for (size_t i = 0; i < inputs.size(); i++) { if (!inputs[i].get_node()) @@ -700,7 +701,7 @@ std::shared_ptr XmlDeserializer::createNode(const std::vector>(pr_data.value()); + rtInfo.emplace(ov::PrimitivesPriority::get_type_info_static(), ov::PrimitivesPriority{pr_data.value()}); } const auto aw_data = dn.attribute("alt_width"); if (aw_data) { @@ -721,24 +722,33 @@ std::shared_ptr XmlDeserializer::createNode(const std::vectorvisit_attributes(attribute_visitor)) { - rt_info[type_info] = attr; + auto attr = attrs_factory.create_by_type_info(type_info); + if (!attr.empty()) { + if (attr.is()) { + RTInfoDeserializer attribute_visitor(item); + if (attr.as().visit_attributes(attribute_visitor)) { + auto res = rt_info.emplace(type_info, attr); + if (!res.second) { + IE_THROW() << "multiple rt_info attributes are detected: " << attribute_name; + } + } else { + IE_THROW() << "VisitAttributes is not supported for: " << item.name() << " attribute"; + } } else { - IE_THROW() << "VisitAttributes is not supported for: " << attribute_name << " attribute"; + IE_THROW() << "Attribute: " << item.name() << " is not recognized as runtime attribute"; } } else { - IE_THROW() << "Attribute: " << attribute_name << " is not recognized"; + IE_THROW() << "Attribute: " << item.name() << " is not recognized"; } } }; diff --git a/src/frontends/ir/src/ir_deserializer.hpp b/src/frontends/ir/src/ir_deserializer.hpp index 192d8b7276d..e018c3df2cc 100644 --- a/src/frontends/ir/src/ir_deserializer.hpp +++ b/src/frontends/ir/src/ir_deserializer.hpp @@ -10,6 +10,7 @@ #include "ie_ngraph_utils.hpp" #include "ir_frontend/model.hpp" +#include "ngraph/opsets/opset.hpp" #include "openvino/core/attribute_visitor.hpp" #include "openvino/core/op_extension.hpp" #include "openvino/op/loop.hpp" @@ -175,12 +176,12 @@ private: std::shared_ptr createNode(const ov::OutputVector& inputs, const pugi::xml_node& node, - const ov::Weights& weights, + const std::shared_ptr& weights, const GenericLayerParams& params); // -- DATA -- const pugi::xml_node m_node; - const ov::Weights& m_weights; + const std::shared_ptr& m_weights; const std::unordered_map& m_opsets; const std::unordered_map& m_extensions; std::unordered_map>& m_variables; diff --git a/src/frontends/onnx/frontend/include/onnx_frontend/frontend.hpp b/src/frontends/onnx/frontend/include/onnx_frontend/frontend.hpp index d35f3485daf..331975df663 100644 --- a/src/frontends/onnx/frontend/include/onnx_frontend/frontend.hpp +++ b/src/frontends/onnx/frontend/include/onnx_frontend/frontend.hpp @@ -27,11 +27,11 @@ public: void convert(std::shared_ptr partially_converted) const override; std::shared_ptr decode(InputModel::Ptr model) const override; std::string get_name() const override; - bool supported_impl(const std::vector>& variants) const override; + bool supported_impl(const std::vector& variants) const override; void add_extension(const std::shared_ptr& extension) override; protected: - InputModel::Ptr load_impl(const std::vector>& params) const override; + InputModel::Ptr load_impl(const std::vector& params) const override; private: std::shared_ptr m_telemetry; diff --git a/src/frontends/onnx/frontend/src/frontend.cpp b/src/frontends/onnx/frontend/src/frontend.cpp index db73a7fde03..6acec3bd5f3 100644 --- a/src/frontends/onnx/frontend/src/frontend.cpp +++ b/src/frontends/onnx/frontend/src/frontend.cpp @@ -17,10 +17,6 @@ using namespace ov; using namespace ov::frontend; -using VariantString = VariantWrapper; -using VariantWString = VariantWrapper; -using VariantIstreamPtr = VariantWrapper; - ONNX_FRONTEND_C_API FrontEndVersion GetAPIVersion() { return OV_FRONTEND_API_VERSION; } @@ -34,29 +30,29 @@ ONNX_FRONTEND_C_API void* GetFrontEndData() { return res; } -InputModel::Ptr FrontEndONNX::load_impl(const std::vector>& variants) const { +InputModel::Ptr FrontEndONNX::load_impl(const std::vector& variants) const { if (variants.size() == 0) { return nullptr; } - if (ov::is_type(variants[0])) { - const auto path = ov::as_type_ptr(variants[0])->get(); + if (variants[0].is()) { + const auto path = variants[0].as(); return std::make_shared(path, m_telemetry); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - if (ov::is_type(variants[0])) { - const auto path = ov::as_type_ptr(variants[0])->get(); + if (variants[0].is()) { + const auto path = variants[0].as(); return std::make_shared(path, m_telemetry); } #endif - if (ov::is_type(variants[0])) { - const auto stream = ov::as_type_ptr(variants[0])->get(); - if (variants.size() > 1 && ov::is_type(variants[1])) { - const auto path = ov::as_type_ptr(variants[1])->get(); + if (variants[0].is()) { + const auto stream = variants[0].as(); + if (variants.size() > 1 && variants[1].is()) { + const auto path = variants[0].as(); return std::make_shared(*stream, path, m_telemetry); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - if (variants.size() > 1 && ov::is_type(variants[1])) { - const auto path = ov::as_type_ptr(variants[1])->get(); + if (variants.size() > 1 && variants[1].is()) { + const auto path = variants[1].as(); return std::make_shared(*stream, path, m_telemetry); } #endif @@ -107,18 +103,18 @@ private: }; } // namespace -bool FrontEndONNX::supported_impl(const std::vector>& variants) const { +bool FrontEndONNX::supported_impl(const std::vector& variants) const { if (variants.size() == 0) { return false; } std::ifstream model_stream; - if (ov::is_type(variants[0])) { - const auto path = ov::as_type_ptr(variants[0])->get(); + if (variants[0].is()) { + const auto path = variants[0].as(); model_stream.open(path, std::ios::in | std::ifstream::binary); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - else if (ov::is_type(variants[0])) { - const auto path = ov::as_type_ptr(variants[0])->get(); + else if (variants[0].is()) { + const auto path = variants[0].as(); model_stream.open(path, std::ios::in | std::ifstream::binary); } #endif @@ -128,11 +124,12 @@ bool FrontEndONNX::supported_impl(const std::vector>& v model_stream.close(); return is_valid_model; } - if (ov::is_type(variants[0])) { - const auto stream = ov::as_type_ptr(variants[0])->get(); + if (variants[0].is()) { + const auto stream = variants[0].as(); StreamRewinder rwd{*stream}; return ngraph::onnx_common::is_valid_model(*stream); } + return false; } diff --git a/src/frontends/paddlepaddle/include/paddlepaddle_frontend/frontend.hpp b/src/frontends/paddlepaddle/include/paddlepaddle_frontend/frontend.hpp index 2020aa00fee..dd3eb2de449 100644 --- a/src/frontends/paddlepaddle/include/paddlepaddle_frontend/frontend.hpp +++ b/src/frontends/paddlepaddle/include/paddlepaddle_frontend/frontend.hpp @@ -55,14 +55,14 @@ protected: /// \param params Can be path to folder which contains __model__ file or path to /// .pdmodel file /// \return InputModel::Ptr - bool supported_impl(const std::vector>& variants) const override; + bool supported_impl(const std::vector& variants) const override; /// \brief Reads model from 1 or 2 given file names or 1 or 2 std::istream containing /// model in protobuf format and weights /// \param params Can contain path to folder with __model__ file or path to .pdmodel /// file or 1 or 2 streams with model and weights /// \return InputModel::Ptr - InputModel::Ptr load_impl(const std::vector>& params) const override; + InputModel::Ptr load_impl(const std::vector& params) const override; private: static std::shared_ptr convert_each_node( diff --git a/src/frontends/paddlepaddle/src/decoder.cpp b/src/frontends/paddlepaddle/src/decoder.cpp index 56f21f0d15b..816d7864fcd 100644 --- a/src/frontends/paddlepaddle/src/decoder.cpp +++ b/src/frontends/paddlepaddle/src/decoder.cpp @@ -31,39 +31,34 @@ std::map TYPE_MAP{ {proto::VarType_Type::VarType_Type_INT8, ov::element::i8}, {proto::VarType_Type::VarType_Type_BF16, ov::element::bf16}}; -std::shared_ptr DecoderPDPDProto::get_attribute(const std::string& name, - const VariantTypeInfo& type_info) const { +ov::Any DecoderPDPDProto::get_attribute(const std::string& name, const std::type_info& type_info) const { auto attrs = decode_attribute_helper(name); if (attrs.empty()) { - return nullptr; + return {}; } - if (type_info == VariantWrapper::get_type_info_static()) { - return std::make_shared>(attrs[0].s()); - } else if (type_info == VariantWrapper::get_type_info_static()) { - return std::make_shared>(attrs[0].l()); - } else if (type_info == VariantWrapper>::get_type_info_static()) { - auto longs = std::vector(attrs[0].longs().begin(), attrs[0].longs().end()); - return std::make_shared>>(longs); - } else if (type_info == VariantWrapper::get_type_info_static()) { - return std::make_shared>(attrs[0].i()); - } else if (type_info == VariantWrapper>::get_type_info_static()) { - auto ints = std::vector(attrs[0].ints().begin(), attrs[0].ints().end()); - return std::make_shared>>(ints); - } else if (type_info == VariantWrapper::get_type_info_static()) { - return std::make_shared>(attrs[0].f()); - } else if (type_info == VariantWrapper>::get_type_info_static()) { - auto floats = std::vector(attrs[0].floats().begin(), attrs[0].floats().end()); - return std::make_shared>>(floats); - } else if (type_info == VariantWrapper::get_type_info_static()) { - auto data_type = (paddle::framework::proto::VarType_Type)attrs[0].i(); - return std::make_shared>(TYPE_MAP[data_type]); - } else if (type_info == VariantWrapper::get_type_info_static()) { - return std::make_shared>(attrs[0].b()); + if (type_info == typeid(std::string)) { + return attrs[0].s(); + } else if (type_info == typeid(int64_t)) { + return attrs[0].l(); + } else if (type_info == typeid(std::vector)) { + return std::vector(attrs[0].longs().begin(), attrs[0].longs().end()); + } else if (type_info == typeid(int32_t)) { + return attrs[0].i(); + } else if (type_info == typeid(std::vector)) { + return std::vector(attrs[0].ints().begin(), attrs[0].ints().end()); + } else if (type_info == typeid(float)) { + return attrs[0].f(); + } else if (type_info == typeid(std::vector)) { + return std::vector(attrs[0].floats().begin(), attrs[0].floats().end()); + } else if (type_info == typeid(ngraph::element::Type)) { + return TYPE_MAP[static_cast(attrs[0].i())]; + } else if (type_info == typeid(bool)) { + return attrs[0].b(); } // Type is not supported by decoder - return nullptr; + return {}; } std::vector DecoderPDPDProto::get_output_names() const { diff --git a/src/frontends/paddlepaddle/src/decoder.hpp b/src/frontends/paddlepaddle/src/decoder.hpp index 29ef5ee9eda..b56ed58116e 100644 --- a/src/frontends/paddlepaddle/src/decoder.hpp +++ b/src/frontends/paddlepaddle/src/decoder.hpp @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -26,7 +27,7 @@ class DecoderPDPDProto : public pdpd::DecoderBase { public: explicit DecoderPDPDProto(const std::shared_ptr& op) : op_place(op) {} - std::shared_ptr get_attribute(const std::string& name, const VariantTypeInfo& type_info) const override; + ov::Any get_attribute(const std::string& name, const std::type_info& type_info) const override; std::vector get_output_names() const override; diff --git a/src/frontends/paddlepaddle/src/frontend.cpp b/src/frontends/paddlepaddle/src/frontend.cpp index e366eb85e90..e8a75903894 100644 --- a/src/frontends/paddlepaddle/src/frontend.cpp +++ b/src/frontends/paddlepaddle/src/frontend.cpp @@ -13,7 +13,6 @@ #include "framework.pb.h" #include "node_context.hpp" #include "op_table.hpp" -#include "openvino/core/variant.hpp" #include "openvino/opsets/opset7.hpp" #include "paddlepaddle_frontend/exceptions.hpp" #include "paddlepaddle_frontend/model.hpp" @@ -116,16 +115,16 @@ bool normalize_framework_node(const std::shared_ptr& node, return true; } -std::istream* variant_to_stream_ptr(const std::shared_ptr& variant, std::ifstream& ext_stream) { - if (ov::is_type>(variant)) { - return ov::as_type_ptr>(variant)->get(); - } else if (ov::is_type>(variant)) { - const auto& model_path = ov::as_type_ptr>(variant)->get(); +std::istream* variant_to_stream_ptr(const ov::Any& variant, std::ifstream& ext_stream) { + if (variant.is()) { + return variant.as(); + } else if (variant.is()) { + const auto& model_path = variant.as(); ext_stream.open(model_path, std::ios::in | std::ifstream::binary); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - else if (ov::is_type>(variant)) { - const auto& model_path = ov::as_type_ptr>(variant)->get(); + else if (variant.is()) { + const auto& model_path = variant.as(); ext_stream.open(model_path, std::ios::in | std::ifstream::binary); } #endif @@ -205,15 +204,15 @@ std::shared_ptr FrontEndPDPD::convert_each_node( return std::make_shared(result_nodes, parameter_nodes); } -bool FrontEndPDPD::supported_impl(const std::vector>& variants) const { +bool FrontEndPDPD::supported_impl(const std::vector& variants) const { // FrontEndPDPD can only load model specified by one path, one file or two files. if (variants.empty() || variants.size() > 2) return false; // Validating first path, it must contain a model - if (ov::is_type>(variants[0])) { + if (variants[0].is()) { std::string suffix = ".pdmodel"; - std::string model_path = ov::as_type_ptr>(variants[0])->get(); + std::string model_path = variants[0].as(); if (!pdpd::endsWith(model_path, suffix)) { model_path += pdpd::get_path_sep() + "__model__"; } @@ -223,9 +222,9 @@ bool FrontEndPDPD::supported_impl(const std::vector>& v return model_str && model_str.is_open(); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - else if (ov::is_type>(variants[0])) { + else if (variants[0].is()) { std::wstring suffix = L".pdmodel"; - std::wstring model_path = ov::as_type_ptr>(variants[0])->get(); + std::wstring model_path = variants[0].as(); if (!pdpd::endsWith(model_path, suffix)) { model_path += pdpd::get_path_sep() + L"__model__"; } @@ -235,32 +234,32 @@ bool FrontEndPDPD::supported_impl(const std::vector>& v return model_str && model_str.is_open(); } #endif - else if (ov::is_type>(variants[0])) { + else if (variants[0].is()) { // Validating first stream, it must contain a model - auto p_model_stream = ov::as_type_ptr>(variants[0])->get(); + auto p_model_stream = variants[0].as(); paddle::framework::proto::ProgramDesc fw; return fw.ParseFromIstream(p_model_stream); } return false; } -InputModel::Ptr FrontEndPDPD::load_impl(const std::vector>& variants) const { +InputModel::Ptr FrontEndPDPD::load_impl(const std::vector& variants) const { if (variants.size() == 1) { // The case when folder with __model__ and weight files is provided or .pdmodel file - if (ov::is_type>(variants[0])) { - std::string m_path = ov::as_type_ptr>(variants[0])->get(); + if (variants[0].is()) { + std::string m_path = variants[0].as(); return std::make_shared(m_path, m_telemetry); } #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) - else if (ov::is_type>(variants[0])) { - std::wstring m_path = ov::as_type_ptr>(variants[0])->get(); + else if (variants[0].is()) { + std::wstring m_path = variants[0].as(); return std::make_shared(m_path, m_telemetry); } #endif // The case with only model stream provided and no weights. This means model has // no learnable weights - else if (ov::is_type>(variants[0])) { - auto p_model_stream = ov::as_type_ptr>(variants[0])->get(); + else if (variants[0].is()) { + auto p_model_stream = variants[0].as(); return std::make_shared(std::vector{p_model_stream}, m_telemetry); } } else if (variants.size() == 2) { diff --git a/src/frontends/paddlepaddle/src/node_context.hpp b/src/frontends/paddlepaddle/src/node_context.hpp index 28461725362..8b363d1f6ba 100644 --- a/src/frontends/paddlepaddle/src/node_context.hpp +++ b/src/frontends/paddlepaddle/src/node_context.hpp @@ -4,26 +4,11 @@ #pragma once #include "ngraph/compatibility.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/any.hpp" #include "paddlepaddle_frontend/exceptions.hpp" #include "paddlepaddle_frontend/utility.hpp" -#define OPENVINO_VARIANT_DECLARATION(TYPE, info) \ - template <> \ - class VariantWrapper : public VariantImpl { \ - public: \ - OPENVINO_RTTI(info); \ - VariantWrapper(const value_type& value) : VariantImpl(value) {} \ - } - namespace ov { -OPENVINO_VARIANT_DECLARATION(int32_t, "Variant::int32"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::int32_vector"); -OPENVINO_VARIANT_DECLARATION(float, "Variant::float"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::float_vector"); -OPENVINO_VARIANT_DECLARATION(bool, "Variant::bool"); -OPENVINO_VARIANT_DECLARATION(ov::element::Type, "Variant::element_type"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::int64_vector"); namespace frontend { namespace pdpd { using InPortName = std::string; @@ -39,7 +24,7 @@ public: /// \param name Attribute name /// \param type_info Attribute type information /// \return Shared pointer to appropriate value if it exists, 'nullptr' otherwise - virtual std::shared_ptr get_attribute(const std::string& name, const VariantTypeInfo& type_info) const = 0; + virtual ov::Any get_attribute(const std::string& name, const std::type_info& type_info) const = 0; virtual std::vector get_output_names() const = 0; @@ -69,71 +54,26 @@ public: NodeContext(const DecoderBase& _decoder, const NamedInputs& _name_map) : decoder(_decoder), name_map(_name_map) {} /// Returns node attribute by name. Returns 'def' value if attribute does not exist - template >::value, bool>::type = true> + template T get_attribute(const std::string& name, const T& def) const { - std::shared_ptr res; - OPENVINO_SUPPRESS_DEPRECATED_START - res = decoder.get_attribute(name, VariantWrapper::type_info); - OPENVINO_SUPPRESS_DEPRECATED_END - if (res) { - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); - } else { - return def; - } - } - template >::value, bool>::type = true> - T get_attribute(const std::string& name, const T& def) const { - std::shared_ptr res; - res = decoder.get_attribute(name, VariantWrapper::get_type_info_static()); - if (res) { - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); + auto res = decoder.get_attribute(name, typeid(T)); + if (!res.empty()) { + return res.as(); } else { return def; } } - template >::value, bool>::type = true> + template T get_attribute(const std::string& name) const { - std::shared_ptr res; - OPENVINO_SUPPRESS_DEPRECATED_START - res = decoder.get_attribute(name, VariantWrapper::type_info); - OPENVINO_SUPPRESS_DEPRECATED_END - FRONT_END_GENERAL_CHECK(res, "Attribute with name '", name, "' does not exist"); - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); + auto res = decoder.get_attribute(name, typeid(T)); + FRONT_END_GENERAL_CHECK(!res.empty(), "Attribute with name '", name, "' does not exist"); + return res.as(); } - template >::value, bool>::type = true> - T get_attribute(const std::string& name) const { - std::shared_ptr res; - res = decoder.get_attribute(name, VariantWrapper::get_type_info_static()); - FRONT_END_GENERAL_CHECK(res, "Attribute with name '", name, "' does not exist"); - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); - } - - template >::value, bool>::type = true> + template bool has_attribute(const std::string& name) const { - std::shared_ptr res; - OPENVINO_SUPPRESS_DEPRECATED_START - res = decoder.get_attribute(name, VariantWrapper::type_info); - OPENVINO_SUPPRESS_DEPRECATED_END - return res != nullptr; - } - - template >::value, bool>::type = true> - bool has_attribute(const std::string& name) const { - std::shared_ptr res; - OPENVINO_SUPPRESS_DEPRECATED_START - res = decoder.get_attribute(name, VariantWrapper::get_type_info_static()); - OPENVINO_SUPPRESS_DEPRECATED_END - return res != nullptr; + return !decoder.get_attribute(name, typeid(T)).empty(); } /// Detects if there is at least one input attached with a given name diff --git a/src/frontends/paddlepaddle/src/op/conv2d_utils.hpp b/src/frontends/paddlepaddle/src/op/conv2d_utils.hpp index f5632612191..6ed034231f5 100644 --- a/src/frontends/paddlepaddle/src/op/conv2d_utils.hpp +++ b/src/frontends/paddlepaddle/src/op/conv2d_utils.hpp @@ -4,6 +4,7 @@ #pragma once #include "node_context.hpp" +#include "openvino/core/coordinate_diff.hpp" namespace ov { namespace frontend { diff --git a/src/frontends/tensorflow/include/tensorflow_frontend/decoder.hpp b/src/frontends/tensorflow/include/tensorflow_frontend/decoder.hpp index 80ecb99935f..423f030934c 100644 --- a/src/frontends/tensorflow/include/tensorflow_frontend/decoder.hpp +++ b/src/frontends/tensorflow/include/tensorflow_frontend/decoder.hpp @@ -4,8 +4,8 @@ #pragma once -#include "openvino/core/any.hpp" -#include "openvino/core/variant.hpp" +#include + #include "tensorflow_frontend/utility.hpp" namespace ov { @@ -18,8 +18,7 @@ public: /// \param name Attribute name /// \param type_info Attribute type information /// \return Shared pointer to appropriate value if it exists, 'nullptr' otherwise - virtual std::shared_ptr get_attribute(const std::string& name, - const VariantTypeInfo& type_info) const = 0; + virtual ov::Any get_attribute(const std::string& name, const std::type_info& type_info) const = 0; /// \brief Get a number of inputs virtual size_t get_input_size() const = 0; diff --git a/src/frontends/tensorflow/include/tensorflow_frontend/frontend.hpp b/src/frontends/tensorflow/include/tensorflow_frontend/frontend.hpp index 2a34b501abb..0066ac00267 100644 --- a/src/frontends/tensorflow/include/tensorflow_frontend/frontend.hpp +++ b/src/frontends/tensorflow/include/tensorflow_frontend/frontend.hpp @@ -12,7 +12,6 @@ #include "common/telemetry_extension.hpp" #include "openvino/core/any.hpp" #include "openvino/core/node_vector.hpp" -#include "openvino/core/variant.hpp" #include "tensorflow_frontend/utility.hpp" namespace ov { @@ -72,9 +71,9 @@ public: protected: /// \brief Check if FrontEndTensorflow can recognize model from given parts - bool supported_impl(const std::vector>& variants) const override; + bool supported_impl(const std::vector& variants) const override; - ov::frontend::InputModel::Ptr load_impl(const std::vector>& variants) const override; + ov::frontend::InputModel::Ptr load_impl(const std::vector& variants) const override; private: void translate_graph(const ov::frontend::InputModel::Ptr& model, diff --git a/src/frontends/tensorflow/include/tensorflow_frontend/graph_iterator.hpp b/src/frontends/tensorflow/include/tensorflow_frontend/graph_iterator.hpp index 613035894f5..de8a548fd14 100644 --- a/src/frontends/tensorflow/include/tensorflow_frontend/graph_iterator.hpp +++ b/src/frontends/tensorflow/include/tensorflow_frontend/graph_iterator.hpp @@ -5,15 +5,16 @@ #pragma once #include "openvino/core/any.hpp" -#include "openvino/core/variant.hpp" #include "tensorflow_frontend/decoder.hpp" #include "tensorflow_frontend/utility.hpp" namespace ov { namespace frontend { /// Abstract representation for an input model graph that gives nodes in topologically sorted order -class TF_API GraphIterator { +class TF_API GraphIterator : ov::RuntimeAttribute { public: + OPENVINO_RTTI("Variant::GraphIterator"); + using Ptr = std::shared_ptr; /// \brief Get a number of operation nodes in the graph @@ -36,15 +37,3 @@ public: }; } // namespace frontend } // namespace ov - -namespace ov { -/// Keep GraphIterator::Ptr object and type information for type-safe -/// dynamic conversions without using C++ RTTI -template <> -class TF_API VariantWrapper<::ov::frontend::GraphIterator::Ptr> - : public VariantImpl<::ov::frontend::GraphIterator::Ptr> { -public: - OPENVINO_RTTI("Variant::GraphIterator::Ptr"); - VariantWrapper(const value_type& value) : VariantImpl(value) {} -}; -} // namespace ov diff --git a/src/frontends/tensorflow/src/decoder_proto.cpp b/src/frontends/tensorflow/src/decoder_proto.cpp index 31ec6657ebe..5a7a26c4356 100644 --- a/src/frontends/tensorflow/src/decoder_proto.cpp +++ b/src/frontends/tensorflow/src/decoder_proto.cpp @@ -6,8 +6,6 @@ #include "node_context.hpp" -using namespace std; - namespace ov { namespace frontend { namespace tf { @@ -29,72 +27,62 @@ const std::map<::tensorflow::DataType, ov::element::Type>& TYPE_MAP() { } } // namespace -template -bool is_type(const VariantTypeInfo& type_info) { - return type_info == VariantWrapper::get_type_info_static(); -} - -template -shared_ptr> create_variant(const T& data) { - return make_shared>(data); -} - -shared_ptr DecoderTFProto::get_attribute(const string& name, const VariantTypeInfo& type_info) const { +ov::Any DecoderTFProto::get_attribute(const std::string& name, const std::type_info& type_info) const { auto attrs = decode_attribute_helper(name); if (attrs.empty()) { - return nullptr; + return {}; } - if (is_type(type_info)) { - return create_variant(attrs[0].s()); - } else if (is_type(type_info)) { - return create_variant(attrs[0].i()); - } else if (is_type>(type_info)) { - vector longs; + if (type_info == typeid(std::string)) { + return attrs[0].s(); + } else if (type_info == typeid(int64_t)) { + return attrs[0].i(); + } else if (type_info == typeid(std::vector)) { + std::vector longs; longs.reserve(attrs[0].list().i_size()); for (size_t idx = 0; idx < attrs[0].list().i_size(); ++idx) { longs.push_back(attrs[0].list().i(idx)); } - return create_variant>(longs); - } else if (is_type(type_info)) { - return create_variant(static_cast(attrs[0].i())); - } else if (is_type>(type_info)) { - vector ints; + return longs; + } else if (type_info == typeid(int32_t)) { + return static_cast(attrs[0].i()); + } else if (type_info == typeid(std::vector)) { + std::vector ints; ints.reserve(attrs[0].list().i_size()); for (size_t idx = 0; idx < attrs[0].list().i_size(); ++idx) { ints.push_back(static_cast(attrs[0].list().i(idx))); } - return create_variant>(ints); - } else if (is_type(type_info)) { - return create_variant(attrs[0].f()); - } else if (is_type>(type_info)) { - vector floats; + return ints; + } else if (type_info == typeid(float)) { + return attrs[0].f(); + } else if (type_info == typeid(std::vector)) { + std::vector floats; floats.reserve(attrs[0].list().i_size()); for (size_t idx = 0; idx < attrs[0].list().i_size(); ++idx) { floats.push_back(attrs[0].list().f(idx)); } - return create_variant>(floats); - } else if (is_type(type_info)) { + return floats; + } else if (type_info == typeid(ov::element::Type)) { auto data_type = attrs[0].type(); - return create_variant(TYPE_MAP().at(data_type)); - } else if (is_type(type_info)) { - return create_variant(attrs[0].b()); - } else if (is_type<::tensorflow::DataType>(type_info)) { - return create_variant<::tensorflow::DataType>(attrs[0].type()); - } else if (is_type<::tensorflow::TensorProto>(type_info)) { - return create_variant<::tensorflow::TensorProto>(attrs[0].tensor()); - } else if (is_type<::ov::PartialShape>(type_info)) { - vector dims; + return TYPE_MAP().at(data_type); + } else if (type_info == typeid(bool)) { + return attrs[0].b(); + } else if (type_info == typeid(::tensorflow::DataType)) { + return attrs[0].type(); + } else if (type_info == typeid(::tensorflow::TensorProto)) { + return attrs[0].tensor(); + } else if (type_info == typeid(::ov::PartialShape)) { + std::vector dims; auto tf_shape = attrs[0].shape(); for (int i = 0; i < tf_shape.dim_size(); i++) { - dims.emplace_back(tf_shape.dim(i).size()); + dims.push_back(tf_shape.dim(i).size()); } auto pshape = ov::PartialShape(dims); - return create_variant<::ov::PartialShape>(pshape); + return pshape; } // type is not supported by decoder - return nullptr; + return {}; } size_t DecoderTFProto::get_input_size() const { @@ -102,14 +90,14 @@ size_t DecoderTFProto::get_input_size() const { } void DecoderTFProto::get_input_node(size_t input_port_idx, - string& producer_name, + std::string& producer_name, size_t& producer_output_port_index) const { // TODO: handle body graph nodes with a couple of columns - string producer_port_name = m_node_def->input(input_port_idx); + std::string producer_port_name = m_node_def->input(input_port_idx); auto delim_pos = producer_port_name.find(':'); - if (delim_pos != string::npos) { + if (delim_pos != std::string::npos) { producer_name = producer_port_name.substr(0, delim_pos); - producer_output_port_index = stoi(producer_port_name.substr(delim_pos)); + producer_output_port_index = std::stoi(producer_port_name.substr(delim_pos)); return; } producer_name = producer_port_name; @@ -124,11 +112,16 @@ const std::string& DecoderTFProto::get_op_name() const { return m_node_def->name(); } -vector<::tensorflow::AttrValue> DecoderTFProto::decode_attribute_helper(const string& name) const { +std::vector<::tensorflow::AttrValue> DecoderTFProto::decode_attribute_helper(const std::string& name) const { auto attr_map = m_node_def->attr(); - if (attr_map.contains(name)) - return {m_node_def->attr().at(name)}; - return {}; + FRONT_END_GENERAL_CHECK(attr_map.contains(name), + "An error occurred while parsing the ", + name, + " attribute of ", + this->get_op_type(), + "node"); + auto value = m_node_def->attr().at(name); + return {value}; } } // namespace tf } // namespace frontend diff --git a/src/frontends/tensorflow/src/decoder_proto.hpp b/src/frontends/tensorflow/src/decoder_proto.hpp index 087986deeb3..a0871b20aa7 100644 --- a/src/frontends/tensorflow/src/decoder_proto.hpp +++ b/src/frontends/tensorflow/src/decoder_proto.hpp @@ -20,8 +20,7 @@ class DecoderTFProto : public DecoderBase { public: explicit DecoderTFProto(const ::tensorflow::NodeDef* node_def) : m_node_def(node_def) {} - std::shared_ptr get_attribute(const std::string& name, - const VariantTypeInfo& type_info) const override; + ov::Any get_attribute(const std::string& name, const std::type_info& type_info) const override; size_t get_input_size() const override; diff --git a/src/frontends/tensorflow/src/frontend.cpp b/src/frontends/tensorflow/src/frontend.cpp index 5c6c83fb9d9..1ad1b019c82 100644 --- a/src/frontends/tensorflow/src/frontend.cpp +++ b/src/frontends/tensorflow/src/frontend.cpp @@ -268,38 +268,38 @@ void FrontEndTF::translate_graph(const ov::frontend::InputModel::Ptr& model, } /// \brief Check if FrontEndTensorflow can recognize model from given parts -bool FrontEndTF::supported_impl(const std::vector>& variants) const { +bool FrontEndTF::supported_impl(const std::vector& variants) const { // TODO: Support other TensorFlow formats: SavedModel, .meta, checkpoint, pbtxt if (variants.size() != 1) return false; // Validating first path, it must contain a model - if (ov::is_type>(variants[0])) { + if (variants[0].is()) { std::string suffix = ".pb"; - std::string model_path = ov::as_type_ptr>(variants[0])->get(); + std::string model_path = variants[0].as(); if (ov::util::ends_with(model_path, suffix.c_str())) { return true; } - } else if (ov::is_type>(variants[0])) { + } else if (variants[0].is()) { return true; } return false; } -ov::frontend::InputModel::Ptr FrontEndTF::load_impl(const std::vector>& variants) const { +ov::frontend::InputModel::Ptr FrontEndTF::load_impl(const std::vector& variants) const { // TODO: Support other TensorFlow formats: SavedModel, .meta, checkpoint, pbtxt if (variants.size() == 1) { // a case when binary protobuf format is provided - if (ov::is_type>(variants[0])) { + if (variants[0].is()) { std::string suffix = ".pb"; - std::string model_path = ov::as_type_ptr>(variants[0])->get(); + std::string model_path = variants[0].as(); if (ov::util::ends_with(model_path, suffix.c_str())) { return std::make_shared( std::make_shared<::ov::frontend::tf::GraphIteratorProto>(model_path), m_telemetry); } - } else if (ov::is_type>(variants[0])) { - auto graph_iterator = ov::as_type_ptr>(variants[0])->get(); + } else if (variants[0].is()) { + auto graph_iterator = variants[0].as(); return std::make_shared(graph_iterator, m_telemetry); } } diff --git a/src/frontends/tensorflow/src/model.cpp b/src/frontends/tensorflow/src/model.cpp index 695217d22e4..e3e1aa76531 100644 --- a/src/frontends/tensorflow/src/model.cpp +++ b/src/frontends/tensorflow/src/model.cpp @@ -116,12 +116,10 @@ void InputModelTF::InputModelTFImpl::loadPlaces() { m_op_places.push_back(op_place); m_op_places_map[op_name] = op_place; if (op_type == "Placeholder") { - auto pshape = std::dynamic_pointer_cast>( - node_decoder->get_attribute("shape", VariantWrapper::get_type_info_static())); - auto type = std::dynamic_pointer_cast>( - node_decoder->get_attribute("dtype", VariantWrapper::get_type_info_static())); + auto pshape = node_decoder->get_attribute("shape", typeid(ov::PartialShape)).as(); + auto type = node_decoder->get_attribute("dtype", typeid(ov::element::Type)).as(); std::vector names = {op_name}; - auto tensor_place = std::make_shared(m_input_model, pshape->get(), type->get(), names); + auto tensor_place = std::make_shared(m_input_model, pshape, type, names); m_tensor_places[op_name] = tensor_place; m_inputs.push_back(tensor_place); } diff --git a/src/frontends/tensorflow/src/node_context.hpp b/src/frontends/tensorflow/src/node_context.hpp index 28a8729140f..2913a2fa799 100644 --- a/src/frontends/tensorflow/src/node_context.hpp +++ b/src/frontends/tensorflow/src/node_context.hpp @@ -3,38 +3,14 @@ // #pragma once +#include #include "exceptions.hpp" -#include "openvino/core/any.hpp" -#include "openvino/core/variant.hpp" #include "place.hpp" #include "tensor.pb.h" #include "tensorflow_frontend/utility.hpp" #include "types.pb.h" -#define OPENVINO_VARIANT_DECLARATION(TYPE, info) \ - template <> \ - class VariantWrapper : public VariantImpl { \ - public: \ - OPENVINO_RTTI(info); \ - VariantWrapper(const value_type& value) : VariantImpl(value) {} \ - } - -namespace ov { -OPENVINO_VARIANT_DECLARATION(int32_t, "Variant::int32"); -OPENVINO_VARIANT_DECLARATION(uint64_t, "Variant::uint64_t"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::int32_vector"); -OPENVINO_VARIANT_DECLARATION(float, "Variant::float"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::float_vector"); -OPENVINO_VARIANT_DECLARATION(bool, "Variant::bool"); -OPENVINO_VARIANT_DECLARATION(ov::element::Type, "Variant::ov_element_type"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::int64_vector"); -OPENVINO_VARIANT_DECLARATION(ov::PartialShape, "Variant:ov_PartialShape"); -OPENVINO_VARIANT_DECLARATION(std::vector, "Variant::string_vector"); -OPENVINO_VARIANT_DECLARATION(::tensorflow::DataType, "Variant::DataType"); -OPENVINO_VARIANT_DECLARATION(::tensorflow::TensorProto, "Variant::TensorProto"); -} // namespace ov - namespace ov { namespace frontend { namespace tf { @@ -55,11 +31,9 @@ public: /// Returns node attribute by name. Returns 'def' value if attribute does not exist template T get_attribute(const std::string& name, const T& def) const { - auto res = m_decoder.get_attribute(name, VariantWrapper::get_type_info_static()); - if (res) { - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); + auto res = m_decoder.get_attribute(name, typeid(T)); + if (!res.empty()) { + return res.as(); } return def; } @@ -67,17 +41,15 @@ public: /// Returns node attribute by name template T get_attribute(const std::string& name) const { - auto res = m_decoder.get_attribute(name, VariantWrapper::get_type_info_static()); - FRONT_END_GENERAL_CHECK(res, "Attribute with name '", name, "' does not exist"); - auto ret = std::dynamic_pointer_cast>(res); - FRONT_END_GENERAL_CHECK(ret, "Attribute with name '", name, "' has invalid type"); - return ret->get(); + auto res = m_decoder.get_attribute(name, typeid(T)); + FRONT_END_GENERAL_CHECK(!res.empty(), "Attribute with name '", name, "' does not exist"); + return res.as(); } /// Check if an attribute of a given name exists template bool has_attribute(const std::string& name) const { - return m_decoder.get_attribute(name, VariantWrapper::get_type_info_static()) != nullptr; + return !m_decoder.get_attribute(name, typeid(T)).empty(); } /// Detects if there is at least one input attached with a given name diff --git a/src/frontends/tensorflow/src/utils.hpp b/src/frontends/tensorflow/src/utils.hpp index 7ab6aa9f093..6de7e432ee2 100644 --- a/src/frontends/tensorflow/src/utils.hpp +++ b/src/frontends/tensorflow/src/utils.hpp @@ -91,18 +91,8 @@ void get_const_input(const NodeContext& node, int64_t input_index, std::vector void values_from_const_node(const NodeContext& node, ov::Shape* const_tensor_shape, std::vector* values) { TF_OP_VALIDATION_CHECK(node, node.get_op_type() == "Const", "Node is expected to be Constant."); - const auto* decoder = node.get_decoder(); - auto dt1 = decoder->get_attribute("dtype", ::ov::VariantWrapper<::tensorflow::DataType>::get_type_info_static()); - FRONT_END_GENERAL_CHECK(dt1 != nullptr); - auto dt = std::dynamic_pointer_cast<::ov::VariantWrapper<::tensorflow::DataType>>(dt1)->get(); - - auto tensor_proto_var = - decoder->get_attribute("value", ::ov::VariantWrapper<::tensorflow::TensorProto>::get_type_info_static()); - FRONT_END_GENERAL_CHECK(tensor_proto_var != nullptr); - - auto tensor_proto = - std::dynamic_pointer_cast<::ov::VariantWrapper<::tensorflow::TensorProto>>(tensor_proto_var)->get(); - + auto dt = node.get_attribute<::tensorflow::DataType>("dtype"); + auto tensor_proto = node.get_attribute<::tensorflow::TensorProto>("value"); const tensorflow::TensorShapeProto& shape = tensor_proto.tensor_shape(); ov::PartialShape pshape; tf_shape_to_ov_shape(shape, &pshape); diff --git a/src/inference/include/openvino/runtime/core.hpp b/src/inference/include/openvino/runtime/core.hpp index 1044a2d91bf..6fee2f38b41 100644 --- a/src/inference/include/openvino/runtime/core.hpp +++ b/src/inference/include/openvino/runtime/core.hpp @@ -17,6 +17,7 @@ #include "ie_plugin_config.hpp" #include "openvino/core/extension.hpp" +#include "openvino/core/function.hpp" #include "openvino/core/op_extension.hpp" #include "openvino/core/version.hpp" #include "openvino/op/op.hpp" @@ -27,13 +28,10 @@ namespace InferenceEngine { class IExtension; -class RemoteContext; } // namespace InferenceEngine namespace ov { -class Function; - namespace runtime { /** @@ -51,100 +49,100 @@ public: * * See register_plugins for more details. * - * @param xmlConfigFile A path to .xml file with plugins to load from. If XML configuration file is not specified, + * @param xml_config_file A path to .xml file with plugins to load from. If XML configuration file is not specified, * then default OpenVINO Runtime plugins are loaded from the default plugin.xml file. */ - explicit Core(const std::string& xmlConfigFile = {}); + explicit Core(const std::string& xml_config_file = {}); /** * @brief Returns plugins version information * - * @param deviceName Device name to identify plugin + * @param device_name Device name to identify plugin * @return A vector of versions */ - std::map get_versions(const std::string& deviceName) const; + std::map get_versions(const std::string& device_name) const; #ifdef OPENVINO_ENABLE_UNICODE_PATH_SUPPORT /** * @brief Reads models from IR and ONNX formats - * @param modelPath path to model - * @param binPath path to data file + * @param model_path path to model + * @param bin_path path to data file * For IR format (*.bin): * * if path is empty, will try to read bin file with the same name as xml and * * if bin file with the same name was not found, will load IR without weights. * For ONNX format (*.onnx): - * * binPath parameter is not used. - * @return Function + * * bin_path parameter is not used. + * @return Model */ - std::shared_ptr read_model(const std::wstring& modelPath, const std::wstring& binPath = {}) const; + std::shared_ptr read_model(const std::wstring& model_path, const std::wstring& bin_path = {}) const; #endif /** * @brief Reads models from IR and ONNX formats - * @param modelPath path to model - * @param binPath path to data file + * @param model_path path to model + * @param bin_path path to data file * For IR format (*.bin): * * if path is empty, will try to read bin file with the same name as xml and * * if bin file with the same name was not found, will load IR without weights. * For ONNX format (*.onnx): - * * binPath parameter is not used. - * @return Function + * * bin_path parameter is not used. + * @return Model */ - std::shared_ptr read_model(const std::string& modelPath, const std::string& binPath = {}) const; + std::shared_ptr read_model(const std::string& model_path, const std::string& bin_path = {}) const; /** * @brief Reads models from IR and ONNX formats * @param model string with model in IR or ONNX format * @param weights shared pointer to constant tensor with weights * Reading ONNX models doesn't support loading weights from data tensors. - * @note Created Function object shares the weights with `weights` object. - * So, do not create `weights` on temporary data which can be later freed, since the network + * @note Created model object shares the weights with `weights` object. + * So, do not create `weights` on temporary data which can be later freed, since the model * constant data becomes to point to invalid memory. - * @return Function + * @return Model */ std::shared_ptr read_model(const std::string& model, const Tensor& weights) const; /** - * @brief Creates an executable network from a network object. + * @brief Creates an executable network from a model object. * - * Users can create as many networks as they need and use - * them simultaneously (up to the limitation of the hardware resources) + * Users can create as many executable networks as they need and use + * them simultaneously (up to the limitation of the hardware resources) * - * @param network Function object acquired from Core::read_model - * @param deviceName Name of device to load network to + * @param model Model object acquired from Core::read_model + * @param device_name Name of device to load model to * @param config Optional map of pairs: (config parameter name, config parameter value) relevant only for this load * operation * @return An executable network reference */ - ExecutableNetwork compile_model(const std::shared_ptr& network, - const std::string& deviceName, + ExecutableNetwork compile_model(const std::shared_ptr& model, + const std::string& device_name, const ConfigMap& config = {}); /** * @brief Reads model and creates an executable network from IR or ONNX file * - * This can be more efficient than using read_model + compile_model(Function) flow - * especially for cases when caching is enabled and cached model is available + * This can be more efficient than using read_model + compile_model(Model) flow + * especially for cases when caching is enabled and cached model is available * - * @param modelPath path to model - * @param deviceName Name of device to load network to + * @param model_path path to model + * @param device_name Name of device to load model to * @param config Optional map of pairs: (config parameter name, config parameter value) relevant only for this load * operation/ * * @return An executable network reference */ - ExecutableNetwork compile_model(const std::string& modelPath, - const std::string& deviceName, + ExecutableNetwork compile_model(const std::string& model_path, + const std::string& device_name, const ConfigMap& config = {}); /** * @brief Creates an executable network from a network object within a specified remote context. - * @param network Function object acquired from Core::read_model + * @param model Model object acquired from Core::read_model * @param context Pointer to RemoteContext object * @param config Optional map of pairs: (config parameter name, config parameter value) relevant only for this load * operation * @return An executable network object */ - ExecutableNetwork compile_model(const std::shared_ptr& network, + ExecutableNetwork compile_model(const std::shared_ptr& model, const RemoteContext& context, const ConfigMap& config = {}); @@ -154,12 +152,14 @@ public: * @param extension Pointer to already loaded extension */ OPENVINO_DEPRECATED("Please use add_extension(ov::Extension) or add_extension(path_to_library) instead.") - void add_extension(const std::shared_ptr& extension); + void add_extension(const std::shared_ptr& extension); + /** * @brief Registers extension * @param library_path path to library with ov::Extension */ void add_extension(const std::string& library_path); + #ifdef OPENVINO_ENABLE_UNICODE_PATH_SUPPORT /** * @brief Registers extension @@ -167,11 +167,13 @@ public: */ void add_extension(const std::wstring& library_path); #endif + /** * @brief Registers extension * @param extension Pointer to base extension */ void add_extension(const std::shared_ptr& extension); + /** * @brief Registers extensions * @param extensions Vector of loaded base extensions @@ -225,78 +227,78 @@ public: } /** - * @brief Creates an executable network from a previously exported network - * @param networkModel network model stream - * @param deviceName Name of device load executable network on + * @brief Creates an executable network from a previously exported one + * @param model_stream Model stream + * @param device_name Name of device load executable network on * @param config Optional map of pairs: (config parameter name, config parameter value) relevant only for this load * operation* * @return An executable network reference */ - ExecutableNetwork import_model(std::istream& networkModel, - const std::string& deviceName, + ExecutableNetwork import_model(std::istream& model_stream, + const std::string& device_name, const ConfigMap& config = {}); /** - * @brief Creates an executable network from a previously exported network within a specified + * @brief Creates an executable network from a previously exported one within a specified * remote context. * - * @param networkModel Network model stream + * @param model_stream Model stream * @param context Pointer to RemoteContext object * @param config Optional map of pairs: (config parameter name, config parameter value) relevant only for this load * operation * @return An executable network reference */ - ExecutableNetwork import_model(std::istream& networkModel, + ExecutableNetwork import_model(std::istream& model_stream, const RemoteContext& context, const ConfigMap& config = {}); /** - * @brief Query device if it supports specified network with specified configuration + * @brief Query device if it supports specified model with specified configuration * - * @param deviceName A name of a device to query - * @param network Network object to query + * @param device_name A name of a device to query + * @param model Model object to query * @param config Optional map of pairs: (config parameter name, config parameter value) * @return An object containing a map of pairs a operation name -> a device name supporting this operation. */ - SupportedOpsMap query_model(const std::shared_ptr& network, - const std::string& deviceName, + SupportedOpsMap query_model(const std::shared_ptr& model, + const std::string& device_name, const ConfigMap& config = {}) const; /** * @brief Sets configuration for device, acceptable keys can be found in ie_plugin_config.hpp * - * @param deviceName An optional name of a device. If device name is not specified, the config is set for all the + * @param device_name An optional name of a device. If device name is not specified, the config is set for all the * registered devices. * * @param config Map of pairs: (config parameter name, config parameter value) */ - void set_config(const ConfigMap& config, const std::string& deviceName = {}); + void set_config(const ConfigMap& config, const std::string& device_name = {}); /** * @brief Gets configuration dedicated to device behaviour. * * The method is targeted to extract information which can be set via set_config method. * - * @param deviceName - A name of a device to get a configuration value. + * @param device_name - A name of a device to get a configuration value. * @param name - config key. * @return Value of config corresponding to config key. */ - Any get_config(const std::string& deviceName, const std::string& name) const; + Any get_config(const std::string& device_name, const std::string& name) const; /** * @brief Gets general runtime metric for dedicated hardware. * - * The method is needed to request common device properties - * which are executable network agnostic. It can be device name, temperature, other devices-specific values. + * The method is needed to request common device properties. + * It can be device name, temperature, other devices-specific values. * - * @param deviceName - A name of a device to get a metric value. + * @param device_name - A name of a device to get a metric value. * @param name - metric name to request. * @return Metric value corresponding to metric key. */ - Any get_metric(const std::string& deviceName, const std::string& name) const; + Any get_metric(const std::string& device_name, const std::string& name) const; /** - * @brief Returns devices available for neural networks inference + * @brief Returns devices available for inference * * @return A vector of devices. The devices are returned as { CPU, GPU.0, GPU.1, MYRIAD } * If there more than one device of specific type, they are enumerated with .# suffix. @@ -306,22 +308,22 @@ public: /** * @brief Register new device and plugin which implement this device inside OpenVINO Runtime. * - * @param pluginName A name of plugin. Depending on platform pluginName is wrapped with shared library suffix and - * prefix to identify library full name + * @param plugin_name A name of plugin. Depending on platform `plugin_name` is wrapped with shared library suffix + * and prefix to identify library full name * - * @param deviceName A device name to register plugin for. If device name is not specified, then it's taken from + * @param device_name A device name to register plugin for. If device name is not specified, then it's taken from * plugin itself. */ - void register_plugin(const std::string& pluginName, const std::string& deviceName); + void register_plugin(const std::string& plugin_name, const std::string& device_name); /** * @brief Unloads previously loaded plugin with a specified name from OpenVINO Runtime * The method is needed to remove plugin instance and free its resources. If plugin for a * specified device has not been created before, the method throws an exception. * - * @param deviceName Device name identifying plugin to remove from OpenVINO Runtime + * @param device_name Device name identifying plugin to remove from OpenVINO Runtime */ - void unload_plugin(const std::string& deviceName); + void unload_plugin(const std::string& device_name); /** @brief Registers plugin to OpenVINO Runtime Core instance using XML configuration file with * plugins description. @@ -349,25 +351,25 @@ public: * - Properties are set to plugin via the `set_config` method. * - Extensions are set to plugin via the `add_extension` method. * - * @param xmlConfigFile A path to .xml file with plugins to register. + * @param xml_config_file A path to .xml file with plugins to register. */ - void register_plugins(const std::string& xmlConfigFile); + void register_plugins(const std::string& xml_config_file); /** * @brief Create a new shared context object on specified accelerator device * using specified plugin-specific low level device API parameters (device handle, pointer, etc.) - * @param deviceName Name of a device to create new shared context on. + * @param device_name Name of a device to create new shared context on. * @param params Map of device-specific shared context parameters. * @return A shared pointer to a created remote context. */ - RemoteContext create_context(const std::string& deviceName, const ParamMap& params); + RemoteContext create_context(const std::string& device_name, const ParamMap& params); /** * @brief Get a pointer to default(plugin-supplied) shared context object for specified accelerator device. - * @param deviceName - A name of a device to get create shared context from. + * @param device_name - A name of a device to get create shared context from. * @return A shared pointer to a default remote context. */ - RemoteContext get_default_context(const std::string& deviceName); + RemoteContext get_default_context(const std::string& device_name); }; } // namespace runtime } // namespace ov diff --git a/src/inference/include/openvino/runtime/executable_network.hpp b/src/inference/include/openvino/runtime/executable_network.hpp index 7c966434445..76cc0867717 100644 --- a/src/inference/include/openvino/runtime/executable_network.hpp +++ b/src/inference/include/openvino/runtime/executable_network.hpp @@ -34,7 +34,7 @@ class Core; */ class OPENVINO_RUNTIME_API ExecutableNetwork { std::shared_ptr _so; - std::shared_ptr _impl; + std::shared_ptr _impl; /** * @brief Constructs ExecutableNetwork from the initialized std::shared_ptr @@ -42,7 +42,8 @@ class OPENVINO_RUNTIME_API ExecutableNetwork { * object is destroyed. * @param impl Initialized shared pointer */ - ExecutableNetwork(const std::shared_ptr& so, const std::shared_ptr& impl); + ExecutableNetwork(const std::shared_ptr& so, + const std::shared_ptr& impl); friend class ov::runtime::Core; public: @@ -64,12 +65,14 @@ public: * @return vector of inputs */ std::vector> inputs() const; + /** * @brief Get input of executable graph function * * @return Function input or throw ov::Exception in case of several outputs */ ov::Output input() const; + /** * @brief Get input of executable graph function * @@ -77,6 +80,7 @@ public: * @return Function input or throw ov::Exception if input wasn't found */ ov::Output input(size_t i) const; + /** * @brief Get input of executable graph function * @@ -91,12 +95,14 @@ public: * @return vector of outputs */ std::vector> outputs() const; + /** * @brief Get output of executable graph function * * @return Function output or throw ov::Exception in case of several outputs */ ov::Output output() const; + /** * @brief Get output of executable graph function * @@ -104,6 +110,7 @@ public: * @return Function output or throw ov::Exception if output wasn't found */ ov::Output output(size_t i) const; + /** * @brief Get output of executable graph function * @@ -113,7 +120,7 @@ public: ov::Output output(const std::string& tensor_name) const; /** - * @brief Creates an inference request object used to infer the network. + * @brief Creates an inference request object used to infer the model. * * The created request has allocated input and output blobs (that can be changed later). * @@ -124,11 +131,11 @@ public: /** * @brief Exports the current executable network. * - * @see Core::ImportNetwork + * @see Core::import_model * - * @param networkModel Network model output stream + * @param model_stream Model output stream */ - void export_model(std::ostream& networkModel); + void export_model(std::ostream& model_stream); /** * @brief Sets configuration for current executable network @@ -153,7 +160,7 @@ public: /** * @brief Gets general runtime metric for an executable network. * - * It can be network name, actual device ID on + * It can be model name, actual device ID on * which executable network is running or all other properties which cannot be changed dynamically. * * @param name metric name to request diff --git a/src/inference/include/openvino/runtime/infer_request.hpp b/src/inference/include/openvino/runtime/infer_request.hpp index 665508b290f..3a2ade5163c 100644 --- a/src/inference/include/openvino/runtime/infer_request.hpp +++ b/src/inference/include/openvino/runtime/infer_request.hpp @@ -35,7 +35,7 @@ class ExecutableNetwork; */ class OPENVINO_RUNTIME_API InferRequest { std::shared_ptr _so; - std::shared_ptr _impl; + std::shared_ptr _impl; /** * @brief Constructs InferRequest from the initialized std::shared_ptr @@ -43,7 +43,7 @@ class OPENVINO_RUNTIME_API InferRequest { * destroyed. * @param impl Initialized shared pointer */ - InferRequest(const std::shared_ptr& so, const std::shared_ptr& impl); + InferRequest(const std::shared_ptr& so, const std::shared_ptr& impl); friend class ov::runtime::ExecutableNetwork; public: @@ -56,31 +56,34 @@ public: * @brief Sets input/output data to infer * * @param name Name of input or output tensor. - * @param tensor Reference to input or output tensor. The type of a tensor must match the network input/output + * @param tensor Reference to input or output tensor. The type of a tensor must match the model input/output * precision and size. */ + void set_tensor(const std::string& name, const Tensor& tensor); /** * @brief Sets input/output data to infer * * @param port Port of input or output tensor. - * @param tensor Reference to input or output tensor. The type of a tensor must match the network input/output + * @param tensor Reference to input or output tensor. The type of a tensor must match the model input/output * precision and size. */ + void set_tensor(const ov::Output& port, const Tensor& tensor); /** * @brief Sets input/output data to infer * * @param port Port of input or output tensor. - * @param tensor Reference to input or output tensor. The type of a tensor must match the network input/output + * @param tensor Reference to input or output tensor. The type of a tensor must match the model input/output * precision and size. */ void set_tensor(const ov::Output& port, const Tensor& tensor); + /** * @brief Sets input tensor to infer * * @param idx Index of input tensor. - * @param tensor Reference to input tensor. The type of a tensor must match the network input precision and size. + * @param tensor Reference to input tensor. The type of a tensor must match the model input precision and size. */ void set_input_tensor(size_t idx, const Tensor& tensor); /** @@ -88,13 +91,15 @@ public: * * @param tensor Reference to input tensor. If model has several inputs, an exception is thrown. */ + void set_input_tensor(const Tensor& tensor); /** * @brief Sets output tensor to infer * * @param idx Index of output tensor. - * @param tensor Reference to output tensor. The type of a tensor must match the network output precision and size. + * @param tensor Reference to output tensor. The type of a tensor must match the model output precision and size. */ + void set_output_tensor(size_t idx, const Tensor& tensor); /** * @brief Sets output tensor to infer models with single output @@ -109,6 +114,7 @@ public: * @param name A name of tensor to get * @return A Tensor with a name @p name. If a tensor is not found, an exception is thrown. */ + Tensor get_tensor(const std::string& name); /** * @brief Gets input/output tensor for inference @@ -116,6 +122,7 @@ public: * @param port Port of tensor to get * @return A Tensor for the port @p port. If a tensor with specified @p port is not found, an exception is thrown. */ + Tensor get_tensor(const ov::Output& port); /** * @brief Gets input/output tensor for inference @@ -123,6 +130,7 @@ public: * @param port Port of tensor to get * @return A Tensor for the port @p port. If a tensor with specified @p port is not found, an exception is thrown. */ + Tensor get_tensor(const ov::Output& port); /** * @brief Gets input tensor for inference @@ -131,6 +139,7 @@ public: * @return A Tensor with an input index @p idx. If a tensor with specified @p idx is not found, an exception is * thrown. */ + Tensor get_input_tensor(size_t idx); /** * @brief Gets input tensor for inference @@ -138,6 +147,7 @@ public: * @return An input Tensor for the model. If model has several inputs, an exception is thrown. */ Tensor get_input_tensor(); + /** * @brief Gets output tensor for inference * @@ -146,6 +156,7 @@ public: * thrown. */ Tensor get_output_tensor(size_t idx); + /** * @brief Gets output tensor for inference * @@ -170,7 +181,7 @@ public: * @brief Queries performance measures per layer to get feedback of what is the most time consuming layer * * @note not all plugins provide meaningful data - * @return Vector of profiling information for layers in network + * @return Vector of profiling information for operations in model */ std::vector get_profiling_info() const; @@ -206,7 +217,7 @@ public: /** * @brief Gets state control interface for given infer request. * - * State control essential for recurrent networks + * State control essential for recurrent modells * @return A vector of Memory State objects */ std::vector query_state(); diff --git a/src/inference/include/openvino/runtime/remote_context.hpp b/src/inference/include/openvino/runtime/remote_context.hpp index 73b27110bb3..e8176d3e117 100644 --- a/src/inference/include/openvino/runtime/remote_context.hpp +++ b/src/inference/include/openvino/runtime/remote_context.hpp @@ -37,8 +37,8 @@ class ExecutableNetwork; */ class OPENVINO_RUNTIME_API RemoteContext { protected: - std::shared_ptr _so; //!< Reference to shared object that loaded implementation - std::shared_ptr _impl; //!< Pointer to remote context implementation + std::shared_ptr _so; //!< Reference to shared object that loaded implementation + std::shared_ptr _impl; //!< Pointer to remote context implementation /** * @brief Constructs RemoteContext from the initialized std::shared_ptr @@ -46,7 +46,7 @@ protected: * object is destroyed. * @param impl Initialized shared pointer */ - RemoteContext(const std::shared_ptr& so, const std::shared_ptr& impl); + RemoteContext(const std::shared_ptr& so, const std::shared_ptr& impl); friend class ov::runtime::Core; friend class ov::runtime::ExecutableNetwork; diff --git a/src/inference/include/openvino/runtime/remote_tensor.hpp b/src/inference/include/openvino/runtime/remote_tensor.hpp index b0f60afe2e5..a4b7337d4e1 100644 --- a/src/inference/include/openvino/runtime/remote_tensor.hpp +++ b/src/inference/include/openvino/runtime/remote_tensor.hpp @@ -58,5 +58,6 @@ public: */ std::string get_device_name() const; }; + } // namespace runtime } // namespace ov \ No newline at end of file diff --git a/src/inference/include/openvino/runtime/variable_state.hpp b/src/inference/include/openvino/runtime/variable_state.hpp index dc065671db0..4cbe637cdb4 100644 --- a/src/inference/include/openvino/runtime/variable_state.hpp +++ b/src/inference/include/openvino/runtime/variable_state.hpp @@ -30,7 +30,7 @@ class InferRequest; */ class OPENVINO_RUNTIME_API VariableState { std::shared_ptr _so; - std::shared_ptr _impl; + std::shared_ptr _impl; /** * @brief Constructs VariableState from the initialized std::shared_ptr @@ -38,7 +38,8 @@ class OPENVINO_RUNTIME_API VariableState { * @param so Optional: Plugin to use. This is required to ensure that VariableState can work properly even if plugin * object is destroyed. */ - VariableState(const std::shared_ptr& so, const std::shared_ptr& impl); + VariableState(const std::shared_ptr& so, + const std::shared_ptr& impl); friend class ov::runtime::InferRequest; diff --git a/src/inference/src/cnn_network_ngraph_impl.cpp b/src/inference/src/cnn_network_ngraph_impl.cpp index 69741ad39bf..4a704d2f600 100644 --- a/src/inference/src/cnn_network_ngraph_impl.cpp +++ b/src/inference/src/cnn_network_ngraph_impl.cpp @@ -24,6 +24,7 @@ #include "ngraph/pass/manager.hpp" #include "openvino/core/except.hpp" #include "openvino/pass/serialize.hpp" +#include "transformations/fix_rt_info.hpp" #include "transformations/smart_reshape/set_batch_size.hpp" #include "transformations/smart_reshape/smart_reshape.hpp" #include "transformations/utils/utils.hpp" @@ -132,6 +133,11 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(const std::shared_ptr& nGra : _ngraph_function(nGraph), _ie_extensions(exts), _new_api(newAPI) { + { + ov::pass::Manager m; + m.register_pass(); + m.run_passes(_ngraph_function); + } // Restore usual attributes for CNNNetwork auto keep_input_info = [=](CNNNetworkNGraphImpl& network, const DataPtr& inData) { InputInfo::Ptr info(new InputInfo()); @@ -188,6 +194,11 @@ CNNNetworkNGraphImpl::CNNNetworkNGraphImpl(const CNNNetwork& network) { } _ngraph_function = ngraph::clone_function(*network.getFunction()); + { + ov::pass::Manager m; + m.register_pass(); + m.run_passes(_ngraph_function); + } validateFunctionNames(); InputsDataMap inputs = network.getInputsInfo(); OutputsDataMap outputs = network.getOutputsInfo(); @@ -427,9 +438,8 @@ void CNNNetworkNGraphImpl::reshape(const std::map(); manager.register_pass(); manager.register_pass<::ngraph::pass::ConvertNMS5ToLegacyMatcher>(false); - // TODO [DS NMS]: remove when nodes from models where nms is not last node in model supports DS - manager.register_pass<::ngraph::pass::ConvertMulticlassNmsToMulticlassNmsIE>(false); - manager.register_pass<::ngraph::pass::ConvertMatrixNmsToMatrixNmsIE>(false); + manager.register_pass<::ngraph::pass::ConvertMulticlassNmsToMulticlassNmsIE>(); + manager.register_pass<::ngraph::pass::ConvertMatrixNmsToMatrixNmsIE>(); manager.register_pass<::ngraph::pass::DisableConvertConstantFoldingOnConstPath>(); manager.register_pass<::ngraph::pass::ConstantFolding>(); // OneHotToLegacy changes output precision diff --git a/src/inference/src/compilation_context.cpp b/src/inference/src/compilation_context.cpp index 1f22668c0da..c2228677fdd 100644 --- a/src/inference/src/compilation_context.cpp +++ b/src/inference/src/compilation_context.cpp @@ -19,6 +19,7 @@ #include "ngraph/opsets/opset6.hpp" #include "ngraph/variant.hpp" #include "openvino/pass/manager.hpp" +#include "transformations/fix_rt_info.hpp" #include "transformations/hash.hpp" #include "transformations/rt_info/fused_names_attribute.hpp" #include "transformations/rt_info/primitives_priority_attribute.hpp" @@ -72,6 +73,7 @@ std::string NetworkCompilationContext::computeHash(const CNNNetwork& network, // 1. Calculate hash on function CNNNetwork net(network); ov::pass::Manager m; + m.register_pass(); m.register_pass(seed); m.run_passes(net.getFunction()); @@ -85,18 +87,9 @@ std::string NetworkCompilationContext::computeHash(const CNNNetwork& network, const auto& rt = op->get_rt_info(); for (const auto& rtMapData : rt) { seed = hash_combine(seed, rtMapData.first); - - if (auto stringData = std::dynamic_pointer_cast>(rtMapData.second)) { - seed = hash_combine(seed, stringData->get()); - } else if (auto intData = - std::dynamic_pointer_cast>(rtMapData.second)) { - seed = hash_combine(seed, intData->get()); - } else if (auto fNames = - std::dynamic_pointer_cast>(rtMapData.second)) { - seed = hash_combine(seed, fNames->get().getNames()); - } else if (auto prim = std::dynamic_pointer_cast(rtMapData.second)) { - seed = hash_combine(seed, prim->get()); - } + std::stringstream strm; + rtMapData.second.print(strm); + seed = hash_combine(seed, strm.str()); } } diff --git a/src/inference/src/cpp_interfaces/interface/ie_iplugin_internal.cpp b/src/inference/src/cpp_interfaces/interface/ie_iplugin_internal.cpp index 1d0469e1ba1..0ffa5e1e1b2 100644 --- a/src/inference/src/cpp_interfaces/interface/ie_iplugin_internal.cpp +++ b/src/inference/src/cpp_interfaces/interface/ie_iplugin_internal.cpp @@ -29,7 +29,7 @@ #include "openvino/core/deprecated.hpp" #include "openvino/core/except.hpp" #include "openvino/core/function.hpp" -#include "openvino/core/variant.hpp" +#include "openvino/core/runtime_attribute.hpp" #include "transformations/utils/utils.hpp" namespace InferenceEngine { @@ -296,7 +296,7 @@ void IInferencePlugin::SetExeNetworkInfo(const std::shared_ptrget_rt_info(); const auto it = rt_info.find("version"); if (it != rt_info.end()) { - auto ir_version_impl = std::dynamic_pointer_cast>(it->second); + auto ir_version_impl = std::dynamic_pointer_cast>(it->second); OPENVINO_ASSERT(ir_version_impl != nullptr, "Failed to extract IR version from 'version' attribute"); const int64_t ir_version = ir_version_impl->get(); // here we decide whether we need to add operation_names as tensor names for diff --git a/src/inference/src/ie_core.cpp b/src/inference/src/ie_core.cpp index ebfc8ed0bd8..9f6ef5bebef 100644 --- a/src/inference/src/ie_core.cpp +++ b/src/inference/src/ie_core.cpp @@ -204,7 +204,7 @@ class CoreImpl : public ie::ICore, public std::enable_shared_from_this supportedMetricKeys = plugin.get_metric(METRIC_KEY(SUPPORTED_METRICS), {}); auto it = std::find(supportedMetricKeys.begin(), supportedMetricKeys.end(), METRIC_KEY(IMPORT_EXPORT_SUPPORT)); - bool supported = + auto supported = (it != supportedMetricKeys.end()) && plugin.get_metric(METRIC_KEY(IMPORT_EXPORT_SUPPORT), {}).as(); return supported; } diff --git a/src/inference/src/ie_network_reader.cpp b/src/inference/src/ie_network_reader.cpp index 3d2dd117128..c4ca40e531b 100644 --- a/src/inference/src/ie_network_reader.cpp +++ b/src/inference/src/ie_network_reader.cpp @@ -301,7 +301,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, using namespace ov::preprocess; PrePostProcessor prepost(function); - auto ir_version_impl = std::dynamic_pointer_cast>(it->second); + auto ir_version_impl = it->second.as>>(); OPENVINO_ASSERT(ir_version_impl != nullptr, "Failed to extract IR version from 'version' attribute"); const int64_t ir_version = ir_version_impl->get(); @@ -347,7 +347,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, function = prepost.build(); // Set version to 10 - rt_info["version"] = std::make_shared>(10); + rt_info["version"] = std::make_shared>(10); } else if (ir_version == 11 && !newAPI) { const std::string& old_api_map_key_order = ov::OldApiMapOrder::get_type_info_static(); const std::string& old_api_map_key_type = ov::OldApiMapElementType::get_type_info_static(); @@ -359,9 +359,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, const auto it_type = rtInfo.find(old_api_map_key_type); auto& pre_input = prepost.input(i); if (it_type != rtInfo.end()) { - const auto old_api_map_attr = std::dynamic_pointer_cast(it_type->second); - OPENVINO_ASSERT(old_api_map_attr != nullptr, "Failed to cast to ov::OldApiMapElementType"); - const auto type = old_api_map_attr->get(); + auto type = it_type->second.as().value; pre_input.tensor().set_element_type(type); OPENVINO_ASSERT(!type.is_dynamic(), "Old API map does not support dynamic type"); @@ -369,9 +367,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, } const auto it_order = rtInfo.find(old_api_map_key_order); if (it_order != rtInfo.end()) { - const auto old_api_map_attr = std::dynamic_pointer_cast(it_order->second); - OPENVINO_ASSERT(old_api_map_attr != nullptr, "Failed to cast to ov::OldApiMapOrder"); - const auto order = old_api_map_attr->get(); + const auto order = it_order->second.as().value; pre_input.preprocess().convert_layout(order); rtInfo.erase(it_order); } @@ -385,9 +381,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, if (it == rtInfo.end()) continue; - const auto old_api_map_attr = std::dynamic_pointer_cast(it->second); - OPENVINO_ASSERT(old_api_map_attr != nullptr, "Failed to cast to ov::OldApiMapOrder"); - const auto order = old_api_map_attr->get(); + const auto order = it->second.as().value; auto& post_output = prepost.output(i); post_output.postprocess().convert_layout(order); @@ -396,7 +390,7 @@ CNNNetwork convert_to_cnnnetwork(std::shared_ptr& function, } // Set version to 10 - rt_info["version"] = std::make_shared>(10); + rt_info["version"] = std::make_shared>(10); function = prepost.build(); } @@ -459,7 +453,7 @@ CNNNetwork details::ReadNetwork(const std::string& modelPath, ov::frontend::FrontEnd::Ptr FE; ov::frontend::InputModel::Ptr inputModel; - ov::VariantVector params{ov::make_variant(model_path)}; + ov::RuntimeAttributeVector params{model_path}; if (!binPath.empty()) { #if defined(OPENVINO_ENABLE_UNICODE_PATH_SUPPORT) && defined(_WIN32) @@ -467,7 +461,7 @@ CNNNetwork details::ReadNetwork(const std::string& modelPath, #else const std::string& weights_path = binPath; #endif - params.emplace_back(ov::make_variant(weights_path)); + params.emplace_back(weights_path); } FE = manager.load_by_model(params); @@ -525,12 +519,12 @@ CNNNetwork details::ReadNetwork(const std::string& model, ov::frontend::FrontEnd::Ptr FE; ov::frontend::InputModel::Ptr inputModel; - ov::VariantVector params{ov::make_variant(&modelStream)}; + ov::RuntimeAttributeVector params{&modelStream}; if (weights) { char* data = weights->cbuffer().as(); std::shared_ptr weights_buffer = std::make_shared>(data, weights->byteSize(), weights); - params.emplace_back(ov::make_variant(weights_buffer)); + params.emplace_back(weights_buffer); } FE = manager.load_by_model(params); diff --git a/src/plugins/auto/CMakeLists.txt b/src/plugins/auto/CMakeLists.txt index f693139be8d..b144bfb4866 100644 --- a/src/plugins/auto/CMakeLists.txt +++ b/src/plugins/auto/CMakeLists.txt @@ -6,15 +6,28 @@ set (TARGET_NAME "ov_auto_plugin") file(GLOB SOURCES ${CMAKE_CURRENT_SOURCE_DIR}/*.cpp ${CMAKE_CURRENT_SOURCE_DIR}/utils/*.cpp) -ie_add_plugin(NAME ${TARGET_NAME} - DEVICE_NAME "MULTI" - SOURCES ${SOURCES} - VERSION_DEFINES_FOR plugin.cpp) +if(ENABLE_AUTO AND ENABLE_MULTI) + ie_add_plugin(NAME ${TARGET_NAME} + DEVICE_NAME "MULTI" + SOURCES ${SOURCES} + VERSION_DEFINES_FOR plugin.cpp) -ie_add_plugin(NAME ${TARGET_NAME} - DEVICE_NAME "AUTO" - PSEUDO_PLUGIN_FOR "MULTI" - DEFAULT_CONFIG "MULTI_WORK_MODE_AS_AUTO:YES") + ie_add_plugin(NAME ${TARGET_NAME} + DEVICE_NAME "AUTO" + PSEUDO_PLUGIN_FOR "MULTI" + DEFAULT_CONFIG "MULTI_WORK_MODE_AS_AUTO:YES") +elseif(ENABLE_AUTO) + ie_add_plugin(NAME ${TARGET_NAME} + DEVICE_NAME "AUTO" + SOURCES ${SOURCES} + VERSION_DEFINES_FOR plugin.cpp + DEFAULT_CONFIG "MULTI_WORK_MODE_AS_AUTO:YES") +elseif(ENABLE_MULTI) + ie_add_plugin(NAME ${TARGET_NAME} + DEVICE_NAME "MULTI" + SOURCES ${SOURCES} + VERSION_DEFINES_FOR plugin.cpp) +endif() target_link_libraries(${TARGET_NAME} PRIVATE ngraph inference_engine_transformations) diff --git a/src/plugins/auto/executable_network.cpp b/src/plugins/auto/executable_network.cpp index 616ca12d95e..a7b435c30d4 100644 --- a/src/plugins/auto/executable_network.cpp +++ b/src/plugins/auto/executable_network.cpp @@ -93,8 +93,14 @@ MultiDeviceExecutableNetwork::MultiDeviceExecutableNetwork(const DeviceMapGetMetric(METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)).as(); @@ -181,13 +187,15 @@ MultiDeviceExecutableNetwork::MultiDeviceExecutableNetwork(const std::string& if (CPUIter != metaDevices.end()) { _loadContext[CPU].isEnabled = true; _loadContext[CPU].deviceInfo = *CPUIter; + _loadContext[CPU].deviceInfo.config[CONFIG_KEY(PERFORMANCE_HINT)] = + InferenceEngine::PluginConfigParams::LATENCY; + _loadContext[CPU].workName = "CPU_HELP"; LOG_INFO("[AUTOPLUGIN]:will load CPU for accelerator"); } else { _loadContext[CPU].isEnabled = false; } } - // initialize the rest members of load context for (int i = 0; i < CONTEXTNUM; i++) { if (_loadContext[i].isEnabled) { @@ -196,7 +204,10 @@ MultiDeviceExecutableNetwork::MultiDeviceExecutableNetwork(const std::string& _loadContext[i].task = [this, contextPtr, modelPath, network]() mutable { TryToLoadNetWork(*contextPtr, modelPath, network); if (contextPtr->isLoadSuccess) { - GenerateWorkers(contextPtr->deviceInfo.deviceName, contextPtr->executableNetwork); + if (contextPtr->workName.empty()) { + contextPtr->workName = contextPtr->deviceInfo.deviceName; + } + GenerateWorkers(contextPtr->workName, contextPtr->executableNetwork); //need lock { std::lock_guard lock(_confMutex); @@ -246,6 +257,9 @@ MultiDeviceExecutableNetwork::MultiDeviceExecutableNetwork(const std::string& _workerRequests[device.deviceName]; _inferPipelineTasksDeviceSpecific[device.deviceName] = nullptr; } + _idleWorkerRequests["CPU_HELP"]; + _workerRequests["CPU_HELP"]; + _inferPipelineTasksDeviceSpecific["CPU_HELP"] = nullptr; _executor->run(_loadContext[CPU].task); _executor->run(_loadContext[ACTUALDEVICE].task); } else { @@ -302,10 +316,30 @@ void MultiDeviceExecutableNetwork::TryToLoadNetWork(AutoLoadContext& context, return; } - // if selec device is CPU, do not need to load CPU again, context[CPU] must have loaded CPU + // if the select device is CPU, need to check the config of _loadContext[CPU] + // if they are same, do not need to load again curDevIsCPU = (context.deviceInfo.deviceName.find("CPU") != std::string::npos); if (curDevIsCPU) { - return; + auto compare = [](std::map& a, + std::map& b) -> bool { + if (a.size() != b.size()) { + return false; + } + for (auto& item : a) { + auto bIter = b.find(item.first); + if (bIter != b.end()) { + if (bIter->second != item.second) { + return false; + } + } else { + return false; + } + } + return true; + }; + if (compare(context.deviceInfo.config, _loadContext[CPU].deviceInfo.config)) { + return; + } } LOG_DEBUG("[AUTOPLUGIN] try to load %s", context.deviceInfo.deviceName.c_str()); @@ -381,7 +415,11 @@ void MultiDeviceExecutableNetwork::ScheduleToWorkerInferRequest(Task inferPipeli if (_loadContext[ACTUALDEVICE].isAlready) { devices.push_back(_loadContext[ACTUALDEVICE].deviceInfo); } else { - devices.push_back(_loadContext[CPU].deviceInfo); + // replace deviceName with workName, so schedule can select correct + // idleWorkerQueue + auto deviceInfo = _loadContext[CPU].deviceInfo; + deviceInfo.deviceName = _loadContext[CPU].workName; + devices.push_back(std::move(deviceInfo)); } } } else { @@ -613,7 +651,19 @@ InferenceEngine::Parameter MultiDeviceExecutableNetwork::GetConfig(const std::st InferenceEngine::Parameter MultiDeviceExecutableNetwork::GetMetric(const std::string &name) const { if (_workModeIsAUTO) { - // fixme: should we wait actual device? meanwhile it will block inference, how to fix? + if (name == METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)) { + unsigned int real = 0; + if (_loadContext[ACTUALDEVICE].isAlready) { + real = _loadContext[ACTUALDEVICE]. + executableNetwork->GetMetric(name).as(); + } else { + real = _loadContext[CPU]. + executableNetwork->GetMetric(name).as(); + } + unsigned int res = std::max(8u, real); + IE_SET_METRIC_RETURN(OPTIMAL_NUMBER_OF_INFER_REQUESTS, res); + } + if (_loadContext[ACTUALDEVICE].isAlready) { return _loadContext[ACTUALDEVICE].executableNetwork->GetMetric(name); } diff --git a/src/plugins/auto/executable_network.hpp b/src/plugins/auto/executable_network.hpp index f1c255db5cb..c0158f25aeb 100644 --- a/src/plugins/auto/executable_network.hpp +++ b/src/plugins/auto/executable_network.hpp @@ -52,6 +52,11 @@ struct AutoLoadContext { std::string networkPrecision; std::string errMessage; InferenceEngine::Task task; + //ACTUALDEVICE's workName is same as it's deviceName, + //CPU_HELP's workName is "CPU_HELP", and deviceName is "CPU" + //add workName is because of ACTUALDEVICE and CPU maybe all CPU, + //they can't use the same workerQueue + std::string workName = ""; }; enum AutoLoadContextIndex { diff --git a/src/plugins/hetero/executable_network.cpp b/src/plugins/hetero/executable_network.cpp index c79090d9c12..61586fbf7b3 100644 --- a/src/plugins/hetero/executable_network.cpp +++ b/src/plugins/hetero/executable_network.cpp @@ -82,10 +82,8 @@ HeteroExecutableNetwork::HeteroExecutableNetwork(const InferenceEngine::CNNNetwo auto& nodeInfo = node->get_rt_info(); auto itInfo = nodeInfo.find("affinity"); if (itInfo != nodeInfo.end()) { - IE_ASSERT((ngraph::is_type>(itInfo->second))); - queryNetworkResult.supportedLayersMap.emplace( - node->get_friendly_name(), - ngraph::as_type_ptr>(itInfo->second)->get()); + IE_ASSERT(itInfo->second.is()); + queryNetworkResult.supportedLayersMap.emplace(node->get_friendly_name(), itInfo->second.as()); allEmpty = false; } } diff --git a/src/plugins/intel_gpu/include/intel_gpu/plugin/cldnn_primitives_list.hpp b/src/plugins/intel_gpu/include/intel_gpu/plugin/cldnn_primitives_list.hpp index 96246504c79..15d7df5fd98 100644 --- a/src/plugins/intel_gpu/include/intel_gpu/plugin/cldnn_primitives_list.hpp +++ b/src/plugins/intel_gpu/include/intel_gpu/plugin/cldnn_primitives_list.hpp @@ -166,7 +166,7 @@ REGISTER_FACTORY(v3, ScatterNDUpdate); // REGISTER_FACTORY(v3, Bucketize); // REGISTER_FACTORY(v3, GRUCell); // REGISTER_FACTORY(v3, NonZero); -// REGISTER_FACTORY(v3, ROIAlign); + REGISTER_FACTORY(v3, ROIAlign); // REGISTER_FACTORY(v3, ReadValue); // REGISTER_FACTORY(v3, ShapeOf); // REGISTER_FACTORY(v3, TopK); @@ -205,6 +205,7 @@ REGISTER_FACTORY(v5, Loop); REGISTER_FACTORY(v6, CTCGreedyDecoderSeqLen); REGISTER_FACTORY(v6, MVN); REGISTER_FACTORY(v6, GatherElements); +REGISTER_FACTORY(v6, ExperimentalDetectronROIFeatureExtractor); // ------------------------------ Supported v7 ops ------------------------------ // REGISTER_FACTORY(v7, Gather); diff --git a/src/plugins/intel_gpu/src/plugin/cldnn_infer_request.cpp b/src/plugins/intel_gpu/src/plugin/cldnn_infer_request.cpp index 77336141f00..c1f356c36d6 100644 --- a/src/plugins/intel_gpu/src/plugin/cldnn_infer_request.cpp +++ b/src/plugins/intel_gpu/src/plugin/cldnn_infer_request.cpp @@ -163,6 +163,14 @@ void checkOutputBlob(const Blob::Ptr &blob, checkAlloc(blob, str_output_not_allocated); } +bool same_host_mem(cldnn::memory::ptr memPtr, uint8_t* hostPtr) { + uint8_t* bufferMem = nullptr; + if (memPtr->get_allocation_type() == cldnn::allocation_type::usm_host) { + bufferMem = reinterpret_cast(memPtr->get_internal_params().mem); + } + return bufferMem == hostPtr; +} + } // namespace namespace CLDNNPlugin { @@ -562,7 +570,15 @@ void CLDNNInferRequest::wait() { // mapping remote blobs not needed - // let the user take care of them explicitly if (!bptr->is()) { - copy_output_data(outputMemory, bptr); + bool same_mem = false; + { + auto dst_lock = bptr->cbuffer(); + auto dst_ptr = dst_lock.as(); + same_mem = same_host_mem(outputMemory, dst_ptr); + } + if (!same_mem) { + copy_output_data(outputMemory, bptr); + } } } @@ -899,13 +915,14 @@ void CLDNNInferRequest::prepare_input(const cldnn::primitive_id& inputName, Blob if (inputLayoutItr == m_graph->GetInputLayouts().end()) { IE_THROW() << "Input name mismatch."; } - Blob::Ptr reqBlob = _deviceInputs.at(inputName); + auto reqBlob = _deviceInputs.at(inputName)->as(); auto _nw_ptr = m_graph->GetNetwork(); cldnn::primitive_id internalName = "parameter:" + inputName; const auto& prec = inputBlob->getTensorDesc().getPrecision(); auto remote_ptr = inputBlob->as(); auto& stream = m_graph->GetNetwork()->get_stream(); bool is_dev_input = remote_ptr != nullptr; + switch (prec) { case Precision::FP32: case Precision::FP16: @@ -918,7 +935,7 @@ void CLDNNInferRequest::prepare_input(const cldnn::primitive_id& inputName, Blob case Precision::I64: { auto impl = getBlobImpl(is_dev_input ? remote_ptr : - reqBlob->as()); + reqBlob); if (!impl->is_allocated()) { IE_THROW() << str_input_not_allocated; } @@ -936,8 +953,11 @@ void CLDNNInferRequest::prepare_input(const cldnn::primitive_id& inputName, Blob } } else { auto src_lock = inputBlob->cbuffer(); - auto ev = inputMem->copy_from(stream, src_lock.as()); - dependencies.push_back(ev); + auto src_ptr = src_lock.as(); + if (!same_host_mem(inputMem, src_ptr)) { + auto ev = inputMem->copy_from(stream, src_ptr); + dependencies.push_back(ev); + } } } _nw_ptr->set_input_data(internalName, inputMem); diff --git a/src/plugins/intel_gpu/src/plugin/cldnn_transformations_pipeline.cpp b/src/plugins/intel_gpu/src/plugin/cldnn_transformations_pipeline.cpp index 50a280ed2d6..f32d4be4383 100644 --- a/src/plugins/intel_gpu/src/plugin/cldnn_transformations_pipeline.cpp +++ b/src/plugins/intel_gpu/src/plugin/cldnn_transformations_pipeline.cpp @@ -359,8 +359,7 @@ void TransformationsPipeline::apply(std::shared_ptr func) { OperationPrecisionRestriction::create({ {0, {ngraph::element::u8, ngraph::element::i8}}, {1, {ngraph::element::i8}} - }), - OperationPrecisionRestriction::create({}) + }) }); auto perTensorQuantization = std::vector({ @@ -371,7 +370,6 @@ void TransformationsPipeline::apply(std::shared_ptr func) { ngraph::pass::Manager lptManager; auto lptPassConfig = lptManager.get_pass_config(); - lptPassConfig->disable(); lptPassConfig->set_callback([](const_node_ptr& node) -> bool { if (const auto mulitply = std::dynamic_pointer_cast(node)) { return !MultiplyToGroupConvolutionTransformation::canBeTransformedToGroupConvolution(mulitply); diff --git a/src/plugins/intel_gpu/src/plugin/ops/experimental_detectron_roi_feature_extractor.cpp b/src/plugins/intel_gpu/src/plugin/ops/experimental_detectron_roi_feature_extractor.cpp new file mode 100644 index 00000000000..0f008965ab3 --- /dev/null +++ b/src/plugins/intel_gpu/src/plugin/ops/experimental_detectron_roi_feature_extractor.cpp @@ -0,0 +1,57 @@ +// Copyright (C) 2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// + +#include "cldnn_program.h" +#include "cldnn_common_utils.h" + +#include "ngraph/op/experimental_detectron_roi_feature.hpp" + +#include "cldnn/primitives/mutable_data.hpp" +#include "cldnn/primitives/experimental_detectron_roi_feature_extractor.hpp" + +namespace CLDNNPlugin { + +static void CreateExperimentalDetectronROIFeatureExtractorOp(Program& p, const std::shared_ptr& op) { + auto inputPrimitives = p.GetInputPrimitiveIDs(op); + std::string layerName = layer_type_name_ID(op) + ".0"; + + cldnn::layout mutableLayout = cldnn::layout( + DataTypeFromPrecision(op->get_output_element_type(1)), + DefaultFormatForDims(op->get_output_shape(1).size()), + CldnnTensorFromIEDims(op->get_output_shape(1))); + + cldnn::memory::ptr shared_memory {p.GetEngine().allocate_memory(mutableLayout)}; + + cldnn::primitive_id experimental_detectron_mutable_id_w = layer_type_name_ID(op) + "_md_write"; + cldnn::mutable_data experimenta_detectron_mutable_prim(experimental_detectron_mutable_id_w, + shared_memory, + op->get_friendly_name()); + p.primitiveIDs[experimental_detectron_mutable_id_w] = experimental_detectron_mutable_id_w; + p.AddPrimitive(experimenta_detectron_mutable_prim); + inputPrimitives.push_back(experimental_detectron_mutable_id_w); + + const ov::op::v6::ExperimentalDetectronROIFeatureExtractor::Attributes& operation_attributes = op->get_attrs(); + + cldnn::experimental_detectron_roi_feature_extractor experimentalDetectronPrim(layerName, + inputPrimitives, + operation_attributes.output_size, + operation_attributes.pyramid_scales, + operation_attributes.sampling_ratio, + operation_attributes.aligned); + p.AddPrimitive(experimentalDetectronPrim); + + cldnn::primitive_id experimental_detectron_mutable_id_r = layer_type_name_ID(op) + ".1"; + cldnn::mutable_data experimental_detectron_mutable_prim_r(experimental_detectron_mutable_id_r, + {layerName}, + shared_memory, + op->get_friendly_name()); + p.primitiveIDs[experimental_detectron_mutable_id_r] = experimental_detectron_mutable_id_r; + p.AddPrimitive(experimental_detectron_mutable_prim_r); + + p.AddPrimitiveToProfiler(layerName, op); +} + +REGISTER_FACTORY_IMPL(v6, ExperimentalDetectronROIFeatureExtractor); + +} // namespace CLDNNPlugin diff --git a/src/plugins/intel_gpu/src/plugin/ops/roi_align.cpp b/src/plugins/intel_gpu/src/plugin/ops/roi_align.cpp new file mode 100644 index 00000000000..1eda818d148 --- /dev/null +++ b/src/plugins/intel_gpu/src/plugin/ops/roi_align.cpp @@ -0,0 +1,42 @@ +// Copyright (C) 2018-2021 Intel Corporation +// SPDX-License-Identifier: Apache-2.0 +// +#include "cldnn_program.h" +#include "cldnn_common_utils.h" +#include "ngraph/op/roi_align.hpp" +#include "cldnn/primitives/roi_align.hpp" +#include + +namespace CLDNNPlugin { + +namespace { + +cldnn::roi_align::PoolingMode from(ngraph::op::v3::ROIAlign::PoolingMode mode) { + switch (mode) { + case ngraph::op::v3::ROIAlign::PoolingMode::MAX: + return cldnn::roi_align::PoolingMode::Max; + case ngraph::op::v3::ROIAlign::PoolingMode::AVG: + default: + return cldnn::roi_align::PoolingMode::Avg; + } +} + +void CreateROIAlignOp(Program& p, const std::shared_ptr& op) { + p.ValidateInputs(op, { 3 }); + auto roi_align_prim = cldnn::roi_align(layer_type_name_ID(op), + p.GetInputPrimitiveIDs(op), + op->get_pooled_h(), + op->get_pooled_w(), + op->get_sampling_ratio(), + op->get_spatial_scale(), + from(op->get_mode()), + op->get_friendly_name()); + p.AddPrimitive(roi_align_prim); + p.AddPrimitiveToProfiler(op); +} + +} // anonymous namespace + +REGISTER_FACTORY_IMPL(v3, ROIAlign); + +} // namespace CLDNNPlugin diff --git a/tools/pot/openvino/tools/pot/algorithms/quantization/fast_bias_correction/algorithm.py b/tools/pot/openvino/tools/pot/algorithms/quantization/fast_bias_correction/algorithm.py index e0693af0a32..c096ded5c93 100755 --- a/tools/pot/openvino/tools/pot/algorithms/quantization/fast_bias_correction/algorithm.py +++ b/tools/pot/openvino/tools/pot/algorithms/quantization/fast_bias_correction/algorithm.py @@ -49,6 +49,7 @@ class FastBiasCorrection(Algorithm): :param model: model to apply algo :return with corrected biases for layers with bias """ + mu.nx_type_infer(model) activations_statistics = self._stats_collector.get_statistics_for_algorithm(self.name) nodes_with_bias = mu.get_nodes_by_type(model, [op['type'] for op in OPERATIONS_WITH_BIAS]) self.find_channel_axis(model) diff --git a/tools/pot/openvino/tools/pot/graph/builder.py b/tools/pot/openvino/tools/pot/graph/builder.py index 1a2adf03910..7256f59965d 100644 --- a/tools/pot/openvino/tools/pot/graph/builder.py +++ b/tools/pot/openvino/tools/pot/graph/builder.py @@ -2,11 +2,10 @@ # SPDX-License-Identifier: Apache-2.0 from copy import deepcopy -import numpy as np from mo.graph.graph import Graph -from openvino.tools.pot.graph.node_utils import get_node_input, get_node_inputs +from openvino.tools.pot.graph.node_utils import get_node_data_type, get_node_input, get_node_inputs from .editor import create_node, connect_nodes_by_name, get_node_by_name @@ -105,8 +104,10 @@ def build_graph_for_node(model, input_name, input_shape, node, remove_bias=False :param remove_fake_quantize: remove fake quantize nodes in the generated graph :return: generated graph. """ + input_data_type = get_node_data_type(node, 0) nodes, edges = [], [] - nodes.append((input_name, 'Parameter', {'name': input_name, 'shape': input_shape, 'type': 'Parameter'})) + nodes.append((input_name, 'Parameter', {'name': input_name, 'shape': input_shape, + 'type': 'Parameter', 'data_type': input_data_type})) node_attrs = deepcopy(node.attrs()) if node.has_valid('output') and node.has_valid('get_output_feature_dim'): @@ -146,6 +147,9 @@ def build_graph_for_node(model, input_name, input_shape, node, remove_bias=False weights_node = get_node_input(src_node, 1) weights_node = get_node_input(weights_node, 0) \ if weights_node.type == 'FakeQuantize' else weights_node - if weights_node.out_port(0).get_data_type() == np.float16: + weights_out_dtype = weights_node.out_port(0).get_data_type() + src_out_dtype = src_node.out_port(0).get_data_type() + if weights_out_dtype != src_out_dtype: weights_node.out_node(0)['Insert_Convert_operation_after'] = True + return graph diff --git a/tools/pot/openvino/tools/pot/graph/model_utils.py b/tools/pot/openvino/tools/pot/graph/model_utils.py index 1f29a66486b..dd5fc858681 100644 --- a/tools/pot/openvino/tools/pot/graph/model_utils.py +++ b/tools/pot/openvino/tools/pot/graph/model_utils.py @@ -3,6 +3,7 @@ import networkx as nx from mo.graph.graph import Node +from mo.middle.passes.infer import type_infer from . import editor as ge, builder as gb from .nx_model import NXModel @@ -127,3 +128,9 @@ def models_union(first_model, second_model): union_dict['model'].graph.update(model_1.graph) union_dict['model'].graph.update(model_2.graph) return union + +def nx_type_infer(model): + """ Apply type_infer for each model in NXModel wrapper + """ + for model_dict in model.models: + type_infer(model_dict['model']) diff --git a/tools/pot/openvino/tools/pot/graph/node_utils.py b/tools/pot/openvino/tools/pot/graph/node_utils.py index 51cae922f43..e3ce1099a42 100644 --- a/tools/pot/openvino/tools/pot/graph/node_utils.py +++ b/tools/pot/openvino/tools/pot/graph/node_utils.py @@ -267,10 +267,10 @@ def get_lstm_ends(read_value, assigns, ignore_nodes): return lstm_outputs -def get_node_data_type(node): - if node.type != 'Const' and node.in_port(0).get_source() is not None \ - and node.in_port(0).get_source().is_data_type_defined(): - return node.in_port(0).get_source().get_data_type() +def get_node_data_type(node, port_id=0): + if node.type != 'Const' and node.in_port(port_id).get_source() is not None \ + and node.in_port(port_id).get_source().is_data_type_defined(): + return node.in_port(port_id).get_source().get_data_type() return None diff --git a/tools/pot/openvino/tools/pot/graph/passes.py b/tools/pot/openvino/tools/pot/graph/passes.py index 1c8f11a55a4..38fbd3f6de0 100644 --- a/tools/pot/openvino/tools/pot/graph/passes.py +++ b/tools/pot/openvino/tools/pot/graph/passes.py @@ -22,7 +22,6 @@ from mo.graph.port import Port from mo.middle.pattern_match import apply_pattern from mo.ops.const import Const from mo.middle.passes.convert_data_type import convert_blob -from mo.middle.passes.infer import type_infer from . import editor as ge from . import node_utils as nu @@ -221,7 +220,6 @@ class FakeQuantizePropagation(BackReplacementPattern): } def delete_fq_non_quantizable_node_precision(self, graph): - type_infer(graph) fq_removal = RemoveFakeQuantize() fq_removal.quantize_agnostic_operations = self.quantize_agnostic_operations fq_removal.quantize_operations = self.quantize_operations @@ -713,14 +711,14 @@ def create_bias_node(graph: Graph, src_node): add_bias.out_node(0)['Insert_Convert_operation_after'] = True -def create_fake_quantize_node(graph: Graph, name): +def create_fake_quantize_node(graph: Graph, name, data_type=np.float32): fq = FakeQuantize(graph, {'name': name, 'levels': 0, 'stop_value_propagation': True}).create_node() - input_low = Const(graph, {'value': np.array(0.0).astype(np.float32)}).create_node() - input_height = Const(graph, {'value': np.array(0.0).astype(np.float32)}).create_node() - output_low = Const(graph, {'value': np.array(0.0).astype(np.float32)}).create_node() - output_height = Const(graph, {'value': np.array(0.0).astype(np.float32)}).create_node() + input_low = Const(graph, {'value': np.array(0.0, dtype=data_type)}).create_node() + input_height = Const(graph, {'value': np.array(0.0, dtype=data_type)}).create_node() + output_low = Const(graph, {'value': np.array(0.0, dtype=data_type)}).create_node() + output_height = Const(graph, {'value': np.array(0.0, dtype=data_type)}).create_node() input_low.out_port(0).connect(fq.in_port(1)) input_height.out_port(0).connect(fq.in_port(2)) @@ -764,9 +762,11 @@ def insert_fake_quantize(graph, node, ports=None, names=None): if port_name is not None and idx in port_name: name = port_name[idx] + port_data_type = nu.get_node_data_type(node, idx) + port_data_type = port_data_type if port_data_type else np.float32 # Create FakeQuantize operations fq_input = create_fake_quantize_node( - graph, '{node_name}/{name}_{idx}'.format(node_name=node.name, name=name, idx=idx)) + graph, '{node_name}/{name}_{idx}'.format(node_name=node.name, name=name, idx=idx), port_data_type) # Insert FakeQuantize after input if node.type == 'Result': @@ -841,6 +841,7 @@ def traverse_graph(node, move_fn, stop_criteria_fn=None, criteria_fns=None): def compress_weights(model: Graph): """Apply transformations to save model weights to INT8.""" + add_removed_converts(model) CompressQuantizeWeights().find_and_replace_pattern(model) model.clean_up() ForceStrictPrecision().find_and_replace_pattern(model) @@ -906,18 +907,25 @@ def add_removed_converts(graph: Graph): data_node = Node(graph, data_node_name) # Get access to Const node connected to data node const_op = data_node.in_node(0) - assert const_op.data_type == np.float32, "Error when try to insert Convert operation after Const: {}".\ - format(const_op.soft_get('name')) + + if const_op.type != 'Const': + logger.debug('Error when try to insert Convert operation after {} with {} type'.\ + format(const_op.soft_get('name'), const_op.soft_get('type'))) + continue + + if const_op.data_type != np.float32: + logger.debug('Error when try to insert Convert operation after Const: {}'.\ + format(const_op.soft_get('name'))) + continue convert_op = Cast(graph, {'dst_type': np.float32, 'name': const_op.name + '/restored_convert', 'stop_value_propagation': True}).create_node() # Insert Convert operation after Const operation - consumer_port = const_op.out_port(0).get_connection().get_destination() - const_op.out_port(0).get_connection().set_destination(convert_op.in_port(0)) - convert_op.out_port(0).connect(consumer_port) + const_op.out_port(0).get_connection().insert_node(convert_op) + convert_op.out_node().value = None - # Convert Const value to FP32 to make types in graph consistent + # Convert Const value to FP16 to make types in graph consistent const_op.value, _, _ = convert_blob(const_op.value, np.float16) const_op.infer(const_op) diff --git a/tools/pot/openvino/tools/pot/graph/transformer.py b/tools/pot/openvino/tools/pot/graph/transformer.py index 179b6c3c392..eafd91d91fc 100644 --- a/tools/pot/openvino/tools/pot/graph/transformer.py +++ b/tools/pot/openvino/tools/pot/graph/transformer.py @@ -1,6 +1,8 @@ # Copyright (C) 2020-2021 Intel Corporation # SPDX-License-Identifier: Apache-2.0 +from mo.middle.passes.infer import type_infer + from .editor import add_fullname_for_nodes from .special_operations import QUANTIZE_AGNOSTIC_OPERATIONS from .passes import InsertFakeQuantize, FakeQuantizePropagation, FakeQuantizeOptimization, RemoveFakeQuantize, \ @@ -46,6 +48,8 @@ class GraphTransformer: self.nodes_marker.mark_ignored_blocks(graph, self.target_device) graph.clean_up() + type_infer(graph) + self.fq_insertion.find_and_replace_pattern(graph) graph.clean_up()