IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Implement ExperimentalDetectronDetectionOutput and ExperimentalDetectronPriorGridGenerator operations as nGraph ops (#3374)

Browse Source 
* Commit.

* Started to write nGraph operation ExperimentalDetectronDetectionOutput. Written draft of the header file.

* Written draft of the cpp-file for nGraph operation ExperimentalDetectronDetectionOutput.

* Small fix.

* Added reading of ExperimentalDetectronDetectionOutput as nGraph operation.

* Some fix.

* Unregistered old shape infer function of the operation ExperimentalDetectronDetectionOutput.

* Written the header file for the operation ExperimentalDetectronPriorGridGenerator.

* Small refactoring.

* Small fix.

* Added set_output_size(3) into op::ExperimentalDetectronDetectionOutput::validate_and_infer_types().

* Added check for number of inputs of ExperimentalDetectronDetectionOutput.

* Reverted some changes.

* Changed IR for ExperimentalDetectronDetectionOutput serialization test.

* Written cpp-file of nGraph operation ExperimentalDetectronPriorGridGenerator.

* Small fix.

* Some fixes.

* Fixes in type and shape infer functions of the MO operation ExperimentalDetectronDetectionOutput.

* Now ExperimentalDetectronPriorGridGenerator is readed as nGraph operation.

* Fixed the infer function of the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Started to write tests for the shape infer function of the nGraph operation ExperimentalDetectronDetectionOutput.

* Written the draft of the test for the shape infer function of the nGraph operation ExperimentalDetectronDetectionOutput.

* Small fix.

* Fixed ngraph/test/CMakeLists.txt.

* Started to write tests for the shape infer function of the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Now the shape infer function of the nGraph operation ExperimentalDetectronPriorGridGenerator performs correctly case dynamic input shapes with static ranks.

* Continued to write test for the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Small fixes.

* Written tests for the shape infer function of the nGraph operation ExperimentalDetectronPriorGridGenerator (case when input shapes are partially dynamic).

* Added test for reading ExperimentalDetectronDetectionOutput as an operation from opset6.

* Some fixes.

* Added some debug outputs.

* Deleted inserted debug output.

* Small fixes.

* Small fix.

* Small fix.

* Small change.

* Added comments to attributes of ExperimentalDetectronDetectionOutput.

* Reverted changes.

* Deleted shape infer for output port 3.

* Small fixes.

* Deleted redundant keyword 'virtual'.

* Deleted redundant usings in header files of nGraph operations ExperimentalDetectronDetectionOutput and ExperimentalDetectronPriorGridGenerator.

* Some fixes.

* Small change.

* Now GridGenerator::validate takes three args (input partial shapes).

* Small fix.

* Deleted some usings.

* Small code style fix.

* Reverted changes in validate_and_infer_types() and validate() of op::v6::ExperimentalDetectronPriorGridGenerator.

* Added description of the class ExperimentalDetectronDetectionOutput.

* Added some comments into the header file of the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Some fixes.

* Added some comments to the class of the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Now the MO operation ExperimentalDetectronDetectionOutput has the attribute 'version' as 'opset6'.

* Now the MO operation ExperimentalDetectronPriorGridGenerator has the attribute 'version' as 'opset6'.

* Some fixes in the MO class ExperimentalDetectronDetectionOutput.

* Fixes in the shape infer function of the nGraph operation ExperimentalDetectronPriorGridGenerator.

* Renamed test XML model for ExperimentalDetectronDetectionOutput serialization tests.

* Added validation of input shapes for the nGraph operation ExperimentalDetectronDetectionOutput.

* Small fixes in the XML models for serialization testing of ExperimentalDetectronDetectionOutput.

* Added tests of shape infer function of the nGraph operation ExperimentalDetectronDetectionOutput for the case when input shapes are partially dynamic.

* Added tests of shape infer function of the nGraph operation ExperimentalDetectronDetectionOutput for the case when some input shapes have dynamic ranks.

* Small fixes.

* Small fix in the MO operation ExperimentalDetectronDetectionOutput shape infer function.

* Fixes in op::v6::ExperimentalDetectronDetectionOutput::validate_and_infer_types().

* Code style fix.

* Small refactoring.

* Added NGRAPH_OP_SCOPE into ExperimentalDetectronDetectionOutput nGraph class.

* Added NGRAPH_OP_SCOPE to the nGraph class ExperimentalDetectronPriorGridGenerator.

* Small fixes.

* Some refactoring.

* Small fix.

* Small fixes.

* Reverted some changes in ExperimentalDetectronDetectionOutput::validate_and_infer_type().

* Now VPU reads the attribute class_agnostic_box_regression of ExperimentalDetectronDetectionOutput as Bool.

* Now MO generates attribute  'class_agnostic_box_regression' of ExperimentalDetectronDetectionOutput only with values false or true.

* Small fix.

* Tabs were replaced by spaces in some XMLs.

* Fixed copyrights.

* Refactoring in op::v6::ExperimentalDetectronDetectionOutput::validate_and_infer_types().

* Refactoring in op::v6::ExperimentalDetectronPriorGridGenerator::validate_and_infer_types().

* Small fixes.

* Started to write ExperimentalDetectronPriorGridGenerator shape infer tests for the case when dynamic input dimensions are intervals.

* Deleted redundant 'return'.

* Written tests for interval values of input shapes of op::v6::ExperimentalDetectronPriorGridGenerator.

* Code style fix.

* Code style fix.
This commit is contained in:
Vladimir Gavrilov 2021-01-18 12:16:40 +03:00 committed by GitHub
parent f5085d3216
commit e20a58d770
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 1016 additions and 156 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
inference-engine/src/inference_engine/generic_ie.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2018-2020 Intel Corporation
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -133,8 +133,7 @@ void ngraph::op::GenericIE::validate_and_infer_types() {
}
// WA: shape infer has to know number of outputs
if ((type == "ExperimentalDetectronROIFeatureExtractor" || type == "ExperimentalDetectronDetectionOutput")
&& parameters.find("num_outputs") == parameters.end()) {
if (type == "ExperimentalDetectronROIFeatureExtractor" && parameters.find("num_outputs") == parameters.end()) {
parameters["num_outputs"] = std::to_string(outputs.size());
}
@ -152,8 +151,7 @@ void ngraph::op::GenericIE::validate_and_infer_types() {
// Extensions are not loaded when we create nGraph function
// First call: create node
if (initialized < 1) {
if ((type == "ExperimentalDetectronROIFeatureExtractor" || type == "ExperimentalDetectronDetectionOutput")
&& outputs.size() < 2) {
if (type == "ExperimentalDetectronROIFeatureExtractor" && outputs.size() < 2) {
// Add fake port
PortIE port;
port.precision = InferenceEngine::Precision::FP32;

6
inference-engine/src/inference_engine/shape_infer/ie_built_in_holder.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2018-2020 Intel Corporation
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -7,8 +7,6 @@
#include <string>
#include "shape_infer/ie_built_in_holder.hpp"
#include "shape_infer/ie_detectionoutput_onnx_shape_infer.hpp"
#include "shape_infer/ie_priorgridgenerator_onnx_shape_infer.hpp"
#include "shape_infer/ie_proposal_onnx_shape_infer.hpp"
#include "shape_infer/ie_proposal_shape_infer.hpp"
#include "shape_infer/ie_rnn_cell_shape_infer.hpp"
@ -68,8 +66,6 @@ public:
#define REG_SHAPE_INFER_FOR_TYPE(__prim, __type) \
static ImplRegisterBase<__prim> __bi_reg__##__type(#__type)
REG_SHAPE_INFER_FOR_TYPE(ExperimentalDetectronDetectionOutputShapeProp, ExperimentalDetectronDetectionOutput);
REG_SHAPE_INFER_FOR_TYPE(ExperimentalDetectronPriorGridGeneratorShapeProp, ExperimentalDetectronPriorGridGenerator);
REG_SHAPE_INFER_FOR_TYPE(ExperimentalDetectronGenerateProposalsSingleImageShapeProp, ExperimentalDetectronGenerateProposalsSingleImage);
REG_SHAPE_INFER_FOR_TYPE(ExperimentalDetectronROIFeatureExtractorShapeProp, ExperimentalDetectronROIFeatureExtractor);
REG_SHAPE_INFER_FOR_TYPE(ExperimentalDetectronTopKROIsShapeProp, ExperimentalDetectronTopKROIs);

46
inference-engine/src/inference_engine/shape_infer/ie_detectionoutput_onnx_shape_infer.hpp
View File

@ -1,46 +0,0 @@
// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <map>
#include <memory>
#include <string>
#include <vector>
#include "ie_built_in_impl.hpp"
namespace InferenceEngine {
namespace ShapeInfer {
/**
* @brief Implementation of Shape inference for ExperimentalDetectronDetectionOutput layer
*/
class ExperimentalDetectronDetectionOutputShapeProp : public BuiltInShapeInferImpl {
protected:
const int ROIS = 0;
const int FEATMAPS = 1;
public:
explicit ExperimentalDetectronDetectionOutputShapeProp(const std::string& type): BuiltInShapeInferImpl(type) {}
void inferShapesImpl(const std::vector<Blob::CPtr>& inBlobs, const std::map<std::string, std::string>& params,
const std::map<std::string, Blob::Ptr>& blobs, std::vector<SizeVector>& outShapes) override {
auto rois_num = GetParamAsUInt("max_detections_per_image", params);
outShapes.push_back({rois_num, 4});
auto num_outputs = GetParamAsUInt("num_outputs", params);
if (num_outputs > 3)
THROW_IE_EXCEPTION << "Incorrect value num_outputs: " << num_outputs;
if (num_outputs >= 2) {
outShapes.push_back({rois_num});
}
if (num_outputs == 3) {
outShapes.push_back({rois_num});
}
}
};
} // namespace ShapeInfer
} // namespace InferenceEngine

49
inference-engine/src/inference_engine/shape_infer/ie_priorgridgenerator_onnx_shape_infer.hpp
View File

@ -1,49 +0,0 @@
// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include <description_buffer.hpp>
#include <map>
#include <memory>
#include <string>
#include <vector>
#include "ie_built_in_impl.hpp"
namespace InferenceEngine {
namespace ShapeInfer {
/**
* @brief Implementation of Shape inference for ExperimentalDetectronPriorGridGenerator layer
*/
class ExperimentalDetectronPriorGridGeneratorShapeProp : public BuiltInShapeInferImpl {
protected:
const int PRIORS = 0;
const int FEATMAP = 1;
const int H = 2;
const int W = 3;
public:
explicit ExperimentalDetectronPriorGridGeneratorShapeProp(const std::string& type): BuiltInShapeInferImpl(type) {}
void inferShapesImpl(const std::vector<Blob::CPtr>& inBlobs, const std::map<std::string, std::string>& params,
const std::map<std::string, Blob::Ptr>& blobs, std::vector<SizeVector>& outShapes) override {
const auto& priors_shape = inShapes.at(PRIORS);
const auto priors_num = priors_shape.at(0);
const auto& featmap_shape = inShapes.at(FEATMAP);
const auto grid_height = featmap_shape.at(H);
const auto grid_width = featmap_shape.at(W);
const bool flatten = GetParamAsBool("flatten", true, params);
if (flatten) {
outShapes.push_back({grid_height * grid_width * priors_num, 4});
} else {
outShapes.push_back({grid_height, grid_width, priors_num, 4});
}
}
};
} // namespace ShapeInfer
} // namespace InferenceEngine

15
inference-engine/src/readers/ir_reader/ie_ir_parser.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2018-2020 Intel Corporation
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -678,7 +678,15 @@ std::shared_ptr<ngraph::Node> V10Parser::XmlDeserializer::createNode(
}
// Try to create operation from loaded opsets
if (!ngraphNode && opsetIt != opsets.end()) {
auto version = params.version;
static const std::unordered_set<std::string> experimental_detectrons = {"ExperimentalDetectronDetectionOutput",
"ExperimentalDetectronPriorGridGenerator"};
if (experimental_detectrons.count(params.type)) {
version = "opset6";
}
if (!ngraphNode && opsets.count(version)) {
auto opset = opsets.at(version);
auto const & type = params.type == "Const"
? "Constant"
: params.type;
@ -691,11 +699,10 @@ std::shared_ptr<ngraph::Node> V10Parser::XmlDeserializer::createNode(
THROW_IE_EXCEPTION << "Cannot create " << params.type << " layer " << params.name << " id:" << params.layerId
<< " from unsupported opset: " << params.version;
}
opset = opsetIt->second;
}
}
auto const & opset = opsetIt->second;
ngraphNode = std::shared_ptr<ngraph::Node>(opset.create_insensitive(type));
if (!ngraphNode) {
THROW_IE_EXCEPTION << "Opset " << params.version << " doesn't contain the operation with type: " << type;

4
inference-engine/src/vpu/graph_transformer/src/stages/exp_detectionoutput.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2018-2020 Intel Corporation
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -97,7 +97,7 @@ void FrontEnd::parseExpDetectionOutput(const Model& model, const ie::CNNLayerPtr
params.max_detections_per_image = layer->GetParamAsInt("max_detections_per_image", 0);
params.num_classes = layer->GetParamAsInt("num_classes", 0);
params.post_nms_count = layer->GetParamAsInt("post_nms_count", 0);
params.class_agnostic_box_regression = layer->GetParamAsFloat("class_agnostic_box_regression", 0) ? 1 : 0;
params.class_agnostic_box_regression = layer->GetParamAsBool("class_agnostic_box_regression", false);
auto inputBoxes = inputs[0]; // [numRois][4]
auto inputDeltas = inputs[1]; // [numRois]([numClasses][4])

138
inference-engine/tests/functional/inference_engine/ir_serialization/models/experimental_detectron_detection_output.xml
View File

@ -1,38 +1,106 @@
<?xml version="1.0" ?>
<!--This is syntetic model created by hand desined only for white-box unit testing-->
<net name="add_abc" version="10">
<layers>
<layer id="0" name="A" type="Parameter" version="opset1">
<data element_type="f32" shape="1"/>
<output>
<port id="0" precision="FP32">
<dim>1</dim>
</port>
</output>
</layer>
<layer id="1" name="DetectionOutput" type="ExperimentalDetectronDetectionOutput" version="experimental">
<data class_agnostic_box_regression="0" deltas_weights="10.0,10.0,5.0,5.0" max_delta_log_wh="4.135166645050049" max_detections_per_image="100" nms_threshold="0.5" num_classes="81" post_nms_count="2000" score_threshold="0.05"/>
<input>
<port id="0">
<dim>1</dim>
</port>
</input>
<output>
<port id="1" precision="FP32">
<dim>1</dim>
</port>
</output>
</layer>
<layer id="2" name="Y" type="Result" version="opset1">
<input>
<port id="0">
<dim>1</dim>
</port>
</input>
</layer>
</layers>
<edges>
<edge from-layer="0" from-port="0" to-layer="1" to-port="0"/>
<edge from-layer="1" from-port="1" to-layer="2" to-port="0"/>
</edges>
<net name="ExperimentalDetectronDetectionOutput" version="10">
<layers>
<layer name="in0" type="Parameter" id="0" version="opset1">
<data shape="1000,4" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>4</dim>
</port>
</output>
</layer>
<layer name="in1" type="Parameter" id="1" version="opset1">
<data shape="1000,324" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>324</dim>
</port>
</output>
</layer>
<layer name="in2" type="Parameter" id="2" version="opset1">
<data shape="1000,81" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>81</dim>
</port>
</output>
</layer>
<layer name="in3" type="Parameter" id="3" version="opset1">
<data shape="1,3" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1</dim>
<dim>3</dim>
</port>
</output>
</layer>
<layer id="4" name="DO" type="ExperimentalDetectronDetectionOutput" version="experimental">
<data class_agnostic_box_regression="0" deltas_weights="10.0,10.0,5.0,5.0" max_delta_log_wh="4.135166645050049" max_detections_per_image="100" nms_threshold="0.5" num_classes="81" post_nms_count="2000" score_threshold="0.05000000074505806"/>
<input>
<port id="0">
<dim>1000</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>1000</dim>
<dim>324</dim>
</port>
<port id="2">
<dim>1000</dim>
<dim>81</dim>
</port>
<port id="3">
<dim>1</dim>
<dim>3</dim>
</port>
</input>
<output>
<port id="4" precision="FP32">
<dim>100</dim>
<dim>4</dim>
</port>
<port id="5" precision="I32">
<dim>100</dim>
</port>
<port id="6" precision="FP32">
<dim>100</dim>
</port>
</output>
</layer>
<layer name="out_0" type="Result" id="5" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
<dim>4</dim>
</port>
</input>
</layer>
<layer name="out_1" type="Result" id="6" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
</port>
</input>
</layer>
<layer name="out_2" type="Result" id="7" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
</port>
</input>
</layer>
</layers>
<edges>
<edge from-layer="0" from-port="0" to-layer="4" to-port="0"/>
<edge from-layer="1" from-port="0" to-layer="4" to-port="1"/>
<edge from-layer="2" from-port="0" to-layer="4" to-port="2"/>
<edge from-layer="3" from-port="0" to-layer="4" to-port="3"/>
<edge from-layer="4" from-port="4" to-layer="5" to-port="0"/>
<edge from-layer="4" from-port="5" to-layer="6" to-port="0"/>
<edge from-layer="4" from-port="6" to-layer="7" to-port="0"/>
</edges>
</net>

106
inference-engine/tests/functional/inference_engine/ir_serialization/models/experimental_detectron_detection_output_opset6.xml Normal file
View File

@ -0,0 +1,106 @@
<?xml version="1.0" ?>
<!--This is syntetic model created by hand desined only for white-box unit testing-->
<net name="ExperimentalDetectronDetectionOutput" version="10">
<layers>
<layer name="in0" type="Parameter" id="0" version="opset1">
<data shape="1000,4" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>4</dim>
</port>
</output>
</layer>
<layer name="in1" type="Parameter" id="1" version="opset1">
<data shape="1000,324" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>324</dim>
</port>
</output>
</layer>
<layer name="in2" type="Parameter" id="2" version="opset1">
<data shape="1000,81" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>
<dim>81</dim>
</port>
</output>
</layer>
<layer name="in3" type="Parameter" id="3" version="opset1">
<data shape="1,3" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1</dim>
<dim>3</dim>
</port>
</output>
</layer>
<layer id="4" name="DO" type="ExperimentalDetectronDetectionOutput" version="opset6">
<data class_agnostic_box_regression="0" deltas_weights="10.0,10.0,5.0,5.0" max_delta_log_wh="4.135166645050049" max_detections_per_image="100" nms_threshold="0.5" num_classes="81" post_nms_count="2000" score_threshold="0.05000000074505806"/>
<input>
<port id="0">
<dim>1000</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>1000</dim>
<dim>324</dim>
</port>
<port id="2">
<dim>1000</dim>
<dim>81</dim>
</port>
<port id="3">
<dim>1</dim>
<dim>3</dim>
</port>
</input>
<output>
<port id="4" precision="FP32">
<dim>100</dim>
<dim>4</dim>
</port>
<port id="5" precision="I32">
<dim>100</dim>
</port>
<port id="6" precision="FP32">
<dim>100</dim>
</port>
</output>
</layer>
<layer name="out_0" type="Result" id="5" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
<dim>4</dim>
</port>
</input>
</layer>
<layer name="out_1" type="Result" id="6" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
</port>
</input>
</layer>
<layer name="out_2" type="Result" id="7" version="opset1">
<input>
<port id="0" precision="FP32">
<dim>100</dim>
</port>
</input>
</layer>
</layers>
<edges>
<edge from-layer="0" from-port="0" to-layer="4" to-port="0"/>
<edge from-layer="1" from-port="0" to-layer="4" to-port="1"/>
<edge from-layer="2" from-port="0" to-layer="4" to-port="2"/>
<edge from-layer="3" from-port="0" to-layer="4" to-port="3"/>
<edge from-layer="4" from-port="4" to-layer="5" to-port="0"/>
<edge from-layer="4" from-port="5" to-layer="6" to-port="0"/>
<edge from-layer="4" from-port="6" to-layer="7" to-port="0"/>
</edges>
</net>

3
inference-engine/tests/functional/inference_engine/ir_serialization/serialize.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2020 Intel Corporation
// Copyright (C) 2017-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -55,6 +55,7 @@ INSTANTIATE_TEST_CASE_P(IRSerialization, SerializationTest,
std::make_tuple("add_abc_initializers.xml"),
std::make_tuple("experimental_detectron_roi_feature_extractor.xml"),
std::make_tuple("experimental_detectron_detection_output.xml"),
std::make_tuple("experimental_detectron_detection_output_opset6.xml"),
std::make_tuple("nms5.xml"),
std::make_tuple("shape_of.xml")));

4
inference-engine/tests/functional/inference_engine/ngraph_reshape_tests.cpp
View File

@ -1,4 +1,4 @@
// Copyright (C) 2018-2020 Intel Corporation
// Copyright (C) 2018-2021 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
@ -559,7 +559,7 @@ TEST_F(NGraphReshapeTests, ReshapeEDDetectionOutput) {
</output>
</layer>
<layer name="in2" type="Parameter" id="2" version="opset1">
<data shape="1000,324" element_type="f32"/>
<data shape="1000,81" element_type="f32"/>
<output>
<port id="0" precision="FP32">
<dim>1000</dim>

12
model-optimizer/extensions/ops/detectionoutput_onnx.py
View File

@ -1,5 +1,5 @@
"""
Copyright (C) 2018-2020 Intel Corporation
Copyright (C) 2018-2021 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@ -28,7 +28,7 @@ class ExperimentalDetectronDetectionOutput(Op):
mandatory_props = dict(
type=__class__.op,
op=__class__.op,
version='experimental',
version='opset6',
infer=__class__.infer,
type_infer=self.type_infer,
in_ports_count=4,
@ -39,7 +39,7 @@ class ExperimentalDetectronDetectionOutput(Op):
def backend_attrs(self):
return [
'class_agnostic_box_regression',
('class_agnostic_box_regression', lambda node: str(bool(node['class_agnostic_box_regression'])).lower()),
'max_detections_per_image',
'nms_threshold',
'num_classes',
@ -54,7 +54,9 @@ class ExperimentalDetectronDetectionOutput(Op):
# boxes
node.out_node(0).shape = np.array([rois_num, 4], dtype=np.int64)
# classes, scores, batch indices
for port_ind in range(1, 3):
# We use range(1, 1 + max(node.out_ports().keys())) instead of range(1, 3), because there are incorrectly
# generated models where ExperimentalDetectronDetectionOutput has 4 outputs.
for port_ind in range(1, 1 + max(node.out_ports().keys())):
if not node.out_port(port_ind).disconnected():
node.out_port(port_ind).data.set_shape(int64_array([rois_num]))
@ -64,3 +66,5 @@ class ExperimentalDetectronDetectionOutput(Op):
node.out_port(0).set_data_type(in_data_type)
node.out_port(1).set_data_type(np.int32) # the second output contains class indices
node.out_port(2).set_data_type(in_data_type)
if node.is_out_port_connected(3):
node.out_port(3).set_data_type(np.int32) # the fourth output contains batch indices

4
model-optimizer/extensions/ops/priorgridgenerator_onnx.py
View File

@ -1,5 +1,5 @@
"""
Copyright (C) 2018-2020 Intel Corporation
Copyright (C) 2018-2021 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
@ -26,7 +26,7 @@ class ExperimentalDetectronPriorGridGenerator(Op):
mandatory_props = dict(
type=__class__.op,
op=__class__.op,
version='experimental',
version='opset6',
infer=__class__.infer,
)
super().__init__(graph, mandatory_props, attrs)

88
ngraph/core/include/ngraph/op/experimental_detectron_detection_output.hpp Normal file
View File

@ -0,0 +1,88 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#pragma once
#include <cstddef>
#include <vector>
#include "ngraph/attribute_adapter.hpp"
#include "ngraph/op/op.hpp"
#include "ngraph/op/util/attr_types.hpp"
namespace ngraph
{
namespace op
{
namespace v6
{
/// \brief An operation ExperimentalDetectronDetectionOutput, according to
/// the repository https://github.com/openvinotoolkit/training_extensions
/// (see pytorch_toolkit/instance_segmentation/segmentoly/rcnn/detection_output.py).
class NGRAPH_API ExperimentalDetectronDetectionOutput : public Op
{
public:
NGRAPH_RTTI_DECLARATION;
/// \brief Structure that specifies attributes of the operation
struct Attributes
{
// specifies score threshold
float score_threshold;
// specifies NMS threshold
float nms_threshold;
// specifies maximal delta of logarithms for width and height
float max_delta_log_wh;
// specifies number of detected classes
int64_t num_classes;
// specifies maximal number of detections per class
int64_t post_nms_count;
// specifies maximual number of detections per image
size_t max_detections_per_image;
// a flag specifies whether to delete background classes or not
// `true` means background classes should be deleted,
// `false` means background classes shouldn't be deleted.
bool class_agnostic_box_regression;
// specifies deltas of weights
std::vector<float> deltas_weights;
};
ExperimentalDetectronDetectionOutput() = default;
/// \brief Constructs a ExperimentalDetectronDetectionOutput operation.
///
/// \param input_rois Input rois
/// \param input_deltas Input deltas
/// \param input_scores Input scores
/// \param input_im_info Input image info
/// \param attrs Attributes attributes
ExperimentalDetectronDetectionOutput(const Output<Node>& input_rois,
const Output<Node>& input_deltas,
const Output<Node>& input_scores,
const Output<Node>& input_im_info,
const Attributes& attrs);
bool visit_attributes(AttributeVisitor& visitor) override;
void validate_and_infer_types() override;
std::shared_ptr<Node>
clone_with_new_inputs(const OutputVector& new_args) const override;
/// \brief Returns attributes of the operation ExperimentalDetectronDetectionOutput
const Attributes& get_attrs() const { return m_attrs; }
private:
Attributes m_attrs;
};
}
}
}

82
ngraph/core/include/ngraph/op/experimental_detectron_prior_grid_generator.hpp Normal file
View File

@ -0,0 +1,82 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#pragma once
#include <cstdint>
#include <vector>
#include "ngraph/attribute_adapter.hpp"
#include "ngraph/op/op.hpp"
#include "ngraph/op/util/attr_types.hpp"
namespace ngraph
{
namespace op
{
namespace v6
{
/// \brief An operation ExperimentalDetectronPriorGridGenerator, according to
/// the repository https://github.com/openvinotoolkit/training_extensions
/// (see pytorch_toolkit/instance_segmentation/segmentoly/rcnn/prior_box.py).
class NGRAPH_API ExperimentalDetectronPriorGridGenerator : public Op
{
public:
NGRAPH_RTTI_DECLARATION;
/// \brief Structure that specifies attributes of the operation
struct Attributes
{
// Specifies whether the output tensor should be 2D or 4D
// `true` means the output tensor should be 2D tensor,
// `false` means the output tensor should be 4D tensor.
bool flatten;
// Specifies number of cells of the generated grid with respect to height.
int64_t h;
// Specifies number of cells of the generated grid with respect to width.
int64_t w;
// Specifies the step of generated grid with respect to x coordinate
float stride_x;
// Specifies the step of generated grid with respect to y coordinate
float stride_y;
};
ExperimentalDetectronPriorGridGenerator() = default;
/// \brief Constructs a ExperimentalDetectronDetectionOutput operation.
///
/// \param priors Input priors
/// \param feature_map Input feature map
/// \param im_data Image data
/// \param attrs attributes
ExperimentalDetectronPriorGridGenerator(const Output<Node>& priors,
const Output<Node>& feature_map,
const Output<Node>& im_data,
const Attributes& attrs);
bool visit_attributes(AttributeVisitor& visitor) override;
void validate_and_infer_types() override;
std::shared_ptr<Node>
clone_with_new_inputs(const OutputVector& new_args) const override;
/// \brief Returns attributes of this operation.
const Attributes& get_attrs() const { return m_attrs; }
private:
Attributes m_attrs;
void validate();
};
}
}
}

2
ngraph/core/include/ngraph/ops.hpp
View File

@ -59,6 +59,8 @@
#include "ngraph/op/equal.hpp"
#include "ngraph/op/erf.hpp"
#include "ngraph/op/exp.hpp"
#include "ngraph/op/experimental_detectron_detection_output.hpp"
#include "ngraph/op/experimental_detectron_prior_grid_generator.hpp"
#include "ngraph/op/extractimagepatches.hpp"
#include "ngraph/op/fake_quantize.hpp"
#include "ngraph/op/floor.hpp"

4
ngraph/core/include/ngraph/opsets/opset6_tbl.hpp
View File

@ -174,5 +174,7 @@ NGRAPH_OP(Round, ngraph::op::v5)
// New operations added in opset6
NGRAPH_OP(CTCGreedyDecoderSeqLen, ngraph::op::v6)
NGRAPH_OP(MVN, ngraph::op::v6)
NGRAPH_OP(ExperimentalDetectronDetectionOutput, ngraph::op::v6)
NGRAPH_OP(ExperimentalDetectronPriorGridGenerator, ngraph::op::v6)
NGRAPH_OP(GatherElements, ngraph::op::v6)
NGRAPH_OP(MVN, ngraph::op::v6)

152
ngraph/core/src/op/experimental_detectron_detection_output.cpp Normal file
View File

@ -0,0 +1,152 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#include <memory>
#include "itt.hpp"
#include "ngraph/attribute_visitor.hpp"
#include "ngraph/op/experimental_detectron_detection_output.hpp"
#include "ngraph/runtime/host_tensor.hpp"
using namespace std;
using namespace ngraph;
NGRAPH_RTTI_DEFINITION(op::v6::ExperimentalDetectronDetectionOutput,
"ExperimentalDetectronDetectionOutput",
6);
op::v6::ExperimentalDetectronDetectionOutput::ExperimentalDetectronDetectionOutput(
const Output<Node>& input_rois,
const Output<Node>& input_deltas,
const Output<Node>& input_scores,
const Output<Node>& input_im_info,
const Attributes& attrs)
: Op({input_rois, input_deltas, input_scores, input_im_info})
, m_attrs(attrs)
{
constructor_validate_and_infer_types();
}
bool op::v6::ExperimentalDetectronDetectionOutput::visit_attributes(AttributeVisitor& visitor)
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronDetectionOutput_visit_attributes);
visitor.on_attribute("score_threshold", m_attrs.score_threshold);
visitor.on_attribute("nms_threshold", m_attrs.nms_threshold);
visitor.on_attribute("max_delta_log_wh", m_attrs.max_delta_log_wh);
visitor.on_attribute("num_classes", m_attrs.num_classes);
visitor.on_attribute("post_nms_count", m_attrs.post_nms_count);
visitor.on_attribute("max_detections_per_image", m_attrs.max_detections_per_image);
visitor.on_attribute("class_agnostic_box_regression", m_attrs.class_agnostic_box_regression);
visitor.on_attribute("deltas_weights", m_attrs.deltas_weights);
return true;
}
void op::v6::ExperimentalDetectronDetectionOutput::validate_and_infer_types()
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronDetectionOutput_validate_and_infer_types);
size_t rois_num = m_attrs.max_detections_per_image;
auto input_et = get_input_element_type(0);
auto rois_shape = get_input_partial_shape(0);
auto deltas_shape = get_input_partial_shape(1);
auto scores_shape = get_input_partial_shape(2);
auto im_info_shape = get_input_partial_shape(3);
set_output_type(0, input_et, Shape{rois_num, 4});
set_output_type(1, element::Type_t::i32, Shape{rois_num});
set_output_type(2, input_et, Shape{rois_num});
if (rois_shape.rank().is_static())
{
NODE_VALIDATION_CHECK(
this, rois_shape.rank().get_length() == 2, "Input rois rank must be equal to 2.");
NODE_VALIDATION_CHECK(this,
rois_shape[1].is_static() && rois_shape[1].get_length() == 4u,
"The last dimension of the 'input_rois' input must be equal to 4. "
"Got: ",
rois_shape[1]);
}
if (deltas_shape.rank().is_static())
{
NODE_VALIDATION_CHECK(
this, deltas_shape.rank().get_length() == 2, "Input deltas rank must be equal to 2.");
if (deltas_shape[1].is_static())
{
NODE_VALIDATION_CHECK(this,
deltas_shape[1].get_length() == m_attrs.num_classes * 4,
"The last dimension of the 'input_deltas' input must be equal to "
"the value of the attribute 'num_classes' * 4. Got: ",
deltas_shape[1]);
}
}
if (scores_shape.rank().is_static())
{
NODE_VALIDATION_CHECK(
this, scores_shape.rank().get_length() == 2, "Input scores rank must be equal to 2.");
if (scores_shape[1].is_static())
{
NODE_VALIDATION_CHECK(this,
scores_shape[1].get_length() == m_attrs.num_classes,
"The last dimension of the 'input_scores' input must be equal to "
"the value of the attribute 'num_classes'. Got: ",
scores_shape[1]);
}
}
if (im_info_shape.rank().is_static())
{
NODE_VALIDATION_CHECK(this,
im_info_shape.rank().get_length() == 2,
"Input image info rank must be equal to 2.");
}
if (rois_shape.rank().is_static() && deltas_shape.rank().is_static() &&
scores_shape.rank().is_static())
{
const auto num_batches_rois = rois_shape[0];
const auto num_batches_deltas = deltas_shape[0];
const auto num_batches_scores = scores_shape[0];
if (num_batches_rois.is_static() && num_batches_deltas.is_static() &&
num_batches_scores.is_static())
{
NODE_VALIDATION_CHECK(this,
num_batches_rois.same_scheme(num_batches_deltas) &&
num_batches_deltas.same_scheme(num_batches_scores),
"The first dimension of inputs 'input_rois', 'input_deltas', "
"'input_scores' must be the same. input_rois batch: ",
num_batches_rois,
"; input_deltas batch: ",
num_batches_deltas,
"; input_scores batch: ",
num_batches_scores);
}
}
}
shared_ptr<Node> op::v6::ExperimentalDetectronDetectionOutput::clone_with_new_inputs(
const OutputVector& new_args) const
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronDetectionOutput_clone_with_new_inputs);
check_new_args_count(this, new_args);
return make_shared<op::v6::ExperimentalDetectronDetectionOutput>(
new_args.at(0), new_args.at(1), new_args.at(2), new_args.at(3), m_attrs);
}

142
ngraph/core/src/op/experimental_detectron_prior_grid_generator.cpp Normal file
View File

@ -0,0 +1,142 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#include <memory>
#include "itt.hpp"
#include "ngraph/attribute_visitor.hpp"
#include "ngraph/op/experimental_detectron_prior_grid_generator.hpp"
#include "ngraph/runtime/host_tensor.hpp"
using namespace std;
using namespace ngraph;
NGRAPH_RTTI_DEFINITION(op::v6::ExperimentalDetectronPriorGridGenerator,
"ExperimentalDetectronPriorGridGenerator",
6);
op::v6::ExperimentalDetectronPriorGridGenerator::ExperimentalDetectronPriorGridGenerator(
const Output<Node>& priors,
const Output<Node>& feature_map,
const Output<Node>& im_data,
const Attributes& attrs)
: Op({priors, feature_map, im_data})
, m_attrs(attrs)
{
constructor_validate_and_infer_types();
}
bool op::v6::ExperimentalDetectronPriorGridGenerator::visit_attributes(AttributeVisitor& visitor)
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronPriorGridGenerator_visit_attributes);
visitor.on_attribute("flatten", m_attrs.flatten);
visitor.on_attribute("h", m_attrs.h);
visitor.on_attribute("w", m_attrs.w);
visitor.on_attribute("stride_x", m_attrs.stride_x);
visitor.on_attribute("stride_y", m_attrs.stride_y);
return true;
}
shared_ptr<Node> op::v6::ExperimentalDetectronPriorGridGenerator::clone_with_new_inputs(
const OutputVector& new_args) const
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronPriorGridGenerator_clone_with_new_inputs);
check_new_args_count(this, new_args);
return make_shared<op::v6::ExperimentalDetectronPriorGridGenerator>(
new_args.at(0), new_args.at(1), new_args.at(2), m_attrs);
}
static constexpr size_t priors_port = 0;
static constexpr size_t featmap_port = 1;
static constexpr size_t im_data_port = 2;
void op::v6::ExperimentalDetectronPriorGridGenerator::validate()
{
auto priors_shape = get_input_partial_shape(priors_port);
auto featmap_shape = get_input_partial_shape(featmap_port);
auto im_data_shape = get_input_partial_shape(im_data_port);
if (priors_shape.rank().is_dynamic() || featmap_shape.rank().is_dynamic())
{
return;
}
NODE_VALIDATION_CHECK(
this, priors_shape.rank().get_length() == 2, "Priors rank must be equal to 2.");
NODE_VALIDATION_CHECK(this,
priors_shape[1].is_static() && priors_shape[1].get_length() == 4u,
"The last dimension of the 'priors' input must be equal to 4. Got: ",
priors_shape[1]);
NODE_VALIDATION_CHECK(
this, featmap_shape.rank().get_length() == 4, "Feature_map rank must be equal to 4.");
if (im_data_shape.rank().is_dynamic())
{
return;
}
NODE_VALIDATION_CHECK(
this, im_data_shape.rank().get_length() == 4, "Im_data rank must be equal to 4.");
const auto num_batches_featmap = featmap_shape[0];
const auto num_batches_im_data = im_data_shape[0];
NODE_VALIDATION_CHECK(this,
num_batches_featmap.same_scheme(num_batches_im_data),
"The first dimension of both 'feature_map' and 'im_data' must match. "
"Feature_map: ",
num_batches_featmap,
"; Im_data: ",
num_batches_im_data);
}
void op::v6::ExperimentalDetectronPriorGridGenerator::validate_and_infer_types()
{
NGRAPH_OP_SCOPE(v6_ExperimentalDetectronPriorGridGenerator_validate_and_infer_types);
auto priors_shape = get_input_partial_shape(priors_port);
auto featmap_shape = get_input_partial_shape(featmap_port);
auto input_et = get_input_element_type(0);
validate();
set_output_size(1);
PartialShape out_shape = {Dimension::dynamic(), Dimension::dynamic(), Dimension::dynamic(), 4};
if (m_attrs.flatten)
{
out_shape = PartialShape{Dimension::dynamic(), 4};
}
if (priors_shape.rank().is_dynamic() || featmap_shape.rank().is_dynamic())
{
set_output_type(0, input_et, out_shape);
return;
}
auto num_priors = priors_shape[0];
auto featmap_height = featmap_shape[2];
auto featmap_width = featmap_shape[3];
if (m_attrs.flatten)
{
out_shape = PartialShape{featmap_height * featmap_width * num_priors, 4};
}
else
{
out_shape = PartialShape{featmap_height, featmap_width, num_priors, 4};
}
set_output_type(0, input_et, out_shape);
}

2
ngraph/test/CMakeLists.txt
View File

@ -129,6 +129,8 @@ set(SRC
type_prop/strided_slice.cpp
type_prop/elu.cpp
type_prop/embeddingbag_offsetssum.cpp
type_prop/experimental_detectron_detection_output.cpp
type_prop/experimental_detectron_prior_grid_generator.cpp
type_prop/extractimagepatches.cpp
type_prop/embeddingbag_packedsum.cpp
type_prop/embedding_segments_sum.cpp

127
ngraph/test/type_prop/experimental_detectron_detection_output.cpp Normal file
View File

@ -0,0 +1,127 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#include <vector>
#include "gtest/gtest.h"
#include "ngraph/ngraph.hpp"
#include "util/type_prop.hpp"
using namespace ngraph;
using Attrs = op::v6::ExperimentalDetectronDetectionOutput::Attributes;
using ExperimentalDetection = op::v6::ExperimentalDetectronDetectionOutput;
TEST(type_prop, detectron_detection_output)
{
Attrs attrs;
attrs.class_agnostic_box_regression = false;
attrs.deltas_weights = {10.0f, 10.0f, 5.0f, 5.0f};
attrs.max_delta_log_wh = 4.135166645050049f;
attrs.max_detections_per_image = 100;
attrs.nms_threshold = 0.5f;
attrs.num_classes = 81;
attrs.post_nms_count = 2000;
attrs.score_threshold = 0.05000000074505806f;
size_t rois_num = static_cast<size_t>(attrs.max_detections_per_image);
auto rois = std::make_shared<op::Parameter>(element::f32, Shape{1000, 4});
auto deltas = std::make_shared<op::Parameter>(element::f32, Shape{1000, 324});
auto scores = std::make_shared<op::Parameter>(element::f32, Shape{1000, 81});
auto im_info = std::make_shared<op::Parameter>(element::f32, Shape{1, 3});
auto detection = std::make_shared<ExperimentalDetection>(rois, deltas, scores, im_info, attrs);
ASSERT_EQ(detection->get_output_element_type(0), element::f32);
ASSERT_EQ(detection->get_output_element_type(1), element::i32);
ASSERT_EQ(detection->get_output_element_type(2), element::f32);
EXPECT_EQ(detection->get_output_shape(0), (Shape{rois_num, 4}));
EXPECT_EQ(detection->get_output_shape(1), (Shape{rois_num}));
EXPECT_EQ(detection->get_output_shape(2), (Shape{rois_num}));
}
TEST(type_prop, detectron_detection_output_dynamic_input_shapes)
{
Attrs attrs;
attrs.class_agnostic_box_regression = false;
attrs.deltas_weights = {10.0f, 10.0f, 5.0f, 5.0f};
attrs.max_delta_log_wh = 4.135166645050049f;
attrs.max_detections_per_image = 100;
attrs.nms_threshold = 0.5f;
attrs.num_classes = 81;
attrs.post_nms_count = 2000;
attrs.score_threshold = 0.05000000074505806f;
size_t rois_num = static_cast<size_t>(attrs.max_detections_per_image);
struct ShapesAndAttrs
{
PartialShape rois_shape;
PartialShape deltas_shape;
PartialShape scores_shape;
PartialShape im_info_shape;
};
const auto dyn_dim = Dimension::dynamic();
const auto dyn_shape = PartialShape::dynamic();
std::vector<ShapesAndAttrs> shapes = {
{{1000, 4}, {1000, 324}, {1000, 81}, {1, 3}},
{{1000, 4}, {1000, 324}, {1000, 81}, {1, dyn_dim}},
{{1000, 4}, {1000, 324}, {1000, 81}, {dyn_dim, 3}},
{{1000, 4}, {1000, 324}, {1000, 81}, {dyn_dim, dyn_dim}},
{{dyn_dim, 4}, {dyn_dim, 324}, {dyn_dim, 81}, {1, 3}},
{{dyn_dim, 4}, {dyn_dim, 324}, {dyn_dim, 81}, {1, dyn_dim}},
{{dyn_dim, 4}, {dyn_dim, 324}, {dyn_dim, 81}, {dyn_dim, 3}},
{{dyn_dim, 4}, {dyn_dim, 324}, {dyn_dim, 81}, {dyn_dim, dyn_dim}},
{{1000, 4}, {1000, 324}, {1000, 81}, dyn_shape},
{{1000, 4}, {1000, 324}, dyn_shape, {1, 3}},
{{1000, 4}, {1000, 324}, dyn_shape, dyn_shape},
{{1000, 4}, dyn_shape, {1000, 81}, {1, 3}},
{{1000, 4}, dyn_shape, {1000, 81}, dyn_shape},
{{1000, 4}, dyn_shape, dyn_shape, {1, 3}},
{{1000, 4}, dyn_shape, dyn_shape, dyn_shape},
{dyn_shape, {1000, 324}, {1000, 81}, {1, 3}},
{dyn_shape, {1000, 324}, {1000, 81}, dyn_shape},
{dyn_shape, {1000, 324}, dyn_shape, {1, 3}},
{dyn_shape, {1000, 324}, dyn_shape, dyn_shape},
{dyn_shape, dyn_shape, {1000, 81}, {1, 3}},
{dyn_shape, dyn_shape, {1000, 81}, dyn_shape},
{dyn_shape, dyn_shape, dyn_shape, {1, 3}},
{dyn_shape, dyn_shape, dyn_shape, dyn_shape},
};
for (const auto& s : shapes)
{
auto rois = std::make_shared<op::Parameter>(element::f32, s.rois_shape);
auto deltas = std::make_shared<op::Parameter>(element::f32, s.deltas_shape);
auto scores = std::make_shared<op::Parameter>(element::f32, s.scores_shape);
auto im_info = std::make_shared<op::Parameter>(element::f32, s.im_info_shape);
auto detection =
std::make_shared<ExperimentalDetection>(rois, deltas, scores, im_info, attrs);
ASSERT_EQ(detection->get_output_element_type(0), element::f32);
ASSERT_EQ(detection->get_output_element_type(1), element::i32);
ASSERT_EQ(detection->get_output_element_type(2), element::f32);
EXPECT_EQ(detection->get_output_shape(0), (Shape{rois_num, 4}));
EXPECT_EQ(detection->get_output_shape(1), (Shape{rois_num}));
EXPECT_EQ(detection->get_output_shape(2), (Shape{rois_num}));
}
}

178
ngraph/test/type_prop/experimental_detectron_prior_grid_generator.cpp Normal file
View File

@ -0,0 +1,178 @@
//*****************************************************************************
// Copyright 2017-2021 Intel Corporation
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//*****************************************************************************
#include <vector>
#include "gtest/gtest.h"
#include "ngraph/ngraph.hpp"
#include "util/type_prop.hpp"
using namespace ngraph;
using Attrs = op::v6::ExperimentalDetectronPriorGridGenerator::Attributes;
using GridGenerator = op::v6::ExperimentalDetectronPriorGridGenerator;
TEST(type_prop, detectron_grid_generator_static_shape_flatten)
{
Attrs attrs;
attrs.flatten = true;
attrs.h = 0;
attrs.w = 0;
attrs.stride_x = 4.0f;
attrs.stride_y = 4.0f;
auto priors = std::make_shared<op::Parameter>(element::f32, Shape{3, 4});
auto feature_map = std::make_shared<op::Parameter>(element::f32, Shape{1, 256, 200, 336});
auto im_data = std::make_shared<op::Parameter>(element::f32, Shape{1, 3, 800, 1344});
auto grid_gen = std::make_shared<GridGenerator>(priors, feature_map, im_data, attrs);
ASSERT_EQ(grid_gen->get_output_element_type(0), element::f32);
EXPECT_EQ(grid_gen->get_output_shape(0), (Shape{201600, 4}));
}
TEST(type_prop, detectron_grid_generator_static_shape_without_flatten)
{
Attrs attrs;
attrs.flatten = false;
attrs.h = 0;
attrs.w = 0;
attrs.stride_x = 4.0f;
attrs.stride_y = 4.0f;
auto priors = std::make_shared<op::Parameter>(element::f32, Shape{3, 4});
auto feature_map = std::make_shared<op::Parameter>(element::f32, Shape{1, 256, 200, 336});
auto im_data = std::make_shared<op::Parameter>(element::f32, Shape{1, 3, 800, 1344});
auto grid_gen = std::make_shared<GridGenerator>(priors, feature_map, im_data, attrs);
ASSERT_EQ(grid_gen->get_output_element_type(0), element::f32);
EXPECT_EQ(grid_gen->get_output_shape(0), (Shape{200, 336, 3, 4}));
}
TEST(type_prop, detectron_grid_generator_dynamic_shapes)
{
Attrs attrs;
attrs.flatten = false;
attrs.h = 0;
attrs.w = 0;
attrs.stride_x = 4.0f;
attrs.stride_y = 4.0f;
struct ShapesAndAttrs
{
PartialShape priors_shape;
PartialShape feature_map_shape;
PartialShape ref_out_shape;
bool flatten;
};
const Shape im_data_shape = Shape{1, 3, 800, 1344};
const auto dyn_dim = Dimension::dynamic();
std::vector<ShapesAndAttrs> shapes = {
{{3, 4}, {1, 256, 200, dyn_dim}, {dyn_dim, 4}, true},
{{3, 4}, {1, 256, dyn_dim, 336}, {dyn_dim, 4}, true},
{{3, 4}, {1, 256, dyn_dim, dyn_dim}, {dyn_dim, 4}, true},
{{dyn_dim, 4}, {1, 256, 200, dyn_dim}, {dyn_dim, 4}, true},
{{dyn_dim, 4}, {1, 256, dyn_dim, 336}, {dyn_dim, 4}, true},
{{dyn_dim, 4}, {1, 256, dyn_dim, dyn_dim}, {dyn_dim, 4}, true},
{{3, 4}, {1, 256, 200, dyn_dim}, {200, dyn_dim, 3, 4}, false},
{{3, 4}, {1, 256, dyn_dim, 336}, {dyn_dim, 336, 3, 4}, false},
{{3, 4}, {1, 256, dyn_dim, dyn_dim}, {dyn_dim, dyn_dim, 3, 4}, false},
{{dyn_dim, 4}, {1, 256, 200, dyn_dim}, {200, dyn_dim, dyn_dim, 4}, false},
{{dyn_dim, 4}, {1, 256, dyn_dim, 336}, {dyn_dim, 336, dyn_dim, 4}, false},
{{dyn_dim, 4}, {1, 256, dyn_dim, dyn_dim}, {dyn_dim, dyn_dim, dyn_dim, 4}, false}};
for (const auto& s : shapes)
{
auto grid_attrs = attrs;
grid_attrs.flatten = s.flatten;
auto priors = std::make_shared<op::Parameter>(element::f32, s.priors_shape);
auto feature_map = std::make_shared<op::Parameter>(element::f32, s.feature_map_shape);
auto im_data = std::make_shared<op::Parameter>(element::f32, im_data_shape);
auto grid_gen = std::make_shared<GridGenerator>(priors, feature_map, im_data, grid_attrs);
ASSERT_EQ(grid_gen->get_output_element_type(0), element::f32);
ASSERT_TRUE(grid_gen->get_output_partial_shape(0).same_scheme(s.ref_out_shape));
}
}
TEST(type_prop, detectron_grid_generator_dynamic_shapes_intervals)
{
Attrs attrs;
attrs.flatten = false;
attrs.h = 0;
attrs.w = 0;
attrs.stride_x = 4.0f;
attrs.stride_y = 4.0f;
struct ShapesAndAttrs
{
PartialShape priors_shape;
PartialShape feature_map_shape;
PartialShape ref_out_shape;
bool flatten;
};
const Shape im_data_shape = Shape{1, 3, 800, 1344};
std::vector<ShapesAndAttrs> shapes = {
{{3, 4}, {1, 256, 200, Dimension(0, 100)}, {Dimension(0, 60000), 4}, true},
{{3, 4}, {1, 256, Dimension(0, 150), 336}, {Dimension(0, 151200), 4}, true},
{{3, 4}, {1, 256, Dimension(0, 150), Dimension(0, 100)}, {Dimension(0, 45000), 4}, true},
{{Dimension(0, 3), 4}, {1, 256, 200, Dimension(0, 150)}, {Dimension(0, 90000), 4}, true},
{{Dimension(0, 3), 4}, {1, 256, Dimension(0, 150), 336}, {Dimension(0, 151200), 4}, true},
{{Dimension(0, 3), 4},
{1, 256, Dimension(0, 150), Dimension(0, 100)},
{Dimension(0, 45000), 4},
true},
{{3, 4}, {1, 256, 200, Dimension(0, 100)}, {200, Dimension(0, 100), 3, 4}, false},
{{3, 4}, {1, 256, Dimension(0, 150), 336}, {Dimension(0, 150), 336, 3, 4}, false},
{{3, 4},
{1, 256, Dimension(0, 150), Dimension(0, 100)},
{Dimension(0, 150), Dimension(0, 100), 3, 4},
false},
{{Dimension(0, 3), 4},
{1, 256, 200, Dimension(0, 100)},
{200, Dimension(0, 100), Dimension(0, 3), 4},
false},
{{Dimension(0, 3), 4},
{1, 256, Dimension(0, 150), 336},
{Dimension(0, 150), 336, Dimension(0, 3), 4},
false},
{{Dimension(0, 3), 4},
{1, 256, Dimension(0, 150), Dimension(0, 100)},
{Dimension(0, 150), Dimension(0, 100), Dimension(0, 3), 4},
false}};
for (const auto& s : shapes)
{
auto grid_attrs = attrs;
grid_attrs.flatten = s.flatten;
auto priors = std::make_shared<op::Parameter>(element::f32, s.priors_shape);
auto feature_map = std::make_shared<op::Parameter>(element::f32, s.feature_map_shape);
auto im_data = std::make_shared<op::Parameter>(element::f32, im_data_shape);
auto grid_gen = std::make_shared<GridGenerator>(priors, feature_map, im_data, grid_attrs);
ASSERT_EQ(grid_gen->get_output_element_type(0), element::f32);
ASSERT_TRUE(grid_gen->get_output_partial_shape(0).same_scheme(s.ref_out_shape));
}
}