IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add support for onnx FasterRCNN model (#3393)

Browse Source 
* Add support for onnx FasterRCNN model

* Add documentation
This commit is contained in:
Maxim Vafin 2020-12-01 18:45:08 +03:00 committed by GitHub
parent 86e5461d4b
commit bbb7478460
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
7 changed files with 67 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md
View File

@ -85,6 +85,7 @@ Model Optimizer produces an Intermediate Representation (IR) of the network, whi
* [Converting a Style Transfer Model from MXNet](prepare_model/convert_model/mxnet_specific/Convert_Style_Transfer_From_MXNet.md)
* [Converting Your Kaldi* Model](prepare_model/convert_model/Convert_Model_From_Kaldi.md)
* [Converting Your ONNX* Model](prepare_model/convert_model/Convert_Model_From_ONNX.md)
* [Converting Faster-RCNN ONNX* Model](prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md)
* [Converting Mask-RCNN ONNX* Model](prepare_model/convert_model/onnx_specific/Convert_Mask_RCNN.md)
* [Converting DLRM ONNX* Model](prepare_model/convert_model/onnx_specific/Convert_DLRM.md)
* [Model Optimizations Techniques](prepare_model/Model_Optimization_Techniques.md)

18
docs/MO_DG/prepare_model/convert_model/onnx_specific/Convert_Faster_RCNN.md Normal file
View File

@ -0,0 +1,18 @@
# Convert ONNX* Faster R-CNN Model to the Intermediate Representation {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Faster_RCNN}
These instructions are applicable only to the Faster R-CNN model converted to the ONNX* file format from the [facebookresearch/maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark).
**Step 1**. Download the pre-trained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn) (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117).
**Step 2**. To generate the Intermediate Representation (IR) of the model, change your current working directory to the Model Optimizer installation directory and run the Model Optimizer with the following parameters:
```sh
python3 ./mo_onnx.py
--input_model FasterRCNN-10.onnx \
--input_shape [3,800,800] \
--mean_values [102.9801,115.9465,122.7717] \
--transformations_config ./extensions/front/onnx/faster_rcnn.json
```
Note that the height and width specified with the `input_shape` command line parameter could be different. Refer to the [documentation](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn) for more information about supported input image dimensions and required pre- and post-processing steps.
**Step 3**. Interpret the outputs. The generated IR file has several outputs: class indices, probabilities and box coordinates. These are outputs from the "DetectionOutput" layer.

1
docs/doxygen/ie_docs.xml
View File

@ -32,6 +32,7 @@
<tab type="user" title="Convert Kaldi* ASpIRE Chain Time Delay Neural Network (TDNN) Model to the Intermediate Representation" url="@ref openvino_docs_MO_DG_prepare_model_convert_model_kaldi_specific_Aspire_Tdnn_Model"/>
</tab>
<tab type="usergroup" title="Converting Your ONNX* Model" url="@ref openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX">
<tab type="user" title="Convert ONNX* Faster R-CNN Model to the Intermediate Representation" url="@ref openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Faster_RCNN"/>
<tab type="user" title="Convert ONNX* Mask R-CNN Model to the Intermediate Representation" url="@ref openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Mask_RCNN"/>
<tab type="user" title="Converting DLRM ONNX* Model" url="@ref openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_DLRM"/>
</tab>

1
model-optimizer/automation/package_BOM.txt
View File

@ -257,6 +257,7 @@ extensions/front/onnx/detectionoutput_ext.py
extensions/front/onnx/dropout_ext.py
extensions/front/onnx/elementwise_ext.py
extensions/front/onnx/expand_ext.py
extensions/front/onnx/faster_rcnn.json
extensions/front/onnx/flatten_ext.py
extensions/front/onnx/flattenONNX_to_reshape.py
extensions/front/onnx/gather_ext.py

20
model-optimizer/extensions/front/onnx/faster_rcnn.json Normal file
View File

@ -0,0 +1,20 @@
[
{
"custom_attributes":
{
"max_detections_per_image": 100,
"max_delta_log_wh": 4.135166645050049,
"score_threshold": 0.05,
"nms_threshold": 0.5,
"post_nms_count": 2000,
"input_fpn_heads": ["486", "454", "422", "390"],
"do_outputs": ["6371", "6373", "6375"],
"box_regressions_input_node": "2614",
"class_predicitons_node": "2615",
"ROIFeatureExtractor2_input": "2335",
"ROIFeatureExtractor2_output": "2592"
},
"id": "ONNXMaskRCNNReplacement",
"match_kind": "general"
}
]

9
model-optimizer/extensions/front/onnx/mask_rcnn.json
View File

@ -6,7 +6,14 @@
"max_delta_log_wh": 4.135166645050049,
"score_threshold": 0.05,
"nms_threshold": 0.5,
"post_nms_count": 2000
"post_nms_count": 2000,
"input_fpn_heads": ["486", "454", "422", "390"],
"do_outputs": ["6530", "6532", "6534"],
"box_regressions_input_node": "2773",
"class_predicitons_node": "2774",
"ROIFeatureExtractor1_output": "6795",
"ROIFeatureExtractor2_input": "2490",
"ROIFeatureExtractor2_output": "2751"
},
"id": "ONNXMaskRCNNReplacement",
"match_kind": "general"

32
model-optimizer/extensions/front/onnx/mask_rcnn_conversion.py
View File

@ -28,8 +28,6 @@ from mo.graph.graph import Graph
from mo.graph.graph import Node
from mo.ops.reshape import Reshape
input_fpn_heads = ('486', '454', '422', '390')
class ONNXMaskRCNNTransformation(FrontReplacementFromConfigFileGeneral):
"""
@ -45,24 +43,25 @@ class ONNXMaskRCNNTransformation(FrontReplacementFromConfigFileGeneral):
replacement_id = 'ONNXMaskRCNNReplacement'
def run_before(self):
# the node "2774" which is used in this transformation is of op SoftMaxONNX. But operations of op SoftMaxONNX
# the class_predicitons_node which is used in this transformation is of op SoftMaxONNX. But operations of op SoftMaxONNX
# will be replaced with a transformation SoftmaxONNXFrontReplacer
return [SoftmaxONNXFrontReplacer]
def transform_graph(self, graph: Graph, replacement_descriptions: dict):
insert_ExperimentalDetectronROIFeatureExtractor2(graph)
insert_ExperimentalDetectronROIFeatureExtractor2(graph, replacement_descriptions)
insert_do(graph, replacement_descriptions)
insert_ExperimentalDetectronROIFeatureExtractor1(graph)
insert_ExperimentalDetectronROIFeatureExtractor1(graph, replacement_descriptions)
def insert_do(graph: Graph, replacement_descriptions):
do_outputs = ['6530', '6532', '6534']
def insert_do(graph: Graph, replacement_descriptions: dict):
do_outputs = replacement_descriptions['do_outputs']
prior_boxes_node = Node(graph, 'ROIFeatureExtractor_2')
num_classes = 81
box_regressions_input_node = Node(graph, replacement_descriptions['box_regressions_input_node'])
box_regressions_node = create_op_node_with_second_input(graph, Reshape, int64_array([-1, 4 * num_classes]),
dict(name='box_regressions'), Node(graph, '2773'))
dict(name='box_regressions'), box_regressions_input_node)
class_predicitons_node = Node(graph, '2774')
class_predicitons_node = Node(graph, replacement_descriptions['class_predicitons_node'])
im_info_node = Parameter(graph, {"name": 'im_info', 'shape': int64_array([1, 3])}).create_node()
do_node = ExperimentalDetectronDetectionOutput(graph, {'name': 'DetectionOutput',
@ -92,8 +91,12 @@ def insert_do(graph: Graph, replacement_descriptions):
do_node.out_port(1).get_connection().insert_node(Cast(graph, {'dst_type': np.int64}).create_node())
def insert_ExperimentalDetectronROIFeatureExtractor1(graph: Graph):
old_output_node = Node(graph, '6795')
def insert_ExperimentalDetectronROIFeatureExtractor1(graph: Graph, replacement_descriptions: dict):
if 'ROIFeatureExtractor1_output' not in replacement_descriptions:
# In case of Faster-RCNN this transformation is not needed and this attribute shouldn't be set
return
input_fpn_heads = replacement_descriptions['input_fpn_heads']
old_output_node = Node(graph, replacement_descriptions['ROIFeatureExtractor1_output'])
input_fpn_head_nodes = [Node(graph, node_id) for node_id in input_fpn_heads]
fpn_roi_align = ExperimentalDetectronROIFeatureExtractor(graph, {'name': 'ROIFeatureExtractor_1',
'distribute_rois_between_levels': 1,
@ -110,8 +113,9 @@ def insert_ExperimentalDetectronROIFeatureExtractor1(graph: Graph):
old_output_node.out_port(0).get_connection().set_source(fpn_roi_align.out_port(0))
def insert_ExperimentalDetectronROIFeatureExtractor2(graph: Graph):
old_output_node = Node(graph, '2751')
def insert_ExperimentalDetectronROIFeatureExtractor2(graph: Graph, replacement_descriptions: dict):
input_fpn_heads = replacement_descriptions['input_fpn_heads']
old_output_node = Node(graph, replacement_descriptions['ROIFeatureExtractor2_output'])
input_fpn_head_nodes = [Node(graph, node_id) for node_id in input_fpn_heads]
fpn_roi_align = ExperimentalDetectronROIFeatureExtractor(graph, {'name': 'ROIFeatureExtractor_2',
'distribute_rois_between_levels': 1,
@ -121,7 +125,7 @@ def insert_ExperimentalDetectronROIFeatureExtractor2(graph: Graph):
'pyramid_scales': int64_array(
[4, 8, 16, 32, 64]),
'sampling_ratio': 2, }).create_node()
fpn_roi_align.in_port(0).connect(Node(graph, '2490').out_port(0))
fpn_roi_align.in_port(0).connect(Node(graph, replacement_descriptions['ROIFeatureExtractor2_input']).out_port(0))
for ind, fpn_node in enumerate(input_fpn_head_nodes):
fpn_roi_align.in_port(ind + 1).connect(fpn_node.out_port(0))