ONNX Loop operation support (#2756)

* Generate TensorIterator without back edges from TensorFlow models

* Added a check in the MarkSubgraphsWithCorrectLayout to not fail when port is not connected

* Updated the 'protobuf2nx' to consume the graph protobuf message

* Cleanup TI from the IRv7 specific code

* Do not run some front transformations recursively

* Draft support for the ONNX Loop operation when 'cond' = True

* LoopToTI transformation changes

* Added draft of Loop operation and parser for ONNX Loop operation body

* Updated Loop body parser + added shape and type infer for the Loop operation

* Fixes for ONNX Loop operation parser

* Moved Loop parsing to Loop op extractor. Added generation of external edges for the Loop body ops

* Added support for ThresholdedRelu using decomposition

* Added support for Min ONNX operation

* Draft fixes for port_map generation for the Loop

* Rename transformation file and fix BOM

* Fixed shape inference for Loop scan outputs (axis is not None)

* Fixed shape inference for ONNX Loop operation

* Refactor checks in the TensorIteratorMerge transformation

* Code refactoring. Enabled commented transformations

* Documentation update for ONNX Loop, ThresholdedRelu and Min

* Fixed typo in the Loop front transformation where execution condition input is connected. Other refactorings

* Fixed in the Loop extractor

* Added printing 'internal_layer_id' attribute in the graph dumper

* Updated calculation of iterations number for the Loop

* Added missing code

* Fixed output port shapes and types generation for Loop operation

* Update function names and variable names in the Loop operation

* Fixed type inference for iteration count input

* Added removal of input/output ports of the Loop if they are not used

* Fixed renumbering Loop operations input/output ports to keep mandatory

* Fixed ThresholdedReluDecomposition transformation

* Updated MO IR Reader to know about Loop operation. But it is still not supported by the MO IR Reader

* Added unit test for Slice op shape infer (reverse the sequence of elements)

* Reverted changes in the ONNX loader function call to protobuf2nx

* Enable Reshape0DToSqueeze transformation recursively

* Refactored Loop operation support implementation

* Changed ThresholdedReluDecomposition to generate Const with shape [1] instead of scalar

* Code style and wording fixes

* Restored accidentally removed 'return' statement in the TI shape infer function

* Fixed comments

* Fixed comment

Co-authored-by: Evgeny Lazarev <elazarev.nnov@gmail.com>

docs/MO_DG/prepare_model/Supported_Frameworks_Layers.md

@@ -357,9 +357,11 @@ Standard ONNX\* operators:
 | LogicalAnd | No |
 | LogicalOr | No |
 | LogSoftmax | No |
+| Loop | No |
 | MatMul | No |
 | MaxPool | No |
 | MeanVarianceNormalization | Reduction over the batch dimension is not supported, reduction over all dimensions except batch and channel ones is obligatory |
+| Min | No |
 | Mul | No |
 | Neg | No |
 | NonMaxSuppression | No |
@@ -406,6 +408,7 @@ Standard ONNX\* operators:
 | Sum | No |
 | Tan | No |
 | Tanh | No |
+| ThresholdedRelu | No |
 | TopK | No |
 | Transpose | No |
 | Unsqueeze | No |

docs/ops/infrastructure/Loop_5.md

@@ -85,7 +85,7 @@ Loop operation description in the IR also has several special sections: `body`,
     * **Port map attributes**:
 
         * *external_port_id*
-            * **Description**: *external_port_id* is a port ID of the `Loop` operation.
+            * **Description**: *external_port_id* is a port ID of the `Loop` operation. The value `-1` means that the body node is not connected to the `Loop` operation.
             * **Range of values**: IDs of the *Loop* outputs
             * **Type**: `int`
             * **Default value**: None

model-optimizer/automation/package_BOM.txt

@@ -266,6 +266,8 @@ extensions/front/onnx/hard_sigmoid_ext.py
 extensions/front/onnx/image_scaler_ext.py
 extensions/front/onnx/instance_normalization_ext.py
 extensions/front/onnx/logsoftmaxONNX_to_logsoftmax.py
+extensions/front/onnx/loop_ext.py
+extensions/front/onnx/LoopNormalize.py
 extensions/front/onnx/lp_normalization_ext.py
 extensions/front/onnx/lrn_ext.py
 extensions/front/onnx/lstm_ext.py
@@ -476,6 +478,7 @@ extensions/front/tf/yolo_v2_voc.json
 extensions/front/tf/yolo_v3.json
 extensions/front/tf/yolo_v3_tiny.json
 extensions/front/tf/yolo_v3_voc.json
+extensions/front/ThresholdedReluDecomposition.py
 extensions/front/TopKNormalize.py
 extensions/front/transformations_config.py
 extensions/front/TransposeOrderNormalizer.py
@@ -633,6 +636,7 @@ extensions/ops/instance_normalization.py
 extensions/ops/interp.py
 extensions/ops/interpolate.py
 extensions/ops/LookupTableInsert.py
+extensions/ops/loop.py
 extensions/ops/LSTM.py
 extensions/ops/lstm_cell.py
 extensions/ops/lstm_sequence.py

model-optimizer/extensions/back/Reshape0DToSqueeze.py

@@ -31,7 +31,6 @@ class Reshape0DToSqueeze(BackReplacementPattern):
     overcome issue the IE doesn't 1D constants with value [0] which is generated for the Reshape to OD case.
     """
     enabled = True
-    run_not_recursively = True
     force_shape_inference = True
 
     def run_before(self):

model-optimizer/extensions/front/ChangePlaceholderTypes.py

@@ -24,6 +24,7 @@ from mo.graph.graph import Graph, Node
 
 class ChangePlaceholderTypes(FrontReplacementPattern):
     enabled = True
+    run_not_recursively = True
 
     @staticmethod
     def is_node_casts_to_float_or_shapeof(node: Node):

model-optimizer/extensions/front/ThresholdedReluDecomposition.py

@@ -0,0 +1,51 @@
+"""
+ Copyright (C) 2018-2020 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+
+from extensions.ops.Cast import Cast
+from extensions.ops.elementwise import Greater, Mul
+from mo.front.common.partial_infer.utils import float_array
+from mo.front.common.replacement import FrontReplacementPattern
+from mo.front.tf.graph_utils import create_op_with_const_inputs
+from mo.graph.graph import Graph, rename_nodes
+from mo.middle.passes.convert_data_type import data_type_str_to_np
+
+
+class ThresholdedReluDecomposition(FrontReplacementPattern):
+    """
+    ThresholdedRelu(x, alpha) = x ? x > alpha : 0
+
+    is replaced with
+
+    ThresholdedRelu(x, alpha) = Mul(x, Cast(Greater(x, alpha), type=float))
+    """
+    enabled = True
+
+    def find_and_replace_pattern(self, graph: Graph):
+        for node in graph.get_op_nodes(op='ThresholdedRelu'):
+            name = node.soft_get('name', node.id)
+
+            greater = create_op_with_const_inputs(graph, Greater, {1: float_array([node.alpha])})
+            greater.in_port(0).connect(node.in_port(0).get_source())
+            float_greater = Cast(graph,
+                                 {'dst_type': data_type_str_to_np(graph.graph['cmd_params'].data_type)}).create_node()
+            greater.out_port(0).connect(float_greater.in_port(0))
+
+            mul = Mul(graph, {}).create_node()
+            node.out_port(0).get_connection().set_source(mul.out_port(0))
+            mul.in_port(0).connect(node.in_port(0).get_source())
+            mul.in_port(1).connect(float_greater.out_port(0))
+
+            rename_nodes([(node, name + '/TBR'), (mul, name)])

model-optimizer/extensions/front/ThresholdedReluDecomposition_test.py

@@ -0,0 +1,60 @@
+"""
+ Copyright (C) 2020 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+import unittest
+
+from extensions.front.ThresholdedReluDecomposition import ThresholdedReluDecomposition
+from mo.front.common.partial_infer.utils import float_array
+from mo.utils.ir_engine.compare_graphs import compare_graphs
+from mo.utils.unittest.graph import build_graph, const
+
+nodes_attributes = {
+    'parameter': {'shape': None, 'type': 'Parameter', 'kind': 'op', 'op': 'Parameter'},
+    'trelu': {'type': None, 'kind': 'op', 'op': 'ThresholdedRelu', 'alpha': 0.75, 'name': 'my_trelu'},
+    'result': {'type': 'Result', 'value': None, 'kind': 'op', 'op': 'Result'},
+
+    'cast': {'type': 'Convert', 'kind': 'op', 'op': 'Cast'},
+    'greater': {'type': 'Greater', 'kind': 'op', 'op': 'Greater'},
+    'mul': {'type': 'Multiply', 'kind': 'op', 'op': 'Mul', 'name': 'my_trelu'},
+    'squeeze2': {'type': 'Squeeze', 'kind': 'op', 'op': 'Squeeze'},
+    **const('alpha', float_array([0.75])),
+}
+
+
+class ThresholdedReluDecompositionTest(unittest.TestCase):
+    def test_trelu(self):
+        graph = build_graph(nodes_attributes,
+                            [('parameter', 'trelu', {'in': 0, 'out': 0}),
+                             ('trelu', 'result', {'in': 0, 'out': 0}),
+                             ], nodes_with_edges_only=True)
+
+        graph_ref = build_graph(nodes_attributes,
+                                [('parameter', 'greater', {'in': 0, 'out': 0}),
+                                 ('alpha', 'greater', {'in': 1, 'out': 0}),
+                                 ('greater', 'cast', {'in': 0, 'out': 0}),
+                                 ('parameter', 'mul', {'in': 0, 'out': 0}),
+                                 ('cast', 'mul', {'in': 1, 'out': 0}),
+                                 ('mul', 'result', {'in': 0, 'out': 0}),
+                                 ], nodes_with_edges_only=True)
+
+        graph.graph['layout'] = 'NCHW'
+        graph.stage = 'front'
+
+        ThresholdedReluDecomposition().find_and_replace_pattern(graph)
+
+        (flag, resp) = compare_graphs(graph, graph_ref, 'result', check_op_attrs=True)
+        self.assertTrue(flag, resp)
+        self.assertTrue(len(graph.get_op_nodes(name='my_trelu')) == 1 and
+                        graph.get_op_nodes(name='my_trelu')[0].op == 'Mul')

model-optimizer/extensions/front/eltwise_n.py

@@ -15,7 +15,7 @@
 """
 
 
-from extensions.ops.elementwise import Add, Maximum, Mul
+from extensions.ops.elementwise import Add, Maximum, Minimum, Mul
 from mo.front.common.replacement import FrontReplacementOp
 from mo.graph.graph import Node, Graph
 
@@ -30,6 +30,7 @@ class EltwiseNReplacement(FrontReplacementOp):
 
     op_to_class_map = {
         'sum': Add,
+        'min': Minimum,
         'max': Maximum,
         'mul': Mul,
     }

model-optimizer/extensions/front/freeze_placeholder_value.py

@@ -31,6 +31,7 @@ class FreezePlaceholderValue(FrontReplacementSubgraph):
     a string and casts it to actual node data type
     """
     enabled = True
+    run_not_recursively = True
     graph_condition = [lambda graph: graph.graph['freeze_placeholder'] is not None]
 
     def run_after(self):

model-optimizer/extensions/front/onnx/LoopNormalize.py

@@ -0,0 +1,67 @@
+"""
+ Copyright (C) 2018-2020 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+import numpy as np
+
+from extensions.front.pass_separator import FrontStart
+from extensions.front.restore_ports import RestorePorts
+from extensions.ops.loop import Loop
+from mo.front.common.partial_infer.utils import int64_array
+from mo.front.common.replacement import FrontReplacementSubgraph
+from mo.front.tf.graph_utils import create_op_with_const_inputs
+from mo.graph.graph import Graph, Node
+from mo.ops.const import Const
+from mo.ops.unsqueeze import Unsqueeze
+
+
+class ONNXLoopNormalize(FrontReplacementSubgraph):
+    enabled = True
+
+    def run_before(self):
+        return [FrontStart]
+
+    def run_after(self):
+        return [RestorePorts]
+
+    def find_and_replace_pattern(self, graph: Graph):
+        for node in graph.get_op_nodes(op='Loop'):
+            self.normalize_body_graph(node)
+
+    @staticmethod
+    def normalize_body_graph(loop_node: Node):
+        loop_name = loop_node.soft_get('name', loop_node.id)
+        # connect "trip count" input if it is not connected with default value "Infinity" (-1)
+        if not loop_node.is_in_port_connected(0):
+            loop_node.add_input_port(0, skip_if_exist=True)
+            Const(loop_node.graph, {'name': loop_name + '/trip_count', 'value': int64_array(-1)}).\
+                create_node().out_port(0).connect(loop_node.in_port(0))
+
+        # connect "execution condition" input if it is not connected with default value True
+        if not loop_node.is_in_port_connected(1):
+            loop_node.add_input_port(1, skip_if_exist=True)
+            Const(loop_node.graph, {'name': loop_name + '/execution_cond', 'value': np.array(True, dtype=np.bool)}).\
+                create_node().out_port(0).connect(loop_node.in_port(1))
+
+        # scan output need Unsqueeze over axis 0
+        for record in loop_node.output_port_map:
+            body_node = Loop.get_body_node_by_internal_id(loop_node, record['internal_layer_id'])
+            assert body_node is not None
+            assert body_node.soft_get('type') == 'Result'
+
+            if record['axis'] is not None:
+                unsqueeze = create_op_with_const_inputs(loop_node.body, Unsqueeze, {1: int64_array([0])})
+                body_node.in_port(0).get_connection().insert_node(unsqueeze)
+
+        Loop.normalize_input_output_ports(loop_node)

model-optimizer/extensions/front/onnx/activation_ext.py

@@ -159,6 +159,17 @@ class FloorExtractor(FrontExtractorOp):
         return cls.enabled
 
 
+class ThresholdedReluExtractor(FrontExtractorOp):
+    op = 'ThresholdedRelu'
+    enabled = True
+
+    @classmethod
+    def extract(cls, node):
+        alpha = onnx_attr(node, 'alpha', 'f', default=1.0)
+        ThresholdedRelu.update_node_stat(node, {'alpha': alpha})
+        return cls.enabled
+
+
 class LeakyReLUExtractor(FrontExtractorOp):
     op = 'LeakyRelu'
     enabled = True

model-optimizer/extensions/front/onnx/elementwise_ext.py

@@ -15,12 +15,12 @@
 """
 import numpy as np
 
-from extensions.ops.elementwise import Add, Sub, Mul, Div, Pow, Less, Equal, Greater, \
-    LogicalAnd, LogicalOr, LogicalXor, Round
+from extensions.ops.elementwise import Add, Sub, Mul, Div, Pow, Less, Equal, Greater, LogicalAnd, LogicalOr, LogicalXor, \
+    Round
 from mo.front.extractor import FrontExtractorOp
 from mo.front.onnx.extractors.utils import onnx_attr
 from mo.graph.graph import Node
-from mo.ops.eltwise_n import EltwiseNAdd, EltwiseNMax
+from mo.ops.eltwise_n import EltwiseNAdd, EltwiseNMax, EltwiseNMin
 from mo.ops.power import AttributedPower
 
 
@@ -130,6 +130,15 @@ class MaxExtractor(FrontExtractorOp):
         return cls.enabled
 
 
+class MinExtractor(FrontExtractorOp):
+    op = 'Min'
+    enabled = True
+
+    @classmethod
+    def extract(cls, node: Node):
+        EltwiseNMin.update_node_stat(node)
+        return cls.enabled
+
 class EqualExtractor(FrontExtractorOp):
     op = 'Equal'
     enabled = True

model-optimizer/extensions/front/onnx/loop_ext.py

@@ -0,0 +1,209 @@
+"""
+ Copyright (C) 2018-2020 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+import copy
+import logging as log
+
+from extensions.ops.loop import Loop
+from extensions.ops.parameter import Parameter
+from mo.front.common.register_custom_ops import check_for_duplicates
+from mo.front.extractor import FrontExtractorOp
+from mo.front.extractor import extract_node_attrs
+from mo.front.onnx.extractor import onnx_op_extractor, onnx_op_extractors
+from mo.front.onnx.extractors.utils import onnx_attr
+from mo.front.onnx.loader import node_id, add_initializers_and_inputs_to_graph
+from mo.graph.graph import Graph, Node, add_opoutput
+from mo.utils.error import Error
+
+
+def connect_body_output(loop_node: Node, loop_output_port_idx: int, internal_result: Node, axis: [int, None] = None,
+                        start: [int, None] = None, end: [int, None] = None, stride: [int, None] = None,
+                        part_size: [int, None] = None):
+    assert loop_node.soft_get('op') == 'Loop'
+    assert internal_result.soft_get('op') == 'Result'
+    assert internal_result.id in loop_node.body
+
+    loop_node.output_port_map.append({'axis': axis, 'stride': stride, 'part_size': part_size, 'start': start,
+                                      'end': end, 'external_port_id': loop_output_port_idx,
+                                      'internal_layer_id': internal_result['internal_layer_id']})
+
+
+def connect_body_input(loop_node: Node, loop_input_port_idx: int, body_parameter: Node,
+                       axis: [int, None] = None, start: [int, None] = None, end: [int, None] = None,
+                       stride: [int, None] = None, part_size: [int, None] = None):
+    assert loop_node.soft_get('op') == 'Loop'
+    assert body_parameter.soft_get('op') == 'Parameter'
+    assert body_parameter.id in loop_node.body
+
+    loop_node.input_port_map.append({'axis': axis, 'stride': stride, 'part_size': part_size, 'start': start,
+                                     'end': end, 'external_port_id': loop_input_port_idx,
+                                     'internal_layer_id': body_parameter['internal_layer_id']})
+
+
+class LoopExtractor(FrontExtractorOp):
+    op = 'Loop'
+    enabled = True
+
+    @classmethod
+    def extract(cls, loop_node):
+        Loop.update_node_stat(loop_node, {})
+
+        body_graph_proto = onnx_attr(loop_node, 'body', 'g', None)
+        main_graph = loop_node.graph
+
+        # create a Graph object for the body and take graph attributes from the main graph
+        body_graph = Graph()
+        main_graph_attrs_copy = copy.deepcopy(main_graph.graph)
+        del main_graph_attrs_copy['tensor_mapping']
+        body_graph.graph.update(main_graph_attrs_copy)
+        loop_node['body'] = body_graph
+
+        # maps a tensor name to a node produced it and the node port: str -> (node_id, node_port)
+        data_nodes_map = {}
+        body_graph.graph['tensor_mapping'] = data_nodes_map  # save mapping for possible Loop inside the Loop
+
+        body_parameters = add_initializers_and_inputs_to_graph(body_graph, body_graph_proto, data_nodes_map)
+
+        external_edges = []  # (src_node, src_out_port), dest_body_parameter_node
+        additional_params = {}  # (src_node, src_out_port) -> parameter_node (for manually added Parameters)
+        # Go through all nodes in the original model order because data nodes are defined on-the-fly and order matters
+        for pb_node in body_graph_proto.node:
+            # create an NX node
+            id = body_graph.unique_id(node_id(pb_node))
+            body_graph.add_node(id, pb=pb_node, kind='op')
+
+            # add incoming edges based on data_nodes_map
+            for dst_port, inp in enumerate(pb_node.input):
+                # should add edge inp --> id
+                if inp not in data_nodes_map:
+                    if inp == '':
+                        # input is omitted; most likely it corresponds to an optional input for an operator
+                        continue
+                    elif inp in main_graph.graph['tensor_mapping']:
+                        log.debug('The edge between outer and inner graphs detected: {} -> {}'.format(inp, id))
+                        if main_graph.graph['tensor_mapping'][inp] not in additional_params:
+                            # create new Parameter body node and connect the body node with the outer graph using it
+                            param_id = str(inp)
+                            body_graph.add_node(param_id, kind='op', op='Parameter', name=param_id, pb=None, shape=None)
+                            parameter_node = Node(body_graph, param_id)
+                            # need to manually update necessary attrs for the node because extractor will not be called
+                            # for it because the node does not have .pb attribute
+                            Parameter.update_node_stat(parameter_node, {})
+                            external_edges.append((main_graph.graph['tensor_mapping'][inp], parameter_node))
+                            src_id, src_port = param_id, 0
+                            additional_params[main_graph.graph['tensor_mapping'][inp]] = parameter_node
+                        else:
+                            src_id, src_port = additional_params[main_graph.graph['tensor_mapping'][inp]].id, 0
+                    else:
+                        raise Error('Reference to "{}" is not satisfied. A node refer not existing data tensor. ONNX '
+                                    'model is not consistent. Protobuf fragment: {}', inp, pb_node)
+                else:
+                    src_id, src_port = data_nodes_map[inp]
+
+                assert (body_graph.has_node(src_id))
+                edge_attrs = {
+                    'out': src_port,
+                    'in': dst_port,
+                    'name': inp,
+                    'fw_tensor_debug_info': [(inp, inp)],
+                    'in_attrs': ['in', 'name'],
+                    'out_attrs': ['out', 'name'],
+                    'data_attrs': ['fw_tensor_debug_info']
+                }
+                body_graph.add_edge(src_id, id, **edge_attrs)
+
+            # add outgoing edges to data_nodes_map
+            for src_port, out in enumerate(pb_node.output):
+                if out in data_nodes_map:
+                    log.debug("Detected reuse of blob {}.".format(out))
+                data_nodes_map[out] = (id, src_port)
+
+        body_results = []
+        for output in body_graph_proto.output:
+            tensor_name = str(output.name)
+            node_name, output_port = data_nodes_map[tensor_name]
+            assert body_graph.has_node(node_name), 'The body graph does not contain output with name "{}"'.format(
+                node_name)
+            body_results.append(Node(body_graph, add_opoutput(body_graph, node_name, output_port, False)))
+
+        # add 'internal_layer_id' attribute which is a must have attribute for the loop body node
+        for idx, body_node in enumerate(body_graph.get_op_nodes()):
+            body_node['internal_layer_id'] = idx
+
+        loop_carried_dependencies_count = len(body_graph_proto.input) - 2
+        scan_outputs_count = len(body_graph_proto.output) - 1 - loop_carried_dependencies_count
+
+        # Loop inputs:
+        #   0 - trip count
+        #   1 - execution condition
+        #   2 .. - loop carried dependencies
+
+        # Loop outputs:
+        #   0 .. loop_carried_dependencies_count - 1 - loop carried dependencies
+        #   loop_carried_dependencies_count .. - scan outputs
+
+        # Body inputs:
+        #   0 - iteration number
+        #   1 - execution condition
+        #   2 .. - loop carried dependencies
+
+        # Body outputs:
+        #   0 - execution condition
+        #   1 .. loop_carried_dependencies_count - loop carried dependencies
+        #   loop_carried_dependencies_count + 1 .. - scan outputs
+
+        body_graph.stage = 'front'
+        # some of the inputs/outputs may not be connected but the normalization transformation will take care of it
+        # connection Loop body nodes with external input edges
+        next_loop_input_port_idx = sorted(loop_node.in_edges().keys())[-1] + 1
+        for (src_node, src_port), body_node in external_edges:
+            main_graph.add_edge(src_node, loop_node.id, **{'out': src_port,
+                                                           'in': next_loop_input_port_idx,
+                                                           'name': src_node,
+                                                           'fw_tensor_debug_info': [(src_node, src_node)],
+                                                           'in_attrs': ['in', 'name'],
+                                                           'out_attrs': ['out', 'name'],
+                                                           'data_attrs': ['fw_tensor_debug_info']}
+                                )
+            connect_body_input(loop_node, next_loop_input_port_idx, body_node)
+            next_loop_input_port_idx += 1
+
+        # mark current iteration input Parameter node
+        Loop.mark_current_iteration_parameter_node(loop_node, body_parameters[0])
+
+        # connect initial value for "execution condition" input of the loop
+        connect_body_input(loop_node, 1, body_parameters[1])
+        # add back edge with "execution condition"
+        Loop.add_back_edge(loop_node, body_parameters[1], body_results[0])
+        # mark "execution condition" Result node
+        Loop.mark_execution_condition_result_node(loop_node, body_results[0])
+
+        # connect initial value for "loop carried" dependencies variables
+        for idx in range(loop_carried_dependencies_count):
+            connect_body_input(loop_node, idx + 2, body_parameters[idx + 2])
+        # add back edge for "loop carried" dependencies variables
+        for idx in range(loop_carried_dependencies_count):
+            Loop.add_back_edge(loop_node, body_parameters[idx + 2], body_results[idx + 1])
+        # connect final value for "loop carried" dependencies variables
+        for idx in range(loop_carried_dependencies_count):
+            connect_body_output(loop_node, idx, body_results[idx + 1])
+
+        # connect "scan outputs" and mark axis for concatenation
+        for idx in range(loop_carried_dependencies_count, loop_carried_dependencies_count + scan_outputs_count):
+            connect_body_output(loop_node, idx, body_results[idx + 1], axis=0)
+
+        # run function to parse body nodes attributes similar to the main graph
+        extract_node_attrs(body_graph, lambda node: onnx_op_extractor(node, check_for_duplicates(onnx_op_extractors)))
+        return cls.enabled

model-optimizer/extensions/front/override_batch.py

@@ -20,6 +20,7 @@ from mo.middle.passes.infer import override_batch
 
 class OverrideBatch(FrontReplacementPattern):
     enabled = True
+    run_not_recursively = True
 
     def find_and_replace_pattern(self, graph: Graph):
         override_batch(graph, graph.graph['cmd_params'].batch)

model-optimizer/extensions/load/onnx/loader.py

@@ -32,6 +32,7 @@ from mo.utils.utils import refer_to_faq_msg
 
 class ONNXLoader(Loader):
     enabled = True
+    run_not_recursively = True
 
     def load(self, graph: Graph):
         argv = graph.graph['cmd_params']

model-optimizer/extensions/middle/MarkSubgraphsWithCorrectLayout.py

@@ -51,14 +51,15 @@ class MarkSubGraphsWithCorrectLayout(MiddleReplacementPattern):
 
     @staticmethod
     def get_input_nodes(node: Node):
-        return [src_port.get_source().node for src_port in node.in_ports().values()]
+        return [src_port.get_source().node for src_port in node.in_ports().values() if not src_port.disconnected()]
 
     @staticmethod
     def get_output_nodes(node: Node):
         result = []
         for out_port in node.out_ports().values():
-            for dest_port in out_port.get_destinations():
-                result.append(dest_port.node)
+            if not out_port.disconnected():
+                for dest_port in out_port.get_destinations():
+                    result.append(dest_port.node)
         return result
 
     def bfs(self, start_nodes: list, visited: set, condition: callable = None, forward: bool = True):

model-optimizer/extensions/middle/PartialInfer.py

@@ -20,6 +20,7 @@ from mo.middle.replacement import MiddleReplacementPattern
 
 class PartialInfer(MiddleReplacementPattern):
     enabled = True
+    run_not_recursively = True
 
     def run_after(self):
         from extensions.front.create_tensor_nodes import CreateTensorNodes

model-optimizer/extensions/middle/TensorIteratorMerge.py

@@ -120,21 +120,22 @@ class TensorIteratorMerge(MiddleReplacementPattern):
     def replace_pattern(graph, match: dict):
         # Here we will found all parts of TI: condition, inputs/outputs, back edges, body and create TensorIterator Op
         # and make all checks needed for TensorIteator work
-        cond_data = match['condition'].out_node(0)
-        time_data = match['condition'].out_node(1) if len(match['condition'].out_nodes()) > 1 else None
+        cond_data = match['condition'].out_node(0) if not match['condition'].out_port(0).disconnected() else None
+        time_data = match['condition'].out_node(1) if len(match['condition'].out_nodes()) >= 1 else None
         name = match['condition'].name
 
         back_edges = []
         inputs = []
         outputs = []
 
-        for node in cond_data.out_nodes():
-            if node['kind'] == 'op' and node['op'] == 'TensorIteratorBackEdge':
-                back_edges.append(node.id)
-            elif node['kind'] == 'op' and node['op'] == 'TensorIteratorInput':
-                inputs.append(node.id)
-            elif node['kind'] == 'op' and node['op'] == 'TensorIteratorOutput':
-                outputs.append(node.id)
+        if cond_data is not None:
+            for node in cond_data.out_nodes():
+                if node['kind'] == 'op' and node['op'] == 'TensorIteratorBackEdge':
+                    back_edges.append(node.id)
+                elif node['kind'] == 'op' and node['op'] == 'TensorIteratorInput':
+                    inputs.append(node.id)
+                elif node['kind'] == 'op' and node['op'] == 'TensorIteratorOutput':
+                    outputs.append(node.id)
 
         if time_data is not None:
             for node in time_data.out_nodes():
@@ -147,12 +148,14 @@ class TensorIteratorMerge(MiddleReplacementPattern):
                     assert False
         condition = match['condition']
         tensor_sequence_length = condition.in_node(0)
-        graph.remove_nodes_from([condition.id, cond_data.id, tensor_sequence_length.id])
-        if time_data is not None:
-            graph.remove_nodes_from([time_data.id])
+
+        nodes_to_remove = [n.id for n in (condition, cond_data, time_data, tensor_sequence_length) if n is not None]
+        graph.remove_nodes_from(nodes_to_remove)
 
         body_nodes, extra_inputs = get_body(graph, inputs, outputs)
-        body_nodes = list(set(body_nodes) - set([cond_data]))
+
+        if cond_data is not None:
+            body_nodes = list(set(body_nodes) - set([cond_data]))
 
         inputs += extra_inputs
 
@@ -281,11 +284,7 @@ class TensorIteratorMerge(MiddleReplacementPattern):
                 assert not ext_inp['internal_data_id'].has_valid('value')
                 new_input_data = Op._create_data_node(body, ext_inp['internal_data_id'].name + '/UnsqueezedInput',
                                                       dict(shape=np.insert(shape, ext_inp['axis'], 1)))
-                dim = shape.copy()
-                # try to do it dynamically reshapable along one of the axis
-                # it is practically useful to reshape along batch dimension, but here we cannot detect where it is
-                # so, we are guessing based on other transformations that it is the major dimension
-                dim[0] = -1
+
                 reshape_op = Squeeze(body, dict(name=ext_inp['internal_data_id'].name + '/InputSqueeze'))
                 reshape_dim_data = Const(body, {'name': ext_inp['internal_data_id'].name + '/ReshapeDim',
                                                 'value': ext_inp['axis']}).create_node_with_data()
@@ -318,9 +317,6 @@ class TensorIteratorMerge(MiddleReplacementPattern):
 
             if ext_out['axis'] is not None:
                 # Insert unsqueezing resize at output port that has partitioning
-                dim = ext_out['internal_data_id'].shape.copy()
-                # trying to make it dynamically reshapable (see related comment above for the first Reshape)
-                dim[0] = -1
                 assert not ext_out['internal_data_id'].has_valid('value')
                 reshape_op = Unsqueeze(body, dict(name=ext_out['internal_data_id'].name + '/OutputUnsqueeze'))
                 reshape_dim_data = Const(body, {'name': ext_out['internal_data_id'].name + '/ReshapeDim',

model-optimizer/extensions/ops/activation_ops.py

@@ -180,6 +180,27 @@ class Elu(Activation):
         return ['alpha']
 
 
+class ThresholdedRelu(Activation):
+    # The operation will be decomposed to primitive operations
+    op = 'ThresholdedRelu'
+
+    def __init__(self, graph: Graph, attrs):
+        trelu_attrs = {'alpha': 1.0, 'type': None}
+        trelu_attrs.update(attrs)
+        super().__init__(graph, trelu_attrs)
+
+    @staticmethod
+    def thresholded_relu(values: np.ndarray, alpha: float):
+        values = values.astype(float)
+        for index, x in np.ndenumerate(values):
+            values[index] = values[index] * (x > alpha)
+        return values
+
+    @classmethod
+    def infer(cls, node: Node):
+        return eltwise_infer(node, lambda x, alpha: ThresholdedRelu.thresholded_relu(x, alpha), alpha=node.alpha)
+
+
 class LeakyReLU(Op):
     op = 'LeakyReLU'
 

model-optimizer/extensions/ops/loop.py

@@ -0,0 +1,437 @@
+"""
+ Copyright (C) 2017-2020 Intel Corporation
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+"""
+import logging as log
+
+import numpy as np
+
+from extensions.ops.tensor_iterator import TensorIterator
+from mo.front.common.partial_infer.utils import int64_array
+from mo.graph.graph import Node, Graph
+from mo.graph.port import Port
+from mo.middle.passes.infer import partial_infer
+from mo.ops.const import Const
+
+
+class Loop(TensorIterator):
+    """
+    Loop layer that iterates over tensors and execute embedded sub-graph. The main difference from the TensorIterator is
+    that Loop operation performs explicit slicing of data using special input called "current_iteration". Also the Loop
+    has special input determining the execution condition and special output producing execution condition for the next
+    iteration.
+    """
+    op = 'Loop'
+
+    def __init__(self, graph: Graph, attrs: dict):
+        base_attrs = {
+            'type': self.op,
+            'op': self.op,
+            'version': 'opset5',
+            'input_port_map': [],  # a list of dicts with such attrs as external_port_id, etc.
+            'output_port_map': [],  # a list of dicts with such attrs as external_port_id, etc.
+            'back_edges': [],  # a list of dicts with such attrs as from_layer, from_port, etc.
+            'body': None,  # an Graph object with a body sub-graph
+            'sub_graphs': ['body'],  # built-in attribute with all sub-graphs
+            'infer': self.infer,
+            'type_infer': self.type_infer,
+        }
+        base_attrs.update(attrs)
+        super().__init__(graph, base_attrs)
+
+    def port_map_attrs(self):
+        return super().port_map_attrs() + ['purpose']
+
+    @staticmethod
+    def generate_port_map(node: Node, src_port_map, dir: str):
+        result_list = []
+        for record in src_port_map:
+            # do not update ids for not-connected output which is used in the Loop operation only
+            if record['external_port_id'] != -1:
+                if dir == 'out':  # increase the output port id by the number of input ports
+                    # update the port id for proper generation of a "ports" section
+                    record['external_port_id'] += len(node.in_ports())
+            record['internal_layer_id'] = TensorIterator.find_internal_layer_id(node.body, record['internal_layer_id'])
+            result_list.append(record)
+        return result_list
+
+    @staticmethod
+    def get_body_node_by_internal_id(loop_node: Node, internal_id: int):
+        suitable_nodes = loop_node.body.get_op_nodes(internal_layer_id=internal_id)
+        assert len(suitable_nodes) <= 1, \
+            'Expected 0 or 1 node with `internal_layer_id`={}, {} found'.format(internal_id, len(suitable_nodes))
+        return suitable_nodes[0] if len(suitable_nodes) == 1 else None
+
+    @staticmethod
+    def updated_body_parameters_shape(loop_node: Node):
+        """
+        Update shape for Loop body parameters.
+        The input shape of the "current_iteration" number input is handled separately because it is not connected with
+        the Loop operation.
+
+        :param loop_node: The Loop node
+        :return: None
+        """
+        for record in loop_node.input_port_map:
+            body_node = Loop.get_body_node_by_internal_id(loop_node, record['internal_layer_id'])
+            # the Parameter may be removed because it was not used in the body, for example, the current iteration
+            # number input
+            if body_node is not None:
+                assert body_node.soft_get('type') == 'Parameter'
+
+                input_shape = int64_array([])  # this is a current iteration number input shape
+                loop_port_idx = record['external_port_id']
+                if loop_port_idx != -1:
+                    input_shape = loop_node.in_port(loop_port_idx).get_connection().get_source().data.get_shape()
+                body_node.shape = input_shape
+                log.debug('Updated shape for the body node with internal_id "{}" with value {}'
+                          ''.format(record['internal_layer_id'], body_node.shape))
+
+    @staticmethod
+    def updated_loop_output_ports_shape_and_value(loop_node: Node):
+        """
+        Update shape and values for Loop output ports. If the number of iterations is dynamic then the corresponding
+        dimension for the scan outputs (having "axis" attribute) are set to 1 because MO cannot generate IR with
+        undefined dimensions.
+
+        :param loop_node: The Loop node
+        :return: None
+        """
+        loop_name = loop_node.soft_get('name', loop_node.id)
+        for record in loop_node.output_port_map:
+            body_node = Loop.get_body_node_by_internal_id(loop_node, record['internal_layer_id'])
+            assert body_node is not None
+            assert body_node.soft_get('type') == 'Result'
+
+            loop_port_idx = record['external_port_id']
+            if loop_port_idx != -1:  # the id = -1 for execution condition output which is not connected anywhere
+                output_value = body_node.in_port(0).data.get_value()
+                output_shape = body_node.in_port(0).data.get_shape()
+                concat_axis = record['axis']
+                if concat_axis is not None:
+                    assert output_shape[concat_axis] == 1, 'Dimension for concatenation is not equal to 1 for scan ' \
+                                                           'output for Loop node "{}" for loop output port "{}"'.\
+                        format(loop_name, loop_port_idx)
+                    num_iters = Loop.iterations_count(loop_node)
+                    if num_iters is None:
+                        log.error('Dynamic number of iterations for Loop node "{}". Consider number to be 1 to be able'
+                                  ' to generate the IR.'.format(loop_name), extra={'is_warning': True})
+                        num_iters = 1
+                    output_shape[concat_axis] = num_iters
+                # MO does not support evaluation of Loop scan outputs with const values
+                if concat_axis is None and output_value is not None:
+                    loop_node.out_port(loop_port_idx).data.set_value(output_value)
+                else:
+                    loop_node.out_port(loop_port_idx).data.set_shape(output_shape)
+
+    @staticmethod
+    def iterations_count(loop_node: Node):
+        """
+        Try to determine the number of loop iterations. If we detect that the number is dynamic then return None.
+
+        :param loop_node: Loop operation node
+        :return: number of iterations or None if the number depends on runtime values.
+        """
+        assert loop_node.soft_get('type') == 'Loop'
+
+        if loop_node.is_in_port_connected(1):
+            execution_condition = loop_node.in_port(1).data.get_value()
+            if execution_condition is None:  # dynamic execution condition
+                return None
+            execution_condition = execution_condition.item()
+            if not execution_condition:  # 0 iterations
+                return 0
+        num_iterations = loop_node.in_port(0).data.get_value()
+        if num_iterations is not None:
+            num_iterations = num_iterations.item(0)
+        return num_iterations
+
+    @staticmethod
+    def update_body_parameters_type(loop_node: Node):
+        """
+        Update the data type for Loop body Parameter nodes based on data type of the outer graph nodes producing data
+        for them.
+
+        :param loop_node: The Loop node
+        :return: None
+        """
+        assert loop_node.soft_get('type') == 'Loop'
+        for record in loop_node.input_port_map:
+            body_node = Loop.get_body_node_by_internal_id(loop_node, record['internal_layer_id'])
+            # the Parameter may be removed because it was not used in the body, for example, the current iteration
+            # number input
+            if body_node is not None:
+                assert body_node.soft_get('type') == 'Parameter'
+
+                loop_port_idx = record['external_port_id']
+                if loop_port_idx != -1:
+                    input_type = loop_node.in_port(loop_port_idx).get_data_type()
+                else:  # this is a current iteration number input node which is not connected to the Loop node
+                    assert record['purpose'] == 'current_iteration'
+                    input_type = np.int64
+
+                body_node.data_type = input_type
+                log.debug('Updated data type for the body node with internal_id "{}" with value {}'
+                          ''.format(record['internal_layer_id'], body_node.data_type))
+
+    @staticmethod
+    def update_loop_output_ports_type(loop_node: Node):
+        """
+        Update the data type for Loop output ports.
+
+        :param loop_node: The Loop node
+        :return: None
+        """
+        assert loop_node.soft_get('type') == 'Loop'
+        for record in loop_node.output_port_map:
+            body_node = Loop.get_body_node_by_internal_id(loop_node, record['internal_layer_id'])
+            assert body_node is not None
+            assert body_node.soft_get('type') == 'Result'
+
+            loop_port_idx = record['external_port_id']
+            if loop_port_idx != -1:  # the id = -1 for execution condition output which is not connected anywhere
+                output_type = body_node.in_port(0).get_data_type()
+                loop_node.out_port(loop_port_idx).set_data_type(output_type)
+
+    @staticmethod
+    def mark_current_iteration_parameter_node(loop_node: Node, body_parameter_node: Node):
+        assert body_parameter_node.id in loop_node.body
+        assert body_parameter_node.soft_get('op') == 'Parameter'
+        assert body_parameter_node.has_valid('internal_layer_id')
+        assert len(loop_node.body.get_op_nodes(purpose='current_iteration')) == 0
+
+        loop_node.input_port_map.append({'axis': None, 'stride': None, 'part_size': None, 'start': None, 'end': None,
+                                         'external_port_id': -1, 'purpose': 'current_iteration',
+                                         'internal_layer_id': body_parameter_node['internal_layer_id']})
+
+    @staticmethod
+    def mark_execution_condition_result_node(loop_node: Node, body_result_node: Node):
+        assert body_result_node.id in loop_node.body
+        assert body_result_node.soft_get('op') == 'Result'
+        assert body_result_node.has_valid('internal_layer_id')
+        assert len(loop_node.body.get_op_nodes(purpose='execution_condition')) == 0
+
+        loop_node.output_port_map.append({'axis': None, 'stride': None, 'part_size': None, 'start': None, 'end': None,
+                                          'external_port_id': -1, 'purpose': 'execution_condition',
+                                          'internal_layer_id': body_result_node['internal_layer_id']})
+
+    @staticmethod
+    def external_port_id_to_body_node(loop_node: Node, external_port_id: int, port_map: dict):
+        """
+        Return the body Node connected to the Loop port with number "external_port_id".
+
+        :param loop_node: the Loop node
+        :param external_port_id: the Loop node port idx
+        :param port_map: the port_map value to look for external_port_id
+        :return: the corresponding body Node
+        """
+        assert loop_node.soft_get('type') == 'Loop'
+        body_graph = loop_node.body
+        result_nodes = []
+        for record in port_map:
+            if record['external_port_id'] == external_port_id:
+                result_nodes.extend(body_graph.get_op_nodes(internal_layer_id=record['internal_layer_id']))
+        assert len(result_nodes) == 1, 'There should be just one body node for external port "{}", but there "{}"' \
+                                       ''.format(external_port_id, len(result_nodes))
+        return result_nodes[0]
+
+    @staticmethod
+    def connect_body_input(loop_input_port: Port, internal_parameter: Node, external_node_out_port: Port = None,
+                           axis: [int, None] = None, start: [int, None] = None, end: [int, None] = None,
+                           stride: [int, None] = None, part_size: [int, None] = None):
+        loop_node = loop_input_port.node
+        assert loop_node.soft_get('op') == 'Loop'
+        assert loop_input_port.type == 'in'
+        assert internal_parameter.soft_get('op') == 'Parameter'
+        assert internal_parameter.id in loop_node.body
+
+        if external_node_out_port is not None:
+            assert loop_input_port.disconnected()
+            assert external_node_out_port.node.id not in loop_node.body
+            loop_input_port.connect(external_node_out_port)
+
+        loop_node.input_port_map.append({'axis': axis, 'stride': stride, 'part_size': part_size, 'start': start,
+                                         'end': end, 'external_port_id': loop_input_port.idx,
+                                         'internal_layer_id': internal_parameter['internal_layer_id']})
+
+    @staticmethod
+    def connect_body_output(loop_output_port: Port, internal_result: Node, external_node_input_ports: list = None,
+                            axis: [int, None] = None, start: [int, None] = None, end: [int, None] = None,
+                            stride: [int, None] = None, part_size: [int, None] = None):
+        loop_node = loop_output_port.node
+        assert loop_node.soft_get('op') == 'Loop'
+        assert loop_output_port.type == 'out'
+        assert internal_result.soft_get('op') == 'Result'
+        assert internal_result.id in loop_node.body
+
+        if external_node_input_ports is not None:
+            assert loop_output_port.disconnected()
+            assert all([port.node.id not in loop_node.body for port in external_node_input_ports])
+            for port in external_node_input_ports:
+                port.disconnect()
+                loop_output_port.connect(port)
+        loop_node.output_port_map.append({'axis': axis, 'stride': stride, 'part_size': part_size, 'start': start,
+                                          'end': end, 'external_port_id': loop_output_port.idx,
+                                          'internal_layer_id': internal_result['internal_layer_id']})
+
+    @staticmethod
+    def add_back_edge(loop_node: Node, internal_parameter: Node, internal_result: Node):
+        assert internal_parameter.id in loop_node.body
+        assert internal_parameter.soft_get('op') == 'Parameter'
+        assert internal_result.id in loop_node.body
+        assert internal_result.soft_get('op') == 'Result'
+
+        loop_node.back_edges.append({'from_layer': internal_result['internal_layer_id'],
+                                     'to_layer': internal_parameter['internal_layer_id'],
+                                     'from_port': 0,
+                                     'to_port': 0})
+
+    @staticmethod
+    def pull_constant_inputs_into_body(loop_node: Node):
+        for port_idx, in_port in reversed(loop_node.in_ports().items()):
+            # TODO add a check that the input does not correspond to execution_condition
+            if not in_port.disconnected() and in_port.get_source().node.soft_get('type') == 'Const':
+                original_const_node = in_port.get_source().node
+                new_const_node = Const(loop_node.body, original_const_node.attrs()).create_node()
+
+                body_parameter = Loop.external_port_id_to_body_node(loop_node, port_idx, loop_node.input_port_map)
+                body_parameter.out_port(0).get_connection().set_source(new_const_node.out_port(0))
+                loop_node.body.remove_nodes_from([body_parameter.id])
+                loop_node.delete_input_port(port_idx)
+
+    @staticmethod
+    def update_port_map_value(port_map: dict, attr: str, original_value: int, new_value: int):
+        matched = 0
+        for record in port_map:
+            if record[attr] == original_value:
+                record[attr] = new_value
+                matched += 1
+        assert matched == 1, 'More than one record in the portmap for attr "{}" wil original value "{}"' \
+                             ''.format(attr, original_value)
+
+    @staticmethod
+    def re_numerate_input_ports(loop_node: Node):
+        """
+        Update input ports ids to be consecutive from 0 to num_input_ports - 1 and update the port_map values of the
+        Loop node.
+
+        :param loop_node: the Loop node
+        :return: None
+        """
+        def re_number_input_port(loop_node: Node, old_port_id: int, new_port_id: int):
+            loop_node.add_input_port(new_port_id, skip_if_exist=True)
+            loop_node.in_port(old_port_id).get_connection().set_destination(loop_node.in_port(new_port_id))
+            Loop.update_port_map_value(loop_node.input_port_map, 'external_port_id', old_port_id, new_port_id)
+
+        if len(loop_node.in_ports()) > 0:
+            max_port_id = sorted(loop_node.in_ports().keys())[-1]
+            new_port_id = 0
+            for port_id in range(max_port_id + 1):
+                if port_id in loop_node.in_ports():
+                    if port_id != new_port_id:
+                        re_number_input_port(loop_node, port_id, new_port_id)
+                    new_port_id += 1
+
+            for port_idx_to_remove in reversed(range(new_port_id, max_port_id + 1)):
+                loop_node.delete_input_port(port_idx_to_remove)
+
+    @staticmethod
+    def re_numerate_output_ports(loop_node: Node):
+        """
+        Update output ports ids to be consecutive from 0 to num_output_ports - 1 and update the port_map values of the
+        Loop node.
+
+        :param loop_node: the Loop node
+        :return: None
+        """
+        assert loop_node.soft_get('type') == 'Loop'
+
+        def re_number_output_port(loop_node: Node, old_port_id: int, new_port_id: int):
+            loop_node.add_output_port(new_port_id, skip_if_exist=True)
+            loop_node.out_port(old_port_id).get_connection().set_source(loop_node.out_port(new_port_id))
+            Loop.update_port_map_value(loop_node.output_port_map, 'external_port_id', old_port_id, new_port_id)
+
+        if len(loop_node.out_ports()) > 0:
+            max_port_id = sorted(loop_node.out_ports().keys())[-1]
+            new_port_id = 0
+            for port_id in range(max_port_id + 1):
+                if port_id in loop_node.out_ports():
+                    if port_id != new_port_id:
+                        re_number_output_port(loop_node, port_id, new_port_id)
+                    new_port_id += 1
+
+            for port_idx_to_remove in reversed(range(new_port_id, max_port_id + 1)):
+                loop_node.delete_output_port(port_idx_to_remove)
+
+    @staticmethod
+    def remove_unused_ops_from_port_map(loop_node: Node, port_map: dict, port_map_attr: str, dir: [None, str] = None):
+        """
+        Find unused operations in the Loop body referenced in the port_map and removes Loop ports connected to it.
+        Loop input port with index 0 and 1 are mandatory so cannot be removed.
+        Output ports of the Loop may not be connected so check for that case also and remove such an ops from the
+        port_map. The only exception is the "execution_condition" output which is a mandatory.
+
+        :param loop_node: the Loop node to update
+        :param port_map: the port_map (input, output or back edges)
+        :param port_map_attr: the port_map attribute containing the `internal_layer_id`
+        :param dir: the direction of the port_map meaning 'in' or 'out' port of the Loop
+        :return:
+        """
+        record_ids_to_remove = []
+        for record_id, record in enumerate(port_map):
+            if len(loop_node.body.get_op_nodes(internal_layer_id=record[port_map_attr])) == 0 or \
+                    (dir == 'out' and record.get('purpose', "") != 'execution_condition' and
+                     record['external_port_id'] not in loop_node.out_ports()):
+                record_ids_to_remove.append(record_id)
+        for record_id_to_remove in reversed(record_ids_to_remove):
+            if dir in ['in', 'out']:
+                port_to_remove = port_map[record_id_to_remove]['external_port_id']
+                if port_to_remove != -1:
+                    if dir == 'in':
+                        if port_to_remove not in [0, 1]:  # input port 0 and 1 are mandatory for the Loop node
+                            loop_node.delete_input_port(port_to_remove)
+                    elif dir == 'out' and port_to_remove in loop_node.out_ports():
+                        loop_node.delete_output_port(port_to_remove)
+            del port_map[record_id_to_remove]
+
+    @staticmethod
+    def normalize_input_output_ports(loop_node: Node):
+        """
+        Remove inputs, outputs, back edges from the port map which are not used in the body and were removed by a
+        graph clean up, for example, in case of not used current_iteration body Parameter. Then renumber input/output
+        ports.
+
+        :param loop_node: the Loop node to normalize
+        :return: None
+        """
+        Loop.remove_unused_ops_from_port_map(loop_node, loop_node.input_port_map, 'internal_layer_id', 'in')
+        Loop.remove_unused_ops_from_port_map(loop_node, loop_node.output_port_map, 'internal_layer_id', 'out')
+        Loop.remove_unused_ops_from_port_map(loop_node, loop_node.back_edges, 'to_layer')
+
+        # remove not connected input/output ports
+        Loop.re_numerate_input_ports(loop_node)
+        Loop.re_numerate_output_ports(loop_node)
+
+    @staticmethod
+    def infer(loop_node: Node):
+        Loop.updated_body_parameters_shape(loop_node)
+        partial_infer(loop_node.body)
+        Loop.updated_loop_output_ports_shape_and_value(loop_node)
+
+    @staticmethod
+    def type_infer(loop_node: Node):
+        from mo.middle.passes.infer import type_infer
+        Loop.update_body_parameters_type(loop_node)
+        type_infer(loop_node.body)
+        Loop.update_loop_output_ports_type(loop_node)

model-optimizer/extensions/ops/tensor_iterator.py

@@ -119,8 +119,7 @@ class TensorIterator(Op):
         for record in ti.back_edges:
             if record[layer_attr_name] != old_layer_id:
                 continue
-            if (port_attr_name in record and record[port_attr_name] == old_port_id) \
-                    or new_port_id is not None:
+            if (port_attr_name in record and record[port_attr_name] == old_port_id) or new_port_id is not None:
                 record[layer_attr_name] = new_layer_id
                 if new_port_id is None:
                     del record[port_attr_name]
@@ -233,13 +232,8 @@ class TensorIterator(Op):
 
         TensorIterator.validate_maps(ti)
 
-    def substitute_ie_attrs(self, new_attrs: dict):
-        """
-        Replace standard list of attribute in layer/data by attributes
-        delivered by backend_attrs
-        """
-
-        port_map_attrs = [
+    def port_map_attrs(self):
+        return [
             'external_port_id',
             'internal_layer_id',
             'internal_port_id',
@@ -247,9 +241,17 @@ class TensorIterator(Op):
             'start',
             'stride',
             'end',
-            'part_size'
+            'part_size',
         ]
 
+    def substitute_ie_attrs(self, new_attrs: dict):
+        """
+        Replace standard list of attribute in layer/data by attributes
+        delivered by backend_attrs
+        """
+
+        port_map_attrs = self.port_map_attrs()
+
         back_edges_attrs = [
             ('from-layer', 'from_layer'),
             ('from-port', 'from_port'),
@@ -265,9 +267,9 @@ class TensorIterator(Op):
                     ('data', self.backend_attrs() + self.default_backend_attrs, []),
                     '@ports',
                     ('port_map', [], [
-                        ('@list', lambda node: self.generate_port_map(node, node.input_port_map),
+                        ('@list', lambda node: self.generate_port_map(node, node.input_port_map, 'in'),
                          ('input', port_map_attrs, [])),
-                        ('@list', lambda node: self.generate_port_map(node, node.output_port_map),
+                        ('@list', lambda node: self.generate_port_map(node, node.output_port_map, 'out'),
                          ('output', port_map_attrs, [])),
                     ]),
                     ('back_edges', [], [
@@ -278,7 +280,7 @@ class TensorIterator(Op):
         })
 
     @staticmethod
-    def find_port_id(node: Node, virtual_id, attr):
+    def find_port_id(node: Node, virtual_id: str, attr: str):
         attrs = node.edge({attr: virtual_id})[2]
         assert bool('in' in attrs) != bool('out' in attrs), attrs
         return attrs['in' if 'in' in attrs else 'out']
@@ -291,13 +293,7 @@ class TensorIterator(Op):
         return internal_nodes[0][0]
 
     @staticmethod
-    def find_internal_layer_and_port(graph: Graph, virtual_layer_id, virtual_port_id):
-        internal_layer_id = __class__.find_internal_layer_id(graph, virtual_layer_id)
-        internal_port_id = __class__.find_port_id(Node(graph, internal_layer_id), virtual_port_id, 'internal_port_id')
-        return internal_layer_id, internal_port_id
-
-    @staticmethod
-    def generate_port_map(node: Node, src_port_map):
+    def generate_port_map(node: Node, src_port_map, dir: str):
         """ Extract port_map attributes from node and node.body attributes.
 
             It iterates over src_port_map and substitute external_port_id, internal_port_id and

model-optimizer/mo/front/onnx/loader.py

@@ -22,7 +22,7 @@ import logging as log
 
 import onnx
 
-from mo.graph.graph import fill_graph_with_nodes, Graph
+from mo.graph.graph import fill_graph_with_nodes, Graph, Node
 from mo.utils.error import Error, FrameworkError
 
 
@@ -54,35 +54,24 @@ def node_id(pb):
         return 'NoNamed'
 
 
-def protobuf2nx(graph, pb):
-    '''Convert proto message with ONNX model to equivalent NX representation.
-    All nodes and edges are restored here as ONNX model has op/data representation,
-    that means that nodes are connected via tensor names. Name of tensors are defined
-    on demand in nodes, so we have a code similar to Caffe here. '''
-    # graph = fill_graph_with_nodes(graph, pb.graph.node, get_id=node_id, get_attrs=protobuf_attrs)
-    # convert initializers to a NX graph for easier control of model consistency and to use it as a dictionary later
-    initializers = Graph()
-    fill_graph_with_nodes(initializers, pb.graph.initializer, get_id=lambda pb: pb.name, get_attrs=protobuf_attrs)
+def protobuf2nx(graph: Graph, pb):
+    """
+    Convert proto message with ONNX model to equivalent NX representation. All nodes and edges are restored here as
+    ONNX model has op/data representation, that means that nodes are connected via tensor names. Name of tensors are
+    defined on demand in nodes, so we have a code similar to Caffe here.
 
+    :param graph: the Graph object to load the graph into
+    :param pb: the ONNX file protobuf message
+    :return: None
+    """
     # maps a tensor name to a node produced it and the node port: str -> (node_id, node_port)
     data_nodes_map = {}
 
-    # first go through all inputs and separate constant from placeholders
-    for inp in pb.graph.input:
-        name = str(inp.name)
-        if graph.has_node(name):
-            raise Error('Name {} of input node already exists, input names are duplicated.', name)
-        elif initializers.has_node(name):
-            # this is a constant
-            graph.add_node(name, kind='op', op='Const', pb=inp, pb_init=initializers.node[name]['pb'])
-        else:
-            # this is a placeholder
-            graph.add_node(name, kind='op', op='Parameter', pb=inp)
-        # add to a tensors map
-        assert not name in data_nodes_map, 'Inconsistency between data_nodes_map and graph.nodes'
-        data_nodes_map[name] = (name, 0)
+    graph_pb = pb.graph
+    add_initializers_and_inputs_to_graph(graph, graph_pb, data_nodes_map)
+
     output_ids = []
-    for outp in pb.graph.output:
+    for outp in graph_pb.output:
         name = str(outp.name)
         if graph.has_node(name):
             log.error('Name {} of output node already exists in graph. Ignoring this output. If the output is required,'
@@ -93,17 +82,9 @@ def protobuf2nx(graph, pb):
             graph.add_node(name, kind='op', op='FakeOutput', pb=outp)
             output_ids.append(name)
 
-    # go over all initializer and make sure that all of them are added to the graph
-    for initializer in initializers.nodes():
-        initializer_id = 'onnx_initializer_node_' + initializer
-        if not graph.has_node(initializer_id):
-            graph.add_node(initializer_id, kind='op', op='Const', pb=initializers.node[initializer]['pb'],
-                           pb_init=initializers.node[initializer]['pb'])
-            data_nodes_map[initializer] = (initializer_id, 0)
-
     # Go through all nodes in the original model order (because data nodes are defined on-the-fly and order is
     # important)
-    for node in pb.graph.node:
+    for node in graph_pb.node:
         # create an NX node
         id = graph.unique_id(node_id(node))
         graph.add_node(id, pb=node, kind='op')
@@ -149,3 +130,41 @@ def protobuf2nx(graph, pb):
             if out in data_nodes_map:
                 log.debug("Detected reuse of blob {}.".format(out))
             data_nodes_map[out] = (id, src_port)
+
+    graph.graph['tensor_mapping'] = data_nodes_map  # save main graph tensor names mapping for Loop op parsing
+
+
+def add_initializers_and_inputs_to_graph(graph: Graph, graph_pb, data_nodes_map: dict):
+    """
+    The function adds nodes specified in the 'initializer' attribute of the pb and input nodes.
+    :param graph: the Graph to add nodes to
+    :param graph_pb: the graph protobuf message
+    :param data_nodes_map: the dictionary with mapping of tensor names to node id and port
+    :return: the list of Parameter nodes
+    """
+    initializers = Graph()
+    fill_graph_with_nodes(initializers, graph_pb.initializer, get_id=lambda pb: pb.name, get_attrs=protobuf_attrs)
+
+    parameters = []
+    # first go through all inputs and separate constant from placeholders
+    for inp in graph_pb.input:
+        name = str(inp.name)
+        if graph.has_node(name):
+            raise Error('Name {} of input node already exists, input names are duplicated.', name)
+        elif initializers.has_node(name):
+            graph.add_node(name, kind='op', op='Const', pb=inp, pb_init=initializers.node[name]['pb'])
+        else:
+            graph.add_node(name, kind='op', op='Parameter', pb=inp)
+            parameters.append(Node(graph, name))
+
+        assert name not in data_nodes_map, 'Inconsistency between data_nodes_map and graph.nodes'
+        data_nodes_map[name] = (name, 0)
+
+    # go over all initializers and make sure that all of them are added to the graph
+    for initializer in initializers.nodes():
+        initializer_id = initializer
+        if not graph.has_node(initializer_id):
+            graph.add_node(initializer_id, kind='op', op='Const', pb=initializers.node[initializer]['pb'],
+                           pb_init=initializers.node[initializer]['pb'])
+            data_nodes_map[initializer] = (initializer_id, 0)
+    return parameters

model-optimizer/mo/graph/graph.py

@@ -112,6 +112,21 @@ class Node:
         # update in_ports_count for consistency but it is unlikely have any effect somewhere in the code
         self['in_ports_count'] = len(self._in_ports)
 
+    def delete_output_port(self, idx, skip_if_absent=False):
+        if not self.has_valid('_out_ports'):
+            raise Error(
+                'Cannot removed ports with indices {} from node {} because node doesn\'t '
+                'have _out_ports attribute'.format(idx, self.soft_get('name')))
+        # no handling of control flow edges -- TODO
+        control_flow = False
+        if not skip_if_absent and idx not in self.out_ports(control_flow=control_flow):
+            raise Error("Input port with index {} doesn't exist in node {}.".format(idx, self.soft_get('name')))
+        if not self.out_port(idx).disconnected():
+            self.out_port(idx).disconnect()
+        del self._out_ports[idx]
+        # update in_ports_count for consistency but it is unlikely have any effect somewhere in the code
+        self['out_ports_count'] = len(self._out_ports)
+
     def add_output_port(self, idx, skip_if_exist=False, **kwargs):
         if not self.has_valid('_out_ports'):
             Node(self.graph, self.id)['_out_ports'] = {}
@@ -580,21 +595,27 @@ class Graph(nx.MultiDiGraph):
             # Also check that all necessary ports are exists
             message = "Attempt to connect {} to {}.".format(u_for_edge, v_for_edge)
             if self.stage == 'front':
-                assert unode.kind == 'op' and vnode.kind == 'op', "{} Wrong add_adge usage! You can connect only two operations in front phase".format(message)
-                assert 'in' in attr and 'out' in attr, "Missing necessary attribute in or out when adding edge between {} and {}".format(u_for_edge, v_for_edge)
+                assert unode.kind == 'op' and vnode.kind == 'op', "{} Wrong add_adge usage! You can connect only two " \
+                                                                  "operations in front phase".format(message)
+                assert 'in' in attr and 'out' in attr, "Missing necessary attribute in or out when adding edge " \
+                                                       "between {} and {}".format(u_for_edge, v_for_edge)
                 is_control_flow = 'control_flow_edge' in attr and attr['control_flow_edge'] is True
                 in_port = 'control_flow_{}'.format(attr['in']) if is_control_flow else attr['in']
                 out_port = 'control_flow_{}'.format(attr['out']) if is_control_flow else attr['out']
-                assert unode.has_port('out', out_port, control_flow=is_control_flow), "{} Missing out port ({}) in {} node".format(message, out_port, unode.name)
-                assert vnode.has_port('in', in_port, control_flow=is_control_flow), "{} Missing in port ({}) in {} node".format(message, in_port, vnode.name)
+                assert unode.has_port('out', out_port, control_flow=is_control_flow), \
+                    "{} Missing out port ({}) in {} node".format(message, out_port, unode.soft_get('name', unode.id))
+                assert vnode.has_port('in', in_port, control_flow=is_control_flow), \
+                    "{} Missing in port ({}) in {} node".format(message, in_port, vnode.soft_get('name', vnode.id))
             elif self.stage in ['middle', 'back']:
                 assert (unode.kind == 'data' and vnode.kind == 'op') or (unode.kind == 'op' and vnode.kind == 'data')
                 if unode.kind == 'data' and vnode.kind == 'op':
                     assert 'in' in attr, "Attribute in is missing when adding edge to {}".format(v_for_edge)
-                    assert vnode.has_port('in', attr['in']), "{} Node {} has no in port ({})".format(message, vnode.name, attr['in'])
+                    assert vnode.has_port('in', attr['in']), "{} Node {} has no in port ({})" \
+                                                             "".format(message, vnode.name, attr['in'])
                 if unode.kind == 'op' and vnode.kind == 'data':
                     assert 'out' in attr, "Attribute out is missing when adding edge from {}".format(u_for_edge)
-                    assert unode.has_port('out', attr['out']), "{} Node {} has no out port ({})".format(message, unode.name, attr['out'])
+                    assert unode.has_port('out', attr['out']), "{} Node {} has no out port ({})" \
+                                                               "".format(message, unode.name, attr['out'])
 
         return super().add_edge(u_for_edge, v_for_edge, key=key, **attr)
 
@@ -759,7 +780,8 @@ class Graph(nx.MultiDiGraph):
                                                                                            list(
                                                                                                node.out_ports().keys())))
 
-    def dump_graph_for_graphviz(self, node_attrs: list = ['kind', 'op', 'shape', 'correct_data_layout', 'nchw_layout'],
+    def dump_graph_for_graphviz(self, node_attrs: list = ['kind', 'op', 'shape', 'correct_data_layout', 'nchw_layout',
+                                                          'internal_layer_id'],
                                 edge_attrs: list = ['in', 'out'], nodes_to_dump: list = None,
                                 save_to_svg=False, highlight_nodes: list = None):
 

model-optimizer/mo/ops/eltwise_n.py

@@ -28,13 +28,13 @@ class EltwiseN(Op):
 
     def __init__(self, graph: Graph, attrs: dict):
         super().__init__(graph, {
-            'op': __class__.op,
+            'op': self.op,
             'type': None,  # type is None because this operation should not appear in IR
             'infer': None,
             'out_ports_count': 1,
         }, attrs)
         if 'operation' not in self.attrs:
-            raise Error('"operation" attribute is not set for operation "{}".'.format(__class__.op))
+            raise Error('"operation" attribute is not set for operation "{}".'.format(self.op))
 
 
 class EltwiseNMul(EltwiseN):
@@ -42,6 +42,11 @@ class EltwiseNMul(EltwiseN):
         super().__init__(graph, {'operation': 'mul'})
 
 
+class EltwiseNMin(EltwiseN):
+    def __init__(self, graph: Graph, attrs: dict):
+        super().__init__(graph, {'operation': 'min'})
+
+
 class EltwiseNMax(EltwiseN):
     def __init__(self, graph: Graph, attrs: dict):
         super().__init__(graph, {'operation': 'max'})

model-optimizer/mo/ops/slice.py

@@ -125,7 +125,7 @@ class Slice(Op):
             'op': 'Slice',
             'in_ports_count': 5,
             'out_ports_count': 1,
-            'infer': __class__.infer
+            'infer': self.infer
         }, attrs)
 
     @staticmethod
@@ -172,17 +172,5 @@ def get_shape_after_slice(input_shape: np.ndarray, slice_idx: List[slice]) -> np
     """
     output_shape = np.zeros(len(input_shape), dtype=np.int32)
     for i, s in enumerate(slice_idx):
-        start, end, step = normalize_slice_indices(input_shape[i], s.start, s.stop, s.step)
-        output_shape[i] = (end - start + step - 1) / step
+        output_shape[i] = len(range(*s.indices(input_shape[i])))
     return output_shape
-
-
-def normalize_slice_indices(size: int, start: int, end: int, step: int) -> (int, int, int):
-    # converts slice indices to format in which size of Slice can be calculated
-    start = size + start if start < 0 else start
-    end = size + end if end < 0 else end
-    start = np.clip(start, 0, size)
-    end = np.clip(end, 0, size)
-    if step < 0:
-        start, end, step = end, start, -step  # if calculate size without values we can do that
-    return start, end, step

model-optimizer/mo/ops/slice_test.py

@@ -51,6 +51,8 @@ class TestSliceOp(unittest.TestCase):
             (None, [1], [3], [0], [4], [1], [4]),
             # with rounding and negative steps (e.g. take from 1st to 3rd with step 4 should give shape 1 not 0)
             (None, [7], [3], [0], [-7], [1], [10]),
+            # reversing the sequence of elements
+            (None, [-1], [np.iinfo(np.int32).min], [0], [-1], [10], [10]),
         ])
         def test_slice_infer(self, inp_value, starts, ends, axes, steps, expected, inp_shape=None):
             if inp_value is None:

model-optimizer/mo/utils/class_registration.py

@@ -282,7 +282,7 @@ def apply_transform(graph: Graph, replacer_cls, **kwargs):
     log.debug("Run replacer {}".format(replacer_cls))
 
     try:
-        if hasattr(replacer, 'run_not_recursively'):
+        if hasattr(replacer, 'run_not_recursively') and replacer.run_not_recursively:
             replacer.find_and_replace_pattern(graph)
         else:
             for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)
@@ -293,8 +293,12 @@ def apply_transform(graph: Graph, replacer_cls, **kwargs):
         if hasattr(replacer, 'force_shape_inference') and replacer.force_shape_inference:
             shape_inference(graph)
 
-        for_graph_and_each_sub_graph_recursively(graph, lambda _: graph.check_empty_graph(replacer_cls))
-        for_graph_and_each_sub_graph_recursively(graph, lambda _: graph.check_shapes_consistency())
+        if hasattr(replacer, 'run_not_recursively') and replacer.run_not_recursively:
+            graph.check_empty_graph(replacer_cls)
+            graph.check_shapes_consistency()
+        else:
+            for_graph_and_each_sub_graph_recursively(graph, lambda _: graph.check_empty_graph(replacer_cls))
+            for_graph_and_each_sub_graph_recursively(graph, lambda _: graph.check_shapes_consistency())
 
     except Error as err:
         raise Error('Exception occurred during running replacer "{}" ({}): {}'.format(

model-optimizer/mo/utils/ir_reader/layer_to_class.py

@@ -25,6 +25,7 @@ from extensions.ops.ReduceOps import ReduceOp
 from extensions.ops.activation_ops import Activation
 from extensions.ops.elementwise import Elementwise, UnaryElementwise, LogicalElementwise, BiasAdd, Div, Mul, Pow, Sub
 from extensions.ops.embedding_bag import EmbeddingBagBase
+from extensions.ops.loop import Loop
 from extensions.ops.psroipooling import DeformablePSROIPoolingOp
 from extensions.ops.scatter import Scatter
 from extensions.ops.scatternd import ScatterNDBase
@@ -51,6 +52,7 @@ custom_ops = {
     'Divide': Div,
     'GroupConvolution': Convolution,
     'GroupConvolutionBackpropData': Deconvolution,
+    'Loop': Loop,
     'MaxPool': Pooling,
     'Multiply': Mul,
     'Power': Pow,