openvino/docs/ops/infrastructure/TensorIterator_1.md

# TensorIterator {#openvino_docs_ops_infrastructure_TensorIterator_1}

**Versioned name**: *TensorIterator-1*

**Category**: *Infrastructure*

**Short description**: *TensorIterator* layer performs recurrent execution of the network, which is described in the `body`, iterating through the data.

**TensorIterator attributes**:

* **Body**:

    `body` is a network that will be recurrently executed. The network is described layer by layer as a typical IR network.

    * **Body attributes**:

            No attributes available.

* **Port map**:

    *port_map* is a set of rules to map input or output data tensors of `TensorIterator` layer onto `body` data tensors. The `port_map` entries can be` input` and `output`. Each entry describes a corresponding mapping rule.

    * **Port map attributes**:

        * *external_port_id*
            * **Description**: *external_port_id* is a port ID of the `TensorIterator` layer.
            * **Range of values**: indexes of the *TensorIterator* outputs
            * **Type**: `int`
            * **Default value**: None
            * **Required**: *yes*

        * *internal_layer_id*

            * **Description**: *internal_layer_id* is a *Parameter* or *Result* layer ID inside the `body` network to map to.
            * **Range of values**: IDs of the *Parameter* layers inside in the *TensorIterator* layer
            * **Type**: `int`
            * **Default value**: None
            * **Required**: *yes*

        * *axis*

            * **Description**: *axis* is an axis to iterate through. It triggers the slicing of this tensor. Only if it is specified, the corresponding `input` or `output` is divided into pieces and start, end and stride attributes define how slicing is performed.
            * **Range of values**: an integer
            * **Type**: `int`
            * **Default value**: None
            * **Required**: *no*

        * *start*

            * **Description**: *start* is an index where the iteration starts from. Negative value means counting indexes from the end. Applies only when the attribute `axis` is specified.
            * **Range of values**: an integer
            * **Type**: `int`
            * **Default value**: 0
            * **Required**: *no*

        * *end*

            * **Description**: *end* is an index where iteration ends. Negative value means counting indexes from the end. Applies only when the attribute `axis` is specified.
            * **Range of values**: an integer
            * **Type**: `int`
            * **Default value**: -1
            * **Required**: *no*

        * *stride*

            * **Description**: *stride* is a step of iteration. Negative value means backward iteration. Applies only when the attribute `axis` is specified.
            * **Range of values**: an integer
            * **Type**: `int`
            * **Default value**: 1
            * **Required**: *no*


* **Back edges**:

    *back_edges* is a set of rules to transfer tensor values from `body` outputs at one iteration to `body` parameters at the next iteration. Back edge connects some *Result* layer in `body` to *Parameter* layer in the same `body`.

    * **Back edge attributes**:

        * *from-layer*

            * **Description**: *from-layer* is a *Result* layer ID inside the `body` network.
            * **Range of values**: IDs of the *Result* layers inside the *TensorIterator*
            * **Type**: `int`
            * **Default value**: None
            * **Required**: *yes*

        * *to-layer*

            * **Description**: *to-layer* is a *Parameter* layer ID inside the `body` network to end mapping.
            * **Range of values**: IDs of the *Parameter* layers inside the *TensorIterator*
            * **Type**: `int`
            * **Default value**: None
            * **Required**: *yes*

**Inputs**

* **Multiple inputs**: Tensors of any type and shape supported type.

**Outputs**

* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape.


**Detailed description**

Similar to other layers, TensorIterator has regular sections: `input` and `output`. It allows connecting TensorIterator to the rest of the IR.
TensorIterator also has several special sections: `body`, `port_map`, `back_edges`. The principles of their work are described below.

How `body` is iterated:

*At the first iteration:*
TensorIterator slices input tensors by a specified axis and iterates over all parts in a specified order. It process input tensors with arbitrary network specified as an IR network in the `body` section. IR is executed as no back-edges are present. Edges from `port map` are used to connect input ports of TensorIterator to `Parameters` in body.

[`inputs`] - `Port map` edges -> [`Parameters:body:Results`]

`Parameter` and `Result` layers are part of the `body`. `Parameters` are stable entry points in the `body`. The results of the execution of the `body` are presented as stable `Result` layers. Stable means that these nodes cannot be fused.

*Next iterations:*
Back edges define which data is copied back to `Parameters` layers from `Results` layers between IR iterations in TensorIterator `body`. That means they pass data from source layer back to target layer. Each layer that is a target for back-edge has also an incoming `port map` edge as an input. The values from back-edges are used instead of corresponding edges from `port map`. After each iteration of the network, all back edges are executed. 
Iterations can be considered as statically unrolled sequence: all edges that flow between two neighbor iterations are back-edges. So in the unrolled loop, each back-edge is transformed to regular edge.

... -> [`Parameters:body:Results`] - back-edges -> [`Parameters:body:Results`] - back-edges -> [`Parameters:body:Results`] - back-edges -> ...

*Calculation of results:*

If `output` entry in the `Port map` doesn't have partitioning (`axis, begin, end, strides`) attributes, then the final value of `output` of TensorIterator is the value of `Result` node from the last iteration. Otherwise the final value of `output` of TensorIterator is a concatenation of tensors in the `Result` node for all `body` iterations. Concatenation order is specified by `stride` attribute.

The last iteration:

[`Parameters:body:Results`] - `Port map` edges -> [`outputs`],  if partitioning attributes are not set.

if there are partitioning attributes, then an output tensor is a concatenation of tensors from all body iterations. If `stride > 0`:
```
output = Concat(S[0], S[1], ..., S[N-1])
```
where `Si` is value of `Result` operation at i-th iteration in the tensor iterator body that corresponds to this output port. If `stride < 0`, then output is concatenated in a reverse order:
```
output = Concat(S[N-1], S[N-2], ..., S[0])
```

**Examples**

*Example 1: a typical TensorIterator structure*
```xml
<layer type="TensorIterator" ... >
    <input> ... </input>
    <output> ... </output>
    <port_map>
        <input external_port_id="0" internal_layer_id="0" axis="1" start="-1" end="0" stride="-1"/>
        <input external_port_id="1" internal_layer_id="1"/>
        ...
        <output external_port_id="3" internal_layer_id="2" axis="1" start="-1" end="0" stride="-1"/>
        ...
    </port_map>
    <back_edges>
        <edge from-layer="1" to-layer="1"/>
        ...
    </back_edges>
    <body>
        <layers> ... </layers>
        <edges> ... </edges>
    </body>
</layer>
```

*Example 2: a full TensorIterator layer*

```xml
<layer type="TensorIterator" ...>
    <input>
        <port id="0">
            <dim>1</dim>
            <dim>25</dim>
            <dim>512</dim>
        </port>
        <port id="1">
            <dim>1</dim>
            <dim>256</dim>
        </port>
        <port id="2">
            <dim>1</dim>
            <dim>256</dim>
        </port>
    </input>
    <output>
        <port id="3" precision="FP32">
            <dim>1</dim>
            <dim>25</dim>
            <dim>256</dim>
        </port>
    </output>
    <port_map>
        <input axis="1" external_port_id="0" internal_layer_id="0" start="0"/>
        <input external_port_id="1" internal_layer_id="3"/>
        <input external_port_id="2" internal_layer_id="4"/>
        <output axis="1" external_port_id="3" internal_layer_id="12"/>
    </port_map>
    <back_edges>
        <edge from-layer="8" to-layer="4"/>
        <edge from-layer="9" to-layer="3"/>
    </back_edges>
    <body>
        <layers>
            <layer id="0" type="Parameter" ...>
                <output>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                        <dim>1</dim>
                        <dim>512</dim>
                    </port>
                </output>
            </layer>
            <layer id="1" type="Const" ...>
                <data offset="0" size="16"/>
                <output>
                    <port id="1" precision="I64">
                        <dim>2</dim>
                    </port>
                </output>
            </layer>
            <layer id="2" type="Reshape" ...>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>1</dim>
                        <dim>512</dim>
                    </port>
                    <port id="1">
                        <dim>2</dim>
                    </port>
                </input>
                <output>
                    <port id="2" precision="FP32">
                        <dim>1</dim>
                        <dim>512</dim>
                    </port>
                </output>
            </layer>
            <layer id="3" type="Parameter" ...>
                <output>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </output>
            </layer>
            <layer id="4" type="Parameter" ...>
                <output>
                    <port id="0" precision="FP32">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </output>
            </layer>
            <layer id="5" type="Const" ...>
                <data offset="16" size="3145728"/>
                <output>
                    <port id="1" precision="FP32">
                        <dim>1024</dim>
                        <dim>768</dim>
                    </port>
                </output>
            </layer>
            <layer id="6" type="Const" ...>
                <data offset="3145744" size="4096"/>
                <output>
                    <port id="1" precision="FP32">
                        <dim>1024</dim>
                    </port>
                </output>
            </layer>
            <layer id="7" type="LSTMCell" ...>
                <data hidden_size="256"/>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>512</dim>
                    </port>
                    <port id="1">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                    <port id="2">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                    <port id="3">
                        <dim>1024</dim>
                        <dim>768</dim>
                    </port>
                    <port id="4">
                        <dim>1024</dim>
                    </port>
                </input>
                <output>
                    <port id="5" precision="FP32">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                    <port id="6" precision="FP32">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </output>
            </layer>
            <layer id="8" type="Result" ...>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </input>
            </layer>
            <layer id="9" type="Result" ...>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </input>
            </layer>
            <layer id="10" type="Const" ...>
                <data offset="3149840" size="24"/>
                <output>
                    <port id="1" precision="I64">
                        <dim>3</dim>
                    </port>
                </output>
            </layer>
            <layer id="11" type="Reshape" ...>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                    <port id="1">
                        <dim>3</dim>
                    </port>
                </input>
                <output>
                    <port id="2" precision="FP32">
                        <dim>1</dim>
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </output>
            </layer>
            <layer id="12" type="Result" ...>
                <input>
                    <port id="0">
                        <dim>1</dim>
                        <dim>1</dim>
                        <dim>256</dim>
                    </port>
                </input>
            </layer>
        </layers>
        <edges>
            <edge from-layer="0" from-port="0" to-layer="2" to-port="0"/>
            <edge from-layer="1" from-port="1" to-layer="2" to-port="1"/>
            <edge from-layer="2" from-port="2" to-layer="7" to-port="0"/>
            <edge from-layer="3" from-port="0" to-layer="7" to-port="1"/>
            <edge from-layer="4" from-port="0" to-layer="7" to-port="2"/>
            <edge from-layer="5" from-port="1" to-layer="7" to-port="3"/>
            <edge from-layer="6" from-port="1" to-layer="7" to-port="4"/>
            <edge from-layer="7" from-port="6" to-layer="8" to-port="0"/>
            <edge from-layer="7" from-port="5" to-layer="9" to-port="0"/>
            <edge from-layer="7" from-port="5" to-layer="11" to-port="0"/>
            <edge from-layer="10" from-port="1" to-layer="11" to-port="1"/>
            <edge from-layer="11" from-port="2" to-layer="12" to-port="0"/>
        </edges>
    </body>
</layer>
```