Added opset docs (#992)

docs/ops/activation/Clamp_1.md

@@ -0,0 +1,53 @@
+## Clamp<a name="Clamp"></a>
+
+**Versioned name**: *Clamp-1*
+
+**Category**: *Activation function*
+
+**Short description**: *Clamp* operation represents clipping activation function.
+
+**Attributes**:
+
+* *min*
+
+  * **Description**: *min* is the lower bound of values in the output. Any value in the input that is smaller than the bound, is replaced with the *min* value. For example, *min* equal 10 means that any value in the input that is smaller than the bound, is replaced by 10.
+  * **Range of values**: non-negative positive floating point number
+  * **Type**: float
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *max*
+
+  * **Description**: *max* is the upper bound of values in the output. Any value in the input that is greater than the bound, is replaced with the *max* value. For example, *max* equals 50 means that any value in the input that is greater than the bound, is replaced by 50.
+  * **Range of values**: positive floating point number
+  * **Type**: float
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: Multidimensional input tensor. Required.
+
+**Outputs**:
+
+*   **1**: Multidimensional output tensor with shape and type matching the input tensor. Required.
+
+**Detailed description**:
+
+*Clamp* does the following with the input tensor element-wise:
+\f[
+clamp( x )=\left\{\begin{array}{ll}
+    max\_value \quad \mbox{if } \quad input( x )>max\_value \\
+    min\_value \quad \mbox{if } \quad input( x )
+\end{array}\right.
+\f]
+
+**Example**
+
+```xml
+<layer ... type="Clamp" ... >
+    <data min="10" max="50" />
+    <input> ... </input>
+    <output> ... </output>
+</layer>
+```

docs/ops/activation/Elu_1.md

@@ -0,0 +1,36 @@
+## Elu<a name="Elu"></a>
+
+**Versioned name**: *Elu-1*
+
+**Category**: *Activation function*
+
+**Short description**: Exponential linear unit element-wise activation function.
+
+**Detailed Description**
+
+For each element from the input tensor calculates corresponding
+element in the output tensor with the following formula:
+\f[
+elu(x) = \left\{\begin{array}{ll}
+    alpha(e^{x} - 1) \quad \mbox{if } x < 0 \\
+    x \quad \mbox{if } x \geq  0
+\end{array}\right.
+\f]
+
+**Attributes**
+
+* *alpha*
+
+  * **Description**: scale for the negative factor
+  * **Range of values**: arbitrary floating point number
+  * **Type**: float
+  * **Default value**: none
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: Input tensor x of any floating point type. Required.
+
+**Outputs**:
+
+*   **1**: Result of Elu function applied to the input tensor *x*. Floating point tensor with shape and type matching the input tensor. Required.

docs/ops/activation/Exp_1.md

@@ -0,0 +1,17 @@
+## Exp<a name="Exp"></a>
+
+**Versioned name**: *Exp-1*
+
+**Category**: *Activation function*
+
+**Short description**: Exponential element-wise activation function.
+
+**Attributes**: has no attributes
+
+**Inputs**:
+
+*   **1**: Input tensor x of any floating point type. Required.
+
+**Outputs**:
+
+*   **1**: Result of Exp function applied to the input tensor *x*. Floating point tensor with shape and type matching the input tensor. Required.

docs/ops/activation/GELU_2.md

@@ -0,0 +1,47 @@
+## GELU- Gaussian Error Linear Unit <a name="Gelu"></a>
+
+**Versioned name**: *Gelu-2*
+
+**Category**: *Activation*
+
+**Short description**: [Reference](https://pytorch.org/docs/stable/nn.functional.html#gelu)
+
+**Detailed description**: [Reference](https://arxiv.org/abs/1606.08415)
+
+**Attributes**: *Gelu* operation has no attributes.
+
+**Mathematical Formulation**
+Gelu(x)=x*Φ(x), where Φ(x) is the Cumulative Distribution Function for Gaussian Distribution.
+The following equivalent combination is recognized and fused into single Gelu op: 
+
+\f[
+    Gelu(x) = 0.5*x*(1 + erf((x) / sqrt(2) )
+\f]
+
+Similarly, the following Gelu approximation (typical for the TensorFlow*) is  recognized and fused into single Gelu op
+\f[
+    Gelu(x) \approx  0.5*x*(1 + tanh((sqrt(2/pi)) * (x + 0.044715 * x ^ 3))
+\f]
+**Inputs**:
+
+*   **1**: Multidimensional input tensor. Required.
+
+**Example**
+
+```xml
+<layer ... type="Gelu">
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>128</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>1</dim>
+            <dim>128</dim>
+        </port>
+    </output>
+</layer>
+
+```

docs/ops/activation/HardSigmoid_1.md

@@ -0,0 +1,48 @@
+## HardSigmoid <a name="HardSigmoid"></a>
+
+**Versioned name**: *HardSigmoid-1*
+
+**Category**: *Activation function*
+
+**Short description**: *HardSigmoid* calculates the hard sigmoid function `y(x) = max(0, min(1, alpha * x + beta))` element-wise with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+* **2**: `alpha` 0D tensor (scalar) of type T. **Required.**
+
+* **3**: `beta` 0D tensor (scalar) of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of the hard sigmoid operation. A tensor of type T.
+
+**Types**
+
+* *T*: any floating point type.
+
+**Examples**
+
+```xml
+<layer ... type="HardSigmoid">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1"/>
+        <port id="2"/>
+    </input>
+    <output>
+        <port id="3">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/activation/PReLU_1.md

@@ -0,0 +1,33 @@
+## PReLU <a name="PReLU"></a>
+
+**Versioned name**: *PReLU-1*
+
+**Category**: Activation function
+
+**Short description**: *PReLU* performs element-wise parametric ReLU operation with negative slope defined by the second input.
+
+**Attributes**: operation has no attributes.
+
+**Inputs**
+
+* **1**: `X` - Input tensor of any supported floating point type T1. Required.
+
+* **2**: `slope` - Tensor with negative slope values of type T2. The shape of the tensor should be broadcastable to input 1. Required.
+
+**Outputs**
+
+* **1**: The result of element-wise PReLU operation applied for tensor from input 1 with slope values from input 2. A tensor of type T1 and shape matching shape of input *x* tensor.
+
+**Types**
+
+* *T1*: arbitrary supported floating point type.
+
+* *T2*: arbitrary supported floating point type.
+
+**Detailed description**
+Before performing addition operation, input tensor 2 with slope values is broadcasted to input 1.
+The broadcasting rules are aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+
+After broadcasting *PReLU* does the following for each input 1 element x:
+
+    f(x) = slope * x for x < 0; x for x >= 0

docs/ops/activation/ReLU_1.md

@@ -0,0 +1,41 @@
+## ReLU <a name="ReLU"></a>
+
+**Versioned name**: *ReLU-1*
+
+**Category**: *Activation*
+
+**Short description**: [Reference](http://caffe.berkeleyvision.org/tutorial/layers/relu.html)
+
+**Detailed description**: [Reference](https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions#rectified-linear-units)
+
+**Attributes**: *ReLU* operation has no attributes.
+
+**Mathematical Formulation**
+
+\f[
+Y_{i}^{( l )} = max(0, Y_{i}^{( l - 1 )})
+\f]
+
+**Inputs**:
+
+*   **1**: Multidimensional input tensor. Required.
+
+**Example**
+
+```xml
+<layer ... type="ReLU">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+
+```

docs/ops/activation/Sigmoid_1.md

@@ -0,0 +1,25 @@
+## Sigmoid<a name="Sigmoid"></a>
+
+**Versioned name**: *Sigmoid-1*
+
+**Category**: *Activation function*
+
+**Short description**: Sigmoid element-wise activation function.
+
+**Attributes**: operations has no attributes.
+
+**Inputs**:
+
+*   **1**: Input tensor *x* of any floating point type. Required.
+
+**Outputs**:
+
+*   **1**: Result of Sigmoid function applied to the input tensor *x*. Floating point tensor with shape and type matching the input tensor. Required.
+
+**Mathematical Formulation**
+
+   For each element from the input tensor calculates corresponding
+    element in the output tensor with the following formula:
+    \f[
+    sigmoid( x ) = \frac{1}{1+e^{-x}}
+    \f]

docs/ops/activation/SoftMax_1.md

@@ -0,0 +1,45 @@
+## SoftMax <a name="SoftMax"></a>
+
+**Versioned name**: *SoftMax-1*
+
+**Category**: *Activation*
+
+**Short description**: [Reference](https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions#softmax)
+
+**Detailed description**: [Reference](http://cs231n.github.io/linear-classify/#softmax)
+
+**Attributes**
+
+* *axis*
+
+  * **Description**: *axis* represents the axis of which the *SoftMax* is calculated. *axis* equal 1 is a default value.
+  * **Range of values**: positive integer value
+  * **Type**: int
+  * **Default value**: 1
+  * **Required**: *no*
+
+
+**Inputs**:
+
+*   **1**: Input tensor with enough number of dimension to be compatible with *axis* attribute. Required.
+
+**Outputs**:
+
+*   **1**: The resulting tensor of the same shape and type as input tensor.
+
+**Detailed description**
+
+\f[
+y_{c} = \frac{e^{Z_{c}}}{\sum_{d=1}^{C}e^{Z_{d}}}
+\f]
+where \f$C\f$ is a size of tensor along *axis* dimension.
+
+**Example**
+
+```xml
+<layer ... type="SoftMax" ... >
+    <data axis="1" />
+    <input> ... </input>
+    <output> ... </output>
+</layer>
+```

docs/ops/arithmetic/Abs_1.md

@@ -0,0 +1,51 @@
+## Abs <a name="Abs"></a>
+
+**Versioned name**: *Abs-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Abs* performs element-wise the absolute value with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise abs operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Abs* does the following with the input tensor *a*:
+
+\f[
+a_{i} = abs(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Abs">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+

docs/ops/arithmetic/Acos_1.md

@@ -0,0 +1,50 @@
+## Acos <a name="Acos"></a>
+
+**Versioned name**: *Acos-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Acos* performs element-wise inverse cosine (arccos) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise acos operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Acos* does the following with the input tensor *a*:
+
+\f[
+a_{i} = acos(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Acos">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Acosh_1.md

@@ -0,0 +1,50 @@
+## Acosh <a name="Acosh"></a>
+
+**Versioned name**: *Acosh-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Acosh* performs element-wise hyperbolic inverse cosine (arccosh) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise acosh operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Acosh* does the following with the input tensor *a*:
+
+\f[
+a_{i} = acosh(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Acosh">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Add_1.md

@@ -0,0 +1,93 @@
+## Add <a name="Add"></a>
+
+**Versioned name**: *Add-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Add* performs element-wise addition operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. Required.
+* **2**: A tensor of type T. Required.
+
+**Outputs**
+
+* **1**: The result of element-wise addition operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Add* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} + b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Add">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Add">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Asin_1.md

@@ -0,0 +1,50 @@
+## Asin <a name="Asin"></a>
+
+**Versioned name**: *Asin-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Asin* performs element-wise inverse sine (arcsin) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise asin operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Asin* does the following with the input tensor *a*:
+
+\f[
+a_{i} = asin(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Asin">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Asinh_1.md

@@ -0,0 +1,50 @@
+## Asinh <a name="Asinh"></a>
+
+**Versioned name**: *Asinh-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Asinh* performs element-wise hyperbolic inverse sine (arcsinh) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise asinh operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Asinh* does the following with the input tensor *a*:
+
+\f[
+a_{i} = asinh(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Asinh">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Atan_1.md

@@ -0,0 +1,50 @@
+## Atan <a name="Atan"></a>
+
+**Versioned name**: *Atan-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Atan* performs element-wise inverse tangent (arctangent) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise atan operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*atan* does the following with the input tensor *a*:
+
+\f[
+a_{i} = atan(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Atan">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Atanh_1.md

@@ -0,0 +1,50 @@
+## Atanh <a name="Atanh"></a>
+
+**Versioned name**: *Atanh-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Atanh* performs element-wise hyperbolic inverse tangent (arctangenth) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise atanh operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Atanh* does the following with the input tensor *a*:
+
+\f[
+a_{i} = atanh(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Atanh">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Ceiling_1.md

@@ -0,0 +1,50 @@
+## Ceiling <a name="Ceiling"></a>
+
+**Versioned name**: *Ceiling-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Ceiling* performs element-wise ceiling operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise ceiling operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Ceiling* does the following with the input tensor *a*:
+
+\f[
+a_{i} = ceiling(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Ceiling">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Cos_1.md

@@ -0,0 +1,50 @@
+## Cos <a name="Cos"></a>
+
+**Versioned name**: *Cos-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Cos* performs element-wise cosine operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise cos operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Cos* does the following with the input tensor *a*:
+
+\f[
+a_{i} = cos(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Cos">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Cosh_1.md

@@ -0,0 +1,50 @@
+## Cosh <a name="Cosh"></a>
+
+**Versioned name**: *Cosh-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Cosh* performs element-wise hyperbolic cosine operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise cosh operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Cosh* does the following with the input tensor *a*:
+
+\f[
+a_{i} = cosh(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Cosh">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/CumSum_3.md

@@ -0,0 +1,121 @@
+## CumSum <a name="CumSum"></a>
+
+**Versioned name**: *CumSum-3*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *CumSum* performs cumulative summation of the input elements along the given axis.
+ 
+**Detailed description**: By default, it will do the sum inclusively meaning the first element is copied as is. Through an "exclusive" attribute, this behavior can change to exclude the first element. It can also perform summation in the opposite direction of the axis. For that, set reverse attribute to `true`.
+
+**Attributes**:
+
+* *exclusive*
+
+  * **Description**: If the attribute is set to `true` then an exclusive sum in which the top element is not included is returned. In other terms, if set to `true`, the `j-th` output element would be the sum of the first `(j-1)` elements. Otherwise, it would be the sum of the first `j` elements.
+  * **Range of values**:
+    * `false` - include the top element
+    * `true` - do not include the top element
+  * **Type**: `boolean`
+  * **Default value**: `false`
+  * **Required**: *no*
+
+* *reverse*
+
+  * **Description**: If set to `true` will perform the sums in reverse direction.
+  * **Range of values**:
+    * `false` - do not perform sums in reverse direction
+    * `true` - perform sums in reverse direction
+  * **Type**: `boolean`
+  * **Default value**: `false`
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+* **2**: Scalar axis of type T_AXIS. Negative value means counting dimensions from the back. Default value is 0. **Optional.**
+
+**Outputs**
+
+* **1**: Output tensor with cumulative sums of the input's elements. A tensor of type T of the same shape as 1st input.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_AXIS*: any integer number.
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="CumSum" exclusive="0" reverse="0">
+    <input>
+        <port id="0">     <!-- input value is: [1., 2., 3., 4., 5.] -->
+            <dim>5</dim>
+        </port>
+        <port id="1"/>     <!-- axis value is: 0 -->
+    </input>
+    <output>
+        <port id="2">     <!-- output value is: [1., 3., 6., 10., 15.] -->
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2*
+
+```xml
+<layer ... type="CumSum" exclusive="1" reverse="0">
+    <input>
+        <port id="0">     <!-- input value is: [1., 2., 3., 4., 5.] -->
+            <dim>5</dim>
+        </port>
+        <port id="1"/>     <!-- axis value is: 0 -->
+    </input>
+    <output>
+        <port id="2">     <!-- output value is: [0., 1., 3., 6., 10.] -->
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 3*
+
+```xml
+<layer ... type="CumSum" exclusive="0" reverse="1">
+    <input>
+        <port id="0">     <!-- input value is: [1., 2., 3., 4., 5.] -->
+            <dim>5</dim>
+        </port>
+        <port id="1"/>     <!-- axis value is: 0 -->
+    </input>
+    <output>
+        <port id="2">     <!-- output value is: [15., 14., 12., 9., 5.] -->
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 4*
+
+```xml
+<layer ... type="CumSum" exclusive="1" reverse="1">
+    <input>
+        <port id="0">     <!-- input value is: [1., 2., 3., 4., 5.] -->
+            <dim>5</dim>
+        </port>
+        <port id="1"/>     <!-- axis value is: 0 -->
+    </input>
+    <output>
+        <port id="2">     <!-- output value is: [14., 12., 9., 5., 0.] -->
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Divide_1.md

@@ -0,0 +1,93 @@
+## Divide <a name="Divide"></a>
+
+**Versioned name**: *Divide-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Divide* performs element-wise division operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise division operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Divide* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} / b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Divide">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Divide">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Erf_1.md

@@ -0,0 +1,51 @@
+## Erf <a name="Erf"></a>
+
+**Versioned name**: *Erf-1*
+
+**Category**: Arithmetic unary operation
+
+**Short description**: *Erf* calculates the Gauss error function element-wise with given tensor.
+
+**Detailed Description**
+
+For each element from the input tensor calculates corresponding element in the output tensor with the following formula:
+\f[
+erf(x) = \pi^{-1} \int_{-x}^{x} e^{-t^2} dt
+\f]
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise operation. A tensor of type T.
+
+**Types**
+
+* *T*: any supported floating point type.
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Erf">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/FloorMod_1.md

@@ -0,0 +1,86 @@
+## FloorMod <a name="FloorMod"></a>
+
+**Versioned name**: *FloorMod-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *FloorMod* returns an element-wise division reminder with two given tensors applying multi-directional broadcast rules. 
+The result here is consistent with a flooring divide (like in Python programming language): `floor(x / y) * y + mod(x, y) = x`. 
+The sign of the result is equal to a sign of the divisor.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. Required.
+* **2**: A tensor of type T. Required.
+
+**Outputs**
+
+* **1**: The element-wise division reminder. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="FloorMod">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="FloorMod">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Floor_1.md

@@ -0,0 +1,50 @@
+## Floor <a name="Floor"></a>
+
+**Versioned name**: *Floor-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Floor* performs element-wise floor operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise floor operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Floor* does the following with the input tensor *a*:
+
+\f[
+a_{i} = floor(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Floor">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Log_1.md

@@ -0,0 +1,50 @@
+## Log <a name="Log"></a>
+
+**Versioned name**: *Log-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Log* performs element-wise natural logarithm operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise log operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Log* does the following with the input tensor *a*:
+
+\f[
+a_{i} = log(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Log">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Maximum_1.md

@@ -0,0 +1,93 @@
+## Maximum <a name="Maximum"></a>
+
+**Versioned name**: *Maximum-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Maximum* performs element-wise maximum operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: First input tensor of type T. Required.
+* **2**: Second input tensor of type T. Required.
+
+**Outputs**
+
+* **1**: The result of element-wise maximum operation. A tensor of type T.
+
+**Types**
+
+* *T*: arbitrary type, which supports less/greater comparison.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Maximum* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = max(a_{i}, b_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Maximum">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Maximum">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Minimum_1.md

@@ -0,0 +1,93 @@
+## Minimum <a name="Minimum"></a>
+
+**Versioned name**: *Minimum-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Minimum* performs element-wise minimum operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: First input tensor of type T. **Required.**
+* **2**: Second input tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise minimum operation. A tensor of type T.
+
+**Types**
+
+* *T*: arbitrary type, which supports less/greater comparison.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Minimum* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = min(a_{i}, b_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Minimum">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Minimum">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Mod_1.md

@@ -0,0 +1,86 @@
+## Mod <a name="Mod"></a>
+
+**Versioned name**: *Mod-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Mod* returns an element-wise division reminder with two given tensors applying multi-directional broadcast rules. 
+The result here is consistent with a truncated divide (like in C programming language): `truncated(x / y) * y + truncated_mod(x, y) = x`. 
+The sign of the result is equal to a sign of a dividend.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. Required.
+* **2**: A tensor of type T. Required.
+
+**Outputs**
+
+* **1**: The element-wise division reminder. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="FloorMod">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="FloorMod">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Multiply_1.md

@@ -0,0 +1,93 @@
+## Multiply <a name="Multiply"></a>
+
+**Versioned name**: *Multiply-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Multiply* performs element-wise multiplication operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise multiplication operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Multiply* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} * b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Multiply">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Multiply">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Negative_1.md

@@ -0,0 +1,50 @@
+## Negative <a name="Negative"></a>
+
+**Versioned name**: *Negative-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Negative* performs element-wise negative operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise negative operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Negative* does the following with the input tensor *a*:
+
+\f[
+a_{i} = -a_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Negative">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Power_1.md

@@ -0,0 +1,93 @@
+## Power <a name="Power"></a>
+
+**Versioned name**: *Power-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Power* performs element-wise power operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. Required.
+* **2**: A tensor of type T. Required.
+
+**Outputs**
+
+* **1**: The result of element-wise power operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Power* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = {a_{i} ^ b}_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Power">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Power">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Selu_1.md

@@ -0,0 +1,65 @@
+## Selu <a name="Selu"></a>
+
+**Versioned name**: *Selu-1*
+
+**Category**: Arithmetic unary operation
+
+**Short description**: *Selu* calculates the SELU activation function (https://arxiv.org/abs/1706.02515) element-wise with given tensor.
+
+**Detailed Description**
+
+For each element from the input tensor calculates corresponding
+element in the output tensor with the following formula:
+\f[
+selu(x) = \lambda \left\{\begin{array}{ll}
+    \alpha(e^{x} - 1) \quad \mbox{if } x \le 0 \\
+    x \quad \mbox{if } x >  0
+\end{array}\right.
+\f]
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+* **2**: `alpha` 1D tensor with one element of type T. **Required.**
+
+* **3**: `lambda` 1D tensor with one element of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise operation. A tensor of type T.
+
+**Types**
+
+* *T*: any supported floating point type.
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Selu">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>1</dim>
+        </port>
+        <port id="2">
+            <dim>1</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Sign_1.md

@@ -0,0 +1,50 @@
+## Sign <a name="Sign"></a>
+
+**Versioned name**: *Sign-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Sign* performs element-wise sign operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise sign operation. A tensor of type T with mapped elements of the input tensor to -1 (if it is negative), 0 (if it is zero), or 1 (if it is positive).
+
+**Types**
+
+* *T*: any numeric type.
+
+*Sign* does the following with the input tensor *a*:
+
+\f[
+a_{i} = sign(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Sign">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Sin_1.md

@@ -0,0 +1,50 @@
+## Sin <a name="Sin"></a>
+
+**Versioned name**: *Sin-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Sin* performs element-wise sine operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise sin operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*sin* does the following with the input tensor *a*:
+
+\f[
+a_{i} = sin(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Sin">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Sinh_1.md

@@ -0,0 +1,50 @@
+## Sinh <a name="Sinh"></a>
+
+**Versioned name**: *Sinh-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Sinh* performs element-wise hyperbolic sine (sinh) operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise sinh operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*sinh* does the following with the input tensor *a*:
+
+\f[
+a_{i} = sinh(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Sinh">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Sqrt_1.md

@@ -0,0 +1,50 @@
+## Sqrt <a name="Sqrt"></a>
+
+**Versioned name**: *Sqrt-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Sqrt* performs element-wise square root operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise sqrt operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Sqrt* does the following with the input tensor *a*:
+
+\f[
+a_{i} = sqrt(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Sqrt">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/SquaredDifference_1.md

@@ -0,0 +1,92 @@
+## SquaredDifference <a name="SquaredDifference"></a>
+
+**Versioned name**: *SquaredDifference-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *SquaredDifference* performs element-wise subtraction operation with two given tensors applying multi-directional broadcast rules, after that each result of the subtraction is squared.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise SquaredDifference operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *SquaredDifference* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = (a_{i} - b_{i})^2
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="SquaredDifference">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+*Example 2: broadcast*
+```xml
+<layer ... type="SquaredDifference">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Subtract_1.md

@@ -0,0 +1,92 @@
+## Subtract <a name="Subtract"></a>
+
+**Versioned name**: *Subtract-1*
+
+**Category**: Arithmetic binary operation
+
+**Short description**: *Subtract* performs element-wise subtraction operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise subtraction operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Subtract* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} - b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Substract">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+*Example 2: broadcast*
+```xml
+<layer ... type="Subtract">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/arithmetic/Tan_1.md

@@ -0,0 +1,51 @@
+## Tan <a name="Tan"></a>
+
+**Versioned name**: *Tan-1*
+
+**Category**: Arithmetic unary operation 
+
+**Short description**: *Tan* performs element-wise tangent operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise tan operation. A tensor of type T.
+
+**Types**
+
+* *T*: any numeric type.
+
+*Tan* does the following with the input tensor *a*:
+
+\f[
+a_{i} = tan(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Tan">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+

docs/ops/arithmetic/Tanh_1.md

@@ -0,0 +1,25 @@
+## Tanh<a name="Tanh"></a>
+
+**Versioned name**: *Tanh-1*
+
+**Category**: *Activation function*
+
+**Short description**: Tanh element-wise activation function.
+
+**Attributes**: has no attributes
+
+**Inputs**:
+
+*   **1**: Input tensor x of any floating point type. Required.
+
+**Outputs**:
+
+*   **1**: Result of Tanh function applied to the input tensor *x*. Floating point tensor with shape and type matching the input tensor. Required.
+
+**Detailed description**
+
+For each element from the input tensor calculates corresponding
+element in the output tensor with the following formula:
+\f[
+tanh ( x ) = \frac{2}{1+e^{-2x}} - 1 = 2sigmoid(2x) - 1
+\f]

docs/ops/comparison/Equal_1.md

@@ -0,0 +1,93 @@
+## Equal <a name="Equal"></a>
+
+**Versioned name**: *Equal-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *Equal* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Equal* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} == b_{i}
+\f]
+
+**Examples** 
+
+*Example 1*
+
+```xml
+<layer ... type="Equal">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Equal">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/comparison/GreaterEqual_1.md

@@ -0,0 +1,93 @@
+## GreaterEqual <a name="GreaterEqual"></a>
+
+**Versioned name**: *GreaterEqual-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *GreaterEqual* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *GreaterEqual* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} >= b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="GreaterEqual">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="GreaterEqual">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/comparison/Greater_1.md

@@ -0,0 +1,93 @@
+## Greater <a name="Greater"></a>
+
+**Versioned name**: *Greater-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *Greater* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Greater* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} > b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Greater">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Greater">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/comparison/LessEqual_1.md

@@ -0,0 +1,93 @@
+## LessEqual <a name="LessEqual"></a>
+
+**Versioned name**: *LessEqual-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *LessEqual* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *LessEqual* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} <= b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="LessEqual">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="LessEqual">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/comparison/Less_1.md

@@ -0,0 +1,93 @@
+## Less <a name="Less"></a>
+
+**Versioned name**: *Less-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *Less* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *Less* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} < b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Less">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="Less">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/comparison/NotEqual_1.md

@@ -0,0 +1,93 @@
+## NotEqual <a name="NotEqual"></a>
+
+**Versioned name**: *NotEqual-1*
+
+**Category**: Comparison binary operation
+
+**Short description**: *NotEqual* performs element-wise comparison operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+* **2**: A tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise comparison operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Detailed description**
+Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *NotEqual* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} != b_{i}
+\f]
+
+**Examples** 
+
+*Example 1*
+
+```xml
+<layer ... type="NotEqual">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="NotEqual">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/condition/Bucketize_3.md

@@ -0,0 +1,70 @@
+## Bucketize <a name="Bucketize"></a>
+
+**Versioned name**: *Bucketize-3*
+
+**Category**: Condition operation
+
+**Short description**: *Bucketize* bucketizes the input based on boundaries. This is similar to [Reference](https://www.tensorflow.org/api_docs/cc/class/tensorflow/ops/bucketize).
+
+**Detailed description**: *Bucketize* computes a bucket index for each element from the first input and outputs a tensor of the first input shape. Buckets are defined with boundaries from the second input.
+
+For example, if the first input tensor is `[[3, 50], [10, -1]]` and the second input is `[0, 5, 10]` with included right bound, the output will be `[[1, 3], [2, 0]]`.
+
+**Attributes**
+
+* *output_type*
+
+  * **Description**: the output tensor type
+  * **Range of values**: "i64" or "i32"
+  * **Type**: string
+  * **Default value**: "i64"
+  * **Required**: *No*
+
+* *with_right_bound*
+
+  * **Description**: indicates whether bucket includes the right or the left edge of interval.
+  * **Range of values**:
+    * True - bucket includes the right interval edge
+    * False - bucket includes the left interval edge
+  * **Type**: `boolean`
+  * **Default value**: True
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: N-D tensor of *T* type with elements for the bucketization. Required.
+*   **2**: 1-D tensor of *T_BOUNDARIES* type with sorted unique boundaries for buckets. Required.
+
+**Outputs**:
+
+*   **1**: Output tensor with bucket indices of *T_IND* type. If the second input is empty, the bucket index for all elements is equal to 0. The output tensor shape is the same as the first input tensor shape.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_BOUNDARIES*: any numeric type.
+
+* *T_IND*: `int32` or `int64`.
+
+**Example**
+
+```xml
+<layer ... type="Bucketize">
+    <input>
+        <port id="0">
+            <dim>49</dim>
+            <dim>11</dim>
+        </port>
+        <port id="1">
+            <dim>5</dim>
+        </port>
+     </input>
+    <output>
+        <port id="1">
+            <dim>49</dim>
+            <dim>11</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/condition/NonZero_3.md

@@ -0,0 +1,56 @@
+## NonZero <a name="NonZero"></a>
+
+**Versioned name**: *NonZero-3*
+
+**Category**: Condition operation
+
+**Short description**: *NonZero* returns the indices of the non-zero elements of the input tensor.
+
+**Detailed description**: *NonZero* returns the indices of the non-zero elements of the input tensor (in row-major order - by dimension).
+The output tensor has shape `[rank(input), num_non_zero]`. For example, for the tensor `[[1, 0], [1, 1]]` the output will be `[[0, 1, 1], [0, 0, 1]]`.
+
+**Attributes**
+
+* *output_type*
+
+  * **Description**: the output tensor type
+  * **Range of values**: "i64" or "i32"
+  * **Type**: string
+  * **Default value**: "i64"
+  * **Required**: *No*
+
+**Inputs**:
+
+*   **1**: `data` tensor of arbitrary rank of type *T*. Required.
+
+**Outputs**:
+
+*   **1**: tensor with indices of non-zero elements of shape `[rank(data), num_non_zero]` of type *T_IND*.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_IND*: `int64` or `int32`.
+
+**Example**
+
+```xml
+<layer ... type="NonZero">
+    <data output_type="i64"/>
+    <input>
+        <port id="0">
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>100</dim>
+            <dim>200</dim>
+        </port>
+     </input>
+    <output>
+        <port id="1">
+            <dim>4</dim>
+            <dim>600000</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/condition/Select_1.md

@@ -0,0 +1,66 @@
+## Select <a name="Select"></a>
+
+**Versioned name**: *Select-1*
+
+**Category**: *Conditions*
+
+**Short description**: *Select* returns a tensor filled with the elements from the second or the third inputs, depending on the condition (the first input) value.
+
+**Detailed description**
+
+*Select* takes elements from `then` input tensor or the `else` input tensor based on a condition mask
+ provided in the first input `cond`. Before performing selection, input tensors `then` and `else` are broadcasted to each other if their shapes are different and `auto_broadcast` attributes is not `none`. Then the `cond` tensor is one-way broadcasted to the resulting shape of broadcasted `then` and `else`. Broadcasting is performed according to `auto_broadcast` value.
+
+**Attributes**
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+
+**Inputs**:
+
+* **1**: `cond` tensor with selection mask of type `boolean`. The tensor can be 0D.
+
+* **2**: `then` the tensor with elements to take where the corresponding element in `cond` is true. Arbitrary type that should match type of `else` input tensor.
+
+* **3**: `else` the tensor with elements to take where the corresponding element in `cond` is false. Arbitrary type that should match type of `then` input tensor.
+
+
+**Outputs**:
+
+* **1**: blended output tensor that is tailored from values of inputs tensors `then` and `else` based on `cond` and broadcasting rules. It has the same type of elements as `then` and `else`.
+
+
+**Example**
+
+```xml
+<layer ... type="Select">
+    <input>
+        <port id="0">     <!-- cond value is: [[false, false], [true, false], [true, true]] -->
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+        <port id="1">     <!-- then value is: [[-1, 0], [1, 2], [3, 4]] -->
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+        <port id="2">     <!-- else value is: [[11, 10], [9, 8], [7, 6]] -->
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">     <!-- output value is: [[11, 10], [1, 8], [3, 4]] -->
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/convolution/BinaryConvolution_1.md

@@ -0,0 +1,38 @@
+## BinaryConvolution<a name="BinaryConvolution"></a>
+
+**Versioned name**: *BinaryConvolution-1*
+
+**Category**: *Convolution*
+
+**Short description**: *BinaryConvolution* convolution with binary weights, binary input and integer output
+
+**Attributes**:
+
+The operation has the same attributes as a regular *Convolution* layer and several unique attributes that are listed below:
+
+* *mode*
+
+  * **Description**: *mode* defines how input tensor 0/1 values and weights 0/1 are interpreted as real numbers and how the result is computed.
+  * **Range of values**:
+    * *xnor-popcount*
+  * **Type**: `string`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pad_value*
+
+  * **Description**: *pad_value* is a floating-point value used to fill pad area.
+  * **Range of values**: a floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: ND tensor with N >= 3, containing integer, float or binary values; filled with 0/1 values of any appropriate type. 0 means -1, 1 means 1 for `mode="xnor-popcount"`. Required.
+
+*   **2**: ND tensor with N >= 3 that represents convolutional kernel filled by integer, float or binary values; filled with 0/1 values. 0 means -1, 1 means 1 for `mode="xnor-popcount"`. Required.
+
+**Outputs**:
+
+*   **1**: output tensor containing float values. Required.

docs/ops/convolution/ConvolutionBackpropData_1.md

@@ -0,0 +1,134 @@
+## ConvolutionBackpropData<a name="ConvolutionBackpropData"></a>
+
+**Versioned name**: *ConvolutionBackpropData-1*
+
+**Category**: Convolution
+
+**Short description**: Computes the gradients of a Convolution operation with respect to the input. Also known as a Deconvolution or a Transposed Convolution.
+
+**Detailed description**:
+
+ConvolutionBackpropData takes the input tensor, weights tensor and output shape and computes the output tensor of a given shape. The shape of the output can be specified as an input 1D integer tensor explicitly or determined by other attributes implicitly. If output shape is specified as an explicit input, shape of the output exactly matches the specified size and required amount of padding is computed.
+
+ConvolutionBackpropData accepts the same set of attributes as a regular Convolution operation, but they are interpreted in a "backward way", so they are applied to the output of ConvolutionBackpropData, but not to the input. Refer to a regular Convolution operation for detailed description of each attribute.
+
+Output shape when specified as an input `output_shape`, specifies only spatial dimensions. No batch or channel dimension should be passed along with H, W or other spatial dimensions. If `output_shape` is omitted, then `pads_begin`, `pads_end` or `auto_pad` are used to determine output spatial shape `[Y_1, Y_2, ..., Y_D]` by input spatial shape `[X_1, X_2, ..., X_D]` in the following way:
+
+```
+if auto_pads != None:
+    pads_begin[i] = 0
+    pads_end[i] = 0
+
+Y_i = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - pads_begin[i] - pads_end[i] + output_padding[i]
+```
+
+where `K_i` filter kernel dimension along spatial axis `i`.
+
+ If `output_shape` is specified, `pads_begin` and `pads_end` are ignored, and `auto_pad` defines how to distribute padding amount around the tensor. In this case pads are determined based on the next formulas to correctly align input and output tensors (similar to ONNX definition at https://github.com/onnx/onnx/blob/master/docs/Operators.md#convtranspose):
+
+```
+total_padding[i] = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - output_shape[i] + output_padding[i]
+if auto_pads != SAME_UPPER:
+    pads_begin[i] = total_padding[i] // 2
+    pads_end[i] = total_padding[i] - pads_begin[i]
+else:
+    pads_end[i] = total_padding[i] // 2
+    pads_begin[i] = total_padding[i] - pads_end[i]
+```
+
+**Attributes**
+
+* *strides*
+
+  * **Description**: *strides* has the same definition as *strides* for a regular Convolution but applied in the backward way, for the output tensor.
+  * **Range of values**: positive integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pads_begin*
+
+  * **Description**: *pads_begin* has the same definition as *pads_begin* for a regular Convolution but applied in the backward way, for the output tensor. May be omitted specified, in which case pads are calculated automatically.
+  * **Range of values**: non-negative integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *pads_end*
+
+  * **Description**: *pads_end* has the same definition as *pads_end* for a regular Convolution but applied in the backward way, for the output tensor. May be omitted, in which case pads are calculated automatically.
+  * **Range of values**: non-negative integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+  
+* *dilations*
+
+  * **Description**: *dilations* has the same definition as *dilations* for a regular Convolution but applied in the backward way, for the output tensor.
+  * **Range of values**: positive integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* has the same definition as *auto_pad* for a regular Convolution but applied in the backward way, for the output tensor. 
+    * None (not specified): use explicit padding values from `pads_begin` and `pads_end`.
+    * *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *no*
+  * **Note**: *pads_begin* and *pads_end* attributes are ignored when *auto_pad* is specified.
+
+* *output_padding*
+
+  * **Description**: *output_padding* adds additional amount of paddings per each spatial axis in the `output` tensor. It unlocks more elements in the output allowing them to be computed. Elements are added at the higher coordinate indices for the spatial dimensions. Number of elements in *output_padding* list matches the number of spatial dimensions in `data` and `output` tensors.
+  * **Range of values**: non-negative integer values
+  * **Type**: int[]
+  * **Default value**: all zeros
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: `data` -- input tensor of rank 3 or greater. Layout is `[N, C_INPUT, X1, ..., XD]`. *Required*.
+
+*   **2**: `filter` -- convolution kernel tensor. Weights have shape `[C_INPUT, C_OUTPUT, K_D, ..., K_1]`. `C_INPUT` is the number of channels in input `data` tensor shape, and `C_OUTPUT` is the number of channels in the `output` tensor. Spatial size of the kernel `[K_D, ..., K_1]` is derived from the shape of this input and aren't specified by any attribute. *Required*.
+
+*   **3**: `output_shape` is 1D integer tensor that specifies spatial shape of the output. *Optional*. If specified, *padding amount* is deduced from relation of input and output spatial shapes according to formulas in the description. If not specified, *output shape* is calculated based on the `pads_begin` and `pads_end` or completely according to `auto_pad`.
+
+**Outputs**:
+
+*   **1**: `output` -- output tensor of the same rank as input `data` tensor and shape `[N, C_OUTPUT, Y1, ..., YD]`.
+
+**Example**
+
+```xml
+<layer id="5" name="upsampling_node" type="ConvolutionBackpropData">
+    <data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>20</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+        <port id="1">
+            <dim>20</dim>
+            <dim>10</dim>
+            <dim>3</dim>
+            <dim>3</dim>
+        </port>
+    </input>
+    <output>
+        <port id="0" precision="FP32">
+            <dim>1</dim>
+            <dim>10</dim>
+            <dim>447</dim>
+            <dim>447</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/convolution/Convolution_1.md

@@ -0,0 +1,114 @@
+## Convolution<a name="Convolution"></a>
+
+**Versioned name**: *Convolution-1*
+
+**Category**: Convolution
+
+**Short description**: [Reference](http://caffe.berkeleyvision.org/tutorial/layers/convolution.html)
+
+**Detailed description**: [Reference](http://cs231n.github.io/convolutional-networks/#conv)
+
+
+*   For the convolutional layer, the number of output features in each dimension is calculated using the formula:
+\f[
+n_{out} = \left ( \frac{n_{in} + 2p - k}{s} \right ) + 1
+\f]
+*   The receptive field in each layer is calculated using the formulas:
+    *   Jump in the output feature map:
+        \f[
+        j_{out} = j_{in} * s
+        \f]
+    *   Size of the receptive field of output feature:
+        \f[
+        r_{out} = r_{in} + ( k - 1 ) * j_{in}
+        \f]
+    *   Center position of the receptive field of the first output feature:
+        \f[
+        start_{out} = start_{in} + ( \frac{k - 1}{2} - p ) * j_{in}
+        \f]
+    *   Output is calculated using the following formula:
+        \f[
+        out = \sum_{i = 0}^{n}w_{i}x_{i} + b
+        \f]
+
+**Attributes**
+
+* *strides*
+
+  * **Description**: *strides* is a distance (in pixels) to slide the filter on the feature map over the (z, y, x) axes for 3D convolutions and (y, x) axes for 2D convolutions. For example, *strides* equal *4,2,1* means sliding the filter 4 pixel at a time over depth dimension, 2 over height dimension and 1 over width dimension.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pads_begin*
+
+  * **Description**: *pads_begin* is a number of pixels to add to the beginning along each axis. For example, *pads_begin* equal *1,2* means adding 1 pixel to the top of the input and 2 to the left of the input.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *pads_end*
+
+  * **Description**: *pads_end* is a number of pixels to add to the ending along each axis. For example, *pads_end* equal *1,2* means adding 1 pixel to the bottom of the input and 2 to the right of the input.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *dilations*
+
+  * **Description**: *dilations* denotes the distance in width and height between elements (weights) in the filter. For example, *dilation* equal *1,1* means that all the elements in the filter are neighbors, so it is the same as for the usual convolution. *dilation* equal *2,2* means that all the elements in the filter are matched not to adjacent elements in the input matrix, but to those that are adjacent with distance 1.
+  * **Range of values**: integer value starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* how the padding is calculated. Possible values:
+    * None (not specified): use explicit padding values.
+    * *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *no*
+  * **Note**: *pads_begin* and *pads_end* attributes are ignored when *auto_pad* is specified.
+
+**Inputs**:
+
+*   **1**: Input tensor of rank 3 or greater. Required.
+*   **2**: Convolution kernel tensor. Weights layout is OIYX (OIZYX for 3D convolution), which means that *X* is changing the fastest, then *Y*, then *Input*, then *Output*. The size of the kernel is derived from the shape of this input and not specified by any attribute. Required.
+
+**Example**
+
+```xml
+<layer type="Convolution" ...>
+    <data dilations="1,1" pads_begin="2,2" pads_end="2,2" strides="1,1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+        <port id="1">
+            <dim>64</dim>
+            <dim>3</dim>
+            <dim>5</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2" precision="FP32">
+            <dim>1</dim>
+            <dim>64</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/convolution/DeformableConvolution_1.md

@@ -0,0 +1,88 @@
+## DeformableConvolution<a name="DeformableConvolution"></a>
+
+**Versioned name**: *DeformableConvolution-1*
+
+**Category**: Convolution
+
+**Detailed description**: [Reference](https://arxiv.org/abs/1703.06211)
+
+**Attributes**
+
+* *strides*
+
+  * **Description**: *strides* is a distance (in pixels) to slide the filter on the feature map over the (z, y, x) axes for 3D convolutions and (y, x) axes for 2D convolutions. For example, *strides* equal *4,2,1* means sliding the filter 4 pixel at a time over depth dimension, 2 over height dimension and 1 over width dimension.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pads_begin*
+
+  * **Description**: *pads_begin* is a number of pixels to add to the beginning along each axis. For example, *pads_begin* equal *1,2* means adding 1 pixel to the top of the input and 2 to the left of the input.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *pads_end*
+
+  * **Description**: *pads_end* is a number of pixels to add to the ending along each axis. For example, *pads_end* equal *1,2* means adding 1 pixel to the bottom of the input and 2 to the right of the input.
+  * **Range of values**: integer values starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *dilations*
+
+  * **Description**: *dilations* denotes the distance in width and height between elements (weights) in the filter. For example, *dilation* equal *1,1* means that all the elements in the filter are neighbors, so it is the same as for the usual convolution. *dilation* equal *2,2* means that all the elements in the filter are matched not to adjacent elements in the input matrix, but to those that are adjacent with distance 1.
+  * **Range of values**: integer value starting from 0
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* how the padding is calculated. Possible values:
+    * None (not specified): use explicit padding values.
+    * *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *no*
+  * **Note**: *pads_begin* and *pads_end* attributes are ignored when *auto_pad* is specified.
+
+* *group*
+
+  * **Description**: *group* is the number of groups which *output* and *input* should be split into. For example, *group* equal to 1 means that all filters are applied to the whole input (usual convolution), *group* equal to 2 means that both *input* and *output* channels are separated into two groups and the *i-th output* group is connected to the *i-th input* group channel. *group* equal to a number of output feature maps implies depth-wise separable convolution.
+  * **Range of values**: integer value starting from 1
+  * **Type**: int
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *deformable_group*
+
+  * **Description**: *deformable_group* is the number of groups which deformable values and *output* should be split into along the channel axis. Apply the deformable convolution using the i-th part of the offset part on the i-th out.
+  * **Range of values**: integer value starting from 1
+  * **Type**: int
+  * **Default value**: 1
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: Input tensor of rank 3 or greater. Required.
+
+*   **2**: Deformable values tensor of rank 3 or higher. Required.
+
+*   **3**: Convolution kernel tensor. Weights layout is OIYX (OIZYX for 3D convolution), which means that *X* is changing the fastest, then *Y*, then *Input* then *Output*. The size of kernel is derived from the shape of this input and not specified by any attribute. Required.
+
+**Example**
+
+```xml
+<layer ... type="DeformableConvolution" ... >
+        <data dilations="1,1" pads_begin="2,2" pads_end="3,3" strides="2,2"/>
+        <input> ... </input>
+        <output> ... </output>
+</layer>
+```

docs/ops/convolution/GroupConvolutionBackpropData_1.md

@@ -0,0 +1,137 @@
+## GroupConvolutionBackpropData<a name="GroupConvolutionBackpropData"></a>
+
+**Versioned name**: *GroupConvolutionBackpropData-1*
+
+**Category**: Convolution
+
+**Short description**: Computes the gradients of a GroupConvolution operation with respect to the input. Also known as Deconvolution or Transposed Convolution.
+
+**Detailed description**:
+
+GroupConvolutionBackpropData is similar to ConvolutionBackpropData but also specifies the group processing in a way similar to how GroupConvolution extends behavior of a regular Convolution operation.
+
+GroupConvolutionBackpropData takes input tensor, weights tensor and output shape and computes output tensor of a given shape. The shape of the output can be specified as an input 1D integer tensor explicitly or determined according to other attributes implicitly. If the output shape is specified as an explicit input, shape of the output exactly matches the specified size and required amount of padding is computed.
+
+GroupConvolutionBackpropData accepts the same set of attributes as a regular GroupConvolution operation, but they are interpreted in a "backward way", so they are applied to the output of GroupConvolutionBackpropData, but not to the input. Refer to a regular GroupConvolution operation for detailed description of each attribute.
+
+Output shape when specified as an input `output_shape`, specifies only spatial dimensions. No batch or channel dimension should be passed along with H, W or other spatial dimensions. If `output_shape` is omitted, then `pads_begin`, `pads_end` or `auto_pad` are used to determine output spatial shape `[Y_1, Y_2, ..., Y_D]` by input spatial shape `[X_1, X_2, ..., X_D]` in the following way:
+
+```
+if auto_pads != None:
+    pads_begin[i] = 0
+    pads_end[i] = 0
+
+Y_i = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - pads_begin[i] - pads_end[i] + output_padding[i]
+```
+
+where `K_i` filter kernel dimension along spatial axis `i`.
+
+ If `output_shape` is specified, `pads_begin` and `pads_end` are ignored, and `auto_pad` defines how to distribute padding amount around the tensor. In this case pads are determined based on the next formulas to correctly align input and output tensors (similar to ONNX definition at https://github.com/onnx/onnx/blob/master/docs/Operators.md#convtranspose):
+
+```
+total_padding[i] = stride[i] * (X_i - 1) + ((K_i - 1) * dilations[i] + 1) - output_shape[i] + output_padding[i]
+if auto_pads != SAME_UPPER:
+    pads_begin[i] = total_padding[i] // 2
+    pads_end[i] = total_padding[i] - pads_begin[i]
+else:
+    pads_end[i] = total_padding[i] // 2
+    pads_begin[i] = total_padding[i] - pads_end[i]
+```
+
+**Attributes**
+
+* *strides*
+
+  * **Description**: *strides* has the same definition as *strides* for a regular Convolution but applied in the backward way, for the output tensor.
+  * **Range of values**: positive integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pads_begin*
+
+  * **Description**: *pads_begin* has the same definition as *pads_begin* for a regular Convolution but applied in the backward way, for the output tensor. May be omitted, in which case pads are calculated automatically.
+  * **Range of values**: non-negative integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *pads_end*
+
+  * **Description**: *pads_end* has the same definition as *pads_end* for a regular Convolution but applied in the backward way, for the output tensor. May be omitted, in which case pads are calculated automatically.
+  * **Range of values**: non-negative integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified.
+
+* *dilations*
+
+  * **Description**: *dilations* has the same definition as *dilations* for a regular Convolution but applied in the backward way, for the output tensor.
+  * **Range of values**: positive integers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* has the same definition as *auto_pad* for a regular Convolution but applied in the backward way, for the output tensor. 
+    * None (not specified): use explicit padding values from `pads_begin` and `pads_end`.
+    * *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *no*
+  * **Note**: *pads_begin* and *pads_end* attributes are ignored when *auto_pad* is specified.
+
+* *output_padding*
+
+  * **Description**: *output_padding* adds additional amount of paddings per each spatial axis in the `output` tensor. It unlocks more elements in the output allowing them to be computed. Elements are added at the higher coordinate indices for the spatial dimensions. Number of elements in *output_padding* list matches the number of spatial dimensions in `data` and `output` tensors.
+  * **Range of values**: non-negative integer values
+  * **Type**: int[]
+  * **Default value**: all zeros
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: `data` -- input tensor of rank 3 or greater. Layout is `[N, C_INPUT * GROUPS, X1, ..., XD]`, where `GROUPS` is the number of groups that is specified as a dedicated dimension in `filter` input. *Required*.
+
+*   **2**: `filter` -- convolution kernel tensor. Weights have shape `[GROUPS, C_INPUT, C_OUTPUT, K_D, ..., K_1]`. `C_INPUT` is the number of channels in input `data` tensor shape, and `C_OUTPUT` is the number of channels in the `output` tensor. `GROUPS` is the number of groups in input/output channel dimension. Spatial size of the kernel `[K_D, ..., K_1]` is derived from the shape of this input and not specified by any attribute. *Required*.
+
+*   **3**: `output_shape` is 1D integer tensor that specifies spatial shape of the output. *Optional*. If specified, *padding amount* is deduced from relation of input and output spatial shapes according to formulas in the description. If not specified, *output shape* is calculated based on the `pads_begin` and `pads_end` or completely according to `auto_pad`.
+
+**Outputs**:
+
+*   **1**: `output` -- output tensor of the same rank as input `data` tensor and shape `[N, GROUPS * C_OUTPUT, Y1, ..., YD]`, where `GROUPS` is the number of groups that is specified as a dedicated dimension in `filter` input.
+
+**Example**
+
+```xml
+<layer id="5" name="upsampling_node" type="GroupConvolutionBackpropData">
+    <data dilations="1,1" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>20</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+        <port id="1">
+            <dim>4</dim>
+            <dim>5</dim>
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>3</dim>
+        </port>
+    </input>
+    <output>
+        <port id="0" precision="FP32">
+            <dim>1</dim>
+            <dim>8</dim>
+            <dim>447</dim>
+            <dim>447</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/convolution/GroupConvolution_1.md

@@ -0,0 +1,117 @@
+## GroupConvolution <a name="GroupConvolution"></a>
+
+**Versioned name**: *GroupConvolution-1*
+
+**Category**: Convolution
+
+**Short description**: [Reference](http://caffe.berkeleyvision.org/tutorial/layers/convolution.html)
+
+**Detailed description**: [Reference](http://cs231n.github.io/convolutional-networks/#conv)
+
+**Attributes**
+
+* *strides*
+
+  * **Description**: *strides* is a distance (in pixels) to slide the filter on the feature map over the (z, y, x) axes for 3D convolutions and (y, x) axes for 2D convolutions. For example, *strides* equal *4,2,1* means sliding the filter 4 pixel at a time over depth dimension, 2 over height dimension and 1 over width dimension.
+  * **Range of values**: positive integer numbers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pads_begin*
+
+  * **Description**: *pads_begin* is a number of pixels to add to the beginning along each axis. For example, *pads_begin* equal *1,2* means adding 1 pixel to the top of the input and 2 to the left of the input.
+  * **Range of values**: positive integer numbers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified. 
+
+* *pads_end*
+
+  * **Description**: *pads_end* is a number of pixels to add to the ending along each axis. For example, *pads_end* equal *1,2* means adding 1 pixel to the bottom of the input and 2 to the right of the input.
+  * **Range of values**: positive integer numbers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+  * **Note**: the attribute is ignored when *auto_pad* attribute is specified. 
+
+* *dilations*
+
+  * **Description**: *dilations* denotes the distance in width and height between elements (weights) in the filter. For example, *dilation* equal *1,1* means that all the elements in the filter are neighbors, so it is the same as for the usual convolution. *dilation* equal *2,2* means that all the elements in the filter are matched not to adjacent elements in the input matrix, but to those that are adjacent with distance 1.
+  * **Range of values**: positive integer numbers
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* how the padding is calculated. Possible values:
+    * None (not specified): use explicit padding values.
+    * *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *no*
+  * **Note**: *pads_begin* and *pads_end* attributes are ignored when *auto_pad* is specified.
+
+**Inputs**:
+
+*   **1**: 4D or 5D input tensor. Required.
+
+*   **2**: Convolution kernel tensor. Weights layout is GOIYX (GOIZYX for 3D convolution), which means that *X* is changing the fastest, then *Y*, then *Input*, *Output* and *Group*. The size of kernel and number of groups are derived from the shape of this input and aren't specified by any attribute. Required.
+
+
+**Mathematical Formulation**
+
+*   For the convolutional layer, the number of output features in each dimension is calculated using the formula:
+\f[
+n_{out} = \left ( \frac{n_{in} + 2p - k}{s} \right ) + 1
+\f]
+*   The receptive field in each layer is calculated using the formulas:
+    *   Jump in the output feature map:
+        \f[
+        j_{out} = j_{in} * s
+        \f]
+    *   Size of the receptive field of output feature:
+        \f[
+        r_{out} = r_{in} + ( k - 1 ) * j_{in}
+        \f]
+    *   Center position of the receptive field of the first output feature:
+        \f[
+        start_{out} = start_{in} + ( \frac{k - 1}{2} - p ) * j_{in}
+        \f]
+    *   Output is calculated using the following formula:
+        \f[
+        out = \sum_{i = 0}^{n}w_{i}x_{i} + b
+        \f]
+
+**Example**
+
+```xml
+<layer type="GroupConvolution" ...>
+    <data dilations="1,1" pads_begin="2,2" pads_end="2,2" strides="1,1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>12</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+        <port id="1">
+            <dim>4</dim>
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>5</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2" precision="FP32">
+            <dim>1</dim>
+            <dim>4</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </output>
+```

docs/ops/detection/DeformablePSROIPooling_1.md

@@ -0,0 +1,121 @@
+## DeformablePSROIPooling <a name="DeformablePSROIPooling"></a>
+
+**Versioned name**: *DeformablePSROIPooling-1*
+
+**Category**: Object detection
+
+**Short description**: *DeformablePSROIPooling* computes position-sensitive pooling on regions of interest specified by input.
+
+**Detailed description**: [Reference](https://arxiv.org/abs/1703.06211).
+
+*DeformablePSROIPooling* operation takes two or three input tensors: with feature maps, with regions of interests (box coordinates) and an optional tensor with transformation values.
+The box coordinates are specified as five element tuples: *[batch_id, x_1, y_1, x_2, y_2]* in absolute values.
+
+**Attributes**
+
+* *output_dim*
+
+  * **Description**: *output_dim* is a pooled output channel number.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *group_size*
+
+  * **Description**: *group_size* is the number of groups to encode position-sensitive score maps.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *spatial_scale*
+
+  * **Description**: *spatial_scale* is a multiplicative spatial scale factor to translate ROI coordinates from their input scale to the scale used when pooling.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *mode*
+  * **Description**: *mode* specifies mode for pooling.
+  * **Range of values**:
+    * *bilinear_deformable* - perform pooling with bilinear interpolation and deformable transformation
+  * **Type**: string
+  * **Default value**: *bilinear_deformable*
+  * **Required**: *no*
+
+* *spatial_bins_x*
+  * **Description**: *spatial_bins_x* specifies numbers of bins to divide the input feature maps over width.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *spatial_bins_y*
+  * **Description**: *spatial_bins_y* specifies numbers of bins to divide the input feature maps over height.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *trans_std*
+  * **Description**: *trans_std* is the value that all transformation (offset) values are multiplied with.
+  * **Range of values**: floating point number
+  * **Type**: `float`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *part_size*
+  * **Description**: *part_size* is the number of parts the output tensor spatial dimensions are divided into. Basically it is the height and width of the third input 
+  with transformation values.
+  * **Range of values**: positive integer number
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: 4D input tensor with feature maps. Required.
+
+*   **2**: 2D input tensor describing box consisting of five element tuples: `[batch_id, x_1, y_1, x_2, y_2]`. Required.
+
+*   **3**: 4D input blob with transformation [values](https://arxiv.org/abs/1703.06211) (offsets). Optional.
+
+**Outputs**:
+
+*   **1**: 4D output tensor with areas copied and interpolated from the 1st input tensor by coordinates of boxes from the 2nd input and transformed according to values from the 3rd input.
+
+**Example**
+
+```xml
+<layer ... type="DeformablePSROIPooling" ... >
+    <data group_size="7" mode="bilinear_deformable" no_trans="False" output_dim="8" part_size="7" pooled_height="7" pooled_width="7" spatial_bins_x="4" spatial_bins_y="4" spatial_scale="0.0625" trans_std="0.1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>392</dim>
+            <dim>38</dim>
+            <dim>63</dim>
+        </port>
+        <port id="1">
+            <dim>300</dim>
+            <dim>5</dim>
+        </port>
+        <port id="2">
+            <dim>300</dim>
+            <dim>2</dim>
+            <dim>7</dim>
+            <dim>7</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3" precision="FP32">
+            <dim>300</dim>
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>7</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/detection/DetectionOutput_1.md

@@ -0,0 +1,153 @@
+## DetectionOutput <a name="DetectionOutput"></a>
+
+**Versioned name**: *DetectionOutput-1*
+
+**Category**: *Object detection*
+
+**Short description**: *DetectionOutput* performs non-maximum suppression to generate the detection output using information on location and confidence predictions.
+
+**Detailed description**: [Reference](https://arxiv.org/pdf/1512.02325.pdf). The layer has 3 mandatory inputs: tensor with box logits, tensor with confidence predictions and tensor with box coordinates (proposals). It can have 2 additional inputs with additional confidence predictions and box coordinates described in the [article](https://arxiv.org/pdf/1711.06897.pdf). The 5-input version of the layer is supported with Myriad plugin only. The output tensor contains information about filtered detections described with 7 element tuples: *[batch_id, class_id, confidence, x_1, y_1, x_2, y_2]*. The first tuple with *batch_id* equal to *-1* means end of output.
+
+At each feature map cell, *DetectionOutput* predicts the offsets relative to the default box shapes in the cell, as well as the per-class scores that indicate the presence of a class instance in each of those boxes. Specifically, for each box out of k at a given location, *DetectionOutput* computes class scores and the four offsets relative to the original default box shape. This results in a total of \f$(c + 4)k\f$ filters that are applied around each location in the feature map, yielding \f$(c + 4)kmn\f$ outputs for a *m \* n* feature map.
+
+**Attributes**:
+
+* *num_classes*
+
+  * **Description**: number of classes to be predicted
+  * **Range of values**: positive integer number
+  * **Type**: int
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *background_label_id*
+
+  * **Description**: background label id. If there is no background class, set it to -1.
+  * **Range of values**: integer values
+  * **Type**: int
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *top_k*
+
+  * **Description**: maximum number of results to be kept per batch after NMS step. -1 means keeping all bounding boxes.
+  * **Range of values**: integer values
+  * **Type**: int
+  * **Default value**: -1
+  * **Required**: *no*
+
+* *variance_encoded_in_target*
+
+  * **Description**: *variance_encoded_in_target* is a flag that denotes if variance is encoded in target. If flag is false then it is necessary to adjust the predicted offset accordingly.
+  * **Range of values**: False or True
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+* *keep_top_k*
+
+  * **Description**: maximum number of bounding boxes per batch to be kept after NMS step. -1 means keeping all bounding boxes after NMS step.
+  * **Range of values**: integer values
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *code_type*
+
+  * **Description**: type of coding method for bounding boxes
+  * **Range of values**: "caffe.PriorBoxParameter.CENTER_SIZE", "caffe.PriorBoxParameter.CORNER"
+  * **Type**: string
+  * **Default value**: "caffe.PriorBoxParameter.CORNER"
+  * **Required**: *no*
+
+* *share_location*
+
+  * **Description**: *share_location* is a flag that denotes if bounding boxes are shared among different classes.
+  * **Range of values**: 0 or 1
+  * **Type**: int
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *nms_threshold*
+
+  * **Description**: threshold to be used in the NMS stage
+  * **Range of values**: floating point values
+  * **Type**: float
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *confidence_threshold*
+
+  * **Description**: only consider detections whose confidences are larger than a threshold. If not provided, consider all boxes.
+  * **Range of values**: floating point values
+  * **Type**: float
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *clip_after_nms*
+
+  * **Description**: *clip_after_nms* flag that denotes whether to perform clip bounding boxes after non-maximum suppression or not.
+  * **Range of values**: 0 or 1
+  * **Type**: int
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *clip_before_nms*
+
+  * **Description**: *clip_before_nms* flag that denotes whether to perform clip bounding boxes before non-maximum suppression or not.
+  * **Range of values**: 0 or 1
+  * **Type**: int
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *decrease_label_id*
+
+  * **Description**: *decrease_label_id* flag that denotes how to perform NMS.
+  * **Range of values**:
+    * 0 - perform NMS like in Caffe\*.
+    * 1 - perform NMS like in MxNet\*.
+  * **Type**: int
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *normalized*
+
+  * **Description**: *normalized* flag that denotes whether input tensors with boxes are normalized. If tensors are not normalized then *input_height* and *input_width* attributes are used to normalize box coordinates.
+  * **Range of values**: 0 or 1
+  * **Type**: int
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *input_height (input_width)*
+
+  * **Description**: input image height (width). If the *normalized* is 1 then these attributes are not used.
+  * **Range of values**: positive integer number
+  * **Type**: int
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *objectness_score*
+
+  * **Description**: threshold to sort out confidence predictions. Used only when the *DetectionOutput* layer has 5 inputs.
+  * **Range of values**: non-negative float number
+  * **Type**: float
+  * **Default value**: 0
+  * **Required**: *no*
+  
+**Inputs**
+
+* **1**: 2D input tensor with box logits. Required.
+* **2**: 2D input tensor with class predictions. Required.
+* **3**: 3D input tensor with proposals. Required.
+* **4**: 2D input tensor with additional class predictions information described in the [article](https://arxiv.org/pdf/1711.06897.pdf). Optional.
+* **5**: 2D input tensor with additional box predictions information described in the [article](https://arxiv.org/pdf/1711.06897.pdf). Optional.
+
+**Example**
+
+```xml
+<layer ... type="DetectionOutput" ... >
+    <data num_classes="21" share_location="1" background_label_id="0" nms_threshold="0.450000" top_k="400" input_height="1" input_width="1" code_type="caffe.PriorBoxParameter.CENTER_SIZE" variance_encoded_in_target="0" keep_top_k="200" confidence_threshold="0.010000"/>
+    <input> ... </input>
+    <output> ... </output>
+</layer>
+```

docs/ops/detection/PSROIPooling_1.md

@@ -0,0 +1,100 @@
+## PSROIPooling <a name="PSROIPooling"></a>
+
+**Versioned name**: *PSROIPooling-1*
+
+**Category**: Object detection
+
+**Short description**: *PSROIPooling* computes position-sensitive pooling on regions of interest specified by input.
+
+**Detailed description**: [Reference](https://arxiv.org/pdf/1703.06211.pdf).
+
+*PSROIPooling* operation takes two input blobs: with feature maps and with regions of interests (box coordinates). 
+The latter is specified as five element tuples: *[batch_id, x_1, y_1, x_2, y_2]*. 
+ROIs coordinates are specified in absolute values for the average mode and in normalized values (to *[0,1]* interval) for bilinear interpolation.
+
+**Attributes**
+
+* *output_dim*
+
+  * **Description**: *output_dim* is a pooled output channel number.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *group_size*
+
+  * **Description**: *group_size* is the number of groups to encode position-sensitive score maps. Use for *average* mode only.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *spatial_scale*
+
+  * **Description**: *spatial_scale* is a multiplicative spatial scale factor to translate ROI coordinates from their input scale to the scale used when pooling.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *mode*
+  * **Description**: *mode* specifies mode for pooling.
+  * **Range of values**:
+    * *average* - perform average pooling
+    * *bilinear* - perform pooling with bilinear interpolation
+  * **Type**: string
+  * **Default value**: *average*
+  * **Required**: *no*
+
+* *spatial_bins_x*
+  * **Description**: *spatial_bins_x* specifies numbers of bins to divide the input feature maps over width. Used for "bilinear" mode only.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *spatial_bins_y*
+  * **Description**: *spatial_bins_y* specifies numbers of bins to divide the input feature maps over height.  Used for "bilinear" mode only.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: 4D input blob with feature maps. Required.
+
+*   **2**: 2D input blob describing box consisting of five element tuples: `[batch_id, x_1, y_1, x_2, y_2]`. Required.
+
+**Outputs**:
+
+*   **1**: 4D output tensor with areas copied and interpolated from the 1st input tensor by coordinates of boxes from the 2nd input.
+
+**Example**
+
+```xml
+<layer ... type="PSROIPooling" ... >
+    <data group_size="6" mode="bilinear" output_dim="360" spatial_bins_x="3" spatial_bins_y="3" spatial_scale="1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3240</dim>
+            <dim>38</dim>
+            <dim>38</dim>
+        </port>
+        <port id="1">
+            <dim>100</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>100</dim>
+            <dim>360</dim>
+            <dim>6</dim>
+            <dim>6</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/detection/PriorBoxClustered_1.md

@@ -0,0 +1,129 @@
+## PriorBoxClustered <a name="PriorBoxClustered"></a>
+
+**Versioned name**: *PriorBoxClustered-1*
+
+**Category**: Object detection
+
+**Short description**: *PriorBoxClustered* operation generates prior boxes of specified sizes normalized to the input image size.
+
+**Attributes**
+
+* *width (height)*
+
+  * **Description**: *width (height)* specifies desired boxes widths (heights) in pixels.
+  * **Range of values**: floating point positive numbers
+  * **Type**: float[]
+  * **Default value**: 1.0
+  * **Required**: *no*
+
+* *clip*
+
+  * **Description**: *clip* is a flag that denotes if each value in the output tensor should be clipped within [0,1].
+  * **Range of values**:
+    * False - clipping is not performed
+    * True  - each value in the output tensor is within [0,1]
+  * **Type**: boolean
+  * **Default value**: True
+  * **Required**: *no*
+
+* *step (step_w, step_h)*
+
+  * **Description**: *step (step_w, step_h)* is a distance between box centers. For example, *step* equal 85 means that the distance between neighborhood prior boxes centers is 85. If both *step_h* and *step_w* are 0 then they are updated with value of *step*. If after that they are still 0 then they are calculated as input image width(height) divided with first input width(height). 
+  * **Range of values**: floating point positive number
+  * **Type**: float
+  * **Default value**: 0.0
+  * **Required**: *no*
+
+* *offset*
+
+  * **Description**: *offset* is a shift of box respectively to top left corner. For example, *offset* equal 85 means that the shift of neighborhood prior boxes centers is 85.
+  * **Range of values**: floating point positive number
+  * **Type**: float
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *variance*
+
+  * **Description**: *variance* denotes a variance of adjusting bounding boxes.
+  * **Range of values**: floating point positive numbers
+  * **Type**: float[]
+  * **Default value**: []
+  * **Required**: *no*
+
+* *img_h (img_w)*
+
+  * **Description**: *img_h (img_w)* specifies height (width) of input image. These attributes are taken from the second input `image_size` height(width) unless provided explicitly as the value for this attributes.
+  * **Range of values**: floating point positive number
+  * **Type**: float
+  * **Default value**: 0
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: `output_size` - 1D tensor with two integer elements `[height, width]`. Specifies the spatial size of generated grid with boxes. Required.
+
+*   **2**: `image_size` - 1D tensor with two integer elements `[image_height, image_width]` that specifies shape of the image for which boxes are generated. Optional.
+
+**Outputs**:
+
+*   **1**: 2D tensor of shape `[2, 4 * height * width * priors_per_point]` with box coordinates. The `priors_per_point` is the number of boxes generated per each grid element. The number depends on layer attribute values.
+
+**Detailed description**
+
+*PriorBoxClustered* computes coordinates of prior boxes by following:
+1.  Calculates the *center_x* and *center_y* of prior box:
+    \f[
+    W \equiv Width \quad Of \quad Image
+    \f]
+    \f[
+    H \equiv Height \quad Of \quad Image
+    \f]
+    \f[
+    center_x=(w+offset)*step
+    \f]
+    \f[
+    center_y=(h+offset)*step
+    \f]
+    \f[
+    w \subset \left( 0, W \right )
+    \f]
+    \f[
+    h \subset \left( 0, H \right )
+    \f]
+2.  For each \f$s \subset \left( 0, W \right )\f$ calculates the prior boxes coordinates:
+    \f[
+    xmin = \frac{center_x - \frac{width_s}{2}}{W}
+    \f]
+    \f[
+    ymin = \frac{center_y - \frac{height_s}{2}}{H}
+    \f]
+    \f[
+    xmax = \frac{center_x - \frac{width_s}{2}}{W}
+    \f]
+    \f[
+    ymax = \frac{center_y - \frac{height_s}{2}}{H}
+    \f]
+If *clip* is defined, the coordinates of prior boxes are recalculated with the formula:
+\f$coordinate = \min(\max(coordinate,0), 1)\f$
+
+**Example**
+
+```xml
+<layer type="PriorBoxClustered" ... >
+    <data clip="0" flip="1" height="44.0,10.0,30.0,19.0,94.0,32.0,61.0,53.0,17.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2" width="86.0,13.0,57.0,39.0,68.0,34.0,142.0,50.0,23.0"/>
+    <input>
+        <port id="0">
+            <dim>2</dim>        <!-- [10, 19] -->
+        </port>
+        <port id="1">
+            <dim>2</dim>        <!-- [180, 320] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>2</dim>
+            <dim>6840</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/detection/PriorBox_1.md

@@ -0,0 +1,179 @@
+## PriorBox<a name="PriorBox"></a>
+
+**Versioned name**: *PriorBox-1*
+
+**Category**: Object detection
+
+**Short description**: *PriorBox* operation generates prior boxes of specified sizes and aspect ratios across all dimensions.
+
+**Attributes**:
+
+* *min_size (max_size)*
+
+  * **Description**: *min_size (max_size)* is the minimum (maximum) box size (in pixels). For example, *min_size (max_size)* equal 15 means that the minimum (maximum) box size is 15.
+  * **Range of values**: positive floating point numbers
+  * **Type**: float[]
+  * **Default value**: []
+  * **Required**: *no*
+
+* *aspect_ratio*
+
+  * **Description**: *aspect_ratio* is a variance of aspect ratios. Duplicate values are ignored. For example, *aspect_ratio* equal "2.0,3.0" means that for the first box aspect_ratio is equal to 2.0 and for the second box is 3.0.
+  * **Range of values**: set of positive integer numbers
+  * **Type**: float[]
+  * **Default value**: []
+  * **Required**: *no*
+
+* *flip*
+
+  * **Description**: *flip* is a flag that denotes that each *aspect_ratio* is duplicated and flipped. For example, *flip* equals 1 and *aspect_ratio* equals to "4.0,2.0" mean that aspect_ratio is equal to "4.0,2.0,0.25,0.5".
+  * **Range of values**:
+    * False - each *aspect_ratio* is flipped
+    * True  - each *aspect_ratio* is not flipped
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+* *clip*
+
+  * **Description**: *clip* is a flag that denotes if each value in the output tensor should be clipped to [0,1] interval.
+  * **Range of values**:
+    * False - clipping is not performed
+    * True - each value in the output tensor is clipped to [0,1] interval.
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+* *step*
+
+  * **Description**: *step* is a distance between box centers. For example, *step* equal 85 means that the distance between neighborhood prior boxes centers is 85.
+  * **Range of values**: floating point non-negative number
+  * **Type**: float
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *offset*
+
+  * **Description**: *offset* is a shift of box respectively to top left corner. For example, *offset* equal 85 means that the shift of neighborhood prior boxes centers is 85.
+  * **Range of values**: floating point non-negative number
+  * **Type**: float
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *variance*
+
+  * **Description**: *variance* denotes a variance of adjusting bounding boxes. The attribute could contain 0, 1 or 4 elements.
+  * **Range of values**: floating point positive numbers
+  * **Type**: float[]
+  * **Default value**: []
+  * **Required**: *no*
+
+* *scale_all_sizes*
+
+  * **Description**: *scale_all_sizes* is a flag that denotes type of inference. For example, *scale_all_sizes* equals 0 means that the PriorBox layer is inferred in MXNet-like manner. In particular, *max_size* attribute is ignored.
+  * **Range of values**:
+    * False - *max_size* is ignored
+    * True  - *max_size* is used
+  * **Type**: boolean
+  * **Default value**: True
+  * **Required**: *no*
+
+* *fixed_ratio*
+
+    * **Description**: *fixed_ratio* is an aspect ratio of a box. For example, *fixed_ratio* equal to 2.000000 means that the aspect ratio for the first box aspect ratio is 2.
+    * **Range of values**: a list of positive floating-point numbers
+    * **Type**: `float[]`
+    * **Default value**: None
+    * **Required**: *no*
+
+* *fixed_size*
+
+    * **Description**: *fixed_size* is an initial box size (in pixels). For example, *fixed_size* equal to 15 means that the initial box size is 15.
+    * **Range of values**: a list of positive floating-point numbers
+    * **Type**: `float[]`
+    * **Default value**: None
+    * **Required**: *no*
+
+* *density*
+
+    * **Description**: *density* is the square root of the number of boxes of each type. For example, *density* equal to 2 means that the first box generates four boxes of the same size and with the same shifted centers.
+    * **Range of values**: a list of positive floating-point numbers
+    * **Type**: `float[]`
+    * **Default value**: None
+    * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: `output_size` - 1D tensor with two integer elements `[height, width]`. Specifies the spatial size of generated grid with boxes. Required.
+
+*   **2**: `image_size` - 1D tensor with two integer elements `[image_height, image_width]` that specifies shape of the image for which boxes are generated. Required.
+
+**Outputs**:
+
+*   **1**: 2D tensor of shape `[2, 4 * height * width * priors_per_point]` with box coordinates. The `priors_per_point` is the number of boxes generated per each grid element. The number depends on layer attribute values.
+
+**Detailed description**:
+
+*PriorBox* computes coordinates of prior boxes by following:
+1.  First calculates *center_x* and *center_y* of prior box:
+    \f[
+    W \equiv Width \quad Of \quad Image
+    \f]
+    \f[
+    H \equiv Height \quad Of \quad Image
+    \f]
+    *   If step equals 0:
+        \f[
+        center_x=(w+0.5)
+        \f]
+        \f[
+        center_y=(h+0.5)
+        \f]
+    *   else:
+        \f[
+        center_x=(w+offset)*step
+        \f]
+        \f[
+        center_y=(h+offset)*step
+        \f]
+        \f[
+        w \subset \left( 0, W \right )
+        \f]
+        \f[
+        h \subset \left( 0, H \right )
+        \f]
+2.  Then, for each \f$ s \subset \left( 0, min_sizes \right ) \f$ calculates coordinates of prior boxes:
+    \f[
+    xmin = \frac{\frac{center_x - s}{2}}{W}
+    \f]
+    \f[
+    ymin = \frac{\frac{center_y - s}{2}}{H}
+    \f]
+    \f[
+    xmax = \frac{\frac{center_x + s}{2}}{W}
+    \f]
+    \f[
+    ymin = \frac{\frac{center_y + s}{2}}{H}
+    \f]
+
+**Example**
+
+```xml
+<layer type="PriorBox" ...>
+    <data aspect_ratio="2.0" clip="0" density="" fixed_ratio="" fixed_size="" flip="1" max_size="38.46" min_size="16.0" offset="0.5" step="16.0" variance="0.1,0.1,0.2,0.2"/>
+    <input>
+        <port id="0">
+            <dim>2</dim>        <!-- values: [24, 42] -->
+        </port>
+        <port id="1">
+            <dim>2</dim>        <!-- values: [384, 672] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>2</dim>
+            <dim>16128</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/detection/Proposal_1.md

@@ -0,0 +1,160 @@
+## Proposal <a name="Proposal"></a>
+
+**Versioned name**: *Proposal-1*
+
+**Category**: *Object detection*
+
+**Short description**: *Proposal* operation filters bounding boxes and outputs only those with the highest prediction confidence.
+
+**Detailed description**
+
+*Proposal* has three inputs: a tensor with probabilities whether particular bounding box corresponds to background and foreground, a tensor with logits for each of the bounding boxes, a tensor with input image size in the [`image_height`, `image_width`, `scale_height_and_width`] or [`image_height`, `image_width`, `scale_height`, `scale_width`] format. The produced tensor has two dimensions `[batch_size * post_nms_topn, 5]`.
+*Proposal* layer does the following with the input tensor:
+1.  Generates initial anchor boxes. Left top corner of all boxes is at (0, 0). Width and height of boxes are calculated from *base_size* with *scale* and *ratio* attributes.
+2.  For each point in the first input tensor:
+    *   pins anchor boxes to the image according to the second input tensor that contains four deltas for each box: for *x* and *y* of center, for *width* and for *height*
+    *   finds out score in the first input tensor
+3.  Filters out boxes with size less than *min_size*
+4.  Sorts all proposals (*box*, *score*) by score from highest to lowest
+5.  Takes top *pre_nms_topn* proposals
+6.  Calculates intersections for boxes and filter out all boxes with \f$intersection/union > nms\_thresh\f$
+7.  Takes top *post_nms_topn* proposals
+8.  Returns top proposals
+
+
+* *base_size*
+
+  * **Description**: *base_size* is the size of the anchor to which *scale* and *ratio* attributes are applied.
+  * **Range of values**: a positive integer number
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pre_nms_topn*
+
+  * **Description**: *pre_nms_topn* is the number of bounding boxes before the NMS operation. For example, *pre_nms_topn* equal to 15 means that the minimum box size is 15.
+  * **Range of values**: a positive integer number
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *post_nms_topn*
+
+  * **Description**: *post_nms_topn* is the number of bounding boxes after the NMS operation. For example, *post_nms_topn* equal to 15 means that the maximum box size is 15.
+  * **Range of values**: a positive integer number
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *nms_thresh*
+
+  * **Description**: *nms_thresh* is the minimum value of the proposal to be taken into consideration. For example, *nms_thresh* equal to 0.5 means that all boxes with prediction probability less than 0.5 are filtered out.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *feat_stride*
+
+  * **Description**: *feat_stride* is the step size to slide over boxes (in pixels). For example, *feat_stride* equal to 16 means that all boxes are analyzed with the slide 16.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *min_size*
+
+  * **Description**: *min_size* is the minimum size of box to be taken into consideration. For example, *min_size* equal 35 means that all boxes with box size less than 35 are filtered out.
+  * **Range of values**: a positive integer number
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *ratio*
+
+  * **Description**: *ratio* is the ratios for anchor generation.
+  * **Range of values**: a list of floating-point numbers
+  * **Type**: `float[]`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *scale*
+
+  * **Description**: *scale* is the scales for anchor generation.
+  * **Range of values**: a list of floating-point numbers
+  * **Type**: `float[]`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *clip_before_nms*
+
+  * **Description**: *clip_before_nms* flag that specifies whether to perform clip bounding boxes before non-maximum suppression or not.
+  * **Range of values**: True or False
+  * **Type**: `boolean`
+  * **Default value**: True
+  * **Required**: *no*
+
+* *clip_after_nms*
+
+  * **Description**: *clip_after_nms* is a flag that specifies whether to perform clip bounding boxes after non-maximum suppression or not.
+  * **Range of values**: True or False
+  * **Type**: `boolean`
+  * **Default value**: False
+  * **Required**: *no*
+
+* *normalize*
+
+  * **Description**: *normalize* is a flag that specifies whether to perform normalization of output boxes to *[0,1]* interval or not.
+  * **Range of values**: True or False
+  * **Type**: `boolean`
+  * **Default value**: False
+  * **Required**: *no*
+
+* *box_size_scale*
+
+  * **Description**: *box_size_scale* specifies the scale factor applied to logits of box sizes before decoding.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: 1.0
+  * **Required**: *no*
+
+* *box_coordinate_scale*
+
+  * **Description**: *box_coordinate_scale* specifies the scale factor applied to logits of box coordinates before decoding.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: 1.0
+  * **Required**: *no*
+
+* *framework*
+
+  * **Description**: *framework* specifies how the box coordinates are calculated.
+  * **Range of values**:
+    * "" (empty string) - calculate box coordinates like in Caffe*
+    * *tensorflow* - calculate box coordinates like in the TensorFlow* Object Detection API models
+  * **Type**: string
+  * **Default value**: "" (empty string)
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: 4D input floating point tensor with class prediction scores. Required.
+
+*   **2**: 4D input floating point tensor with box logits. Required.
+
+*   **3**: 1D input floating tensor 3 or 4 elements:  [`image_height`, `image_width`, `scale_height_and_width`] or [`image_height`, `image_width`, `scale_height`, `scale_width`]. Required.
+
+**Outputs**:
+
+*   **1**: Floating point tensor of shape `[batch_size * post_nms_topn, 5]`.
+
+**Example**
+
+```xml
+<layer ... type="Proposal" ... >
+    <data base_size="16" feat_stride="16" min_size="16" nms_thresh="0.6" post_nms_topn="200" pre_nms_topn="6000"
+     ratio="2.67" scale="4.0,6.0,9.0,16.0,24.0,32.0"/>
+    <input> ... </input>
+    <output> ... </output>
+</layer>
+```

docs/ops/detection/ROIAlign_3.md

@@ -0,0 +1,112 @@
+## ROIAlign <a name="ROIAlign"></a>
+
+**Versioned name**: *ROIAlign-3*
+
+**Category**: Object detection
+
+**Short description**: *ROIAlign* is a *pooling layer* used over feature maps of non-uniform input sizes and outputs a feature map of a fixed size.
+
+**Detailed description**: [Reference](https://arxiv.org/abs/1703.06870).
+
+*ROIAlign* performs the following for each Region of Interest (ROI) for each input feature map:
+1. Multiply box coordinates with *spatial_scale* to produce box coordinates relative to the input feature map size.
+2. Divide the box into bins according to the *sampling_ratio* attribute.
+3. Apply bilinear interpolation with 4 points in each bin and apply maximum or average pooling based on *mode* attribute to produce output feature map element.
+
+**Attributes**
+
+* *pooled_h*
+
+  * **Description**: *pooled_h* is the height of the ROI output feature map.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pooled_w*
+
+  * **Description**: *pooled_w* is the width of the ROI output feature map.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *sampling_ratio*
+
+  * **Description**: *sampling_ratio* is the number of bins over height and width to use to calculate each output feature map element. If the value
+  is equal to 0 then use adaptive number of elements over height and width: `ceil(roi_height / pooled_h)` and `ceil(roi_width / pooled_w)` respectively.
+  * **Range of values**: a non-negative integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *spatial_scale*
+
+  * **Description**: *spatial_scale* is a multiplicative spatial scale factor to translate ROI coordinates from their input spatial scale to the scale used when pooling.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *mode*
+
+  * **Description**: *mode* specifies a method to perform pooling to produce output feature map elements.
+  * **Range of values**: 
+    * *max* - maximum pooling
+    * *avg* - average pooling 
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: 4D input tensor of shape `[N, C, H, W]` with feature maps of type *T*. Required.
+
+*   **2**: 2D input tensor of shape `[NUM_ROIS, 4]` describing box consisting of 4 element tuples: `[x_1, y_1, x_2, y_2]` in relative coordinates of type *T*.
+The box height and width are calculated the following way: `roi_width = max(spatial_scale * (x_2 - x_1), 1.0)`, 
+`roi_height = max(spatial_scale * (y_2 - y_1), 1.0)`, so the malformed boxes are expressed as a box of size `1 x 1`. Required.
+
+*   **3**: 1D input tensor of shape `[NUM_ROIS]` with batch indices of type *IND_T*. Required.
+
+**Outputs**:
+
+*   **1**: 4D output tensor of shape `[NUM_ROIS, C, pooled_h, pooled_w]` with feature maps of type *T*.
+
+**Types**
+
+* *T*: any supported floating point type.
+
+* *IND_T*: any supported integer type.
+
+
+**Example**
+
+```xml
+<layer ... type="ROIAlign" ... >
+    <data pooled_h="6" pooled_w="6" spatial_scale="16.0" sampling_ratio="2" mode="avg"/>
+    <input>
+        <port id="0">
+            <dim>7</dim>
+            <dim>256</dim>
+            <dim>200</dim>
+            <dim>200</dim>
+        </port>
+        <port id="1">
+            <dim>1000</dim>
+            <dim>4</dim>
+        </port>
+        <port id="2">
+            <dim>1000</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3" precision="FP32">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>6</dim>
+            <dim>6</dim>
+        </port>
+    </output>    
+</layer>
+```
+

docs/ops/detection/ROIPooling_1.md

@@ -0,0 +1,63 @@
+## ROIPooling <a name="ROIPooling"></a>
+
+**Versioned name**: *ROIPooling-1*
+
+**Category**: Object detection
+
+**Short description**: *ROIPooling* is a *pooling layer* used over feature maps of non-uniform input sizes and outputs a feature map of a fixed size.
+
+**Detailed description**: [deepsense.io reference](https://blog.deepsense.ai/region-of-interest-pooling-explained/)
+
+**Attributes**
+
+* *pooled_h*
+
+  * **Description**: *pooled_h* is the height of the ROI output feature map. For example, *pooled_h* equal to 6 means that the height of the output of *ROIPooling* is 6.
+  * **Range of values**: a non-negative integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *pooled_w*
+
+  * **Description**: *pooled_w* is the width of the ROI output feature map. For example, *pooled_w* equal to 6 means that the width of the output of *ROIPooling* is 6.
+  * **Range of values**: a non-negative integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *spatial_scale*
+
+  * **Description**: *spatial_scale* is the ratio of the input feature map over the input image size.
+  * **Range of values**: a positive floating-point number
+  * **Type**: `float`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *method*
+
+  * **Description**: *method* specifies a method to perform pooling. If the method is *bilinear*, the input box coordinates are normalized to the `[0, 1]` interval.
+  * **Range of values**: *max* or *bilinear*
+  * **Type**: string
+  * **Default value**: *max*
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: 4D input tensor of shape `[1, C, H, W]` with feature maps. Required.
+
+*   **2**: 2D input tensor of shape `[NUM_ROIS, 5]` describing box consisting of 5 element tuples: `[batch_id, x_1, y_1, x_2, y_2]`. Required.
+
+**Outputs**:
+
+*   **1**: 4D output tensor of shape `[NUM_ROIS, C, pooled_h, pooled_w]` with feature maps. Required.
+
+**Example**
+
+```xml
+<layer ... type="ROIPooling" ... >
+        <data pooled_h="6" pooled_w="6" spatial_scale="0.062500"/>
+        <input> ... </input>
+        <output> ... </output>
+    </layer>
+```

docs/ops/detection/RegionYolo_1.md

@@ -0,0 +1,133 @@
+## RegionYolo <a name="RegionYolo"></a>
+
+**Versioned name**: *RegionYolo-1*
+
+**Category**: *Object detection*
+
+**Short description**: *RegionYolo* computes the coordinates of regions with probability for each class.
+
+**Detailed description**: This operation is directly mapped to the original YOLO layer. [Reference](https://arxiv.org/pdf/1612.08242.pdf)
+
+**Attributes**:
+
+* *anchors*
+
+  * **Description**: *anchors* codes a flattened list of pairs `[width, height]` that codes prior box sizes. This attribute is not used in output computation, but it is required for post-processing to restore real box coordinates.
+  * **Range of values**: list of any length of positive floating point number
+  * **Type**: `float[]`
+  * **Default value**: None
+  * **Required**: *no*
+
+* *axis*
+
+  * **Description**: starting axis index in the input tensor `data` shape that will be flattened in the output; the end of flattened range is defined by `end_axis` attribute.
+  * **Range of values**: `-rank(data) .. rank(data)-1`
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *coords*
+
+  * **Description**: *coords* is the number of coordinates for each region.
+  * **Range of values**: an integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *classes*
+
+  * **Description**: *classes* is the number of classes for each region.
+  * **Range of values**: an integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *end_axis*
+
+  * **Description**: ending axis index in the input tensor `data` shape that will be flattened in the output; the beginning of the flattened range is defined by `axis` attribute.
+  * **Range of values**: `-rank(data)..rank(data)-1`
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *num*
+
+  * **Description**: *num* is the number of regions.
+  * **Range of values**: an integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *do_softmax*
+
+  * **Description**: *do_softmax* is a flag that specifies the inference method and affects how the number of regions is determined. It also affects output shape. If it is 0, then output shape is 4D, and 2D otherwise.
+  * **Range of values**:
+    * *False* - do not perform softmax
+    * *True* - perform softmax
+  * **Type**: `boolean`
+  * **Default value**: True
+  * **Required**: *no*
+
+* *mask*
+
+  * **Description**: *mask* specifies the number of regions. Use this attribute instead of *num* when *do_softmax* is equal to 0.
+  * **Range of values**: a list of integers
+  * **Type**: `int[]`
+  * **Default value**: `[]`
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: `data` - 4D input tensor with floating point elements and shape `[N, C, H, W]`. Required.
+
+**Outputs**:
+
+*   **1**: output tensor of rank 4 or less that codes detected regions. Refer to the original YOLO paper to decode the output as boxes. `anchors` should be used to decode real box coordinates. If `do_softmax` is set to 0, then the output shape is `[N, (classes + coords + 1)*len(mask), H, W]`. If `do_softmax` is set to 1, then output shape is partially flattened and defined in the following way:
+
+    flat_dim = data.shape[axis] * data.shape[axis+1] * ... * data.shape[end_axis]
+    output.shape = [data.shape[0], ..., data.shape[axis-1], flat_dim, data.shape[end_axis + 1], ...]
+
+**Example**
+
+```xml
+<!-- YOLO V3 example -->
+<layer type="RegionYolo" ... >
+    <data anchors="10,14,23,27,37,58,81,82,135,169,344,319" axis="1" classes="80" coords="4" do_softmax="0" end_axis="3" mask="0,1,2" num="6"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>255</dim>
+            <dim>26</dim>
+            <dim>26</dim>
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>1</dim>
+            <dim>255</dim>
+            <dim>26</dim>
+            <dim>26</dim>
+        </port>
+    </output>
+</layer>
+
+<!-- YOLO V2 Example -->
+<layer type="RegionYolo" ... >
+    <data anchors="1.08,1.19,3.42,4.41,6.63,11.38,9.42,5.11,16.62,10.52" axis="1" classes="20" coords="4" do_softmax="1" end_axis="3" num="5"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>125</dim>
+            <dim>13</dim>
+            <dim>13</dim>
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>1</dim>
+            <dim>21125</dim>
+        </port>
+    </output>
+</layer>
+
+```

docs/ops/detection/ReorgYolo_1.md

@@ -0,0 +1,53 @@
+## ReorgYolo Layer <a name="ReorgYolo"></a>
+
+**Versioned name**: *ReorgYolo-1*
+
+**Category**: *Object detection*
+
+**Short description**: *ReorgYolo* reorganizes input tensor taking into account strides.
+
+**Detailed description**:
+
+[Reference](https://arxiv.org/pdf/1612.08242.pdf)
+
+**Attributes**
+
+* *stride*
+
+  * **Description**: *stride* is the distance between cut throws in output blobs.
+  * **Range of values**: positive integer
+  * **Type**: `int`
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: 4D input tensor of any type and shape `[N, C, H, W]`. `H` and `W` should be divisible by `stride`. Required.
+
+**Outputs**:
+
+*   **1**: 4D output tensor of the same type as input tensor and shape `[N, C*stride*stride, H/stride, W/stride]`. Required.
+
+**Example**
+
+```xml
+<layer id="89" name="ExtractImagePatches" type="ReorgYolo">
+    <data stride="2"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>64</dim>
+            <dim>26</dim>
+            <dim>26</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1" precision="f32">
+            <dim>1</dim>
+            <dim>256</dim>
+            <dim>13</dim>
+            <dim>13</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/generation/Range_1.md

@@ -0,0 +1,76 @@
+## Range<a name="Range"></a>
+
+**Versioned name**: *Range-1*
+
+**Category**: Generation
+
+**Short description**: *Range* operation generates a sequence of numbers according input values [start, stop) with a step.
+
+**Attributes**:
+
+No attributes available.
+
+**Inputs**:
+
+* **1**: "start" - If a value is not given then *start* = 0. A scalar of type T. **Required.**
+* **2**: "stop" - A scalar of type T. **Required.**
+* **3**: "step" - If a value is not given then *step* = 1. A scalar of type T. **Required.**
+
+**Outputs**:
+
+* **1**: A tensor with type matching 2nd tensor.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Detailed description**:
+
+*Range* operation generates a sequence of numbers starting from the value in the first input (start) up to but not including the value in the second input (stop) with a step equal to the value in the third input, according to the following formula:
+
+\f[
+[start, start + step, start + 2 * step, ..., start + K * step], where K is the maximal integer value that satisfies condition start + K*step < stop, then step is positive value and start + K*step > stop, then step is negative value.
+\f]
+
+**Examples**
+
+*Example 1: positive step*
+
+```xml
+<layer ... type="Range">
+    <input>
+        <port id="0">  <!-- start value: 2 -->
+        </port>
+        <port id="1">  <!-- stop value: 23 -->
+        </port>
+        <port id="2">  <!-- step value: 3 -->
+        </port>
+    </input>
+    <output>
+        <port id="3">
+            <dim>7</dim> <!-- [ 2,  5,  8, 11, 14, 17, 20] -->
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: negative step*
+
+```xml
+<layer ... type="Range">
+    <input>
+        <port id="0">  <!-- start value: 23 -->
+        </port>
+        <port id="1">  <!-- stop value: 2 -->
+        </port>
+        <port id="2">  <!-- step value: -3 -->
+        </port>
+    </input>
+    <output>
+        <port id="3">
+            <dim>7</dim> <!-- [23, 20, 17, 14, 11, 8, 5] -->
+        </port>
+    </output>
+</layer>
+```
+

docs/ops/image/Interpolate_1.md

@@ -0,0 +1,98 @@
+## Interpolate <a name="Interpolate"></a>
+
+**Versioned name**: *Interpolate-1*
+
+**Category**: Image processing
+
+**Short description**: *Interpolate* layer performs interpolation of independent slices in input tensor by specified dimensions and attributes.
+
+**Attributes**
+
+* *axes*
+
+  * **Description**: `axes` specify spatial dimension indices where interpolation is applied. Other dimensions are treated as batch dimensions. The order of elements in `axes` attribute matters and mapped directly to elements with the same indices in the 2nd input `target_spatial_shape`.
+  * **Range of values**: list of non-negative integer numbers
+  * **Type**: `int[]`
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *mode*
+
+  * **Description**: specifies type of interpolation
+  * **Range of values**: one of `nearest`, `linear`, `cubic`, `area`
+  * **Type**: string
+  * **Default value**: none
+  * **Required**: *yes*
+
+* *align_corners*
+
+  * **Description**: *align_corners* is a flag that specifies whether to align corners or not. 1 means the alignment is applied, 0 means the alignment isn't applied.
+  * **Range of values**: True or False
+  * **Type**: `boolean`
+  * **Default value**: True
+  * **Required**: *no*
+
+* *antialias*
+
+  * **Description**: *antialias* is a flag that specifies whether to perform anti-aliasing.
+  * **Range of values**:
+    * False - do not perform anti-aliasing
+    * True - perform anti-aliasing
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+* *pads_begin*
+
+  * **Description**: *pads_beg* specify the number of pixels to add to the beginning of the image being interpolated.
+This is a scalar that specifies padding for each spatial dimension.
+  * **Range of values**: list of non-negative integer numbers
+  * **Type**: `int`
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *pads_end*
+
+  * **Description**: *pads_end* specify the number of pixels to add to the beginning of the image being interpolated.
+This is a scalar that specifies padding for each spatial dimension.
+  * **Range of values**: list of non-negative integer numbers
+  * **Type**: `int`
+  * **Default value**: 0
+  * **Required**: *no*
+
+**Inputs**
+
+*   **1**: `data` - Input tensor with data for interpolation. Type of elements is any supported floating point type. Required.
+
+*   **2**: `target_spatial_shape` - 1D tensor describing output shape for spatial axes. Number of elements matches the number of indices in *axes* attribute, the order matches as well. Required.
+
+**Outputs**
+
+*   **1**: Resulting interpolated tensor with elements of the same type as input `data` tensor. The shape of the output matches input `data` shape except spatial dimensions mentioned in `axes` attribute. For other dimensions shape matches sizes from `target_spaticl_shape` in order specified in `axes`.
+
+**Example**
+
+```xml
+<layer ... type="Interpolate" ...>
+    <data axes="2,3" align_corners="0" pads_begin="0" pads_end="0" mode="linear"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>2</dim>
+            <dim>48</dim>
+            <dim>80</dim>
+        </port>
+        <port id="1">
+            <dim>2</dim>  <!--The values in this input are [50, 60] -->
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>1</dim>
+            <dim>2</dim>
+            <dim>50</dim>
+            <dim>60</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/infrastructure/Assign_3.md

@@ -0,0 +1,48 @@
+## Assign <a name="Assign"></a>
+
+**Versioned name**: *Assign-3*
+
+**Category**: *Infrastructure*
+
+**Short description**: *Assign* sets an input value to the `variable_id` variable.
+
+**Detailed description**:
+
+*Assign* operation sets an input value to the `variable_id` variable and 
+optionally returns it as an output. This value will be returned by *ReadValue* operation on next infer if variable was not reset.
+The operation checks that the type and shape of the input are the same as 
+declared in `variable_id` and returns an error otherwise.
+
+**Attributes**: 
+
+* *variable_id*
+
+  * **Description**: identificator of the variable to be updated
+  * **Range of values**: any non-empty string
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *Yes*
+
+**Inputs**
+
+*   **1**: `new_value` - input tensor of any supported type. **Required**.
+
+**Outputs**
+
+*   **1**: tensor with the same shape and type as `new_value`
+
+**Example**
+
+```xml
+<layer ... type="Assign" ...>
+    <data variable_id="lstm_state_1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </input>
+</layer>
+```

docs/ops/infrastructure/Constant_1.md

@@ -0,0 +1,55 @@
+## Constant <a name="Constant"></a>
+
+**Versioned name**: *Constant-1*
+
+**Category**: *Infrastructure*
+
+**Short description**: *Constant* operation produces a tensor with content read from binary file by offset and size.
+
+**Attributes**
+
+* *offset*
+
+  * **Description**: specifies position in binary file with weights where the content of the constant begins; value in bytes
+  * **Range of values**: non-negative integer value
+  * **Type**: int
+  * **Default value**: none
+  * **Required**: *yes*
+
+* *size*
+
+  * **Description**: size of constant content in binary files; value in bytes
+  * **Range of values**: positive integer bigger than zero
+  * **Type**: int
+  * **Default value**: none
+  * **Required**: *yes*
+
+* *element_type*
+
+  * **Description**: the type of element of output tensor
+  * **Range of values**: u1, u8, u16, u32, u64, i8, i16, i32, i64, f16, f32, boolean, bf16
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *Yes*
+
+* *shape*
+
+  * **Description**: the shape of the output tensor
+  * **Range of values**: list of non-negative integers, empty list is allowed, which means 0D or scalar tensor
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *Yes*
+
+**Example**
+
+```xml
+<layer ... type="Constant">
+    <data offset="1000" size="256" element_type="f32" shape="8,8"/>
+    <output>
+        <port id="1">
+            <dim>8</dim>
+            <dim>8</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/infrastructure/Parameter_1.md

@@ -0,0 +1,41 @@
+## Parameter <a name="Parameter"></a>
+
+**Versioned name**: *Parameter-1*
+
+**Category**: *Infrastructure*
+
+**Short description**: *Parameter* layer specifies input to the model.
+
+**Attributes**:
+
+* *element_type*
+
+  * **Description**: the type of element of output tensor
+  * **Range of values**: u8, u16, u32, u64, i8, i16, i32, i64, f16, f32, boolean, bf16
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *Yes*
+
+* *shape*
+
+  * **Description**: the shape of the output tensor
+  * **Range of values**: list of non-negative integers, emty list is allowed that means 0D or scalar tensor
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *Yes*
+
+**Example**
+
+```xml
+<layer ... type="Parameter" ...>
+    <data>element_type="f32" shape="1,3,224,224"</data>
+    <output>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/infrastructure/ReadValue_3.md

@@ -0,0 +1,57 @@
+## ReadValue <a name="ReadValue"></a>
+
+**Versioned name**: *ReadValue-3*
+
+**Category**: *Infrastructure*
+
+**Short description**: *ReadValue* returns value of the `variable_id` variable.
+
+**Detailed description**: 
+
+*ReadValue* returns value from the corresponding `variable_id` variable if the variable was set already by *Assign* operation and was not reset. 
+The operation checks that the type and shape of the output are the same as
+declared in `variable_id` and returns an error otherwise. If the corresponding variable was not set or was reset,
+the operation returns the value from the 1 input, and initializes the `variable_id` shape and type
+with the shape and type from the 1 input.
+
+**Attributes**:
+
+* *variable_id*
+
+  * **Description**: identificator of the variable to be read
+  * **Range of values**: any non-empty string
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *Yes*
+
+**Inputs**
+
+*   **1**: `init_value` - input tensor with constant values of any supported type. **Required**.
+
+**Outputs**
+
+*   **1**: tensor with the same shape and type as `init_value`
+
+**Example**
+
+```xml
+<layer ... type="ReadValue" ...>
+    <data variable_id="lstm_state_1"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/infrastructure/Result_1.md

@@ -0,0 +1,34 @@
+## Result <a name="Result"></a>
+
+**Versioned name**: *Result-1*
+
+**Category**: *Infrastructure*
+
+**Short description**: *Result* layer specifies output of the model.
+
+**Attributes**: 
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required.**
+
+**Types**
+
+* *T*: arbitrary supported type.
+
+**Example**
+
+```xml
+<layer ... type="Result" ...>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>224</dim>
+            <dim>224</dim>
+        </port>
+    </input>
+</layer>
+```

docs/ops/infrastructure/TensorIterator_1.md

@@ -0,0 +1,374 @@
+## TensorIterator <a name="TensorIterator"></a>
+
+**Versioned name**: *TensorIterator-1*
+
+**Category**: Infrastructure
+
+**Short description**: *TensorIterator* layer performs recurrent execution of the network, which is described in the `body`, iterating through the data.
+
+**TensorIterator attributes**:
+
+* **Body**:
+
+    `body` is a network that will be recurrently executed. The network is described layer by layer as a typical IR network.
+
+    * **Body attributes**:
+
+            No attributes available.
+
+* **Port map**:
+
+    *port_map* is a set of rules to map input or output data tensors of `TensorIterator` layer onto `body` data tensors. The `port_map` entries can be` input` and `output`. Each entry describes a corresponding mapping rule.
+
+    * **Port map attributes**:
+
+        * *external_port_id*
+            * **Description**: *external_port_id* is a port ID of the `TensorIterator` layer.
+            * **Range of values**: indexes of the *TensorIterator* outputs
+            * **Type**: `int`
+            * **Default value**: None
+            * **Required**: *yes*
+
+        * *internal_layer_id*
+
+            * **Description**: *internal_layer_id* is a *Parameter* or *Result* layer ID inside the `body` network to map to.
+            * **Range of values**: IDs of the *Parameter* layers inside in the *TensorIterator* layer
+            * **Type**: `int`
+            * **Default value**: None
+            * **Required**: *yes*
+
+        * *axis*
+
+            * **Description**: *axis* is an axis to iterate through. It triggers the slicing of this tensor. Only if it is specified, the corresponding `input` or `output` is divided into pieces and start, end and stride attributes define how slicing is performed.
+            * **Range of values**: an integer
+            * **Type**: `int`
+            * **Default value**: None
+            * **Required**: *no*
+
+        * *start*
+
+            * **Description**: *start* is an index where the iteration starts from. Negative value means counting indexes from the end. Applies only when the attribute `axis` is specified.
+            * **Range of values**: an integer
+            * **Type**: `int`
+            * **Default value**: 0
+            * **Required**: *no*
+
+        * *end*
+
+            * **Description**: *end* is an index where iteration ends. Negative value means counting indexes from the end. Applies only when the attribute `axis` is specified.
+            * **Range of values**: an integer
+            * **Type**: `int`
+            * **Default value**: -1
+            * **Required**: *no*
+
+        * *stride*
+
+            * **Description**: *stride* is a step of iteration. Negative value means backward iteration. Applies only when the attribute `axis` is specified.
+            * **Range of values**: an integer
+            * **Type**: `int`
+            * **Default value**: 1
+            * **Required**: *no*
+
+
+* **Back edges**:
+
+    *back_edges* is a set of rules to transfer tensor values from `body` outputs at one iteration to `body` parameters at the next iteration. Back edge connects some *Result* layer in `body` to *Parameter* layer in the same `body`.
+
+    * **Back edge attributes**:
+
+        * *from-layer*
+
+            * **Description**: *from-layer* is a *Result* layer ID inside the `body` network.
+            * **Range of values**: IDs of the *Result* layers inside the *TensorIterator*
+            * **Type**: `int`
+            * **Default value**: None
+            * **Required**: *yes*
+
+        * *to-layer*
+
+            * **Description**: *to-layer* is a *Parameter* layer ID inside the `body` network to end mapping.
+            * **Range of values**: IDs of the *Parameter* layers inside the *TensorIterator*
+            * **Type**: `int`
+            * **Default value**: None
+            * **Required**: *yes*
+
+**Inputs**
+
+* **Multiple inputs**: Tensors of any type and shape supported type.
+
+**Outputs**
+
+* **Multiple outputs**: Results of execution of the `body`. Tensors of any type and shape.
+
+
+**Detailed description**
+
+Similar to other layers, TensorIterator has regular sections: `input` and `output`. It allows connecting TensorIterator to the rest of the IR.
+TensorIterator also has several special sections: `body`, `port_map`, `back_edges`. The principles of their work are described below.
+
+How `body` is iterated:
+
+*At the first iteration:*
+TensorIterator slices input tensors by a specified axis and iterates over all parts in a specified order. It process input tensors with arbitrary network specified as an IR network in the `body` section. IR is executed as no back-edges are present. Edges from `port map` are used to connect input ports of TensorIterator to `Parameters` in body.
+
+[`inputs`] - `Port map` edges -> [`Parameters:body:Results`]
+
+`Parameter` and `Result` layers are part of the `body`. `Parameters` are stable entry points in the `body`. The results of the execution of the `body` are presented as stable `Result` layers. Stable means that these nodes cannot be fused.
+
+*Next iterations:*
+Back edges define which data is copied back to `Parameters` layers from `Results` layers between IR iterations in TensorIterator `body`. That means they pass data from source layer back to target layer. Each layer that is a target for back-edge has also an incoming `port map` edge as an input. The values from back-edges are used instead of corresponding edges from `port map`. After each iteration of the network, all back edges are executed. 
+Iterations can be considered as statically unrolled sequence: all edges that flow between two neighbor iterations are back-edges. So in the unrolled loop, each back-edge is transformed to regular edge.
+
+... -> [`Parameters:body:Results`] - back-edges -> [`Parameters:body:Results`] - back-edges -> [`Parameters:body:Results`] - back-edges -> ...
+
+*Calculation of results:*
+
+If `output` entry in the `Port map` doesn't have partitioning (`axis, begin, end, strides`) attributes, then the final value of `output` of TensorIterator is the value of `Result` node from the last iteration. Otherwise the final value of `output` of TensorIterator is a concatenation of tensors in the `Result` node for all `body` iterations. Concatenation order is specified by `stride` attribute.
+
+The last iteration:
+
+[`Parameters:body:Results`] - `Port map` edges -> [`outputs`],  if partitioning attributes are not set.
+
+if there are partitioning attributes, then an output tensor is a concatenation of tensors from all body iterations. If `stride > 0`:
+```
+output = Concat(S[0], S[1], ..., S[N-1])
+```
+where `Si` is value of `Result` operation at i-th iteration in the tensor iterator body that corresponds to this output port. If `stride < 0`, then output is concatenated in a reverse order:
+```
+output = Concat(S[N-1], S[N-2], ..., S[0])
+```
+
+**Examples**
+
+*Example 1: a typical TensorIterator structure*
+```xml
+<layer type="TensorIterator" ... >
+    <input> ... </input>
+    <output> ... </output>
+    <port_map>
+        <input external_port_id="0" internal_layer_id="0" axis="1" start="-1" end="0" stride="-1"/>
+        <input external_port_id="1" internal_layer_id="1"/>
+        ...
+        <output external_port_id="3" internal_layer_id="2" axis="1" start="-1" end="0" stride="-1"/>
+        ...
+    </port_map>
+    <back_edges>
+        <edge from-layer="1" to-layer="1"/>
+        ...
+    </back_edges>
+    <body>
+        <layers> ... </layers>
+        <edges> ... </edges>
+    </body>
+</layer>
+```
+
+*Example 2: a full TensorIterator layer*
+
+```xml
+<layer type="TensorIterator" ...>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>25</dim>
+            <dim>512</dim>
+        </port>
+        <port id="1">
+            <dim>1</dim>
+            <dim>256</dim>
+        </port>
+        <port id="2">
+            <dim>1</dim>
+            <dim>256</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3" precision="FP32">
+            <dim>1</dim>
+            <dim>25</dim>
+            <dim>256</dim>
+        </port>
+    </output>
+    <port_map>
+        <input axis="1" external_port_id="0" internal_layer_id="0" start="0"/>
+        <input external_port_id="1" internal_layer_id="3"/>
+        <input external_port_id="2" internal_layer_id="4"/>
+        <output axis="1" external_port_id="3" internal_layer_id="12"/>
+    </port_map>
+    <back_edges>
+        <edge from-layer="8" to-layer="4"/>
+        <edge from-layer="9" to-layer="3"/>
+    </back_edges>
+    <body>
+        <layers>
+            <layer id="0" type="Parameter" ...>
+                <output>
+                    <port id="0" precision="FP32">
+                        <dim>1</dim>
+                        <dim>1</dim>
+                        <dim>512</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="1" type="Const" ...>
+                <data offset="0" size="16"/>
+                <output>
+                    <port id="1" precision="I64">
+                        <dim>2</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="2" type="Reshape" ...>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>1</dim>
+                        <dim>512</dim>
+                    </port>
+                    <port id="1">
+                        <dim>2</dim>
+                    </port>
+                </input>
+                <output>
+                    <port id="2" precision="FP32">
+                        <dim>1</dim>
+                        <dim>512</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="3" type="Parameter" ...>
+                <output>
+                    <port id="0" precision="FP32">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="4" type="Parameter" ...>
+                <output>
+                    <port id="0" precision="FP32">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="5" type="Const" ...>
+                <data offset="16" size="3145728"/>
+                <output>
+                    <port id="1" precision="FP32">
+                        <dim>1024</dim>
+                        <dim>768</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="6" type="Const" ...>
+                <data offset="3145744" size="4096"/>
+                <output>
+                    <port id="1" precision="FP32">
+                        <dim>1024</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="7" type="LSTMCell" ...>
+                <data hidden_size="256"/>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>512</dim>
+                    </port>
+                    <port id="1">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                    <port id="2">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                    <port id="3">
+                        <dim>1024</dim>
+                        <dim>768</dim>
+                    </port>
+                    <port id="4">
+                        <dim>1024</dim>
+                    </port>
+                </input>
+                <output>
+                    <port id="5" precision="FP32">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                    <port id="6" precision="FP32">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="8" type="Result" ...>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </input>
+            </layer>
+            <layer id="9" type="Result" ...>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </input>
+            </layer>
+            <layer id="10" type="Const" ...>
+                <data offset="3149840" size="24"/>
+                <output>
+                    <port id="1" precision="I64">
+                        <dim>3</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="11" type="Reshape" ...>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                    <port id="1">
+                        <dim>3</dim>
+                    </port>
+                </input>
+                <output>
+                    <port id="2" precision="FP32">
+                        <dim>1</dim>
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </output>
+            </layer>
+            <layer id="12" type="Result" ...>
+                <input>
+                    <port id="0">
+                        <dim>1</dim>
+                        <dim>1</dim>
+                        <dim>256</dim>
+                    </port>
+                </input>
+            </layer>
+        </layers>
+        <edges>
+            <edge from-layer="0" from-port="0" to-layer="2" to-port="0"/>
+            <edge from-layer="1" from-port="1" to-layer="2" to-port="1"/>
+            <edge from-layer="2" from-port="2" to-layer="7" to-port="0"/>
+            <edge from-layer="3" from-port="0" to-layer="7" to-port="1"/>
+            <edge from-layer="4" from-port="0" to-layer="7" to-port="2"/>
+            <edge from-layer="5" from-port="1" to-layer="7" to-port="3"/>
+            <edge from-layer="6" from-port="1" to-layer="7" to-port="4"/>
+            <edge from-layer="7" from-port="6" to-layer="8" to-port="0"/>
+            <edge from-layer="7" from-port="5" to-layer="9" to-port="0"/>
+            <edge from-layer="7" from-port="5" to-layer="11" to-port="0"/>
+            <edge from-layer="10" from-port="1" to-layer="11" to-port="1"/>
+            <edge from-layer="11" from-port="2" to-layer="12" to-port="0"/>
+        </edges>
+    </body>
+</layer>
+```

docs/ops/logical/LogicalAnd_1.md

@@ -0,0 +1,93 @@
+## LogicalAnd <a name="LogicalAnd"></a>
+
+**Versioned name**: *LogicalAnd-1*
+
+**Category**: Logical binary operation
+
+**Short description**: *LogicalAnd* performs element-wise logical AND operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required**.
+* **2**: A tensor of type T. **Required**.
+
+**Outputs**
+
+* **1**: The result of element-wise logical AND operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: boolean type.
+
+**Detailed description**
+Before performing logical operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *LogicalAnd* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} and b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="LogicalAnd">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="LogicalAnd">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/logical/LogicalNot_1.md

@@ -0,0 +1,50 @@
+## LogicalNot <a name="LogicalNot"></a>
+
+**Versioned name**: *LogicalNot-1*
+
+**Category**: Logical unary operation 
+
+**Short description**: *LogicalNot* performs element-wise logical negation operation with given tensor.
+
+**Attributes**:
+
+    No attributes available.
+
+**Inputs**
+
+* **1**: An tensor of type T. **Required.**
+
+**Outputs**
+
+* **1**: The result of element-wise logical negation operation. A tensor of type T.
+
+**Types**
+
+* *T*: boolean type.
+
+*LogicalNot* does the following with the input tensor *a*:
+
+\f[
+a_{i} = not(a_{i})
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="LogicalNot">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/logical/LogicalOr_1.md

@@ -0,0 +1,93 @@
+## LogicalOr <a name="LogicalOr"></a>
+
+**Versioned name**: *LogicalOr-1*
+
+**Category**: Logical binary operation
+
+**Short description**: *LogicalOr* performs element-wise logical OR operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required**.
+* **2**: A tensor of type T. **Required**.
+
+**Outputs**
+
+* **1**: The result of element-wise logical OR operation. A tensor of type boolean.
+
+**Types**
+
+* *T*: boolean type.
+
+**Detailed description**
+Before performing logical operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *LogicalOr* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} or b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="LogicalOr">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="LogicalOr">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/logical/LogicalXor_1.md

@@ -0,0 +1,93 @@
+## LogicalXor <a name="LogicalXor"></a>
+
+**Versioned name**: *LogicalXor-1*
+
+**Category**: Logical binary operation
+
+**Short description**: *LogicalXor* performs element-wise logical XOR operation with two given tensors applying multi-directional broadcast rules.
+
+**Attributes**:
+
+* *auto_broadcast*
+
+  * **Description**: specifies rules used for auto-broadcasting of input tensors.
+  * **Range of values**:
+    * *none* - no auto-broadcasting is allowed, all input shapes should match
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+**Inputs**
+
+* **1**: A tensor of type T. **Required**.
+* **2**: A tensor of type T. **Required**.
+
+**Outputs**
+
+* **1**: The result of element-wise logical XOR operation. A tensor of type T.
+
+**Types**
+
+* *T*: boolean type.
+
+**Detailed description**
+Before performing logical operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attributes is not `none`. Broadcasting is performed according to `auto_broadcast` value.
+
+After broadcasting *LogicalXor* does the following with the input tensors *a* and *b*:
+
+\f[
+o_{i} = a_{i} xor b_{i}
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="LogicalXor">
+    <input>
+        <port id="0">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+        <port id="1">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>256</dim>
+            <dim>56</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: broadcast*
+```xml
+<layer ... type="LogicalXor">
+    <input>
+        <port id="0">
+            <dim>8</dim>
+            <dim>1</dim>
+            <dim>6</dim>
+            <dim>1</dim>
+        </port>
+        <port id="1">
+            <dim>7</dim>
+            <dim>1</dim>
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>8</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+            <dim>5</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/matrix/MatMul_1.md

@@ -0,0 +1,168 @@
+## MatMul <a name="MatMul"></a>
+
+**Versioned name**: *MatMul-1*
+
+**Category**: Matrix multiplication
+
+**Short description**: Generalized matrix multiplication
+
+**Detailed description**
+
+*MatMul* operation takes two tensors and performs usual matrix-matrix multiplication, matrix-vector multiplication or vector-matrix multiplication depending on argument shapes. Input tensors can have any rank >= 1. Two right-most axes in each tensor are interpreted as matrix rows and columns dimensions while all left-most axes (if present) are interpreted as multi-dimensional batch: [BATCH_DIM_1, BATCH_DIM_2,..., BATCH_DIM_K, ROW_INDEX_DIM, COL_INDEX_DIM]. The operation supports usual broadcast semantics for batch dimensions. It enables multiplication of batch of pairs of matrices in a single shot.
+
+Before matrix multiplication, there is an implicit shape alignment for input arguments. It consists of the following steps:
+
+1. If rank of an input less than 2 it is unsqueezed to 2D tensor by adding axes with size 1 to the left of the shape. For example, if input has shape `[S]` it will be reshaped to `[1, S]`. It is applied for each input independently.
+
+2. Applied transpositions specified by optional `transpose_a` and `transpose_b` attributes.
+
+3. If ranks of input arguments are different after steps 1 and 2, each is unsqueezed from the left side of the shape by necessary number of axes to make both shapes of the same rank.
+
+3. Usual rules of the broadcasting are applied for batch dimensions.
+
+Two attributes, transpose_a and transpose_b specifies embedded transposition for two right-most dimension for the first and the second input tensors correspondingly. It implies swapping of ROW_INDEX_DIM and COL_INDEX_DIM in the corresponding input tensor. Batch dimensions are not affected by these attributes.
+
+**Attributes**
+
+* *transpose_a*
+
+  * **Description**: transposes dimensions ROW_INDEX_DIM and COL_INDEX_DIM of the 1st input; 0 means no transpose, 1 means transpose
+  * **Range of values**: False or True
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+* *transpose_b*
+
+  * **Description**: transposes dimensions ROW_INDEX_DIM and COL_INDEX_DIM of the 2nd input; 0 means no transpose, 1 means transpose
+  * **Range of values**: False or True
+  * **Type**: boolean
+  * **Default value**: False
+  * **Required**: *no*
+
+
+**Inputs**:
+
+*   **1**: Input batch of matrices A. Rank >= 1. Required.
+
+*   **2**: Input batch of matrices B. Rank >= 1. Required.
+
+
+**Example**
+
+*Vector-matric multiplication*
+
+```xml
+<layer ... type="MatMul">
+    <input>
+        <port id="0">
+            <dim>1024</dim>
+        </port>
+        <port id="1">
+            <dim>1024</dim>
+            <dim>1000</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>1000</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Matrix-matrix multiplication (like FullyConnected with batch size 1)*
+
+```xml
+<layer ... type="MatMul">
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>1024</dim>
+        </port>
+        <port id="1">
+            <dim>1024</dim>
+            <dim>1000</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>1000</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Matrix-vector multiplication with embedded transposition of the second matrix*
+
+```xml
+<layer ... type="MatMul">
+    <data transpose_b="true"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>1024</dim>
+        </port>
+        <port id="1">
+            <dim>1000</dim>
+            <dim>1024</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>1000</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Matrix-matrix multiplication (like FullyConnected with batch size 10)*
+
+```xml
+<layer ... type="MatMul">
+    <input>
+        <port id="0">
+            <dim>10</dim>
+            <dim>1024</dim>
+        </port>
+        <port id="1">
+            <dim>1024</dim>
+            <dim>1000</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>10</dim>
+            <dim>1000</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Multiplication of batch of 5 matrices by a one matrix with broadcasting*
+
+```xml
+<layer ... type="MatMul">
+    <input>
+        <port id="0">
+            <dim>5</dim>
+            <dim>10</dim>
+            <dim>1024</dim>
+        </port>
+        <port id="1">
+            <dim>1024</dim>
+            <dim>1000</dim>
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>5</dim>
+            <dim>10</dim>
+            <dim>1000</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/BatchToSpace_2.md

@@ -0,0 +1,78 @@
+## BatchToSpace <a name="BatchToSpace"></a>
+
+**Versioned name**: *BatchToSpace-2*
+
+**Category**: *Data movement*
+
+**Short description**: The *BatchToSpace* operation reshapes the "batch" dimension 0 into N - 1 dimensions of shape `block_shape` + [batch] and interleaves these blocks back into the grid defined by the spatial dimensions `[1, ..., N - 1]` to obtain a result with the same rank as `data` input. The spatial dimensions of this intermediate result are then optionally cropped according to `crops_begin` and `crops_end` to produce the output. This is the reverse of the *SpaceToBatch* operation.
+
+**Detailed description**:
+
+The *BatchToSpace* operation is similar to the TensorFlow* operation [BatchToSpaceND](https://www.tensorflow.org/api_docs/python/tf/batch_to_space_nd)
+
+The operation is equivalent to the following transformation of the input tensors `data` with shape `[batch, D_1, D_2 ... D_{N-1}]` and `block_shape`, `crops_begin`, `crops_end` of shape `[N]` to *Y* output tensor.
+
+    note: B_0 is expected to be 1.
+    x' = reshape(`data`, [B_1, ..., B_{N - 1}, batch / (B_1 * ... B_{N - 1}), D_1, D_2, ..., D_{N - 1}]), where B_i = block_shape[i]
+
+    x'' = transpose(x', [N, N + 1, 0, N + 2, 1, ..., N + N - 1, N - 1])
+
+    x''' = reshape(x'', [batch / (B_1 * ... * B_{N - 1}), D_1 * B_1, D_2 * B_2, ... , D_{N - 1} * B_{N - 1}])
+
+   Crop the start and end of dimensions according to `crops_begin`, `crops_end` to produce the output of shape:
+   note: `crops_begin[0], crops_end[0]` are expected to be 0.
+    `y = [batch / (B_1 * ... * B_{N - 1}), crop(D_1 * B_1, crops_begin[1], crops_end[1]), crop(D_2 * B_2, crops_begin[2], crops_end[2]), ... , crop(D_{N - 1} * B_{N - 1}, crops_begin[N - 1], crops_end[N - 1])]`
+
+**Attributes**
+
+    No attributes available.
+
+**Inputs**
+
+*   **1**: `data` - input N-D tensor `[batch, D_1, D_2 ... D_{N-1}]` of *T1* type with rank >= 2. **Required.**
+*   **2**: `block_shape` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be >= 1.`block_shape[0]` is expected to be 1. **Required.** 
+*   **3**: `crops_begin` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be non-negative. crops_begin specifies the amount to crop from the beginning along each axis of `data` input . It is required that `crop_start[i] + crop_end[i] <= block_shape[i] * input_shape[i]`. `crops_begin[0]` is expected to be 0. **Required.**
+*   **4**: `crops_end` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be non-negative. crops_end specifies the amount to crop from the ending along each axis of `data` input. It is required that `crop_start[i] + crop_end[i] <= block_shape[i] * input_shape[i]`. `crops_end[0]` is expected to be 0. **Required.**
+
+**Outputs**
+
+*   **1**: N-D tensor with shape `[batch / (block_shape[0] * block_shape[1] * ... * block_shape[N - 1]), D_1 * block_shape[1] - crops_begin[1] - crops_end[1], D_2 * block_shape[2] - crops_begin[2] - crops_end[2], ..., D_{N - 1} * block_shape[N - 1] - crops_begin[N - 1] - crops_end[N - 1]` of the same type as `data` input. 
+
+**Types**
+
+* *T1*: any supported type.
+* *T2*: any supported integer type.
+
+**Example**
+
+```xml
+<layer type="BatchToSpace" ...>
+    <input>
+        <port id="0">       <!-- data -->
+            <dim>48</dim>   <!-- batch -->
+            <dim>3</dim>    <!-- spatial dimension 1 -->   
+            <dim>3</dim>    <!-- spatial dimension 2 -->
+            <dim>1</dim>    <!-- spatial dimension 3 -->
+            <dim>3</dim>    <!-- spatial dimension 4 -->
+        </port>
+        <port id="1">       <!-- block_shape value: [1, 2, 4, 3, 1] -->
+            <dim>5</dim>  
+        </port>
+        <port id="2">       <!-- crops_begin value: [0, 0, 1, 0, 0] -->
+            <dim>5</dim>
+        </port>
+        <port id="3">       <!-- crops_end value: [0, 0, 1, 0, 0] -->
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3">      
+            <dim>2</dim>    <!-- data.shape[0] / (block_shape.shape[0] * block_shape.shape[1] * ... * block_shape.shape[4]) -->
+            <dim>6</dim>    <!-- data.shape[1] * block_shape.shape[1] - crops_begin[1] - crops_end[1]-->
+            <dim>10</dim>   <!-- data.shape[2] * block_shape.shape[2] - crops_begin[2] - crops_end[2] -->
+            <dim>3</dim>    <!-- data.shape[3] * block_shape.shape[3] - crops_begin[3] - crops_end[3] -->
+            <dim>3</dim>    <!-- data.shape[4] * block_shape.shape[4] - crops_begin[4] - crops_end[4] -->
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Broadcast_1.md

@@ -0,0 +1,117 @@
+## Broadcast <a name="Broadcast"></a>
+
+**Versioned name**: *Broadcast-1*
+
+**Category**: Data movement
+
+**Short description**: *Broadcast* replicates data on the first input to fit a given shape on the second input.
+
+**Detailed description**:
+
+*Broadcast* takes the first tensor `data` and, following broadcasting rules that are specified by `mode` attribute and the 3rd input `axes_mapping`, builds a new tensor with shape matching the 2nd input tensor `target_shape`. `target_shape` input is a 1D integer tensor that represents required shape of the output.
+
+Attribute `mode` and the 3rd input `axes_mapping` are relevant for cases when rank of the input `data` tensor doesn't match the size of the `target_shape` input. They both define how axes from `data` shape are mapped to the output axes. If `mode` is set to `numpy`, it means that the standard one-directional numpy broadcasting rules are applied. They are similar to rules that applied in all binary element-wise operations in case when `auto_broadcasting` attribute is set to `numpy`, and are similar to rules described at [here](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html#general-broadcasting-rules), when only one-directional broadcasting is applied: input tensor `data` is broadcasted to `target_shape` but not vice-versa.
+
+In case if `mode` is set to `explicit`, then 3rd input `axes_mapping` comes to play. It contains a list of axis indices, each index maps an axis from the 1st input tensor `data` to axis in the output. The size of `axis_mapping` should match the rank of input `data` tensor, so all axes from `data` tensor should be mapped to axes of the output.
+
+For example, `axes_mapping = [1]` enables broadcasting of a tensor with shape `[C]` to shape `[N,C,H,W]` by replication of initial tensor along dimensions 0, 2 and 3. Another example is broadcasting of tensor with shape `[H,W]` to shape `[N,H,W,C]` with `axes_mapping = [1, 2]`. Both examples requires `mode` set to `explicit` and providing mentioned `axes_mapping` input, because such operations cannot be expressed with `axes_mapping` set to `numpy`.
+
+
+**Attributes**:
+
+* *mode*
+
+  * **Description**: specifies rules used for mapping of `input` tensor axes to output shape axes.
+  * **Range of values**:
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.; only one-directional broadcasting is applied from `data` to `target_shape`. If this attribute value is used, then the 3rd input for the operation shouldn't be provided.
+    * *explicit* - mapping of the input `data` shape axes to output shape is provided as an explicit 3rd input.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+
+**Inputs**:
+
+*   **1**: `data` - source tensor of any type and shape that is being broadcasted. Required.
+
+*   **2**: `taget_shape` - 1D integer tensor describing output shape. Required.
+
+*   **3**: `axes_mapping` - 1D integer tensor describing a list of axis indices, each index maps an axis from the 1st input tensor `data` to axis in the output. The index values in this tensor should be sorted, that disables on-the-fly transpositions of input `data` tensor while the broadcasting. `axes_mapping` input is optional depending on `mode` value.
+
+**Outputs**:
+
+*   **1**: Output tensor with replicated content from the 1st tensor `data` and with shape matched `target_shape`.
+
+**Example**
+
+```xml
+<layer ... type="Broadcast" ...>
+    <data mode="numpy"/>
+    <input>
+        <port id="0">
+            <dim>16</dim>
+            <dim>1</dim>
+            <dim>1</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
+        </port>
+        <!-- the 3rd input shouldn't be provided with mode="numpy" -->
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>16</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+
+<layer ... type="Broadcast" ...>
+    <data mode="explicit"/>
+    <input>
+        <port id="0">
+            <dim>16</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
+        </port>
+        <port id="1">
+            <dim>1</dim>   <!--The tensor contains 1 elements: [1] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>16</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+
+<layer ... type="Broadcast" ...>
+    <data mode="explicit"/>
+    <input>
+        <port id="0">
+            <dim>50</dim>
+            <dim>50</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 50, 50, 16] -->
+        </port>
+        <port id="1">
+            <dim>2</dim>   <!--The tensor contains 2 elements: [1, 2] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+            <dim>16</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Broadcast_3.md

@@ -0,0 +1,149 @@
+## Broadcast <a name="Broadcast"></a>
+
+**Versioned name**: *Broadcast-3*
+
+**Category**: Data movement
+
+**Short description**: *Broadcast* replicates data on the first input to fit a given shape on the second input.
+
+**Detailed description**:
+
+*Broadcast* takes the first tensor `data` and, following broadcasting rules that are specified by `mode` attribute and the 3rd input `axes_mapping`, builds a new tensor with shape matching the 2nd input tensor `target_shape`. `target_shape` input is a 1D integer tensor that represents required shape of the output.
+
+Attribute `mode` and the 3rd input `axes_mapping` are relevant for cases when rank of the input `data` tensor doesn't match the size of the `target_shape` input. They both define how axes from `data` shape are mapped to the output axes. If `mode` is set to `numpy`, it means that the standard one-directional numpy broadcasting rules are applied. They are similar to rules that applied in all binary element-wise operations in case when `auto_broadcasting` attribute is set to `numpy`, and are similar to rules described at [here](https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html#general-broadcasting-rules), when only one-directional broadcasting is applied: input tensor `data` is broadcasted to `target_shape` but not vice-versa.
+
+In case if `mode` is set to `bidirectional`, then the broadcast rule is similar to `numpy.array(input) * numpy.ones(target_shape)`. Dimensions are right alignment. Two corresponding dimension must have the same value, or one of them is equal to 1. If this attribute value is used, then the 3rd input for the operation shouldn't be provided. The behaviour of such kind of broadcasting is equivalent to ONNX operation [Expand](https://github.com/onnx/onnx/blob/rel-1.7.0/docs/Operators.md#Expand). 
+
+In case if `mode` is set to `explicit`, then 3rd input `axes_mapping` comes to play. It contains a list of axis indices, each index maps an axis from the 1st input tensor `data` to axis in the output. The size of `axis_mapping` should match the rank of input `data` tensor, so all axes from `data` tensor should be mapped to axes of the output.
+
+For example, `axes_mapping = [1]` enables broadcasting of a tensor with shape `[C]` to shape `[N,C,H,W]` by replication of initial tensor along dimensions 0, 2 and 3. Another example is broadcasting of tensor with shape `[H,W]` to shape `[N,H,W,C]` with `axes_mapping = [1, 2]`. Both examples requires `mode` set to `explicit` and providing mentioned `axes_mapping` input, because such operations cannot be expressed with `axes_mapping` set to `numpy`.
+
+
+**Attributes**:
+
+* *mode*
+
+  * **Description**: specifies rules used for mapping of `input` tensor axes to output shape axes.
+  * **Range of values**:
+    * *numpy* - numpy broadcasting rules, aligned with ONNX Broadcasting. Description is available in <a href="https://github.com/onnx/onnx/blob/master/docs/Broadcasting.md">ONNX docs</a>.; only one-directional broadcasting is applied from `data` to `target_shape`. If this attribute value is used, then the 3rd input for the operation shouldn't be provided.
+    * *explicit* - mapping of the input `data` shape axes to output shape is provided as an explicit 3rd input.
+    * *bidirectional* - the broadcast rule is similar to `numpy.array(input) * numpy.ones(target_shape)`. Dimensions are right alignment. Two corresponding dimension must have the same value, or one of them is equal to 1. If this attribute value is used, then the 3rd input for the operation shouldn't be provided.
+  * **Type**: string
+  * **Default value**: "numpy"
+  * **Required**: *no*
+
+
+**Inputs**:
+
+*   **1**: `data` - source tensor of type *T* and shape that is being broadcasted. Required.
+
+*   **2**: `target_shape` - 1D tensor of type *T_SHAPE* describing output shape. Required.
+
+*   **3**: `axes_mapping` - 1D tensor of type *T_SHAPE* describing a list of axis indices, each index maps an axis from the 1st input tensor `data` to axis in the output. The index values in this tensor should be sorted, that disables on-the-fly transpositions of input `data` tensor while the broadcasting. `axes_mapping` input is needed for `mode` equal to *explicit* only.
+
+**Outputs**:
+
+*   **1**: Output tensor of `data` tensor type with replicated content from the 1st tensor `data` and with shape matched `target_shape`.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_SHAPE*: any integer type.
+
+**Example**
+
+```xml
+<layer ... type="Broadcast" ...>
+    <data mode="numpy"/>
+    <input>
+        <port id="0">
+            <dim>16</dim>
+            <dim>1</dim>
+            <dim>1</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
+        </port>
+        <!-- the 3rd input shouldn't be provided with mode="numpy" -->
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>16</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+
+<layer ... type="Broadcast" ...>
+    <data mode="explicit"/>
+    <input>
+        <port id="0">
+            <dim>16</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 16, 50, 50] -->
+        </port>
+        <port id="1">
+            <dim>1</dim>   <!--The tensor contains 1 elements: [1] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>16</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+
+<layer ... type="Broadcast" ...>
+    <data mode="explicit"/>
+    <input>
+        <port id="0">
+            <dim>50</dim>
+            <dim>50</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 50, 50, 16] -->
+        </port>
+        <port id="1">
+            <dim>2</dim>   <!--The tensor contains 2 elements: [1, 2] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+            <dim>16</dim>
+        </port>
+    </output>
+</layer>
+
+<layer ... type="Broadcast" ...>
+    <data mode="bidirectional"/>
+    <input>
+        <port id="0">
+            <dim>16</dim>
+            <dim>1</dim>
+            <dim>1</dim>
+       </port>
+        <port id="1">
+            <dim>4</dim>   <!--The tensor contains 4 elements: [1, 1, 50, 50] -->
+        </port>
+        <!-- the 3rd input shouldn't be provided with mode="bidirectional" -->
+    </input>
+    <output>
+        <port id="2">
+            <dim>1</dim>
+            <dim>16</dim>
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Concat_1.md

@@ -0,0 +1,66 @@
+## Concat <a name="Concat"></a>
+
+**Versioned name**: *Concat-1*
+
+**Category**: data movement operation.
+
+**Short description**: Concatenates arbitrary number of input tensors to a single output tensor along one axis.
+
+**Attributes**:
+
+* *axis*
+
+  * **Description**: *axis* specifies dimension to concatenate along
+  * **Range of values**: integer number. Negative value means counting dimension from the end
+  * **Type**: int
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1..N**: Arbitrary number of input tensors of type *T*. Types of all tensors should match. Rank of all tensors should match. The rank is positive, so scalars as inputs are not allowed. Shapes for all inputs should match at every position except `axis` position. At least one input is required.
+
+**Outputs**:
+
+*   **1**: Tensor of the same type *T* as input tensor and shape `[d1, d2, ..., d_axis, ...]`, where `d_axis` is a sum of sizes of input tensors along `axis` dimension.
+
+**Types**
+
+* *T*: any numeric type.
+
+**Example**
+
+```xml
+<layer id="1" type="Concat">
+    <data axis="1" />
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>8</dim>  <!-- axis for concatenation -->
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+        <port id="1">
+            <dim>1</dim>
+            <dim>16</dim>  <!-- axis for concatenation -->
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+        <port id="2">
+            <dim>1</dim>
+            <dim>32</dim>  <!-- axis for concatenation -->
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>1</dim>
+            <dim>56</dim>  <!-- concatenated axis: 8 + 16 + 32 = 48 -->
+            <dim>50</dim>
+            <dim>50</dim>
+        </port>
+    </output>
+</layer>
+
+```

docs/ops/movement/DepthToSpace_1.md

@@ -0,0 +1,80 @@
+## DepthToSpace <a name="DepthToSpace"></a>
+
+**Versioned name**: *DepthToSpace-1*
+
+**Category**: *Data movement*
+
+**Short description**: *DepthToSpace* operation rearranges data from the depth dimension of the input tensor into spatial dimensions of the output tensor.
+
+**Attributes**
+
+* *block_size*
+
+  * **Description**: *block_size* specifies the size of the value block to be moved. The depth dimension size must be evenly divided by `block_size ^ (len(input.shape) - 2)`.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *mode*
+
+  * **Description**: specifies how the input depth dimension is split to block coordinates and the new depth dimension.
+  * **Range of values**:
+    * *blocks_first*: the input depth is divided to `[block_size, ..., block_size,  new_depth]`
+    * *depth_first*: the input depth is divided to `[new_depth, block_size, ..., block_size]`
+  * **Type**: `string`
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**
+
+*   **1**: `data` - input tensor of any type with rank >= 3. Required.
+
+**Outputs**
+
+*   **1**: permuted tensor with shape `[N, C / block_size ^ K, D1 * block_size, D2 * block_size, ..., DK * block_size]`.
+
+**Detailed description**
+
+*DepthToSpace* operation permutes elements from the input tensor with shape `[N, C, D1, D2, ..., DK]`, to the output tensor where values from the input depth dimension (features) `C` are moved to spatial blocks in `D1`, ..., `DK`. Refer to the [ONNX* specification](https://github.com/onnx/onnx/blob/master/docs/Operators.md#DepthToSpace) for an example of the 4D input tensor case.
+
+The operation is equivalent to the following transformation of the input tensor `data` with `K` spatial dimensions of shape `[N, C, D1, D2, ..., DK]` to *Y* output tensor. If `mode = blocks_first`:
+
+    x' = reshape(data, [N, block_size, block_size, ..., block_size, C / (block_size ^ K), D1, D2, ..., DK])
+
+    x'' = transpose(x', [0,  K + 1,  K + 2, 1, K + 3, 2, K + 4, 3, ..., K + (K + 1), K])
+
+    y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size])
+
+If `mode = depth_first`:
+
+    x' = reshape(data, [N, C / (block_size ^ K), block_size, block_size, ..., block_size, D1, D2, ..., DK])
+
+    x'' = transpose(x', [0,  1,  K + 2, 2, K + 3, 3, K + 4, 4, ..., K + (K + 1), K + 1])
+
+    y = reshape(x'', [N, C / (block_size ^ K), D1 * block_size, D2 * block_size, D3 * block_size, ..., DK * block_size])
+
+
+**Example**
+
+```xml
+<layer type="DepthToSpace" ...>
+    <data block_size="2" mode="blocks_first"/>
+    <input>
+        <port id="0">
+            <dim>5</dim>
+            <dim>28</dim>
+            <dim>2</dim>
+            <dim>3</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>5</dim>  <!-- data.shape[0] -->
+            <dim>7</dim>  <!-- data.shape[1] / (block_size ^ 2) -->
+            <dim>4</dim>  <!-- data.shape[2] * block_size -->
+            <dim>6</dim>  <!-- data.shape[3] * block_size -->
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ExtractImagePatches_3.md

@@ -0,0 +1,341 @@
+## ExtractImagePatches <a name="ExtractImagePatches"></a>
+
+**Versioned name**: *ExtractImagePatches-3*
+
+**Category**: *Data movement*
+
+**Short description**: The *ExtractImagePatches* operation collects patches from the input tensor, as if applying a convolution. All extracted patches are stacked in the depth dimension of the output.
+
+**Detailed description**:
+
+The *ExtractImagePatches* operation is similar to the TensorFlow* operation [ExtractImagePatches](https://www.tensorflow.org/api_docs/python/tf/image/extract_patches).
+
+This op extracts patches of shape `sizes` which are `strides` apart in the input image. The output elements are taken from the input at intervals given by the `rate` argument, as in dilated convolutions.
+
+The result is a 4D tensor containing image patches with size `size[0] * size[1] * depth` vectorized in the "depth" dimension.
+
+The "auto_pad" attribute has no effect on the size of each patch, it determines how many patches are extracted.
+
+
+**Attributes**
+
+* *sizes*
+
+  * **Description**: *sizes* is a size `[size_rows, size_cols]` of the extracted patches.
+  * **Range of values**: non-negative integer number
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *strides*
+
+  * **Description**: *strides* is a distance `[stride_rows, stride_cols]` between centers of two consecutive patches in an input tensor.
+  * **Range of values**: non-negative integer number
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *rates*
+
+  * **Description**: *rates* is the input stride `[rate_rows, rate_cols]`, specifying how far two consecutive patch samples are in the input. Equivalent to extracting patches with `patch_sizes_eff = patch_sizes + (patch_sizes - 1) * (rates - 1)`, followed by subsampling them spatially by a factor of rates. This is equivalent to rate in dilated (a.k.a. Atrous) convolutions. 
+  * **Range of values**: non-negative integer number
+  * **Type**: int[]
+  * **Default value**: None
+  * **Required**: *yes*
+
+* *auto_pad*
+
+  * **Description**: *auto_pad* how the padding is calculated. Possible values:
+    * *same_upper (same_lower)* the input is padded by zeros to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
+    * *valid* - do not use padding.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**
+
+*   **1**: `data` the 4-D tensor of type *T* with shape `[batch, depth, in_rows, in_cols]`. **Required.**
+
+**Outputs**
+
+*   **1**: 4-D tensor with shape `[batch, size[0] * size[1] * depth, out_rows, out_cols]` with type equal to `data` tensor. Note `out_rows` and `out_cols` are the dimensions of the output patches.
+
+**Types**
+
+* *T*: any supported type.
+
+
+**Example**
+
+```xml
+<layer type="ExtractImagePatches" ...>
+    <data sizes="3,3" strides="5,5" rates="1,1" auto_pad="valid"/>
+    <input>
+        <port id="0">
+            <dim>64</dim>
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>10</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1" precision="f32">      
+            <dim>64</dim>
+            <dim>27</dim>
+            <dim>2</dim>
+            <dim>2</dim>
+        </port>
+    </output>
+</layer>
+```
+
+Image is a `1 x 1 x 10 x 10` array that contains the numbers 1 through 100. We use the symbol `x` to mark output patches.
+
+1. `sizes="3,3", strides="5,5", rates="1,1", auto_pad="valid"`
+
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp; 4 &nbsp; 5 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp; 9 10    
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;14 15 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x 19 20  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;24 25 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x 29 30  
+  31 32 33 34 35 36 37 38 39 40  
+  41 42 43 44 45 46 47 48 49 50  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;54 55 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x 59 60  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;64 65 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x 69 70  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;74 75 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x 79 80  
+  81 82 83 84 85 86 87 88 89 90  
+  91 92 93 94 95 96 97 98 99 100  
+  
+  output:  
+
+    [[[[ 1  6]
+       [51 56]]
+
+      [[ 2  7]
+       [52 57]]
+
+      [[ 3  8]
+       [53 58]]
+
+      [[11 16]
+       [61 66]]
+
+      [[12 17]
+       [62 67]]
+
+      [[13 18]
+       [63 68]]
+
+      [[21 26]
+       [71 76]]
+
+      [[22 27]
+       [72 77]]
+
+      [[23 28]
+       [73 78]]]] 
+
+  output shape: `[1, 9, 2, 2]`
+  
+2. `sizes="4,4", strides="8,8", rates="1,1", auto_pad="valid"`
+
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;&nbsp;5 &nbsp;&nbsp;6 &nbsp;&nbsp;7 &nbsp;&nbsp;8 &nbsp;&nbsp;9 10  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;15 16 17 18 19 20  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;25 26 27 28 29 30  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;35 36 37 38 39 40  
+  41 42 43 44 45 46 47 48 49 50  
+  51 52 53 54 55 56 57 58 59 60  
+  61 62 63 64 65 66 67 68 69 70  
+  71 72 73 74 75 76 77 78 79 80  
+  81 82 83 84 85 86 87 88 89 90  
+  91 92 93 94 95 96 97 98 99 100  
+  
+  output:  
+
+    [[[[ 1]]
+
+      [[ 2]]
+
+      [[ 3]]
+
+      [[ 4]]
+
+      [[11]]
+
+      [[12]]
+
+      [[13]]
+
+      [[14]]
+
+      [[21]]
+
+      [[22]]
+
+      [[23]]
+
+      [[24]]
+
+      [[31]]
+
+      [[32]]
+
+      [[33]]
+
+      [[34]]]] 
+      
+  output shape: `[1, 16, 1, 1]`
+
+3. `sizes="4,4", strides="9,9", rates="1,1", auto_pad="same_upper"`
+
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;&nbsp;4 &nbsp;&nbsp;5 &nbsp;&nbsp;6 &nbsp;&nbsp;7 &nbsp;&nbsp;8 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;14 15 16 17 18 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;24 25 26 27 28 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;0 31 32 33 34 35 36 37 38 39 40 &nbsp;&nbsp;0 &nbsp;&nbsp;0  
+   &nbsp;&nbsp;0 41 42 43 44 45 46 47 48 49 50 &nbsp;&nbsp;0 &nbsp;&nbsp;0  
+   &nbsp;&nbsp;0 51 52 53 54 55 56 57 58 59 60 &nbsp;&nbsp;0 &nbsp;&nbsp;0  
+   &nbsp;&nbsp;0 61 62 63 64 65 66 67 68 69 70 &nbsp;&nbsp;0 &nbsp;&nbsp;0  
+   &nbsp;&nbsp;0 71 72 73 74 75 76 77 78 79 80 &nbsp;&nbsp;0 &nbsp;&nbsp;0  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;84 85 86 87 88 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;94 95 96 97 98 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;0 &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;x    
+
+  output:  
+
+    [[[[  0   0]
+       [  0  89]]
+
+      [[  0   0]
+       [ 81  90]]
+
+      [[  0   0]
+       [ 82   0]]
+
+      [[  0   0]
+       [ 83   0]]
+
+      [[  0   9]
+       [  0  99]]
+
+      [[  1  10]
+       [ 91 100]]
+
+      [[  2   0]
+       [ 92   0]]
+
+      [[  3   0]
+       [ 93   0]]
+
+      [[  0  19]
+       [  0   0]]
+
+      [[ 11  20]
+       [  0   0]]
+
+      [[ 12   0]
+       [  0   0]]
+
+      [[ 13   0]
+       [  0   0]]
+
+      [[  0  29]
+       [  0   0]]
+
+      [[ 21  30]
+       [  0   0]]
+
+      [[ 22   0]
+       [  0   0]]
+
+      [[ 23   0]
+       [  0   0]]]] 
+
+  output shape: `[1, 16, 2, 2]`
+
+4. `sizes="3,3", strides="5,5", rates="2,2", auto_pad="valid"`  
+This time we use the symbols `x`, `y`, `z` and `k` to distinguish the patches:
+  
+   &nbsp;&nbsp;x &nbsp;&nbsp;2 &nbsp;&nbsp;x &nbsp;&nbsp;4 &nbsp;&nbsp;x &nbsp;&nbsp;y &nbsp;&nbsp;7 &nbsp;&nbsp;y &nbsp;&nbsp;9 &nbsp;&nbsp;y  
+  11 12 13 14 15 16 17 18 19 20  
+   &nbsp;&nbsp;x &nbsp;22 &nbsp;&nbsp;x 24 &nbsp;&nbsp;x &nbsp;&nbsp;y 27 &nbsp;&nbsp;y 29 &nbsp;&nbsp;y  
+  31 32 33 34 35 36 37 38 39 40  
+   &nbsp;&nbsp;x &nbsp;42 &nbsp;&nbsp;x 44 &nbsp;&nbsp;x &nbsp;&nbsp;y 47 &nbsp;&nbsp;y 49 &nbsp;&nbsp;y  
+   &nbsp;&nbsp;z &nbsp;52 &nbsp;&nbsp;z 54 &nbsp;&nbsp;z &nbsp;&nbsp;k 57 &nbsp;&nbsp;k 59 &nbsp;&nbsp;k  
+  61 62 63 64 65 66 67 68 69 70  
+   &nbsp;&nbsp;z &nbsp;72 &nbsp;&nbsp;z 74 &nbsp;&nbsp;z &nbsp;&nbsp;k 77 &nbsp;&nbsp;k 79 &nbsp;&nbsp;k  
+  81 82 83 84 85 86 87 88 89 90  
+   &nbsp;&nbsp;z &nbsp;92 &nbsp;&nbsp;z 94 &nbsp;&nbsp;z &nbsp;&nbsp;k 97 &nbsp;&nbsp;k 99 &nbsp;&nbsp;k
+   
+  output:
+
+    [[[[  1   6]
+       [ 51  56]]
+
+      [[  3   8]
+       [ 53  58]]
+
+      [[  5  10]
+       [ 55  60]]
+
+      [[ 21  26]
+       [ 71  76]]
+
+      [[ 23  28]
+       [ 73  78]]
+
+      [[ 25  30]
+       [ 75  80]]
+
+      [[ 41  46]
+       [ 91  96]]
+
+      [[ 43  48]
+       [ 93  98]]
+
+      [[ 45  50]
+       [ 95 100]]]] 
+
+  output_shape: `[1, 9, 2, 2]`
+
+5. `sizes="2,2", strides="3,3", rates="1,1", auto_pad="valid"`  
+Image is a `1 x 2 x 5 x 5` array that contains two feature maps where feature map with coordinate 0 contains numbers in a range `[1, 25]` and feature map with coordinate 1 contains numbers in a range `[26, 50]`  
+
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;&nbsp;3 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;6 &nbsp;&nbsp;7 &nbsp;&nbsp;8 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+  11 12 13 14 15  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;18 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;23 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;28 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;33 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+  36 37 38 39 40  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;43 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+   &nbsp;&nbsp;x &nbsp;&nbsp;x &nbsp;48 &nbsp;&nbsp;x &nbsp;&nbsp;x  
+
+  output:
+
+    [[[[ 1  4]
+       [16 19]]
+
+      [[26 29]
+       [41 44]]
+
+      [[ 2  5]
+       [17 20]]
+
+      [[27 30]
+       [42 45]]
+
+      [[ 6  9]
+       [21 24]]
+
+      [[31 34]
+       [46 49]]
+
+      [[ 7 10]
+       [22 25]]
+
+      [[32 35]
+       [47 50]]]] 
+   
+  output shape: `[1, 8, 2, 2]`

docs/ops/movement/GatherTree_1.md

@@ -0,0 +1,80 @@
+## GatherTree <a name="GatherTree"></a>
+
+**Versioned name**: *GatherTree-1*
+
+**Category**: Beam search post-processing
+
+**Short description**: Generates the complete beams from the ids per each step and the parent beam ids.
+
+**Detailed description**
+
+GatherTree operation implements the same algorithm as GatherTree operation in TensorFlow. Please see complete documentation [here](https://www.tensorflow.org/versions/r1.12/api_docs/python/tf/contrib/seq2seq/gather_tree?hl=en).
+
+Pseudo code:
+
+```python
+for batch in range(BATCH_SIZE):
+    for beam in range(BEAM_WIDTH):
+        max_sequence_in_beam = min(MAX_TIME, max_seq_len[batch])
+
+        parent = parent_idx[max_sequence_in_beam - 1, batch, beam]
+
+        for level in reversed(range(max_sequence_in_beam - 1)):
+            final_idx[level, batch, beam] = step_idx[level, batch, parent]
+
+            parent = parent_idx[level, batch, parent]
+```
+
+Element data types for all input tensors should match each other.
+
+**Attributes**: *GatherTree* has no attributes
+
+**Inputs**
+
+* **1**:  `step_ids` -- a tensor of shape `[MAX_TIME, BATCH_SIZE, BEAM_WIDTH]` of type `T` with indices from per each step. Required.
+
+* **2**:  `parent_idx` -- a tensor of shape `[MAX_TIME, BATCH_SIZE, BEAM_WIDTH]` of type `T` with parent beam indices. Required.
+
+* **3**:  `max_seq_len` -- a tensor of shape `[BATCH_SIZE]` of type `T` with maximum lengths for each sequence in the batch. Required.
+
+* **4**:  `end_token` -- a scalar tensor of type `T` with value of the end marker in a sequence. Required.
+
+
+**Outputs**
+
+* **1**: `final_idx` -- a tensor of shape `[MAX_TIME, BATCH_SIZE, BEAM_WIDTH]` of type `T`.
+
+**Types**
+
+* *T*: `float32` or `int32`; `float32` should have integer values only.
+
+**Example**
+
+```xml
+<layer type="GatherTree" ...>
+    <input>
+        <port id="0">
+            <dim>100</dim>
+            <dim>1</dim>
+            <dim>10</dim>
+        </port>
+        <port id="1">
+            <dim>100</dim>
+            <dim>1</dim>
+            <dim>10</dim>
+        </port>
+        <port id="2">
+            <dim>1</dim>
+        </port>
+        <port id="3">
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>100</dim>
+            <dim>1</dim>
+            <dim>10</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Gather_1.md

@@ -0,0 +1,61 @@
+## Gather <a name="Gather"></a>
+
+**Versioned name**: *Gather-1*
+
+**Category**: Data movement operations
+
+**Short description**: *Gather* operation takes slices of data in the first input tensor according to the indices specified in the second input tensor and axis from the third input.
+
+**Detailed description**
+
+    output[:, ... ,:, i, ... , j,:, ... ,:] = input1[:, ... ,:, input2[i, ... ,j],:, ... ,:]
+
+Where `i` is the value from the third input.
+
+**Attributes**: *Gather* has no attributes
+
+**Inputs**
+
+* **1**:  Tensor with arbitrary data. Required.
+
+* **2**:  Tensor with indices to gather. The values for indices are in the range `[0, input1[axis] - 1]`. Required.
+
+* **3**:  Scalar or 1D tensor *axis* is a dimension index to gather data from. For example, *axis* equal to 1 means that gathering is performed over the first dimension. Negative value means reverse indexing. Allowed values are from `[-len(input1.shape), len(input1.shape) - 1]`. Required.
+
+**Outputs**
+
+* **1**: The resulting tensor that consists of elements from the first input tensor gathered by indices from the second input tensor. Shape of the tensor is `[input1.shape[:axis], input2.shape, input1.shape[axis + 1:]]`
+
+**Example**
+
+```xml
+<layer id="1" type="Gather" ...>
+    <input>
+        <port id="0">
+            <dim>6</dim>
+            <dim>12</dim>
+            <dim>10</dim>
+            <dim>24</dim>
+        </port>
+        <port id="1">
+            <dim>15</dim>
+            <dim>4</dim>
+            <dim>20</dim>
+            <dim>28</dim>
+        </port>
+        <port id="2"/>   <!--  axis = 1  -->
+    </input>
+    <output>
+        <port id="2">
+            <dim>6</dim>      <!-- embedded dimension from the 1st input -->
+            <dim>15</dim>     <!-- embedded dimension from the 2nd input -->
+            <dim>4</dim>      <!-- embedded dimension from the 2nd input -->
+            <dim>20</dim>     <!-- embedded dimension from the 2nd input -->
+            <dim>28</dim>     <!-- embedded dimension from the 2nd input -->
+            <dim>10</dim>     <!-- embedded dimension from the 1st input -->
+            <dim>24</dim>     <!-- embedded dimension from the 1st input -->
+        </port>
+    </output>
+</layer>
+```
+

docs/ops/movement/Pad_1.md

@@ -0,0 +1,125 @@
+## Pad <a name="Pad"></a>
+
+**Versioned name**: *Pad-1*
+
+**Category**: *Data movement operations*
+
+**Short description**: *Pad* operation extends an input tensor on edges. The amount and value of padded elements are defined by inputs and attributes.
+
+**Attributes**
+
+* *pad_mode*
+
+  * **Description**: *pad_mode* specifies the method used to generate new element values.
+  * **Range of values**: Name of the method in string format:
+    * `constant` - padded values are equal to the value of the *pad_value* operation attribute.
+    * `edge` - padded values are copied from the respective edge of the input `data` tensor.
+    * `reflect` - padded values are a reflection of the input `data` tensor; values on the edges are not duplicated. `pads_begin[D]` and `pads_end[D]` must be not greater than `data.shape[D] – 1` for any valid `D`.
+    * `symmetric` - padded values are symmetrically added from the input `data` tensor. This method is similar to the `reflect`, but values on edges are duplicated. Refer to the examples below for more details. `pads_begin[D]` and `pads_end[D]` must be not greater than `data.shape[D]` for any valid `D`.
+  * **Type**: string
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**
+
+* **1**: `data` - input tensor to be padded. Required.
+
+* **2**: `pads_begin` - specifies the number of padding elements at the beginning of each axis. A list of non-negative integers. The length of the list must be equal to the number of dimensions in the input tensor. Required.
+
+* **3**: `pads_end` - specifies the number of padding elements at the beginning of each axis. A list of non-negative integers. The length of the list must be equal to the number of dimensions in the input tensor. Required.
+
+* **4**: `pad_value` - scalar tensor of type matching type of elements in `data` tensor to be replicated in padded area. Used with the `pad_mode = "constant"` only. All new elements are populated with this value. Optional for `pad_mode = "constant"`. If not provided, 0 of appropriate type is used. Shouldn't be set for other `pad_mode` values.
+
+
+**Outputs**
+
+* **1**: Output padded tensor with dimensions `pads_begin[D] + data.shape[D] + pads_end[D]` for each `D` from `0` to `len(data.shape) - 1`.
+
+
+**Detailed Description**
+
+The attributes specify a number of elements to add along each axis and a rule by which new element values are generated: for example, whether they are filled with a given constant or generated based on the input tensor content.
+
+The following examples illustrate how output tensor is generated for the *Pad* layer for a given input tensor:
+```
+INPUT =
+[[ 1  2  3  4 ]
+[  5  6  7  8 ]
+[  9 10 11 12 ]]
+```
+with the following attributes:
+```
+pads_begin = [0, 1]
+pads_end = [2, 3]
+```
+depending on the *pad_mode*.
+* `pad_mode = "constant"`:
+```
+OUTPUT =
+[[ 0  1  2  3  4  0  0  0 ]
+[  0  5  6  7  8  0  0  0 ]
+[  0  9 10 11 12  0  0  0 ]
+[  0  0  0  0  0  0  0  0 ]
+[  0  0  0  0  0  0  0  0 ]]
+```
+* `pad_mode = "edge"`:
+```
+OUTPUT =
+[[ 1  1  2  3  4  4  4  4 ]
+[  5  5  6  7  8  8  8  8 ]
+[  9  9 10 11 12 12 12 12 ]
+[  9  9 10 11 12 12 12 12 ]
+[  9  9 10 11 12 12 12 12 ]]
+```
+* `pad_mode = "reflect"`:
+```
+OUTPUT =
+[[ 2  1  2  3  4  3  2  1 ]
+[  6  5  6  7  8  7  6  5 ]
+[ 10  9 10 11 12 11 10  9 ]
+[  6  5  6  7  8  7  6  5 ]
+[  2  1  2  3  4  3  2  1 ]]
+```
+* `pad_mode = "symmetric"`:
+```
+OUTPUT =
+[[ 1  1  2  3  4  4  3  2 ]
+[  5  5  6  7  8  8  7  6 ]
+[  9  9 10 11 12 12 11 10 ]
+[  9  9 10 11 12 12 11 10 ]
+[  5  5  6  7  8  8  7  6 ]]
+```
+
+**Example**
+
+```xml
+<layer ... type="Pad" ...>
+    <data pad_mode="constant"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>3</dim>
+            <dim>32</dim>
+            <dim>40</dim>
+        </port>
+        <port id="1">
+            <dim>4</dim>     <!-- pads_begin = [0, 5, 2, 1]  -->
+        </port>
+        <port id="2">
+            <dim>4</dim>     <!-- pads_end = [1, 0, 3, 7] -->
+        </port>
+        <port id="3">
+                             <!-- pad_value = 15.0 -->
+        </port>
+    </input>
+    <output>
+        <port id="0">
+            <dim>2</dim>     <!-- 2 = 0 + 1 + 1 = pads_begin[0] + input.shape[0] + pads_end[0] -->
+            <dim>8</dim>     <!-- 8 = 5 + 3 + 0 = pads_begin[1] + input.shape[1] + pads_end[1] -->
+            <dim>37</dim>    <!-- 37 = 2 + 32 + 3 = pads_begin[2] + input.shape[2] + pads_end[2] -->
+            <dim>48</dim>    <!-- 48 = 1 + 40 + 7 = pads_begin[3] + input.shape[3] + pads_end[3] -->
+                             <!-- all new elements are filled with 15.0 value -->
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ReverseSequence_1.md

@@ -0,0 +1,60 @@
+## ReverseSequence <a name="ReverseSequence"></a>
+
+**Versioned name**: *ReverseSequence-1*
+
+**Category**: data movement operation
+
+**Short description**: *ReverseSequence* reverses variable length slices of data.
+
+**Detailed description**: *ReverseSequence* slices input along the dimension specified in the *batch_axis*, and for each slice *i*, reverses the first *lengths[i]* (the second input) elements along the dimension specified in the *seq_axis*.
+
+**Attributes**
+
+* *batch_axis*
+
+  * **Description**: *batch_axis* is the index of the batch dimension.
+  * **Range of values**: an integer. Can be negative.
+  * **Type**: `int`
+  * **Default value**: 0
+  * **Required**: *no*
+
+* *seq_axis*
+
+  * **Description**: *seq_axis* is the index of the sequence dimension.
+  * **Range of values**: an integer. Can be negative.
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: tensor with input data to reverse. Required.
+
+*   **2**: 1D tensor populated with integers with sequence lengths in the 1st input tensor. Required.
+
+**Example**
+
+```xml
+<layer ... type="ReverseSequence">
+    <data batch_axis="0" seq_axis="1"/>
+    <input>
+        <port id="0">
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>100</dim>
+            <dim>200</dim>
+        </port>
+        <port id="1">
+            <dim>3</dim>
+        </port>
+     </input>
+    <output>
+        <port id="2">
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>100</dim>
+            <dim>200</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Reverse_1.md

@@ -0,0 +1,68 @@
+## Reverse <a name="Reverse"></a>
+
+**Versioned name**: *Reverse-1*
+
+**Category**: data movement operation
+
+**Short description**: *Reverse* operations reverse specified axis in an input tensor.
+
+**Detailed description**: *Reverse* produces a tensor with the same shape as the first input tensor and with elements reversed along dimensions specified in the second input tensor. The axes can be represented either by dimension indices or as a mask. The interpretation of the second input is determined by *mode* attribute. 
+
+If `index` mode is used, the second tensor should contain indices of axes that should be reversed. The length of the second tensor should be in a range from 0 to rank of the 1st input tensor.
+
+In case if `mask` mode is used, then the second input tensor length should be equal to the rank of the 1st input. And each value has boolean value `True` or `False`. `True` means the corresponding axes should be reverted, `False` means it should be untouched.
+
+If no axis specified, that means either the second input is empty if `index` mode is used or second input has only `False` elements if `mask` mode is used, then *Reverse* just passes the source tensor through output not doing any data movements.
+
+**Attributes**
+
+* *mode*
+
+  * **Description**: specifies how the second input tensor should be interpreted: as a set of indices or a mask
+  * **Range of values**: `index`, `mask`
+  * **Type**: `string`
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**:
+
+*   **1**: `data` the tensor of type *T1* with input data to reverse. **Required.**
+
+*   **2**: `axis` 1D tensor of type *T2* populated with indices of reversed axes if *mode* attribute is set to `index`, otherwise 1D tensor of type *T3* and with a length equal to the rank of `data` input that specifies a mask for reversed axes.
+
+**Outputs**:
+
+*   **1**: output reversed tensor with shape and type equal to `data` tensor.
+
+**Types**
+
+* *T1*: any supported type.
+* *T2*: any supported integer type.
+* *T3*: boolean type.
+
+**Example**
+
+```xml
+<layer ... type="Reverse">
+    <data mode="index"/>
+    <input>
+        <port id="0">
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>100</dim>
+            <dim>200</dim>
+        </port>
+        <port id="1">
+            <dim>1</dim>   <!-- reverting along single axis -->
+        </port>
+     </input>
+    <output>
+        <port id="2">
+            <dim>3</dim>
+            <dim>10</dim>
+            <dim>100</dim>
+            <dim>200</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ScatterElementsUpdate_3.md

@@ -0,0 +1,87 @@
+## ScatterElementsUpdate <a name="ScatterElementsUpdate"></a>
+
+**Versioned name**: *ScatterElementsUpdate-3*
+
+**Category**: Data movement operations
+
+**Short description**: Creates a copy of the first input tensor with updated elements specified with second and third input tensors.
+
+**Detailed description**: For each entry in `updates`, the target index in `data` is obtained by combining the corresponding entry in
+`indices` with the index of the entry itself: the index-value for dimension equal to `axis` is obtained from the value of the corresponding entry in
+`indices` and the index-value for dimension not equal to `axis` is obtained from the index of the entry itself.
+
+For instance, in a 3D tensor case, the update corresponding to the `[i][j][k]` entry is performed as below:
+
+```
+output[indices[i][j][k]][j][k] = updates[i][j][k] if axis = 0,
+output[i][indices[i][j][k]][k] = updates[i][j][k] if axis = 1,
+output[i][j][indices[i][j][k]] = updates[i][j][k] if axis = 2
+```
+
+`update` tensor dimensions are less or equal to the corresponding `data` tensor dimensions. 
+
+**Attributes**: *ScatterElementsUpdate* does not have attributes.
+
+**Inputs**:
+
+*   **1**: `data` tensor of arbitrary rank `r` and of type *T*. Required.
+
+*   **2**: `indices` tensor with indices of type *T_IND*. The rank of the tensor is equal to the rank of `data` tensor.
+All index values are expected to be within bounds `[0, s - 1]` along axis of size `s`. If multiple indices point to the
+same output location then the order of updating the values is undefined. If an index points to non-existing output
+tensor element or is negative then exception is raised. Required.
+
+*   **3**: `updates` tensor of shape equal to the shape of `indices` tensor and of type *T*. Required.
+
+*   **4**: `axis` tensor with scalar or 1D tensor with one element of type *T_AXIS* specifying axis for scatter.
+The value can be in range `[-r, r - 1]` where `r` is the rank of `data`. Required.
+
+**Outputs**:
+
+*   **1**: tensor with shape equal to `data` tensor of the type *T*.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_IND*: any integer numeric type.
+
+* *T_AXIS*: any integer numeric type.
+
+**Example**
+
+```xml
+<layer ... type="ScatterElementsUpdate">
+    <input>
+        <port id="0">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>7</dim>
+            <dim>7</dim>
+        </port>
+        <port id="1">
+            <dim>125</dim>
+            <dim>20</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+        </port>
+        <port id="2">
+            <dim>125</dim>
+            <dim>20</dim>
+            <dim>7</dim>
+            <dim>6</dim>
+        </port>
+        <port id="3">     <!-- value [0] -->
+            <dim>1</dim>
+        </port>
+    </input>
+    <output>
+        <port id="4" precision="FP32">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>7</dim>
+            <dim>7</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ScatterNDUpdate_3.md

@@ -0,0 +1,94 @@
+## ScatterNDUpdate <a name="ScatterNDUpdate"></a>
+
+**Versioned name**: *ScatterNDUpdate-3*
+
+**Category**: Data movement operations
+
+**Short description**: Creates a copy of the first input tensor with updated elements specified with second and third input tensors. This is similar to [Reference](https://github.com/onnx/onnx/blob/master/docs/Operators.md#ScatterND)
+
+**Detailed description**: The operation produces a copy of `data` tensor and updates its value to values specified 
+by `updates` at specific index positions specified by `indices`. The output shape is the same as the shape of `data`. 
+`indices` tensor must not have duplicate entries. In case duplicate entries in `indices` the result is undefined.
+
+The last dimension of `indices` can be at most the rank of `data.shape`. 
+The last dimension of `indices` corresponds to indices into elements if `indices.shape[-1]` = `data.shape.rank` or slices 
+if `indices.shape[-1]` < `data.shape.rank`. `updates` is a tensor with shape `indices.shape[:-1] + data.shape[indices.shape[-1]:]`
+
+Example 1 that shows update of four single elements in `data`:
+
+```
+data    = [1, 2, 3, 4, 5, 6, 7, 8]
+indices = [[4], [3], [1], [7]]
+updates = [9, 10, 11, 12]
+output  = [1, 11, 3, 10, 9, 6, 7, 12]
+```
+
+Example 2 that shows update of two slices of `4x4` shape in `data`:
+
+```
+data    = [[[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
+           [[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
+           [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]],
+           [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]
+indices = [[0], [2]]
+updates = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],
+           [[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]]]
+output  = [[[5, 5, 5, 5], [6, 6, 6, 6], [7, 7, 7, 7], [8, 8, 8, 8]],
+           [[1, 2, 3, 4], [5, 6, 7, 8], [8, 7, 6, 5], [4, 3, 2, 1]],
+           [[1, 1, 1, 1], [2, 2, 2, 2], [3, 3, 3, 3], [4, 4, 4, 4]],
+           [[8, 7, 6, 5], [4, 3, 2, 1], [1, 2, 3, 4], [5, 6, 7, 8]]]
+```
+
+
+**Attributes**: *ScatterNDUpdate* does not have attributes.
+
+**Inputs**:
+
+*   **1**: `data` tensor of arbitrary rank `r` >= 1 and of type *T*. Required.
+
+*   **2**: `indices` tensor with indices of arbitrary rank `q` >= 1 and of type *T_IND*. All index values `i_j` in index entry `(i_0, i_1, ...,i_k)` (where `k = indices.shape[-1]`) must be within bounds `[0, s_j - 1]` where `s_j = data.shape[j]`. `k` must be at most `r`. Required.
+
+*   **3**: `updates` tensor of rank `r - indices.shape[-1] + q - 1` of type *T*. Required.
+
+**Outputs**:
+
+*   **1**: tensor with shape equal to `data` tensor of the type *T*.
+
+**Types**
+
+* *T*: any supported type.
+
+* *T_IND*: any supported integer types.
+
+**Example**
+
+```xml
+<layer ... type="ScatterNDUpdate">
+    <input>
+        <port id="0">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>10</dim>
+            <dim>15</dim>
+        </port>
+        <port id="1">
+            <dim>25</dim>
+            <dim>125</dim>
+            <dim>3</dim>
+        </port>
+        <port id="2">
+            <dim>25</dim>
+            <dim>125</dim>
+            <dim>15</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>10</dim>
+            <dim>15</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ScatterUpdate_3.md

@@ -0,0 +1,93 @@
+## ScatterUpdate <a name="ScatterUpdate"></a>
+
+**Versioned name**: *ScatterUpdate-3*
+
+**Category**: Data movement operations
+
+**Short description**: *ScatterUpdate* creates a copy of the first input tensor with updated elements specified with second and third input tensors.
+
+**Detailed description**: *ScatterUpdate* creates a copy of the first input tensor with updated elements in positions specified with `indices` input
+and values specified with `updates` tensor starting from the dimension with index `axis`. For the `data` tensor of shape `[d_0, d_1, ..., d_n]`,
+`indices` tensor of shape `[i_0, i_1, ..., i_k]` and `updates` tensor of shape
+`[d_0, d_1, ... d_(axis - 1), i_0, i_1, ..., i_k, d_(axis + k + 1), ..., d_n]` the operation computes
+for each `m, n, ..., p` of the `indices` tensor indices:
+
+```
+data[..., indices[m, n, ..., p], ...] = updates[..., m, n, ..., p, ...]
+```
+
+where first `...` in the `data` corresponds to first `axis` dimensions, last `...` in the `data` corresponds to the
+`rank(data) - (axis + 1)` dimensions.
+
+Several examples for case when `axis = 0`:
+1. `indices` is a 0D tensor: `data[indices, ...] = updates[...]`
+2. `indices` is a 1D tensor (for each `i`): `data[indices[i], ...] = updates[i, ...]`
+3. `indices` is a ND tensor (for each `i, ..., j`): `data[indices[i, ..., j], ...] = updates[i, ..., j, ...]`
+
+This operation is similar to TensorFlow* operation [ScatterUpdate](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/scatter_update)
+but allows scattering for the arbitrary axis.
+
+**Attributes**: *ScatterUpdate* does not have attributes.
+
+**Inputs**:
+
+*   **1**: `data` tensor of arbitrary rank `r` and of type *T*. Required.
+
+*   **2**: `indices` tensor with indices of type *T_IND*.
+All index values are expected to be within bounds `[0, s - 1]` along axis of size `s`. If multiple indices point to the
+same output location then the order of updating the values is undefined. If an index points to non-existing output
+tensor element or is negative then an exception is raised. Required.
+
+*   **3**: `updates` tensor of type *T*. Required.
+
+*   **4**: `axis` tensor with scalar or 1D tensor with one element of type *T_AXIS* specifying axis for scatter.
+The value can be in range `[-r, r - 1]` where `r` is the rank of `data`. Required.
+
+**Outputs**:
+
+*   **1**: tensor with shape equal to `data` tensor of the type *T*.
+
+**Types**
+
+* *T*: any numeric type.
+
+* *T_IND*: any supported integer types.
+
+* *T_AXIS*: any supported integer types.
+
+**Example**
+
+```xml
+<layer ... type="ScatterUpdate">
+    <input>
+        <port id="0">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>10</dim>
+            <dim>15</dim>
+        </port>
+        <port id="1">
+            <dim>125</dim>
+            <dim>20</dim>
+        </port>
+        <port id="2">
+            <dim>1000</dim>
+            <dim>125</dim>
+            <dim>20</dim>
+            <dim>10</dim>
+            <dim>15</dim>
+        </port>
+        <port id="3">     <!-- value [1] -->
+            <dim>1</dim>
+        </port>
+    </input>
+    <output>
+        <port id="4" precision="FP32">
+            <dim>1000</dim>
+            <dim>256</dim>
+            <dim>10</dim>
+            <dim>15</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/ShuffleChannels_1.md

@@ -0,0 +1,71 @@
+## ShuffleChannels <a name="ShuffleChannels"></a>
+
+**Versioned name**: *ShuffleChannels-1*
+
+**Name**: *ShuffleChannels*
+
+**Category**: Data movement
+
+**Short description**: *ShuffleChannels* permutes data in the channel dimension of the input tensor.
+
+**Attributes**:
+
+* *axis*
+
+  * **Description**: *axis* specifies the index of a channel dimension.
+  * **Range of values**: an integer number in the range [-4, 3]
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *No*
+
+* *group*
+
+  * **Description**: *group* specifies the number of groups to split the channel dimension into. This number must evenly divide the channel dimension size.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *No*
+
+**Inputs**:
+
+*   **1**: 4D input tensor of any supported data type. Required.
+
+**Outputs**:
+
+*   **1**: 4D input tensor with shape and element type as for the input tensor.
+
+**Mathematical Formulation**
+
+The operation is the equivalent with the following transformation of the input tensor *x* of shape *[N, C, H, W]*:
+
+```
+x' = reshape(x, [N, group, C / group, H * W])
+x'' = transpose(x', [0, 2, 1, 3])
+y = reshape(x'', [N, C, H, W])
+```
+
+where `group` is the layer parameter described above and the `axis = 1`.
+
+**Example**
+
+```xml
+<layer ... type="ShuffleChannels" ...>
+    <data group="3" axis="1"/>
+    <input>
+        <port id="0">
+            <dim>5</dim>
+            <dim>12</dim>
+            <dim>200</dim>
+            <dim>400</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>5</dim>
+            <dim>12</dim>
+            <dim>200</dim>
+            <dim>400</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/SpaceToBatch_2.md

@@ -0,0 +1,78 @@
+## SpaceToBatch <a name="SpaceToBatch"></a>
+
+**Versioned name**: *SpaceToBatch-2*
+
+**Category**: *Data movement*
+
+**Short description**: The *SpaceToBatch* operation divides "spatial" dimensions `[1, ..., N - 1]` of the `data` input into a grid of blocks of shape `block_shape`, and interleaves these blocks with the batch dimension (0) such that in the output, the spatial dimensions `[1, ..., N - 1]` correspond to the position within the grid, and the batch dimension combines both the position within a spatial block and the original batch position. Prior to division into blocks, the spatial dimensions of the input are optionally zero padded according to `pads_begin` and `pads_end`. 
+
+**Detailed description**:
+
+The *SpaceToBatch* operation is similar to the TensorFlow* operation [SpaceToBatchND](https://www.tensorflow.org/api_docs/python/tf/space_to_batch_nd)
+
+The operation is equivalent to the following transformation of the input tensor `data` of shape `[batch, D_1, D_2 ... D_{N - 1}]` and `block_shape`, `pads_begin`, `pads_end` of shapes `[N]` to *Y* output tensor.
+
+    Zero-pad the start and end of dimensions [D_0, ..., D_{N - 1}] of the input according to `pads_begin` and `pads_end`:
+    note: P_0 for batch dimension is expected to be 0 (no-padding).
+    x = [batch + P_0, D_1 + P_1, D_2 + P_2, ..., D_{N - 1} + P_{N - 1}], where P_i = pads_begin[i] + pads_end[i]
+
+    note: B_0 for batch is ignored.
+    x' = reshape(x, [batch, (D_1 + P_1) / B_1, B_1, (D_2 + P_2) / B_2, B_2, ..., (D_{N - 1} + P_{N - 1}) / B_{N - 1}, B_{N - 1}]), where B_i = block_shape[i]
+
+    x'' = transpose(x',  [2, 4, ..., (N - 1) + (N - 1), 0, 1, 3, ..., N + (N - 1)])
+
+    y = reshape(x'', [batch * B_1 * ... * B_{N - 1}, (D_1 + P_1) / B_1, (D_2 + P_2) / B_2, ... , (D_{N - 1} + P_{N - 1}) / B_{N - 1}])
+
+**Attributes**
+
+    No attributes available.
+
+**Inputs**
+
+*   **1**: `data` - input N-D tensor `[batch, D_1, D_2 ... D_{N - 1}]` of *T1* type with rank >= 2. **Required.**
+*   **2**: `block_shape` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be >= 1.  `block_shape[0]` is expected to be 1. **Required.**
+*   **3**: `pads_begin` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be non-negative. `pads_begin` specifies the padding for the beginning along each axis of `data` input . It is required that `block_shape[i]` divides `data_shape[i] + pads_begin[i] + pads_end[i]`. `pads_begin[0]` is expected to be 0. **Required.**
+*   **4**: `pads_end` - input 1-D tensor of *T2* type with shape `[N]` that is equal to the size of `data` input shape. All values must be non-negative. `pads_end` specifies the padding for the ending along each axis of `data` input. It is required that `block_shape[i]` divides `data_shape[i] + pads_begin[i] + pads_end[i]`. `pads_end[0]` is expected to be 0. **Required.**
+ 
+**Outputs**
+
+*   **1**: N-D tensor with shape `[batch * block_shape[0] * block_shape[1] * ... * block_shape[N - 1], (pads_begin[1] + D_1 + pads_end[1]) / block_shape[1], (pads_begin[2] + D_2 + pads_end[2]) / block_shape[2], ..., (pads_begin[N - 1] + D_{N - 1} + pads_end[N - 1]) / block_shape[N - 1]` of the same type as `data` input. 
+
+**Types**
+
+* *T1*: any supported type.
+* *T2*: any supported integer type.
+
+**Example**
+
+```xml
+<layer type="SpaceToBatch" ...>
+    <input>
+        <port id="0">       <!-- data -->
+            <dim>2</dim>    <!-- batch -->
+            <dim>6</dim>    <!-- spatial dimension 1 -->
+            <dim>10</dim>   <!-- spatial dimension 2 -->
+            <dim>3</dim>    <!-- spatial dimension 3 -->
+            <dim>3</dim>    <!-- spatial dimension 4 -->
+        </port> 
+        <port id="1">       <!-- block_shape value: [1, 2, 4, 3, 1] -->
+            <dim>5</dim>  
+        </port>
+        <port id="2">       <!-- pads_begin value: [0, 0, 1, 0, 0] -->
+            <dim>5</dim>
+        </port>
+        <port id="3">       <!-- pads_end value: [0, 0, 1, 0, 0] -->
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="3">
+            <dim>48</dim>   <!-- data.shape[0] * block_shape.shape[0] * block_shape.shape[1] *... * block_shape.shape[4] -->
+            <dim>3</dim>    <!-- (data.shape[1] + pads_begin[1] + pads_end[1]) / block_shape.shape[1]  -->
+            <dim>3</dim>    <!-- (data.shape[2] + pads_begin[2] + pads_end[2]) / block_shape.shape[2] -->
+            <dim>1</dim>    <!-- (data.shape[3] + pads_begin[3] + pads_end[3]) / block_shape.shape[3] -->
+            <dim>3</dim>    <!-- (data.shape[4] + pads_begin[4] + pads_end[4]) / block_shape.shape[4] -->
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/SpaceToDepth_1.md

@@ -0,0 +1,80 @@
+## SpaceToDepth <a name="SpaceToDepth"></a>
+
+**Versioned name**: *SpaceToDepth-1*
+
+**Category**: *Data movement*
+
+**Short description**: *SpaceToDepth* operation rearranges data from the spatial dimensions of the input tensor into depth dimension of the output tensor.
+
+**Attributes**
+
+* *block_size*
+
+  * **Description**: *block_size* specifies the size of the value block to be moved. The depth dimension size must be evenly divided by `block_size ^ (len(input.shape) - 2)`.
+  * **Range of values**: a positive integer
+  * **Type**: `int`
+  * **Default value**: 1
+  * **Required**: *no*
+
+* *mode*
+
+  * **Description**: specifies how the output depth dimension is gathered from block coordinates and the old depth dimension.
+  * **Range of values**:
+    * *blocks_first*: the output depth is gathered from `[block_size, ..., block_size,  C]`
+    * *depth_first*: the output depth is gathered from `[C, block_size, ..., block_size]`
+  * **Type**: `string`
+  * **Default value**: None
+  * **Required**: *yes*
+
+**Inputs**
+
+*   **1**: `data` - input tensor of any type with rank >= 3. Required.
+
+**Outputs**
+
+*   **1**: permuted tensor with shape `[N, C * (block_size ^ K), D1 / block_size, D2 / block_size, ..., DK / block_size]`.
+
+**Detailed description**
+
+*SpaceToDepth* operation permutes element from the input tensor with shape `[N, C, D1, D2, ..., DK]`, to the output tensor where values from the input spatial dimensions `D1, D2, ..., DK` are moved to the new depth dimension. Refer to the [ONNX* specification](https://github.com/onnx/onnx/blob/master/docs/Operators.md#SpaceToDepth) for an example of the 4D input tensor case.
+
+The operation is equivalent to the following transformation of the input tensor `data` with `K` spatial dimensions of shape `[N, C, D1, D2, ..., DK]` to *Y* output tensor. If `mode = blocks_first`:
+
+    x' = reshape(data, [N, C, D1/block_size, block_size, D2/block_size, block_size, ... , DK/block_size, block_size])
+
+    x'' = transpose(x',  [0,  3, 5, ..., K + (K + 1), 1,  2, 4, ..., K + K])
+
+    y = reshape(x'', [N, C * (block_size ^ K), D1 / block_size, D2 / block_size, ... , DK / block_size])
+
+If `mode = depth_first`:
+
+    x' = reshape(data, [N, C, D1/block_size, block_size, D2/block_size, block_size, ..., DK/block_size, block_size])
+
+    x'' = transpose(x', [0,  1, 3, 5, ..., K + (K + 1),  2, 4, ..., K + K])
+
+    y = reshape(x'', [N, C * (block_size ^ K), D1 / block_size, D2 / block_size, ..., DK / block_size])
+
+
+**Example**
+
+```xml
+<layer type="SpaceToDepth" ...>
+    <data block_size="2" mode="blocks_first"/>
+    <input>
+        <port id="0">
+            <dim>5</dim>
+            <dim>7</dim>
+            <dim>4</dim>
+            <dim>6</dim>
+        </port>
+    </input>
+    <output>
+        <port id="1">
+            <dim>5</dim>    <!-- data.shape[0] -->
+            <dim>28</dim>   <!-- data.shape[1] * (block_size ^ 2) -->
+            <dim>2</dim>    <!-- data.shape[2] / block_size -->
+            <dim>3</dim>    <!-- data.shape[3] / block_size -->
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Split_1.md

@@ -0,0 +1,80 @@
+## Split <a name="Split"></a>
+
+**Versioned name**: *Split-1*
+
+**Category**: *Data movement operations*
+
+**Short description**: *Split* operation splits an input tensor into pieces of the same length along some axis.
+
+**Attributes**
+
+* *num_splits*
+
+  * **Description**: it specifies the number of outputs into which the initial "*data*" tensor will be split along *"axis"*
+  * **Range of values**: a positive integer less than or equal to the size of the dimension being split over
+  * **Type**: any integer type
+  * **Default value**: None
+  * **Required**: *Yes*
+
+**Inputs**
+
+* **1**: "data" - A tensor of type T1. **Required.**
+
+* **2**: "axis" - axis along *"data"* to split. A scalar of type T2 with value from range `-rank(data) .. rank(data)-1`. Negative values address dimensions from the end. **Required.**
+
+**Outputs**
+
+* **Multiple outputs**: Tensors of the same type as the 1st input tensor. The shape of the i-th output has the same shape as the *"data"* except along dimension *"axis"* where the size is `data.shape[i]/num_splits`.
+
+**Detailed Description**
+
+*Split* operation splits the *"data"* input tensor into pieces of the same length along *"axis"*. The i-th shape of output tensor will be equal to the *"data"* shape except along dimension *"axis"* where the shape will be `data.shape[i]/num_splits`. The sum of elements of split_lengths must match `data.shape[axis]`.
+
+Shape of output tensor will be:
+\f[
+shape_output_tensor = shape_input_tensor[shape_input_tensor[0], shape_input_tensor[1], ... ,split_lengths[axis], ... shape_input_tensor[D-1]], where D rank of input tensor.
+\f]
+
+
+**Types**
+
+* *T1*: arbitrary supported type.
+* *T2*: any integer type.
+
+**Example**
+
+```xml
+<layer id="1" type="Split" ...>
+    <data num_splits="3" />
+    <input>
+        <port id="0">       <!-- some data -->
+            <dim>6</dim>
+            <dim>12</dim>
+            <dim>10</dim>
+            <dim>24</dim>
+        </port>
+        <port id="1">       <!-- axis: 1 -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>6</dim>
+            <dim>4</dim>
+            <dim>10</dim>
+            <dim>24</dim>
+        </port>
+        <port id="3">
+            <dim>6</dim>
+            <dim>4</dim>
+            <dim>10</dim>
+            <dim>24</dim>
+        </port>
+        <port id="4">
+            <dim>6</dim>
+            <dim>4</dim>
+            <dim>10</dim>
+            <dim>24</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/StridedSlice_1.md

@@ -0,0 +1,99 @@
+## StridedSlice <a name="StridedSlice"></a>
+
+**Versioned name**: *StridedSlice-1*
+
+**Category**: Data movement operation
+
+**Short description**: *StridedSlice* extracts a strided slice of a tensor.
+ It is similar to generalized array indexing in Python\*.
+
+**Attributes**
+
+*   *begin_mask*
+
+    * **Description**: *begin_mask* is a bit mask. *begin_mask[i]* equal to 1 means that the corresponding dimension of the `begin` input is ignored and the 'real' beginning of the tensor is used along corresponding dimension.
+    * **Range of values**: a list of `0`s and `1`s
+    * **Type**: `int[]`
+    * **Default value**: None
+    * **Required**: *yes*
+
+*   *end_mask*
+
+    * **Description**: *end_mask* is a bit mask. If *end_mask[i]* is 1, the corresponding dimension of the `end` input is ignored and the real 'end' of the tensor is used along corresponding dimension.
+    * **Range of values**: a list of `0`s and `1`s
+    * **Type**: `int[]`
+    * **Default value**: None
+    * **Required**: *yes*
+
+*   *new_axis_mask*
+
+    * **Description**: *new_axis_mask* is a bit mask. If *new_axis_mask[i]* is 1, a length 1 dimension is inserted on the `i`-th position of input tensor.
+    * **Range of values**: a list of `0`s and `1`s
+    * **Type**: `int[]`
+    * **Default value**: `[0]`
+    * **Required**: *no*
+
+*   *shrink_axis_mask*
+
+    * **Description**: *shrink_axis_mask* is a bit mask. If *shrink_axis_mask[i]* is 1, the dimension on the `i`-th position is deleted.
+    * **Range of values**: a list of `0`s and `1`s
+    * **Type**: `int[]`
+    * **Default value**: `[0]`
+    * **Required**: *no*
+
+*   *ellipsis_mask*
+
+    * **Description**: *ellipsis_mask* is a bit mask. It inserts missing dimensions on a position of a non-zero bit.
+    * **Range of values**: a list of `0`s and `1`. Only one non-zero bit is allowed.
+    * **Type**: `int[]`
+    * **Default value**: `[0]`
+    * **Required**: *no*
+
+**Inputs**:
+
+*   **1**: Multidimensional input tensor to be sliced. Required.
+
+*   **2**: `begin` input - 1D input tensor with begin indexes for input tensor slicing. Required.
+           Out-of-bounds values are silently clamped. If `begin_mask[i]` is 1, the value of `begin[i]` is ignored
+           and the range of the appropriate dimension starts from 0.
+           Negative values mean indexing starts from the end. For example, if `foo=[1,2,3]`, `begin[0]=-1` means `begin[0]=3`.
+
+*   **3**: `end` input - 1D input tensor with end indexes for input tensor slicing. Required.
+           Out-of-bounds values will be silently clamped. If `end_mask[i]` is 1, the value of `end[i]` is ignored
+           and the full range of the appropriate dimension is used instead.
+           Negative values mean indexing starts from the end. For example, if `foo=[1,2,3]`, `end[0]=-1` means `end[0]=3`.
+
+*   **4**: `stride` input - 1D input tensor with strides. Optional.
+
+**Example**
+```xml
+<layer ... type="StridedSlice" ...>
+    <data begin_mask="1,0,1,1,1" ellipsis_mask="0,0,0,0,0" end_mask="1,0,1,1,1" new_axis_mask="0,0,0,0,0" shrink_axis_mask="0,1,0,0,0"/>
+    <input>
+        <port id="0">
+            <dim>1</dim>
+            <dim>2</dim>
+            <dim>384</dim>
+            <dim>640</dim>
+            <dim>8</dim>
+        </port>
+        <port id="1">
+            <dim>5</dim>
+        </port>
+        <port id="2">
+            <dim>5</dim>
+        </port>
+        <port id="3">
+            <dim>5</dim>
+        </port>
+    </input>
+    <output>
+        <port id="4">
+            <dim>1</dim>
+            <dim>384</dim>
+            <dim>640</dim>
+            <dim>8</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Tile_1.md

@@ -0,0 +1,110 @@
+## Tile<a name="Tile"></a>
+
+**Versioned name**: *Tile-1*
+
+**Category**: Data movement
+
+**Short description**: *Tile* operation repeats an input tensor *"data"* the number of times given by *"repeats"* input tensor along each dimension. 
+* If number of elements in *"repeats"* is more than shape of *"data"*, then *"data"* will be promoted to "*repeats*" by prepending new axes, e.g. let's shape of *"data"* is equal to (2, 3) and *"repeats"* is equal to [2, 2, 2], then shape of *"data"* will be promoted to (1, 2, 3) and result shape will be (2, 4, 6).
+* If number of elements in *"repeats"* is less than shape of *"data"*, then *"repeats"* will be promoted to "*data*" by prepending 1's to it, e.g. let's shape of *"data"* is equal to (4, 2, 3) and *"repeats"* is equal to [2, 2], then *"repeats"* will be promoted to [1, 2, 2] and result shape will be (4, 4, 6)
+
+**Attributes**:
+
+No attributes available.
+
+**Inputs**:
+
+* **1**: "data" - an input tensor to be padded. A tensor of type T1. **Required.**
+* **2**: "repeats" - a per-dimension replication factor. For example, *repeats* equal to 88 means that the output tensor gets 88 copies of data from the specified axis. A tensor of type T2. **Required.**
+
+**Outputs**:
+
+* **1**: The count of dimensions in result shape will be equal to the maximum from count of dimensions in "data" shape and number of elements in "repeats". A tensor with type matching 1st tensor. 
+
+**Types**
+
+* *T1*: arbitrary supported type.
+* *T2*: any integer type.
+
+**Detailed description**:
+
+*Tile* operation extends input tensor and filling in output tensor by the following rules:
+
+\f[out_i=input_i[inner_dim*t]\f] \f[ t \in \left ( 0, \quad tiles \right ) \f]
+
+**Examples**
+
+*Example 1: number elements in "repeats" is equal to shape of data*
+
+```xml
+<layer ... type="Tile">
+    <input>
+        <port id="0">
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+        <port id="1">
+            <dim>3</dim>  <!-- [1, 2, 3] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>2</dim>
+            <dim>6</dim>
+            <dim>12</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: number of elements in "repeats" is more than shape of "data"*
+
+```xml
+<layer ... type="Tile">
+    <input>
+        <port id="0">  <!-- will be promoted to shape (1, 2, 3, 4) -->
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+        <port id="1">
+            <dim>4</dim>  <!-- [5, 1, 2, 3] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>5/dim>
+            <dim>2</dim>
+            <dim>6</dim>
+            <dim>12</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 3: number of elements in "repeats" is less than shape of "data"*
+
+```xml
+<layer ... type="Tile">
+    <input>
+        <port id="0">
+            <dim>5</dim>       
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+        <port id="1">
+            <dim>3</dim>  <!-- [1, 2, 3] will be promoted to [1, 1, 2, 3] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>5</dim>  
+            <dim>2</dim>
+            <dim>6</dim>
+            <dim>12</dim>
+        </port>
+    </output>
+</layer>
+```

docs/ops/movement/Transpose_1.md

@@ -0,0 +1,103 @@
+## Transpose<a name="Transpose"></a>
+
+**Versioned name**: *Transpose-1*
+
+**Category**: Data movement
+
+**Short description**: *Transpose* operation reorders the input tensor dimensions.
+
+**Attributes**:
+
+No attributes available.
+
+**Inputs**:
+
+* **1**: "arg" - the tensor to be transposed. A tensor of type T1. **Required.**
+* **2**: "input_order" - the permutation to apply to the axes of the input shape. Must be a vector of element T2 type, with shape [n], where n is the rank of "arg". The tensor's value must contain every integer in the range [0,n-1]. If an empty list is specified [] then the axes will be inverted. A tensor of type T2. **Required.**
+
+**Outputs**:
+
+*   **1**: A tensor with shape and type matching 1st tensor.
+
+**Types**
+
+* *T1*: arbitrary supported type.
+* *T2*: any integer type.
+
+**Detailed description**:
+
+*Transpose* operation reorders the input tensor dimensions. Source indexes and destination indexes are bound by the formula: 
+\f[
+    output[i(order[0]), i(order[1]), ..., i(order[N-1])] = input[i(0), i(1), ..., i(N-1)], where i(j) in range 0..(input.shape[j]-1).
+\f]
+
+**Examples**
+
+*Example 1*
+
+```xml
+<layer ... type="Transpose">
+    <input>
+        <port id="0">
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+        <port id="1">
+            <dim>3</dim>  <!-- [2, 0, 1] -->
+        </port>
+    </input>
+    <output>
+        <port id="2">
+            <dim>4</dim>
+            <dim>2</dim>
+            <dim>3</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 2: input_order in not specified*
+
+```xml
+<layer ... type="Transpose">
+    <input>
+        <port id="0">
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+    </input>
+    <output>         <!-- input_order = [2, 1, 0] if input_order is not set -->
+        <port id="1">
+            <dim>4</dim>
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+    </output>
+</layer>
+```
+
+*Example 3: input_order = empty_list []*
+
+```xml
+<layer ... type="Transpose">
+    <input>
+        <port id="0">
+            <dim>2</dim>
+            <dim>3</dim>
+            <dim>4</dim>
+        </port>
+        <port id="1">
+            <dim>0</dim> <!-- input_order = [2, 1, 0] if input_order is empty list -->
+        </port>
+    </input>
+    <output>         
+        <port id="2">
+            <dim>4</dim>
+            <dim>3</dim>
+            <dim>2</dim>
+        </port>
+    </output>
+</layer>
+```