IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

DOCS shift to rst (#17375)

Browse Source
This commit is contained in:
Sebastian Golebiewski
2023-05-05 10:55:40 +02:00
committed by GitHub
parent 175e169537
commit 963f30a2fe
13 changed files with 1328 additions and 1141 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
docs/ops/activation/Mish_4.md
View File

@@ -1,5 +1,7 @@
# Mish {#openvino_docs_ops_activation_Mish_4}
@sphinxdirective
**Versioned name**: *Mish-4*
**Category**: *Activation function*
@@ -8,23 +10,24 @@
**Detailed description**
*Mish* is a self regularized non-monotonic neural activation function proposed in this [article](https://arxiv.org/abs/1908.08681v2).
*Mish* is a self regularized non-monotonic neural activation function proposed in this `article <https://arxiv.org/abs/1908.08681v2>`__.
*Mish* performs element-wise activation function on a given input tensor, based on the following mathematical formula:
\f[
Mish(x) = x\cdot\tanh\big(SoftPlus(x)\big) = x\cdot\tanh\big(\ln(1+e^{x})\big)
\f]
.. math::
Mish(x) = x\cdot\tanh\big(SoftPlus(x)\big) = x\cdot\tanh\big(\ln(1+e^{x})\big)
**Attributes**: *Mish* operation has no attributes.
**Inputs**:
* **1**: A tensor of type *T* and arbitrary shape. **Required.**
* **1**: A tensor of type *T* and arbitrary shape. **Required.**
**Outputs**:
* **1**: The result of element-wise *Mish* function applied to the input tensor. A tensor of type *T* and the same shape as input tensor.
* **1**: The result of element-wise *Mish* function applied to the input tensor. A tensor of type *T* and the same shape as input tensor.
**Types**
@@ -32,19 +35,23 @@ Mish(x) = x\cdot\tanh\big(SoftPlus(x)\big) = x\cdot\tanh\big(\ln(1+e^{x})\big)
**Example**
```xml
<layer ... type="Mish">
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="3">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
```
.. code-block: cpp
<layer ... type="Mish">
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="3">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
@endsphinxdirective

115
docs/ops/arithmetic/Maximum_1.md
View File

@@ -1,5 +1,7 @@
# Maximum {#openvino_docs_ops_arithmetic_Maximum_1}
@sphinxdirective
**Versioned name**: *Maximum-1*
**Category**: *Arithmetic binary*
@@ -7,13 +9,14 @@
**Short description**: *Maximum* performs element-wise maximum operation with two given tensors applying broadcasting rule specified in the *auto_broadcast* attribute.
**Detailed description**
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to `auto_broadcast` attribute specification. As a second step *Maximum* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to ``auto_broadcast`` attribute specification. As a second step *Maximum* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
After broadcasting *Maximum* does the following with the input tensors *a* and *b*:
\f[
o_{i} = max(a_{i},\ b_{i})
\f]
.. math::
o_{i} = max(a_{i},\ b_{i})
**Attributes**:
@@ -21,8 +24,10 @@ o_{i} = max(a_{i},\ b_{i})
* **Description**: specifies rules used for auto-broadcasting of input tensors.
* **Range of values**:
* *none* - no auto-broadcasting is allowed, all input shapes must match
* *numpy* - numpy broadcasting rules, description is available in [Broadcast Rules For Elementwise Operations](../broadcast_rules.md)
* *numpy* - numpy broadcasting rules, description is available in :doc:`Broadcast Rules For Elementwise Operations <openvino_docs_ops_broadcast_rules>`
* **Type**: string
* **Default value**: "numpy"
* **Required**: *no*
@@ -44,52 +49,58 @@ o_{i} = max(a_{i},\ b_{i})
*Example 1 - no broadcasting*
```xml
<layer ... type="Maximum">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Maximum">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
*Example 2: numpy broadcasting*
```xml
<layer ... type="Maximum">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Maximum">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
@endsphinxdirective

115
docs/ops/arithmetic/Minimum_1.md
View File

@@ -1,5 +1,7 @@
# Minimum {#openvino_docs_ops_arithmetic_Minimum_1}
@sphinxdirective
**Versioned name**: *Minimum-1*
**Category**: *Arithmetic binary*
@@ -7,11 +9,12 @@
**Short description**: *Minimum* performs element-wise minimum operation with two given tensors applying broadcasting rule specified in the *auto_broadcast* attribute.
**Detailed description**
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to `auto_broadcast` attribute specification. As a second step *Minimum* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to ``auto_broadcast`` attribute specification. As a second step *Minimum* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
.. math::
o_{i} = min(a_{i},\ b_{i})
\f[
o_{i} = min(a_{i},\ b_{i})
\f]
**Attributes**:
@@ -19,8 +22,10 @@ o_{i} = min(a_{i},\ b_{i})
* **Description**: specifies rules used for auto-broadcasting of input tensors.
* **Range of values**:
* *none* - no auto-broadcasting is allowed, all input shapes must match
* *numpy* - numpy broadcasting rules, description is available in [Broadcast Rules For Elementwise Operations](../broadcast_rules.md)
* *numpy* - numpy broadcasting rules, description is available in :doc:`Broadcast Rules For Elementwise Operations <openvino_docs_ops_broadcast_rules>`
* **Type**: string
* **Default value**: "numpy"
* **Required**: *no*
@@ -42,52 +47,58 @@ o_{i} = min(a_{i},\ b_{i})
*Example 1 - no broadcasting*
```xml
<layer ... type="Minimum">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Minimum">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
*Example 2: numpy broadcasting*
```xml
<layer ... type="Minimum">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Minimum">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
@endsphinxdirective

116
docs/ops/arithmetic/Mod_1.md
View File

@@ -1,5 +1,7 @@
# Mod {#openvino_docs_ops_arithmetic_Mod_1}
@sphinxdirective
**Versioned name**: *Mod-1*
**Category**: *Arithmetic binary*
@@ -7,13 +9,14 @@
**Short description**: *Mod* performs an element-wise modulo operation with two given tensors applying broadcasting rule specified in the *auto_broadcast* attribute.
**Detailed description**
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to `auto_broadcast` attribute specification. As a second step *Mod* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
As a first step input tensors *a* and *b* are broadcasted if their shapes differ. Broadcasting is performed according to ``auto_broadcast`` attribute specification. As a second step *Mod* operation is computed element-wise on the input tensors *a* and *b* according to the formula below:
\f[
o_{i} = a_{i} \mod b_{i}
\f]
.. math::
*Mod* operation computes a reminder of a truncated division. It is the same behaviour like in C programming language: `truncated(x / y) * y + truncated_mod(x, y) = x`. The sign of the result is equal to a sign of a dividend. The result of division by zero is undefined.
o_{i} = a_{i} \mod b_{i}
*Mod* operation computes a reminder of a truncated division. It is the same behavior like in C programming language: ``truncated(x / y) * y + truncated_mod(x, y) = x``. The sign of the result is equal to a sign of a dividend. The result of division by zero is undefined.
**Attributes**:
@@ -21,8 +24,10 @@ o_{i} = a_{i} \mod b_{i}
* **Description**: specifies rules used for auto-broadcasting of input tensors.
* **Range of values**:
* *none* - no auto-broadcasting is allowed, all input shapes must match
* *numpy* - numpy broadcasting rules, description is available in [Broadcast Rules For Elementwise Operations](../broadcast_rules.md)
* *numpy* - numpy broadcasting rules, description is available in :doc:`Broadcast Rules For Elementwise Operations <openvino_docs_ops_broadcast_rules>`
* **Type**: string
* **Default value**: "numpy"
* **Required**: *no*
@@ -45,52 +50,57 @@ o_{i} = a_{i} \mod b_{i}
*Example 1 - no broadcasting*
```xml
<layer ... type="Mod">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Mod">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
*Example 2: numpy broadcasting*
```xml
<layer ... type="Mod">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Mod">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
@endsphinxdirective

117
docs/ops/arithmetic/Multiply_1.md
View File

@@ -1,5 +1,7 @@
# Multiply {#openvino_docs_ops_arithmetic_Multiply_1}
@sphinxdirective
**Versioned name**: *Multiply-1*
**Category**: *Arithmetic binary*
@@ -7,12 +9,13 @@
**Short description**: *Multiply* performs element-wise multiplication operation with two given tensors applying broadcasting rule specified in the *auto_broacast* attribute.
**Detailed description**
Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and `auto_broadcast` attribute is not `none`. Broadcasting is performed according to `auto_broadcast` value.
Before performing arithmetic operation, input tensors *a* and *b* are broadcasted if their shapes are different and ``auto_broadcast`` attribute is not ``none``. Broadcasting is performed according to ``auto_broadcast`` value.
After broadcasting *Multiply* performs multiplication operation for the input tensors *a* and *b* using the formula below:
\f[
o_{i} = a_{i} \cdot b_{i}
\f]
.. math::
o_{i} = a_{i} \cdot b_{i}
**Attributes**:
@@ -20,9 +23,11 @@ o_{i} = a_{i} \cdot b_{i}
* **Description**: specifies rules used for auto-broadcasting of input tensors.
* **Range of values**:
* *none* - no auto-broadcasting is allowed, all input shapes must match,
* *numpy* - numpy broadcasting rules, description is available in [Broadcast Rules For Elementwise Operations](../broadcast_rules.md),
* *pdpd* - PaddlePaddle-style implicit broadcasting, description is available in [Broadcast Rules For Elementwise Operations](../broadcast_rules.md).
* *numpy* - numpy broadcasting rules, description is available in :doc:`Broadcast Rules For Elementwise Operations <openvino_docs_ops_broadcast_rules>`,
* *pdpd* - PaddlePaddle-style implicit broadcasting, description is available in :doc:`Broadcast Rules For Elementwise Operations <openvino_docs_ops_broadcast_rules>`.
* **Type**: string
* **Default value**: "numpy"
* **Required**: *no*
@@ -45,52 +50,58 @@ o_{i} = a_{i} \cdot b_{i}
*Example 1*
```xml
<layer ... type="Multiply">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Multiply">
<data auto_broadcast="none"/>
<input>
<port id="0">
<dim>256</dim>
<dim>56</dim>
</port>
<port id="1">
<dim>256</dim>
<dim>56</dim>
</port>
</input>
<output>
<port id="2">
<dim>256</dim>
<dim>56</dim>
</port>
</output>
</layer>
*Example 2: broadcast*
```xml
<layer ... type="Multiply">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="Multiply">
<data auto_broadcast="numpy"/>
<input>
<port id="0">
<dim>8</dim>
<dim>1</dim>
<dim>6</dim>
<dim>1</dim>
</port>
<port id="1">
<dim>7</dim>
<dim>1</dim>
<dim>5</dim>
</port>
</input>
<output>
<port id="2">
<dim>8</dim>
<dim>7</dim>
<dim>6</dim>
<dim>5</dim>
</port>
</output>
</layer>
@endsphinxdirective

272
docs/ops/matrix/MatMul_1.md
View File

@@ -1,5 +1,7 @@
# MatMul {#openvino_docs_ops_matrix_MatMul_1}
@sphinxdirective
**Versioned name**: *MatMul-1*
**Category**: *Matrix multiplication*
@@ -8,15 +10,16 @@
**Detailed description**
*MatMul* operation takes two tensors and performs usual matrix-matrix multiplication, matrix-vector multiplication or vector-matrix multiplication depending on argument shapes. Input tensors can have any rank >= 1. Two right-most axes in each tensor are interpreted as matrix rows and columns dimensions while all left-most axes (if present) are interpreted as multi-dimensional batch: [BATCH_DIM_1, BATCH_DIM_2,..., BATCH_DIM_K, ROW_INDEX_DIM, COL_INDEX_DIM]. The operation supports usual broadcast semantics for batch dimensions. It enables multiplication of batch of pairs of matrices in a single shot.
*MatMul* operation takes two tensors and performs usual matrix-matrix multiplication, matrix-vector multiplication or vector-matrix multiplication depending on argument shapes. Input tensors can have any rank >= 1. Two right-most axes in each tensor are interpreted as matrix rows and columns dimensions while all left-most axes (if present) are interpreted as multi-dimensional batch: ``[BATCH_DIM_1, BATCH_DIM_2,..., BATCH_DIM_K, ROW_INDEX_DIM, COL_INDEX_DIM]``. The operation supports usual broadcast semantics for batch dimensions. It enables multiplication of batch of pairs of matrices in a single shot.
Before matrix multiplication, there is an implicit shape alignment for input arguments. It consists of the following steps:
1. Applying transpositions specified by optional `transpose_a` and `transpose_b` attributes. Only the two right-most dimensions are transposed, other dimensions remain the same. Transpose attributes are ignored for 1D tensors.
1. Applying transpositions specified by optional ``transpose_a`` and ``transpose_b`` attributes. Only the two right-most dimensions are transposed, other dimensions remain the same. Transpose attributes are ignored for 1D tensors.
2. One-dimensional tensors unsqueezing is applied for each input independently. The axes inserted in this step are not included in the output shape.
* If rank of the **first** input is equal to 1, it is always unsqueezed to 2D tensor **row vector** (regardless of `transpose_a`) by adding axes with size 1 at ROW_INDEX_DIM, to the **left** of the shape. For example `[S]` will be reshaped to `[1, S]`.
* If rank of the **second** input is equal to 1, it is always unsqueezed to 2D tensor **column vector** (regardless of `transpose_b`) by adding axes with size 1 at COL_INDEX_DIM, to the **right** of the shape. For example `[S]` will be reshaped to `[S, 1]`.
* If rank of the **first** input is equal to 1, it is always unsqueezed to 2D tensor **row vector** (regardless of ``transpose_a``) by adding axes with size 1 at ROW_INDEX_DIM, to the **left** of the shape. For example ``[S]`` will be reshaped to ``[1, S]``.
* If rank of the **second** input is equal to 1, it is always unsqueezed to 2D tensor **column vector** (regardless of ``transpose_b``) by adding axes with size 1 at COL_INDEX_DIM, to the **right** of the shape. For example ``[S]`` will be reshaped to ``[S, 1]``.
3. If ranks of input arguments are different after steps 1 and 2, the tensor with a smaller rank is unsqueezed from the left side of the shape by necessary number of axes to make both shapes of the same rank.
@@ -24,17 +27,17 @@ Before matrix multiplication, there is an implicit shape alignment for input arg
Temporary axes inserted in **step 2** are removed from the final output shape after multiplying.
After vector-matrix multiplication, the temporary axis inserted at ROW_INDEX_DIM is removed. After matrix-vector multiplication, the temporary axis inserted at COL_INDEX_DIM is removed.
Output shape of two 1D tensors multiplication `[S] x [S]` is squeezed to scalar.
Output shape of two 1D tensors multiplication ``[S] x [S]`` is squeezed to scalar.
Output shape inference logic examples (ND here means bigger than 1D):
* 1D x 1D: `[X] x [X] -> [1, X] x [X, 1] -> [1, 1] => []` (scalar)
* 1D x ND: `[X] x [B, ..., X, Y] -> [1, X] x [B, ..., X, Y] -> [B, ..., 1, Y] => [B, ..., Y]`
* ND x 1D: `[B, ..., X, Y] x [Y] -> [B, ..., X, Y] x [Y, 1] -> [B, ..., X, 1] => [B, ..., X]`
* ND x ND: `[B, ..., X, Y] x [B, ..., Y, Z] => [B, ..., X, Z]`
* 1D x 1D: ``[X] x [X] -> [1, X] x [X, 1] -> [1, 1] => []`` (scalar)
* 1D x ND: ``[X] x [B, ..., X, Y] -> [1, X] x [B, ..., X, Y] -> [B, ..., 1, Y] => [B, ..., Y]``
* ND x 1D: ``[B, ..., X, Y] x [Y] -> [B, ..., X, Y] x [Y, 1] -> [B, ..., X, 1] => [B, ..., X]``
* ND x ND: ``[B, ..., X, Y] x [B, ..., Y, Z] => [B, ..., X, Z]``
Two attributes, `transpose_a` and `transpose_b` specify embedded transposition for two right-most dimensions for the first and the second input tensors correspondingly. It implies swapping of ROW_INDEX_DIM and COL_INDEX_DIM in the corresponding input tensor. Batch dimensions and 1D tensors are not affected by these attributes.
Two attributes, ``transpose_a`` and ``transpose_b`` specify embedded transposition for two right-most dimensions for the first and the second input tensors correspondingly. It implies swapping of ROW_INDEX_DIM and COL_INDEX_DIM in the corresponding input tensor. Batch dimensions and 1D tensors are not affected by these attributes.
**Attributes**
@@ -57,13 +60,13 @@ Two attributes, `transpose_a` and `transpose_b` specify embedded transposition f
**Inputs**:
* **1**: Tensor of type *T* with matrices A. Rank >= 1. **Required.**
* **1**: Tensor of type *T* with matrices A. Rank >= 1. **Required.**
* **2**: Tensor of type *T* with matrices B. Rank >= 1. **Required.**
* **2**: Tensor of type *T* with matrices B. Rank >= 1. **Required.**
**Outputs**
* **1**: Tensor of type *T* with results of the multiplication.
* **1**: Tensor of type *T* with results of the multiplication.
**Types**:
@@ -73,135 +76,144 @@ Two attributes, `transpose_a` and `transpose_b` specify embedded transposition f
*Vector-matrix multiplication*
```xml
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
*Matrix-vector multiplication*
```xml
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1000</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1000</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
*Matrix-matrix multiplication (like FullyConnected with batch size 1)*
```xml
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<input>
<port id="0">
<dim>1</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>1</dim>
<dim>1000</dim>
</port>
</output>
</layer>
*Vector-matrix multiplication with embedded transposition of the second matrix*
```xml
<layer ... type="MatMul">
<data transpose_b="true"/>
<input>
<port id="0">
<dim>1024</dim>
</port>
<port id="1">
<dim>1000</dim>
<dim>1024</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<data transpose_b="true"/>
<input>
<port id="0">
<dim>1024</dim>
</port>
<port id="1">
<dim>1000</dim>
<dim>1024</dim>
</port>
</input>
<output>
<port id="2">
<dim>1000</dim>
</port>
</output>
</layer>
*Matrix-matrix multiplication (like FullyConnected with batch size 10)*
```xml
<layer ... type="MatMul">
<input>
<port id="0">
<dim>10</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>10</dim>
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<input>
<port id="0">
<dim>10</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>10</dim>
<dim>1000</dim>
</port>
</output>
</layer>
*Multiplication of batch of 5 matrices by a one matrix with broadcasting*
```xml
<layer ... type="MatMul">
<input>
<port id="0">
<dim>5</dim>
<dim>10</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>5</dim>
<dim>10</dim>
<dim>1000</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatMul">
<input>
<port id="0">
<dim>5</dim>
<dim>10</dim>
<dim>1024</dim>
</port>
<port id="1">
<dim>1024</dim>
<dim>1000</dim>
</port>
</input>
<output>
<port id="2">
<dim>5</dim>
<dim>10</dim>
<dim>1000</dim>
</port>
</output>
</layer>
@endsphinxdirective

180
docs/ops/normalization/MVN_1.md
View File

@@ -1,79 +1,98 @@
# MVN {#openvino_docs_ops_normalization_MVN_1}
@sphinxdirective
**Versioned name**: *MVN-1*
**Category**: *Normalization*
**Short description**: Calculates mean-variance normalization of the input tensor. Supports two normalization techniques: [Instance/Contrast Normalization](https://arxiv.org/abs/1607.08022) and [Layer Normalization](https://arxiv.org/abs/1607.06450).
**Short description**: Calculates mean-variance normalization of the input tensor. Supports two normalization techniques: `Instance/Contrast Normalization <https://arxiv.org/abs/1607.08022>`__ and `Layer Normalization <https://arxiv.org/abs/1607.06450>`__.
**Detailed description**
Based on `across_channels` attribute mean value is calculated using one of formulas below:
Based on ``across_channels`` attribute mean value is calculated using one of formulas below:
1. if `true` mean value is calculated using Layer Normalization:
\f[
\mu_{n} = \frac{\sum_{c}^{C}\sum_{h}^{H}\sum_{w}^{W} i_{nchw}}{C * H * W}
\f]
2. if `false` mean value is calculated using Instance/Contrast Normalization:
\f[
\mu_{nc} = \frac{\sum_{h}^{H}\sum_{w}^{W} i_{nchw}}{H * W}
\f]
1. If ``true`` mean value is calculated using Layer Normalization:
where \f$i_{nchw}\f$ is an input tensor parametrized by \f$n\f$ batches, \f$c\f$ channels and \f$h,w\f$ spatial dimesnions.
.. math::
\mu_{n} = \frac{\sum_{c}^{C}\sum_{h}^{H}\sum_{w}^{W} i_{nchw}}{C * H * W}
2. If ``false`` mean value is calculated using Instance/Contrast Normalization:
.. math::
\mu_{nc} = \frac{\sum_{h}^{H}\sum_{w}^{W} i_{nchw}}{H * W}
where :math:`i_{nchw}` is an input tensor parametrized by :math:`n` batches, math:`c` channels and math:`h,w` spatial dimensions.
If ``reduction_axes`` attribute is provided mean value is calculated based on formula:
.. math::
\mu_{n} = ReduceMean(i_{k}, reduction_axes)
If `reduction_axes` attribute is provided mean value is calculated based on formula:
\f[
\mu_{n} = ReduceMean(i_{k}, reduction_axes)
\f]
Afterwards *MVN* subtracts mean value from the input blob.
If *normalize_variance* is set to `true`, the output blob is divided by variance:
\f[
o_{i}=\frac{o_{i}}{\sqrt {\sum {\sigma_{k}^2}+\epsilon}}
\f]
If *normalize_variance* is set to ``true``, the output blob is divided by variance:
where \f$\sigma_{k}^2\f$ is the variance calculated based on mean value, \f$\epsilon\f$ is a value added to the variance for numerical stability and corresponds to `epsilon` attribute.
.. math::
o_{i}=\frac{o_{i}}{\sqrt {\sum {\sigma_{k}^2}+\epsilon}}
where :math:`\sigma_{k}^2` is the variance calculated based on mean value, :math:`\epsilon` is a value added to the variance for numerical stability and corresponds to ``epsilon`` attribute.
**Attributes**
* *across_channels*
* **Description**: *across_channels* is a flag that specifies whether mean values are shared across channels. If `true` mean values and variance are calculated for each sample across all channels and spatial dimensions (Layer Normalization), otherwise calculation is done for each sample and for each channel across spatial dimensions (Instance/Contrast Normalization).
* **Description**: *across_channels* is a flag that specifies whether mean values are shared across channels. If ``true`` mean values and variance are calculated for each sample across all channels and spatial dimensions (Layer Normalization), otherwise calculation is done for each sample and for each channel across spatial dimensions (Instance/Contrast Normalization).
* **Range of values**:
* `false` - do not share mean values across channels
* `true` - share mean values across channels
* **Type**: `boolean`
* ``false`` - do not share mean values across channels
* ``true`` - share mean values across channels
* **Type**: ``boolean``
* **Required**: *yes*
* *reduction_axes*
* **Description**: 1D tensor of unique elements and type *T_IND* which specifies indices of dimensions in `data` that define normalization slices. Negative value means counting dimensions from the back.
* **Range of values**: allowed range of axes is `[-r; r-1]` where `r = rank(data)`, the order cannot be sorted
* **Type**: `int`
* **Description**: 1D tensor of unique elements and type *T_IND* which specifies indices of dimensions in ``data`` that define normalization slices. Negative value means counting dimensions from the back.
* **Range of values**: allowed range of axes is ``[-r; r-1]`` where ``r = rank(data)``, the order cannot be sorted
* **Type**: ``int``
* **Required**: *yes*
* *normalize_variance*
* **Description**: *normalize_variance* is a flag that specifies whether to perform variance normalization.
* **Range of values**:
* `false` - do not normalize variance
* `true` - normalize variance
* **Type**: `boolean`
* ``false`` - do not normalize variance
* ``true`` - normalize variance
* **Type**: ``boolean``
* **Required**: *yes*
* *eps*
* **Description**: *eps* is the number to be added to the variance to avoid division by zero when normalizing the value. For example, *epsilon* equal to 0.001 means that 0.001 is added to the variance.
* **Range of values**: a positive floating-point number
* **Type**: `double`
* **Type**: ``double``
* **Required**: *yes*
* **Note** Important: it is necessary to use only one of `across_channels` or `reduction_axes` attributes, they cannot be defined together.
*
.. important::
It is necessary to use only one of ``across_channels`` or ``reduction_axes`` attributes, they cannot be defined together.
**Inputs**
* **1**: `data` - input tensor of type *T* and arbitrary shape. **Required.**
* **1**: ``data`` - input tensor of type *T* and arbitrary shape. **Required.**
**Outputs**
@@ -82,54 +101,59 @@ where \f$\sigma_{k}^2\f$ is the variance calculated based on mean value, \f$\eps
**Types**
* *T*: any floating point type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Examples**
*Example: with `across_channels` attribute*
*Example: with* ``across_channels`` *attribute*
```xml
<layer ... type="MVN">
<data across_channels="true" eps="1e-9" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
*Example: with `reduction_axes` attribute*
<layer ... type="MVN">
<data across_channels="true" eps="1e-9" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
*Example: with* ``reduction_axes`` *attribute*
.. code-block:: cpp
<layer ... type="MVN">
<data reduction_axes="2,3" eps="1e-9" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
@endsphinxdirective
```xml
<layer ... type="MVN">
<data reduction_axes="2,3" eps="1e-9" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
```

111
docs/ops/normalization/MVN_6.md
View File

@@ -1,5 +1,7 @@
# MVN {#openvino_docs_ops_normalization_MVN_6}
@sphinxdirective
**Versioned name**: *MVN-6*
**Category**: *Normalization*
@@ -9,20 +11,27 @@
**Detailed description**
*MVN* subtracts mean value from the input blob:
\f[
o_{i} = i_{i} - ReduceMean(i_{k}, axes)
\f]
If *normalize_variance* is set to `true`, the output blob is divided by variance. When normalizing the value, the number `eps` is added to the variance to avoid division by zero. According to the `eps_mode` flag's value, `eps` is added inside or outside the sqrt:
.. math::
o_{i} = i_{i} - ReduceMean(i_{k}, axes)
If *normalize_variance* is set to ``true``, the output blob is divided by variance. When normalizing the value, the number ``eps`` is added to the variance to avoid division by zero. According to the ``eps_mode`` flag's value, ``eps`` is added inside or outside the sqrt:
* If ``eps_mode`` is ``inside_sqrt``:
.. math::
o_{i}=\frac{o_{i}}{\sqrt {\sum {o_{k}^2}+\epsilon}}
* If ``eps_mode`` is ``outside_sqrt``:
.. math::
o_{i}=\frac{o_{i}}{\sqrt {\sum {o_{k}^2}}+\epsilon}
* If `eps_mode` is `inside_sqrt`:
\f[
o_{i}=\frac{o_{i}}{\sqrt {\sum {o_{k}^2}+\epsilon}}
\f]
* If `eps_mode` is `outside_sqrt`:
\f[
o_{i}=\frac{o_{i}}{\sqrt {\sum {o_{k}^2}}+\epsilon}
\f]
**Attributes**
@@ -30,65 +39,73 @@ o_{i}=\frac{o_{i}}{\sqrt {\sum {o_{k}^2}}+\epsilon}
* **Description**: *normalize_variance* is a flag that specifies whether to perform variance normalization.
* **Range of values**:
* `false` - do not normalize variance
* `true` - normalize variance
* **Type**: `boolean`
* ``false`` - do not normalize variance
* ``true`` - normalize variance
* **Type**: ``boolean``
* **Required**: *yes*
* *eps*
* **Description**: *eps* is the number to be added to the variance to avoid division by zero when normalizing the value.
* **Range of values**: a positive floating-point number
* **Type**: `float`
* **Type**: ``float``
* **Required**: *yes*
* *eps_mode*
* **Description**: Choose where to add epsilon.
* **Range of values**:
* `inside_sqrt` - add epsilon inside sqrt
* `outside_sqrt` - add epsilon outside of sqrt
* **Type**: `string`
* ``inside_sqrt`` - add epsilon inside sqrt
* ``outside_sqrt`` - add epsilon outside of sqrt
* **Type**: ``string``
* **Required**: *yes*
**Inputs**
* **1**: `data` - Input tensor to be normalized of type *T* and arbitrary shape. **Required.**
* **1**: ``data`` - Input tensor to be normalized of type *T* and arbitrary shape. **Required.**
* **2**: `axes` - 1D tensor which specifies indices of dimensions in `data` that define normalization slices. Allowed range of axes is `[-r; r-1]` where `r = rank(data)`, the order can be not sorted. Negative value means counting dimensions from the back. Type *T_IND*. **Required.**
* **2**: ``axes`` - 1D tensor which specifies indices of dimensions in ``data`` that define normalization slices. Allowed range of axes is ``[-r; r-1]`` where ``r = rank(data)``, the order can be not sorted. Negative value means counting dimensions from the back. Type *T_IND*. **Required.**
**Outputs**
* **1**: Output tensor of the same shape and type as the `data` input tensor.
* **1**: Output tensor of the same shape and type as the ``data`` input tensor.
**Types**
* *T*: any floating point type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Example**
```xml
<layer ... type="MVN">
<data eps="1e-9" eps_mode="inside_sqrt" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
<port id="1">
<dim>3</dim> <!-- value of [0,2,3] means independent normalization per channels -->
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MVN">
<data eps="1e-9" eps_mode="inside_sqrt" normalize_variance="true"/>
<input>
<port id="0">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
<port id="1">
<dim>3</dim> < !-- value of [0,2,3] means independent normalization per channels -->
</port>
</input>
<output>
<port id="2">
<dim>6</dim>
<dim>12</dim>
<dim>10</dim>
<dim>24</dim>
</port>
</output>
</layer>
@endsphinxdirective

336
docs/ops/pooling/MaxPool_1.md
View File

@@ -1,14 +1,16 @@
# MaxPool {#openvino_docs_ops_pooling_MaxPool_1}
@sphinxdirective
**Versioned name**: *MaxPool-1*
**Category**: *Pooling*
**Short description**: Performs max pooling operation on input.
**Detailed description**: Input shape can be either 3D, 4D or 5D. Max Pooling operation is performed with the respect to input shape from the third dimension to the last dimension. If paddings are used then during the pooling calculation their value are `-inf`. The Max Pooling operation involves sliding a filter over each channel of feature map and downsampling by choosing the biggest value within the region covered by the filter. [Article about max pooling in Convolutional Networks](https://deeplizard.com/learn/video/ZjM_XQa5s6s).
**Detailed description**: Input shape can be either 3D, 4D or 5D. Max Pooling operation is performed with the respect to input shape from the third dimension to the last dimension. If paddings are used then during the pooling calculation their value are ``-inf``. The Max Pooling operation involves sliding a filter over each channel of feature map and downsampling by choosing the biggest value within the region covered by the filter. `Article about max pooling in Convolutional Networks <https://deeplizard.com/learn/video/ZjM_XQa5s6s>`__.
**Attributes**: *Pooling* attributes are specified in the `data` node, which is a child of the layer node.
**Attributes**: *Pooling* attributes are specified in the ``data`` node, which is a child of the layer node.
* *strides*
@@ -44,8 +46,10 @@
* **Description**: *rounding_type* is a type of rounding to be used to compute output shape.
* **Range of values**:
* *ceil*
* *floor*
* **Type**: string
* **Default value**: *floor*
* **Required**: *no*
@@ -53,9 +57,11 @@
* *auto_pad*
* **Description**: *auto_pad* how the padding is calculated. Possible values:
* *explicit*: use explicit padding values from `pads_begin` and `pads_end`.
* *explicit*: use explicit padding values from ``pads_begin`` and ``pads_end``.
* *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value an extra padding is added at the end (at the beginning).
* *valid* - do not use padding.
* **Type**: string
* **Default value**: *explicit*
* **Required**: *no*
@@ -63,188 +69,200 @@
**Inputs**:
* **1**: 3D, 4D or 5D input tensor of type *T*. **Required.**
* **1**: 3D, 4D or 5D input tensor of type *T*. **Required.**
**Outputs**:
* **1**: Input shape can be either `[N, C, H]`, `[N, C, H, W]` or `[N, C, H, W, D]`. Then the corresponding output shape will be `[N, C, H_out]`, `[N, C, H_out, W_out]` or `[N, C, H_out, W_out, D_out]`. Output tensor has the same data type as input tensor.
* **1**: Input shape can be either ``[N, C, H]``, ``[N, C, H, W]`` or ``[N, C, H, W, D]``. Then the corresponding output shape will be ``[N, C, H_out]``, ``[N, C, H_out, W_out]`` or ``[N, C, H_out, W_out, D_out]``. Output tensor has the same data type as input tensor.
**Types**
* *T*: floating-point or integer type.
**Mathematical Formulation**
Output shape calculation based on `auto_pad` and `rounding_type`:
* `auto_pad = explicit` and `rounding_type = floor`
`H_out = floor(H + pads_begin[0] + pads_end[0] - kernel[0] / strides[0]) + 1`
`W_out = floor(W + pads_begin[1] + pads_end[1] - kernel[1] / strides[1]) + 1`
`D_out = floor(D + pads_begin[2] + pads_end[2] - kernel[2] / strides[2]) + 1`
* `auto_pad = valid` and `rounding_type = floor`
`H_out = floor(H - kernel[0] / strides[0]) + 1`
`W_out = floor(W - kernel[1] / strides[1]) + 1`
`D_out = floor(D - kernel[2] / strides[2]) + 1`
Output shape calculation based on ``auto_pad`` and ``rounding_type``:
* `auto_pad = same_upper/same_lower` and `rounding_type = floor`
`H_out = H`
`W_out = W`
`D_out = D`
* ``auto_pad = explicit`` and ``rounding_type = floor``
``H_out = floor(H + pads_begin[0] + pads_end[0] - kernel[0] / strides[0]) + 1``
``W_out = floor(W + pads_begin[1] + pads_end[1] - kernel[1] / strides[1]) + 1``
``D_out = floor(D + pads_begin[2] + pads_end[2] - kernel[2] / strides[2]) + 1``
* `auto_pad = explicit` and `rounding_type = ceil`
`H_out = ceil(H + pads_begin[0] + pads_end[0] - kernel[0] / strides[0]) + 1`
`W_out = ceil(W + pads_begin[1] + pads_end[1] - kernel[1] / strides[1]) + 1`
`D_out = ceil(D + pads_begin[2] + pads_end[2] - kernel[2] / strides[2]) + 1`
* ``auto_pad = valid`` and ``rounding_type = floor``
``H_out = floor(H - kernel[0] / strides[0]) + 1``
``W_out = floor(W - kernel[1] / strides[1]) + 1``
``D_out = floor(D - kernel[2] / strides[2]) + 1``
* `auto_pad = valid` and `rounding_type = ceil`
`H_out = ceil(H - kernel[0] / strides[0]) + 1`
`W_out = ceil(W - kernel[1] / strides[1]) + 1`
`D_out = ceil(D - kernel[2] / strides[2]) + 1`
* ``auto_pad = same_upper/same_lower`` and ``rounding_type = floor``
``H_out = H``
``W_out = W``
``D_out = D``
* `auto_pad = same_upper/same_lower` and `rounding_type = ceil`
`H_out = H`
`W_out = W`
`D_out = D`
* ``auto_pad = explicit`` and ``rounding_type = ceil``
``H_out = ceil(H + pads_begin[0] + pads_end[0] - kernel[0] / strides[0]) + 1``
``W_out = ceil(W + pads_begin[1] + pads_end[1] - kernel[1] / strides[1]) + 1``
``D_out = ceil(D + pads_begin[2] + pads_end[2] - kernel[2] / strides[2]) + 1``
If `H + pads_begin[i] + pads_end[i] - kernel[i]` is not divided by `strides[i]` evenly then the result is rounded with the respect to `rounding_type` attribute.
* ``auto_pad = valid`` and ``rounding_type = ceil``
``H_out = ceil(H - kernel[0] / strides[0]) + 1``
``W_out = ceil(W - kernel[1] / strides[1]) + 1``
``D_out = ceil(D - kernel[2] / strides[2]) + 1``
Example 1 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = explicit`
* ``auto_pad = same_upper/same_lower`` and ``rounding_type = ceil``
``H_out = H``
``W_out = W``
``D_out = D``
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
pads_begin = [1, 1]
pads_end = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
output = [[[[-1, 2, 3, 3],
[4, 5, 5, -6],
[4, 8, 9, 9],
[-7, 8, 9, 9]]]]
```
If ``H + pads_begin[i] + pads_end[i] - kernel[i]`` is not divided by ``strides[i]`` evenly then the result is rounded with the respect to ``rounding_type`` attribute.
Example 2 shows how *MaxPool* operates with 3D input using 1D kernel and `auto_pad = valid`
1. Example 1 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = explicit``
```
input = [[[-1, 2, 3, 5, -7, 9, 1]]]
strides = [1]
kernel = [3]
rounding_type = "floor"
auto_pad = "valid"
output = [[[3, 5, 5, 9, 9]]]
```
.. code-block:: sh
Example 3 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = same_lower`
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
pads_begin = [1, 1]
pads_end = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
output = [[[[-1, 2, 3, 3],
[4, 5, 5, -6],
[4, 8, 9, 9],
[-7, 8, 9, 9]]]]
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_lower"
output = [[[[-1, 2, 3],
[4, 5, 5]
[4, 8, 9]]]]
```
Example 4 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = same_upper`
2. Example 2 shows how *MaxPool* operates with 3D input using 1D kernel and ``auto_pad = valid``
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]],
[[2, -1, 5],
[6, -7, 1],
[8, 2, -3]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_upper"
output = [[[[5, 5, 3],
[8, 9, 9]
[8, 9, 9]],
[[6, 5, 5],
[8, 2, 1],
[8, 2, -3]]]]
```
.. code-block:: sh
Example 5 shows how *MaxPool* operates with 4D input using 2D kernel, `auto_pad = valid` and `rounding_type = ceil`
input = [[[-1, 2, 3, 5, -7, 9, 1]]]
strides = [1]
kernel = [3]
rounding_type = "floor"
auto_pad = "valid"
output = [[[3, 5, 5, 9, 9]]]
3. Example 3 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = same_lower``
.. code-block:: sh
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_lower"
output = [[[[-1, 2, 3],
[4, 5, 5]
[4, 8, 9]]]]
4. Example 4 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = same_upper``
.. code-block:: sh
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]],
[[2, -1, 5],
[6, -7, 1],
[8, 2, -3]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_upper"
output = [[[[5, 5, 3],
[8, 9, 9]
[8, 9, 9]],
[[6, 5, 5],
[8, 2, 1],
[8, 2, -3]]]]
5. Example 5 shows how *MaxPool* operates with 4D input using 2D kernel, ``auto_pad = valid`` and ``rounding_type = ceil``
.. code-block:: sh
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [2, 2]
kernel = [2, 2]
rounding_type = "ceil"
auto_pad = "valid"
output = [[[[5, 3],
[8, 9]]]]
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [2, 2]
kernel = [2, 2]
rounding_type = "ceil"
auto_pad = "valid"
output = [[[[5, 3],
[8, 9]]]]
```
**Examples**
```xml
<layer ... type="MaxPool" ... >
<data auto_pad="same_upper" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</output>
</layer>
.. code-block:: cpp
<layer ... type="MaxPool" ... >
<data auto_pad="explicit" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="same_upper" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="explicit" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="valid" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
</output>
</layer>
@endsphinxdirective
<layer ... type="MaxPool" ... >
<data auto_pad="valid" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
</output>
</layer>
```

524
docs/ops/pooling/MaxPool_8.md
View File

@@ -1,14 +1,16 @@
## MaxPool {#openvino_docs_ops_pooling_MaxPool_8}
@sphinxdirective
**Versioned name**: *MaxPool-8*
**Category**: *Pooling*
**Short description**: Performs the max pooling operation on input.
**Detailed description**: Input shape can be either 3D, 4D, or 5D. The max pooling operation is performed with respect to input shape from the third dimension to the last dimension. If paddings are used, during the pooling calculation their values are `-inf`. The max pooling operation involves sliding a filter over each channel of a feature map and downsampling by choosing the largest value within the region covered by the filter.
**Detailed description**: Input shape can be either 3D, 4D, or 5D. The max pooling operation is performed with respect to input shape from the third dimension to the last dimension. If paddings are used, during the pooling calculation their values are ``-inf``. The max pooling operation involves sliding a filter over each channel of a feature map and downsampling by choosing the largest value within the region covered by the filter.
**Attributes**: *Pooling* attributes are specified in the `data` node, which is a child of the layer node.
**Attributes**: *Pooling* attributes are specified in the ``data`` node, which is a child of the layer node.
* *strides*
@@ -19,11 +21,11 @@
* *dilations*
* **Description**: *dilations* specify the index of the next pixel to select when pooling. If not present, the dilation defaults to 1, meaning the adjacent pixel is chosen. A value of 2 indicates that one pixel is skipped and every other pixel is considered. Dilations specify one value for each spatial axis of the kernel: `(z, y, x)` for 3D poolings and `(y, x)` for 2D poolings.
* **Range of values**: integer values starting from 0
* **Type**: int[]
* **Default value**: `[1,1,...]`
* **Required**: *no*
* **Description**: *dilations* specify the index of the next pixel to select when pooling. If not present, the dilation defaults to 1, meaning the adjacent pixel is chosen. A value of 2 indicates that one pixel is skipped and every other pixel is considered. Dilations specify one value for each spatial axis of the kernel: ``(z, y, x)`` for 3D poolings and ``(y, x)`` for 2D poolings.
* **Range of values**: integer values starting from 0
* **Type**: int[]
* **Default value**: ``[1,1,...]``
* **Required**: *no*
* *pads_begin*
@@ -52,8 +54,10 @@
* **Description**: *rounding_type* is a type of rounding to be used to compute output shape.
* **Range of values**:
* *ceil*
* *floor*
* **Type**: string
* **Default value**: *floor*
* **Required**: *no*
@@ -61,9 +65,11 @@
* *auto_pad*
* **Description**: *auto_pad* how the padding is calculated. Possible values:
* *explicit*: explicit padding values from `pads_begin` and `pads_end` are used.
* *explicit*: explicit padding values from ``pads_begin`` and ``pads_end`` are used.
* *same_upper (same_lower)* the input is padded to match the output size. In case of odd padding value, an extra padding is added at the end (at the beginning).
* *valid* padding is not used.
* **Type**: string
* **Default value**: *explicit*
* **Required**: *no*
@@ -71,290 +77,308 @@
* *index_element_type*
* **Description**: the type of output tensor with indices
* **Range of values**: "i64" or "i32"
* **Type**: string
* **Default value**: "i64"
* **Required**: *No*
* **Description**: the type of output tensor with indices
* **Range of values**: "i64" or "i32"
* **Type**: string
* **Default value**: "i64"
* **Required**: *No*
* *axis*
* **Description**: indicator of the first dimension in the input shape that should be used to calculate the upper bound of allowed index output values. The upper bound is the product of dimensions starting from the one pointed by the 'axis' attribute until the end of the input shape.
* **Range of values**: integer number. Negative value means counting dimension from the end. The range is `[-R, R - 1]`, where `R` is the rank of the input tensor.
* **Type**: int
* **Default value**: 0
* **Required**: *No*
* **Description**: indicator of the first dimension in the input shape that should be used to calculate the upper bound of allowed index output values. The upper bound is the product of dimensions starting from the one pointed by the 'axis' attribute until the end of the input shape.
* **Range of values**: integer number. Negative value means counting dimension from the end. The range is ``[-R, R - 1]``, where ``R`` is the rank of the input tensor.
* **Type**: int
* **Default value**: 0
* **Required**: *No*
**Inputs**:
* **1**: 3D, 4D, or 5D input tensor of type T. Required.
* **1**: 3D, 4D, or 5D input tensor of type T. Required.
**Outputs**:
* **1**: Input shape can be either `[N, C, H]`, `[N, C, H, W]`, or `[N, C, H, W, D]`. The corresponding output shape is `[N, C, H_out]`, `[N, C, H_out, W_out]` or `[N, C, H_out, W_out, D_out]`. Output tensor has the same data type as the input tensor.
* **1**: Input shape can be either ``[N, C, H]``, ``[N, C, H, W]``, or ``[N, C, H, W, D]``. The corresponding output shape is ``[N, C, H_out]``, ``[N, C, H_out, W_out]`` or ``[N, C, H_out, W_out, D_out]``. Output tensor has the same data type as the input tensor.
* **2**: Output tensor of type *T_IND* with indices of values selected by the pooling operation.
Shape of this output matches the first output. The type of this output can be specified using the `index_element_type` attribute.
Shape of this output matches the first output. The type of this output can be specified using the ``index_element_type`` attribute.
Values are computed as indices in a tensor flattened to 1D, not considering padding. Examples for a 5D input tensor:
* When `axis == 0`, the values are in the range `[0, N * C * H * W * D)`.
* When `axis == 2`, the values are in the range `[0, H * W * D)`.
Note: the values of this output can only be calculated correctly if `pads_value` is set to `-infinity`.
* When ``axis == 0``, the values are in the range ``[0, N * C * H * W * D)``.
* When ``axis == 2``, the values are in the range ``[0, H * W * D)``.
.. note::
The values of this output can only be calculated correctly if ``pads_value`` is set to ``-infinity``.
**Types**
* *T*: floating point or integer type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Mathematical Formulation**
Output shape calculation based on `auto_pad` and `rounding_type`:
* `auto_pad = explicit` and `rounding_type = floor`
`H_out = floor((H + pads_begin[0] + pads_end[0] - ((kernel[0] - 1) * dilations[0] + 1)) / strides[0] + 1)`
`W_out = floor((W + pads_begin[1] + pads_end[1] - ((kernel[1] - 1) * dilations[1] + 1)) / strides[1] + 1)`
`D_out = floor((D + pads_begin[2] + pads_end[2] - ((kernel[2] - 1) * dilations[2] + 1)) / strides[2] + 1)`
* `auto_pad = explicit` and `rounding_type = ceil`
`H_out = ceil((H + pads_begin[0] + pads_end[0] - ((kernel[0] - 1) * dilations[0] + 1)) / strides[0] + 1)`
`W_out = ceil((W + pads_begin[1] + pads_end[1] - ((kernel[1] - 1) * dilations[1] + 1)) / strides[1] + 1)`
`D_out = ceil((D + pads_begin[2] + pads_end[2] - ((kernel[2] - 1) * dilations[2] + 1)) / strides[2] + 1)`
Output shape calculation based on ``auto_pad`` and ``rounding_type``:
* `auto_pad = valid`
`H_out = ceil((H - ((kernel[0] - 1) * dilations[0] + 1) + 1) / strides[0])`
`W_out = ceil((W - ((kernel[1] - 1) * dilations[1] + 1) + 1) / strides[1])`
`D_out = ceil((D - ((kernel[2] - 1) * dilations[2] + 1) + 1) / strides[2])`
* ``auto_pad = explicit`` and ``rounding_type = floor``
``H_out = floor((H + pads_begin[0] + pads_end[0] - ((kernel[0] - 1) * dilations[0] + 1)) / strides[0] + 1)``
``W_out = floor((W + pads_begin[1] + pads_end[1] - ((kernel[1] - 1) * dilations[1] + 1)) / strides[1] + 1)``
``D_out = floor((D + pads_begin[2] + pads_end[2] - ((kernel[2] - 1) * dilations[2] + 1)) / strides[2] + 1)``
* `auto_pad = same_upper / same_lower`
`H_out = H`
`W_out = W`
`D_out = D`
* ``auto_pad = explicit`` and ``rounding_type = ceil``
``H_out = ceil((H + pads_begin[0] + pads_end[0] - ((kernel[0] - 1) * dilations[0] + 1)) / strides[0] + 1)``
``W_out = ceil((W + pads_begin[1] + pads_end[1] - ((kernel[1] - 1) * dilations[1] + 1)) / strides[1] + 1)``
``D_out = ceil((D + pads_begin[2] + pads_end[2] - ((kernel[2] - 1) * dilations[2] + 1)) / strides[2] + 1)``
* ``auto_pad = valid``
``H_out = ceil((H - ((kernel[0] - 1) * dilations[0] + 1) + 1) / strides[0])``
``W_out = ceil((W - ((kernel[1] - 1) * dilations[1] + 1) + 1) / strides[1])``
``D_out = ceil((D - ((kernel[2] - 1) * dilations[2] + 1) + 1) / strides[2])``
* ``auto_pad = same_upper / same_lower``
``H_out = H``
``W_out = W``
``D_out = D``
If `H + pads_begin[i] + pads_end[i] - kernel[i]` is not divisible by `strides[i]` evenly, the result is rounded with respect to the `rounding_type` attribute.
If ``H + pads_begin[i] + pads_end[i] - kernel[i]`` is not divisible by ``strides[i]`` evenly, the result is rounded with respect to the ``rounding_type`` attribute.
Example 1 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = explicit`.
1. Example 1 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = explicit``.
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
pads_begin = [1, 1]
pads_end = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
output0 = [[[[-1, 2, 3, 3],
[4, 5, 5, -6],
[4, 8, 9, 9],
[-7, 8, 9, 9]]]]
output1 = [[[[0, 1, 2, 2],
[3, 4, 4, 5],
[3, 7, 8, 8],
[6, 7, 8, 8]]]]
```
.. code-block:: sh
Example 2 shows how *MaxPool* operates with 3D input using 1D kernel and `auto_pad = valid`.
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
pads_begin = [1, 1]
pads_end = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
output0 = [[[[-1, 2, 3, 3],
[4, 5, 5, -6],
[4, 8, 9, 9],
[-7, 8, 9, 9]]]]
output1 = [[[[0, 1, 2, 2],
[3, 4, 4, 5],
[3, 7, 8, 8],
[6, 7, 8, 8]]]]
```
input = [[[-1, 2, 3, 5, -7, 9, 1]]]
strides = [1]
kernel = [3]
rounding_type = "floor"
auto_pad = "valid"
output0 = [[[3, 5, 5, 9, 9]]]
output1 = [[[2, 3, 3, 5, 5]]]
```
Example 3 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = same_lower`.
2. Example 2 shows how *MaxPool* operates with 3D input using 1D kernel and ``auto_pad = valid``.
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_lower"
output0 = [[[[-1, 2, 3],
[4, 5, 5]
[4, 8, 9]]]]
output1 = [[[[0, 1, 2],
[3, 4, 4]
[3, 7, 8]]]]
```
.. code-block:: sh
Example 4 shows how *MaxPool* operates with 4D input using 2D kernel and `auto_pad = same_upper`.
input = [[[-1, 2, 3, 5, -7, 9, 1]]]
strides = [1]
kernel = [3]
rounding_type = "floor"
auto_pad = "valid"
output0 = [[[3, 5, 5, 9, 9]]]
output1 = [[[2, 3, 3, 5, 5]]]
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]],
[[2, -1, 5],
[6, -7, 1],
[8, 2, -3]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_upper"
output0 = [[[[5, 5, 3],
[8, 9, 9]
[8, 9, 9]],
[[6, 5, 5],
[8, 2, 1],
[8, 2, -3]]]]
output1 = [[[[4, 4, 2],
[7, 8, 8]
[7, 8, 8]],
[[12, 11, 11],
[15, 16, 14],
[15, 16, 17]]]]
```
Example 5 shows how *MaxPool* operates with 4D input using 2D kernel, `auto_pad = valid` and `rounding_type = ceil`.
3. Example 3 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = same_lower``.
```
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [2, 2]
kernel = [2, 2]
rounding_type = "ceil"
auto_pad = "valid"
output0 = [[[[5, 3],
[8, 9]]]]
output1 = [[[[4, 2],
[7, 8]]]]
```
.. code-block:: sh
Example 6 shows how *MaxPool* operates on 4D input using dilated 2D kernel, `auto_pad = explicit` and `rounding_type = floor`.
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_lower"
output0 = [[[[-1, 2, 3],
[4, 5, 5]
[4, 8, 9]]]]
output1 = [[[[0, 1, 2],
[3, 4, 4]
[3, 7, 8]]]]
```
input = [[[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
dilations = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
pads_begin = [1, 1]
pads_end = [1, 1]
output0 = [[[[5, 6, 5],
[8, 9, 8],
[5, 6, 5]]]]
output1 = [[[[4, 5, 4],
[7, 8, 7],
[4, 5, 4]]]]
```
Example 7 shows how *MaxPool* operates on 4D input using 2D kernel, with non-default `axis` value.
4. Example 4 shows how *MaxPool* operates with 4D input using 2D kernel and ``auto_pad = same_upper``.
.. code-block:: sh
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]],
[[2, -1, 5],
[6, -7, 1],
[8, 2, -3]]]]
strides = [1, 1]
kernel = [2, 2]
rounding_type = "floor"
auto_pad = "same_upper"
output0 = [[[[5, 5, 3],
[8, 9, 9]
[8, 9, 9]],
[[6, 5, 5],
[8, 2, 1],
[8, 2, -3]]]]
output1 = [[[[4, 4, 2],
[7, 8, 8]
[7, 8, 8]],
[[12, 11, 11],
[15, 16, 14],
[15, 16, 17]]]]
5. Example 5 shows how *MaxPool* operates with 4D input using 2D kernel, ``auto_pad = valid`` and ``rounding_type = ceil``.
.. code-block:: sh
input = [[[[-1, 2, 3],
[4, 5, -6],
[-7, 8, 9]]]]
strides = [2, 2]
kernel = [2, 2]
rounding_type = "ceil"
auto_pad = "valid"
output0 = [[[[5, 3],
[8, 9]]]]
output1 = [[[[4, 2],
[7, 8]]]]
6. Example 6 shows how *MaxPool* operates on 4D input using dilated 2D kernel, ``auto_pad = explicit`` and ``rounding_type = floor``.
.. code-block:: sh
input = [[[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]]]]
strides = [1, 1]
kernel = [2, 2]
dilations = [2, 2]
rounding_type = "floor"
auto_pad = "explicit"
pads_begin = [1, 1]
pads_end = [1, 1]
output0 = [[[[5, 6, 5],
[8, 9, 8],
[5, 6, 5]]]]
output1 = [[[[4, 5, 4],
[7, 8, 7],
[4, 5, 4]]]]
7. Example 7 shows how *MaxPool* operates on 4D input using 2D kernel, with non-default ``axis`` value.
.. code-block:: sh
input = [[[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]],
[[10, 11, 12],
[13, 14, 15],
[16, 17, 18]]
]]
strides = [1, 1]
kernel = [2, 2]
dilations = [1, 1]
rounding_type = "floor"
auto_pad = "explicit"
pads_begin = [0, 0]
pads_end = [0, 0]
axis = 2
output0 = [[[[5, 6],
[8, 9]],
[[14, 15],
[17, 18]]]]
output1 = [[[[4, 5],
[7, 8]],
[[4, 5],
[7, 8]]]]
```
input = [[[[1, 2, 3],
[4, 5, 6],
[7, 8, 9]],
[[10, 11, 12],
[13, 14, 15],
[16, 17, 18]]
]]
strides = [1, 1]
kernel = [2, 2]
dilations = [1, 1]
rounding_type = "floor"
auto_pad = "explicit"
pads_begin = [0, 0]
pads_end = [0, 0]
axis = 2
output0 = [[[[5, 6],
[8, 9]],
[[14, 15],
[17, 18]]]]
output1 = [[[[4, 5],
[7, 8]],
[[4, 5],
[7, 8]]]]
```
**Examples**
```xml
<layer ... type="MaxPool" ... >
<data auto_pad="same_upper" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</output>
</layer>
.. code-block:: cpp
<layer ... type="MaxPool" ... >
<data auto_pad="explicit" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="same_upper" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="explicit" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>17</dim>
<dim>17</dim>
</port>
</output>
</layer>
<layer ... type="MaxPool" ... >
<data auto_pad="valid" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
</output>
</layer>
@endsphinxdirective
<layer ... type="MaxPool" ... >
<data auto_pad="valid" kernel="2,2" pads_begin="1,1" pads_end="1,1" strides="2,2"/>
<input>
<port id="0">
<dim>1</dim>
<dim>3</dim>
<dim>32</dim>
<dim>32</dim>
</port>
</input>
<output>
<port id="1">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
<port id="2">
<dim>1</dim>
<dim>3</dim>
<dim>16</dim>
<dim>16</dim>
</port>
</output>
</layer>
```

153
docs/ops/sort/MatrixNMS_8.md
View File

@@ -1,5 +1,7 @@
## MatrixNonMaxSuppression {#openvino_docs_ops_sort_MatrixNms_8}
@sphinxdirective
**Versioned name**: *MatrixNonMaxSuppression-8*
**Category**: *Sorting and maximization*
@@ -8,88 +10,95 @@
**Detailed description**: The operation performs the following:
1. Selects candidate bounding boxes with scores higher than `score_threshold`.
2. For each class, selects at most `nms_top_k` candidate boxes.
3. Decays scores of the candidate boxes according to the Matrix NMS algorithm [Wang et al](https://arxiv.org/abs/2003.10152.pdf). This algorithm is applied independently to each class and each batch element. Boxes of `background_class` are skipped and thus eliminated during the process.
4. Selects boxes with the decayed scores higher than `post_threshold`, and selects at most `keep_top_k` scoring candidate boxes per batch element.
1. Selects candidate bounding boxes with scores higher than ``score_threshold``.
2. For each class, selects at most ``nms_top_k`` candidate boxes.
3. Decays scores of the candidate boxes according to the Matrix NMS algorithm `Wang et al <https://arxiv.org/abs/2003.10152.pdf>`__. This algorithm is applied independently to each class and each batch element. Boxes of ``background_class`` are skipped and thus eliminated during the process.
4. Selects boxes with the decayed scores higher than ``post_threshold``, and selects at most ``keep_top_k`` scoring candidate boxes per batch element.
The Matrix NMS algorithm is described below:
1. Sort descending the candidate boxes by score, and compute `n*n` pairwise IOU (IntersectionOverUnion) matrix `X` for the top `n` boxes. Suppose `n` is the number of candidate boxes.
2. Set the lower triangle and diagonal of `X` to 0. Therefore get the upper triangular matrix `X`.
3. Take the column-wise max of `X` to compute a vector `K` of maximum IOU for each candidate box.
4. Repeat element value of `K` along axis 1. Suppose this gets a matrix `X_cmax`.
5. Compute the decay factor: `decay_factor = exp((X_cmax**2 - X**2) * gaussian_sigma)` if `decay_function` is `guassian`, else `decay_factor = (1 - X) / (1 - X_cmax)`.
6. Take the column-wise min of `decay_factor`, and element-wise multiply with scores to decay them.
1. Sort descending the candidate boxes by score, and compute ``n*n`` pairwise IOU (IntersectionOverUnion) matrix ``X`` for the top ``n`` boxes. Suppose ``n`` is the number of candidate boxes.
2. Set the lower triangle and diagonal of ``X`` to 0. Therefore get the upper triangular matrix ``X``.
3. Take the column-wise max of ``X`` to compute a vector ``K`` of maximum IOU for each candidate box.
4. Repeat element value of ``K`` along axis 1. Suppose this gets a matrix ``X_cmax``.
5. Compute the decay factor: ``decay_factor = exp((X_cmax**2 - X**2) * gaussian_sigma)`` if ``decay_function`` is ``guassian``, else ``decay_factor = (1 - X) / (1 - X_cmax)``.
6. Take the column-wise min of ``decay_factor``, and element-wise multiply with scores to decay them.
**Attributes**:
* *sort_result*
* **Description**: *sort_result* specifies the order of output elements.
* **Range of values**: `class`, `score`, `none`
* **Range of values**: ``class``, ``score``, ``none``
* *class* - sort selected boxes by class id (ascending).
* *score* - sort selected boxes by score (descending).
* *none* - do not guarantee the order.
* **Type**: `string`
* **Default value**: `none`
* **Type**: ``string``
* **Default value**: ``none``
* **Required**: *no*
* *sort_result_across_batch*
* **Description**: *sort_result_across_batch* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not.
* **Range of values**: true or false
* *true* - sort selected boxes across batches.
* *false* - do not sort selected boxes across batches (boxes are sorted per batch element).
* **Type**: boolean
* **Default value**: false
* **Required**: *no*
* *output_type*
* **Description**: the tensor type of outputs `selected_indices` and `valid_outputs`.
* **Range of values**: `i64` or `i32`
* **Type**: `string`
* **Default value**: `i64`
* **Description**: the tensor type of outputs ``selected_indices`` and ``valid_outputs``.
* **Range of values**: ``i64`` or ``i32``
* **Type**: ``string``
* **Default value**: ``i64``
* **Required**: *no*
* *score_threshold*
* **Description**: minimum score to consider box for the processing.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
* *nms_top_k*
* **Description**: maximum number of boxes to be selected per class.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *keep_top_k*
* **Description**: maximum number of boxes to be selected per batch element.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *background_class*
* **Description**: the background class id.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all classes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all classes
* **Required**: *no*
* *normalized*
* **Description**: *normalized* is a flag that indicates whether `boxes` are normalized or not.
* **Description**: *normalized* is a flag that indicates whether ``boxes`` are normalized or not.
* **Range of values**: true or false
* *true* - the box coordinates are normalized.
* *false* - the box coordinates are not normalized.
* **Type**: boolean
* **Default value**: True
* **Required**: *no*
@@ -97,78 +106,82 @@ The Matrix NMS algorithm is described below:
* *decay_function*
* **Description**: decay function used to decay scores.
* **Range of values**: `gaussian`, `linear`
* **Type**: `string`
* **Default value**: `linear`
* **Range of values**: ``gaussian``, ``linear``
* **Type**: ``string``
* **Default value**: ``linear``
* **Required**: *no*
* *gaussian_sigma*
* **Description**: gaussian_sigma parameter for gaussian decay_function.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `2.0`
* **Type**: ``float``
* **Default value**: ``2.0``
* **Required**: *no*
* *post_threshold*
* **Description**: threshold to filter out boxes with low confidence score after decaying.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
**Inputs**:
* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. The box cooridnates are layout as `[xmin, ymin, xmax, ymax]`. **Required.**
* **1**: ``boxes`` - tensor of type *T* and shape ``[num_batches, num_boxes, 4]`` with box coordinates. The box coordinates are layout as ``[xmin, ymin, xmax, ymax]``. **Required.**
* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. The tensor type should be same with `boxes`. **Required.**
* **2**: ``scores`` - tensor of type *T* and shape ``[num_batches, num_classes, num_boxes]`` with box scores. The tensor type should be same with ``boxes``. **Required.**
**Outputs**:
* **1**: `selected_outputs` - tensor of type *T* which should be same with `boxes` and shape `[number of selected boxes, 6]` containing the selected boxes with score and class as tuples `[class_id, box_score, xmin, ymin, xmax, ymax]`.
* **1**: ``selected_outputs`` - tensor of type *T* which should be same with ``boxes`` and shape ``[number of selected boxes, 6]`` containing the selected boxes with score and class as tuples ``[class_id, box_score, xmin, ymin, xmax, ymax]``.
* **2**: `selected_indices` - tensor of type *T_IND* and shape `[number of selected boxes, 1]` the selected indices in the flattened input `boxes`, which are absolute values cross batches. Therefore possible valid values are in the range `[0, num_batches * num_boxes - 1]`.
* **2**: ``selected_indices`` - tensor of type *T_IND* and shape ``[number of selected boxes, 1]`` the selected indices in the flattened input ``boxes``, which are absolute values cross batches. Therefore possible valid values are in the range ``[0, num_batches * num_boxes - 1]``.
* **3**: `selected_num` - 1D tensor of type *T_IND* and shape `[num_batches]` representing the number of selected boxes for each batch element.
* **3**: ``selected_num`` - 1D tensor of type *T_IND* and shape ``[num_batches]`` representing the number of selected boxes for each batch element.
When there is no box selected, `selected_num` is filled with `0`. `selected_outputs` is an empty tensor of shape `[0, 6]`, and `selected_indices` is an empty tensor of shape `[0, 1]`.
When there is no box selected, ``selected_num`` is filled with ``0``. ``selected_outputs`` is an empty tensor of shape ``[0, 6]``, and ``selected_indices`` is an empty tensor of shape ``[0, 1]``.
**Types**
* *T*: floating-point type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Example**
```xml
<layer ... type="MatrixNonMaxSuppression" ... >
<data decay_function="guassian" sort_result="score" output_type="i64"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> <!-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MatrixNonMaxSuppression" ... >
<data decay_function="gaussian" sort_result="score" output_type="i64"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> < !-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
@endsphinxdirective

152
docs/ops/sort/MulticlassNonMaxSuppression_8.md
View File

@@ -1,5 +1,7 @@
## MulticlassNonMaxSuppression {#openvino_docs_ops_sort_MulticlassNonMaxSuppression_8}
@sphinxdirective
**Versioned name**: *MulticlassNonMaxSuppression-8*
**Category**: *Sorting and maximization*
@@ -8,97 +10,103 @@
**Detailed description**: *MulticlassNonMaxSuppression* is a multi-phase operation. It implements non-maximum suppression algorithm as described below:
1. Let `B = [b_0,...,b_n]` be the list of initial detection boxes, `S = [s_0,...,s_N]` be the list of corresponding scores.
2. Let `D = []` be an initial collection of resulting boxes. Let `adaptive_threshold = iou_threshold`.
3. If `B` is empty, go to step 9.
4. Take the box with highest score. Suppose that it is the box `b` with the score `s`.
5. Delete `b` from `B`.
6. If the score `s` is greater than or equal to `score_threshold`, add `b` to `D`, else go to step 9.
7. If `nms_eta < 1` and `adaptive_threshold > 0.5`, update `adaptive_threshold *= nms_eta`.
8. For each input box `b_i` from `B` and the corresponding score `s_i`, set `s_i = 0` when `iou(b, b_i) > adaptive_threshold`, and go to step 3.
9. Return `D`, a collection of the corresponding scores `S`, and the number of elements in `D`.
1. Let ``B = [b_0,...,b_n]`` be the list of initial detection boxes, ``S = [s_0,...,s_N]`` be the list of corresponding scores.
2. Let ``D = []`` be an initial collection of resulting boxes. Let ``adaptive_threshold = iou_threshold``.
3. If ``B`` is empty, go to step 9.
4. Take the box with highest score. Suppose that it is the box ``b`` with the score ``s``.
5. Delete ``b`` from ``B``.
6. If the score ``s`` is greater than or equal to ``score_threshold``, add ``b`` to ``D``, else go to step 9.
7. If ``nms_eta < 1`` and ``adaptive_threshold > 0.5``, update ``adaptive_threshold *= nms_eta``.
8. For each input box ``b_i`` from ``B`` and the corresponding score ``s_i``, set ``s_i = 0`` when ``iou(b, b_i) > adaptive_threshold``, and go to step 3.
9. Return ``D``, a collection of the corresponding scores ``S``, and the number of elements in ``D``.
This algorithm is applied independently to each class of each batch element. The operation feeds at most `nms_top_k` scoring candidate boxes to this algorithm.
The total number of output boxes of each batch element must not exceed `keep_top_k`.
Boxes of `background_class` are skipped and thus eliminated.
This algorithm is applied independently to each class of each batch element. The operation feeds at most ``nms_top_k`` scoring candidate boxes to this algorithm.
The total number of output boxes of each batch element must not exceed ``keep_top_k``.
Boxes of ``background_class`` are skipped and thus eliminated.
**Attributes**:
* *sort_result*
* **Description**: *sort_result* specifies the order of output elements.
* **Range of values**: `class`, `score`, `none`
* **Range of values**: ``class``, ``score``, ``none``
* *class* - sort selected boxes by class id (ascending).
* *score* - sort selected boxes by score (descending).
* *none* - do not guarantee the order.
* **Type**: `string`
* **Default value**: `none`
* **Type**: ``string``
* **Default value**: ``none``
* **Required**: *no*
* *sort_result_across_batch*
* **Description**: *sort_result_across_batch* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not.
* **Range of values**: true or false
* *true* - sort selected boxes across batches.
* *false* - do not sort selected boxes across batches (boxes are sorted per batch element).
* **Type**: boolean
* **Default value**: false
* **Required**: *no*
* *output_type*
* **Description**: the tensor type of outputs `selected_indices` and `valid_outputs`.
* **Range of values**: `i64` or `i32`
* **Type**: `string`
* **Default value**: `i64`
* **Description**: the tensor type of outputs ``selected_indices`` and ``valid_outputs``.
* **Range of values**: ``i64`` or ``i32``
* **Type**: ``string``
* **Default value**: ``i64``
* **Required**: *no*
* *iou_threshold*
* **Description**: intersection over union threshold.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
* *score_threshold*
* **Description**: minimum score to consider box for the processing.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
* *nms_top_k*
* **Description**: maximum number of boxes to be selected per class.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *keep_top_k*
* **Description**: maximum number of boxes to be selected per batch element.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *background_class*
* **Description**: the background class id.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all classes.
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all classes.
* **Required**: *no*
* *normalized*
* **Description**: *normalized* is a flag that indicates whether `boxes` are normalized or not.
* **Description**: *normalized* is a flag that indicates whether ``boxes`` are normalized or not.
* **Range of values**: true or false
* *true* - the box coordinates are normalized.
* *false* - the box coordinates are not normalized.
* **Type**: boolean
* **Default value**: True
* **Required**: *no*
@@ -106,62 +114,66 @@ Boxes of `background_class` are skipped and thus eliminated.
* *nms_eta*
* **Description**: eta parameter for adaptive NMS.
* **Range of values**: a floating-point number in close range `[0, 1.0]`.
* **Type**: `float`
* **Default value**: `1.0`
* **Range of values**: a floating-point number in close range ``[0, 1.0]``.
* **Type**: ``float``
* **Default value**: ``1.0``
* **Required**: *no*
**Inputs**:
* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. The box coordinates are layout as `[xmin, ymin, xmax, ymax]`. **Required.**
* **1**: ``boxes`` - tensor of type *T* and shape ``[num_batches, num_boxes, 4]`` with box coordinates. The box coordinates are layout as ``[xmin, ymin, xmax, ymax]``. **Required.**
* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. The tensor type should be same with `boxes`. **Required.**
* **2**: ``scores`` - tensor of type *T* and shape ``[num_batches, num_classes, num_boxes]`` with box scores. The tensor type should be same with ``boxes``. **Required.**
**Outputs**:
* **1**: `selected_outputs` - tensor of type *T* which should be same with `boxes` and shape `[number of selected boxes, 6]` containing the selected boxes with score and class as tuples `[class_id, box_score, xmin, ymin, xmax, ymax]`.
* **1**: ``selected_outputs`` - tensor of type *T* which should be same with ``boxes`` and shape ``[number of selected boxes, 6]`` containing the selected boxes with score and class as tuples ``[class_id, box_score, xmin, ymin, xmax, ymax]``.
* **2**: `selected_indices` - tensor of type *T_IND* and shape `[number of selected boxes, 1]` the selected indices in the flattened `boxes`, which are absolute values cross batches. Therefore possible valid values are in the range `[0, num_batches * num_boxes - 1]`.
* **2**: ``selected_indices`` - tensor of type *T_IND* and shape ``[number of selected boxes, 1]`` the selected indices in the flattened ``boxes``, which are absolute values cross batches. Therefore possible valid values are in the range ``[0, num_batches * num_boxes - 1]``.
* **3**: `selected_num` - 1D tensor of type *T_IND* and shape `[num_batches]` representing the number of selected boxes for each batch element.
* **3**: ``selected_num`` - 1D tensor of type *T_IND* and shape ``[num_batches]`` representing the number of selected boxes for each batch element.
When there is no box selected, `selected_num` is filled with `0`. `selected_outputs` is an empty tensor of shape `[0, 6]`, and `selected_indices` is an empty tensor of shape `[0, 1]`.
When there is no box selected, ``selected_num`` is filled with ``0``. ``selected_outputs`` is an empty tensor of shape ``[0, 6]``, and ``selected_indices`` is an empty tensor of shape ``[0, 1]``.
**Types**
* *T*: floating-point type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Example**
```xml
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> <!-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> < !-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
@endsphinxdirective

227
docs/ops/sort/MulticlassNonMaxSuppression_9.md
View File

@@ -1,5 +1,7 @@
## MulticlassNonMaxSuppression<a name="MulticlassNonMaxSuppression"></a> {#openvino_docs_ops_sort_MulticlassNonMaxSuppression_9}
@sphinxdirective
**Versioned name**: *MulticlassNonMaxSuppression-9*
**Category**: *Sorting and maximization*
@@ -8,97 +10,103 @@
**Detailed description**: *MulticlassNonMaxSuppression* is a multi-phase operation. It implements non-maximum suppression algorithm as described below:
1. Let `B = [b_0,...,b_n]` be the list of initial detection boxes, `S = [s_0,...,s_N]` be the list of corresponding scores.
2. Let `D = []` be an initial collection of resulting boxes. Let `adaptive_threshold = iou_threshold`.
3. If `B` is empty, go to step 9.
4. Take the box with highest score. Suppose that it is the box `b` with the score `s`.
5. Delete `b` from `B`.
6. If the score `s` is greater than or equal to `score_threshold`, add `b` to `D`, else go to step 9.
7. If `nms_eta < 1` and `adaptive_threshold > 0.5`, update `adaptive_threshold *= nms_eta`.
8. For each input box `b_i` from `B` and the corresponding score `s_i`, set `s_i = 0` when `iou(b, b_i) > adaptive_threshold`, and go to step 3.
9. Return `D`, a collection of the corresponding scores `S`, and the number of elements in `D`.
1. Let ``B = [b_0,...,b_n]`` be the list of initial detection boxes, ``S = [s_0,...,s_N]`` be the list of corresponding scores.
2. Let ``D = []`` be an initial collection of resulting boxes. Let ``adaptive_threshold = iou_threshold``.
3. If ``B`` is empty, go to step 9.
4. Take the box with highest score. Suppose that it is the box ``b`` with the score ``s``.
5. Delete ``b`` from ``B``.
6. If the score ``s`` is greater than or equal to ``score_threshold``, add ``b`` to ``D``, else go to step 9.
7. If ``nms_eta < 1`` and ``adaptive_threshold > 0.5``, update ``adaptive_threshold *= nms_eta``.
8. For each input box ``b_i`` from ``B`` and the corresponding score ``s_i``, set ``s_i = 0`` when ``iou(b, b_i) > adaptive_threshold``, and go to step 3.
9. Return ``D``, a collection of the corresponding scores ``S``, and the number of elements in ``D``.
This algorithm is applied independently to each class of each batch element. The operation feeds at most `nms_top_k` scoring candidate boxes to this algorithm.
The total number of output boxes of each batch element must not exceed `keep_top_k`.
Boxes of `background_class` are skipped and thus eliminated.
This algorithm is applied independently to each class of each batch element. The operation feeds at most ``nms_top_k`` scoring candidate boxes to this algorithm.
The total number of output boxes of each batch element must not exceed ``keep_top_k``.
Boxes of ``background_class`` are skipped and thus eliminated.
**Attributes**:
* *sort_result*
* **Description**: *sort_result* specifies the order of output elements.
* **Range of values**: `class`, `score`, `none`
* **Range of values**: ``class``, ``score``, ``none``
* *class* - sort selected boxes by class id (ascending).
* *score* - sort selected boxes by score (descending).
* *none* - do not guarantee the order.
* **Type**: `string`
* **Default value**: `none`
* **Type**: ``string``
* **Default value**: ``none``
* **Required**: *no*
* *sort_result_across_batch*
* **Description**: *sort_result_across_batch* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not.
* **Range of values**: true or false
* *true* - sort selected boxes across batches.
* *false* - do not sort selected boxes across batches (boxes are sorted per batch element).
* **Type**: boolean
* **Default value**: false
* **Required**: *no*
* *output_type*
* **Description**: the tensor type of outputs `selected_indices` and `valid_outputs`.
* **Range of values**: `i64` or `i32`
* **Type**: `string`
* **Default value**: `i64`
* **Description**: the tensor type of outputs ``selected_indices`` and ``valid_outputs``.
* **Range of values**: ``i64`` or ``i32``
* **Type**: ``string``
* **Default value**: ``i64``
* **Required**: *no*
* *iou_threshold*
* **Description**: intersection over union threshold.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
* *score_threshold*
* **Description**: minimum score to consider box for the processing.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Type**: ``float``
* **Default value**: ``0``
* **Required**: *no*
* *nms_top_k*
* **Description**: maximum number of boxes to be selected per class.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *keep_top_k*
* **Description**: maximum number of boxes to be selected per batch element.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all boxes
* **Required**: *no*
* *background_class*
* **Description**: the background class id.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all classes.
* **Type**: ``int``
* **Default value**: ``-1`` meaning to keep all classes.
* **Required**: *no*
* *normalized*
* **Description**: *normalized* is a flag that indicates whether `boxes` are normalized or not.
* **Description**: *normalized* is a flag that indicates whether ``boxes`` are normalized or not.
* **Range of values**: true or false
* *true* - the box coordinates are normalized.
* *false* - the box coordinates are not normalized.
* **Type**: boolean
* **Default value**: True
* **Required**: *no*
@@ -106,103 +114,112 @@ Boxes of `background_class` are skipped and thus eliminated.
* *nms_eta*
* **Description**: eta parameter for adaptive NMS.
* **Range of values**: a floating-point number in close range `[0, 1.0]`.
* **Type**: `float`
* **Default value**: `1.0`
* **Range of values**: a floating-point number in close range ``[0, 1.0]``.
* **Type**: ``float``
* **Default value**: ``1.0``
* **Required**: *no*
**Inputs**:
There are 2 kinds of input formats. The first one is of two inputs. The boxes are shared by all classes.
* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. The box coordinates are layout as `[xmin, ymin, xmax, ymax]`. **Required.**
* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. The tensor type should be same with `boxes`. **Required.**
* **1**: ``boxes`` - tensor of type *T* and shape ``[num_batches, num_boxes, 4]`` with box coordinates. The box coordinates are layout as ``[xmin, ymin, xmax, ymax]``. **Required.**
* **2**: ``scores`` - tensor of type *T* and shape ``[num_batches, num_classes, num_boxes]`` with box scores. The tensor type should be same with ``boxes``. **Required.**
The second format is of three inputs. Each class has its own boxes that are not shared.
* **1**: `boxes` - tensor of type *T* and shape `[num_classes, num_boxes, 4]` with box coordinates. The box coordinates are layout as `[xmin, ymin, xmax, ymax]`. **Required.**
* **1**: ``boxes`` - tensor of type *T* and shape ``[num_classes, num_boxes, 4]`` with box coordinates. The box coordinates are layout as ``[xmin, ymin, xmax, ymax]``. **Required.**
* **2**: `scores` - tensor of type *T* and shape `[num_classes, num_boxes]` with box scores. The tensor type should be same with `boxes`. **Required.**
* **2**: ``scores`` - tensor of type *T* and shape ``[num_classes, num_boxes]`` with box scores. The tensor type should be same with ``boxes``. **Required.**
* **3**: `roisnum` - tensor of type *T_IND* and shape `[num_batches]` with box numbers in each image. `num_batches` is the number of images. Each element in this tensor is the number of boxes for corresponding image. The sum of all elements is `num_boxes`. **Required.**
* **3**: ``roisnum`` - tensor of type *T_IND* and shape ``[num_batches]`` with box numbers in each image. ``num_batches`` is the number of images. Each element in this tensor is the number of boxes for corresponding image. The sum of all elements is ``num_boxes``. **Required.**
**Outputs**:
* **1**: `selected_outputs` - tensor of type *T* which should be same with `boxes` and shape `[number of selected boxes, 6]` containing the selected boxes with score and class as tuples `[class_id, box_score, xmin, ymin, xmax, ymax]`.
* **1**: ``selected_outputs`` - tensor of type *T* which should be same with ``boxes`` and shape ``[number of selected boxes, 6]`` containing the selected boxes with score and class as tuples ``[class_id, box_score, xmin, ymin, xmax, ymax]``.
* **2**: `selected_indices` - tensor of type *T_IND* and shape `[number of selected boxes, 1]` the selected indices in the flattened `boxes`, which are absolute values cross batches. Therefore possible valid values are in the range `[0, num_batches * num_boxes - 1]`.
* **2**: ``selected_indices`` - tensor of type *T_IND* and shape ``[number of selected boxes, 1]`` the selected indices in the flattened ``boxes``, which are absolute values cross batches. Therefore possible valid values are in the range ``[0, num_batches * num_boxes - 1]``.
* **3**: `selected_num` - 1D tensor of type *T_IND* and shape `[num_batches]` representing the number of selected boxes for each batch element.
* **3**: ``selected_num`` - 1D tensor of type *T_IND* and shape ``[num_batches]`` representing the number of selected boxes for each batch element.
When there is no box selected, `selected_num` is filled with `0`. `selected_outputs` is an empty tensor of shape `[0, 6]`, and `selected_indices` is an empty tensor of shape `[0, 1]`.
When there is no box selected, ``selected_num`` is filled with ``0``. ``selected_outputs`` is an empty tensor of shape ``[0, 6]``, and ``selected_indices`` is an empty tensor of shape ``[0, 1]``.
**Types**
* *T*: floating-point type.
* *T_IND*: `int64` or `int32`.
* *T_IND*: ``int64`` or ``int32``.
**Example**
```xml
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64" sort_result_across_batch="false" iou_threshold="0.2" score_threshold="0.5" nms_top_k="-1" keep_top_k="-1" background_class="-1" normalized="false" nms_eta="0.0"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> <!-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64" sort_result_across_batch="false" iou_threshold="0.2" score_threshold="0.5" nms_top_k="-1" keep_top_k="-1" background_class="-1" normalized="false" nms_eta="0.0"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> < !-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
Another possible example with 3 inputs could be like:
```xml
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64" sort_result_across_batch="false" iou_threshold="0.2" score_threshold="0.5" nms_top_k="-1" keep_top_k="-1" background_class="-1" normalized="false" nms_eta="0.0"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>100</dim>
</port>
<port id="2">
<dim>10</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> <!-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
```
.. code-block:: cpp
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64" sort_result_across_batch="false" iou_threshold="0.2" score_threshold="0.5" nms_top_k="-1" keep_top_k="-1" background_class="-1" normalized="false" nms_eta="0.0"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>100</dim>
</port>
<port id="2">
<dim>10</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> < !-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
@endsphinxdirective