IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

MulticlassNms-8 spec. (#5907)

Browse Source 
* MulticlassNms-8 spec.

This Op functionally extends NonMaxSuppression-5, to perform more post-processing phases, and lay out
the detection outputs in the way of PaddlePaddle detection.

* Update MulticlassNMS_8.md

Clarify the meaning of "nms_top_k".

* Update MulticlassNMS_8.md

* Update MulticlassNMS_8.md

* Update MulticlassNMS_8.md

* Update MulticlassNMS_8.md

* Update MulticlassNMS_8.md
This commit is contained in:
cecilia peng
2021-06-18 14:20:00 +08:00
committed by GitHub
parent 94352874a5
commit 73ef2ffb41
2 changed files with 162 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
docs/doxygen/ie_docs.xml
View File

@@ -194,6 +194,7 @@ limitations under the License.
<tab type="user" title="Minimum-1" url="@ref openvino_docs_ops_arithmetic_Minimum_1"/>
<tab type="user" title="Mish-4" url="@ref openvino_docs_ops_activation_Mish_4"/>
<tab type="user" title="Mod-1" url="@ref openvino_docs_ops_arithmetic_Mod_1"/>
<tab type="user" title="MulticlassNonMaxSuppression-8" url="@ref openvino_docs_ops_sort_MulticlassNonMaxSuppression_8"/>
<tab type="user" title="Multiply-1" url="@ref openvino_docs_ops_arithmetic_Multiply_1"/>
<tab type="user" title="Negative-1" url="@ref openvino_docs_ops_arithmetic_Negative_1"/>
<tab type="user" title="NonMaxSuppression-1" url="@ref openvino_docs_ops_sort_NonMaxSuppression_1"/>

161
docs/ops/sort/MulticlassNMS_8.md Normal file
View File

@@ -0,0 +1,161 @@
## MulticlassNonMaxSuppression<a name="MulticlassNonMaxSuppression"></a> {#openvino_docs_ops_sort_MulticlassNonMaxSuppression_8}
**Versioned name**: *MulticlassNonMaxSuppression-8*
**Category**: *Sorting and maximization*
**Short description**: *MulticlassNonMaxSuppression* performs multi-class non-maximum suppression of the boxes with predicted scores.
**Detailed description**: *MulticlassNonMaxSuppression* is a multi-phase operation. It implements non-maximum suppression algorithm as described below:
1. Let `B = [b_0,...,b_n]` be the list of initial detection boxes, `S = [s_0,...,s_N]` be the list of corresponding scores.
2. Let `D = []` be an initial collection of resulting boxes. Let `adaptive_threshold = iou_threshold`.
3. If `B` is empty, go to step 9.
4. Take the box with highest score. Suppose that it is the box `b` with the score `s`.
5. Delete `b` from `B`.
6. If the score `s` is greater than or equal to `score_threshold`, add `b` to `D`, else go to step 9.
7. If `nms_eta < 1` and `adaptive_threshold > 0.5`, update `adaptive_threshold *= nms_eta`.
8. For each input box `b_i` from `B` and the corresponding score `s_i`, set `s_i = 0` when `iou(b, b_i) > adaptive_threshold`, and go to step 3.
9. Return `D`, a collection of the corresponding scores `S`, and the number of elements in `D`.
This algorithm is applied independently to each class of each batch element. The operation feeds at most `nms_top_k` scoring candidate boxes to this algorithm.
The total number of output boxes of each batch element must not exceed `keep_top_k`.
Boxes of `background_class` are skipped and thus eliminated.
**Attributes**:
* *sort_result*
* **Description**: *sort_result* specifies the order of output elements.
* **Range of values**: `class`, `score`, `none`
* *class* - sort selected boxes by class id (ascending).
* *score* - sort selected boxes by score (descending).
* *none* - do not guarantee the order.
* **Type**: `string`
* **Default value**: `none`
* **Required**: *No*
* *sort_result_across_batch*
* **Description**: *sort_result_across_batch* is a flag that specifies whenever it is necessary to sort selected boxes across batches or not.
* **Range of values**: true or false
* *true* - sort selected boxes across batches.
* *false* - do not sort selected boxes across batches (boxes are sorted per batch element).
* **Type**: boolean
* **Default value**: false
* **Required**: *No*
* *output_type*
* **Description**: the tensor type of outputs `selected_indices` and `valid_outputs`.
* **Range of values**: `i64` or `i32`
* **Type**: `string`
* **Default value**: `i64`
* **Required**: *No*
* *iou_threshold*
* **Description**: intersection over union threshold.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Required**: *No*
* *score_threshold*
* **Description**: minimum score to consider box for the processing.
* **Range of values**: a floating-point number
* **Type**: `float`
* **Default value**: `0`
* **Required**: *No*
* *nms_top_k*
* **Description**: maximum number of boxes to be selected per class.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Required**: *No*
* *keep_top_k*
* **Description**: maximum number of boxes to be selected per batch element.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all boxes
* **Required**: *No*
* *background_class*
* **Description**: the background class id.
* **Range of values**: an integer
* **Type**: `int`
* **Default value**: `-1` meaning to keep all classes.
* **Required**: *No*
* *nms_eta*
* **Description**: eta parameter for adaptive NMS.
* **Range of values**: a floating-point number in close range `[0, 1.0]`.
* **Type**: `float`
* **Default value**: `1.0`
* **Required**: *No*
**Inputs**:
* **1**: `boxes` - tensor of type *T* and shape `[num_batches, num_boxes, 4]` with box coordinates. The box coordinates are layout as `[xmin, ymin, xmax, ymax]`. **Required.**
* **2**: `scores` - tensor of type *T* and shape `[num_batches, num_classes, num_boxes]` with box scores. **Required.**
**Outputs**:
* **1**: `selected_outputs` - tensor of type *T_THRESHOLDS* and shape `[number of selected boxes, 6]` containing the selected boxes with score and class as tuples `[class_id, box_score, xmin, ymin, xmax, ymax]`.
* **2**: `selected_indices` - tensor of type *T_IND* and shape `[number of selected boxes, 1]` the selected indices in the flattened `boxes`, which are absolute values cross batches. Therefore possible valid values are in the range `[0, num_batches * num_boxes - 1]`.
* **3**: `selected_num` - 1D tensor of type *T_IND* and shape `[num_batches]` representing the number of selected boxes for each batch element.
When there is no box selected, `selected_num` is filled with `0`. `selected_outputs` is an empty tensor of shape `[0, 6]`, and `selected_indices` is an empty tensor of shape `[0, 1]`.
**Types**
* *T*: floating point type.
* *T_MAX_BOXES*: integer type.
* *T_THRESHOLDS*: floating point type.
* *T_IND*: `int64` or `int32`.
**Example**
```xml
<layer ... type="MulticlassNonMaxSuppression" ... >
<data sort_result="score" output_type="i64"/>
<input>
<port id="0">
<dim>3</dim>
<dim>100</dim>
<dim>4</dim>
</port>
<port id="1">
<dim>3</dim>
<dim>5</dim>
<dim>100</dim>
</port>
</input>
<output>
<port id="5" precision="FP32">
<dim>-1</dim> <!-- "-1" means a undefined dimension calculated during the model inference -->
<dim>6</dim>
</port>
<port id="6" precision="I64">
<dim>-1</dim>
<dim>1</dim>
</port>
<port id="7" precision="I64">
<dim>3</dim>
</port>
</output>
</layer>
```