IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Mish specification refactoring (#5035)

Browse Source 
* Review spec of Mish operation

* Add minor changes

   * Updated reference paper to a newer version
   * Fix typo in SoftPlus op

* Minor change in example section

* Fix minor wording issues
This commit is contained in:
Gabriele Galiero Casay 2021-04-01 15:08:24 +02:00 committed by GitHub
parent f7863847ad
commit 10d72a6631
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 16 additions and 16 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
docs/ops/activation/Mish_4.md
View File

@ -2,35 +2,35 @@
**Versioned name**: *Mish-4* **Versioned name**: *Mish-4*
**Category**: *Activation* **Category**: *Activation function*
**Short description**: Mish is a Self Regularized Non-Monotonic Neural Activation Function. **Short description**: *Mish* is a Self Regularized Non-Monotonic Neural Activation Function.
**Detailed description**: Mish is a self regularized non-monotonic neural activation function proposed in the [article](https://arxiv.org/abs/1908.08681). **Detailed description**
**Attributes**: operation has no attributes. *Mish* is a self regularized non-monotonic neural activation function proposed in this [article](https://arxiv.org/abs/1908.08681v2).
*Mish* performs element-wise activation function on a given input tensor, based on the following mathematical formula:
\f[
Mish(x) = x\cdot\tanh\big(SoftPlus(x)\big) = x\cdot\tanh\big(\ln(1+e^{x})\big)
\f]
**Attributes**: *Mish* operation has no attributes.
**Inputs**: **Inputs**:
* **1**: Input tensor *x* of any floating point type T. Required. * **1**: A tensor of type `T` and arbitrary shape. **Required**.
**Outputs**: **Outputs**:
* **1**: Floating point tensor with shape and type matching the input tensor. * **1**: The result of element-wise *Mish* function applied to the input tensor. A tensor of type `T` and the same shape as input tensor.
**Types** **Types**
* *T*: any floating point type. * *T*: arbitrary supported floating-point type.
**Mathematical Formulation** **Example**
For each element from the input tensor calculates corresponding
element in the output tensor with the following formula:
\f[
Mish(x) = x*tanh(ln(1.0+e^{x}))
\f]
**Examples**
```xml ```xml
<layer ... type="Mish"> <layer ... type="Mish">