IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

revise GRU cell/sequence (#7901)

Browse Source 
* add typepro/vistor tests

Signed-off-by: fishbell <bell.song@intel.com>

* remove redundant test file

Signed-off-by: fishbell <bell.song@intel.com>

* fix clang format

Signed-off-by: fishbell <bell.song@intel.com>

* typo

Signed-off-by: fishbell <bell.song@intel.com>

* update doc

Signed-off-by: fishbell <bell.song@intel.com>
This commit is contained in:
song, bell
2021-10-21 11:26:53 +03:00
committed by GitHub
parent e0062fc274
commit ff14899a96
5 changed files with 350 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/ops/sequence/GRUCell_3.md
+17 -2
View File
@@ -6,6 +6,21 @@
**Short description**: *GRUCell* represents a single GRU Cell that computes the output using the formula described in the [paper](https://arxiv.org/abs/1406.1078).
**Detailed description**: *GRUCell* computes the output *Ht* for the current time step based on the followint formula:
```
Formula:
* - matrix multiplication
(.) - Hadamard product(element-wise)
[,] - concatenation
f, g - are activation functions.
zt = f(Xt*(Wz^T) + Ht-1*(Rz^T) + Wbz + Rbz)
rt = f(Xt*(Wr^T) + Ht-1*(Rr^T) + Wbr + Rbr)
ht = g(Xt*(Wh^T) + (rt (.) Ht-1)*(Rh^T) + Rbh + Wbh) # default, when linear_before_reset = 0
ht = g(Xt*(Wh^T) + (rt (.) (Ht-1*(Rh^T) + Rbh)) + Wbh) # when linear_before_reset != 0
Ht = (1 - zt) (.) ht + zt (.) Ht-1
```
**Attributes**
* *hidden_size*
@@ -20,7 +35,7 @@
* **Description**: activation functions for gates
* **Range of values**: any combination of *relu*, *sigmoid*, *tanh*
* **Type**: a list of strings
* **Default value**: *sigmoid,tanh*
* **Default value**: *sigmoid* for f, *tanh* for g
* **Required**: *no*
* *activations_alpha, activations_beta*
@@ -57,7 +72,7 @@
* **4**: `R` - 2D tensor of type *T* `[3 * hidden_size, hidden_size]`, the recurrence weights for matrix multiplication, gate order: zrh. **Required.**
* **5**: `B` - 1D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape is `[4 * hidden_size]` - the sum of biases for z and r gates (weights and recurrence weights), the biases for h gate are placed separately. Otherwise the shape is `[3 * hidden_size]`, the sum of biases (weights and recurrence weights). **Required.**
* **5**: `B` - 1D tensor of type *T*. If *linear_before_reset* is set to 1, then the shape is `[4 * hidden_size]` - the sum of biases for z and r gates (weights and recurrence weights), the biases for h gate are placed separately. Otherwise the shape is `[3 * hidden_size]`, the sum of biases (weights and recurrence weights). **Optional.**
**Outputs**