XLNET models bugs (#1199)

* Started to write equality comparator for StridedSlice layers.

* Now considered only unique StridedSlice consumers of non-constant data nodes.

* Fixed test for the transformation ConvertGroupedStridedSlice.

* Deleted commented code.

* Small fixes.

* Moved functions unique_by and group_by_with_binary_predicate to mo/utils/utils.py.

* Deleted function collect_sequences.

* Added asserts into the constructor of StridedSlice.

* Written test for the case when 1) there are 4 StridedSlice operations; 2) 2 of StridedSlice have the same data; 3) 2 others StridedSlice have the same data; 4) all StridedSlice operations outputs are consumed by different operations.

* Added some comments.
This commit is contained in:
Vladimir Gavrilov authored and GitHub committed 2020-11-19 10:07:30 +03:00
1 parent 2ec0904d40
commit 3aed19a46a
5 files changed
+258 -91

No files matched your search

+39
View File
@@ -18,6 +18,8 @@ import os
import re
import warnings
from typing import Callable
import numpy as np
@@ -108,3 +110,40 @@ def get_mo_root_dir():
"""
return os.path.normpath(os.path.join(os.path.dirname(os.path.abspath(os.path.realpath(__file__))), os.pardir,
os.pardir))
def group_by_with_binary_predicate(xs: list, predicate: Callable) -> list:
"""
It is an analogue of the function groupby from itertools, but with a binary predicate.
In other words, group_by_with_binary_predicate generates a break or new group every time
the value of the predicate function is False.
:param xs: list of grouped value
:param predicate: criterion of equality
:return: grouped list
"""
if not xs:
return []
prev = xs[0]
sequence = [prev]
result = []
for x in xs[1:]:
if predicate(prev, x):
sequence.append(x)
prev = x
else:
result.append(sequence)
prev = x
sequence = [prev]
result.append(sequence)
return result
def unique_by(xs: list, predicate: Callable) -> list:
"""
This function groups elements of the list xs using 'predicate', and then takes one element from each group.
:param xs: input list
:param predicate: grouping criterion which is some binary predicate
:return: list with unique elements
"""
groups = group_by_with_binary_predicate(xs, predicate)
return [group[0] for group in groups]