* Draft version of the Swish nGraph operation and fusing transformations for different approaches to express the operation
* Swish fusing transformation refactoring
* Added Swish operation and extractor for TF. Removed unfolding transformation for the operation.
* Added SwishIE. Implemented transformation to convert Swish to SwishIE.
* Code style fixes
* Updated Swish reference implementation. Added tests for shape and value inference
* Fixed code style for Python API
* Fixed unit test
* Apply review comments
* Use matcher_pass_callback
* Make m_alpha attribute protected in the SwishIE operation
* Fixed Swish op PythonAPI test
* Added Caffe Slice_ext
* Added TFSlice, AttributedSlice (both with extractors and replacers), corrected SliceConverter and added unittests for all cases
* added comments to each type of Slice operation; optimized shape inference; moved mxlice inside of slice.py; renamed slice_replacers
* removed type annotation for get_shape_after_slice routine
* replaced zeros_like with zeros
* Corrected preserving node names, renamed attributes names, added tests fro slice_replacer onnx phase
* Renamed slice_replacers.py
* added more unittest cases
* added type annotations, moved to more relevant place routines for shape calculation, and some other minor corrections
* corrected a typo `normalize_slice_indices` comment
* corrected shape calculation for Nonconstant inputs
* corrected a few typos
* corrected type declarations
* corrected shape inference with rounding
* refactored unit-tests for front transforms of Slice
* added error raising for negative and zero shapes
* removed magic_num
* corrected AttributedSlice, clarified comments
* fixed unit-test for AttributedSliceToSlice
* typo in type hints corrected
* removed supported_attrs
* returned back default None for attrs of Slice
SparseToDense used in Wide and Deep model is expressed through ScatterND operation.
ScatterND is more functional than SparseToDense. Hence, it was decided to replace SparseToDense
with ScatterND. ScatterND is more useful for other models.
Remove SparseToDense from the previous opset
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* LayerNorm(PyTorch/HuggingFace pattern)->MVN+Mul+Add. Improves perf on BERT by 5%
* deducing the across_channels from axes passed to the MVN op.
axes are normalized. if no axes is specified, falling back to the (previously) default across_channel value
Co-authored-by: myshevts <maim.y.shevtsov@intel.com>
* Removed back phase transformations related to IRv7
* Fixed setting value for the input port using the 'set_value' method
* Removed front and middle phase transformations related to IRv7
* Cleanup the rest of the Model Optimizer transformations from IRv7 specific transformations
* Final cleanup of the deprecated IR v7 related code
* Removed 'blobs_as_input' usage in the Model Optimizer.
* Removed function '_fuse_add' from the Model Optimizer since it is not used anymore.
* Removed 'keep_in_IR' node attribute for FakeQuantize ops in the MO
* Disabled failing gpu_engine.user_context test
* [MO] Implement EmbeddingBag_3
* Transform dynamic sub-graph of Wide and Deep into EmbeddingSegmentsSum
- Expressed SparseWeightedSum sub-graph through EmbeddingSegmentsSum
- Removed experimental SparseWeightedSum layer
- Implemented tests for the transformation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix EmbeddingBag shape infer
* Fix EmbeddingSegmentsSum transformation for Wide and Deep
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix EmbeddingSegmentSum replacer after ports swap
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Update package_BOM.txt
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Add unit tests for EmbeddingXXX shape infer
* Fix ATen resolver
* Remove deleted files from BOM
* Add opset version to embedding_bag
* Use base class for EmbeddingBag
* Fix per_sample_weights case
* Fix EmbeddingSegmentsSum transformation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix EmbeddingBag checks
* Fix ATen front transformation and merge conflicts
* Fix BOM
* Work around limitation for I64 input of W&D model
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Cleanup where operation to fix affect of WhereDecomposition transform
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix BOM
* Correct EmbeddingSegmentSum transform for Wide and Deep
Add casting segment ids to i32 and remove ConstToResult sub-graph.
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Update BOM with RemoveConstToResult transform
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Add more comments for RemoveConstToResult transformation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Remove useless logging in EmbeddingSegmentsSum transformation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Small fixes
* Move EmbeddingBag resolving back to front phase
* Improve error messages
* Fix typo in unittests
* Reimplement sparse_reshape middle transform
Avoid deprecated API.
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Clean-up graph after sparse_reshape and ConstToResult transformation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix clean-up for transformations
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix clean-up for transformation #2
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Roman Kazantsev
Maxim Vafin
Evgeny Lazarev
Pavel Esir
Yegor Kruglov
Maxim Shevtsovandmyshevts
Evgenya Stepyreva
iimironov
Vladimir Gavrilov
Alexey Suhov