* Move infer_convolution_forward to dev API
* Move infer_auto_padding to dev API
* Move evaluate_as_partial_shape to dev API
* Move default_label_evaluator to dev API
* Move generate_transpose_default_order to dev API
* Move is_valid_axes_order to dev API
* Move has_no_labels to dev API
* Remove usage of legacy get_constant_from_source
from core and transformations
* Move normalize_axes to dev API
* Move normalize_axis to dev API
* Deprecate functions to be removed later
* extract cpu func tests into a reusable workflow
* add needs
* check paths
* correct paths
* rm usage of setupvars
* extract samples
* use yaml string
* split mac workflow
* check variables
* try github workspace
* check full dir
* set variables
* try with sub directory
* Revert "try with sub directory"
This reverts commit d891ee3f28.
* extract c++ tests from linux
* change
* use cxx workflow in mac workflows
* add noninteractive
* extract cpu func tests
* rm unused
* enable mac
* fix reusable workflow name
* correct path for sparse-checkout
* extract python UT
* remove unnecessary asterisk
* remove another unnecessary asterisk
* use reusable action for linux cc
* add check for setupvars existence
* check with manually installed opencl
* add components input to samples workflow; rm pr triggers for mac workflows
* add missing needs
* add missing option for deps install script
* use disables in test themselves instead of ifs in workflows
* use reusable workflow for cxx tests in linux arm
* use python reusable workflow in linux arm
* add missing endif
* use self-hosted for samples, add x86_64 constraint for jax
* check paths
* find gompby partial name
* skip failing tests on arm; correct gomp finding for ovc
* check tests
* add debian packages job; use job_ prefix for reusable workflows with jobs
* extract tf hub model tests
* extract tf model hub perf tests
* extract pytorch models tests
* do not use container on GHA runners
* extract onnx runtime
* add missing deps
* skip test for linux arm
* rm always()s
* fix quotes
* correct paths
* correct ifs, check dir for onnxruntime
* correct path for onnxruntime utils folder; install python3
* use self-hosted as input
* check for self-hosted runner via name, pass version
* skip cpu plugin unittest
* check cxx tests
* rm pr trigger
* add arm as a matrix for build job
* uncomment
* comment
* try inside pipeline
* check location
* another dirs
* try to privide correct action path
* use corrected action
* use newer commit
* use newer commit
* use newer commit
* use newer action commit
* add setting
* rm from pipeline, adapt action iteslf
* add missing deps
* enable samples and debian jobs
* correct yml
* correct image name
* correct syntax, use self-hosted option
* enable onnx runtime and c++, use newer action
* enable Python and CPU Func tests
* add missing deps for arm64
* increase timeout for python tests
* disable some tests, add more time
* skip failing tests
* skip speech sample test on arm
* dummy chang
* skip mxnet mo on arm, run all tests
* rm quotes
* separate linux x86 and arm64 workflows
* rm unused matrix refs, add timeouts
* add skips for c++ tests and some Python tests
* correct cache keys, extend timeout
* skip more python tests
* add more skips: for python and CPU func
* extend cpu func list with skips
* disable cpu func tests and python api 2.0 tests
* rm disable job
* styling, rm pr trigger, rm always(), rm unnecessary changes
* revert
* use ifs instead of comments, provide better wording for skips
* [Core] Support String Tensors
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add String Constant implementation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue in tests
* Add cast_vector for Constant of ov::string type
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue: ambiguous type in GNA
* Fix ambiguous build issue in GNA tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style
* Fix code-style
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix ambiguous build issue in GNA tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix ambiguous build issue in TF FE tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update openvino.style for naming convention check
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix compilation error in core unit tests - need typename
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add test for new element_type
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update src/inference/src/dev/make_tensor.cpp
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Add support of string Tensors for Constant
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix copying string tensor value for Constant
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Complete template methods for Constant
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Improve performance for initialization and destruction of string Tensor for set_shape
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add check for string value in test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove unused variable
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update src/inference/src/dev/make_tensor.cpp
* Fix copy_to for ITensor of string type and add tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add tests for Constant of string type and serialization
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use memset_allocation to switch initialization
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add additional documentation for host_ptr
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update src/core/src/op/constant.cpp
* Use OPENVINO_THROW
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update src/core/include/openvino/op/constant.hpp
* Update src/core/include/openvino/op/constant.hpp
Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>
* Apply code-review feedback: use string_size
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Recover evaluate impl for non-string type
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code for creating of string constant for legacy non HostTensor tensor
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: simplify copy_to method
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use StringAlignedBuffer to store string Constant values
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove not needed methods in StringAlignedBuffer
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Refactor set_shape method
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>
* Update reverse infer to allow changing shape if it already partially defined
* Update tests
* Remove changes in If
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* add to DisableShapeOfConstantFolding flag; set it in ts_gather; add unit test
* fix ts_general; use check_shape flag in EnableShapeOfConstantFolding
* set EnableShapeOfConstantFolding constructor explicit
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* Added experimental ScaledDotProductAttention operation in opset12. Supported in PT FE for aten::scaled_dot_product_attention translation. Decomposed in the common optimizations as functional reference.
* Better ScaledDotProductAttention
- Moved decomposition to the decomposing transformation
- Implemented more ctors for the op
- Renamed is_causal to causal
- Shape/type inference native code instead of using decomposition
- Moved the op from opset12 to opset13
- Added Python wrapper for ScaledDotProductAttention
* Fix test that counts ops in the opsets
* Update src/core/src/op/scaled_dot_product_attention.cpp
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
* Update src/core/src/op/scaled_dot_product_attention.cpp
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
* Move ScaledDotProductAttentionDecomposition from fusions to decompositions.
* Remove not used legacy shape inference in ScaledDotProductAttention
* Better namespace usage
* Register all nodes in ScaledDotProductDecomposition for correct tracking of nodes and running next mather passes on all new nodes.
* Don't use register_new_node_
* ScaledDotProductAttention specification (with an extra scale argument)
* Code style fix
* Scale input implementation for ScaledDotProductAttention
* Handle attention_mask=0 case in the op spec
* Better description of scale input
* N->M in scale description
* Code style fix, remove debug print.
* Apply suggestions from code review
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Mateusz Mikolajczyk <mateusz.mikolajczyk@intel.com>
* Fix for case when is_causal is not passed
* Extended description of ScaledDotProduct op
* Better description in py op wrapper
* Basic shape propagation tests for ScaledDotProductAttention
* Added ScaledDotProductAttention to toc.
* Add op impl check
---------
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Mateusz Mikolajczyk <mateusz.mikolajczyk@intel.com>
* Fix constant folding in MulMulMulFusion
by add f64 precision in Multiply to perform evaluate for const folding
* Do not transform if input has not supported type
* Handle Reshape's special zero in SimplifySecondInputOfReshape
SimplifySecondInputOfReshape detects ShapeOf->Gather->Concat
subgraphs on Reshape's second input and replaces ShapeOf->Gather
with a Constant with zero(s). Currently it works only with Reshapes
that have special_zero set to true, but it can work for Reshapes
with special_zero == false if non-Gather inputs to Concat are Constants
and don't contain any zero.
Ticket: CVS-123434
* fix no default output
* Handle Reshape in SplitSqueezeConcatFusion
Ticket: CVS-122455
* move check for squeeze/reshape
* add some comments
* review comments
* add use_shapes flag to SplitSqueezeConcatFusion
* Add static shape adapter
- Adapters holds CPU dimension which can be reference to it or vector
- Add ov::optional for holding optional result from shape inference
- Add new `infer` function in `IStaticShapeInfer`
* Temporary support of StaticShape
* Minor corrections in ShapeInferenceTA
* Migrate shape_infer to new interface version
* Replace StaticShape by adapter implementation
* Replace IShapeInferCommon by IStaticShapeInfer
* Correct code formatting
* Fix build issues
* NodeValidationFailure::create for StaticShapeRef
* Review shape inference for reshape operator
- review shape_infer implementation
- add more unit test for static and dynamic shapes
* Fix build issues
* Correct minus one dim calculation
* Fix build issues on windows
* Improve resolving special minus one
* Use NODE_SHAPE_INFER_CHECK
* Update product in/out calculations
* Temporary add ngraph header to solve build issue
* Correct minus one dim calc when static part same
* Add check for scalar input
* Remove debug message
* Fix `minus one` dynamic dimension calculation
* Fix `minus one` dynamic dimension calculation
* Fix merge issues in reshape
Minor refactor reshape evaluate
* Don't pass input label on minus one pattern
when input dimension will be modified.
* Add Rotation support to primitive and kernel
* Add unit tests
* Add transformation for NMSRotated
* add single-layer tests
* Fix: angle value for the same box may have its sign changed several times passing through iterations of batch and class loops.
* fix review comments
Transformation fuses Transpose on first or second MatMul's input
and sets MatMul's transpose_a/transpose_b accordingly.
TransposeMatMul is already part of SmartReshape, but it can be added
to MOCTransformations as well so native models that are don't use reshape
can benefit from that.
Ticket: CVS-118908
Current implementation tries to leverage branchless approach, but it's not correct
if scale is 0. In that case - zero point can can become inf or nan and multiplication
by 0 doesn't change its value. That causes another issue - infinite or NaN zero point
cannot be optimized out later.
Ticket: CVS-122931
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* [TF FE] Provide full support of TF1 Control flow and TensorArray ops
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add missed header for TensorArrayV3 op
* Temporarily disable GRU cell fusion
* Update src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp
* Fix a case when element_shape for TensorArrayV3
* Fix translator for TensorArrayCloseV3
* Update summarize graph with TensorArrayCloseV3
* Add layer tests for TensorArrayScatterV3, Close, Size, Array
* Fix output shape for Merge node
* Remove unused variable
* Fix translator for TensorArrayConcatV3
* Fix translator for TensorArrayConcatV3
* Add layer tests for TensorArrayWriteV3, Gather, and Concat
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add translator for GatherTree
* Fix TF FE unit-test for GatherTree
* Fix GatherTree translator
* Fix GatherTree translator to handle 1d end_token
* Fix undeclared parameter issue
* Fix GatherTree unit-test
* Add TensorArrayV3Replacer transformation
* Temporarily disable dangling transformation
* Recover RemoveMultiSubGraphOpDanglingParamsResults transformation
* Recover GRUCellFusion transformation
* Simplify check for GRUCellFusion transformation
* Use proper name for unit-tests
* Simplify translator for TensorArrayWriteV3
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix RemoveMultiSubgraphOpDanglingParamsResults transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Additional fix for remove_multi_subgraph_op_dangling_params
* Make static TI run a dynamic subgraph
* Dedicated SL test
* Change condition to respect stat shapes
* Adjust test to cover the code path properly
* Recover fallback for still failing case GNMT
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
* Reference implementation for u4 constant compression from pytorch model based on bitwise ops pattern
* Fixed order of 4-bit halfs in byte
* Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case).
* Moved u4 compression to utils_quantize. Implemented not-interleaved version of u4 compression
* Removed debug output
* Added aten::matmul to the list of exceptions in may_produce_alias as a workaround for gptq models
* Added patching for gptq models applied automatically in convert_model
* WA for an inssue with u4 with earlier convert to fp16
* U4 blocked repacking for gptq patched model layout
* Deleted obsolete u4 re-packing based on aten::cat. Fixed the resulting u4 constant shape. Removed debug output.
* Revert "Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case)."
This reverts commit 0ef1455e70.
* Update src/frontends/pytorch/src/op/cat.cpp
* Check mask and shift values in u4 pattern. deque -> OutputVector for u4_compression_stack
* Convert to a given floating type instead of half in gptq patching. Better structured code.
* Code style fix
* Removed deque include
* Code style fixes
* Trailing space removed
* Fixed patched_forward and ts_decoder after unvalidated commits.
* Swap nibbles in u4/i4
* Better exception handling around jit.trace and gptq.patch_model
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Code style
* Revers int4 byte order
* Fixed core tests
* Fixed unguarded dynamic_cast result
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
* Fixed transformation tests
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Prevent patching of non-gptq models
* Removed extra calling of quantized weights decompression patterns
* Better detection of supported AutoGPTQ models + more diagnostics
* Accurate diagnostics in case when aten::stack has multiple axes
---------
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Maxim VafinandIvan Tikhonov
Evgenya Nugmanova
Tomasz Jankowski
Mateusz Mikolajczyk
Oleg Pipikin
Vitaliy Urusovskij
Georgy Krivoruchko
Andrey Kashchikhin
River Li
Xuejun Zhai
Vladislav Golubev
Evgeny KotovandIvan Tikhonov
Aleksandr Voron
Nikolay Shchegolev
Pavel EsirandAndrei Kochin
Vladimir Paramuzov
Oleksii Khovan
