* [TF FE] Fix TF1 SSD PPN model conversion
It contains a case when one Merge node eliminated different conditional flows.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add layer test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Added experimental ScaledDotProductAttention operation in opset12. Supported in PT FE for aten::scaled_dot_product_attention translation. Decomposed in the common optimizations as functional reference.
* Better ScaledDotProductAttention
- Moved decomposition to the decomposing transformation
- Implemented more ctors for the op
- Renamed is_causal to causal
- Shape/type inference native code instead of using decomposition
- Moved the op from opset12 to opset13
- Added Python wrapper for ScaledDotProductAttention
* Fix test that counts ops in the opsets
* Update src/core/src/op/scaled_dot_product_attention.cpp
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
* Update src/core/src/op/scaled_dot_product_attention.cpp
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
* Move ScaledDotProductAttentionDecomposition from fusions to decompositions.
* Remove not used legacy shape inference in ScaledDotProductAttention
* Better namespace usage
* Register all nodes in ScaledDotProductDecomposition for correct tracking of nodes and running next mather passes on all new nodes.
* Don't use register_new_node_
* ScaledDotProductAttention specification (with an extra scale argument)
* Code style fix
* Scale input implementation for ScaledDotProductAttention
* Handle attention_mask=0 case in the op spec
* Better description of scale input
* N->M in scale description
* Code style fix, remove debug print.
* Apply suggestions from code review
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Mateusz Mikolajczyk <mateusz.mikolajczyk@intel.com>
* Fix for case when is_causal is not passed
* Extended description of ScaledDotProduct op
* Better description in py op wrapper
* Basic shape propagation tests for ScaledDotProductAttention
* Added ScaledDotProductAttention to toc.
* Add op impl check
---------
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Mateusz Mikolajczyk <mateusz.mikolajczyk@intel.com>
* [TF FE] Fix conversion of TF1 OD models out-of-the-box
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add test While with nested If operation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update tests/layer_tests/tensorflow_tests/test_tf_While.py
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [CPU] CPU plugin migrates to plugin API 2.0
* Fix legacy config/metric issue
* Fix some issue of ov_cpu_func_tests
1. set_tensors_impl segment fault
2. ov::loaded_from_cache unsupported issue
* Resolve some comments
1. ov::loaded_from_cache issue
2. throw_if_cancelled issue
3. import_model issue
4. set_tensor_impl issue
5. batched_inference issue
* Fix dynamic shape inference issue
* Fix build error
* keep original model info in infer_request
* Fix minor error
* cache internal tensors for input/output precision change
* Disable import model test cases with precision changes
* fix precision issue
* Fix issue for import model
* Fix InferRequestCancellationTests exception issue
* Skip InferRequestIOBBlobTest.*secondCallGetInputDoNotReAllocateData due to new plugin api have different behavior
* Fix graph name issue
* Fix ROI issues
* Fix Transpose shape issue
* Skip vie::Version test due to change to ov::Version
* Solve input port name changes issue
* Solve preprocess layout issue
* Fix minor issue
* tidy up code
* Fix conflict after rebase
* Fix Windows build warning
* Add aux tensors for precision change issue
* Fix import/export model issue
* WA single layer name changed by preprocess
* Revert "WA single layer name changed by preprocess"
This reverts commit bc8fcdd43c.
* Skip some legacy tests due to plugin api 2.0 is enabled
1. skip some python legacy tests for plugin api 2.0 some different behaviors
2. skip some smoke tests due to output port name was changed
* Fix 2 build warnings
* Skip some AUTO plugin tests
* Fix property issue caused by AUTO plugin
* Skip PSROIPooling issues
* Follow header files reference policy
* Split out transformation fixing for nop_elimination
* Fix AUTO plugin mismatch issue for get_tensor function
* Fix aux tensor shape issue
* Fix tensor shape issue
* WA python sync inference sample's segmentfault issue
* Fix reshape issue for dynamic inference
* Fixed incorrect tensor name in e2e test
Fixe issue: e2e ONNX_Customized_Cascade_Rcnn_api_2_True_batch_1_device_CPU_precision_FP325den8cnk
* Fix python segmentfault issue of plugin api 2.0
* Fix python segmentfault issue of plugin api 2.0
* Revert "Fix python segmentfault issue of plugin api 2.0"
This reverts commit 6f502e5d86.
* Fix onnx_duplicated_output_name due to empty tensor
Co-authored-by: Bell, Song <bell.song@intel.com>
* Remove redundant code
* Remove python segment fault WA
* Keep rt_info to fix test failure in case of legacy public api
* Fix output port names missing issue
* Adress some reviewers' comments
* Restore OnnxBackendNodeModelTest::test_maxpool_with_argmax_2d_precomputed_pads_cpu after fixing has been merged
* Resolve tensor sharing issue when there are same name output port name
In some case, model has 2 or more same name input/output ports, they aslo have the same
precision and partial_shape. Compiled_model will share the same ov::Descriptor::Tensor pointer
and ov::Tensor between multiple such ports.
Considered solving python segment fault issue to create seperated input/output ports, which also
need handle such tensor shared case, this patch will do it.
* Resolve tensor sharing issue when there are same name output port name
In some case, model has 2 or more same name input/output ports, they aslo have the same
precision and partial_shape. Compiled_model will share the same ov::Descriptor::Tensor pointer
and ov::Tensor between multiple such ports.
Considered solving python segment fault issue to create seperated input/output ports, which also
need handle such tensor shared case, this patch will do it.
* Better method to find shrared tensor desc
* rename with snake_case style
* Remove ngraph header files
* Keep external_ptr naming
* Add OPENVINO_SUPPRESS_DEPRECATED for some legacy code
* Use port's tensor_ptr to replace creating new tensor_ptr
* Resolve some reviewer comments
* Implement ov::IInferRequestInternalWrapper::GetPreProcess to recover python GetPrepProcess tests
* Remove unnecessary header files reference
* Assert the risk of precision change and reorder at the same time
* Modify legacy python test to fit plugin api 2.0 behavior
* Recover smoke_Transpose(2|4|5|6)D/TransposeLayerTest.CompareWithRefs due to fixing is merged
* Fix typo issue
* Address reviewer's comments
* Disable precision coversion
* Fix error when CpuBlockedMemoryDesc
* Remove precision mismatch WA
* WA precision issue for query_model
* Solve precision mismatch between compiled model and graph
* Fixe failure of query_model
* Rebase to new plugin api update
* Recover the test cases of precision mismatch
* Try to fix name changing for graph model
* Remove tets code
* Remove fp64
* Rebase to new plugin api update
* Update for some failure cases
* Fix bert_benchmark failure issue
* Avoid segment fault in arm acl
Legacy public api + cpu plugin api will add convert op by preprocess by default for unsupported precision,
but ACLConvertExecutor cannot support dimension > 6, so this test will be segment fault due to dimension > 6
smoke_TestNumpyBroadcastNgraphEvaluate/BroadcastLayerTest.CompareWithRefs/targetShape=(1.2.3.4.5.6.7.8.9.10)_axesMapping=()_mode=numpy_inShape=(1.2.1.4.1.6.1.8.1.10)_inNPrec=I8_trgDev=CPU
smoke_TestNumpyBroadcastNgraphEvaluate/BroadcastLayerTest.CompareWithRefs/targetShape=(1.2.3.4.5.6.7.8.9.10)_axesMapping=()_mode=numpy_inShape=(1.2.1.4.1.6.1.8.1.10)_inNPrec=U8_trgDev=CPU
* Remove precision change from preprocess to avoid ACL unsupport convert dim > 6
* ACLConvertExecutor cannot support dimension > 6, don't let preprocess to add Convert
* Revert "ACLConvertExecutor cannot support dimension > 6, don't let preprocess to add Convert"
This reverts commit fd7a8b35af.
* Revert "Remove precision change from preprocess to avoid ACL unsupport convert dim > 6"
This reverts commit 3c2d9a5f17.
* Debug
* Debug incorrect precision checking issue
* Debug Eltwise FP64 unsupported issue
* Add logs for precision
* debug log
* Update for new dependent PRs merged
* Fix failure caused by preprocess
Fix below failures due to cannot find ops by name
smoke_LPT/ReduceMaxTransformation.CompareWithRefImpl/f32_[1,3,10,10]_CPU_f32__256*
* Fix build error
* Fix failure caused by missing code during rebase
* Add debug
* Fix precision unsupport issue
* U16/I16/U64 precision support
* Resolve the issue of f64 reorder
Fix below issue:
Cannot create reorder primitive: unsupported reorder case
* Fix convert multiple child edge issue
* Solve ROI tensor failure issues
* Temporarily disable num_nodes comparison
* Only change convert precision for fp64
* Put convert precision change before reorder to avoid confusion
* Add debug log for transformation
* Fix rebase confilict
* Fix clang issue
* Temporarily disable test_infer_mixed_values python test of bf16
* Solve issue of smoke_ConvertCPULayerTest_BOOL_Dynamic_inputPRC=BF16 choose FP32 primType rather than BP16 primType
* Fix issue of pytorch_tests/test_outer.py
There are 2 output ports, but with the same port name, they should share the same tensor.
* Fix arm cannot find Eltwise executor issue
smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=INPUT_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL
will report below error:
[ GENERAL_ERROR ] Supported Eltwise executor is not found
It need change convert precision to avoid such problem.
* Fix memory overwritten issue
* Temporarily skip arm fp16 SetBlobTest
* Fix compile error after rebase
* Restore smoke_IsOp test due to fixing pr merged
* Fix float to bf16 issue in avx2 isa
* solve onnx test xfail issue
* Skip test cases that ARM Eltwise executor FP16 is not supported
smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=INPUT_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL
smoke_SetBlobCPU/SetBlobTest.CompareWithRefs/Type=BOTH_Device=CPU_PrecisionInNet=FP16_PrecisionInNgraph=BOOL
[ GENERAL_ERROR ] Supported Eltwise executor is not found
* [CPU] improve reorder to support any precision
* Implement ReorderExecutor
* Fix builld error
* Not cache executor due to its primitive has been cached
* Keep convert one time at most
At most insert one convert if needed, if still cannot do reorder it will throw exception rather than insert the second convert
For example, below reorder will not be supported:
FP64<->I64/U64/U32
U32<->I64/U64
U32<->I16/U16
FP64<->FP64
BIN<->BIN
* Only do conversion if layout is same
* update for only convert case
* Update for reviewer comments
* update for failure cases
* Address reviewer comments
* Update rebase issue
* minor update
* Solve unsupported precision issue in tranfromation rather than init_edge
* Remove unnecessary convert in init_edge
* Minor changes
* Update Reorder::reorderData
* Solve issue if only coversion without reorder
* Address reviewer comments
* Address reviewer comments
* Keep exception for unsuported precision
* update
* Revert reorder executor implement
* Solve float->bool issue on transformation pipeline
* Solve I64 is not supported issues
* Solve reviewer's comments
* Fixed dynamic top_k node issue
* Skip nhwc and nChw16c test cases for ConvertLayer
* Update for reviewers' comments
* Fix some failures
* Update for several failure cases
* Update for apiConformanceTests failures
* Fix incorrect node name after import model
* update
* update comments
* Solve issue of smoke_MatMul_NoTranspose and smoke_MatMul_BothTranspose
* Fixed AlignMatMulInputRanks scalar issue
* Address reviewers' comments, remove redundant path in graph.cpp
* Remove test_div_uint8_cpu from xfail_issue_58676
* Solve invalid number of nodes for smoke_Snippets_BroadcastSelect
* ConstantResultSubgraphTest of u16/i16/u32/i64/u64
* restore smoke_SetBlobCPU BOOL tests for arm
* [CPU] Fix ARM precision issue
ARM64 ACL prefers fp16 than fp32, API 2.0 requires input/output precision not changes,
then fp32 input will trigger convert node is added to convert fp32 to fp16.
* Solve some ARM64 failures
* Fix arm64 InferRequestVariableStateTest tests out of memory issue
ARM64 will force fp16 precision, which cause states memory can be fp16, so memcpy to state_memory
cannot use float * element_size, else it will be out of memory bound.
* Skip 2 arm64 tests caused by forcing fp16 precision
* Revert "Fix arm64 InferRequestVariableStateTest tests out of memory issue"
This reverts commit 3e12bd48c2.
* Fix python test_get_profiling_info failure issue
---------
Co-authored-by: Bell, Song <bell.song@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Remove `ngraph` from PT FE and FE tests utils
* Remove `ngraph` from Paddle FE
* Remove `InferenceEngine` from some ONNX FE test
* Port `generate_embedding.py` to API2.0
* CLangFormat
* Fix comments
* [TF FE] Support complex tensors
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Align output type for Real and Imag operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update decoding complex types
* Add support for ComplexAbs, FFT and IFFT operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Correct axes based on a number of inner-most dimensions
* Add layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update supported ops documentation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add a comment for ComplexTypeMark
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Switch off TF1 While support totally
This is a total switch off due to GPU limitation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Need additional fallback in Enter to avoid shapes problem
* Disable tests with While op
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Disable layer test for TF1 While
* Remove extra spaces
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* fix paddle load model from memory
* fix coding style
* ignore the deprecated api
* fix a istream bug; add test case
* simplify func variant_to_stream_ptr
* restore the previous impl for less memory affect
* fix memory leak
* Set sort_result_descending attr false for nms
* Add onnx test
* Tmp exclude for CPU
* Update test values
* Update test vals to avoid cast warning
* Fix typo
* [TF FE] Fix body graph injection, CumSum and SparseFillEmptyRows
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Do not handle non-parameters in body
* Update layer test to cover default parameter and attribute values
* Fix layer tests
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add support for ONNX GroupNormalization
* Enable onnx compliance tests
* Add f to the test values to avoid cast warning
* Use tolerance as fp instead of bits
* [TF FE] Provide full support of TF1 Control flow and TensorArray ops
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add missed header for TensorArrayV3 op
* Temporarily disable GRU cell fusion
* Update src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp
* Fix a case when element_shape for TensorArrayV3
* Fix translator for TensorArrayCloseV3
* Update summarize graph with TensorArrayCloseV3
* Add layer tests for TensorArrayScatterV3, Close, Size, Array
* Fix output shape for Merge node
* Remove unused variable
* Fix translator for TensorArrayConcatV3
* Fix translator for TensorArrayConcatV3
* Add layer tests for TensorArrayWriteV3, Gather, and Concat
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add translator for GatherTree
* Fix TF FE unit-test for GatherTree
* Fix GatherTree translator
* Fix GatherTree translator to handle 1d end_token
* Fix undeclared parameter issue
* Fix GatherTree unit-test
* Add TensorArrayV3Replacer transformation
* Temporarily disable dangling transformation
* Recover RemoveMultiSubGraphOpDanglingParamsResults transformation
* Recover GRUCellFusion transformation
* Simplify check for GRUCellFusion transformation
* Use proper name for unit-tests
* Simplify translator for TensorArrayWriteV3
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix RemoveMultiSubgraphOpDanglingParamsResults transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Additional fix for remove_multi_subgraph_op_dangling_params
* Make static TI run a dynamic subgraph
* Dedicated SL test
* Change condition to respect stat shapes
* Adjust test to cover the code path properly
* Recover fallback for still failing case GNMT
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
* add m1 mac pipelines as a matrix parameter
* Update mac.yml
disable java_api because of macos arm64 - Java is not available on macOS arm64 runners
* Update mac.yml
added always condition for all tests
* Update mac.yml
* Update mac.yml
* Update mac.yml
* Update setup.py
temp commit
* Update tools/openvino_dev/setup.py
* use matrix for var
* add mxnet to extras only for x86_64
* skip failing tests
* use xfail for Python tests; add missing filter for transformations tests
* skip CPU func tests on x86_64 mac; skip some tests from CPU func tests on arm mac
* Update mac.yml
* skip tests on mac arm
* skip tests on darwin; apply review
* add more skips for python and c++ tests
* skip tf tests
* skip more tf tests; skip more Python UT stages
* rm alwayses, rm triggers, add nightly trigger
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Siddhant Chauhan
Pawel Raasz
Maciej Smyk
Roman Kazantsev
Maxim VafinandAlina Kladieva
Vitaliy Urusovskij
Surya Siddharth Pemmaraju
rsato10andRoman Kazantsev
Xiuchuan Zhai
Ilya Lavrenov
Karan JakharandMaxim Vafin
Mikhail Ryzhov
Ilya Churaev
Andrey KashchikhinandIlya Lavrenov
