* Handle Reshape's special zero in SimplifySecondInputOfReshape
SimplifySecondInputOfReshape detects ShapeOf->Gather->Concat
subgraphs on Reshape's second input and replaces ShapeOf->Gather
with a Constant with zero(s). Currently it works only with Reshapes
that have special_zero set to true, but it can work for Reshapes
with special_zero == false if non-Gather inputs to Concat are Constants
and don't contain any zero.
Ticket: CVS-123434
* fix no default output
Co-authored-by: Mateusz Tabaka <mateusz.tabaka@intel.com>
* Corrected memory consumption log, removed unused log_level info.
* Minor correction.
* Trace memory only if verbose is set.
* Use get_traced_memory().
* [TF FE] Switch off TF1 While support totally
This is a total switch off due to GPU limitation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Need additional fallback in Enter to avoid shapes problem
* Disable tests with While op
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Disable layer test for TF1 While
* Remove extra spaces
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* fix paddle load model from memory
* fix coding style
* ignore the deprecated api
* fix a istream bug; add test case
* simplify func variant_to_stream_ptr
* restore the previous impl for less memory affect
* fix memory leak
* Migrate NotEqual operator to new API
* Remove `visit_attributes` is same as base
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Set sort_result_descending attr false for nms
* Add onnx test
* Tmp exclude for CPU
* Update test values
* Update test vals to avoid cast warning
* Fix typo
* [TF FE] Fix body graph injection, CumSum and SparseFillEmptyRows
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Do not handle non-parameters in body
* Update layer test to cover default parameter and attribute values
* Fix layer tests
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Improved warning message for BA + OpenCL case
* Update samples/cpp/benchmark_app/CMakeLists.txt
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
---------
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* Drop HostTensor and move to ov namespace
* Style
* Optimize vector assignment
* Optimize vector assignment
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Migrate Concat op to new API
* Move shape validation to shape_infer
* Fix getting concat axis in shape inference
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* choose Pcore to compile model for GPU plugin
* provide function to update executor config
* set callback executor to nullptr for GPU plugin
* fix code style
* fix warning
* optimize duplicate code
* set callback executor to nullptr for another gpu compile_model
* add description for new function
* add smoke test
* fix code style
* modify function definition
---------
Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
* Migrate Transpose to new API
* Move shape validation to shape_infer
* Remove visit_attributes is same as base
* Correct transpose order shape check
for static shapes
- correct creation of order shape for static shape tests
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Add static shape adapter
- Adapters holds CPU dimension which can be reference to it or vector
- Add ov::optional for holding optional result from shape inference
- Add new `infer` function in `IStaticShapeInfer`
* Temporary support of StaticShape
* Minor corrections in ShapeInferenceTA
* Migrate shape_infer to new interface version
* Replace StaticShape by adapter implementation
* Replace IShapeInferCommon by IStaticShapeInfer
* Correct code formatting
* Fix build issues
* NodeValidationFailure::create for StaticShapeRef
* Review shape inference for reshape operator
- review shape_infer implementation
- add more unit test for static and dynamic shapes
* Fix build issues
* Correct minus one dim calculation
* Fix build issues on windows
* Improve resolving special minus one
* Use NODE_SHAPE_INFER_CHECK
* Update product in/out calculations
* Temporary add ngraph header to solve build issue
* Correct minus one dim calc when static part same
* Add check for scalar input
* Remove debug message
* Fix `minus one` dynamic dimension calculation
* Fix `minus one` dynamic dimension calculation
* Fix merge issues in reshape
Minor refactor reshape evaluate
* Don't pass input label on minus one pattern
when input dimension will be modified.
Deconvolution::createDescriptor will call createDescriptorInternalDefault() to create fwd_conv_pd,
sometimes ref_convolution_fwd_t will be chosen to return its primitive_desc, but ref_convolution_fwd_t
primitive will not be created finally, then CC will not put this primitive into convolution_impl_list
in selective build stage, the final CC package will fail due to cannot create fwd_conv_pd of ref_convolution_fwd_t.
* [GPU] Fix outputs are not allocated in loop_inst
* Fill empty padding when the number of output paddings is less than num_outputs
* Fill empty data types when the number of output data types is less than num_outputs
* Modify postprocess_output_memory to set output memory without set_output_memory function
* In postprocess_output_memory, get concatenated_output_mem using input_info including output idx
* Modify gpu functional tests for dynamic loop to check multiple outputs of dynamic loop
* update postprocessing for condition
* Fix empty dimension issue for scalar value
* change code to get output paddings and output data type in primitive
* allocate memory for scalar data type with zero dimension
* Fix mismatch issue of input layout with shape and data types in body_network
* Fix output setting in post-processing
* pass bytes_count to gpu_usm params
* Fix condition gpu functional test issue
* Revert "allocate memory for scalar data type with zero dimension"
This reverts commit 2f10f3687c78406b20d52b6e37b1be2a30b4b73f.
* reinterpret one dimension memory buffer to zer dimension memor buffer to avoid zero byte memory allocation issue
* Add support for ONNX GroupNormalization
* Enable onnx compliance tests
* Add f to the test values to avoid cast warning
* Use tolerance as fp instead of bits
* Migrate Equal to new API
* Remove `visit_attributes` is same as base
* Remove i4, u4 from evaluate in Equal
reference implementation not handle binary precisions correctly
* Sync precisions in `has_evaluate` with `evaluate`
* Fix all equal check for lower bound
- make broadcast test assertion more strict
- remove deprecated functions from broadcast test
* [CPU]fix mergeTransposeAndReorder when Transpose and Reorder do opposite permutation to each other as expected, but isOptimized is false due to some reason like inPlace memory. The permuation is asked because the dnnl reorder primitive requires the source and destination tensors to have the same shape.
* add unit test
* Update src/plugins/intel_cpu/src/graph_optimizer.cpp
Co-authored-by: Maksim Kutakov <maxim.kutakov@gmail.com>
* simplify unit test
* update comments
---------
Co-authored-by: Maksim Kutakov <maxim.kutakov@gmail.com>
* build dynamic in win main; transfer win main to MSVC 2019; use toolchain for win cc
* Update windows_conditional_compilation.yml
use cmake toolchain globally in build stage
* Update windows_conditional_compilation.yml
* Update windows_conditional_compilation.yml
* use quotes
* try w/o protobuf
* do not restore cache
* return
* revert
* add missing shell
* skip Template OpImpl tests
* skip OV C API tests
* rm pr trigger
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Migrate Negative operator to new API
* Remove `visit_attributes` is same as base
* Use std::negate instead of lambda
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Migrate slice to new API
* Remove visit_attributes, is same as base class
* Move shape checks to shape_infer
- minor refactor Slice op
* Move `get_tensors_partial_shapes` to dev API
* Correct comment
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
---------
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
* skip excessive mem alloc request in build
* update mem check function
* fix os behavior
* update mem size check location
* only dynamic shape case takes check_allocatable
* update check condition
* `RNNSequenceTest` to API2.0
* `Result` to API2.0
* `Reshape` to API2.0
* `ReorgYolo` to API2.0
* `RegionYolo` to API2.0
* Alignment fixes
* Skip more `RNNSequenceTest` cases
* Migrate Less operator to new API
* Migrate Greater operator to new API
- use less implementation in greater to reduce bin size
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* [workflows/linux] Switch to sccache and Azure Blob Storage
* Install curl
* Remove --show-config
* Add sccache to other Linux workflows
* sccache to Android, curl to riscv and CC
* Use sccache action instead of manual install
* Oops, missed sccache manual installation in two places
* Use env vars instead of hardcoded CMAKE_C(XX)_COMPILER_LAUNCHER
* Forgot one more stage in Linux CC pipeline
* Temporarily disable Blob Storage for RISC-V
For some reason sccache has no effect on build time and show 0 hits
and 0 compilation requests despite being in CMake calls
* forgot to add sccache installation to Linux CC
* Revert "Temporarily disable Blob Storage for RISC-V"
This reverts commit b528f41dad583a38b9ef93121e38044b9dccb71b.
* Missing container option for CC build
* Remove curl installation
* Remove CCACHE* variables which have no effect on sccache
* Revert sccache changes for Linux RISC-V workflow
* Add Rotation support to primitive and kernel
* Add unit tests
* Add transformation for NMSRotated
* add single-layer tests
* Fix: angle value for the same box may have its sign changed several times passing through iterations of batch and class loops.
* fix review comments
* Migrate Minimum op to new API
* Refactor evaluates to reduce binary size
- add infer_broadcast_shape, get shapes from tensors reduce OV_ASSERT
- refactor Evaluate structures to reduce binary size
---------
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
Transformation fuses Transpose on first or second MatMul's input
and sets MatMul's transpose_a/transpose_b accordingly.
TransposeMatMul is already part of SmartReshape, but it can be added
to MOCTransformations as well so native models that are don't use reshape
can benefit from that.
Ticket: CVS-118908
* Preserve partial values on mod inputs
- static values full range of integers
- intervals only if not negatives
* Fix bounds evaluate when inputs are scalars
Current implementation tries to leverage branchless approach, but it's not correct
if scale is 0. In that case - zero point can can become inf or nan and multiplication
by 0 doesn't change its value. That causes another issue - infinite or NaN zero point
cannot be optimized out later.
Ticket: CVS-122931
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* try to fix memory leak issue
cpustreamer is released, but there are still thread id in t_stream_count_map
* fix threadlocal affect all threads
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* add comment for local() function to avoid mistaken modification
in the future
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* use custom stread id
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* fix review comments
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* fix format issue
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* create shared_ptr before assert
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
---------
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* Initial implementation of primitive, kernel selector, dummy kernel for RMS Norm
Signed-off-by: Andrew Park <andrew.park@intel.com>
* RMS ref kernel implementation with single WI
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add TC and reference func for ov_gpu_unit_tests
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add internal RMS norm op
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add transformation which fuse RMS decompsition pattern to RMS internal op
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Fix pattern for RMS fusion transformation
* Update rms ref kernel for optimization and additional planar format suuport
* Initial impl for optimized rms kernel excluding leftovers handling and case smaller than vector size
* Update the initial version to handle leftovers and case smaller than vector size
* Fuse pre decom and post comp reorders additionally
* Enable dynamic impl for rms again
* Revert fuse pre decomp and post comp reorders additionally
* Add subgraph TC for ov_gpu_func_tests
* decrease error margin for f32 data type
* update description
Signed-off-by: Andrew Park <andrew.park@intel.com>
* update test param for input shapes
* Apply comments
* Fix failed TC for invalid gamma element type
* Apply comments
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Update pattern that fuse post reorder together
* Apply comments
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
* [TF FE] Provide full support of TF1 Control flow and TensorArray ops
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add missed header for TensorArrayV3 op
* Temporarily disable GRU cell fusion
* Update src/common/transformations/src/transformations/common_optimizations/moc_transformations.cpp
* Fix a case when element_shape for TensorArrayV3
* Fix translator for TensorArrayCloseV3
* Update summarize graph with TensorArrayCloseV3
* Add layer tests for TensorArrayScatterV3, Close, Size, Array
* Fix output shape for Merge node
* Remove unused variable
* Fix translator for TensorArrayConcatV3
* Fix translator for TensorArrayConcatV3
* Add layer tests for TensorArrayWriteV3, Gather, and Concat
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add translator for GatherTree
* Fix TF FE unit-test for GatherTree
* Fix GatherTree translator
* Fix GatherTree translator to handle 1d end_token
* Fix undeclared parameter issue
* Fix GatherTree unit-test
* Add TensorArrayV3Replacer transformation
* Temporarily disable dangling transformation
* Recover RemoveMultiSubGraphOpDanglingParamsResults transformation
* Recover GRUCellFusion transformation
* Simplify check for GRUCellFusion transformation
* Use proper name for unit-tests
* Simplify translator for TensorArrayWriteV3
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix RemoveMultiSubgraphOpDanglingParamsResults transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Additional fix for remove_multi_subgraph_op_dangling_params
* Make static TI run a dynamic subgraph
* Dedicated SL test
* Change condition to respect stat shapes
* Adjust test to cover the code path properly
* Recover fallback for still failing case GNMT
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
* add m1 mac pipelines as a matrix parameter
* Update mac.yml
disable java_api because of macos arm64 - Java is not available on macOS arm64 runners
* Update mac.yml
added always condition for all tests
* Update mac.yml
* Update mac.yml
* Update mac.yml
* Update setup.py
temp commit
* Update tools/openvino_dev/setup.py
* use matrix for var
* add mxnet to extras only for x86_64
* skip failing tests
* use xfail for Python tests; add missing filter for transformations tests
* skip CPU func tests on x86_64 mac; skip some tests from CPU func tests on arm mac
* Update mac.yml
* skip tests on mac arm
* skip tests on darwin; apply review
* add more skips for python and c++ tests
* skip tf tests
* skip more tf tests; skip more Python UT stages
* rm alwayses, rm triggers, add nightly trigger
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Introduce WA to improve performance of find_port() method
* Add mutex
* Remove redindant lock
* Reduce the number of get_tensor_ptr calls
* Fixed typo
* Removed WAs from Hetero plugin
* Update model list and install cpu torch
* Move to hub tests
* Update tests/model_hub_tests/torch_tests/requirements.txt
* Make pytorch mainteiners owners of torch tests
* TorchFX: Constant value pass optimization
* Replace op.Constant with make_constant in fx_decoder
* Using shared memory for constant value passing
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
---------
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Fix command for Building with Ninja
Removing current directory from the command.
* Update docs/dev/build_windows.md
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Reference implementation for u4 constant compression from pytorch model based on bitwise ops pattern
* Fixed order of 4-bit halfs in byte
* Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case).
* Moved u4 compression to utils_quantize. Implemented not-interleaved version of u4 compression
* Removed debug output
* Added aten::matmul to the list of exceptions in may_produce_alias as a workaround for gptq models
* Added patching for gptq models applied automatically in convert_model
* WA for an inssue with u4 with earlier convert to fp16
* U4 blocked repacking for gptq patched model layout
* Deleted obsolete u4 re-packing based on aten::cat. Fixed the resulting u4 constant shape. Removed debug output.
* Revert "Switched PyTorch FE to dev mode: in case if model cannot be fully converted, give partially converted model with PTFrameworkNode's with a printed warning (normally would raise an exception in case)."
This reverts commit 0ef1455e70.
* Update src/frontends/pytorch/src/op/cat.cpp
* Check mask and shift values in u4 pattern. deque -> OutputVector for u4_compression_stack
* Convert to a given floating type instead of half in gptq patching. Better structured code.
* Code style fix
* Removed deque include
* Code style fixes
* Trailing space removed
* Fixed patched_forward and ts_decoder after unvalidated commits.
* Swap nibbles in u4/i4
* Better exception handling around jit.trace and gptq.patch_model
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
* Code style
* Revers int4 byte order
* Fixed core tests
* Fixed unguarded dynamic_cast result
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
* Fixed transformation tests
* Update src/bindings/python/src/openvino/frontend/pytorch/gptq.py
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Prevent patching of non-gptq models
* Removed extra calling of quantized weights decompression patterns
* Better detection of supported AutoGPTQ models + more diagnostics
* Accurate diagnostics in case when aten::stack has multiple axes
---------
Co-authored-by: Alexander Kozlov <alexander.kozlov@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: Evgenya Nugmanova <eva.my.link@gmail.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
* Migrate VariadicSlice to new API
- refactor to reduce bin size
* Move `get_tensors_partial_shapes` to dev API
* Use get_tensors_partial_shapes in VariadicSplit
* Remove `visit_attributes` is same as base
* Gather needs to keep the original input/output rank
- because the parameters as indices, batch_dims and axis depend on the rank.
- add input_rank to gather primitive.
* don't query on set_preferred_formats pass
-when the force_implementations is set.
-when forcing_impl is not onednn.
* Add Multinomial-13 to MO
* Add Multinomial tests for MO IR reader
* Move convert_type check
* Imports clean up
* Update pacgage BOM file
* Avoid files collision in tests
* Check for ReduceProd + SoftMax fix
* Check for ReduceProd + SoftMax fix
* Fix after moving on get_constant_max_of_type
* Extended tests and added coverage for other types
* Code optimization
* [GPU] Fixed data generation for f16 fusion tests
* [GPU] Temporary tolerance increase for failed tests on iGPU
* [GPU] Temporary skip or tolerance increase for failed tests on dGPU
* Enable device properties setting via AUTO plugin.
* Update.
* Update.
* Update.
* Update.
* Update.
* Update.
* Update docs/OV_Runtime_UG/auto_device_selection.md
Co-authored-by: yanlan song <bell.song@intel.com>
* Add for stateful model
* Fix the link
---------
Co-authored-by: yanlan song <bell.song@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Add group_normalization_kernel_selector
* Define group_normalization GPU primitive and its instantiation
* Add GroupNormalization operation builder
* Add test class for GroupNormalization operator
* Add instantiation of GroupNormalization test for GPU Plugin
* Disable GroupNormalizationDecomposition transformation in GPU Plugin
* Add GroupNormalizationKernelRef implementation
* Add GroupNormalization unit tests which cover blocked layout support
* [GPU] enable dynamic loop
- support multiple outputs
- support dynamic loop memory allocation
- support negative num_iterations
- implement calc_output_layouts
- add dynamic loop functional / unit tests
* Fix fail to check memory to set when original 1d data
- follow up code reviews
* Fix unit test failures
* Follow up code review
* Modify concat memory map creation process
* Check whether or not first input of loop is num_iteration_id
* Follow up code review
- refactoring preprocess_backedge_memory
* * Fix ci failures
* Clear custom_outputs_vec for condition
* Add num_outputs for condition and loop
* *Fix constant and param of body network have mismatched layouts
* Set consts.needsBatchInterpretation for const
* * refactoring is_dynamic in loop_inst::execute
* * remove wait_for_events in body_network execution loop
* * Remove redundant events
* * follow-up code review - modify OPENVNO_ASSERT
* * Remove redundant codes in loop_inst::execute
* * add current iteration update nodes into the ov::Model
* * rollback some codes for the performance degradation
* Migrate TopK to new API
* Refactor compare_max for TopK
* Unify check of k for const and non-const input
* Update src/core/include/openvino/op/util/evaluate_helpers.hpp
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
* Move `get_tensors_partial_shapes` to dev API
---------
Co-authored-by: Tomasz Jankowski <tomasz1.jankowski@intel.com>
* Refactor shape_size util to reduce bin size
* Make `check_new_args_count` non-template function
* Use as not template check_new_args_count
in multi-nominal
* Migrated all subgraph tests after quantization graphs
* Fixed skip tests config
* Add alias for NPU
* Try to fix NPU
* Remove unsupported precisions
* Remove GNA test
* Remove more tests
* Use precise op version - Parameter
* Use precise op version - Constant
* Use precise op version - Result
* Use precise op version - Add, Concat, Subtract
* Use precise op version - NMS
* Use precise op version - TopK, Broadcast
* Use precise op version
* Include openvino headers in quotes
* Use precise op version - LSTMCell
* Fix code style
* [TF FE] Support TF 2.14 and add OnesLike translator
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update tests constraints
* Update open_model_zoo
* Adopt TF Lite test to 2.14 TF
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Support TF Lite layer tests for diffrent TF versions
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* only skip test if mac
* unskip
* unskip trigger
* skip for onnx fe as well
* do not skip
* return skips and unskip test_backend in Python API 1.0
* rm pr trigger
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* skip gna-related tests on mac; rm unused in pipelines
* rm unused env var
* use if instead of commenting out
* add name
* return pr trigger
* rm pr trigger
* enable eltwise skip test
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* enable skipped test of onehot
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* remove shapeof OD test. cpu node shape_of don't support it
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* enable skipped test case of strideslice
remove default stride test
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* enable skipped test case of matmul
remove some test cases, custom matmul only support some rank
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* fix eltwise autob issue
powerstatic don't get the autob attibute from origin op
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* Revert "fix eltwise autob issue"
This reverts commit 1139296411.
* Revert "enable eltwise skip test"
This reverts commit c9f0a6f225.
* disable eltwise none autob test
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* add error message for OPENVINO_ASSERT
go NgraphShapeInfer branch in matmul node
when input0 rank is not equal to input1 rank
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* create ticket for EltwiseShapeInfer and skip the test case
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* revert test case order in matmul
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
---------
Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
* add new operations as unary
* get unary as input(0) instead of iterating pattern map
* add CumSum + unit tests
* add Tile + unit tests
* add tile
* fix ts_tile
* code review fix: use ADD_MATCHER
* fix bug CI tests
* Optimize CompressQuantizeWeights transformation
- remove CoordinateTransform usage from FakeQuantize reference implementation
- move ZeroPointOptimizer functionality inside CompressQuantizeWeights
- compute scale and zero point in the same loop
Ticket: CVS-119273
* review comments
* clang format
* fix comments
* Do not normalize negative indices for Gather v8
* code style fix
* added transformation test with accuracy check for Gather-v8
* removed GatherNegativeConstIndicesNormalize transformation at all
* ConvertGather8ToGather7 conversion: added more checks
* Introduced shared Gather8withIndicesDataLayerTest: added CPU, GPU instances
* code style fix
* small fix
* review fixes
* do negative indices normalization if possible
* code style fix
* refactor cpu test instances
* code style fix
* add test data for SRF on Linux
* update cpu map detection for Ecore only platform
* update test data for smoke test of streams generation
* update test data
* [GPU] oneDNN3.3 integration.
* Supports new formats from oneDNN3.3 requires.
* Fix Perf regression because of the wrong mvn kernel selection issue.
modnet_webcam_portrait_matting.int8
person-reidentification-retail-0248.int8
* support undefined onednn tag for using any tag instead.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Merge opt_kernel into reference
* Remove get_default_order
* Use ov:: in jit generators
* Remove unused template function
* Add reshape parameter for consistency with Op spec
* Add brief description and such
* Remove unused param from reshape ref
* Use C++ casting
* Remove ngraph namespace from operations without namespace
* Try to fix build
* Additional fixes
* More fixes
* More fixs
* Fix reverse op
* Fixed tests
* Throw an exception if somebody tries to reallocate tensor
* Revert "Throw an exception if somebody tries to reallocate tensor"
This reverts commit 8e06d6d576.
* Remove python test
* Revert "Remove python test"
This reverts commit 37b12148d3.
* Changed evaluate model behavior
* check dirs
* cmake, build, instlll
* use make
* use build dir
* use target
* add missing
* execute conformance tests
* correct path for requirements
* setupvars
* add API conformance
* conformance as a separate job; install all necessary files
* uncomment
* merge
* install deps
* use matrix, upload expected failures to gh cache
* use 8-core
* use the same paths
* uncomment
* comment
* change
* use csv
* add exit if there are failed tests
* always upload logs
* check dir
* use another dir for expected_failures
* upload always
* rm
* new key
* rm unused
* change
* update
* update
* rm unused
* do not exit if update is set
* copy file
* do not fail in failures check
* use specific branch
* run clean
* add expected failures
* uncomment
* comment out
* correctly add failed tests to fix_priority; check for unexpected failures in case of an update
* use azure runners for conformance
* use 4-core
* uncomment
* comment out
* split deps installation
* add missing deps for setup-python
* print error
* install certs
* do not verify ssl
* add ca-certificates install
* uncomment
* comment
* pack artifacts, rm unused deps
* rm unused dep
* always pack artifacts
* rm for func tests
* use less cores
* use var
* store int
* do not exit if non-zero code
* only 4 core
* use gh runner
* use sudo
* add missing sudo
* use expected failures
* uncomment
* comment
* create fresh expected failures
* use expected failures
* use specific ref and repo
* use expected failures
* uncomment
* comment out
* check caches
* fix str
* rm unused
* restore by restore key
* create dir
* use array
* use diff path
* mv after each download
* add sleeping and more logs
* add saving of hash table
* change
* uncomment
* comment
* download cache entries
* check dir
* use better dir
* uncomment
* rm unused
* add skip configs
* update lists
* rm unused dir; add docs
* rm unused
* rm hardcoded repo ref
* rm unused options; use better name for job
* rm unnecessary dir creation
* GPU primitive and kernel changes to support ScatterElementsUpdate-12
* Add single-layer test for ScatterElementsUpdate-12
* Unit tests:
- fix formats test - add separate data type for indices;
- add tests for Reduction.
* add pipeline
* rm triggers
* address comments
* use uninteractive as env
* rm triggers
* rm unused testdata
* use better concurrency group
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* use aks runner
* correct path
* provide path
* add missing cmake options; rm unnecessary dir creation
* use image from private docker
* split OV and plugin cmake & build; do not fail on warning for plugin build
* use different build_dir for nvidia plugin
* add missing options
* rm unnecessary options; add target for build
* Apply suggestions from code review
try fix for NVIDIA plugin
* Apply suggestions from code review
revert to default contrib repo, used ccache for CUDA files
---------
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* [GPU] Added more debug print for prepare input/output
Skip enqueuemempcy for zero count tensor
* Fix mem alloc limitation
* Simplify memory check condition
* Fix not to wait for null event
* Additional fixes
- apply review comments (disable 0 size memcpy for other mem types) && return user event instead of nullptr
- fixd get_max_memory to return cpu memory instead of global memorysize (Because in windows, max_global_memory returns shared gpu memory instead of physical memory size)
* Fixed functest failure
* [C API] support string size for char pointer
* rename function name
* Add deprecated flag
* Add macro to ignore deprecated
* Fix build error in windows
---------
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* [TF FE] Support TF1 While Control flow
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-style fix
* Update API for OpPlace to store back edge
* Fix build: no rvalue by reference passing
* Fix build issue: correct type
* Fix TF FE unit-tests
* Apply code-review feedback: remove unused vars
* Fix fusing complicated case of TF1 While
* Remove unused variable
* Update MO unit test
* Fix layer tests for While
* Handle Switch and NextIteration nodes connected directly
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Migrate ReduceL1, ReduceL2 to new API
- add some new utils which are deprecated
* Hide helper functions from public API
* Migrate reductions ops to new API
* Migrate get_constant_from_source to dev API
* Rename ref max to reduce_max
* Rename ref min to reduce_min
* Rename ref mean to reduce_mean
* Rename ref sum to reduce_sum
* Rename ref product to reduce_prod
- minor optimization in ReduceProd operator
* Restore custom isfinite for ov float types
* Fix type name in reduce_max.hpp
* Add missing include in shape_util.hpp
* Make count same type as data type in reduce mean
* Correct reduce sum doxy comment
* [Hub Tests] Move Hub tests to self-host runners
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Not all directories possible to clean up
* Move Hub models tests to AKS nodes
* Add install actions python setup
* Add ca-certificate deps to install
* Fix bug for PT models
* Update .github/workflows/linux.yml
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Update .github/workflows/linux.yml
* Revert changes for PT models tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Install correct version of OV
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* fix compilation issue on Windows arm64
* fix build issue
* fix build issue
* disable clang compiler for Windows on ARM
* disable clang compiler for cpu plugin
* Update CMakeLists.txt
removed fatal_error from cpu cmake
* Update features.cmake
disable Intel CPU plugin by default on Windows ARM64 with clangl-cl compiler
* update comments
* fix build issue
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* [transformations] WeightsDequantizeToFakeQuantize: Extend pattern matching with the case when both Subtract inputs are Convert
* [transformations] WeightsDequantizeToFakeQuantize: Added new tests to cover the extention added to pattern match
* Fix review comments
* Use FindPython3.cmake
* Fixed compilation on macOS 14 with new core development tools
* Try to use Python3_SOABI instead of PYTHON_MODULE_EXTENSION
* Use Development.Module
* Keep specifying only Python3_EXECUTABLE
* Print PYTHON_MODULE_EXTENSION
* Added check for minimal cmake version for python API
* Returned Python3_INCLUDE_DIR for cross-compilation case
* Try to allow cmake older than 3.18
* Use build python interpreter to check cython dependency
* revert changes in .ci/openvino-onnx/Dockerfile
* removed unused code
* Fixed issue with variables scope
* Experiment: remove include dirs
* Corrected docs
* Use pybind11 function to set extension
* Revert "Experiment: remove include dirs"
This reverts commit 6f7f90211c.
* Refactor ConvolutionBackpropDataLayerTest, ConvolutionLayerTest, DeformableConvolutionLayerTest (#19810)
* Refactor ConvolutionBackpropDataLayerTest
* Refactor ConvolutionLayerTest
* Refactor DeformableConvolutionLayerTest
* Apply comments
* Apply comments
* Fix
* Updated minimum cmake version for Windows
* Simplified check
* Removed useless message status
* Use puiblic option
---------
Co-authored-by: Oleg Pipikin <oleg.pipikin@intel.com>
* Add EnableShapeOfConstantFolding transformation
Transpose sinking (that is used in TF frontend) disables ShapeOf constant folding
which prevents some optimizations further in the pipeline.
This patch introduces EnableShapeOfConstantFolding that removes DisableConstantFolding
from ShapeOf nodes.
Ticket: CVS-118890
* add description
* review comments
* headers
* Added support of shapes and types from paddle, torch and tf.
* Removed changes from requirements.
* Corrected test.
* Moved helper methods to utils.
* Separated tests by frameworks.
* Removed changes from complex_params test.
* Refactored Linix GHA pipeline
* Move ONNX tests to Unit
* Install tests together with openvino
* Added checks for GNA usage in docs python snippets
* Install openssl for working pip
* Disabled onnx_dyn_shapes_expand_1_dyn_shape ONNX test
* exclude dyn shapes from onnx
* Migrated samples tests
* Fixed ONNX tests
* caches for PT tests dependencies
* Build Contrib separately from main OpenVINO
* More fixes
* Added python install for samples stage
* Installed manifests for ONNX FE tests
* Try to avoid libgl1 dependency in tests
* Clean-up
* Fixed issue with manifests in ONNX tests
* Try to build debian packages in GHA
* Fixed debian packages build for multi-config generators
* changed unit tests
* add unit tests; fixed
* code review fixes
* remove debugging
* remove debug code
* add tests to ov_cpu_func_tests
* add negative padding tests to ov_gpu_func_tests
* code review fixes: change point to check positive indexes; check Pad mode in the list of currently supported
* code review fix: move check in canBeTransformed
* [TEMPLATE] Disable ReferenceTopKTest topk_min/max_sort_none on MacOs
* Apply suggestions from code review
disabled tests only for macOS x86_64, because they pass on Apple M1 / M2
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Refactored Linix GHA pipeline
* Move ONNX tests to Unit
* Install tests together with openvino
* Added checks for GNA usage in docs python snippets
* Install openssl for working pip
* Disabled onnx_dyn_shapes_expand_1_dyn_shape ONNX test
* exclude dyn shapes from onnx
* Migrated samples tests
* Fixed ONNX tests
* caches for PT tests dependencies
* Build Contrib separately from main OpenVINO
* More fixes
* Added python install for samples stage
* Installed manifests for ONNX FE tests
* Try to avoid libgl1 dependency in tests
* Clean-up
* Fixed issue with manifests in ONNX tests
* Migrate ops evaluate
* Remove using ngraph and std from ops
* Use OPENVINO_ASSERT instead of NGRAPH_CHECK
* Move `shape_util.hpp` to `dev_api/openvino/core/`
* Remove visit_attributes, same as base impl
* Fix build issues
* Fix build issues
* [workflows/liux] Trying AKS runners
* Remove sudo usages
* Fix repositories path
* Well, adding debug output
* execution bit?
* more debug output
* using bash...
* We should use shell syntax for env variables in 'run' sections
* again, may be we should use bash...
* fix cloning path for openvino repo
* more debug output
* Trying to populate WORKSPACE var in other way
* Trying different WORKSPACE
* export ALL variables for samples tests
* Trying to put openvino build into correct directory
* Replace more occurences of BUILD_DIR...
* Use shell expansion instead of Actions syntax in run steps
* Skip unit test failures
* Oops, there's no pytest for MO unit tests
* Oops, typo
* Fix LAYER_TESTS_INSTALL_DIR variable expansion
* Fix LAYER_TESTS_INSTALL_DIR for new stages
* Replace all the varibles
* Comment out CCACHE_DIR
I want to test worst-case scenario
* Comment setup ccache step
* Revert "Comment out CCACHE_DIR"
This reverts commit 046663a9f1.
* Just checking something
* Update with new stages from master
* One more stage to fix
* Fix INSTALL_DIR again (new stages were merged to master?)
* Remove concurrency limit (for test purposes)
* Trigger one more pipeline
* Disable concurrency settings
* Trigger another build
* Disable ccache (for testing purposes)
* trigger another build to measure duration without ccache
* And one more test commit
* Return back ccache
* Testing new image
* Enable concurrency back
* Using new image
* And another image
* Trying build on the clean image
* Install git
Upstream ubuntu image doesn't have git
* Oops, forgot apt-get update
* Upstream Ubuntu image, but using ACR cache
* Remove debug output
* Use Ubuntu image from ACR cache for the rest of the stages
* Display version of ccache
* Swith to ubuntu 20 temporarily (I hope)
* Set CCACHE_TEMP_DIR outside of shared space
* Revert "Swith to ubuntu 20 temporarily (I hope)"
This reverts commit 2e53109d83.
* trying ccche nohardlink option
* Trying to disable ccache file cloning
* fix CCACHE_TEMPDIR
* Install git to Python unit tests stage
* Install libsnappy-dev as tests require it
* Install python to test stage
* Fix env vars syntax in few places
* Update docker image name
* Increase timeout for CPU tests
* Debug output - trying to find out what's wrong with test reports
* Change file name in debug-output 'find' command
* let's try shell syntax for upload artifact action...
It most likely won't work but I see strange output in the logs:
Error: No files were found with the provided path: /__w/openvino/openvino/install/tests/TEST*.xml
/runner/_work/openvino/openvino/install/tests/logs/failed/*.log
/runner/_work/openvino/openvino/install/tests/logs/crashed/*.log
/runner/_work/openvino/openvino/install/tests/logs/hanged/*.log
/runner/_work/openvino/openvino/install/tests/logs/interapted/*.log
/runner/_work/openvino/openvino/install/tests/logs/disabled_tests.log. No
artifacts will be uploaded.
So the first path is translated to the path inside the container
correctly and the rest are not O_o
* Revert "let's try shell syntax for upload artifact action..."
This reverts commit eb19b4f319.
* Leave just disabled_tests.log (as it is the only one present)
* Use my own directory instead of github.workspace
Inside-container path translation doesn't work for upload-artifact
actions (it works only for the first path, but not for the rest)
* Revert "Leave just disabled_tests.log (as it is the only one present)"
This reverts commit 4321827880.
* Okay, let's try hardcoded workspace path...
* Okay, path should be inside "official" workspace. Hardcode it
* Missed some variables, replace github.workspace to hard-coded value
* Remove debug output 'find' command
* Remove old-style cache setup
* Clean ccache stats after cmake
* Change docker image to the one with non-root user
* Oops, switch to Ubuntu 20
* Playing with concurrency group name
* Oops, forgot sudo for ninja installation
* Revert "Oops, forgot sudo for ninja installation"
This reverts commit a7e1730279.
* Revert "Oops, switch to Ubuntu 20"
This reverts commit 19fe75f4af.
* Revert "Change docker image to the one with non-root user"
This reverts commit 553116492d.
* Use Ubuntu 20 DockerHub image
* Trying to force apt-get packages not to wait for user input
* Make DEBIAN_FRONTED for all stages
* Update reason of skipped MO unit test
* Add DEBIAN_FRONTED to all jobs
* Fix environment variables expansion in newly-added stages
* Update .github/workflows/linux.yml
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Update .github/workflows/linux.yml
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Fix review comments
* More review comment fixes
* Remove DEBIAN_FRONTEND from Python Models job
* "Get tools versions" step is absent on master
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* add cpu pinning on windows
* remove pinning limitation on windows
* only support the machine with one numa node
* fix code style
* fix build error on macos
* set mask initial value
* fix test failure on window
* fix build failure on macos, add limitation on windows machine with two sockets
* fix test failure on windows
* fix test failure
* fix comments
* [OVC] do not parse inputs
* fix unit-tests
* remove redundant lines, add test case
* add one more unit-test
* skip None values
* replace str with List in test_mo_import_from_memory
* corrected type hints, added a safety assert
Eltwise order of the supported primitive descriptors affects the
performance of the model.
Often only one of the port descriptors matches with the layout
of the parent descriptors, i.e. when two parent ports have mixed
layout "nchw nhwc".
So either nchw or nhwc layout will be used for the eltwise node
and reorder will be used for either of the ports.
The shape of the ports also can be different (when one of the inputs is
broadcasted). So reorders on different ports have different
performance impact.
The layout of the eltwise node child has an effect on the performance
as well, since it may or may not require reorder on input.
* Optimization for gemm & fc in iGPU.
FC: fake alignment for 16 is better in iGPU.
Gemm: permute + gemm_tiled_opt is better than transposed_input + gemm_ref kernel for unaligned shapes to 16. Note that this is an temporal optimization and will be removed once the final solution (i.e., support unaligned transposed input shape in gemm_tiled_opt kernel) is availalbe.
* Fix unittest
* Fix for model_cache
* Fix unittest
* Init rotated non max suppresion spec
* Add opset13 docs
* Apply minor refactor from review
* Update boxes definition
* Update example format from cpp to xml
* Add version in op list
* Add clockwise attr to the example
* Align indent
* Remove redundant input from example
* Add steps of iou_rotated
* Add default values for attributes
* Drop box encoding attribute
* Rephrase nput description
* Applay grammatical suggestions
* use input memory buffer as output memeory when input1/2 are empty
* fix wrong rebase
* add func test
* implement in on_execute()
* remove deleted function definitioin
* remove unused header files
* fix include error
* update condition of empty input check
* Symbolic shape inference and graph optimizations
- Prepares a place in CommonOptimizations pipeline for symbolic optimizations
- Introduces symbolic propagation and symbolic optimizations for ChainedMaximum, NopBroadcast and shape sub-graph optimization
- Introduces utility runtime info for TableOfEquivalence passing and disabling of value invalidation during shape inference
* Executes NgramFusion in a symbolic environment. Relaxes Ngram fusion pattern utilizing symbolic knowledge
* Remove debug model visualization
* rt_info copying to new Add operation
* Fix visualization and place validation in nicer place in symbolic transformation
* Fix Slice operation not to propagate labels if input and output dimension is fully dynamic
* Covering Vladislav comments
* Replace value invalidation followed by validation to revalidation since it does the same thing
* Adding back invalidation of cached values to Symbolic Propagation pass
* Fix StridedSlice label propagation. Code style
* Update src/common/transformations/tests/symbolic_transformations/nop_broadcast.cpp
* [GPU] Fix canonicalization for fused dep's shape
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Update TC to reproducible on the latest master
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Fix custom canonicalize shapes for Gather
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Avoid Constant casting / printing when OV_VISUALIZE_TREE_CONST_MAX_ELEMENTS==0
Cast only requested amount of elements in Constant::cast_vector<>
* Refactor
* Revert style back
* Fix signed/unsigned comparison
* test
* Style
* Style
* [GPU] Fix gpu functional test failures
* set m_max_batch to 1
* add debug log for condition operation
* Add debug logs for condition and constant
* To fix zero byte allocation issue, Convert zero dimension to 1 dimension in constant
* Add the code to check if output shape is dynamic in split offset calculation and checking allow_new_shape_infer in program_builder
* Add unit test for fix checking output shape
* Add test case for zero dimennsion allocation and debug message
* Fix build failure for condition unit test
* Follow up code review
* [TF Hub] Cover TF Hub use cases with adoption to OpenVINO
This is necessarily to demonstrate support of models programmed with TF Hub API
through OV notebooks.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Preserve original keras input and output tensor names
* Add tests with TF Hub API models
* No KerasLayer handling
* Handle specific signature
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* introduce linux cc
* disable some triggers
* check dirs
* use another model
* return model
* rm triggers
---------
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* changed OS build runner to 20.04
* fixed PR
* changed version for test jobs
* Updated title
* Update .github/workflows/linux.yml
* Update linux.yml for PT models
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* lpt transformations: transition to api 2.0, ngraph -> openvino
* use ov namespace for lpt transformations
* fix low_precision usings
* includes refactoring
* delete RecurrentGraphRewrite and RecurrentMatcher as unused classes
* use ov header for itt; delete the disabled test
* delete the unused function
* suppress doxygen warning
* fix link in the documentation
* Switched ONNX to 1.14
* Updated IR_VERSION in the tests
* Assigned an extended tests to issues
* Switched ONNX to 1.14.1
* Slightly relaxed requirements for ONNX 1.14.1 and updated conan.lock
* [TF Hub] Set seed for input data generation and fix integer input data
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Clean-up workflow
* Update precommit model scope
* Avoid legacy generator
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* GPU primitive and kernel changes to support Pad-12
* Exclude Pad-12 from GPU transformations pipeline
* add unit tests
* add single-layet test for Pad-12
* [TF Hub][GA] Use Ubuntu 20.04 for TensorFlow Hub Models validation and Separate job
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply review comments: ubuntu-20.04 use and install deps
* Simplify validation pipeline for TF Hub Models
* Remove extra deps installations
* Remove not needed code
* Try to fix
* Try 22.04
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Handle negative values in GroupedSliceToVSplitOptimization
CVS-118897
* change the way of getting slice inputs
* clamp value
---------
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
* Migrate ReduceL1, ReduceL2 to new API
- add some new utils which are deprecated
* Add missing include
* Remove debug message
* Hide helper functions from public API
* [TF Hub][TF FE] Preserve outputs of ConcreteFunction from signature and their names
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix naming and complete TODO
* Apply code-review: extra assert to check input_signature
* Fix inputs for fw
* Fix input data preparation and import convert_model
* Correct variable detection among all inputs
* Handle special input and output signature
* Fix adjust_saved_model_names
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Move BroadcastTransition to MOC
Broadcast that could be eliminated by BroadcastElementwiseFusion are moved down the graph
(by BroadcastTransition registered in the plugins). That prevents BroadcastElementwiseFusion
to eliminate them.
Ticket: CVS-118899
* dont count const layers
* remove virtual inheritance
* [TF Hub][GA] Set correct scheduler and model scope for regular validation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove empty line
* Update .github/workflows/linux.yml
Co-authored-by: Andrey Kashchikhin <andrey.kashchikhin@intel.com>
* Correct a path to output html report
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Andrey Kashchikhin <andrey.kashchikhin@intel.com>
* Reduce number of rank checks
* Preserve data shape if signal_size input is not provided
* Add bounds propagation on fft input
* Improved preserving bounds on fft input
* Remove size_t rank cast and have_axes variable
* Check refactor
* Use ge helper for rank comparison
* Make bounds constexpr
* Pass raw pointer instead of unique_ptr ref
* Use normalize_axes helper
* Ensure to call set label if it's not zero
* [TF Hub][TF FE] Use multiprocessing based tests for TF Hub validation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix import and initialization
* [TF Hub][TF FE] Make TF Hub validation more robust and add convenient marking for failing cases
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Moved inference unit tests to new API
* Added infer request and variable state
* Try to fix LTO
* Try to avoid warning from gmock
* Try to fix azure build
* Try to fix Windows build
* Comment all variable_state_test file for future investigation
* Start migration of caching tests to new API
* Removed legacy API from mock_plugin
* Fixed more tests
* Remove redundant code
* Enable more tests
* Move all tests, need to reenable tests
* Revert incorrect change
* Cosmetic changes
* Fixed AUTO BATCH tests and disabled hetero tests
* Fixed crash in HETERO tests
* Restored opset1::Reshape label peropagation for -1 special value
* Lets opset1::Reshape keep same shape infer. Makes FindBatch transformation keep labels in output shapes of Result node
* uses Parameter from correct namespace
* Add all scope of models for testing
* Fix signature in test
* Fix workaround with output tensor names
* Fix unknown rank input resource case
* [TF Hub][TF FE] Fix TF Hub model validation and extend precommit
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove unneeded comments
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [GPU] Fix input feature map indexing with pad and batch indices for ROIAlign
* Fix failed TCs for ov_gpu_func_tests
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Fix to do batch interpretation for inconsistency between ROIALign input and const 1D tensor
Signed-off-by: Andrew Park <andrew.park@intel.com>
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
* fix parsing output
* use always coma backspace to separate outputs ', '
* update docstring
* call parser only for ovc cli tool
* docstring correction
* separate docs for cli tool and convert_model; other minor changes
* drop redundant arg from cli_parser_test.py
* more solid test cases added
* remove redundant argv.framework from cli_parser_test.py
* shape correction in concat
* Apply suggestions from code review
fix: coma -> comma
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* apply review suggestions
* remove extra ')'
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Moved inference unit tests to new API
* Added infer request and variable state
* Try to fix LTO
* Try to avoid warning from gmock
* Try to fix azure build
* Try to fix Windows build
* Comment all variable_state_test file for future investigation
* TransposeSinking - add support for ShapeOf
It's often in the model that transposes have two consumers: ShapeOf and
another transpose and with ShapeOf absent - the transpose pair could be eliminated.
This patch's approach is to propagate a transpose through ShapeOf by creating
"ShapeOf->Gather" subgraph and replacing ShapeOf's input with transpose input.
Ticket: CVS-118896
* enhance docs
* [PyOV] clean up in tests
* use generated model in tests
* fix ci
* return back fild
* fix ci 1
* fix ci2
* update
* move models
* return back deleted test
* move model creation from conftest
* fix ci
* fix ci
* Use API 2.0 in operators evaluate
- Drop ngraph namespace in ops
- Refactor reference implementation for modified ops
* Apply code style
* Fix build issue in reference impl
* Fix code style
* Fix compile warnings
* Add inputs check and set output shape in evaluates
* trying with pytest
* update as per suggested
* pytest testing on compress_quantized_weights_test
* resolved warning of assert on tuple
---------
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Create performance.yml
create new performance issue template for customers to report issue that related to performance
* Update performance.yml
Amend based on latest comment
* Update performance.yml
* Update performance.yml
amend based on suggestion
* API 2.0 replace ngraph headers and namespaces in CPU, LPT etc transformations tests
* revert the incorrect renaming
* function -> model in arm transformation tests
* Reuse common shape validation for fft base
* Align helper names
* Common test class for fft ops
* Move all (I)DFT test cases to the common test class
* More test cases for param axes
* Init labels validation
* More label tests
* Labels validation for non const signal size
* Init label tests for IRDFT
* More label test for irdft
* Labels tests for RDFT
* Remove duplicated tests
* Rename common validation file
* Rename shape infer tests file
* Use node shape infer check
* Headers order alignment
* Add const to the test params vector
* Use this make_op
* Use OV_EXPECT_THROW in common fft tests
* Use OV_EXPECT_THROW iin rdft an irdft tests
* Pass input shapes and use SHAPE_INFER_CHECK
* Shorter error messages
* Update to use ov namespace in typeprop tests
* Move files to new directories
* Use quotes for openvino includes
* Provide proxy calls for transition
of dependant components.
* Correct includes style
* Redo proxies
* Fix deprecated
* Move aliases to proxy files
* Apply code style
* [TF Hub][TF FE] Support TensorListLength and TensorListResize operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add test with empty tensor list
* remove assert
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [conformance] Fix same name for models
* add model name
* Update hash wwith input_info and graph_priority
* Fix double cache
* Read meta from file
* temp to check
* Move loop with caches and add attr to rename
* [conformance] Enable local cache using in SubgraphsDumper tool
- [scatter_update] Use input index for input buffer instead of output index
- [concat cpu impl] Sync input layout and mem_ptr when input host tensor creation
- Add unit tests for scatter_update and concat cpu impl
* Fix error in CVS-115961, caused by wrapper covert 1.0 req to 2.0 req create 2 more threads
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* Eable the test of compareAutoBatchingToSingleBatch with batch size 4 & num req 64, after fix issue 115961
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
---------
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [CONFORMANCE] Extend conformance runner to use in GA with expected_failures filelist
* fix
* exclude failed tests from run in case without update
* Small refactoring
* [JAX][TF Hub][TF FE] Introduce JAX layer tests and support of XLA operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix JAX layer tests infa
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Extend run for JAX layer tests
* Use ovc convert_model
* Fix translator and extend layer test cases
* Exclude jax testing on Windows
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Introduce model reader which works only with new API
* Fix GNA compilation
* Removed old code
* Fixed Windows build
* Remove legacy headers from core_impl
* Fixed caching tests if plugin on legacy API call ReadNetwork
* [TF FE] Use regular Convolution in case dynamic input channels
This solution is aligned with the legacy frontend but it has limitation.
This is a temporal solution until the core obtains ShapeOf evaluator.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove unused variable from the test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix unit-test
* Update mo unit-test
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add `recreate_and_infer_in_thread` memleak test
* Add custom threshold for memleak test
* Update tests/stress_tests/common/ie_pipelines/pipelines.cpp
For some (e.g. quantized) models, W, R, B inputs to sequence operations
are not direct constants, but a subgraph that can be constfolded later.
In that case, we don't need to have them as inputs to TensorIterator,
they can be pulled into its body instead.
Ticket: CVS-117544
* WIP Postpone fp16 in CompressFloatConstantsImpl
* Apply suggestions from code review
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* WIP: Compression to FP16 in Serialize
* Prepared for efficient fp32 to fp16 conversion
* Update src/core/reference/src/runtime/reference/convert.cpp
* Called real slow reference implementations in the place where the optimized versions are supposed to be implemented
* Code style
* Fixed 0 values in the fast f64 to f16 compression
* Optimized convert_from_f32_to_f16_with_clamp
* Added optimized f32->f16 instance of change_constant_precision
* compression transformation Python test
* use tmp dir, minor corrections
* Update src/bindings/python/tests/test_transformations/test_compression.py
* Update src/bindings/python/tests/test_transformations/test_compression.py
* style fix
* define rt_info for postponed_fp16_compression
* remove redundant class
* fix temp dir for Win in test_compression.py
* update definitions in convert.hpp
* Update implementation in convert.cpp
* Update serialize.cpp
* Update compress_float_constants.cpp
* added macros for ARM/non_x86 in convert.cpp
* fix macros in convert.cpp
* change fixme placement in serialize.cpp
* style_fix
* Update src/core/reference/src/runtime/reference/convert.cpp
* style_fix
* Optimized count_out_of_f16_range
* Code style
* Revert unused
* Update src/core/src/pass/serialize.cpp
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Update src/core/reference/src/runtime/reference/convert.cpp
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* use optimized convert_from_f32_to_f16_with_clamp for non postponed
* minor corrections
* Update src/common/transformations/src/transformations/common_optimizations/compress_float_constants.cpp
* Update compress_float_constants.cpp
* Switched mo and ovc to save_model instead of serialize to leverage performance improvements in fp32->fp16
* Applied minor code imporvements to address review feedback
* Minor changes in code
* Update tools/ovc/openvino/tools/ovc/main.py
* Apply suggestions from code review
* Fixed failed test in case when both usual xml compression and fp16 compression are applied simultaneously (disabled for now)
* Added description for CompressFloatConstantImpl postponed parameter
* Description of postponed parameter for CompressFloatConstants
* Reverted switching to save_model in mo as the compression can be applied not only via CLI and old code should be kept for Python path (not applicable for ovc)
* Removed remining committed test artefacts and reverted remaining changes in mo
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: dmitrygo <dmitry.gorokhov@intel.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Co-authored-by: Pavel Esir <pavel.esir@intel.com>
Co-authored-by: Pavel Esir <pavel.esir@gmail.com>
+ Fixed black output image by Nan output.
+ Resolved fp16 overflow of gemm primitive before softmax
+ Added fused post ops of clamp activation not to get inf which caused Nan output
+ Added new pass modify_fused_ops
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* Added support of tuple in input, removed type syntax from OVC tool.
* Removed type syntax tests.
* Apply suggestions from code review
* Method annotation corrected.
* Type annotation corrected.
---------
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* TorchFX caching bugfix and improvements
* Fixed inconsistent env variable for Backend device
* Identify PyTorch FrontEnd Decoder type
* Added import statement in init files
* Registered ts_openvino as a separate backend
* Added caching fix and removed extraneous code
* Changed the name of ts backend
* Fixed issue with local temporary object
* Removed import statement from init files
* Changed the documentation
* Added get_supported_ops method for decoders
---------
Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com>
Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
* Added check that nncf was imported.
* Added check that nncf was imported in MO.
* Added check that nncf was imported in MO.
* Apply suggestions from code review
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* Removed not needed import.
* Pylint fix.
---------
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* add set property case
* add set property case
* free key
* fix code style
* add test of get_property() for ov_property_key_auto_batch_timeout
* get_property first
* add device name
* device_name.c_str()
* reset name
* test set_property
* remove comment
* pass local
* get property first
* fix bug
* use device from GetParam
* fix char to string
* use EXPECT_STREQ(target.c_str(), result);
* fix comment
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Moved mmap to openvino/util as a shared functionality
* Enabled MMAP for SavedModel and MetaGraph
* Fixed CMake
* Fixed a lost line
* Simplified code for compilers
* Aligned with an actual master
* Enabled mmap by default and added test
* Suppressed warning, added test for MetaGraph, additional messages
---------
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* Removed 'example_output' from ovc and ovc.convert_model, used output for this purpose
* Update tools/ovc/openvino/tools/ovc/convert.py
* Update tools/ovc/openvino/tools/ovc/convert_impl.py
* Reverted mo parts not affected by remove of example_output
* fix PDPD convert_model tests
---------
Co-authored-by: Xiuchuan Zhai <xiuchuan.zhai@intel.com>
* [PT FE] Fix behaivior of convert_partially method to align with new description
* Simplify
* Do not add a message to exception of get_converted_model
* Update src/frontends/pytorch/src/frontend.cpp
* Supress normalize exception only when model is valid
* Create TranslateSession in own scope
* [TF FE] Fix conversion of SavedModel with only numerical names operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Compare tensor names in the unit-test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
The Python version uses `app_inputs_info` to represent different input configurations, but the C++ version extends that use case and uses `app_inputs_info` to represent different input images as well. That means that the assumption that if `app_input_info.size() > 1`, then input shape is dynamic, doesn’t always hold for C++
Ticket 117673
* Added shape and type infer for result nodes in MOC transformations.
* Clang format.
* Added validate_nodes_and_infer_types() pass at the end of MOC pipeline.
* Clang format.
* Added test.
* Clang format.
* [FIX][CPU] Convert CF: added convert childs number check
* code style fix
* WIP: [CPU][TESTS] Convert CF: Added subgraph test with two outputs for Convert node
* MatMulDecompressConvertTest refactoring + instances fixes
* removed KeepConstAndDecompressionForMatMul pass and added cpu callback for KeepConstAndDecompression
* MatMulDecompressConvertTest2: added graphs and small problem description
* small review fix
* Fixed output_model logic.
* Removed InputCutInfo, disabled input cut in ovc.
* Disabled output cut, added tests for setting shapes or types for not all inputs.
* Returned support of numpy type.
* Separated MO and OVC python API tests.
* Small corrections.
* Added output dir test, exceptions test.
* Tests fixed.
* Corrected extension param description.
* Corrected input description, minor code corrections.
* [PT FE] Use weight share switch in frontend
* Return static for function
* Update src/bindings/python/src/openvino/frontend/pytorch/ts_decoder.py
* Fix issue with quantized constants
* Add tests for shared
* Change fused_names algo -> cut subgraphs
* Added extractor name to serialization dir + meta_info
* Uncomment log
* Add this_op_cnt to model_info, fix in_info for model
* Replace clone node second time to relace input node
* fix small problem
* small fixes
* Switch off repeat extractor
* remove model serialization
* fused_names
* change default device in fused_names extractor
* fused_names
* Small speed up
* Move replace of const by param to cache
* Move alignament of in_info to extractorManager
* Sort model by size (check mem fragmentation)
* Fix problem with opset12
* Update manager.cpp
* Serialize cache in case of long
* Add test
* Update graph_cache.cpp
* Update graph_cache.cpp
* Graph cache size
* test other approach
* remove extra
* Fix issue with replae
* try with 1gb limitatiom
* to merge
* revert
* Change `VPUX`/`VPU` occurrences to `NPU`
* Switch `HARDWARE_AWARE_IGNORED_PATTERNS` VPU to NPU
* Rename `MYRIAD plugin`
* Rename vpu_patterns to npu_patterns in tools/pot
* Rename vpu.json to npu.json in tools/pot
* Rename restrict_for_vpu to restrict_for_npu in tools/pot
* Change keembayOptimalBatchNum to npuOptimalBatchNum
---------
Co-authored-by: Dan <mircea-aurelian.dan@intel.com>
* Fix issue with kwargs in signature
* Update src/bindings/python/src/openvino/frontend/pytorch/ts_decoder.py
* Fix problem with some ops in detectron2
* Use debug name for extra input signature
* [TF FE] Support MaxPoolWithArgmax operation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add ticket number for TS crash
* Correct error message
* Skip crashing tests
* Set additional tensor name for MaxPool
---------
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [HETERO] Add ConstantFolding in compile modelto avoid unexpected dynamism after model split. Add new property, which shows number of subgraphs
* Remove check for dynamic subgraph
* Removed legacy API from core_impl
* Revert old extension in old API
* Fixed unit tests
* Wrap new extensions in old API
* Wrap extensions in all legacy API
* Fixed legacy exceptions
* Fixed ONNX tests
* Try to fix LTO
Decompression attribute (that is present in models with FP16 precision)
prevents the weights to be constantfolded. Weights constantfolding is
required by CompressQuantizeWeights to compress the weights to low
precision format.
Ticket: CVS-117310
* Shared ShapeOf transformation via SharedOpOptimization. Allow EliminateGatherUnsqueeze pattern to have intermediate binary operation. Changes covered with transformation tests
* Move Reshape up through binary op to be optimized by NopElimination transformations
* Optimizes Shared Convert
* SharedOpOptimization: give higher priority for nodes of later versions to become root nodes
* Deleted old transformations
* Added revalidation for modified nodes
* Added binary op revalidation to the EliminateGatherUnsqueeze in order to have correct shapes for other matchers in the same graph rewrite
* Made PrepareShapeOpsForEliminationAroundBE independable of the upper node
* Introduces GroupedSliceToVSplitOptimization optimization
* Preserve output names during GroupedSliceToVSplitOptimization
* Revert "Made PrepareShapeOpsForEliminationAroundBE independable of the upper node"
This reverts commit 96785b24c9.
* Comments are adressed
* SharedOpOptimization: removes Base classes from the rules
* Advanced `mmap` test for IR FE
* Move memory functions to `CommonTestUtils`
* CppLint + ClangFormat
* Refactor IR FE `mmap` tests
1) Remove `compile_model` stage as not required, so also remove
"CPU" dependency
2) Run test in separate process to make RAM values more stable
3) Update RAM reference as "binsize / 2"
* Skip test on Apple platform
* Remove `getRssFileInKB()` as unused
+ Added is_padded_spatial to program_node
+ Added reorder to remove padded input in spatial axis for mvn
+ case applied only for blocked formats of implemented mvn opt kernel
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* updated to enqueue only fc for async build
* updated use_async_compilation(), make_task_executor_config() and disabled gemm_onednn.impl_replacement_with_cldnn
* added _num_async_build_threads
* added gemm to the async compliation targets
* removed priorbox in mark_if_constant
* fix priorbox operation for dynamic shape
* restore share test classes and disabled the test cases
* add exception throw for PriorBoxClustered
* Allow StridedSlice as predecessor for in place concat
* Enable padding support for strided slice
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add prepare_buffer_fusing TC for ov_gpu_unit_tests
---------
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Skip reorder at runtime if data type and format are not changedt
* Update shape of reorder user at predecessor node so that we can allocate pred nodes' output to host mem if needed
* Reinterpret reorder memory at runtime if needed
(e.g., input is fake-aligned fc and reorder uses that memory)
* Add debug config
* Fix CI test failure
* Do not skip after optimized reshape
* Do not skip user reorder if the user reorder is output and current node is static, and the memory is allocated to device
* Disable skip reorder user if current node has fused node
* Update src/plugins/intel_gpu/src/graph/include/reorder_inst.h
Co-authored-by: Eddy Kim <eddy.kim@intel.com>
* Minor fix for compilation error
* Do not skip reorder if the reorder's user is optimizable concat
* Fix CI failures
* No need to wait for input_layout because the events is already resolved in dgpu
* Fixed corner case where only some of the multiple output layouts are static
---------
Co-authored-by: Eddy Kim <eddy.kim@intel.com>
* * Not to reuse internal memory for dynamic shape because of the current inefficiency in the pool
* Added a new debug config for dump runtime memory pool
* Apply DisableMemoryReuse for all usages
* Resolved perf issue of memory reuse from pool : Previously original ibuf record was not released when we allocate new memory for that buf.
After releasing the memory, # of the memory pool record does not increase => no longer inefficient memory pool retireval.
* Added test
* Remove ngraph headers from some core source files
* Suppress some warnings
* Suppress more warnings
* Try to fix some compilation issues
* Suppress more warnings
* Supress clone_model
* Suppress warnings for Windows
* Suppress more warnings for Windows
* Suppress more warnings for Windows
* Additional suppress
* More Windows warnings
* Additional warning
* Suppress more warnings
* Suppress warning from python API
* [WIP] Added load_extension on Python side
* Added load_extension to NodeFactory as an alternative way to expose them to user (openvino path only, no changes in ngraph legacy path
* Reverted adding load_extensions in openvino.runtime
* Renamed load_extension to add_extension to be aligned with other part of extension API
* Applied code style rules
* Shorter description of NodeFactory.add_extension
* Fixed accidentally deleted indent
* Explicit error when custom op without intpus is attempted to be created, better help for NodeFactory.add_extension (op version clarification)
* Style fixes
* Test to cover NodeFactory.add_extension
* Minor wording changes
* Code style
* Fix code style
* Limit NodeFactory.add_extension test to specific test configurations
* [AUTO BATCH PLUGIN] update time out from set property
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] update test case from time out property
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
* [AUTO BATCH PLUGIN] fix review comment
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
---------
Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
Co-authored-by: Haiqi Pan <haiqi.pan@intel.com>
* Extract axes normalization and validation in separate functions in Interpolate op
* Update resample primitive declaration
* Update output layout calculation for Interpolate v11
* Update Interpolate op builder
* Add a shared test instance for Interpolate from 11th opset
* Add basic tests for Interpolate from opset 11
* Add new resample types and appropriate flags in ParamsKey
* Replace map which holds axes and scales with two separate vectors in resample_params
* Add resample kernel implementation
* [dGPU] Enable user scratchpad mode.
* Reuse intermediate buffer.
* Add own id to the memory dependencies at the c-tor of program_node
+ Allocate intermediate memory with memory_pool::get_memory() function.
+ Assign scratchpad memory desc in load() function for onednn primitive
serialization
* Allocate device mem for onednn scratchpad mem
* [Bug Template] New format bug template
This new bug template provides an easy interface for customers to fill in the information before submitting a new issue.
* [Update][Bug Template] New format bug template
- use 2021.4 LTS as an example for the OpenVINO version (this is where the customer will fill up the version)
- Adding macOS Systems for Intel CPU, macOS Systems for Apple Silicon, Android Systems, Raspbian Stretch OS, and WebAssembly to the OS list
- Adding GNA, AUTO, HETERO, and BATCH to the devices list
- Remove "Engine" in "Inference Engine"
* Update bug.yml v3
removed #endhere
* Apply suggestions from code review
add some suggested changes by @p-wysicki
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
* Update bug.yml
---------
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
* ResolveNamesCollisions transformation refactoring; enable it in MOC
* fix the description
* call ResolveNamesCollisions transformation in the frontends; resolve review comments
* Resolve review comments
* fix EliminateUnsqueezeGather and AlignMixedTypes transformations
* [Snippets][CI] Added Snippets Func Tests to Azure Windows
* [Snippets][CI] Added Snippets Func Tests to Azure Linux
* [Snippets][CI] Added Snippets Func Tests to GitHib workflow Linux
* [Snippets][CI] Added Snippets Func Tests to GitHib workflow Windows
* [Snippets][CI] Added Snippets Func Tests to Azure Linux Debian
OV_PYTHON_VERSION:3.11.2# Full version of Python its required for LD_LIBRARY_PATH. More details https://github.com/microsoft/azure-pipelines-tool-lib/blob/master/docs/overview.md#tool-cache
steps:
- task:UsePythonVersion@0
inputs:
versionSpec:'$(OV_PYTHON_VERSION)'# Setting only major & minor version will download latest release from GH repo example 3.10 will be 3.10.10.
OV_PYTHON_VERSION:3.11.2# Full version of Python its required for LD_LIBRARY_PATH. More details https://github.com/microsoft/azure-pipelines-tool-lib/blob/master/docs/overview.md#tool-cache
steps:
- task:UsePythonVersion@0
inputs:
versionSpec:'$(OV_PYTHON_VERSION)'# Setting only major & minor version will download latest release from GH repo example 3.10 will be 3.10.10.
OV_PYTHON_VERSION:$(OV_PYTHON_VERSION_MAJOR_MINOR).2# Full version of Python its required for LD_LIBRARY_PATH. More details https://github.com/microsoft/azure-pipelines-tool-lib/blob/master/docs/overview.md#tool-cache
steps:
- task:UsePythonVersion@0
inputs:
versionSpec:'$(OV_PYTHON_VERSION)'# Setting only major & minor version will download latest release from GH repo example 3.10 will be 3.10.10.
OV_PYTHON_VERSION:3.11.2# Full version of Python its required for LD_LIBRARY_PATH. More details https://github.com/microsoft/azure-pipelines-tool-lib/blob/master/docs/overview.md#tool-cache
steps:
- task:UsePythonVersion@0
inputs:
versionSpec:'$(OV_PYTHON_VERSION)'# Setting only major & minor version will download latest release from GH repo example 3.10 will be 3.10.10.
OV_PYTHON_VERSION:3.11.2# Full version of Python its required for LD_LIBRARY_PATH. More details https://github.com/microsoft/azure-pipelines-tool-lib/blob/master/docs/overview.md#tool-cache
steps:
- task:UsePythonVersion@0
inputs:
versionSpec:'$(OV_PYTHON_VERSION)'# Setting only major & minor version will download latest release from GH repo example 3.10 will be 3.10.10.
description:What is the pre-release version you are using?
placeholder:2023.0.0.dev20230427
validations:
required:true
- type:textarea
id:feedback
attributes:
label:Pre-release feedback
description:What is the issue or feedback on the pre-release?
placeholder:There is an inference performance drop in OpenVINO 2022.4.
validations:
required:true
- type:textarea
id:thoughts
attributes:
label:New Feature Feedback
description:Do you have any feedback on the new features released in the pre-release?
placeholder:Any thoughts on the new features are welcome.
validations:
required:false
- type:markdown
attributes:
value:|
By submitting this issue, you agree to follow our [Code of Conduct](https://github.com/intel/intel-one-mono/blob/main/CODE_OF_CONDUCT.md)
Have a new feature you would like to see in OpenVINO? File a feature request <a href="https://github.com/openvinotoolkit/openvino/issues/new/choose">HERE</a>.
You can also implement the feature by following the <a href="https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md">CONTRIBUTING.MD</a>
message:Thank you for looking into this issue! Please let us know if you have any questions or require any help.
issueCurrentlyAssignedMessage:Thanks for being interested in this issue. It looks like this ticket is already assigned to a contributor. Please communicate with the assigned contributor to confirm the status of the issue.
We welcome community contributions to OpenVINO™. Please read the following guide to learn how to find ideas for contribution, follow best practices for pull requests, and test your changes with our established checks.
## How to contribute to the OpenVINO project
OpenVINO™ is always looking for opportunities to improve and your contributions
play a big role in this process. There are several ways you can make the
product better:
## Before you start contributing you should
### Provide Feedback
- Make sure you agree to contribute your code under [OpenVINO™ (Apache 2.0) license](https://github.com/openvinotoolkit/openvino/blob/master/LICENSE).
- Decide what you’re going to contribute. If you are not sure what you want to work on, check out [Contributions Welcome](https://github.com/openvinotoolkit/openvino/issues/17502). See if there isn't anyone already working on the subject you choose, in which case you may still contribute, providing support and suggestions for the given issue or pull request.
- If you are going to fix a bug, check if it still exists. You can do it by building the latest master branch and making sure that the error is still reproducible there. We do not fix bugs that only affect older non-LTS releases like 2020.2, for example (see more details about our [branching strategy](https://github.com/openvinotoolkit/openvino/wiki/Branches)).
* **Report bugs / issues**
If you experience faulty behavior in OpenVINO or its components, you can
[create a new issue](https://github.com/openvinotoolkit/openvino/issues)
in the GitHub issue tracker.
* **Propose new features / improvements**
If you have a suggestion for improving OpenVINO or want to share your ideas, you can open a new
* **User documentation** is built from several sources and published at
[docs.openvino.ai](docs.openvino.ai), which is the recommended place for reading
these documents. Use the files maintained in this repository only for editing purposes.
* The easiest way to help with documentation is to review it and provide feedback on the
existing articles. Whether you notice a mistake, see the possibility of improving the text,
or think more information should be added, you can reach out to any of the documentation
contributors to discuss the potential changes.
You can also create a Pull Request directly, following the [editor's guide](./docs/CONTRIBUTING_DOCS.md).
## "Fork & Pull Request model" for code contribution
### Promote and Support OpenVINO
### [](https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md#the-instruction-in-brief)The instruction in brief
* **Popularize OpenVINO**
Articles, tutorials, blog posts, demos, videos, and any other involvement
in the OpenVINO community is always a welcome contribution. If you discuss
or present OpenVINO on various social platforms, you are raising awareness
of the product among A.I. enthusiasts and enabling other people to discover
the toolkit. Feel free to reach out to OpenVINO developers if you need help
with making such community-based content.
- Register at GitHub. Create your fork of the OpenVINO™ repository [https://github.com/openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino) (see [https://help.github.com/articles/fork-a-repo](https://help.github.com/articles/fork-a-repo) for details).
- Install Git.
- Set your user name and email address in Git configuration according to the GitHub account (see [First-Time-Git-Setup](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) for details).
- Choose a task for yourself. It may be a bugfix or an entirely new piece of code.
- Choose a base branch for your work. More details about branches and policies are here: [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches)
- Clone your fork to your computer.
- Create a new branch (give it a meaningful name) from the base branch of your choice.
- Modify / add the code, following our [Coding Style Guide](./docs/dev/coding_style.md).
- If you want to add a new sample, please have a look at the [Guide for contributing to C++/C/Python IE samples](https://github.com/openvinotoolkit/openvino/wiki/SampleContribute)
- If you want to contribute to the documentation and want to add a new guide, follow that instruction [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation)
- Run testsuite locally:
- execute each test binary from the artifacts directory, e.g. `<source dir>/bin/intel64/Release/ieFuncTests`
- When you are done, make sure that your branch is up to date with latest state of the branch you want to contribute to (e.g. `git fetch upstream && git merge upstream/master`). If so, push your branch to your GitHub fork and create a pull request from your branch to the base branch (see [using-pull-requests](https://help.github.com/articles/using-pull-requests) for details).
## Making a good pull request
Following these guidelines will increase the likelihood of your pull request being accepted:
- One PR – one issue.
- Build perfectly on your local system.
- Choose the right base branch, based on our [Branch Guidelines](https://github.com/openvinotoolkit/openvino/wiki/Branches).
- Follow the [Coding Style Guide](./docs/dev/coding_style.md) for your code.
- Document your contribution, if you decide it may benefit OpenVINO users. You may do it yourself by editing the files in the "docs" directory or contact someone working with documentation to provide them with the right information.
- Cover your changes with test.
- Add the license statement at the top of new files [C++ example](https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/classification_sample_async/main.cpp#L1-L2), [Python example](https://github.com/openvinotoolkit/openvino/blob/master/samples/python/hello_classification/hello_classification.py#L3-L4).
- Add proper information to the PR: a meaningful title, the reason why you made the commit, and a link to the issue page, if it exists.
- Remove changes unrelated to the PR.
- If it is still WIP and you want to check CI test results early, use a _Draft_ PR.
- Submit your PR and become an OpenVINO™ contributor!
* **Help Other Community Members**
If you are an experienced OpenVINO user and want to help, you can always
share your expertise with the community. Check GitHub Discussions and
Issues to see if you can help someone.
## Testing and merging pull requests
## License
Your pull request will be automatically tested by OpenVINO™'s precommit (testing statuses are automatically reported as "green" or "red" circles in precommit steps on the PR page). If any builders fail, you need to fix the issues before the PR can be merged. If you push any changes to your branch on GitHub the tests will re-run automatically. No need to close pull request and open a new one!
When an assigned reviewer accepts the pull request and the pre-commit is "green", the review status is set to "Approved", which informs OpenVINO™ maintainers that they can merge your pull request.
By contributing to the OpenVINO project, you agree that your contributions will be
licensed under the terms stated in the [LICENSE](./LICENSE.md) file.
OpenVINO documentation is built using Sphinx and the reStructuredText formatting.
That means the basic formatting rules need to be used:
### White Spaces
OpenVINO documentation is developed to be easily readable in both html and
reStructuredText. Here are some suggestions on how to make it render nicely
and improve document clarity.
### Headings (including the article title)
They are made by "underscoring" text with punctuation marks (at least as
many marks as letters in the underscored header). We use the following convention:
```
H1
====================
H2
####################
H3
++++++++++++++++++++
H4
--------------------
H5
....................
```
### Line length
In programming, a limit of 80 characters per line is a common BKM. It may also apply
to reading natural languages fairly well. For this reason, we aim at lines of around
70 to 100 characters long. The limit is not a strict rule but rather a guideline to
follow in most cases. The breaks will not translate to html, and rightly so, but will
make reading and editing documents in GitHub or an editor much easier.
### Tables
Tables may be difficult to implement well in websites. For example, longer portions
of text, like descriptions, may render them difficult to read (e.g. improper cell
widths or heights). Complex tables may also be difficult to read in source files.
To prevent that, check the [table directive documentation](https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#table-directives)
and see our custom directives. Use the following guidelines for easier editing:
* For very big and complex data sets: use a list instead of a table or remove
the problematic content from the table and implement it differently.
* For very big and complex data sets that need to use tables: use an external
file (e.g. PDF) and link to it.
* For medium tables that look bad in source (e.g. due to long lines of text),
use the reStructuredText list table format.
* For medium and small tables, use the reStructuredText grid or simple table formats.
## Cross-linking
There are several directives Sphinx uses for linking, each has its purpose and format.
Follow these guidelines for consistent results:
* Avoid absolute references to internal documents as much as possible (link to source, not html).
* Note that sphinx uses the "back-tick" character and not the "inverted-comma" => ` vs. '
* When a file path starts at the same directory is used, put "./" at its beginning.
* Always add a space before the opening angle bracket ("<") for target files.
Use the following formatting for different links:
* link to an external page / file
* `` `text <url>`__ ``
* use a double underscore for consistency
* link to an internal documentation page / file
* `` :doc:`a docs page <relativefilepath>` ``
* Link to an rst or md file within our documentation, so that it renders properly in html
* link to a header on the same page
* `` 'a header in the same article <this-is-section-header-title>`__ ``
* anchors are created automatically for all existing headers
* such anchor looks like the header, with minor adjustments:
* all letters are lower case,
* remove all special glyphs, like brackets,
* replace spaces with hyphens
* Create an anchor in an article
* `` .. _anchor-in-the target-article:: ``
* put it before the header to which you want to link
* See the rules for naming anchors / labels at the bottom of this article
* link to an anchor on a different page in our documentation
* `` :ref:`the created anchor <anchor-in-thetarget-article>` ``
* link to the anchor using just its name
* anchors / labels
Read about anchors
Sphinx uses labels to create html anchors, which can be linked to from anywhere in documentation.
Although they may be put at the top of any article to make linking to it very easy, we do not use
this approach. Every label definition starts with an underscore, the underscore is not used in links.
Most importantly, every label needs to be globally unique. It means that it is always a good
practice to start their labels with a clear identifier of the article they reside in.
@@ -33,7 +33,7 @@ OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference.
- Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud
This open-source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, GNA, multi device and heterogeneous plugins to accelerate deep learning inference on Intel® CPUs and Intel® Processor Graphics.
This open-source version includes several components: namely [OpenVINO Model Converter (OVC)], [OpenVINO™ Runtime], as well as CPU, GPU, GNA, multi device and heterogeneous plugins to accelerate deep learning inference on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from [Open Model Zoo], along with 100+ open
source and public models in popular formats such as TensorFlow, ONNX, PaddlePaddle, MXNet, Caffe, Kaldi.
@@ -48,8 +48,7 @@ source and public models in popular formats such as TensorFlow, ONNX, PaddlePadd
* [python](./src/bindings/python) - Python API for OpenVINO™ Runtime
* [Plugins](./src/plugins) - contains OpenVINO plugins which are maintained in open-source by the OpenVINO team. For more information, take a look at the [list of supported devices](#supported-hardware-matrix).
* [Frontends](./src/frontends) - contains available OpenVINO frontends that allow reading models from the native framework format.
* [Model Optimizer] - is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
* [Post-Training Optimization Tool] - is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning, for example, post-training 8-bit quantization.
* [OpenVINO Model Converter (OVC)] - is a cross-platform command-line tool that facilitates the transition between training and deployment environments, and adjusts deep learning models for optimal execution on end-point target devices.
* [Samples] - applications in C, C++ and Python languages that show basic OpenVINO use cases.
## Supported Hardware matrix
@@ -62,30 +61,30 @@ The OpenVINO™ Runtime can infer models on different hardware devices. This sec
<td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td>
<td>Multi plugin enables simultaneous inference of the same model on several devices in parallel</td>
</tr>
@@ -129,6 +128,16 @@ OpenVINO™ Toolkit also contains several plugins which simplify loading models
OpenVINO™ Toolkit is licensed under [Apache License Version 2.0](LICENSE).
By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
## Telemetry
OpenVINO™ collects software performance and usage data for the purpose of improving OpenVINO™ tools. This data is collected directly by OpenVINO™ or through the use of Google Analytics 4.
You can opt-out at any time by running the command:
``` bash
opt_in_out --opt_out
```
More Information is available at https://docs.openvino.ai/latest/openvino_docs_telemetry_information.html.
## Documentation
### User documentation
@@ -155,10 +164,9 @@ The list of OpenVINO tutorials:
## System requirements
The system requirements vary depending on platform and are available on dedicated pages:
- Broader model and hardware support - Optimize & deploy with ease across an expanded range of deep learning models including NLP, and access AI acceleration across an expanded range of hardware.
- NEW: Support for Intel 13th Gen Core Processor for desktop (code named Raptor Lake).
- NEW: Preview support for Intel’s discrete graphics cards, Intel® Data Center GPU Flex Series and Intel® Arc™ GPU for DL inferencing workloads in intelligent cloud, edge and media analytics workloads. Hundreds of models enabled.
@@ -18,6 +18,6 @@
- NEW: Introducing new performance hint (”Cumulative throughput”) in AUTO device, enabling multiple accelerators (e.g. multiple GPUs) to be used at once to maximize inferencing performance.
- NEW: Introducing Intel® FPGA AI Suite support which enables real-time, low-latency, and low-power deep learning inference in this easy-to-use package
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.