* Addapt ExperimentalDetectronTopKRois to support blocked layouts
* Reformat arg_max_gpu_test
* Add support and tests to arg_max_min 2D case to support for ExperimentalDetectronTopKROIs operation
* [TF FE] Refactor Pad, PadV2, MirrorPad and add layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Recover translators for DynamicStitch
* Apply code-review feedback and workaround for i32 paddings
* Return required type for convert_pad_mode function
* Work around IR reader limitation with i32 type for Pad
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Moved pdpd tests to folder with pdpd frontend
* Restored old requirements.txt for paddle temporarily
* FIxed requirements path for pdpd
* Restore old dependency
If gemm input dimensions are not multiple of 16 and any of
transpose_a/transpose_b attribute is set - cldnn picks
'gemm_ref' kernel in favor of faster 'gemm_tiled_opt'.
By emplacing explicit permute operation on the gemm input that it requires,
we make cldnn to pick 'gemm_tiled_opt', which in result improves
performance.
For some input shapes, transpose(s) + gemm_tiled_opt can be slower than
just gemm_ref. Based on benchmarks - the cutoff point was set for inputs shapes > (64, 64).
Ticket: 67271
* Replacing image with a scalable svg file
Replacing image in Protecting Model Guide -changing the png image to scalable svg format.
Removing image project files.
* added gather tree blocked layout support
* gather tree blocked layout support review cleanup
* gather tree blocked layout support review cleanup
* gather tree blocked layout support review cleanup
* gather tree blocked layout support review cleanup
* gather tree blocked layout support review cleanup
* build fixed
* usage of program and program_node are changed to use kernerl_impl_params
* pointer to network is replaced with reference
* moved primitive specific parameters out of kernel_impl_params
* use impl_params instead of program_node in onednn_impl
* relocate params from impl_params to primitive_inst
* added conditional directives around fused_desc_onednn
* removal of unnecessary changes
* node_output is separated from impl_param->output
* Improvements in rpm / debian build
* Fixed several debian warnings
* Supported old gflags from CentOS 7
* Reverted back OpenCV version
* Fixed clang-format
* [TF FE] Refactor translators for Select and SelectV2 and test it
It fixes a case when the condition of a smaller rank than operands in Select
Separately added tests for Select and SelectV2
Do not mix-up Select with Where, so tests for Where are moved to test_tf_Where.py
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert extra changes
* Apply code-review feedback: support undefined rank
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* [TF FE] Refactor translators for Reverse, ReverseV2 and test it
Make these operations reshapeable. Add layer tests for them to the pre-commit
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: simplify checks in Reverse
* Apply the rest of code-review feedback: simplify code for Reverse
* Remove redundant check for axes
* Apply code-review feedback: support dynamic rank
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [GPU] add blocked format to experimental detectron detection output
* [GPU] add this kernel in whitelist of program.cpp
Co-authored-by: ozhydkov-lohika <ozhydkov@lohika.com>
* cldnn serialization
* read layout from _impl_param instead of node
* changed ref in kernel_impl_param to pointer
* removed serialization utils
* removed serialzation related changes
* restored references in function arguments
* remove trailing spaces
* revert change in bs_x_bsv16
* fix to rebase
* Improving Readability of Further Low-Level Implementation Details
The changes include recreation of the graphics to improve the readability of the article. Minor proofreading corrections have been applied as well.
Failure: Comparing reshape->output_shape with a new tensor() object was not working as intended.
Since the original intention was to check whether cldnn::tensor type output_shape is set or not,
so replaced the comparision to use size of the output_shape.
* support control flow, tensor array
* fix clang error
* support ppdet2.3 model
* 1. code clean; 2. more comments inlined.
* fix after clone ov::Model no more in need.
* support dynamic shape; more comments
* only process same rank
* remove unused function
* simplify the loop logic
* fix review comments
* support shape{1} {}
* disable FoldSubgraphEmptyInputs because loop in loop
* fix review comments
* remove scalar{}->shape{1} testcase
* allow re-infer shape when backedge changes input shape
* fix condition shape to {1}
* support output rank is not same or dynamic
* fix refactor error
* apply review comments
* fix win warnings
* remove TransformEliminateConvert
Co-authored-by: jialipen <cecilia.peng@intel.com>
Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
* Bump Cmake for Linux
* Fixed environment
* Change CI script
* Disable some warnings
* Fixed compiler flag
* Enabled Warnings as error for Linux
* Revert "Enabled Warnings as error for Linux"
This reverts commit bfb09efb71.
* Use prebuild cmake
* Remove redundant check
* Use non-blocking assertions in unsqueeze tests
for independent properties
* Review unsqueeze interval shape propagation:
- extend partial shape propagation tests
- add invalid axis value tests
- fix issues when repeated axes on input
* Shape inference test, unsqueeze using default ctor
- private fields set by setters
* Review unsqeeze bounds propagation:
- preserve and propagate labels
- bounds propagation lowe/upper
* Add template shape inference for unsqueeze
- extract current implementation and unify it for shape types
- add unit test for static shape inference
- add unsqueeze in/out indexes
* Unify axes normalization
* Use common fixture for static shape inference
* Fix build issue in GPU plugin
- include unsqueeze shape inference
* Remove enum with in/out indexes
Due to build issue on Windows
* Remove make move iterator
minor changes validation util
* Add test for label propagation
- expand static shape inference tests
* Add common validation for axes input
* Fix build issues
Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
* cmake/packaging/rpm.cmake: Added Frontends
* rpm_post_build: Added link to Fedoraproject wiki with rpmlint errors info
* RPM: Throw warning for rpmlint findings instead of FATAL ERROR till there is no filtering/suppression of errors
debian.cmake: Added one more error to suppress
thirdparty/CMakeLists.txt: added CentOS(RHEL) for gflags shared libs case
install_openvino_dependencies: Added gflags installation
* Fixed gflags issues for RHEL/CentOS
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Initial dynamic shape smoke test for GPU
* Bert dynamic runs without crash
* Additional fix to resolve error in bert-large-uncased-whole-word-masking-squad-emb-0001
* Fix error in unfusion function: input nodes of the current (fused) node need to be updated with the latest dependency if they are fuesed to other nodes
* Several fixes
(1) Fix program to clear _kernels after all build_program steps finished
(2) Fix update_kernel not to init_kernel when impl_cache hit
(3) Fix update_kernel to clear kernels_cache::_kernels after adding the new impl to impl_cache
(4) No longer need to remove kernel from kernels_cache::_kernels after the corresponding impl is dropped from impl_cache
* Fix crash of bert_emd_4layer
* Applied review comment
* Applied review comment : fix add_required_reorder
* Fix broadcast to propagate dynamic shape properly & reverted change on constant
* Added a new unfusion unittest
* Fix broadcast single input case to use predefined shape properly
* Fixed count_non_zero output to result only one element of count
* Removed output_layout str for gather_nonzero
Removed unused ov_input_rank for count_nonzero
* Fixed create_host_blob to use usm if the target layout is not dynamic.
(Previous impl has an error when the network is dynamic but the output is static)
Moved dyn shape smoke test under dynamic directory
* Fix lint error
* Fix gather type_prop test build issue on windows
* Add pytest-forked to test requirements
* Update src/bindings/python/requirements_test.txt
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Remove warning suppression for core tests
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
This commit contains two features which are split into two commits:
1) linear onnx interpolation of 5d formats
2) Optimize blocked formats for linear_onnx and nearest formats
- contain correspondent unit and single layer tests
* Create device property for passing nthreads or pin to AUTO or MULTI in benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update the logic of the multi device properties setting for the same target device.
* refactor the logic to pass nthreads and pin to AUTO and MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* [TF FE] Add translators for IsInf and IsNan and test it
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add changes in tf_utils
* Apply code-review: add xfail and use of unary op
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update to the latest cmake for Windows
* Skip new warnings
* Added compiler flags for old cmake
* Disabled some warnings
* Suppress new warnings
* Fixed some errors
* Disable some warning for lpt
* Fix some warnings
* Disable new warnings
* Fix some warnings
* Disabled Was error for arm
* Fixed some errors
* Fixed some new warnings
* Fixed auto plugin
* Fixed Windows warning
* Fixed Windows
* Disable some warnings
* Some changes
* Small test fixes
* Fixed python build
* Added skip warnings for legacy python API
* fix benchmark build error caused by string compare
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [Onlt Test] only for test mac build, will revert after validation
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* fix benchmark build error caused by string compare
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [benchmark] change string compare func from strcmp() to std::string::compare
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Revert "[Onlt Test] only for test mac build, will revert after validation"
This reverts commit 313e8428d4.
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
Port from #13549
Fixed the following issues:
Switching between C++ and Python docs for "Shape Inference",
Removed repetitions,
Quote background in bullet list at the beginning of "Multi-device execution",
Broken note directives,
Fixed video player size in "Inference with OpenVINO Runtime",
Standardize Additional Resources throughout Runtime Inference.
* Interval value propagation tests for Add and Sub
* Add interval value prop tests for Divide and Power
* Add interval value prop tests for Max and Min
* Add mixed ops subgraph tests
* Tests update
* Style apply
* Restore tests
* Fix elvaluate bounds for Divide op
* Split tests into separate cases
* Back to const auto& for inputs
* Separate checks for evaluate tensor
* Clean up commented tests
* Introduce OV_EXPECT_THROW test macro
- wrapper for gtest macro with additional matcher for exception what()
- apply OV_EXPECT_THROW in gather type prop tests
(cherry picked from commit f174319339)
* Revert gather type_prop tests changes
* Use OV_EXPECT_THROW in the tests
Co-authored-by: Raasz, Pawel <pawel.raasz@intel.com>
* add parameterized test
* add blocked layouts support
* support for mixed input formats
* fix RHEL 8.2 build
* add scatter_elements_update to whitelist for blocked formats
* Added bs_fs_yx_bsv16_fsv32 format
* Change stress tests db fields
* Add option for upload data to database by API handler
* Push modified data only to new database
* Fix adding log info to new db
Static gflags is not available on Fedora, leading to a configure
error. So use the shared version.
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Tom Rix <trix@redhat.com>
If ENABLE_NCC_STYLE=OFF, the check for ncc.py still happens.
A confusing warning is reported and a redundant setting of
ENABLE_NCC_STYLE to OFF is done.
Check that ENABLE_NCC_SYTLE is enabled before looking for ncc.py
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Tom Rix <trix@redhat.com>
* Added reading of meta information from IR
* Fixed code style
* Added tests to cover multi-threading and removed file
* Fixed asserts
* Changed parsing of meta information
* Added meta information serialization
* Fixed unit tests
* Support unset_cli_parameters
* Changed get meta API
* Fixed threading tests
* Fixed mutex
* Removed serialization
* Fixed tests
* Fixed build
* Add quantization_parameters
* Use rt_info instead of meta section
* Disable old rt_info serialization
* Try to fix MO Reader for new rt info
* Fixed property name
* Added test to new meta format
* Added tests for new PoT config
* Added tests for new rt_info attributes
* Remove redundant code
* Revert old rt_info style for nodes
* Add get API for python
* Added python tests
* Fixed python code style
* Fixed tests code style
* Fix flake8
* Fixed python tests
* Add has_rt_info to test
* Added more comments
* Extended tests and API
* Changed error message
* Use new API to get model version
* Fixed comments on python API
* Fixed comments
* Fixed comments
* Update src/bindings/python/src/pyopenvino/graph/model.cpp
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Update src/bindings/python/src/pyopenvino/graph/model.cpp
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Fixed comments
* Fixed python check
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Build code snippets together with doc
* Added separate step for code snippets
* Try to enable CI step
* Disabe myriad
* Try to fix Windows
* Detect number of cores
* Reduced build time of code snippets
* add unit test for reorg_yolo
* add validation to reorg_yolo kernel
* add blocked formats support
* remove non-working yxfb optimization
* add reorg_yolo to whitelist for blocked formats
* [TF FE] Refactor translation for Concat and ConcatV2 operations and add tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Correct the layer tests for passing
* Apply code-review feedback: remove redundant function
* Fix a name of function to create Concat model in the layer test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* move transformation to common; use opset9; fix Gelu and Softsign operations; misc fixes
* cpplint fixes
* rename DoTransformation; use TransformationTestsF for testing
* test register_new_nodes working by creating multiple unary operations
* fix transformation constraints; add unit test on it
* remove unneeded headers
* remove unneeded comments
* fix tests iteration types
* rename unit test
* parametrized input shape and type
* swap nodes instead of creating new
* transpose can have not just a constant node on the 1-st input
* Update src/tests/functional/inference_engine/transformations/common_optimizations/transpose_sinking_unary.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update src/tests/functional/inference_engine/transformations/common_optimizations/transpose_sinking_unary.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* Update src/tests/functional/inference_engine/transformations/common_optimizations/transpose_sinking_unary.cpp
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
* clang cleanup
* return back nodes swapping; wrote additional unit tests
* add unit tests
* fix unit tests namespace
* add unit tests
* remove debug
* cpplint fixes
* use 2 ways of swapping: nodes and outputs
* clang fixes
* move choise descision into separate function
Co-authored-by: Nadezhda Ageeva <nkogteva@gmail.com>
reorder_input has the issue when using generated reorder case.
And handle_reshape has only handle 4 dims (bfyx), so support 5, 6 dimes
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Benchmark_app python and c/c++ script add op profiling feature
* Fix compile bug about benchmark c/c++ op profiling
* Fix Compile Bug
* Fix issue in PR#12350 : 1. report_type info update; 2. priority of -pc and -pcsort; 3.align format between c++ and python; 4.check 'proportion' for 0 and print N/A; 5.detect the confusing print info
* Fix 8/17 review suggestion
* Fix 8/23 suggestion
* Fix the clang-format issue
* Fix the win-cc issue
* Fix win-cc issue about conversion from 'double' to 'float', possible loss of data
* Push PR by ODT group account
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: Fiona Zhao <fiona.zhao@intel.com>
* Moved MVN reference to ov::core.
* Code style.
* Applied comments.
* Code style.
* Returned evaluate in evaluates_map()
* Added constant folding test.
* Use new API.
* Use new API.
* Use new API.
Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
* Changed GNA library version from 1455.2 to 1815
* fixed new URL to GNA in cmake
* fixed gna lib version in linux CI
* use Gna2ExportTlv from new library
* Updated GNA to 1815.1
* [C API DOC] fix mac doc build error caused by ov_output_const_port_t vs ov_output_port_t
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [Only for Test] validate mac build error fix, need to revert before merge
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Revert "[Only for Test] validate mac build error fix, need to revert before merge"
This reverts commit 1082ab32ba.
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Use eltwise_shape_infer form shape_inference in validate_and_infer_elementwise_args
* Align Unary ops, remove redundant validate_and_infer_elementwise_args usage
* Add test with default constructor for BinaryElementwiseArithmetic ops
* Style apply
* Fix expected error message
* Add common shape_infer tests for BinaryElementiwiseArithmetic ops
* Remove old Add test
* Update NGRAPH_CHECK to OV ASSERT
* Removal of redundant autob param to the validate function
* Tests update
* [GPU] Fixes for infer request impl in dynamic cases
* [GPU] Fixed incosistent output shapes for LSTMSequence op
* [GPU] Update network::get_output_layout method
* [GPU] WA for USM memory allocations with 0 bytes
* docs: Rewrite What's Next section on PyPI page
* docs: fix link to Python example
* docs: add link to additional configurations page
* Update docs/install_guides/installing-openvino-pip.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/install_guides/installing-openvino-pip.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* docs: remove Whats Next link
* docs: fix what's next link
* Update docs/install_guides/installing-openvino-pip.md
* Update docs/install_guides/installing-openvino-pip.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Add missing div/mul/sub type_prop test files to CMakeLists
* Align license note in missed test files
* Add interval dimension tests for binary arithmetic ops
* Align test file style
* Add test for broadcast none
* Add more numpy test cases
* Add pdpd broadcat tests
* Add more intervals numpy broadcast tests
* add attr types header
* Register tests in ArithmeticOperator suite
* Change all assert_eq to expect_eq
* pdpd broadcast tests clean up
* Add tests for dim labels numpy broadcast
* Style format
* Tests improvement
* More label tests
* Comment update
* Add more tests for numpy and none broadcast
* PDPD tests cleanup
* Refactor tests to use get/set_shape_labels helper
* Add more label tests
* Add more tests, use TableOfEquivalence
* Propagate d2 label if labels are not equal
* Make common `merge_labels` helper function
* Update MatMul labels test
* Fully dynamic shape test numpy/none/pdpd
* Separate and refactor test cases
* Add interval dim test to Concat op
- use common test fixture for PartialShape OK scenario
* Use get_shape_labels in Concat tests
* Add label_evaluate test for Concat op
* Add validation util to check no labels in tensor
* Add template shape inference for Concat op
* Move Conact evaluate_label test to evaluate bounds
* Fix cpplint issue
* Migrate to PDPD 2.3
* update supported paddle version to 2.3.2
* update paddle test with paddle new static API on 2.3.2
* fix issue of different output shape with paddle's
* fix clang format issue
* fix compiling issue
* integrate paddle static save model api to original save model api
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
* SplitConcatElimination transformation
* ov namespace
* fix: add an additional check for split
* Fix lpt tests: added unary op between Split and Concat to avoid elimination
* add additional check in SplitConcatElimination, add new tests
* apply review comments, add detailed description of the special case
* apply review comments
* clean up
* apply review comments
* Apply suggestions from code review
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* codestyle
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* optimize models on batch16
* add is_os_yx_isv16_osv8 to format_traits_map
* choose onednn deconv
- remove checking deconv cross-precision
- implicit concat could not support byxf.
Co-authored-by: Jade Cho <jade.cho@intel.com>
* Update primitive, primitive_inst, and program_node for multiple outputs support
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Update arg_max_min kernel for multiple outputs support
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Fix failed unittests TCs
- remove duplicate output memory
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Clean up code
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Apply code review
Signed-off-by: Andrew Park <andrew.park@intel.com>
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Update the config test for Myriad because config checking logic is updated in the MULTI/AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Fixes for brew support on OSX
* Added new policy
* added quotes
* Used OpenVINODeveloperPackage
* Extra fixes
* Use standard verbose variable
* Fixed pkg-config generation
* [TF FE] Remove export API for helper transformation
Also, it skips layer tests for SegmentSum and DynamicPartition in case legacy frontend
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Split test suites for DynamicPartition
* Update CMake to link additional sources for the tests
* Update Cmake to fix unresolved symbol get_message_prefix_tf
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [C API DOC] update C API user guide pipeline
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] C API user guide pipeline add Link & Build part for C
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] C API user guide pipeline add objects release for C
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] C API user guide pipeline clear annotates
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] fix build error
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] Reconstruct the guide about integration with OpenVINO Runtime for C
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Revert "[C API DOC] Reconstruct the guide about integration with OpenVINO Runtime for C"
This reverts commit 9552054e7e.
* [C API DOC] split project structure & Cmake
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] using correct input shape
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] align with C++ code example
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [C API DOC] fix build error
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [TF FE] Support Unique operation via transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix other part: place header for UniqueReplacer and test name
* Remove redundant included headers in the transformation test
* Add one more case in the transformation test
* Implement the right way for mapping tensor names
* Apply code-review: rename out_idx, add check for 1D
* Fix unique_replacer test
* Fix typo in the test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add FunctionsComparator CmpValues ACCURACY to the test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
- Enable per-tensor zp on AVX512_VNNI and AVX512_AMX ISA to use BRGCONV kernel.
- Fix brgconv should not be disabled correctly in some INT8 model which has binary postops
- Merge optimization of convolution attribute PR to lower down the load network time latency
- Only enable conv/group conv S8 input on AMX. Non-AMX same with 2022.1 LPT input precision config.
* [GNA] Move tests to plugin dir
* reverted temporary fix
* Added linux specific compiler flags
Co-authored-by: Adam Tumialis <adam.tumialis@intel.com>
* Remove deprecated and not used functions:
- ngraph::is_valid_permutation
- ngraph::apply_permutation
* Use not blocking assertion in transpose test,
type prop
* Set 'UNDEFINED' as the performancehint default value. Update benchmark app to pass performance hint with none value to MULTI/AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Using 'UNDEFINED' as the default value of hint in AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Adding a flag instead of adding a default value of hint to check if user configures the performance hint to AUTO/MULTI plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update benchmark python version.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add static_output attribute to enable GPU implementation of GenerateProposals.
This change may be reverted when dynamic shapes support will be implemented
in GPU plugin.
* - add GPU single layer test;
- add normalized attribute to SLT.
* add GPU primitive for GenerateProposals
* add kernel for GenerateProposals
* add unit test for GenerateProposals
* add blocked layouts support
* tidy up
* support blocked layouts also for 2nd and 3d outputs
* Transformation to deal with dynamic output in GPU
- revert changes in GenerateProposals interface and shape inferenece;
- add transformation;
- custom compare method in single-layer test.
* address review comments
* fix after rebase
* fix after rebase
* review comment: added nms_eta to testcase name generation
* - added input types combination to functional tests;
- fix float16 comparison.
* fix after rebase
* use vector for input ids
* fix after rebase
* [TF FE] Support DynamicPartition operation
Add support for DynamicPartition operation and validate it with the layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add implementation
* Rescale partition indices to provide stable sorting
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Test to interval shape propagated by transpose
* Test to propagate labels by transpose
* Add template transpose shape inference
* Fixes to transpose shape inference
* Update names for shapes:
input -> input_shape
order -> order_shape
* Not fill output shape for dynamic range
* Add constexpr to SeqGen and Between comparator
* Correct StaticShape creation in test
* Tests check partial value propagate in arg input
* Add evaluate upper, lower, label to transpose
- add test
* Add common methods for inference and evaluate
* Move helpers to shape_inference
* Move transpose attribute to transpose op
* Fix include in transpose operator
* Correct label generation and type
* Fix null conversion
* Use uint64_t for labels tensor
* Fix compare labels
* Use order length as output rank
* Update transpose transformation test
* Move helpers to validation_util
* Correct test assertion for expected shape
* Transpose evaluate use common function
for output calculation
* Remove redundant helpers from transpose test
* marks tf2 keras tests as precommit for new FE
* adds tf2 layers tests for new FE in precommit
* removes redundant tabulations
* review changes
* fix field name
* fix syntax error
* removes failing tests
* removes failed test
* removes failed test
* Fix benchmark_app commandline parsing
Object 'str' does not have a contains() method. Replaced by
"searchstring in object" query.
* Fix perf counter output and use consistent units
When ProfilingInfo was bound by pybind11, those entries turned into
Python timedelta objects. This caused two problems:
1. There was a division by 1000 to go to milliseconds. This caused a lot
of precision lost, because `timedelta(microseconds=2300)/1000.0 =
timedelta(2)` and `timedelta(microseconds=33) = timedelta(0)`.
2. When converting those timedelta objects to str, the output is in the
form of (HH:MM:SS.XXXXXX). This is not very useful microsecond based
performance counters.
This change simply reverts everything to printing plain microsecond
based integers.
* Align counter output to ms between Python/C++
* [GPU] mark constants as supported on QN. Fetch common part for CPU and GPU.
* Code style
* Apply review comments. Fix TEMPLATE plugin FP16, use common QN helper.
* Apply review comments: CNNNetwork -> Model inside GetSupportedNodes
* Apply review comments: move functino from plugin interface
* parent 6e7016ccda
author Ilya Churaev <ilya.churaev@intel.com> 1664281499 +0400
committer Ilya Churaev <ilya.churaev@intel.com> 1664510018 +0400
Fixed warnings on local machine
* Added CMAKE_COMPILE_WARNING_AS_ERROR usage
* Fixed style
* Fixed merge conflicts
* Fixed typo
* Fixed myriad build for macOS
* Fixed warning
* Fixed tests
* Disabled incorrect test
* Try to fix linux tests
* Revert "Try to fix linux tests"
This reverts commit 29224c93ff.
* Fixed tests
* Revert logic with incorrect cast
* Fixed log softmax
* Disable warning as error for cuda
* Try to fix inference_engine_s
* Fixed cmake
* Revert "Fixed cmake"
This reverts commit 87e9e4e674.
* Revert "Try to fix inference_engine_s"
This reverts commit a1adca8b05.
* WA for static symbols in inference_engine_s test library
* Fixed code style
* Fixed static definition for master
* Revert "Fixed static definition for master"
This reverts commit 20d00d215a.
* Revert "Fixed code style"
This reverts commit 0eb2362543.
* Revert "WA for static symbols in inference_engine_s test library"
This reverts commit 75ef86a79d.
* Fixed linker issue for Windows
* Disable WaE by default
* Disable warning as error in the developer package
* Try to fix dev package
* Try to fix Windows Jenkins
* Revert old behavior for tread_warn_as_err variable
* [GPU] Split and VariadicSplit new shape infer (#13216)
GPU update test script
* [GPU] Added W/A for crop offset (#13216)
Co-authored-by: Ahn, Paul Y <paul.y.ahn@intel.com>
* [GPU] Fix crop gpu test failures (#13216)
- Fixed by review comments
* [GPU] Move input offsets calculation to crop_inst::calc_output_layouts (#13216)
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
* [TF FE] Switch on fallback for TF FE even for use_new_frontend option
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add unit-test for TF FE transformation config fallback
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Support DeepSpeech model by TF FE
Add the final part to support BlockLSTM operation with sliced state cell
from the last time step.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: use get_pattern_map, no nullptr
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [GPU] Implement (I)RDFT-9
* Added GPU implementation of RDFT operation
* Added basic GPU implementation of IRDFT operation, but still needs adjusting
* Added single layer tests for (I)RDFT
* Added unit tests for (I)RDFT
* Added blocked layout support for all DFTs operation
* [GPU] Added changes after review.
* [GPU] Added single-layer test for single axis case for DFTs.
* [GPU] Simplified attach_dft_impl method.
* [GPU] Moved negative axes cases to separate test for DFTs.
* [GPU] Adjusted tests to cover all signal sizes cases (less, equal and more than input size) for DFTs.
* [GPU] Fixed IRDFT implementation.
* [GPU] Used RDFTLayerTest.
* [GPU] Fixes after rebase.
* [GPU] Renamed kind enum to direction.
There is build error on RHEL 9
openvino/src/plugins/intel_gna/backend/pwl_border_values_counter_identity.cpp:70:51: error: ‘numeric_limits’ is not a member of ‘std’
70 | const int16_t y_min = is_low_precision ? std::numeric_limits<int8_t>::min() : std::numeric_limits<int16_t>::min();
numerical_limits is declared in limits, so include it.
Signed-off-by: Tom Rix <trix@redhat.com>
Signed-off-by: Tom Rix <trix@redhat.com>
* [GNA] Add extra segments to PWL only if it is fused with Conv2D layer
* added checking if extra segments are needed by checking if activation
functions are fused with Conv2D or Conv2D with Pooling applied
* added unit tests verifying implmentation
* Update src/tests/unit/gna/gna_extra_pwl_segments_tests.cpp
fix license
Add test
Update src/common/transformations/include/transformations/common_optimizations/fused_names_cleanup.hpp
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Review comments
Cleanup test includes
* Moved template_plugin
* Move template extension
* Changed cmake configs
* Try to fix build
* Moved plugin to standalone folder and fixed build
* Try to fix doc
* Try to fix doc
* Update CMakeLists.txt
* Moved extensions to src/core
* SequenceFusion transformation and tests
* Enable SequenceFusion transformation in MOC
* add missed includes
* fix type
* fix build, apply review comments
* fix build
* fix build
* fix build again
* use ov namespace in has_result_consumers function
* fix win build
* try to fix win build
* investigate issue on win platform
* investigate issue on win platform
* investigate issue on win
* issue on win platform
* remove the transformation from MOC
* fix LSTMCell fusion, simplify transformation implementation, fix copying tensor and friendly names
* clean up
* add support for LSTMCell v0, resolve review comments, enable additional tests
* Avoid duplicated code
Change-Id: I86fe59bab13010aedc62a2c92102b00e060ceeec
* Add more test cased for element type
Change-Id: I1f02254cee715d53ede0422c14a4732267d3db27
* Apply INSTANTIATE_TEST_SUITE_P for tensor element type test
Change-Id: I8bbeeb00ae872b7b286b5a776a5c2681e6b5ee94
* [TF FE] WA: Support Non-frozen Formats
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Normalize path to the intermediate model
* Apply feedback: add path to return and correct tests
* Check that the file is not empty in the test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update LD_LIBRARY_PATH for python API tests
* Update LD_LIBRARY_PATH for python API tests
* Remove PYTHONPATH, OV_FRONTEND_PATH
echo PYTHONPATH and LD_LIBRARY_PATH
* Update get environ variables when are not set
* Set OV_FRONTEND_PATH variable
* Add openvino installation path to OV_FRONTEND_PATH
* Add PYTHONPATH to point install tests dir
* Add debug message for PYTHONPATH
* Update LD_LIBRARY_PATH to point openvino wheel
install directory
* Add comments for environ vars for python API tests
* Remove not required paths from linux CI
* Minor comments update
* marks specified tests with precommit_tf_fe mark
* adds tf layer tests for new fe into the precommit
* removes redundant tab
* removes redundant tab
* clean-ups
* clean-ups
* change import path for build_graph func
* set path to mo
* set path to mo for PYTHONPATH
* get back imports
* left only CPU runs
* return back skip for LogSoftmax op since it still was not fixed
* adds test for GRUBlockCell op
* [MO, Common] Fix SplitConcatPairToInterpolateFusion transformation
It fixes a case when Split go with disconnected output port.
For this case, this transformation must not be applied.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix grammar mistake in the comment
* Remove ref_function from the test
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* The RegistersPool class was implemented. It is usefull for managing registers while developing jit kernels.
* The unit tests for RegistersPool were added.
* The virtual function call in the RegistersPool constructor bug was fixed.
* The Reg8, Reg16 were added to the RegisterPool. The tests were added for different ISA and different Xbyak register types.
* The release register on the move leakage was fixed.
* The default constructor was added for the IsaRegistersPool class, which exclude some critical registers by default.
* The implementation for is_any_of was added and used in the static_assert. The std::is_base_of and Xbyak::Mmx was used to identify vector registers. The factory method was added for RegistersPool with isa non template parameter.
* The -Wswitch and -Wunused-value errors was fixed for macos build.
* Add 1d to 1d weight kernel when creating ConstantOp
* Fix performance regression of dna model
* Add testcase for 1d input
* Fix blocksize range and simplify the operation
* Separate codes for block1 from block8
* Use OUTPUT_BLOCK_WRITE
* Add unit testcases for feature of multiple and non-multiple of 16 and for fusion test
* Fix testcase suffix as _1d_depthwise_convolution and remove redundant codes
* Fix some potential segmentfault in test cases
Change-Id: Ie12956e32805d950fa1eb82a1e49d532b26bc390
* Use OV_ASSERT_OK to replace OV_EXPECT_OK
Change-Id: I6780072cca4b4a6485105db4832ef3085c990df0
* Fix compile error
Change-Id: Ib4594093ddc60e504e45c6581410fa22d09bd9a6
* Update usage of ASSERT and EXCEPT
Change-Id: I7cb5d0bda4a60a2de899132c47fbf745bc760305
* Use EXPECT by default
Use EXPECT by default, unless you require something to hold for the remainder of the tests.
Change-Id: I2d2d8d8d9632e9e8742c4d923353d319e597a46a
* Avoid the issue of ov::hint::PerformanceMode::UNDEFINED
If convert ov::hint::PerformanceMode::UNDEFINED to string format, the string value is 0 (string_value=0, not "UNDEFINED")
We should avoid to copy this empty string.
Change-Id: I9d4a080b697a863762342f6f890d8ac7434e1864
* DOCS: Fixing version selector dropdown
Fixing the version selector dropdown, to avoid horizontal scrollbar and trimming text.
* Adding overflow
* Update custom.css
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Change debug informaiton format from Z7 to Zi for transformation library. Temporary workaround for LNK1248: image size (1004722F6) exceeds maximum allowable size (FFFFFFFF)
* add WIN32 check
* Updating NNCF documentation
* nncf-doc-update-ms
* Merge branch 'nncf-documentation-for-22.2' of https://github.com/sgolebiewski-intel/openvino into nncf-documentation-for-22.2
* Adding python files
* Changing ID of Range Supervision
* Minor fixes
Fixing formatting and renaming ID
* Proofreading
Minor corrections and removal of Neural Network Compression Framework article
Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* [GNA] Add extra segment to PWL to ensure crossing (0,0)
* implemented new mechanism of generating PWL segments for identity:
* adding extra segment at (0,0) for PWL if given data does not pass (0,0).
* updating xBases for segments on the left and right side of (0,0) segment.
* fixed dumpin of descriptor to *.dat file
* added regression tests in unit tests and functional tests
* Fix review comments
- Precise spatial shape inference for dynamic shapes
- Deprecated set_adding_above method and given alternative set_pads_end
- Enabled shape inference for default-constructed Convultions (private field `num_spatials` is no more required to call shape_infer)
- Test adjusted for dynamic spatial shapes calculation
Tickets: 82949, 82963
* Unicode support
Change-Id: I6da095b0f0b86eb0fa9c44ddb41fa40f82fbded2
* Unicode support load compile model from file and add test cases
Change-Id: Ic66c2b9d5bc7da44db85772fc55abb9a699f4b7d
* Fix build error in Windows
Change-Id: I98c686f5bef17bef63ec6937035fac011224a790
* Fix compile error caused by rebase
Change-Id: I1457ba6bfd6743855bf55d77a2103a558548a8a0
* Fix refefinition issue in Windows
In windows it reports below error:
include\10.0.18362.0\um\winnt.h(1131): error C2365: 'BOOLEAN': redefinition; previous definition was 'enumerator'
openvino\src\bindings\c\include\openvino/c/ov_common.h(105): note: see declaration of 'BOOLEAN'
Change-Id: Ic84b7f0d6e3de5ba6c37af39533a3ca2f2d17ad8
* Link ov::util to call unicode code function
Change-Id: Ib324173524b8fe9b7f523fe6b838e155ffbe0842
* rename BL0 to be BOOLEAN
* Boolean should be 8bits data
* fix benchmark filed to pass on MULTI device when enable num_streams.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove part of test cases because MULTI plguin will not check if the unspported property is valid now.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove some incorrect config for MULTI test case since MULTI will pass through those unrecognized config without any exception.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support MULTI to set nstreams to multi target devices by using ov::device:properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Support AUTO to set nstreams with multi target devices bu using ov:device::properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Fixed NonZero to have sorted result.
Fixed NonZero cldnn unittest to compare with ngraph reference code.
* Applied review comments
* Added more tests for 1d,2d,3d
* Fix local mem to be read from the param
* [TF FE] Support GRUBlockCell
Currently, we support only hidden state output from GRUBlockCell due to
OpenVINO GRUCell capability
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style issue
* Return Softsign translator
* Add tests for GRUBlockCellReplacer
* Fix issue for bias in GRUBlockCellReplacer
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* ShapeOf test: convert from LayerTestsCommon to SubgraphBaseTest and enable it
* Drop tail underline from description
* Rename filterCPUInfoForDevice -> filterFormatsInfoForDevice
* Use more suitable function name and reuse makeNgraphFunction
* Allow to compile wheel package based on OpenVINO developer package
* Added wheel specific runtime dir to avoid issues with generator expressions
* Added backward compatibility
* Trying to fix RHEL, U20
* Fixed cmake options in setup.py
* [TF FE] Support BlockLSTM operation
Due to restrictions of LSTMSequence, currently we support
only concatenated hidden states from all time steps
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply feedback: opset9 use, remove redundant code, preserve tensor name
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply feedback: check for dynamic dimension, non-nullptr for decoder use
* Copy run-time info using NodeRegistry
* Prepare test placeholder for internal transformations and operations
Clean-up includes
* Implement unit-tests for BlockLSTMToLSTMSequenceOneOutput transformation
* Apply code-style changes
* Fix computation of hidden_size
* Fix output tensor name
* Revert incorrect changes
* Fix a type of hidden_size in tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Added auto-detect for SHUTDOWN_PROTOBUF option
* Hide some helper variables as internal ones
* Update frontends cmake files not to include tests folders
* Added an ability to test private API for TF FE
* Added an ability to test private API for TF FE
* Improved mock frontends in python tests
* Fixed compilation error
* Fixed link on WIndows
* convert() method added.
* Moved conversion to convert() method.
* Fixed commits.
* Output dir fix.
* Added objects support for extesions param.
* Added support for transformations_config extension objects.
* Input to str unit tests.
* Added tests, added comments.
* Updated BOM.
* Removed commented code.
* Fixed extension passing.
* Small corrections.
* Fixed for python 3.6.
* Small fix.
* Moved dir creating to ov.serialize(), removed mo.serialize(), small fixes.
* Small fix.
* Small correction.
* Removed coping of params, moved convert implemetation to separate module.
* Import fixes.
* Moved hiding of exceptions to main().
* Updated comment.
* Fixed unit tests.
* Comment changed.
* Fixed dir creating.
* Tests fixed.
* Small fixes.
* Test fix.
* Added meta data generation, removed printing of execution time for silent mode.
* Import fix.
* Conflict fix.
* Fixed error.
* Fix for custom config.
* Added version, data_type params to help.
* Added mo.convert() full-functional tests.
* Small corrections.
* Comment correction.
* Moved convert to openvino package, moved LayotMap and InputCutInfo to openvino.convert.
* Added help param.
* Wrong change removed.
* Small fix.
* Removed unnecessary comments.
* Removed .xml extension check from append_ir_info.
* Added missed file.
* Fixed error.
* Fix for bool value in InputCutInfo.
* Moved InputCutInfo, LayoutMap to openvino.tools.mo.
* Moved InputCutInfo, LayoutMap to openvino.tools.mo.
* Moved check and read_model to emit_ir.
* Small correction.
* Added comment.
* Added unit_tests with convert().
* Small corrections.
* Removed convert alias from openvino.
* Fixed conflicting unit tests.
* Removed unnecessary warnings.
* Params check fix.
* Small correction.
* Added paths checks.
* Added negative tests for to_str methods, fixed errors.
* Added tuples support in input parameter.
* Moved reminders to update OV and use API 2.0 to main().
* Returned .mapping file generating.
* Added positional input_model param.
* Added test for unnamed input_model.
* Optimize imports.
* Added more informative error for brackets syntax in --input.
* Conflict fix.
* Conflict fix.
* [CPU] rnn: prepare memory according to pd of real primitive used
* Remove internalBlobDesc initialization, add node name to error message
* Add CPU functional test switching between batch sizes
* fix LSTMCell input data range
* Change W/R/B initial value ranges and abs_threshold only for dynamic batch test
* GRUCell fusion transformation and tests
* Enable GRUCell fusion transformation in MOC
* add missed include
* add missed include again
* fix warning in the tests
* Apply review comments
* Use NodeRegistry in GRUCellFusion transformation, apply review comments
* Add support for both formats zrh and rzh
* move additional checks to pattern, remove the transformation from MOC
* fix conflict with master branch
* apply review comments
* Update src/tests/functional/inference_engine/transformations/common_optimizations/gru_cell_fusion.cpp
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* fix warning
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* Update CI trigger rules
* Fixed CI run
* Fixed CI trigers
* Disable trigger for .ci
* Revert "Disable trigger for .ci"
This reverts commit 92c812ccb0.
When runtime exception was thrown during infer call, the infer
request handle was never relinquished nor condition variable notified.
That caused the get_idle_request_id function wait forever.
* Adding Tensorflow2 Keras tests to layer tests
* Fixed usage of use_new_frontend parameter in layer tests of
Tensorflow2 Keras
* Tests changed to use --use_old_api instead of --api_2 argument
* Added copyrights in files
* Added TF2-Keras Activation test to CI check
* detection_output kernel support cross-type(fp16/fp32),
so graph-optimization also support cross-type detection_output
Signed-off-by: hyunback <hyunback.kim@intel.com>
* When set batch_size, disable auto batching
* auto-batch only supprt auto plugin
* Set property allow_auto_batching for devices
* cancel static
* Clean Code
* Add test case for testing allow_auto_batching
* Fix the issue after merging latest master branch code
* Add allow_auto_batching in python benchmark_app
* Fix the quotes issue
* When load_network is set to allow-auto-batching, there is no need to read the value of global
* Fix the clang-format
* Change the location
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* port installing from archive
* fix formatting in code
* Updating archive names in qsg (#12927)
* port 2022.2 install guide changes to master
* small fix
Co-authored-by: Artyom Anokhov <artyom.anokhov@intel.com>
* New Homepage for Tutorials
Modifying tutorials.md
Adding separate article for notebooks installation guide: notebooks-installation.md
Updated to version from 13.07.2022
https://repository.toolbox.iotg.sclab.intel.com/projects/ov-notebook/0.1.0-latest/20220713220805/dist/rst_files/
* Update consts.py
* Update consts.py
* Updating tutorials to version from 13.09
Adding missing notebooks:
115, 203, 219, 220, 221, 222, 223,
* Update docs/tutorials.md
* Updating meta tags
Updating meta description and keywords
* Update notebooks-installation.md
* Update tutorials.md
* [CPU] Added support AVX2 and SSE for jit_emu_vcvtneps2bf16
* Upped thresholds
* check condition
* Removed thresholds
* Added kernel test
* Fixed AVX2 and SSE impl: removed AVX512 instructions
* come back
* Fixed jit_emu_vcvtneps2bf16 sse in=out
* update store_emitter for sse
* Added I32 test and updated tests
* united avx2 and sse impls
* Fixed ymm/xmm for avx512
* United with avx512_bf16
* fixed unused variable warning
* unified float2bfloat in convert_emitter
* Solve some code style issues
Change-Id: I06c31ad9513463c758db96dbe33031983a8fd7a5
* Refine ov_dimension with C style structure
Use C style definition to replace opaque C++ handle:
1). Improve partial shape
2). Improve dimension/rank
3). Improve shape
Change-Id: I680fd3e76c63d0c06d9d8b07c67af0440fb15bb7
* Some minor update
Change-Id: I237490a765e6ecd6f3896b1bd75d5f962304f43b
* add partial_shape_is_dynamic and update doc
Change-Id: Ief358f27de8386e35485d08847c8243d1490157f
* Rename property and add basic test cases
Change-Id: Ifbed0bbf7a5e836d2cab186763d7e5f8f2cb7d44
* address reviewer's comments
Change-Id: I9a5d1ede6d086906f3777c3d29237986a505db52
* Fix compile error
Change-Id: I72267a8dda32dd8f43c1ea1522d07b392052358f
* Use ov_any_t to replace ov_property_value_t
Change-Id: Ia4062d0da0941c901990bf5f16497ae3c8f2eb0f
* update doc for ov_model
Change-Id: I80183394f16dd1e07ff998993e26f8b903f2402e
* Property key apply string variables replace enum variable
Change-Id: I0e1699862de3913ccdc06ded5b8f8ac5a2bf8773
* Fix compiling error in windows
Change-Id: I4f9c7c746e218db94491c5f4bc5aeda1b3fde62b
* Change init/deinit method to create/free, remove unnecessary methods
Change-Id: Ibc5de039da94e92db494583ab1ff1a63e0586c73
* Pass by value for single C structure data
Change-Id: I289eb56e2ba1ab90942c8c801c585ab0f792952f
* fix dllexport issue in windows
Change-Id: I1632375e5e651fdfa076cdab2d8d1c3e83986549
* Fix static build error
Change-Id: I6633fe4887edad0b3a8b7e10730fb9d36a6ac0cd
* Use pass by value for all transparent structure
Change-Id: I652f6e70b420ef7c5af5cff3f5d482911cf0d75a
* Unify parameter order in C APIs, input first and then output parameters
Change-Id: Ib89c77d81c314e3a81bfcec9917630abbc896f5e
* Rename preprocess api to be more readable
Change-Id: I0a8ecd005e73e4ba8e90a8f25bedf0739fe31836
* Simplfy property implement for set and get property
Change-Id: Icad41606368cef9520b1623f6207784c295f1b9f
* check string type property issue
Change-Id: Ie8ee6995dda401f1db0cc098e061ffcfd24de92a
* Fix memory leak
Change-Id: Ife8980736be320314eca6fb3f3605e019663e1da
* Check string property data
Change-Id: I542a136ff874cb345c67f7d3c1dbc74981a909df
* Fix issues caused by rebase
Change-Id: I3d675a16cf76980a6414e369a83464a46dec1e49
* Remove some redundant code
Change-Id: Iab6bf7f355ec7a7424d884e620319ed9db80ecf5
* Remove property type from interface
ov_core_set_property will have 2 cases:
1) enum type: <const char* property_key, ov_property_xxx_e property_value>
2) other: <const char* property_key, const char* property_value>
Change-Id: Id7dec1ac40948d45c4f8baee687652aee92ea75d
* Remove redundant code
Change-Id: I66865d3313e305b0109cb63bd049772a01c0688f
* Remove exception log and update propery size argument
Change-Id: Ia1a60c07c9adf08d175d94a7d5e4298adb27bf80
* Apply string for enum property
Change-Id: I591b1db453a52b5ea9000e0cc3e11edf0ef5f790
* Add test case for MULTI_DEVICE_PRIORITIES
Change-Id: I59e4307564a2560724b4dacdb83c3ed45ea1ce8d
* Fix test issues and add more negative test case
Change-Id: Ib849fb8c098e487a537c1b1ab881f5e905871f73
* [MO] Extend support for TF BlockLSTM with the single output, concatenated hidden states
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Remove extra comments
* Apply the code review feedback: return useful logs, more logs for asserts
* Add unit-test for shape infer
* Add unit-test for transformation
* Fix a bug with adding input port for BlockLSTM
* Apply code-review feedback: no need to create existing port, add more comments and correct
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* refactor of renaming libraries in layer tests
* 1. adds check for old API and new FE usafe
2. refactor of api_2 arg
* fix for tf_NMS test preprocessing
* take libs path from LD_LIBRARY_PATH env
* convert str to Path object
* use wheels path to libs
* print lib paths
* print lib paths
* use ov_frontend_path env
* also check if file to rename exists
* removes redundant prints
* copy instead of rename
* 1. copy instead of rename
2. adds some details to readme
* Building python frontend tests
* Enable mock tests in new python API:
- Fix mock_py frontend library name
- Set same destination for mock_py frontend as others frontends
- Fix importing mock_py frontend
- Minor changes in mock_py tests
* Fix flake8 issue
* Fix linking issue on windows
* Remove DISABLE_PYBIND11 option
* Build mock_py always as shared library
- put mock_py into ov::frontend namespace
- minor re-factor of mock_py
* Add mock c API fix warning on windows
* Fix installation of mock_py frontend
* Install mock_py frontend library in tests
* More detailed error message in python test
when cannot import frontend
* Restore IMPLEMENT_OPENVINO_API for windows build
of pybind_mock_frontend
* Set env paths for python API test on Azure CI
- skip tensorflow conversion ext test when not installed
* Correct skip tensorflow test decorator
* Add C API macro for mock1 frontend
* Ignore E999 error reported by flake8
* Fix conversion extension name for tf imports
* Fix flake8 issues
* Remove python3.8 path from LD_LIBRARY_PATH
on debian CI
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Initial
* packaging: Fixed syntax
rpm.cmake: downgraded PYOPENVINO version to 3.6 as default
* install_build_dependencies: Added rpm-build
rpm.cmake: Removed SHLIBDEPS settings for RPMs creation. Replaced CPACK_RPM_PACKAGE_HOMEPAGE with CPACK_RPM_PACKAGE_URL. Set CPACK_RPM_PACKAGE_AUTOREQPROV to ON.
* rpm_post_build.cmake: Fixed warning if rpmlint not installed
rpm.cmake: Removed SHLIBDEPS usage.
* rpm.cmake: Updated comments
* Fixed naming
* developer_package/rpm.cmake: Fixed ov_rpm_add_latest_component macro with using lower case for CPACK_COMPONENT_XXX_DEPENDS set.
* DOCS-doc_structure_step_2
- adjustments to the previous change based on feedback
- changes focusing on ModelOptimizer section to mitigate the removal of ONNX and PdPd articles
* remove 2 files we brought back after 22.1
* [ONNX FE Tests] Fix ONNX FE tests on Debian
Fixing error "Cannot load library libtest_builtin_extensions_1.so libopenvino_tensorflow_fe.so"
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update .ci/azure/linux_debian.yml
* Export TF FE for PDPD tests since it uses TF FE extensions
* Export TF FE for PDPD tests since it uses TF FE extensions
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Required by external data mechanism in ONNX standard, where Tensor object has
to be able to find external data file based on the location field and model
directory path. Previously, it was done by a transformation, but it handled
initializers only, but in order to handle Constant nodes we need one more
loop, but over all of the model's nodes. Propagating model directory path to
Tensor allows us to reduce that overhead.
Ticket: 91271
* add paddle where_index op
* add more test cases
* add more test cases
* add more test cases and support boolen
* fix clang format
* use paddle.nonzero instead of LayerHelper
* remove opset6
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
Co-authored-by: cecilia peng <cecilia.peng@intel.com>
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
+ Resolved failure of oneDNN reduction which has fused output layout.
+ Removed planar format from oneDNN reduction which selected ref kernel.
+ Modified relevant test-cases
Signed-off-by: Min, Byungil <byungil.min@intel.com>
StridesOptimization propagates strides attributes up the graph.
This attribute is kept in Input<Node> runtime info.
There is a case in CUDA plugin, where StridesOptimization is called
twice and if strides attribute is kept in node's runtime info,
the second run of this transformation tries to propagate strides
once again which can result in shape inference failure.
* [Frontend, TF FE] Fix RTTI for ConversionExtension on MacOS
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Put only destructor into cpp
* Remove extra white-space
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* add paddle op sum
* new version
* add unit test for other precision
* change layer.sum to add_n
* remove redundant code
Co-authored-by: Bo Liu <bo4.liu@intel.com>
Co-authored-by: cecilia peng <cecilia.peng@intel.com>
* Add ov_model_output_by tensor name and index function
* Fix the clang format issue
* clang format
* Add ov_output_node_t in input and shape function
* Add ov_oputput_node_t in model_output function
* Fix clang-format
* change ov_output_node_list_t to ov_output_const_node_list_t
* Add ov_output_node_list_t new struct
* Rename oputput_node to be port
* Rename output_node to be port in CAPI test and samples
* Add ov_model_output/outputs/input/inputs function and testcases
* Rename output_ports to ports
* Chage ov_node_list_get_element_type_by_index to ov_node_list_get_element_type
* Change ov_output_node to ov_output_port
* Delete node list and modify the function that used the node list structure before
* Fix clang-format in ov_infer_request_test.cpp
* change node to port in funcation name
* Rebase new master
* Add new API in compile model and model,such as get input/output size
* Add err info in hello_classification_c
* Fix clang-format in hello_classification_c
* Merge the new infer_request
* Initialize ov_output_const_port
* Change the location of initialization
Co-authored-by: River Li <river.li@intel.com>
* Refactor eye decomposition:
- match style of other transformations
- add NodeRegister class to make and collect created nodes
- use `NodeRegister` in transformation for copy runtime info
- use `NodeRegister` in `MatcherPass` to replace new `register_new_node`
* Fix CI LTO optimization issue for Ubuntu 20
* Move NodeRegister to dev API
To fix LTO issue for eye decomposition
* Change NodeRegister to NodeRegistry
* Move NodeRegistry to openvino/pass
* Remove NodeRegistry from public API
* Only rename cpuUnitTests and cpuFuncTests to ov_cpu_unit_tests and ov_cpu_func_tests
* Change ov_cpu_unit_tests's displayName to readable, from 'CPU UT' to 'Intel CPU Unit Tests'.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* [TF FE] Support Wide&Deep family models by TensorFlow FrontEnd
It adds support for operations SparseToDense, transfomations for SparseReshape and
fusing transformation for EmbeddingSegmentSum operation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code style for sparse_fill_empty_rows
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Complete transformation for embedding_segments_feature_fusing
* Fix signatures of constructors of internal ops: SparseFillEmptyRows, etc.
* Add override for validate_and_infer_types methods of the internal ops
* Remove unused variable in Unique
* Revert incorrect changes for default type in case graph cutting
* Make InternalOperation class and apply code feedback
* Revert changes for pipeline.py
* Remove apply_middle_transformation from frontend.hpp
* Recover apply_additional_conversion_rules
* Eliminate a number of constructors for internal operations
* Improve code readibility for embedding_segments_feature_fusing transformation
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue: returning reference to temporary
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add access to the hidden Node::evaluate method in each Ops
* Fix the test
* Add new line in EOF
* Add comment about using ov::op::Op
* Use opset9
* Add more detailed comment
* Added support of pugixml search via pkg-config
* Updated old InferencEngineDEveloperPackage.cmake.in
* Removed debug print
* Fixed cmake stage for U18
* Added WA for pugixml on U18 for DevPackage as well
* Disable pkg-config generation for OpenVINO for U18
* Added option for pkg-config
* Now triplet is aut-detected
* Fixed U18 case
* Fix the new infer request to set output memory normally during enqueue stage
* Fix to use the generated outputsMap instead of graph API to retrieve outputID
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add CC support for tbbbind
* Use throw to replace return
* Tbbbinder will be optimized out for non-NUMA
1. With NUMA we may expect that some TBBBind API is optimized out, but some of the API is definitely here
2. The same for HYBRID cores on Alder lake
3. For no NUMA nor HYBRID I expect no TBBBind API is used
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Debian arm cross-compilation
* Pugixml::shared as system library
* Removed debug print of openvino.pc
* Set debian package arch for cross-compilation manually
* Solve some code style issues
Change-Id: I06c31ad9513463c758db96dbe33031983a8fd7a5
* Refine ov_dimension with C style structure
Use C style definition to replace opaque C++ handle:
1). Improve partial shape
2). Improve dimension/rank
3). Improve shape
Change-Id: I680fd3e76c63d0c06d9d8b07c67af0440fb15bb7
* Some minor update
Change-Id: I237490a765e6ecd6f3896b1bd75d5f962304f43b
* add partial_shape_is_dynamic and update doc
Change-Id: Ief358f27de8386e35485d08847c8243d1490157f
* Rename property and add basic test cases
Change-Id: Ifbed0bbf7a5e836d2cab186763d7e5f8f2cb7d44
* address reviewer's comments
Change-Id: I9a5d1ede6d086906f3777c3d29237986a505db52
* Fix compile error
Change-Id: I72267a8dda32dd8f43c1ea1522d07b392052358f
* Use ov_any_t to replace ov_property_value_t
Change-Id: Ia4062d0da0941c901990bf5f16497ae3c8f2eb0f
* update doc for ov_model
Change-Id: I80183394f16dd1e07ff998993e26f8b903f2402e
* Property key apply string variables replace enum variable
Change-Id: I0e1699862de3913ccdc06ded5b8f8ac5a2bf8773
* Fix compiling error in windows
Change-Id: I4f9c7c746e218db94491c5f4bc5aeda1b3fde62b
* Change init/deinit method to create/free, remove unnecessary methods
Change-Id: Ibc5de039da94e92db494583ab1ff1a63e0586c73
* Pass by value for single C structure data
Change-Id: I289eb56e2ba1ab90942c8c801c585ab0f792952f
* fix dllexport issue in windows
Change-Id: I1632375e5e651fdfa076cdab2d8d1c3e83986549
* Fix static build error
Change-Id: I6633fe4887edad0b3a8b7e10730fb9d36a6ac0cd
* Use pass by value for all transparent structure
Change-Id: I652f6e70b420ef7c5af5cff3f5d482911cf0d75a
* Unify parameter order in C APIs, input first and then output parameters
Change-Id: Ib89c77d81c314e3a81bfcec9917630abbc896f5e
* Rename preprocess api to be more readable
Change-Id: I0a8ecd005e73e4ba8e90a8f25bedf0739fe31836
* Generate openvino.pc pkg-config file
* Added libva-dev as a dependency
* Fixed typo in install_build_dependencies.sh
* samples on CI
* Revert changes in samples; use pkg-config in tests
* Debug print
* Revert changes in fuzz tests
* Fixed TBB usage
* Fixed pkg-config usage for cross-compilation
* Fixed pkg_config_tbb_lib_dir
* Don't use PKG_CONFIG_EXECUTABLE unconditionally
* Fixed copy-patse
* Fixe
* Generate pkg-config file for Apple as well
* Fixes for ubuntu 22.04
* Disable apple
* [TF FE] Add loaders: EuclidianNorm, Erf, Bucketize, BroadcastTo, BroadcastArgs
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix a name for translate_broadcast_args_op
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add loaders for Einsum and Mish
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* WA for Benchmark App: Remote tensor use for internaly dynamic original models (NMS) that turn static due to transformation to legacy op and lack of dynamism support
* Style
* Fixed dependencies in case of debian package building
* Removed duplication of cython-based libraries in wheel
* Fixed that TBB is not part of the wheel
* Fixes for yocto
* Add NVIDIA device name along to CUDA device name (temporary solution)
* Disable nvidia_plugin along to cuda_plugin in .ci/azure/ (temporary solution)
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* Updated ia32 toolchain with system / processor
* Allow pkg-config during cross-compilation
* added error message
* Search TBB using tbb.pc as well
* WA
* Fix for yocto
* Added print
* Use tbb.pc only when system libs are enabled
* Improved OpenVINOConfig.cmake
* Don't find pkg-config as required
* Add eye decomposition transformation
* Fix EyeLike generation when diagonal shift
outside dimensions
* Add batch shape to eye decomposition
* Eye decomposition clean-up
* Remove reference part if no eye decompose in
decomposition tests
* Eye-Like use eye operator
* Disable eye decomposition for CPU plugin
* Use opset9 instead of ops in eye decomposition
* Apply transformations header style
to eye_decomposition.hpp
* Add model reference in eye decomposition tests
- use opset9 instead of ov::op:vX namespace
* Refactor eye decomposition:
- match style of other transformations
- add NodeRegister class to make and collect created nodes
- use `NodeRegister` in transformation for copy runtime info
- use `NodeRegister` in `MatcherPass` to replace new `register_new_node`
* Fix CI LTO optimization issue for Ubuntu 20
* Move NodeRegister to dev API
To fix LTO issue for eye decomposition
* Change NodeRegister to NodeRegistry
Avoid extra transposes in conversion of FakeQuantWithMinMaxVars and
add translator FakeQuantWithMinMaxVarsPerChannels.
It allows to convert MobileNetV2 with FakeQuantWithMinMaxVars and
FakeQuantWithMinMaxVarsPerChannels operations.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Add Transpose Sinking for Prelu operation
Now it covers a case with a scalar slope.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add unit-tests for Transpose sinking of Prelu
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix non-scalar slope case
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Move onnx_tests target dir
* Move onnx_frontend_tests target dir
* Move standalone_build target dir
* Move onnxImporterUnitTests target dir
* Combine onnx edytor tests
* Remove excessive target
* Merge onnx tests into one binary
part 1 : onnxImporterUnitTests into onnx_tests
* Merge onnx tests into one binary
part 2 : onnx_frontend_tests into onnx_tests
* Fix code style
* Update CI files
* Move onnx model test files
* Rename onnx tests target
* Avoid plurarity in target name
Rationale: Letter s and the end tells about implementation and
structure, while the purpose is singular - to validate OpenVINO ONNX
Frontend.
* Avoid loosing manifest filter flag
Issue occurred when gtest_filter was provided from command line
* Remove conflicting dependency
Linking to gtest_main (from googletest) via commonTestUtils caused
conflict while choosing main() function. OV defined in gtest_main_manifest
is prefered here as consider manifest file along with CLI arguments.
* Use proper naming
- Implement static TileScheduler to handle compile params processing. Now compile params are accessed only here
- TileScheduler should emit code only for necessary scalar/vector Tiles
- Perform abstract-to-physical register mapping in one place (currently KernelEmitter constructor)
- Implement more precise register mapping, so larger subgraphs could be created (now up to 12 i/o regs instead of 7)
Co-authored-by: Ivan Novoselov <ivan.novoselov@intel.com>
* tf_fe lib renaming to switch to new fe
* Apply suggestions from code review
* 1. implements renaming for win os
2. --api_2 key is equal to True by default
* --ir_version key is equal to '11' by default
* resolves merge conflicts
* check if file to rename exists
Co-authored-by: Evgenya Stepyreva <eva.my.link@gmail.com>
* fix unicode issue that ie.ReadNetwork fails if the exe's folder contains Chinese characters
* fix clang format issue
* fix clang format issue
* modify '/' to file_separator in get_directory
* fix ci issue
* fix test failure on windows
* fix test issue
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* [SAMPLES] Add GNA library version log to speech_sample
* [GNA] Add OV version to TLV export file
Co-authored-by: Szymon Irzabek <szymon.jakub.irzabek@intel.com>
* Revert "Fix experimental detectron do ref impl (#10621)"
This reverts commit d87233863d.
* Disabled Experimental Detectron per agreement with GPU team. Ticket to fix it: 90209
when cpuFuncTest and libopenvino.so in sampe directory
on redhat8, cpuFuncTest can't find front library path
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* Add eye decomposition transformation
* Fix EyeLike generation when diagonal shift
outside dimensions
* Add batch shape to eye decomposition
* Eye decomposition clean-up
* Remove reference part if no eye decompose in
decomposition tests
* Eye-Like use eye operator
* Disable eye decomposition for CPU plugin
* Use opset9 instead of ops in eye decomposition
* Apply transformations header style
to eye_decomposition.hpp
* Add model reference in eye decomposition tests
- use opset9 instead of ov::op:vX namespace
* Refactor eye decomposition:
- match style of other transformations
- add NodeRegister class to make and collect created nodes
- use `NodeRegister` in transformation for copy runtime info
- use `NodeRegister` in `MatcherPass` to replace new `register_new_node`
* Fix CI LTO optimization issue for Ubuntu 20
* update logic of setting TPUT by default for MULTI and AUTO.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set config to MULTI instaed of target device when using -d MULTI:target_device in benchmark app.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update logit to check if perfomance hint is set.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Set the correct _so from passthrough executable network when batch plugin is disabel to fix lifecycle coredump.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Same update for AUTO plugin
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* CVS-89672 Make model reshape and track batch
* Minor refactoring
* Changed mechanism of constant replacement to more mature
* Update src/common/transformations/include/transformations/smart_reshape/lstm_states_broadcast.hpp
* Update src/common/transformations/src/transformations/smart_reshape/lstm_states_broadcast.cpp
* [GPU] Integrate oneDNNv2.7
- Support bsv16_fsv2, bsv16_fsv4
- Update qunatize LWS opt and common GetOptimalLocalWorkGroupSizes.
- Enable bs_fs_zyx_bsv16_fsv2 in eltwise.
- Update to use binary_mul in case NOT int8 conv with oscale.
- Update oneDNN v2.7pc2 branch in onednn submmodule
- Apply shallow conv in/out condition, using acdb,acdeb
mixed precision b32, f32 case, newly add weight reorder format
- Update to fix remains ref_conv selelction issue and running failure issues.
- Update to fix shallow out and normal in case creates unnecessary reorder.
* Fix different in/out format issue in concat
- Concat supports in/out different format, but it has issue calculating concat axis.
After moving cpuFuncTest to cpu_plugin folder, there is more stricter grammar check.
ngPrc is declared twice, but for the first declaration, the ngPrc is not used.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Disable python for configuration with libraries suffix
* Fixed python detection on arm
* More fixes
* Used find host package
* Don't use PYTHON_VERSION which is used by pybind
* Add eye decomposition transformation
* Fix EyeLike generation when diagonal shift
outside dimensions
* Add batch shape to eye decomposition
* Eye decomposition clean-up
* Remove reference part if no eye decompose in
decomposition tests
* Eye-Like use eye operator
* Disable eye decomposition for CPU plugin
* Use opset9 instead of ops in eye decomposition
* Apply transformations header style
to eye_decomposition.hpp
* Add model reference in eye decomposition tests
- use opset9 instead of ov::op:vX namespace
* Refactor eye decomposition:
- match style of other transformations
- add NodeRegister class to make and collect created nodes
- use `NodeRegister` in transformation for copy runtime info
- use `NodeRegister` in `MatcherPass` to replace new `register_new_node`
* [TF FE] Refactor translators for Resize operations and correct Pooling
It allows to convert magenta_arbitrary-image-stylization model
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Align TF FE tranlator for Resize with legacy frontend
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Do minor fix for MaxPool
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add overrided method to generating vector of strings
* Trim the value from the the left and right
* Add test to verify that output names are correctly read from IR
* Use spaces instead of tabs
* Add C++ tests for read model contains outputs with whitespaces
* Fix test for add output
* Remove python test
* Warning suppression for C-API.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Add compiler flags for CI verification.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Update parent cmake compiler flag, and make it work for total bindings folder.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Open compiler check.
Move "sign-compare" to below, and make it only work for bindings. Because plugins have many compare errors.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Remove sign-compare.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Support a branch without masks in matmul
* Init transpose mask propagation
* Transpose mask propagation + tests
* Unsqueeze mask propagation support / Fix reshape unsqueeze
* Init Matmul batch dims mask propagation support
* Matmul batch dims mask propagation support
* Broadcasted elementwise support
* Unsuccessfull attempt to remove masks from unit dims in eltwise
* Stricted way to pass broadcasted dims through elementwise
* Refactor reshape
* ReshapedPassThrough propagation
* Hack to enable matmul mask propagation up from right branch to left
* Make ReshapePassThrough common (tests are needed)
* Hacks to enable Bert int8 pruning
* Not reshapeable and big pattern for ReshapedElementwise operation version of transformer pruning
* Reshape shape subgraph support
* Init shrink reshape mask propataion support
* Init reshape shrink mask propagation
* Shrink extend reshape mask propagation suport init
* Complex reshape test
* Duplicated code removed
* Fix accuracy check tests
* Fix rebase
* Fix broadcasted elementwise / update tests
* Fix types
* Remove redundant methods from Mask
* Fix transpose propagation in Matmul mask initialization
* Migrate to CoordinateTransformer
* Add comments
* Fix types
* Make elementwise inputs deterministic
* Fix tests
* Refactor shape size collecting
* Fix rebase
* Fix comments
* Fix comments
* Test MatMul with two outputs is added
* Mask methods comments was added / get_constant_from_node test
* [TF FE] Add dynamic rank support for Convolutional and Pooling operations
Refactor DepthwiseConv2D, AvgPool, and FusedBatchNorm operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue with rvalue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue with climit
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Skip duplication of Parameter nodes
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert changes in StridedSlice and add check for AvgPool operation type
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert the rest of changes for StridedSlice
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix translator for AvgPool: add pad mode
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Introduce helper default_op_checks
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Add Transpose Sinking for additional unary-wise Operations
It helps to fix performance degradation for MobileNet models
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add LogicalNot for Transpose sinking
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* checking config priorities is set by checking if the value of priorities is empty instead of not founding in the config map.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add test case for MULTI without setting device priority.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Created dedicated debian packages for frontends
* Fixed Azure CI
* Install TF FE with both names
* Disabled 2 QN tests
* Install python samples differently
* Fix openvino_contrib version
* [linux_arm64] Fix openvino contrib version
* [linux_debian] Fix openvino contrib version
* Azure: Try to take into account result of previous step as well
* Removed excess contrib var
* Disable CUDA plugin build via cmake flags
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
* Fix openvino_contrib version
* [linux_arm64] Fix openvino contrib version
* [linux_debian] Fix openvino contrib version
* Disable CUDA plugin build via cmake flags
* add value verificatioin for the key ov::device::priorities and corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Retrieve the device name via the DeviceIDParser rather than via the priorities directly.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add logic to support devices like CPU(x),GPU(x).
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add 'mock' as a available device name when checking if ov::device::priorities is correct.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update format.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* do not check if priorities is available when value of priorigies is empty.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* implement can_memory_reuse flag
* apply the option to reduce opt kernel
* add reduce opt kernel for the option
* apply feedback
* consider when user->selected_impl() = nullptr
* apply recursive function to check optimized user
* support blocked format for broadcast
* remove redundant reorders and reshape so broadcast selects blocked format
* implement broadcast ngraph reference and added blocked format test cases with values bigger than block size
* fix cpplint error
* fix cpplint error 2
* review comments applied
* review comments applied: cleaner GetInputBlockND function
* change from program_helpers::are_layouts_identical() to layout::compatible()
* change from layout::compatible() to layout::identical()
* [TF FE] Implement translators for ExtractImagePatches and MatrixDiag
It allows to convert Inpaint model and infer it correctly
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: correct comments, use set
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply suggestions from code review
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Fixed:
SC2070: -n doesn't work with unquoted arguments. Quote or use [[ ]].
SC2086: Double quote to prevent globbing and word splitting
in TBB related part of the script.
* Create ONNX FrontEnd GenerateProposals op
* Add onnx GP Op validation
* Add batch 2 test
* Improve code readability
.. per review comments
* Fix test model paths
* Use heterogeneous test values
* Update linux and mac part of BUILDING.md
* Fix variable names and minor other fixes
* Remove unused macos deps
* Follow up on comments including cmake instructions
* Update wheel instructions
* [TF FE] Correct Result node names and output tensor names
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Return transformation pass with TransposeSinking
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Switch off TransposeSinking transformation due to bugs with preserving tensor names
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use just set_friendly_name for Result nodes
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use get_friendly_name for composing Result node name
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix tensorflow tests for partially convert
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Recover header for manager
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
The convert node was extended to support the conversion from Precision::BIN precision. The issue with FakeQuantize-1 conformance tests failures is related to the convert node. In the case of binarization, the FakeQuantize node is transformed into a binarization node and the convert node. But the convert node didn't support conversion from Precision::BIN precision. (#12149)
* [C API 2.0]Redifine partial shape and property wrapper
1. Use dimension object to initialize partial_shape rather than string
2. Use void* to unify property value rather than union
3. rename some C api name to strict align with C++ method.
Change-Id: I64b5c521461264dba2d23543808584632fbd6d4b
* [C API 2.0]Memory check and implement all reshape interface
1. Memory safe create and free
2. Implement all reshape interface align with C++ interface
3. rename some api to align with C++ interface
Change-Id: Ib5e4192bdbd8a11cdd7e30b1dc84881ba3f2d505
* Rename prepostprocess to strict align with C++ name
Change-Id: I7a4d0a6e835b2d6ed01cd218ac81b1b621f600bf
* [C API 2.0]redefine ov_node and ov_model interface
1. redefine ov_node and ov_model interface
2. rename some api to aligne with C++ interface
3. remove some redundant code
4. align CMakeLists.txt with OpenVINO 2.0 convention
Change-Id: I4d5e92157e7891319c9754da8e70b9c6150ae2e3
* Redefine ov_layout to support more than one char
Change-Id: I39e5389246cf3edcc2f4734d13157457773d89b8
* Add interface to get partial_shape from node
Change-Id: I8cef77db581b43d2f0a9ac48cfdc09a86e39b694
* Use unique_ptr prevent memory leaks in case of exception
Change-Id: I150b375108a3eded400bdde087ab5c858958c25f
* Put legacy C API and 2.0 C API into a library
Change-Id: I067a55a00e78b80cdede5ae7adad316ee98cabd1
* Only keep OV 2.0 C sample and move legacy C sample to legacy directory
1. Move legacy C samples to tools/legacy/c directory
2. Keep OV 2.0 C samples in samples/c directory
Change-Id: I05880d17ee7cb7eafc6853ebb5394f3969258592
* Fix format and log issues
Change-Id: I05d909b3d7046d41b807e35808a993bb09672e68
* Restore documents update
Change-Id: I82dd92081c0aa1a2d7dca7f114cf6a35131d6f92
* Change config map data be const
Change-Id: I9043859e8308c01d80794dc8280ae236947f3bbb
* Update api document
Change-Id: I35bc149bad0de17424d95f48c3027030b708e147
* Add clang enable
Change-Id: I335639c05fb5fb38e682dbb72bfaf78380c0adaf
* Fix clang issue after enable clang for ie_c_api.c
Change-Id: Idcb4dda9d66e47a169eb79a9c4fe7a7d4df838db
* split header file and c file into multiple files
Change-Id: I7c3398966809ef70d7fcb799f2d612a33b471e31
* Fix clang format issue
Change-Id: Ibd18b45537c8f3bcbb5b995c90ae28999161d54d
* Add single ov_dimension_create method
Change-Id: Icd06b50e4f4df8f7897c7c4327edb67178162544
* Remove all legacy c samples completely
Change-Id: I098360a0d9002340e8769074181f7997b43bce8f
* Update ov_property_value to replace only ptr
Change-Id: I9f5a11b4cf07e759c1998e78e2624f0a1266d9b0
* Split more header files, add static dimension api
Change-Id: I14e4fb8585fc629480c06b86bd8219e75a9682f7
* Change ov_dimensions_create to be ov_dimensions_create_dynamic
Change-Id: I50c02749cea96f12bcea702b53a89c65b289550e
* rename status and get_out_tensor
Change-Id: I762c1d0c5a069454506fe3c04283c63ddbfacf31
* Split ov_c_api_test.cpp
* Split OV2.0 CAPI tests
* move var into Setup
* Merge legacy and 2.0 C API test
* Merge InferenceEngineCAPITests into openvino_capi_test
1. put InferenceEngineCAPITests into openvino_capi_test
2. resolve some format issues
Change-Id: I47bbba6bd70a871ee063becbd80eb57919fa9fb0
* legacy api test skips clang format
Change-Id: Id54ecdba827cf98c99b92295c0a0772123098b63
* Fix clang format issue
Change-Id: I7ed510d8178971fe04a895e812c261db99d8b9f2
* Restore InferenceEngineCAPITests
Change-Id: I4d641ffb1de9ce4d20ebecf35fc036fa7bd73e55
* rename openvino_capi_test to ov_capi_test
Change-Id: I6b6fe0cdb89aab7210abb17f32dbfdcdce72ba25
* unify list size name and refine ov_core_version_t
Change-Id: I137fc6d990c7b07f597ee94fa3b98d07ae843cb6
* align header file path to be openvino/c/openvino.h
Change-Id: I1a4552e1d558098af704942fe45488b0d6d53a90
* Fix path issue
Change-Id: I84d425d25e3b08c1516cbcc842fb9cb75574bf17
* move ov_color_format and remove opencv depenency
Change-Id: I486145f9e92e8bbf2e937d3572334aa9f0e68841
* Resolve some memory allocation error handling issues and read model with empty weight issue
Change-Id: Icd8e3b6de9741147993fa215a0c7cfd7debd5500
* Add GPU test cases
Change-Id: I13324ef019b5b1af79259ca932a36a0cec792c27
* Fix clang issue
Change-Id: I9bb4c47de301d142b5e2a77a39f667689ad9fe38
* Resolve CI test failure
Change-Id: Ia327d5edab19d8dd44ac369670f190d5c57aca79
* Redefine ov_shape and add default ov_core_create
Change-Id: I3e47d607f8aad65cb99cdddacaecf7bf34b1361b
* Remove some unnecessary api of node
Remove the unnecessary node api:
ov_node_get_any_name(ov_output_const_node_t* node, char** tensor_name)
ov_node_get_element_type(ov_output_const_node_t* node, ov_element_type_e* tensor_type)
Change-Id: I80a3243676800263a9e56afa3cfffce7b4bd2ae7
* Rename reshape api
ov_model_reshape should be common case which allow to reshape any models with different number of input.
Change-Id: I26bafeeb8a3dda7cd5164cda15fdb338db8668cb
* Rename ov_node api
Change-Id: I03114ecb6de5c46b6d02c909b6f6fb6c8bfd5cba
* Remove subfolder out of source code
Change-Id: Ib033ae7712cc0460d6fc21a0f89818381ae503c0
* apply absolute path for all header files
Change-Id: I8024c897d424b407025e21460ed4b62829b853d2
* Fix CI issue ov_capi_test failed to find libgna
Change-Id: I166e79a818498c6721fe956f43873f36d9ae1e07
* Resolve build issue to align with PR12214
Change-Id: I9e6094db213b431ee1b46e0d64199131db33bb36
Co-authored-by: ruiqi <ruiqi.yang@intel.com>
* OpenCV build switched off by default
* Removal of OCV copy from the "install tests" stage
* Switch off OCV dependent tests
* cmake code repositioning
* More OCV-dependent tests disabled
* OCV removed entirely from the C-API tests
* Final cleanup
* removed references to program_node
* removed member variables in primitive_impl
* removed redundant variables
* removed whithspace at the end of line
* added ctor for updated primitive_impls
* changed init_kernels to require kernels_cache
* removed program_node ref from custom_primitive
* added set_node_params to update the lastest status of program_node
* removed const from get_kernel
+ Creation of onednn_engine in ocl_engine is changed to on-demand.
+ benchmark cache_dir option takes longer than cl_cache_dir env in loading network.
+ For clDNN execution, benchmark cache_dir created onednn_engine if just ONEDNN_ENABLE config is ON.
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* [TF FE] Implement conversion for Attention OCR model
The following scope of work is done to make Attention OCR convertable:
1. Refactored translators for BiasAdd, Slice, and ArgMax operations. Add translation for StopGradient operation.
2. The previous traversing algorithm to compute topological sorted nodes list was incorrect. Now it is implemented based on topologically_sorted function from core/graph_util.hpp.
3. The unsupported data types are now preliminary converted to undefined type for the purpose of to have them cut off.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert workaround for support of non-frozen models
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Correct third-party dependencies
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply feedback from code-review
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback from PR12444
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply the rest of comments from the code-review
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Previously INPUT/OUTPUT_SIZES was declared as an unnamed array macro.
However current ocl compiler deals with the value obtained from such an array non-constant, so that it results in a side effect in compiler optimization.
Here, a workaround for this issue is applied, i.e., declare the INPUT_OUTPUT_SIZES as a __const array instead of jit constants.
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* First version of Windows installer
* WIndows NSIS installer
* Improved version of debian packages
* Improvements
* Improvements
* Debian packages now look good
* Library versioning
* Fixed tests to run against debian packages
* Fixed frontend tests
* Fixed code style
* FIxed Windows
* Fixed python tests
* Fixed paths in tests
* fdvfdv
* Fixes
* USe versioning only for debian packages
* Relocatable tests
* Fixed
* Fixed all tests
* Fixed clang-format
* Fixed more tests
* Fixed some tests
* Absolute paths in .ci
* Fixes
* Added support of OpenCV 3.4
* Trying to fix gnaUnitTests
* Method Node::constant_fold uses new evaluate method with TensorVector
* Simplify conditions for creating tensor
* Add test for evaluate with TensorVector
* Add line at EOF
* the first approach to fixing the Result edge naming issue
* more robust approach, not depending on the split
* out_nodes_ids type change
* missing if clause
* [TF FE] Fix StridedSlice translator for new_axis vector longer input rank
Currently, new_axis vector is cut by input rank that is correct and leads to the loss of new axes.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use int64 type in mask_to_vector function
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* WIP graph tests fixing
* Fix collectiors graph tests
* remove debug code
* Fix rebase
* eps update for scales tests
* Outputs for some reference models was changed
* Sanity reference metrics update for VNNI CI hosts
* Unused hyperopt dependency which broke python3.6 support is commented
* Minor comments fixes
* Update doc for AUTO and AUTO_BATCH
Signed-off-by: Chen Peter <peter.chen@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* [MO] Relax MO upper-bound requirements for TensorFlow and NumPy
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Just debug numpy version
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Pin upper-bounds for NumPy and TensorFlow modules in all reqs files
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update submodule dependency for open_model_zoo
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Install numpy module first
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy version in POT setup.py
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Extend telemetry tests with a set of possible solutions for events
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy module version for layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* get data type of convolution weights from node.weights() when network is internal
* use only instance.node.weights().get_output_layout().data_type
* fix typo
* add unit test for the case
* Add exception handling for calc_output_layout
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Apply comment to error handler
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Add static shared_objects map in FEM
- add unit tests for frontend lib close
- not use static FEM in ie network reader
- add main for gtest which can use manifest file to filter tests
* Move library pointers map to manger impl
- add to manger impl method to make frontend from loaded plugin
* Add shutdown function to ov namespace
it cleans the static resources
* Revert changes related to linking mian for tests
* Add python binding to ov::openvino_shutdown
* Renamed shutdown method and added to legacy C++ API
(cherry picked from commit a8395bd207)
* Added C bindings
(cherry picked from commit d2c9ddc263)
* Move frontend lib close test to ieFunctTest
- moved to not introduced new test binary and modification on CI
the frontend tests use dynamic linked frontend lib which is load
on test application start and mask lib close tests
- remove gtest_main_manifest as not required now
- add ov::shutdown test to expect application crash
* Fix lib_close test
- remove not get_disabled_tests from utils
- revert CMake file formating
* Fix get model path in lib close tests
* Skip frontend lib close tests if static lib build
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* add type caster for ov::Layout, enable load method to take pathlibs.Path as arugment
* fix typo
* fix style
* add missing blank line
* add common function to check if py::object is either Path or string
* fix style
* Update src/bindings/python/src/pyopenvino/graph/preprocess/pre_post_process.hpp
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* add tests, fix style, remove pointer argument overload
* fix style
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* Revert "Fixed 3 naming issue"
This reverts commit a92d3cfff5.
* Revert "Fix CC issues for transformation and snippets"
This reverts commit d08a3f5aac.
* Fix NGRAPH_PASS_CALLBACK issue to make it can work
* Fix matcher name missing issue
* Getting rid of dependency from get_default_param for typed_program_node
* Fix bug
* Enable two pathes to call choose_impl / does_possible_impl_exists / does_an_impl_exists to be able to use given layout
* Replaced impl factory API to get kernel_impl_param's pointer
* Update for recently added primitives
* Add and apply optional_layout
* fix kernel_param_impl to be handled as unique_ptr
* Applied review comments
* Fix rebase conflict
* Fix CI error
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* First version of Windows installer
* WIndows NSIS installer
* Test commit
* Revert "Disable loading of v7 reader for new IR versions (#12252)"
This reverts commit cb6ca7bb89.
* Revert "Test commit"
This reverts commit 977b83f2ba.
* Add const fold check in operators instead pass
- refactor constant fold pass to using ov instead of ngraph
- add constant_folding_is_disabled overload for raw pointer
* Remove Reshape from skip const inferences
in legacy graph transformer
* Const fold test for modified operators
* Add Overview page
* Revert "Add Overview page"
* fix references
* update links
* update the wording to be more clear
* add the error message about Visual studio back
* update links to static html links of 2022.2
* port changes to master
* update description
* Fixed newAPI for case if core was removed
* Fixed code style
* Fixed typo
* Use new API by default
* Create core with template plugin
* Added doxygen comment
* ROIAlign v9 support
* Code changes after review1
* Code changes after review2
* fix of single layer test for Windows
* Since PR #12043 we don't need strong include order of primitive_base.hpp and
impls/implementation map.hpp anymore
* Code changes after review3
* Code changes after review4
* Commits from PR11373:
Fixed handling of transpose after convolution
[GNA] Fixed calculation of dimensions for ConvolutionFilter and PWL primitives
[GNA] Fixed coverity error and failed tests
* Apply comments
* Update src/plugins/intel_gna/gna_graph_compiler.cpp
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* Update src/plugins/intel_gna/gna_graph_compiler.cpp
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* Rollback names
* Separate test data
* Move coverity issue to separate request
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* Add some tip description about enable OpenCL for benchmark_app.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Export doesn't work, we need to add -Dopencl_root_hints=[PATH]/OpenCL-CLHPP/include to cmake command.
Signed-off-by: xipingya <xiping.yan@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* change memory access pattern of fsv layout for permute
* Fix permute_ref to process F first only when (bf...) => (b...f)
* Refactor
Co-authored-by: si-eun-kim <sieun.kim@intel.com>
* fix references
* update links
* update the wording to be more clear
* add the error message about Visual studio back
* update links to static html links of 2022.2
* Fix GPU NonMaxSuppression implementation
* Introduce Nms9 single layer tests
* Adapt internal NMS and GPU implementation for NMS9 implementation
* Adapt CPU implementation in GPU for NMS9
* Add blocked layouts support to NMS
* Add unit tests for blocked formats for NMS
* Fix boxes groups size for the small shapes
* Use ocl implementation for blocked layout input
* Fix templates typedefs to pass win build
* Fix second output to set data in correct format
* Add methods for access to varables information in Program class
* add ReadValue and Assign primitives
* ReadValue and Assign implementations
* Implementation of memory states allocation
* Add output existance check in primitive_inst to avoid crashes if output is set during execution
* Add memory states management functionality in network component
* Integration of memory states feature in inference request component
* Exclude constant path for read_value and assign nodes in cldnn transformations
* Improve memory states test to run on a single inference request
* unit tests for ReadValue and Assign
* single-layer test for ReadValue and Assign
* Add QueryState API implementation
* Add memory state test which covers dynamic batch case
Co-authored-by: Oleksii Khovan <okhovan@lohika.com>
* NonZero operation: uncomment tests since they can be passed now
# Conflicts:
# src/tests/functional/plugin/cpu/shared_tests_instances/skip_tests_config.cpp
* Unbreak tests once more by changing base class from LayerTestsCommon to SubgraphBaseTest
* Unbreak compilation / style
* Add test case for cache
Co-authored-by: Chenhu Wang <chenhu.wang@intel.com>
* Increase zeroes count for NonZero tests
* Correct the change
* Remove my previous changes and add dynamic shapes / repeatable shapes into the correct file
Co-authored-by: Chenhu Wang <chenhu.wang@intel.com>
@@ -21,7 +21,7 @@ We welcome community contributions to OpenVINO™. Please read the following gui
- Choose a base branch for your work. More details about branches and policies are here: [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches)
- Clone your fork to your computer.
- Create a new branch (with a meaningful name) from the base branch you chose.
- Modify / add the code following our [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines).
- Modify / add the code following our [Coding Style Guide](./docs/dev/coding_style.md).
- If you want to add a new sample, please look at this [Guide for contributing to C++/C/Python IE samples](https://github.com/openvinotoolkit/openvino/wiki/SampleContribute)
- If you want to contribute to the documentation and want to add a new guide, follow that instruction [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation)
- Run testsuite locally:
@@ -35,7 +35,7 @@ Following these guidelines will increase the likelihood of your pull request bei
- One PR – one issue.
- Build perfectly on your local system.
- Choose the right base branch [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches).
- Follow the [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines) for your code.
- Follow the [Coding Style Guide](./docs/dev/coding_style.md) for your code.
- Update documentation using [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation) if needed.
- Cover your changes with test.
- Add license at the top of new files [C++ example](https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/classification_sample_async/main.cpp#L1-L2), [Python example](https://github.com/openvinotoolkit/openvino/blob/master/samples/python/hello_classification/hello_classification.py#L3-L4).
@@ -52,4 +52,4 @@ Your pull request will be automatically tested by OpenVINO™'s precommit (testi
## Merging PR
As soon as the reviewer is fine with the pull request and precommit shows "green" status, the "Approved" review status is put, which signals OpenVINO™ maintainers that they can merge your pull request.
When the reviewer accepts the pull request and the pre-commit shows a "green" status, the review status is set to "Approved", which signals to the OpenVINO™ maintainers that they can merge your pull request.
@@ -34,24 +34,24 @@ OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference.
- Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud
This open-source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from the [Open Model Zoo], along with 100+ open
This open-source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inference on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from [Open Model Zoo], along with 100+ open
source and public models in popular formats such as TensorFlow, ONNX, PaddlePaddle, MXNet, Caffe, Kaldi.
### Components
* [OpenVINO™ Runtime] - is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions on the platform of your choice.
* [core](https://github.com/openvinotoolkit/openvino/tree/master/src/core) - provides the base API for model representation and modification.
* [inference](https://github.com/openvinotoolkit/openvino/tree/master/src/inference) - provides an API to infer models on device.
* [transformations](https://github.com/openvinotoolkit/openvino/tree/master/src/common/transformations) - contains the set of common transformations which are used in OpenVINO plugins.
* [low precision transformations](https://github.com/openvinotoolkit/openvino/tree/master/src/common/low_precision_transformations) - contains the set of transformations which are used in low precision models
* [bindings](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings) - contains all awailable OpenVINO bindings which are maintained by OpenVINO team.
* [c](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings/c) - provides C API for OpenVINO™ Runtime
* [python](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings/python) - Python API for OpenVINO™ Runtime
* [Plugins](https://github.com/openvinotoolkit/openvino/tree/master/src/plugins) - contains OpenVINO plugins which are maintained in open-source by OpenVINO team. For more information please taje a look to the [list of supported devices](#supported-hardware-matrix).
* [Frontends](https://github.com/openvinotoolkit/openvino/tree/master/src/frontends) - contains available OpenVINO frontends which allow to read model from native framework format.
* [core](./src/core) - provides the base API for model representation and modification.
* [inference](./src/inference) - provides an API to infer models on the device.
* [transformations](./src/common/transformations) - contains the set of common transformations which are used in OpenVINO plugins.
* [low precision transformations](./src/common/low_precision_transformations) - contains the set of transformations that are used in low precision models
* [bindings](./src/bindings) - contains all available OpenVINO bindings which are maintained by the OpenVINO team.
* [c](./src/bindings/c) - C API for OpenVINO™ Runtime
* [python](./src/bindings/python) - Python API for OpenVINO™ Runtime
* [Plugins](./src/plugins) - contains OpenVINO plugins which are maintained in open-source by the OpenVINO team. For more information, take a look at the [list of supported devices](#supported-hardware-matrix).
* [Frontends](./src/frontends) - contains available OpenVINO frontends that allow reading models from the native framework format.
* [Model Optimizer] - is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
* [Post-Training Optimization Tool] - is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning, for example, post-training 8-bit quantization.
* [Samples] - applications on C, C++ and Python languages which shows basic use cases of OpenVINO usages.
* [Samples] - applications in C, C++ and Python languages that show basic OpenVINO use cases.
## Supported Hardware matrix
@@ -70,7 +70,7 @@ The OpenVINO™ Runtime can infer models on different hardware devices. This sec
<td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td>
<td>Multi plugin enables simultaneous inference of the same model on several devices in parallel</td>
</tr>
</tbody>
@@ -140,11 +140,11 @@ By contributing to the project, you agree to the license and copyright terms the
### User documentation
The latest documentation for OpenVINO™ Toolkit is availabe [here](https://docs.openvino.ai/). This documentation contains detailed information about all OpenVINO components and provides all important information which could be needed if you create an application which is based on binary OpenVINO distribution or own OpenVINO version without source code modification.
The latest documentation for OpenVINO™ Toolkit is available [here](https://docs.openvino.ai/). This documentation contains detailed information about all OpenVINO components and provides all the important information you may need to create an application based on binary OpenVINO distribution or own OpenVINO version without source code modification.
### Developer documentation
[Developer documentation](#todo-add) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
[Developer documentation](./docs/dev/index.md) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
## Tutorials
@@ -161,15 +161,15 @@ The list of OpenVINO tutorials:
## System requirements
The full information about system requirements depends on platform and available in section `System requirement` on dedicated pages:
Please take a look to [OpenVINO Wiki](https://github.com/openvinotoolkit/openvino/wiki#how-to-build) to get more information about OpenVINO build process.
See the [OpenVINO Wiki](https://github.com/openvinotoolkit/openvino/wiki#how-to-build) to get more information about the OpenVINO build process.
## How to contribute
@@ -177,13 +177,13 @@ See [CONTRIBUTING](./CONTRIBUTING.md) for details. Thank you!
## Get a support
Please report questions, issues and suggestions using:
* [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) - a suite of advanced algorithms for model inference optimization including quantization, filter pruning, binarization and sparsity
* [OpenVINO™ Training Extensions (OTE)](https://github.com/openvinotoolkit/training_extensions) - convenient environment to train Deep Learning models and convert them using OpenVINO for optimized inference.
* [OpenVINO™ Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) - a scalable, high-performance solution for serving deep learning models optimized for Intel architectures
* [DL Workbench](https://docs.openvino.ai/nightly/workbench_docs_Workbench_DG_Introduction.html) - An alternative, web-based version of OpenVINO designed to make production of pretrained deep learning models significantly easier.
* [Computer Vision Annotation Tool (CVAT)](https://github.com/openvinotoolkit/cvat) - an online, interactive video and image annotation tool for computer vision purposes.
* [DL Workbench](https://docs.openvino.ai/nightly/workbench_docs_Workbench_DG_Introduction.html) - an alternative, web-based version of OpenVINO designed to facilitate optimization and compression of pre-trained deep learning models.
* [Computer Vision Annotation Tool (CVAT)](https://github.com/opencv/cvat) - an online, interactive video and image annotation tool for computer vision purposes.
* [Dataset Management Framework (Datumaro)](https://github.com/openvinotoolkit/datumaro) - a framework and CLI tool to build, transform, and analyze datasets.
---
\* Other names and brands may be claimed as the property of others.
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
# FIXME: there are compiler failures with LTO and Cross-Compile toolchains. Disabling for now, but
# this must be addressed in a proper way
ie_dependent_option(ENABLE_LTO"Enable Link Time Optimization"OFF"LINUX;NOT CMAKE_CROSSCOMPILING;CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 4.9"OFF)
ie_dependent_option(ENABLE_LTO"Enable Link Time Optimization"OFF"LINUX OR (APPLE AND AARCH64);NOT CMAKE_CROSSCOMPILING;CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 4.9"OFF)
ie_option(OS_FOLDER"create OS dedicated folder in output"OFF)
@@ -17,8 +17,17 @@ else()
ie_option(USE_BUILD_TYPE_SUBFOLDER"Create dedicated sub-folder per build type for output binaries"ON)
endif()
# FIXME: ARM cross-compiler generates several "false positive" warnings regarding __builtin_memcpy buffer overflow
ie_dependent_option(TREAT_WARNING_AS_ERROR "Treat build warnings as errors"ON"X86 OR X86_64"OFF)
if(CI_BUILD_NUMBER)
set(TREAT_WARNING_AS_ERROR_DEFAULTON)
else()
set(TREAT_WARNING_AS_ERROR_DEFAULTOFF)
endif()
ie_dependent_option(TREAT_WARNING_AS_ERROR"WILL BE REMOVED SOON, NEED TO FIX PRIVATE COMPONENTS"ON"X86_64 OR X86"OFF)
-- OpenVINO Developers <openvino@intel.com> Sun, 30 Oct 2022 04:17:51 +0800
openvino (2022.2.0) unstable; urgency=low
* Broader model and hardware support - Optimize & deploy with ease across an expanded range of deep learning models including NLP, and access AI acceleration across an expanded range of hardware.
* NEW: Support for Intel 13th Gen Core Processor for desktop (code named Raptor Lake).
* NEW: Preview support for Intel’s discrete graphics cards, Intel® Data Center GPU Flex Series and Intel® Arc™ GPU for DL inferencing workloads in intelligent cloud, edge and media analytics workloads. Hundreds of models enabled.
* NEW: Test your model performance with preview support for Intel 4th Generation Xeon® processors (code named Sapphire Rapids).
* Broader support for NLP models and use cases like text to speech and voice recognition. Reduced memory consumption when using Dynamic Input Shapes on CPU. Improved efficiency for NLP applications.
* Frameworks Integrations – More options that provide minimal code changes to align with your existing frameworks
* OpenVINO Execution Provider for ONNX Runtime gives ONNX Runtime developers more choice for performance optimizations by making it easy to add OpenVINO with minimal code changes.
* NEW: Accelerate PyTorch models with ONNX Runtime using OpenVINO™ integration with ONNX Runtime for PyTorch (OpenVINO™ Torch-ORT). Now PyTorch developers can stay within their framework and benefit from OpenVINO performance gains.
* OpenVINO Integration with TensorFlow now supports more deep learning models with improved inferencing performance.
* More portability and performance - See a performance boost straight away with automatic device discovery, load balancing & dynamic inference parallelism across CPU, GPU, and more.
* NEW: Introducing new performance hint (”Cumulative throughput”) in AUTO device, enabling multiple accelerators (e.g. multiple GPUs) to be used at once to maximize inferencing performance.
* NEW: Introducing Intel® FPGA AI Suite support which enables real-time, low-latency, and low-power deep learning inference in this easy-to-use package
- Broader model and hardware support - Optimize & deploy with ease across an expanded range of deep learning models including NLP, and access AI acceleration across an expanded range of hardware.
- NEW: Support for Intel 13th Gen Core Processor for desktop (code named Raptor Lake).
- NEW: Preview support for Intel’s discrete graphics cards, Intel® Data Center GPU Flex Series and Intel® Arc™ GPU for DL inferencing workloads in intelligent cloud, edge and media analytics workloads. Hundreds of models enabled.
- NEW: Test your model performance with preview support for Intel 4th Generation Xeon® processors (code named Sapphire Rapids).
- Broader support for NLP models and use cases like text to speech and voice recognition. Reduced memory consumption when using Dynamic Input Shapes on CPU. Improved efficiency for NLP applications.
- Frameworks Integrations – More options that provide minimal code changes to align with your existing frameworks
- OpenVINO Execution Provider for ONNX Runtime gives ONNX Runtime developers more choice for performance optimizations by making it easy to add OpenVINO with minimal code changes.
- NEW: Accelerate PyTorch models with ONNX Runtime using OpenVINO™ integration with ONNX Runtime for PyTorch (OpenVINO™ Torch-ORT). Now PyTorch developers can stay within their framework and benefit from OpenVINO performance gains.
- OpenVINO Integration with TensorFlow now supports more deep learning models with improved inferencing performance.
- More portability and performance - See a performance boost straight away with automatic device discovery, load balancing & dynamic inference parallelism across CPU, GPU, and more.
- NEW: Introducing new performance hint (”Cumulative throughput”) in AUTO device, enabling multiple accelerators (e.g. multiple GPUs) to be used at once to maximize inferencing performance.
- NEW: Introducing Intel® FPGA AI Suite support which enables real-time, low-latency, and low-power deep learning inference in this easy-to-use package
message(FATAL_ERROR"rpmlint has found some mistakes. You can get more info regarding issues on site https://fedoraproject.org/wiki/Common_Rpmlint_issues")
Once you have a model that meets both OpenVINO™ and your requirements, you can choose among several ways of deploying it with your application:
* [Run inference and develop your app with OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
* [Deploy your application locally](../OV_Runtime_UG/deployment/deployment_intro.md).
* [Deploy your model online with the OpenVINO Model Server](@ref ovms_what_is_openvino_model_server).
* [Deploy your application locally](../OV_Runtime_UG/deployment/deployment_intro.md).
* [Deploy your model with OpenVINO Model Server](@ref ovms_what_is_openvino_model_server).
* [Deploy your application for the TensorFlow framework with OpenVINO Integration](./openvino_ecosystem_ovtf.md).
> **NOTE**: Note that [running inference in OpenVINO Runtime](../OV_Runtime_UG/openvino_intro.md) is the most basic form of deployment. Before moving forward, make sure you know how to create a proper Inference configuration.
Deep Learning Workbench (DL Workbench) is an official OpenVINO™ graphical interface designed to make the production of pretrained deep learning Computer Vision and Natural Language Processing models significantly easier.
Minimize the inference-to-deployment workflow timing for neural models right in your browser: import a model, analyze its performance and accuracy, visualize the outputs, optimize and make the final model deployment-ready in a matter of minutes. DL Workbench takes you through the full OpenVINO™ workflow, providing the opportunity to learn about various toolkit components.
DL Workbench enables you to get a detailed performance assessment, explore inference configurations, and obtain an optimized model ready to be deployed on various Intel® configurations, such as client and server CPU, Intel® Processor Graphics (GPU), Intel® Movidius™ Neural Compute Stick 2 (NCS 2), and Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.
DL Workbench also provides the [JupyterLab environment](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Jupyter_Notebooks.html#doxid-workbench-docs-workbench-d-g-jupyter-notebooks) that helps you quick start with OpenVINO™ API and command-line interface (CLI). Follow the full OpenVINO workflow created for your model and learn about different toolkit components.
DL Workbench helps achieve your goals depending on the stage of your deep learning journey.
If you are a beginner in the deep learning field, the DL Workbench provides you with
learning opportunities:
* Learn what neural networks are, how they work, and how to examine their architectures.
* Learn the basics of neural network analysis and optimization before production.
* Get familiar with the OpenVINO™ ecosystem and its main components without installing it on your system.
If you have enough experience with neural networks, DL Workbench provides you with a
convenient web interface to optimize your model and prepare it for production:
* Measure and interpret model performance.
* Tune the model for enhanced performance.
* Analyze the quality of your model and visualize output.
## General Workflow
The diagram below illustrates the typical DL Workbench workflow. Click to see the full-size image:
Get a quick overview of the workflow in the DL Workbench User Interface:
## OpenVINO™ Toolkit Components
The intuitive web-based interface of the DL Workbench enables you to easily use various
OpenVINO™ toolkit components:
Component | Description
|------------------|------------------|
| [Open Model Zoo](https://docs.openvinotoolkit.org/latest/omz_tools_downloader.html)| Get access to the collection of high-quality pre-trained deep learning [public](https://docs.openvinotoolkit.org/latest/omz_models_group_public.html) and [Intel-trained](https://docs.openvinotoolkit.org/latest/omz_models_group_intel.html) models trained to resolve a variety of different tasks.
| [Model Optimizer](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) |Optimize and transform models trained in supported frameworks to the IR format. <br>Supported frameworks include TensorFlow\*, Caffe\*, Kaldi\*, MXNet\*, and ONNX\* format.
| [Benchmark Tool](https://docs.openvinotoolkit.org/latest/openvino_inference_engine_tools_benchmark_tool_README.html)| Estimate deep learning model inference performance on supported devices.
| [Accuracy Checker](https://docs.openvinotoolkit.org/latest/omz_tools_accuracy_checker.html)| Evaluate the accuracy of a model by collecting one or several metric values.
| [Post-Training Optimization Tool](https://docs.openvinotoolkit.org/latest/pot_README.html)| Optimize pretrained models with lowering the precision of a model from floating-point precision(FP32 or FP16) to integer precision (INT8), without the need to retrain or fine-tune models. |
OpenVINO Runtime offers multiple inference modes to allow optimum hardware utilization under different conditions. The most basic one is a single-device mode, which defines just one device responsible for the entire inference workload. It supports a range of Intel hardware by means of plugins embedded in the Runtime library, each set up to offer the best possible performance. For a complete list of supported devices and instructions on how to use them, refer to the [guide on inference devices](../OV_Runtime_UG/supported_plugins/Device_Plugins.md).
The remaining modes assume certain levels of automation in selecting devices for inference. Using them in the deployed solution may potentially increase its performance and portability. The automated modes are:
Every deep learning workflow begins with obtaining a model. You can choose to prepare a custom one, use a ready-made solution and adjust it to your needs, or even download and run a pre-trained network from an online database, such as OpenVINO's [Open Model Zoo](../model_zoo.md).
This section describes how to obtain and prepare your modelfor work with OpenVINO to get the best inference results:
* [Browse a database of models for use in your projects](../model_zoo.md).
[OpenVINO™ supports several model formats](../MO_DG/prepare_model/convert_model/supported_model_formats.md) and allows to convert them to it's own, OpenVINO IR, providing a tool dedicated to this task.
[Model Optimizer](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) reads the original model and creates the OpenVINO IR model (.xml and .bin files) so that inference can ultimately be performed without delays due to format conversion. Optionally, Model Optimizer can adjust the model to be more suitable for inference, for example, by [alternating input shapes](../MO_DG/prepare_model/convert_model/Converting_Model.md), [embedding preprocessing](../MO_DG/prepare_model/Additional_Optimizations.md) and [cutting training parts off](../MO_DG/prepare_model/convert_model/Cutting_Model.md).
The approach to fully convert a model is considered the default choice, as it allows the full extent of OpenVINO features. The OpenVINO IR model format is used by other conversion and preparation tools, such as the Post-Training Optimization Tool, for further optimization of the converted model.
Conversion is not required for ONNX and PaddlePaddle models, as OpenVINO provides C++ and Python APIs for importing them to OpenVINO Runtime directly. It provides a convenient way to quickly switch from framework-based code to OpenVINO-based code in your inference application.
This section describes the how to obtain and prepare your model for work with OpenVINO to get the best inference results:
* [See the supported formats and how to use them in your project](../MO_DG/prepare_model/convert_model/supported_model_formats.md)
* [Convert different model formats to the OpenVINO IR format](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
* [Automate model-related tasks with Model Downloader and additional OMZ Tools](https://docs.openvino.ai/latest/omz_tools_downloader.html).
To begin with, you may want to [browse a database of models for use in your projects](../model_zoo.md).
A suite of advanced algorithms for Neural Network inference optimization with minimal accuracy drop. NNCF applies quantization, filter pruning, binarization and sparsity algorithms to PyTorch and TensorFlow models during training.
@@ -8,7 +8,6 @@ There are two options for using the custom operation configuration file:
* Include a section with your kernels into the automatically-loaded `<lib_path>/cldnn_global_custom_kernels/cldnn_global_custom_kernels.xml` file.
* Call the `ov::Core::set_property()` method from your application with the `"CONFIG_FILE"` key and the configuration file name as a value before loading the network that uses custom operations to the plugin:
@@ -68,7 +68,7 @@ Remaining part of this guide uses Frontend Extension API applicable for new fron
A custom operation class and a new mapping frontend extension class object should be registered to be usable in OpenVINO runtime.
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension/new), which demonstrates extension development details based on minimalistic `Identity` operation that is a placeholder for your real custom operation. You can review the complete code, which is fully compliable, to see how it works.
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new), which demonstrates extension development details based on minimalistic `Identity` operation that is a placeholder for your real custom operation. You can review the complete code, which is fully compliable, to see how it works.
To load the extensions to the `ov::Core` object, use the `ov::Core::add_extension` method, this method allows to load library with extensions or extensions from the code.
@@ -138,7 +138,7 @@ This CMake script finds the OpenVINO™ using the `find_package` CMake command.
To build the extension library, run the commands below:
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box.
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box. This capability requires writing code in C++, so if you are using Python to develop your application you need to build a separate shared library implemented in C++ first and load it in Python using `add_extension` API. Please refer to [Create library with extensions](Intro.md#create-library-with-extensions) for more details on library creation and usage. The remining part of this document describes how to implement an operation class.
@@ -4,7 +4,7 @@ The goal of this chapter is to explain how to use Frontend extension classes to
This API is applicable for new frontends only, which exist for ONNX and PaddlePaddle. If a different model format is used, follow legacy [Model Optimizer Extensions](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) guide.
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension/new), which demonstrates extension development details based on minimalistic `Identity` operation that is a placeholder for your real custom operation. You can review the complete code, which is fully compliable, to see how it works.
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/src/core/template_extension/new), which demonstrates extension development details based on minimalistic `Identity` operation that is a placeholder for your real custom operation. You can review the complete code, which is fully compliable, to see how it works.
OpenVINO™ Runtime has three main transformation types:
@@ -91,7 +91,7 @@ Transformation library has two internal macros to support conditional compilatio
When developing a transformation, you need to follow these transformation rules:
###1. Friendly Names
###1. Friendly Names
Each `ov::Node` has an unique name and a friendly name. In transformations we care only about friendly name because it represents the name from the model.
To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below.
@@ -100,7 +100,7 @@ To avoid losing friendly name when replacing node with other node or subgraph, s
In more advanced cases, when replaced operation has several outputs and we add additional consumers to its outputs, we make a decision how to set friendly name by arrangement.
###2. Runtime Info
###2. Runtime Info
Runtime info is a map `std::map<std::string, ov::Any>` located inside `ov::Node` class. It represents additional attributes in `ov::Node`.
These attributes can be set by users or by plugins and when executing transformation that changes `ov::Model` we need to preserve these attributes as they will not be automatically propagated.
@@ -111,9 +111,9 @@ Currently, there is no mechanism that automatically detects transformation types
When transformation has multiple fusions or decompositions, `ov::copy_runtime_info` must be called multiple times for each case.
**Note**: copy_runtime_info removes rt_info from destination nodes. If you want to keep it, you need to specify them in source nodes like this: copy_runtime_info({a, b, c}, {a, b})
> **NOTE**: `copy_runtime_info` removes `rt_info` from destination nodes. If you want to keep it, you need to specify them in source nodes like this: `copy_runtime_info({a, b, c}, {a, b})`
###3. Constant Folding
###3. Constant Folding
If your transformation inserts constant sub-graphs that need to be folded, do not forget to use `ov::pass::ConstantFolding()` after your transformation or call constant folding directly for operation.
The example below shows how constant subgraph can be constructed.
@@ -140,8 +140,8 @@ In transformation development process:
## Using pass manager <a name="using_pass_manager"></a>
`ov::pass::Manager` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations.
It can register and apply any [transformation pass](#transformations_types) on model.
In addition, `ov::pass::Manager` has extended debug capabilities (find more information in the [how to debug transformations](#how_to_debug_transformations) section).
It can register and apply any [transformation pass](#transformations-types) on model.
In addition, `ov::pass::Manager` has extended debug capabilities (find more information in the [how to debug transformations](#how-to-debug-transformations) section).
The example below shows basic usage of `ov::pass::Manager`
@@ -151,7 +151,7 @@ Another example shows how multiple matcher passes can be united into single Grap
## How to debug transformations <a name="how_to_debug_transformations"></a>
## How to debug transformations <a name="how-to-debug-transformations"></a>
If you are using `ngraph::pass::Manager` to run sequence of transformations, you can get additional debug capabilities by using the following environment variables:
@@ -160,7 +160,7 @@ OV_PROFILE_PASS_ENABLE=1 - enables performance measurement for each transformati
OV_ENABLE_VISUALIZE_TRACING=1 - enables visualization after each transformation. By default, it saves dot and svg files.
```
> **Note**: Make sure that you have dot installed on your machine; otherwise, it will silently save only dot file without svg file.
> **NOTE**: Make sure that you have dot installed on your machine; otherwise, it will silently save only dot file without svg file.
# Build Plugin Using CMake* {#openvino_docs_ie_plugin_dg_plugin_build}
# Build Plugin Using CMake {#openvino_docs_ie_plugin_dg_plugin_build}
Inference Engine build infrastructure provides the Inference Engine Developer Package for plugin development.
OpenVINO build infrastructure provides the OpenVINO Developer Package for plugin development.
Inference Engine Developer Package
OpenVINO Developer Package
------------------------
To automatically generate the Inference Engine Developer Package, run the `cmake` tool during a DLDT build:
To automatically generate the OpenVINO Developer Package, run the `cmake` tool during a OpenVINO build:
```bash
$ mkdir dldt-release-build
$ cddldt-release-build
$ cmake -DCMAKE_BUILD_TYPE=Release ../dldt
$ mkdir openvino-release-build
$ cdopenvino-release-build
$ cmake -DCMAKE_BUILD_TYPE=Release ../openvino
```
Once the commands above are executed, the Inference Engine Developer Package is generated in the `dldt-release-build` folder. It consists of several files:
-`InferenceEngineDeveloperPackageConfig.cmake` - the main CMake script which imports targets and provides compilation flags and CMake options.
-`InferenceEngineDeveloperPackageConfig-version.cmake` - a file with a package version.
-`targets_developer.cmake` - an automatically generated file which contains all targets exported from the OpenVINO build tree. This file is included by `InferenceEngineDeveloperPackageConfig.cmake` to import the following targets:
Once the commands above are executed, the OpenVINO Developer Package is generated in the `openvino-release-build` folder. It consists of several files:
-`OpenVINODeveloperPackageConfig.cmake` - the main CMake script which imports targets and provides compilation flags and CMake options.
-`OpenVINODeveloperPackageConfig-version.cmake` - a file with a package version.
-`targets_developer.cmake` - an automatically generated file which contains all targets exported from the OpenVINO build tree. This file is included by `OpenVINODeveloperPackageConfig.cmake` to import the following targets:
A common plugin consists of the following components:
@@ -56,7 +60,7 @@ A common plugin consists of the following components:
To build a plugin and its tests, run the following CMake scripts:
- Root `CMakeLists.txt`, which finds the Inference Engine Developer Package using the `find_package` CMake command and adds the `src` and `tests` subdirectories with plugin sources and their tests respectively:
- Root `CMakeLists.txt`, which finds the OpenVINO Developer Package using the `find_package` CMake command and adds the `src` and `tests` subdirectories with plugin sources and their tests respectively:
```cmake
cmake_minimum_required(VERSION3.13)
@@ -82,21 +86,15 @@ if(ENABLE_TESTS)
endif()
endif()
```
> **NOTE**: The default values of the `ENABLE_TESTS`, `ENABLE_FUNCTIONAL_TESTS` options are shared via the Inference Engine Developer Package and they are the same as for the main DLDT build tree. You can override them during plugin build using the command below:
> **NOTE**: The default values of the `ENABLE_TESTS`, `ENABLE_FUNCTIONAL_TESTS` options are shared via the OpenVINO Developer Package and they are the same as for the main OpenVINO build tree. You can override them during plugin build using the command below:
> **NOTE**: The `IE::funcSharedTests` static library with common functional Inference Engine Plugin tests is imported via the Inference Engine Developer Package.
> **NOTE**: The `openvino::funcSharedTests` static library with common functional OpenVINO Plugin tests is imported via the OpenVINO Developer Package.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
