For some (e.g. quantized) models, W, R, B inputs to sequence operations
are not direct constants, but a subgraph that can be constfolded later.
In that case, we don't need to have them as inputs to TensorIterator,
they can be pulled into its body instead.
Ticket: CVS-117544
* WIP Postpone fp16 in CompressFloatConstantsImpl
* Apply suggestions from code review
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* WIP: Compression to FP16 in Serialize
* Prepared for efficient fp32 to fp16 conversion
* Update src/core/reference/src/runtime/reference/convert.cpp
* Called real slow reference implementations in the place where the optimized versions are supposed to be implemented
* Code style
* Fixed 0 values in the fast f64 to f16 compression
* Optimized convert_from_f32_to_f16_with_clamp
* Added optimized f32->f16 instance of change_constant_precision
* compression transformation Python test
* use tmp dir, minor corrections
* Update src/bindings/python/tests/test_transformations/test_compression.py
* Update src/bindings/python/tests/test_transformations/test_compression.py
* style fix
* define rt_info for postponed_fp16_compression
* remove redundant class
* fix temp dir for Win in test_compression.py
* update definitions in convert.hpp
* Update implementation in convert.cpp
* Update serialize.cpp
* Update compress_float_constants.cpp
* added macros for ARM/non_x86 in convert.cpp
* fix macros in convert.cpp
* change fixme placement in serialize.cpp
* style_fix
* Update src/core/reference/src/runtime/reference/convert.cpp
* style_fix
* Optimized count_out_of_f16_range
* Code style
* Revert unused
* Update src/core/src/pass/serialize.cpp
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Update src/core/reference/src/runtime/reference/convert.cpp
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* use optimized convert_from_f32_to_f16_with_clamp for non postponed
* minor corrections
* Update src/common/transformations/src/transformations/common_optimizations/compress_float_constants.cpp
* Update compress_float_constants.cpp
* Switched mo and ovc to save_model instead of serialize to leverage performance improvements in fp32->fp16
* Applied minor code imporvements to address review feedback
* Minor changes in code
* Update tools/ovc/openvino/tools/ovc/main.py
* Apply suggestions from code review
* Fixed failed test in case when both usual xml compression and fp16 compression are applied simultaneously (disabled for now)
* Added description for CompressFloatConstantImpl postponed parameter
* Description of postponed parameter for CompressFloatConstants
* Reverted switching to save_model in mo as the compression can be applied not only via CLI and old code should be kept for Python path (not applicable for ovc)
* Removed remining committed test artefacts and reverted remaining changes in mo
---------
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: dmitrygo <dmitry.gorokhov@intel.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Co-authored-by: Pavel Esir <pavel.esir@intel.com>
Co-authored-by: Pavel Esir <pavel.esir@gmail.com>
+ Fixed black output image by Nan output.
+ Resolved fp16 overflow of gemm primitive before softmax
+ Added fused post ops of clamp activation not to get inf which caused Nan output
+ Added new pass modify_fused_ops
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* Added support of tuple in input, removed type syntax from OVC tool.
* Removed type syntax tests.
* Apply suggestions from code review
* Method annotation corrected.
* Type annotation corrected.
---------
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* TorchFX caching bugfix and improvements
* Fixed inconsistent env variable for Backend device
* Identify PyTorch FrontEnd Decoder type
* Added import statement in init files
* Registered ts_openvino as a separate backend
* Added caching fix and removed extraneous code
* Changed the name of ts backend
* Fixed issue with local temporary object
* Removed import statement from init files
* Changed the documentation
* Added get_supported_ops method for decoders
---------
Co-authored-by: Cavus Mustafa <mustafa.cavus@intel.com>
Co-authored-by: ynimmaga <yamini.nimmagadda@intel.com>
* Added check that nncf was imported.
* Added check that nncf was imported in MO.
* Added check that nncf was imported in MO.
* Apply suggestions from code review
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* Removed not needed import.
* Pylint fix.
---------
Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com>
* add set property case
* add set property case
* free key
* fix code style
* add test of get_property() for ov_property_key_auto_batch_timeout
* get_property first
* add device name
* device_name.c_str()
* reset name
* test set_property
* remove comment
* pass local
* get property first
* fix bug
* use device from GetParam
* fix char to string
* use EXPECT_STREQ(target.c_str(), result);
* fix comment
---------
Co-authored-by: Chen Peter <peter.chen@intel.com>
* Moved mmap to openvino/util as a shared functionality
* Enabled MMAP for SavedModel and MetaGraph
* Fixed CMake
* Fixed a lost line
* Simplified code for compilers
* Aligned with an actual master
* Enabled mmap by default and added test
* Suppressed warning, added test for MetaGraph, additional messages
---------
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* Removed 'example_output' from ovc and ovc.convert_model, used output for this purpose
* Update tools/ovc/openvino/tools/ovc/convert.py
* Update tools/ovc/openvino/tools/ovc/convert_impl.py
* Reverted mo parts not affected by remove of example_output
* fix PDPD convert_model tests
---------
Co-authored-by: Xiuchuan Zhai <xiuchuan.zhai@intel.com>
* [PT FE] Fix behaivior of convert_partially method to align with new description
* Simplify
* Do not add a message to exception of get_converted_model
* Update src/frontends/pytorch/src/frontend.cpp
* Supress normalize exception only when model is valid
* Create TranslateSession in own scope
