* Model caching support - Core part
Introducing model caching support
Use core.SetConfig({{CONFIG_KEY(CACHE_DIR), <dir>}}); to enable caching of models
OpenVINO will try to create caching folder if it doesn't exist, but it is recommended for client to create caching folder with necessary permissions before enabling cache in config
For caching, plugins shall support import/export functionality
Plugin requirements:
- Add METRIC_KEY(IMPORT_EXPORT_SUPPORT) in SUPPORTED_METRICS to support caching
If plugin has different device architectures with different caches, i.e.
For "GNA.0" - one cache, for "GNA.10" - another cache
In this case plugin shall support DEVICE_ARCHITECTURE metric and return different strings for different DEVICE_ID's
Added functional tests
* Fix CentOS build issues
* Few updates according to code review
* Revert unnecessary changes for Import/Export core implementation
These changes affect old behavior and may be undesired
For caching support these is no need to change anything in this area
If needed, such removal of 'Magic' usage can be done under separate task in future
* More tests:
1) Verify that Imported data from stream is the same as was exported
2) Verify that cache is not loaded when config in LoadNetwork is changed
3) Verify that if CNN Network is changed between ReadNetwork and LoadNetwork - cache is not loaded
* Update of NetworkCompilationContext
Put back functionality of calculating hash based on runtime information, weights
Implemented OstreamHashWrapper to avoid serialization to buffer
* Correction of CACHE_DIR key description
* Unit tests for compilation_context
Changes:
1) Improved handling of OstreamHashAdapter
2) Improved runtime info serialization (not just PrimitivesPriority and affinity)
3) Removed redundant weights hash calculation
* Fix GCC 4.8 build issues
* Compilation context updates
1) Use hash of sum of serialized data to get hash of network. It is more efficient comparing to weights sum calculation
2) CalculateFileInfo - convert path to absolute ("./test.blob" and "test.blob" shall give same hash)
* Hash - added more rt_info attributes + tests
- PrimitivesPriority
- FusedNames
- Dequantization
* Moved "get_absolute_path" macro to file_utils.h
* Make 'absoluteFilePath' a library API, not macro
* One more unit test for fileName hashing
* Fix compilation error after merge with latest master
* Allow tests to be executed in parallel (stress mode)
* More minor updates for stress testing
Now it allows to execute tests with '--repeat=100' option where one test is executed in multiple processes simultaneously
Example:
./gtest-parallel <openvino_dir>/bin/intel64/Debug/ieFuncTests --gtest_filter=CachingTest* --repeat=10
* Use absolute model file path for calculating blob name
* Added 'createDirectoryRecursive' API to plugin_api/file_utils
* Fix issue with GNA Import/Export.
Application may create ostream and write some app-specific header to beginning of stream.
GNA plug-in shall not 'seekg' to beginning of stream where app-specific content is located.
Instead it shall seek to the original position of stream
Functional test is also updated to cover this case
* Updated according to review comments
- Check return code right after 'istream::tellg'
- Added 'applicationHeader' as parameter to base ImportExport tests
* Error check for startPos + unit test
* Suppress deprecated warnings like in other GNA tests
* Feature/vpu doc fixes 2021 3 (#4635)
* Documentation fixes and updates for VPU
* minor correction
* minor correction
* Fixed links
* updated supported layers list for vpu
* Auto benchmarks for evaluate(...) and extra unit tests.
* Test suite for manual i/o examination.
* Template on input type. Constant folding tests the 'count_ops_of_type' issue.
* Multidim manual tests
* NgOp&Ref: Unit tests Ok. NB evaluates_map
* Stable. Negative indices as if larger than depth. Benchmarks removed.
* Added Serialization SLT
* I/O shapes consistency checks
* Consistency checks in evaluate(). Removed the ref from evaluates_map.
* removed excessive #include from evaluates_map
* Code style
* [GNA] Support Pooling 2D
Refactor Pooling
Remove unused sum pooling
Skip ReorderPoolActivation pass when 2D pooling detected
Enable fusing of Pool2D with other components into GNA Operation
* Support GNA_SW_FP32 mode for Pooling 2D with tests
* Apply review
typo
* Apply review
Restore indentation
* Only executor threads now have top task priority. Other runtime threads have lower priority. Other non-runtime tasks (which do not participate in inference) have much lower priority.
* Initial Subgraphs Dumper and IR runner test inplementation
* Implement matchers configuration
* Refactor MatcherConfig
* Add OP cloners factory
* Remove IR runner test (to be added in separate PR)
* Use different nodes comparison method
* Remove extra include of <any> which fails build with old glibc
* Add README.md
Co-authored-by: Mikhail Treskin <mikhail.treskin@intel.com>
SubGraph might have inputs and outputs in different order and still be
the same Function.
Co-authored-by: Patryk Elszkowski <patryk.elszkowki@intel.com>
* added draft version of doc for networks with state (#3372)
* added draft version of doc for networks with state
* update code snippet
* update example
* review fixes
* fixed png
* review fixes
* image fix
* minor fix
* LowLatency documentation
* apply review comments
* Apply suggestions from code review
Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
* review fixes
Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com>
Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
(cherry picked from commit 99c746f2c3)
* take latest changes from 2021.2
