[First Inference] Read time improvements via using 'mmap/munmap' (#10907)

* Performance improvement for constant creation

The issue is that 'are_all_data_elements_bitwise_identical()' is called every time in Constant constructor, and it potentially checks all buffer which is O(N) complexity.
While it is needed only if client uses 'get_all_data_elements_bitwise_identical'

Solution:
- Defer calculation until first call of 'get_all_data_elements_bitwise_identical'
- Store calculated value in mutable class member to reuse it on next calls of 'get_all_data_elements_bitwise_identical'

Test verifies both cases:
a) that constant creation with shared memory data (now O(1)) is significantly faster than creation+bitwiseCheck O(N)
b) Than once calculated, value is taken from cache, which is significantly faster than re-calculation

* fix clang-format

* Stash - Linux implementation

* Windows mmap implementation + unicode

* Clang for windows

* removed debug print

* Add handling of empty bin file

* fix windows includes

* Fix python test

* Unit tests
Fix for Constant with size > 4GB

* Fix review comments
This commit is contained in:
Mikhail Nosov
2022-03-17 17:16:06 +03:00
committed by GitHub
parent 3a8fd7135e
commit 4b3dd808df
12 changed files with 429 additions and 38 deletions

View File

@@ -106,6 +106,17 @@ macro(ov_add_frontend)
set(FRONTEND_NAMES "${FRONTEND_NAMES}" CACHE INTERNAL "" FORCE)
file(GLOB_RECURSE LIBRARY_SRC ${CMAKE_CURRENT_SOURCE_DIR}/src/*.cpp)
if (WIN32)
# Remove linux specific files
file(GLOB_RECURSE LIN_FILES ${CMAKE_CURRENT_SOURCE_DIR}/src/os/lin/*.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/os/lin/*.hpp)
list(REMOVE_ITEM LIBRARY_SRC "${LIN_FILES}")
else()
# Remove windows specific files
file(GLOB_RECURSE WIN_FILES ${CMAKE_CURRENT_SOURCE_DIR}/src/os/win/*.cpp
${CMAKE_CURRENT_SOURCE_DIR}/src/os/win/*.hpp)
list(REMOVE_ITEM LIBRARY_SRC "${WIN_FILES}")
endif()
file(GLOB_RECURSE LIBRARY_HEADERS ${CMAKE_CURRENT_SOURCE_DIR}/src/*.hpp)
file(GLOB_RECURSE LIBRARY_PUBLIC_HEADERS ${CMAKE_CURRENT_SOURCE_DIR}/include/*.hpp)