IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
12,259 Commits 38 Branches 54 Tags
3e552ad2b596f4e2ae076490dc9be8d64386341a
Commit Graph

494 Commits

Author SHA1 Message Date
Xuejun Zhai
daba3713c0 [Wrapper] Avoid creating new threads when converting legacy inference request to API 2.0 (#19376) 
* Fix error in CVS-115961, caused by wrapper covert 1.0 req to 2.0 req create 2 more threads

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

* Eable the test of compareAutoBatchingToSingleBatch with batch size 4 & num req 64, after fix issue 115961

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>

---------

Signed-off-by: Zhai, Xuejun <xuejun.zhai@intel.com>
 2023-08-24 11:46:02 +04:00
Maksim Kutakov
4996d1f034 TensorMemoryBlob: sync data handle on alloc call (#19149) 2023-08-14 13:22:11 +04:00
Fang Xu
e33de35063 [CPU] Add max_threads_per_core setting into StreamExecutor (#18857) 2023-08-11 14:12:38 +00:00
Wanglei Shen
a957764362 block SOC Ecore to be used in multi-threading on Windows (#18910) 2023-08-11 15:23:29 +04:00
Yuan Hu
85609d4881 [Core] fix Memory Leak caused by create/inference request consequently in separate thread (#18868) 
* try to fix memory issue

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>

* save code

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* fix life cycle issue

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* update comment and save stream of master thread in tbb ThreadLocal

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* update

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* not save the main stream in tbb

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* remote test code and update comment

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* fix mistaken modify

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* fix format issue

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* add test

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>

* improve test

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>

* fix the test

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* remote unused code

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* update the comment of the code

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* fix format issue

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

* revert test case

Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>

---------

Signed-off-by: HU Yuan2 <yuan2.hu@intel.com>
Signed-off-by: Hu Yuan2 <yuan2.hu@intel.com>
 2023-08-11 11:53:07 +04:00
Sun Xiaoxia
0ed1749ad9 [CPU] Removed socket id=-1 workaround on macOS (#18779) 2023-08-11 10:38:21 +04:00
Wanglei Shen
c62ad1add8 Support Win7 in cpu information parser (#19109) 2023-08-11 09:54:13 +04:00
Stefania Hergane
24f8c4105e [EISW-89824] [master] Rename VPUX to NPU (#19004) 
* Change `VPUX`/`VPU` occurrences to `NPU`

* Switch `HARDWARE_AWARE_IGNORED_PATTERNS` VPU to NPU

* Rename `MYRIAD plugin`

* Rename vpu_patterns to npu_patterns in tools/pot

* Rename vpu.json to npu.json in tools/pot

* Rename restrict_for_vpu to restrict_for_npu in tools/pot

* Change keembayOptimalBatchNum to npuOptimalBatchNum

---------

Co-authored-by: Dan <mircea-aurelian.dan@intel.com>
 2023-08-10 00:20:07 +04:00
hyunback kim
58bd8e7f0b [GPU] Add disable winograd convolution in execute_config. (#18964) 
* [GPU] Add disable winograd convolution in execute_config.

Signed-off-by: hyunback <hyunback.kim@intel.com>
 2023-08-09 13:32:36 +09:00
Ilya Churaev
4d84f4d182 Remove legacy API from from template plugin (#19036) 
* Remove legacy API from from template plugin

* Fixed template plugin tests

* Fixed warning for Windows

* Fixed Windows warning
 2023-08-09 07:27:28 +04:00
Ilya Churaev
f86a0ad459 Removed legacy API from core_impl (#19009) 
* Removed legacy API from core_impl

* Revert old extension in old API

* Fixed unit tests

* Wrap new extensions in old API

* Wrap extensions in all legacy API

* Fixed legacy exceptions

* Fixed ONNX tests

* Try to fix LTO
 2023-08-09 07:26:25 +04:00
Ilya Churaev
5ae9d80113 Remove core properties from set_property() call (#19008) 
* Remove core properties from compiled_model call

* Add ENABLE_MMAP NO to python test

* Fixed set_property tests

---------

Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
 2023-08-08 15:55:41 +04:00
Ilya Churaev
28ce25421a Removed some deprecated API from inference component (#18981) 2023-08-04 12:34:33 +04:00
Wanglei Shen
0cad2f1324 [CPU] Move streams calculation before transformation pipeline (#18911) 2023-08-02 11:38:34 +00:00
Ilya Churaev
c8bdefec30 Remove old API from Core implementation (#18892) 2023-08-01 09:00:55 +00:00
Ilya Churaev
3760e084fa Apply global mutex only for old glibc (#18832) 
* Apply global mutex only for old gcc

* Added comment
 2023-07-31 07:23:24 +00:00
Vladimir Paramuzov
3e0b4b337b [GPU] LUID property added (#18815) 2023-07-28 19:16:25 +04:00
Marcin Kacprzak
e1ec17013d [GNA] Fix for GeminiLake detection (#18653) 
* [GNA] Fix for GeminiLake detection

* Added HWGeneration::GNA_1_0_E enumerator
Added DeviceVersion::GNAEmbedded1_0 enumerator, changed the meaning of DeviceVersion::GNA1_0.
Updated ConvLowPrecision test with all supported targets

* [GNA] Extended a few tests with GNA1.0
 2023-07-28 08:18:22 +02:00
Luwei Zhou
5f5df36b60 [CPU] Enable oneDNN avx2 brgemm impls for Matmul/FullyConnected operations (#18467) 2023-07-27 13:09:13 +02:00
Oleg Pipikin
ff0b51f3ca Rename CommonTestUtils namespace (#18590) 
* Rename CommonTestUtils namespace 1

* Rename CommonTestUtils namespace 2

* Rename CommonTestUtils namespace 3

* Rename CommonTestUtils namespace 4

* Fix1

* Fix2

* Fix3

* Apply comments

* Fix4

* Fix5
 2023-07-26 21:06:50 +02:00
Ilya Churaev
18505b99b7 Removed core member remapped devices and use proxy config instead (#18791) 2023-07-26 08:20:43 +00:00
Nadezhda Ageeva
97b4b13074 Adds ov::internal::exclusive_async_requests in ov::internal::supported_properties (#18746) 
* Adds ov::internal::exclusive_async_requests into ov::internal::supported_properties

* Update HETERO tests for ov::internal::exclusive_async_requests
 2023-07-26 09:43:21 +04:00
Ilya Churaev
7767af3529 Fixed proxy plugin initialization with internal names (#18783) 2023-07-25 22:14:42 +04:00
Ilya Churaev
d3fdc761f6 Try to fix get_data() and lto (#18766) 
* Try to fix get_data() and lto

* Added test
 2023-07-25 14:09:21 +04:00
cecilia peng
7fbd3a7ebf [CPU] Zero-copy optimizations for model outputs (#18476) 
- Implement zero-copy output between plugin graph and infer request, thus eliminate memory copy overhead and optimize performance
- Implement double buffer for InferRequest outputs
 2023-07-25 10:33:48 +04:00
Ilya Churaev
e7cd113a47 Add to_string() method to ov::element::Type (#18697) 
* Add to_string() method to ov::element::Type

* Fixed Python legacy API

* Fixed typo
 2023-07-25 09:22:16 +04:00
Ilya Churaev
29ca4b99c5 Add property to configure internal name under the proxy plugin (#18727) 2023-07-25 09:22:02 +04:00
Ilya Lavrenov
86c68fffe4 Removed Intel Myriad headers from new API (#18736) 
* Removed Intel Myriad from new API

* Renamed GPU unit tests
 2023-07-24 16:20:08 +04:00
Wanglei Shen
0974cb10bc add socket id initialization in MacOS (#18717) 2023-07-21 18:49:36 +00:00
Wanglei Shen
98026d420f update proc_type_table for MacOS (#18702) 2023-07-21 21:13:28 +08:00
Ilya Churaev
44cae128cc Restore information about Blob layout for new plugins (#18696) 
* Restore information about Blob layout for new plugins

* Save legacy names
 2023-07-21 14:28:55 +04:00
Sun Xiaoxia
2dfb537bcb Xiaoxia/add get_socket_id interface based on threading2.0 (#18264) 
* add streams_info_table init in the constructor of config

* add refresh _proc_type_table

* add get_org_proc_type_table

* add numa_node per stream in reserve_available_cpus()

* fix warning

* remove log

* fix code style

* fix gpu test build failed issue, modify debug info

* fix code style

* fix build failed on macos

* fix code style

* select socket in reserve cpu on 2 sockets platform

* fix build failed on macos

* modify numa node selecting in reserve_cpu_by_streams_info

* add test case

* fix code style

* modify test case

* fix core dump

* fix core dumped on pin=NUMA

* fix test failed on macos

* fix reserve cpu is wrong when streams_info_table=[1 1 36] proc_type_table=[36 18 0 18]

* add test case in LinuxCpuReserve

* modify test case

* add test case in cpu_reserve_test

* add cpu_stream_info_test

* modify enum

* fix test failed

* change int to size_t

* remove invalid code, fix running failed on macos

* modify LinuxCpuStreamType test case, move ie_cpu_streams_info.hpp to openvino/runtime/threading/

* fix code sytle

* modify enum name

* add comments in test case

* fix build issue

* change IE_ASSERT to OPENVINO_ASSERT

* fix test failed on macos and windows

* updated test cases due to the cpu mapping is changed

* enable numa_node_id and socket_id in streams_info_table

* fix code style issue

* fix document issue

* add get socket id interface

* fix segment fault on machine enabled socket_id=1 with numactl command

* fix numactl failed on four numa nodes machine

* remove compile warning

* fix numa_node_id=-1

* fix test case failed on macos

* fix test failed on macos

* fix numa_node_id=0 on macos

* Solve conflicts with master branch

* separate test cases for Linux and Mac/Windows

* update code style for windows compiler

* fix comments

* fix code style

* fix code style

* remove _plugin_mutex, fix comments

* fix code style

* fix code style

* add get_num_sockets

* fix cpu reserve issue in latency mode,ANY core on RPL machine

* add cpu reserve test case

---------

Co-authored-by: Wanglei Shen <wanglei.shen@intel.com>
 2023-07-20 23:34:21 +08:00
Ilya Churaev
3f675ce396 Extend Proxy tests to cover cache_dir with auto batch (#18632) 
* Added new tests to reproduce issue with auto batch

* Remove cache dir

* Fixed tests with auto batch

* Fixed check

* Added comment

* Remove try catch

* Remove comment

* Revert "Remove comment"

This reverts commit 99e26de4ca.

* Revert "Remove try catch"

This reverts commit 08b478c070.
 2023-07-20 11:11:41 +04:00
River Li
0e76496acc [WA] performance issue for plugin api 2.0 (#18509) 
* [WA] performance issue for plugin api 2.0

* Avoid onnx failure

* Avoid ov_template_func_tests failure

[ RUN      ] smoke_BehaviorTests/OVRemoteTest.inferWithRemoteNoThrow/element_type=f32targetDevice=TEMPLATE__context_parameters=__tensor_parameters=

MEM_USAGE=400336KB
/mnt/data1_1c41/river/openvino_master/openvino/src/tests/functional/plugin/shared/src/behavior/ov_plugin/remote.cpp:107: Failure
Expected: infer_request.set_input_tensor(0, input_remote_tensor) doesn't throw an exception.
  Actual: it throws.
[  FAILED  ] smoke_BehaviorTests/OVRemoteTest.inferWithRemoteNoThrow/element_type=f32targetDevice=TEMPLATE__context_parameters=__tensor_parameters=, where GetParam() = (f32, "TEMPLATE", {}, ({}, {})) (76 ms)
 2023-07-20 08:22:31 +04:00
Marcin Kusmierski
186b1b6bfc Gna 3 5 as default target with transpose fixes (#18373) 
* [GNA] Set GNA 3.5 as default target and add 3.5 to properties

* [GNA] fix release version to 2023.1.

Co-authored-by: Szymon Irzabek <szymon.jakub.irzabek@intel.com>

* [GNA] Fix E2E Tests for Kaldi Framework

 * added fix for InsertConvolutionTransposeHW

---------

Co-authored-by: Szymon Irzabek <szymon.jakub.irzabek@intel.com>
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
 2023-07-19 14:48:01 +02:00
Sofya Balandina
8ac00677a9 [apiConformance] Move related to core tests from api to ov_inference_functional_tests (#18349) 2023-07-19 12:01:26 +03:00
Ilya Churaev
e600a8e45a Revert "Revert "Revert "Fixed cases if cast data from Tensor to compatible type (#18479)" (#18539)" (#18599)" (#18631) 
This reverts commit ea84e73282.
 2023-07-19 12:20:11 +04:00
Vitaliy Urusovskij
f470825faf Revert "Remove extra model from CompiledModelDesc" (#18612) 
* Revert "Remove extra `model` from `CompiledModelDesc`"

This reverts commit fb07fd7b82.

* Prevent use of `get_rt_model` in Hetero `export`
 2023-07-19 12:05:24 +04:00
Ilya Churaev
ea84e73282 Revert "Revert "Fixed cases if cast data from Tensor to compatible type (#18479)" (#18539)" (#18599) 
This reverts commit 3daa387692.
 2023-07-18 19:57:32 +00:00
Oleg Pipikin
cdef86c366 Move unit and func test utils (#18423) 
* Move unit and func test utils. Rename targets

* Fix1
 2023-07-18 16:23:58 +02:00
Chen Xu
9334ad0790 [CPU] Reduce node supports fp16 precision (#18227) 2023-07-18 12:49:25 +00:00
Oleg Pipikin
b211767280 Consolidate common test utils targets (#18385) 
* Consolidate test util targets

* Fix code style

* Fix 1

* Fix code style
 2023-07-17 11:59:13 +00:00
Ilya Churaev
f38566834b Remove unsupported plugin from proxy in order to avoid exception catching (#18518) 
* Remove unsupported plugin from proxy in order to avoid exception
catching

* Remove from alias for

* Globally remove unavailable devices

* Do not load proxy if no available devices

* Try to fix CI

* Add debug messages

* Avoid two times the same plugin

* Revert "Add debug messages"

This reverts commit 562e36c633.

* Cache hidden devices

* Update if fallback order was changed

* Try to fix CI

* Fixed CoreThreading tests

* Fixed typo

* Try to fix plugin initialization

* Fixed initialization

* Fixed lock
 2023-07-17 11:10:59 +04:00
Vladimir Paramuzov
325d02b760 [GPU] Use stream executor and exceptions from new api (#18531) 
* [Common] Handle ov::Exception too in CompiledModel wrapper

* [GPU] Use new threading and exception interfaces where possible
 2023-07-14 20:51:40 +02:00
Ilya Churaev
93e30468b1 Change plugin api (#18441) 
* Remove vector of so objects from Tensor, Any, VariableState and
RemoteContext

* Change IRemoteContext

* Fixed build without Proxy

* Moved API to ITensor

* Remove friends from Tensor class

* Changed query_state API

* Remote SoPtr for some returned values

* Fixed auto plugin

* Add so to converted objects

* Fixed build all on macOS

* Check that tensor pointers != nullptr

* Add SO to converter

* Added new constructors for SO ptrs

* Changed IVariableState API

* Remove proxy friend

* Remove friends and nullptrs from auto

* Fixed build

* Fixed HETERO plugin

* Fixed code style
 2023-07-14 10:47:53 +04:00
Vitaliy Urusovskij
43ff356f1a Hetero 2.0 (#18429) 
* Hetero 2.0: Implement `hetero::Plugin`

* Fix CMakeLists.txt

* Remove duplicated "itt.hpp"

* Remove extra API method

* Delete `HeteroPlugin::Engine`

* Intregrate `ov::hetero::Plugin` into `HeteroExeNet`

* Use `is_new_api()`

* Add `override` modificator

* Rename `GetDevicePlugins` to `get_device_properties`

* Small fixes

* Fix precommit

* Fix Import/Export

* Fix `import_model()`

* Remove extra lines

* TEMPORARY disable one test

* ClangFormat+Refactoring

* Build fix

* Update `hetero::Configuration` logic

* Update `hetero::Plugin::get_property()`

* Fix

* Changes after code review

* Remove extra comments

* Revert "TEMPORARY disable one test"

This reverts commit 13fa48d8102ec5146e79987835e6f7444e469f7a.

* Small fix

* Refactoring + Fixes

* Add `hetero::Plugin::device_supports_model_caching`

* Replace `device_supports_model_caching` with ICore impl

* Code review changes

* Rework `Configuration`

* Implement `hetero::CompiledModel`

* Fix caching_test

* ClangFormat

* Fix inputs/outputs mapping
After some fix in ICompiledModel, m_inputs != model->inputs,
so mapping was updated to be independent from m_inputs
(same for m_outputs)

* Fix types

* Update `get_runtime_model()`

* Remove `ngraph` and `IE_THROW`

* Implement Hetero Sync/Async Infer Requests

* A lot of changes

* Add `submodels_output_to_input` to import/export

* Implement `get_tensors()`/`set_tensors()`

* Fix `Configuration` ctor

* Skip `SetPreProcessTo` legacy tests for hetero

* Change `Output->Input` logic to `Input->PrevOutput`
because for one output may be several inputs
(map key collision)

* Update `CompiledModel` to use shared_ptr vs *

* Update `CompiledModel`  public/private members

* Names refactoring

* Optimize imports

* Fix properties for `query_model()` call

* Use static objects in `Configuration` methods

* `Configuration` refactoring

* Get `ov::supported_properties` from subdevices

* InferRequest +1 import and fwd declaration

* Update `InferRequest` prof_info, remove extra methods

* Remove `InferRequestDesc`

* Remove `InferRequest::cancel()`

* Update CMakeLists.txt

* Remove extra `catch` in `CompiledModel`

* Replace `InferenceEngine::details::contains()`

* ClangFormat

* Use `const auto&`

* Try to fix types

* Remove `friendly_name` for subgraphs connection

* Remove TODO comment

* Remove `CompiledModel::m_model` as unused

* Polish .hpp files

* Fix size_t serialization in `CompiledModel::export_model()`

* Minor changes

* Move `intersect*` IE utils to anonym namespace

* Use `const auto&`

* Update Hetero `get_profiling_info()`

* Minor changes

* Rename `deviceName` to `device_name`

* Add `hetero::VariableState` in order to hold HW so

* Port internal properties to `ov::internal`

* Add `CompiledModel::set_inputs_and_outputs()`

* Add `OPENVINO_SUPPRESS_DEPRECATED_*`

* Fix CompiledModel `ov::device::properties` issue

* Use `get_device_name()` instead of hardcoded name

* Make `get_profiling_info()` names bwd compatible

* Remove `include` folder

* Remove try/catch from CompiledModel

* Update error message

* `CompiledModel::set_property` throw not implemented
 2023-07-13 23:26:04 +04:00
Ilya Lavrenov
3daa387692 Revert "Fixed cases if cast data from Tensor to compatible type (#18479)" (#18539) 
This reverts commit 63b18adf68.
 2023-07-13 18:34:53 +04:00
Ilya Churaev
63b18adf68 Fixed cases if cast data from Tensor to compatible type (#18479) 
* Fixed cases if cast data from Tensor to compatible type

* Allow cast only if both element types are real or not
 2023-07-12 22:51:34 +04:00
Nadezhda Ageeva
3330cd69a7 [GPU] Introduce ov::config_device_id internal property to the new API (#17820) 
* [GPU] Introduce ov::config_device_id internal property to the new API. Fix setting property for specific device

* Introduce ov::internal::supported_properties

* Fix caching tests

* Fix win tests

* Fix Proxy tests

* Replace CONFIG_KEY_INTERNAL(CONFIG_DEVICE_ID) by ov::internal::config_device_id in PROXY plugin

* Review comments
 2023-07-12 10:03:35 +00:00
Ilya Churaev
0927e867b0 Implement ov::IInferRequestInternalWrapper::GetPreProcess to recover python GetPrepProcess tests (#18484) 
Co-authored-by: River.Li <river.li@intel.com>
 2023-07-12 00:02:54 +04:00