[PYTHON API] release GIL (#10810)

* AsyncInferQueue nogil update + refactoring

* nogil in compiled model

* nogil in Core

* fix refactoring

* nogil in infer_request

* add tests

* Fix code style

* update test with incrementing reference counting

* try to fix code style

* fix code style

* release gil in reshape and preprocessing

* make args optional in test

* fix code style

* add docs about GIL

* try to link doc string with docs

* Apply suggestions from code review

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>

* Fix docs

* docs refactoring

* Apply review comments

* Fix code style

Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: Anastasia Kuporosova <anastasia.kuporosova@intel.com>
This commit is contained in:
authored and GitHub committed 2022-03-31 16:12:48 +03:00
1 parent 1d247815be
commit 1efb0a034f
9 files changed
+421 -67

No files matched your search

@@ -75,3 +75,36 @@ Another feature of `AsyncInferQueue` is ability of setting callbacks. When callb
The callback of `AsyncInferQueue` is uniform for every job. When executed, GIL is acquired to ensure safety of data manipulation inside the function.
@snippet docs/snippets/ov_python_exclusives.py asyncinferqueue_set_callback
### Releasing the GIL
Some functions in Python API release the Global Lock Interpreter (GIL) while running work-intensive code. It can help you to achieve more parallelism in your application using Python threads. For more information about GIL please refer to the Python documentation.
@snippet docs/snippets/ov_python_exclusives.py releasing_gil
> **NOTE**: While GIL is released functions can still modify and/or operate on Python objects in C++, thus there is no reference counting. User is responsible for thread safety if sharing of these objects with other thread occurs. It can affects your code only if multiple threads are spawned in Python.:
#### List of functions that release the GIL
- openvino.runtime.AsyncInferQueue.start_async
- openvino.runtime.AsyncInferQueue.is_ready
- openvino.runtime.AsyncInferQueue.wait_all
- openvino.runtime.AsyncInferQueue.get_idle_request_id
- openvino.runtime.CompiledModel.create_infer_request
- openvino.runtime.CompiledModel.infer_new_request
- openvino.runtime.CompiledModel.__call__
- openvino.runtime.CompiledModel.export
- openvino.runtime.CompiledModel.get_runtime_model
- openvino.runtime.Core.compile_model
- openvino.runtime.Core.read_model
- openvino.runtime.Core.import_model
- openvino.runtime.Core.query_model
- openvino.runtime.Core.get_available_devices
- openvino.runtime.InferRequest.infer
- openvino.runtime.InferRequest.start_async
- openvino.runtime.InferRequest.wait
- openvino.runtime.InferRequest.wait_for
- openvino.runtime.InferRequest.get_profiling_info
- openvino.runtime.InferRequest.query_state
- openvino.runtime.Model.reshape
- openvino.preprocess.PrePostProcessor.build