[PYTHON API] Tensor.data property for low precisions + packing (#11131)

* rebase old branch with master

* Fix doc style

* fix test

* update tests

* Add missed param

* Rewrite docstring for tensor and refactor set_input_tensors test

* update python exclusives

* keep compatibility

* remove notes about slices

* fix code style

* Fix code style
This commit is contained in:
Alexey Lebedev authored and GitHub committed 2022-04-01 12:04:04 +03:00
1 parent 701d75eafa
commit 6eaa15745a
10 files changed
+314 -64

No files matched your search

+8 -6
View File
@@ -28,12 +28,6 @@ Python API allows passing data as tensors. `Tensor` object holds a copy of the d
@snippet docs/snippets/ov_python_exclusives.py tensor_shared_mode
### Slices of array's memory
One of the `Tensor` class constructors allows to share the slice of array's memory. When `shape` is specified in the constructor that has the numpy array as first argument, it triggers the special shared memory mode.
@snippet docs/snippets/ov_python_exclusives.py tensor_slice_mode
## Running inference
Python API supports extra calling methods to synchronous and asynchronous modes for inference.
@@ -76,6 +70,14 @@ The callback of `AsyncInferQueue` is uniform for every job. When executed, GIL i
@snippet docs/snippets/ov_python_exclusives.py asyncinferqueue_set_callback
### Working with u1, u4 and i4 element types
Since openvino supports low precision element types there are few ways how to handle them in python.
To create an input tensor with such element types you may need to pack your data in new numpy array which byte size matches original input size:
@snippet docs/snippets/ov_python_exclusives.py packing_data
To extract low precision values from tensor into numpy array you can use next helper:
@snippet docs/snippets/ov_python_exclusives.py unpacking
### Releasing the GIL
+17 -11
View File
@@ -50,17 +50,6 @@ shared_tensor.data[0][2] = 0.6
assert data_to_share[0][2] == 0.6
#! [tensor_shared_mode]
#! [tensor_slice_mode]
data_to_share = np.ones(shape=(2,8))
# Specify slice of memory and the shape
shared_tensor = ov.Tensor(data_to_share[1][:] , shape=ov.Shape([8]))
# Editing of the numpy array affects Tensor's data
data_to_share[1][:] = 2
assert np.array_equal(shared_tensor.data, data_to_share[1][:])
#! [tensor_slice_mode]
infer_request = compiled.create_infer_request()
data = np.random.randint(-5, 3 + 1, size=(8))
@@ -132,6 +121,23 @@ infer_queue.wait_all()
assert all(data_done)
#! [asyncinferqueue_set_callback]
unt8_data = np.ones([100])
#! [packing_data]
from openvino.helpers import pack_data
packed_buffer = pack_data(unt8_data, ov.Type.u4)
# Create tensor with shape in element types
t = ov.Tensor(packed_buffer, [1, 128], ov.Type.u4)
#! [packing_data]
#! [unpacking]
from openvino.helpers import unpack_data
unpacked_data = unpack_data(t.data, t.element_type, t.shape)
assert np.array_equal(unpacked_data , unt8_data)
#! [unpacking]
#! [releasing_gil]
import openvino.runtime as ov
import cv2 as cv