[PYTHON API] Tensor.data property for low precisions + packing (#11131)
* rebase old branch with master * Fix doc style * fix test * update tests * Add missed param * Rewrite docstring for tensor and refactor set_input_tensors test * update python exclusives * keep compatibility * remove notes about slices * fix code style * Fix code style
This commit is contained in:
1 parent
701d75eafa
commit
6eaa15745a
10 files changed
+314
-64
No files matched your search
@@ -28,12 +28,6 @@ Python API allows passing data as tensors. `Tensor` object holds a copy of the d
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py tensor_shared_mode
|
||||
|
||||
### Slices of array's memory
|
||||
|
||||
One of the `Tensor` class constructors allows to share the slice of array's memory. When `shape` is specified in the constructor that has the numpy array as first argument, it triggers the special shared memory mode.
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py tensor_slice_mode
|
||||
|
||||
## Running inference
|
||||
|
||||
Python API supports extra calling methods to synchronous and asynchronous modes for inference.
|
||||
@@ -76,6 +70,14 @@ The callback of `AsyncInferQueue` is uniform for every job. When executed, GIL i
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py asyncinferqueue_set_callback
|
||||
|
||||
### Working with u1, u4 and i4 element types
|
||||
|
||||
Since openvino supports low precision element types there are few ways how to handle them in python.
|
||||
To create an input tensor with such element types you may need to pack your data in new numpy array which byte size matches original input size:
|
||||
@snippet docs/snippets/ov_python_exclusives.py packing_data
|
||||
|
||||
To extract low precision values from tensor into numpy array you can use next helper:
|
||||
@snippet docs/snippets/ov_python_exclusives.py unpacking
|
||||
|
||||
### Releasing the GIL
|
||||
|
||||
|
||||
@@ -50,17 +50,6 @@ shared_tensor.data[0][2] = 0.6
|
||||
assert data_to_share[0][2] == 0.6
|
||||
#! [tensor_shared_mode]
|
||||
|
||||
#! [tensor_slice_mode]
|
||||
data_to_share = np.ones(shape=(2,8))
|
||||
|
||||
# Specify slice of memory and the shape
|
||||
shared_tensor = ov.Tensor(data_to_share[1][:] , shape=ov.Shape([8]))
|
||||
|
||||
# Editing of the numpy array affects Tensor's data
|
||||
data_to_share[1][:] = 2
|
||||
assert np.array_equal(shared_tensor.data, data_to_share[1][:])
|
||||
#! [tensor_slice_mode]
|
||||
|
||||
infer_request = compiled.create_infer_request()
|
||||
data = np.random.randint(-5, 3 + 1, size=(8))
|
||||
|
||||
@@ -132,6 +121,23 @@ infer_queue.wait_all()
|
||||
assert all(data_done)
|
||||
#! [asyncinferqueue_set_callback]
|
||||
|
||||
unt8_data = np.ones([100])
|
||||
|
||||
#! [packing_data]
|
||||
from openvino.helpers import pack_data
|
||||
|
||||
packed_buffer = pack_data(unt8_data, ov.Type.u4)
|
||||
# Create tensor with shape in element types
|
||||
t = ov.Tensor(packed_buffer, [1, 128], ov.Type.u4)
|
||||
#! [packing_data]
|
||||
|
||||
#! [unpacking]
|
||||
from openvino.helpers import unpack_data
|
||||
|
||||
unpacked_data = unpack_data(t.data, t.element_type, t.shape)
|
||||
assert np.array_equal(unpacked_data , unt8_data)
|
||||
#! [unpacking]
|
||||
|
||||
#! [releasing_gil]
|
||||
import openvino.runtime as ov
|
||||
import cv2 as cv
|
||||
|
||||
Reference in new issue
Block a user