IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'upstream/master' into debian-packages

Browse Source
This commit is contained in:
Ilya Lavrenov
2022-02-14 19:29:34 +03:00
parent db2e5d348e 3a5d821219
commit f637f8fbc4
213 changed files with 7315 additions and 5638 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
.ci/azure/linux.yml
View File

@@ -236,8 +236,16 @@ jobs:
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) -pyver 3.8 && python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph $(PYTHON_STATIC_ARGS) --junitxml=TEST-Pyngraph.xml --ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_utils/test_utils.py --ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py --ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph Python Bindings Tests'
. $(SETUPVARS) -pyver 3.8 && python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph $(PYTHON_STATIC_ARGS) --junitxml=TEST-Pyngraph.xml --ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py --ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph and IE Python Bindings Tests'
continueOnError: false
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) -pyver 3.8 && python3 -m pytest -s $(INSTALL_TEST_DIR)/pyopenvino $(PYTHON_STATIC_ARGS) --junitxml=TEST-Pyngraph.xml --ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_utils/test_utils.py --ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_zoo_models.py --ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_backend.py
displayName: 'Python API 2.0 Tests'
continueOnError: false
- script: |
@@ -336,14 +344,6 @@ jobs:
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
cd $(REPO_DIR)/src/bindings/python/tests_compatibility/test_inference_engine
. $(SETUPVARS) -pyver 3.8 && python3 -m pytest --junitxml=TEST-PythonAPI.xml $(PYTHON_STATIC_ARGS)
displayName: 'Python API Tests'
continueOnError: false
- script: |
. $(SETUPVARS)
python3 -m pip install -r requirements.txt

96
docs/api/ie_python_api/api.rst
View File

@@ -1,8 +1,98 @@
Inference Engine Python API
===========================
OpenVINO Python API
===================
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.inference_engine
openvino.runtime
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.op
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset1
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset2
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset3
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset4
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset5
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset6
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset7
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.opset8
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.passes
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.runtime.utils
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.preprocess
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.frontend
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.offline_transformations
.. toctree::
:maxdepth: 2
:hidden:
compatibility

20
docs/api/ie_python_api/compatibility.rst Normal file
View File

@@ -0,0 +1,20 @@
OpenVINO Python API - Compatibility
===================================
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
openvino.inference_engine
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
ngraph
.. autosummary::
:toctree: _autosummary
:template: custom-module-template.rst
_pyngraph

18
docs/dlstreamer.md Normal file
View File

@@ -0,0 +1,18 @@
# Intel® Deep Learning Streamer (Intel® DL Streamer) {#openvino_docs_dlstreamer}
Intel® DL Streamer is a streaming media analytics framework, based on GStreamer* multimedia framework, for creating complex media analytics pipelines.
Intel® DL Streamer makes Media analytics easy:
* Write less code and get better performance
* Quickly develop, optimize, benchmark, and deploy video & audio analytics pipelines in the Cloud and at the Edge
* Analyze video and audio streams, create actionable results, capture results, and send them to the cloud
* Leverage the efficiency and computational power of Intel hardware platforms
Go to [Intel® DL Streamer documentation website](https://dlstreamer.github.io) for information on how to download, install, and use.
**Media analytics** is the analysis of audio & video streams to detect, classify, track, identify and count objects, events and people. The analyzed results can be used to take actions, coordinate events, identify patterns and gain insights across multiple domains.
**Media analytics pipelines** transform media streams into insights through audio / video processing, inference, and analytics operations across multiple IP blocks.
\* Other names and brands may be claimed as the property of others.

16
docs/documentation.md
View File

@@ -49,10 +49,9 @@
.. toctree::
:maxdepth: 1
:hidden:
:caption: Media Processing
:caption: Media Processing and Computer Vision Libraries
DL Streamer API Reference <https://openvinotoolkit.github.io/dlstreamer_gst/>
gst_samples_README
Intel® Deep Learning Streamer <openvino_docs_dlstreamer>
openvino_docs_gapi_gapi_intro
OpenVX Developer Guide <https://software.intel.com/en-us/openvino-ovx-guide>
OpenVX API Reference <https://khronos.org/openvx>
@@ -102,5 +101,12 @@ The toolkit provides a [Performance Optimization Guide](optimization_guide/dldt_
## Graphical Web Interface for OpenVINO™ Toolkit
You can choose to use the [OpenVINO™ Deep Learning Workbench](@ref workbench_docs_Workbench_DG_Introduction), a web-based tool that guides you through the process of converting, measuring, optimizing, and deploying models. This tool also serves as a low-effort introduction to the toolkit and provides a variety of useful interactive charts for understanding performance.
## Media Processing
The OpenVINO™ toolkit comes with several sets of libraries and tools that add capability and flexibility to the toolkit. These include [DL Streamer](@ref gst_samples_README), a utility that eases creation of pipelines via command line or API, and optimized versions of OpenCV and OpenCL.
## Media Processing and Computer Vision Libraries
The OpenVINO™ toolkit also works with the following media processing frameworks and libraries:
* [Intel® Deep Learning Streamer (Intel® DL Streamer)](@ref openvino_docs_dlstreamer) — A streaming media analytics framework based on GStreamer, for creating complex media analytics pipelines optimized for Intel hardware platforms. Go to the Intel® DL Streamer [documentation](https://dlstreamer.github.io/) website to learn more.
* [Intel® oneAPI Video Processing Library (oneVPL)](https://www.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/api-based-programming/intel-oneapi-video-processing-library-onevpl.html) — A programming interface for video decoding, encoding, and processing to build portable media pipelines on CPUs, GPUs, and other accelerators.
You can also add computer vision capabilities to your application using optimized versions of [OpenCV](https://opencv.org/) and [OpenVX](https://khronos.org/openvx).

10
docs/openvino_sphinx_theme/openvino_sphinx_theme/templates/custom-module-template.rst
View File

@@ -3,6 +3,7 @@
.. automodule:: {{ fullname }}
{% block attributes %}
{% if attributes %}
.. rubric:: Module Attributes
.. autosummary::
@@ -10,9 +11,11 @@
{% for attr in attributes %}
{{ attr }}
{% endfor %}
{% endif %}
{% endblock %}
{% block functions %}
{% if functions %}
.. rubric:: Functions
.. autosummary::
@@ -20,9 +23,11 @@
{% for func in functions %}
{{ func }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block classes %}
{% if classes %}
.. rubric:: Classes
.. autosummary::
@@ -31,9 +36,11 @@
{% for cl in classes %}
{{ cl }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block exceptions %}
{% if exceptions %}
.. rubric:: Exceptions
.. autosummary::
@@ -41,9 +48,11 @@
{% for item in exceptions %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block modules %}
{% if modules %}
.. rubric:: Modules
.. autosummary::
@@ -53,4 +62,5 @@
{% for mod in modules %}
{{ mod }}
{%- endfor %}
{% endif %}
{% endblock %}

47
docs/template_plugin/src/template_infer_request.cpp
View File

@@ -118,18 +118,8 @@ static void AllocateImpl(const BlobDataMap& userDataMap,
GetNetworkPrecisionF&& GetNetworkPrecision,
bool isInputBlob = true) {
for (const auto& userData : userDataMap) {
IE_SUPPRESS_DEPRECATED_START
auto partialShape = userData.second->getPartialShape();
IE_SUPPRESS_DEPRECATED_END
SizeVector dims;
if (partialShape.is_static()) {
dims = userData.second->getTensorDesc().getDims();
} else if (partialShape.rank().is_static()) {
dims = SizeVector(partialShape.rank().get_length(), 0);
} else {
dims = SizeVector{0};
}
AllocateImplSingle(userBlobMap, deviceBlobMap, userData, GetNetworkPrecision, dims);
auto tensorDesc = userData.second->getTensorDesc();
AllocateImplSingle(userBlobMap, deviceBlobMap, userData, GetNetworkPrecision, tensorDesc.getDims());
}
}
@@ -436,17 +426,20 @@ InferenceEngine::Blob::Ptr TemplateInferRequest::GetBlob(const std::string& name
} else {
data = _outputs[name];
SizeVector dims;
IE_SUPPRESS_DEPRECATED_START
if (!foundOutput->isDynamic()) {
auto has_zeros = [](const SizeVector& vec) {
return std::any_of(vec.cbegin(), vec.cend(), [](size_t e) {
return e == 0;
});
};
if (!has_zeros(foundOutput->getTensorDesc().getDims())) {
dims = foundOutput->getTensorDesc().getDims();
} else if (_outputTensors[_executableNetwork->_outputIndex.at(name)] &&
_outputTensors[_executableNetwork->_outputIndex.at(name)]->get_partial_shape().is_static()) {
dims = _outputTensors[_executableNetwork->_outputIndex.at(name)]->get_shape();
} else {
auto rank = foundOutput->getPartialShape().rank();
dims = SizeVector(rank.is_dynamic() ? 1 : rank.get_length(), 0);
auto rank = foundOutput->getTensorDesc().getDims().size();
dims = SizeVector(rank == 0 ? 1 : rank, 0);
}
IE_SUPPRESS_DEPRECATED_END
if (data->getTensorDesc().getDims() != dims) {
auto&& results = _executableNetwork->_function->get_results();
@@ -476,17 +469,21 @@ void TemplateInferRequest::SetBlob(const std::string& name, const InferenceEngin
IE_THROW(NotAllocated) << "Failed to set empty blob with name: \'" << name << "\'";
InputInfo::Ptr foundInput;
DataPtr foundOutput;
auto has_zeros = [](const SizeVector& vec) {
return std::any_of(vec.cbegin(), vec.cend(), [](size_t e) {
return e == 0;
});
};
const bool isInput = findInputAndOutputBlobByName(name, foundInput, foundOutput);
const bool compoundBlobPassed = userBlob->is<CompoundBlob>();
const bool remoteBlobPassed = userBlob->is<RemoteBlob>();
if (!compoundBlobPassed && !remoteBlobPassed && userBlob->buffer() == nullptr)
IE_THROW(NotAllocated) << "Input data was not allocated. Input name: \'" << name << "\'";
IE_SUPPRESS_DEPRECATED_START
if (userBlob->size() == 0 &&
!((foundInput && foundInput->getInputData()->isDynamic()) || (foundOutput && foundOutput->isDynamic()))) {
bool input_dynamic = foundInput && has_zeros(foundInput->getInputData()->getDims());
bool output_dynamic = foundOutput && has_zeros(foundOutput->getDims());
if (userBlob->size() == 0 && !(input_dynamic || output_dynamic)) {
IE_THROW() << "Input data is empty. Input name: \'" << name << "\'";
}
IE_SUPPRESS_DEPRECATED_END
size_t dataSize = userBlob->size();
if (isInput) {
@@ -503,13 +500,11 @@ void TemplateInferRequest::SetBlob(const std::string& name, const InferenceEngin
auto devDims = devBlob->getTensorDesc().getDims();
auto devLayout = devBlob->getTensorDesc().getLayout();
auto devPrecision = devBlob->getTensorDesc().getPrecision();
IE_SUPPRESS_DEPRECATED_START
if (foundInput->getInputData()->isDynamic() && (devDims != usrDims || devLayout != usrLayout)) {
if (input_dynamic && (devDims != usrDims || devLayout != usrLayout)) {
devBlob = make_blob_with_precision({devPrecision, usrDims, TensorDesc::getLayoutByDims(usrDims)});
devBlob->allocate();
_deviceInputs[name] = devBlob;
}
IE_SUPPRESS_DEPRECATED_END
const bool preProcRequired = preProcessingRequired(foundInput, userBlob, devBlob);
if (compoundBlobPassed && !preProcRequired) {
IE_THROW(NotImplemented) << "cannot set compound blob: supported only for input pre-processing";
@@ -539,13 +534,11 @@ void TemplateInferRequest::SetBlob(const std::string& name, const InferenceEngin
auto devDims = devBlob->getTensorDesc().getDims();
auto devLayout = devBlob->getTensorDesc().getLayout();
auto devPrecision = devBlob->getTensorDesc().getPrecision();
IE_SUPPRESS_DEPRECATED_START
if (foundOutput->isDynamic() && (devDims != usrDims || devLayout != usrLayout)) {
if (output_dynamic && (devDims != usrDims || devLayout != usrLayout)) {
devBlob = make_blob_with_precision({devPrecision, usrDims, TensorDesc::getLayoutByDims(usrDims)});
devBlob->allocate();
_networkOutputBlobs[name] = devBlob;
}
IE_SUPPRESS_DEPRECATED_END
size_t outputSize = devBlob->getTensorDesc().getLayout() != InferenceEngine::Layout::SCALAR
? details::product(devBlob->getTensorDesc().getDims())
: 1;

197
docs/template_plugin/tests/functional/op_reference/reverse.cpp
View File

@@ -2,22 +2,29 @@
// SPDX-License-Identifier: Apache-2.0
//
#include "openvino/op/reverse.hpp"
#include <gtest/gtest.h>
#include "openvino/op/reverse.hpp"
#include "openvino/op/constant.hpp"
#include "base_reference_test.hpp"
#include "openvino/op/constant.hpp"
using namespace reference_tests;
using namespace ov;
namespace {
struct ReverseParams {
ReverseParams(const reference_tests::Tensor& constantTensor, const op::v1::Reverse::Mode reverseMode,
const reference_tests::Tensor& dataTensor, const reference_tests::Tensor& expectedTensor, const std::string& testcaseName = "") :
constantTensor(constantTensor), reverseMode(reverseMode),
dataTensor(dataTensor), expectedTensor(expectedTensor),
testcaseName(testcaseName) {}
ReverseParams(const reference_tests::Tensor& constantTensor,
const op::v1::Reverse::Mode reverseMode,
const reference_tests::Tensor& dataTensor,
const reference_tests::Tensor& expectedTensor,
const std::string& testcaseName = "")
: constantTensor(constantTensor),
reverseMode(reverseMode),
dataTensor(dataTensor),
expectedTensor(expectedTensor),
testcaseName(testcaseName) {}
reference_tests::Tensor constantTensor;
op::v1::Reverse::Mode reverseMode;
@@ -60,21 +67,17 @@ private:
params.constantTensor.shape,
params.constantTensor.data.data());
const auto reverse = std::make_shared<op::v1::Reverse>(data, constant, params.reverseMode);
return std::make_shared<ov::Model>(NodeVector {reverse}, ParameterVector {data});
return std::make_shared<ov::Model>(NodeVector{reverse}, ParameterVector{data});
}
};
class ReferenceReverseTestAxesRankIndexMode : public ReferenceReverseTest {
};
class ReferenceReverseTestAxesRankIndexMode : public ReferenceReverseTest {};
class ReferenceReverseTestAxesElemsMaskMode : public ReferenceReverseTest {
};
class ReferenceReverseTestAxesElemsMaskMode : public ReferenceReverseTest {};
class ReferenceReverseTestAxesOutOfBounds : public ReferenceReverseTest {
};
class ReferenceReverseTestAxesOutOfBounds : public ReferenceReverseTest {};
class ReferenceReverseTestAxesOutOfBounds4 : public ReferenceReverseTest {
};
class ReferenceReverseTestAxesOutOfBounds4 : public ReferenceReverseTest {};
TEST_P(ReferenceReverseTest, CompareWithRefs) {
Exec();
@@ -82,37 +85,38 @@ TEST_P(ReferenceReverseTest, CompareWithRefs) {
TEST_P(ReferenceReverseTestAxesRankIndexMode, CompareWithRefs) {
const auto Data = std::make_shared<op::v0::Parameter>(element::f32, Shape{2, 2, 2});
const auto Rev_Axes = std::make_shared<op::v0::Parameter>(element::i64, Shape{1, 1}); // correct: 1D
EXPECT_THROW(std::make_shared<ov::Model>(std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
ParameterVector{Data, Rev_Axes}),
const auto Rev_Axes = std::make_shared<op::v0::Parameter>(element::i64, Shape{1, 1}); // correct: 1D
EXPECT_THROW(const auto unused = std::make_shared<ov::Model>(
std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
ParameterVector{Data, Rev_Axes}),
ngraph::NodeValidationFailure);
}
TEST_P(ReferenceReverseTestAxesElemsMaskMode, CompareWithRefs) {
const auto Data = std::make_shared<op::v0::Parameter>(element::f32, Shape{2, 2, 2});
const auto Rev_Axes = std::make_shared<op::v0::Parameter>(element::boolean, Shape{2}); // correct: 3
EXPECT_THROW(std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::MASK),
EXPECT_THROW(const auto unused = std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::MASK),
ngraph::NodeValidationFailure);
}
TEST_P(ReferenceReverseTestAxesOutOfBounds, CompareWithRefs) {
const auto Data = std::make_shared<op::v0::Parameter>(element::f32, Shape{2, 2, 2});
const auto Rev_Axes = op::v0::Constant::create(element::i64, Shape{2}, {1, 10});
EXPECT_THROW(std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
EXPECT_THROW(const auto unused = std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
ngraph::NodeValidationFailure);
}
TEST_P(ReferenceReverseTestAxesOutOfBounds4, CompareWithRefs) {
const auto Data = std::make_shared<op::v0::Parameter>(element::f32, Shape{2, 2, 2});
const auto Rev_Axes = op::v0::Constant::create(element::i64, Shape{4}, {0, 1, 2, 3});
EXPECT_THROW(std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
EXPECT_THROW(const auto unused = std::make_shared<op::v1::Reverse>(Data, Rev_Axes, op::v1::Reverse::Mode::INDEX),
ngraph::NodeValidationFailure);
}
template <element::Type_t IN_ET>
std::vector<ReverseParams> generateParams() {
using T = typename element_type_traits<IN_ET>::value_type;
std::vector<ReverseParams> params {
std::vector<ReverseParams> params{
// nothing_to_reverse
ReverseParams(reference_tests::Tensor({0}, element::i64, std::vector<int64_t>{}),
op::v1::Reverse::Mode::INDEX,
@@ -156,67 +160,74 @@ std::vector<ReverseParams> generateParams() {
reference_tests::Tensor({4, 3}, IN_ET, std::vector<T>{11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0}),
"reverse_2d_01_mask"),
// reverse_3d_0
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}),
"reverse_3d_0"),
ReverseParams(
reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}),
"reverse_3d_0"),
// reverse_3d_1
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{1}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
9, 10, 11, 6, 7, 8, 3, 4, 5, 0, 1, 2, 21, 22, 23, 18, 19, 20, 15, 16, 17, 12, 13, 14}),
"reverse_3d_1"),
ReverseParams(
reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{1}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{9, 10, 11, 6, 7, 8, 3, 4, 5, 0, 1, 2,
21, 22, 23, 18, 19, 20, 15, 16, 17, 12, 13, 14}),
"reverse_3d_1"),
// reverse_3d_2
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
2, 1, 0, 5, 4, 3, 8, 7, 6, 11, 10, 9, 14, 13, 12, 17, 16, 15, 20, 19, 18, 23, 22, 21}),
"reverse_3d_2"),
ReverseParams(
reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{2, 1, 0, 5, 4, 3, 8, 7, 6, 11, 10, 9,
14, 13, 12, 17, 16, 15, 20, 19, 18, 23, 22, 21}),
"reverse_3d_2"),
// reverse_3d_01
ReverseParams(reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{0, 1}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
21, 22, 23, 18, 19, 20, 15, 16, 17, 12, 13, 14, 9, 10, 11, 6, 7, 8, 3, 4, 5, 0, 1, 2}),
"reverse_3d_01"),
ReverseParams(
reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{0, 1}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{21, 22, 23, 18, 19, 20, 15, 16, 17, 12, 13, 14,
9, 10, 11, 6, 7, 8, 3, 4, 5, 0, 1, 2}),
"reverse_3d_01"),
// reverse_3d_02
ReverseParams(reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{0, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
14, 13, 12, 17, 16, 15, 20, 19, 18, 23, 22, 21, 2, 1, 0, 5, 4, 3, 8, 7, 6, 11, 10, 9}),
"reverse_3d_02"),
ReverseParams(
reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{0, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{14, 13, 12, 17, 16, 15, 20, 19, 18, 23, 22, 21,
2, 1, 0, 5, 4, 3, 8, 7, 6, 11, 10, 9}),
"reverse_3d_02"),
// reverse_3d_12
ReverseParams(reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{1, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12}),
"reverse_3d_12"),
ReverseParams(
reference_tests::Tensor({2}, element::i64, std::vector<int64_t>{1, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0,
23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12}),
"reverse_3d_12"),
// reverse_3d_012
ReverseParams(reference_tests::Tensor({3}, element::i64, std::vector<int64_t>{0, 1, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{
23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0}),
"reverse_3d_012"),
ReverseParams(
reference_tests::Tensor({3}, element::i64, std::vector<int64_t>{0, 1, 2}),
op::v1::Reverse::Mode::INDEX,
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,
12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23}),
reference_tests::Tensor({2, 4, 3}, IN_ET, std::vector<T>{23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12,
11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0}),
"reverse_3d_012"),
};
return params;
}
std::vector<ReverseParams> generateCombinedParams() {
const std::vector<std::vector<ReverseParams>> reverseTypeParams {
const std::vector<std::vector<ReverseParams>> reverseTypeParams{
generateParams<element::Type_t::i8>(),
generateParams<element::Type_t::i16>(),
generateParams<element::Type_t::i32>(),
@@ -237,7 +248,7 @@ std::vector<ReverseParams> generateCombinedParams() {
}
std::vector<ReverseParams> generateParamsAxesRankIndexMode() {
std::vector<ReverseParams> params {
std::vector<ReverseParams> params{
// reverse_v1_incorrect_rev_axes_rank_index_mode
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
@@ -249,7 +260,7 @@ std::vector<ReverseParams> generateParamsAxesRankIndexMode() {
}
std::vector<ReverseParams> generateParamsAxesElemsMaskMode() {
std::vector<ReverseParams> params {
std::vector<ReverseParams> params{
// reverse_v1_incorrect_rev_axes_elems_mask_mode
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
@@ -261,7 +272,7 @@ std::vector<ReverseParams> generateParamsAxesElemsMaskMode() {
}
std::vector<ReverseParams> generateParamsAxesOutOfBounds() {
std::vector<ReverseParams> params {
std::vector<ReverseParams> params{
// reverse_v1_axes_out_of_bounds
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
@@ -273,7 +284,7 @@ std::vector<ReverseParams> generateParamsAxesOutOfBounds() {
}
std::vector<ReverseParams> generateParamsAxesOutOfBounds4() {
std::vector<ReverseParams> params {
std::vector<ReverseParams> params{
// reverse_v1_axes_out_of_bounds_4
ReverseParams(reference_tests::Tensor({1}, element::i64, std::vector<int64_t>{0}),
op::v1::Reverse::Mode::INDEX,
@@ -284,18 +295,28 @@ std::vector<ReverseParams> generateParamsAxesOutOfBounds4() {
return params;
}
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs, ReferenceReverseTest,
testing::ValuesIn(generateCombinedParams()), ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs,
ReferenceReverseTest,
testing::ValuesIn(generateCombinedParams()),
ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs, ReferenceReverseTestAxesRankIndexMode,
testing::ValuesIn(generateParamsAxesRankIndexMode()), ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs,
ReferenceReverseTestAxesRankIndexMode,
testing::ValuesIn(generateParamsAxesRankIndexMode()),
ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs, ReferenceReverseTestAxesElemsMaskMode,
testing::ValuesIn(generateParamsAxesElemsMaskMode()), ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs,
ReferenceReverseTestAxesElemsMaskMode,
testing::ValuesIn(generateParamsAxesElemsMaskMode()),
ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs, ReferenceReverseTestAxesOutOfBounds,
testing::ValuesIn(generateParamsAxesOutOfBounds()), ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs,
ReferenceReverseTestAxesOutOfBounds,
testing::ValuesIn(generateParamsAxesOutOfBounds()),
ReferenceReverseTest::getTestCaseName);
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs, ReferenceReverseTestAxesOutOfBounds4,
testing::ValuesIn(generateParamsAxesOutOfBounds4()), ReferenceReverseTest::getTestCaseName);
} // namespace
INSTANTIATE_TEST_SUITE_P(smoke_Reverse_With_Hardcoded_Refs,
ReferenceReverseTestAxesOutOfBounds4,
testing::ValuesIn(generateParamsAxesOutOfBounds4()),
ReferenceReverseTest::getTestCaseName);
} // namespace

10
samples/cpp/benchmark_app/README.md
View File

@@ -85,11 +85,11 @@ Options:
-l "<absolute_path>" Required for CPU custom layers. Absolute path to a shared library with the kernels implementations.
Or
-c "<absolute_path>" Required for GPU custom kernels. Absolute path to an .xml file with the kernels description.
-hint "<throughput(or just 'tput')/latency">
Optional. Performance hint (optimize for latency or throughput).
The hint allows the OpenVINO device to select the right network-specific settings,
as opposite to just accepting specific values from the sample command line.
So you can specify only the hint without setting explicit 'nstreams' or other device-specific options.
-hint "performance hint (latency or throughput or none)" Optional. Performance hint allows the OpenVINO device to select the right network-specific settings.
'throughput' or 'tput': device performance mode will be set to THROUGHPUT.
'latency': device performance mode will be set to LATENCY.
'none': no device performance mode will be set.
Using explicit 'nstreams' or other device-specific options, please set hint to 'none'
-api "<sync/async>" Optional (deprecated). Enable Sync/Async API. Default value is "async".
-niter "<integer>" Optional. Number of iterations. If not specified, the number of iterations is calculated depending on a device.
-nireq "<integer>" Optional. Number of infer requests. Default value is determined automatically for a device.

15
samples/cpp/benchmark_app/benchmark_app.hpp
View File

@@ -33,10 +33,12 @@ static const char model_message[] =
/// @brief message for performance hint
static const char hint_message[] =
"Optional. Performance hint (optimize for latency or throughput)."
" The hint allows the OpenVINO device to select the right network-specific settings,"
" as opposite to just accepting specific values from the sample command line."
" So you can specify only the hint without setting explicit 'nstreams' or other device-specific options";
"Optional. Performance hint allows the OpenVINO device to select the right network-specific settings.\n"
" 'throughput' or 'tput': device performance mode will be set to THROUGHPUT.\n"
" 'latency': device performance mode will be set to LATENCY.\n"
" 'none': no device performance mode will be set.\n"
" Using explicit 'nstreams' or other device-specific options, please set hint to "
"'none'";
/// @brief message for execution mode
static const char api_message[] = "Optional (deprecated). Enable Sync/Async API. Default value is \"async\".";
@@ -371,7 +373,7 @@ static void show_usage() {
std::cout << " -l \"<absolute_path>\" " << custom_cpu_library_message << std::endl;
std::cout << " Or" << std::endl;
std::cout << " -c \"<absolute_path>\" " << custom_cldnn_message << std::endl;
std::cout << " -hint \"performance hint (latency or throughput)\" " << hint_message << std::endl;
std::cout << " -hint \"performance hint (latency or throughput or none)\" " << hint_message << std::endl;
std::cout << " -api \"<sync/async>\" " << api_message << std::endl;
std::cout << " -niter \"<integer>\" " << iterations_count_message << std::endl;
std::cout << " -nireq \"<integer>\" " << infer_requests_count_message << std::endl;
@@ -388,7 +390,8 @@ static void show_usage() {
std::cout << std::endl << " device-specific performance options:" << std::endl;
std::cout << " -nstreams \"<integer>\" " << infer_num_streams_message << std::endl;
std::cout << " -nthreads \"<integer>\" " << infer_num_threads_message << std::endl;
std::cout << " -pin \"YES\"/\"HYBRID_AWARE\"/\"NO\"/\"NUMA\" " << infer_threads_pinning_message << std::endl;
std::cout << " -pin (\"YES\"|\"CORE\")/\"HYBRID_AWARE\"/(\"NO\"|\"NONE\")/\"NUMA\" "
<< infer_threads_pinning_message << std::endl;
#ifdef HAVE_DEVICE_MEM_SUPPORT
std::cout << " -use_device_mem " << use_device_mem_message << std::endl;
#endif

57
samples/cpp/benchmark_app/main.cpp
View File

@@ -55,9 +55,10 @@ bool ParseAndCheckCommandLine(int argc, char* argv[]) {
if (FLAGS_api != "async" && FLAGS_api != "sync") {
throw std::logic_error("Incorrect API. Please set -api option to `sync` or `async` value.");
}
if (!FLAGS_hint.empty() && FLAGS_hint != "throughput" && FLAGS_hint != "tput" && FLAGS_hint != "latency") {
if (!FLAGS_hint.empty() && FLAGS_hint != "throughput" && FLAGS_hint != "tput" && FLAGS_hint != "latency" &&
FLAGS_hint != "none") {
throw std::logic_error("Incorrect performance hint. Please set -hint option to"
"either `throughput`(tput) or `latency' value.");
"`throughput`(tput), `latency' value or 'none'.");
}
if (!FLAGS_report_type.empty() && FLAGS_report_type != noCntReport && FLAGS_report_type != averageCntReport &&
FLAGS_report_type != detailedCntReport) {
@@ -106,6 +107,36 @@ static void next_step(const std::string additional_info = "") {
<< (additional_info.empty() ? "" : " (" + additional_info + ")") << std::endl;
}
ov::hint::PerformanceMode get_performance_hint(const std::string& device, const ov::Core& core) {
ov::hint::PerformanceMode ov_perf_hint = ov::hint::PerformanceMode::UNDEFINED;
auto supported_properties = core.get_property(device, ov::supported_properties);
if (std::find(supported_properties.begin(), supported_properties.end(), ov::hint::performance_mode) !=
supported_properties.end()) {
if (FLAGS_hint != "") {
if (FLAGS_hint == "throughput" || FLAGS_hint == "tput") {
slog::warn << "Device(" << device << ") performance hint is set to THROUGHPUT" << slog::endl;
ov_perf_hint = ov::hint::PerformanceMode::THROUGHPUT;
} else if (FLAGS_hint == "latency") {
slog::warn << "Device(" << device << ") performance hint is set to LATENCY" << slog::endl;
ov_perf_hint = ov::hint::PerformanceMode::LATENCY;
} else if (FLAGS_hint == "none") {
slog::warn << "No device(" << device << ") performance hint is set" << slog::endl;
ov_perf_hint = ov::hint::PerformanceMode::UNDEFINED;
}
} else {
slog::warn << "PerformanceMode was not explicitly specified in command line. "
"Device("
<< device << ") performance hint will be set to THROUGHPUT." << slog::endl;
ov_perf_hint = ov::hint::PerformanceMode::THROUGHPUT;
}
} else {
if (FLAGS_hint != "") {
slog::warn << "Device(" << device << ") does not support performance hint property(-hint)." << slog::endl;
}
}
return ov_perf_hint;
}
/**
* @brief The entry point of the benchmark application
*/
@@ -195,21 +226,6 @@ int main(int argc, char* argv[]) {
slog::info << "GPU extensions is loaded " << ext << slog::endl;
}
if (FLAGS_hint.empty()) {
for (auto& device : devices) {
auto supported_properties = core.get_property(device, ov::supported_properties);
if (std::find(supported_properties.begin(), supported_properties.end(), ov::hint::performance_mode) !=
supported_properties.end()) {
slog::warn << "-hint default value is determined as " << ov::hint::PerformanceMode::THROUGHPUT
<< " automatically for " << device
<< " device. For more detailed information look at README." << slog::endl;
std::stringstream strm;
strm << ov::hint::PerformanceMode::THROUGHPUT;
FLAGS_hint = strm.str();
}
}
}
slog::info << "OpenVINO: " << ov::get_openvino_version() << slog::endl;
slog::info << "Device info: " << slog::endl;
slog::info << core.get_versions(device_name) << slog::endl;
@@ -217,11 +233,6 @@ int main(int argc, char* argv[]) {
// ----------------- 3. Setting device configuration
// -----------------------------------------------------------
next_step();
ov::hint::PerformanceMode ov_perf_hint = ov::hint::PerformanceMode::UNDEFINED;
if (FLAGS_hint == "throughput" || FLAGS_hint == "tput")
ov_perf_hint = ov::hint::PerformanceMode::THROUGHPUT;
else if (FLAGS_hint == "latency")
ov_perf_hint = ov::hint::PerformanceMode::LATENCY;
auto getDeviceTypeFromName = [](std::string device) -> std::string {
return device.substr(0, device.find_first_of(".("));
@@ -248,6 +259,7 @@ int main(int argc, char* argv[]) {
auto& device_config = config[device];
// high-level performance modes
auto ov_perf_hint = get_performance_hint(device, core);
if (ov_perf_hint != ov::hint::PerformanceMode::UNDEFINED) {
device_config.emplace(ov::hint::performance_mode(ov_perf_hint));
if (FLAGS_nireq != 0)
@@ -288,7 +300,6 @@ int main(int argc, char* argv[]) {
auto it_device_nstreams = device_nstreams.find(device);
if (it_device_nstreams != device_nstreams.end()) {
// set to user defined value
auto supported_properties = core.get_property(device, ov::supported_properties);
if (supported(key)) {
device_config[key] = it_device_nstreams->second;
} else if (supported(ov::num_streams.name())) {

198
samples/cpp/classification_sample_async/README.md
View File

@@ -1,33 +1,32 @@
# Image Classification Async C++ Sample {#openvino_inference_engine_samples_classification_sample_async_README}
This sample demonstrates how to execute an inference of image classification networks like AlexNet and GoogLeNet using Asynchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Asynchronous Inference Request API.
Models with only one input and output are supported.
In addition to regular images, the sample also supports single-channel `ubyte` images as an input for LeNet model.
Image Classification Async C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
|OpenVINO Runtime Version| `ov::get_openvino_version` | Get Openvino API version
|Available Devices| `ov::Core::get_available_devices`| Get version information of the devices for inference
| Asynchronous Infer | `ov::InferRequest::start_async`, `ov::InferRequest::set_callback` | Do asynchronous inference with callback
|Custom Extension Kernels| `ov::Core::add_extension`, `ov::Core::set_config`| Load extension library and config to the device
| Model Operations | `ov::set_batch`, `ov::get_batch` | Managing of model, operate with its batch size. Setting batch size using input image count.
| Feature | API | Description |
| :--- | :--- | :--- |
| Asynchronous Infer | `ov::InferRequest::start_async`, `ov::InferRequest::set_callback` | Do asynchronous inference with callback. |
| Model Operations | `ov::Output::get_shape`, `ov::set_batch` | Manage the model, operate with its batch size. Set batch size using input image count. |
| Infer Request Operations | `ov::InferRequest::get_input_tensor` | Get an input tensor. |
| Tensor Operations | `ov::shape_size`, `ov::Tensor::data` | Get a tensor shape size and its data. |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx)
| Validated images | The sample uses OpenCV\* to [read input image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png), single-channel `ubyte` images.
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/classification_sample_async/README.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/classification_sample_async/README.md) |
## How It Works
Upon the start-up, the sample application reads command line parameters and loads specified network and input images (or a
folder with images) to the Inference Engine plugin. The batch size of the network is set according to the number of read images. The batch mode is an independent attribute on the asynchronous mode. Asynchronous mode works efficiently with any batch size.
At startup, the sample application reads command line parameters and loads the specified model and input images (or a
folder with images) to the OpenVINO™ Runtime plugin. The batch size of the model is set according to the number of read images. The batch mode is an independent attribute on the asynchronous mode. Asynchronous mode works efficiently with any batch size.
Then, the sample creates an inference request object and assigns completion callback for it. In scope of the completion callback
handling the inference request is executed again.
@@ -37,14 +36,37 @@ After that, the application starts inference for the first infer request and wai
When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
Run the application with the `-h` option to see the usage instructions:
```
classification_sample_async -h
```
Usage instructions:
```
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
classification_sample_async [OPTION]
Options:
-h Print usage instructions.
-m "<path>" Required. Path to an .xml file with a trained model.
-i "<path>" Required. Path to a folder with images or path to image files: a .ubyte file for LeNet and a .bmp file for other models.
-d "<device>" Optional. Specify the target device to infer on (the list of available devices is shown below). Default value is CPU. Use "-d HETERO:<comma_separated_devices_list>" format to specify the HETERO plugin. Sample will look for a suitable plugin for the device specified.
Available target devices: <devices>
```
To run the sample, you need specify a model and image:
- you can use [public](@ref omz_models_group_public) or [Intel's](@ref omz_models_group_intel) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader).
@@ -52,82 +74,72 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
Running the application with the `-h` option yields the following usage message:
```
<path_to_sample>/classification_sample_async -h
InferenceEngine:
API version ............ <version>
Build .................. <build>
Description ....... API
classification_sample_async [OPTION]
Options:
-h Print a usage message.
-m "<path>" Required. Path to an .xml file with a trained model.
-i "<path>" Required. Path to a folder with images or path to an image files: a .ubyte file for LeNet and a .bmp file for the other networks.
-l "<absolute_path>" Required for CPU plugin custom layers. Absolute path to a shared library with the kernels implementations
Or
-c "<absolute_path>" Required for GPU, MYRIAD, HDDL custom kernels. Absolute path to the .xml config file with the kernels descriptions.
-d "<device>" Optional. Specify the target device to infer on (the list of available devices is shown below). Default value is CPU. Use "-d HETERO:<comma_separated_devices_list>" format to specify HETERO plugin. Sample will look for a suitable plugin for device specified.
-nt "<integer>" Optional. Number of top results. Default value is 10.
Available target devices: <devices>
```
Running the application with the empty list of options yields the usage message given above and an error message.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model using:
```
<path_to_sample>/classification_sample_async -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp -d GPU
omz_downloader --name googlenet-v1
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
omz_converter --name googlenet-v1
```
4. Perform inference of `dog.bmp` using `googlenet-v1` model on a `GPU`, for example:
```
classification_sample_async -m googlenet-v1.xml -i dog.bmp -d GPU
```
## Sample Output
By default the application outputs top-10 inference results for each infer request.
```
[ INFO ] InferenceEngine:
IE version ......... 2021.4.0
Build ........... 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] C:\images\car.bmp
[ INFO ] Loading Inference Engine
[ INFO ] Device info:
GPU
clDNNPlugin version ......... 2021.4.0
Build ........... 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Loading network files:
[ INFO ] C:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (749, 637) to (227, 227)
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[ INFO ] /images/dog.bmp
[ INFO ] Loading model files:
[ INFO ] /models/googlenet-v1.xml
[ INFO ] model name: GoogleNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 224, 224}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
[ INFO ] Read input images
[ INFO ] Set batch size 1
[ INFO ] model name: GoogleNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: u8
[ INFO ] input shape: {1, 224, 224, 3}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
[ INFO ] Loading model to the device GPU
[ INFO ] Create infer request
[ INFO ] Start inference (10 asynchronous executions)
[ INFO ] Start inference (asynchronous executions)
[ INFO ] Completed 1 async request execution
[ INFO ] Completed 2 async request execution
[ INFO ] Completed 3 async request execution
@@ -138,33 +150,29 @@ By default the application outputs top-10 inference results for each infer reque
[ INFO ] Completed 8 async request execution
[ INFO ] Completed 9 async request execution
[ INFO ] Completed 10 async request execution
[ INFO ] Processing output blobs
[ INFO ] Completed async requests execution
Top 10 results:
Image C:\images\car.bmp
Image /images/dog.bmp
classid probability
------- -----------
656 0.6645315
654 0.1121185
581 0.0698451
874 0.0334973
436 0.0259718
817 0.0173190
675 0.0109321
511 0.0109075
569 0.0083093
717 0.0063173
[ INFO ] Execution successful
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
156 0.8935547
218 0.0608215
215 0.0217133
219 0.0105667
212 0.0018835
217 0.0018730
152 0.0018730
157 0.0015745
154 0.0012817
220 0.0010099
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

1
samples/cpp/classification_sample_async/classification_sample_async.h
View File

@@ -52,5 +52,6 @@ static void show_usage() {
std::cout << std::endl;
std::cout << " -h " << help_message << std::endl;
std::cout << " -m \"<path>\" " << model_message << std::endl;
std::cout << " -i \"<path>\" " << image_message << std::endl;
std::cout << " -d \"<device>\" " << target_device_message << std::endl;
}

4
samples/cpp/classification_sample_async/main.cpp
View File

@@ -87,8 +87,8 @@ int main(int argc, char* argv[]) {
std::shared_ptr<ov::Model> model = core.read_model(FLAGS_m);
printInputAndOutputsInfo(*model);
OPENVINO_ASSERT(model->get_parameters().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->get_results().size() == 1, "Sample supports models with 1 output only");
OPENVINO_ASSERT(model->inputs().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->outputs().size() == 1, "Sample supports models with 1 output only");
// -------- Step 3. Configure preprocessing --------
const ov::Layout tensor_layout{"NHWC"};

114
samples/cpp/hello_classification/README.md
View File

@@ -1,39 +1,43 @@
# Hello Classification C++ Sample {#openvino_inference_engine_samples_hello_classification_README}
This sample demonstrates how to execute an inference of image classification networks like AlexNet and GoogLeNet using Synchronous Inference Request API, input auto-resize feature and support of UNICODE paths.
This sample demonstrates how to do inference of image classification models using Synchronous Inference Request API.
Models with only one input and output are supported.
Hello Classification C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
| Basic Infer Flow | `ov::Core::read_model`, `ov::Core::compile_model`, `ov::CompiledModel::create_infer_request`, `ov::InferRequest::get_input_tensor`, `ov::InferRequest::set_input_tensor`, `ov::InferRequest::get_output_tensor` | Common API to do inference: configure input and output tensors, reading model, create infer request
| Synchronous Infer | `ov::InferRequest::infer` | Do synchronous inference
| Model Operations | `ov::Model::inputs`, `ov::Model::outputs` | Managing of model
| Tensor Operations| `ov::Tensor::get_element_type`, `ov::Tensor::get_shape`, `ov::Tensor::data` | Work with storing inputs, outputs of the model, weights and biases of the layers
| Input auto-resize | `ov::preprocess::PreProcessSteps::resize`, `ov::preprocess::InputInfo::model::set_layout` | Set image of the original size as input for a model with other input size. Resize and layout conversions will be performed automatically by the corresponding plugin just before inference
| Feature | API | Description |
| :--- | :--- | :--- |
| OpenVINO Runtime Version | `ov::get_openvino_version` | Get Openvino API version |
| Basic Infer Flow | `ov::Core::read_model`, `ov::Core::compile_model`, `ov::CompiledModel::create_infer_request`, `ov::InferRequest::set_input_tensor`, `ov::InferRequest::get_output_tensor` | Common API to do inference: read and compile a model, create an infer request, configure input and output tensors |
| Synchronous Infer | `ov::InferRequest::infer` | Do synchronous inference |
| Model Operations | `ov::Model::inputs`, `ov::Model::outputs` | Get inputs and outputs of a model |
| Tensor Operations | `ov::Tensor::get_shape` | Get a tensor shape |
| Preprocessing | `ov::preprocess::InputTensorInfo::set_element_type`, `ov::preprocess::InputTensorInfo::set_layout`, `ov::preprocess::InputTensorInfo::set_spatial_static_shape`, `ov::preprocess::PreProcessSteps::resize`, `ov::preprocess::InputModelInfo::set_layout`, `ov::preprocess::OutputTensorInfo::set_element_type`, `ov::preprocess::PrePostProcessor::build` | Set image of the original size as input for a model with other input size. Resize and layout conversions are performed automatically by the corresponding plugin just before inference. |
| Options | Values |
|:--- |:---
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx)
| Validated images | The sample uses OpenCV\* to [read input image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png)
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C](../../../samples/c/hello_classification/README.md), [Python](../../../samples/python/hello_classification/README.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C](../../../samples/c/hello_classification/README.md), [Python](../../../samples/python/hello_classification/README.md) |
## How It Works
Upon the start-up, the sample application reads command line parameters, loads specified network and an image to the Inference Engine plugin.
Then, the sample creates an synchronous inference request object. When inference is done, the application outputs data to the standard output stream.
At startup, the sample application reads command line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin and performs synchronous inference. Then processes output data and write it to a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
```
hello_classification <path_to_model> <path_to_image> <device_name>
```
To run the sample, you need specify a model and image:
- you can use [public](@ref omz_models_group_public) or [Intel's](@ref omz_models_group_intel) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader).
@@ -41,28 +45,36 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model using:
```
<path_to_sample>/hello_classification <path_to_model>/alexnet.xml <path_to_image>/car.bmp GPU
omz_downloader --name googlenet-v1
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
omz_converter --name googlenet-v1
```
4. Perform inference of `car.bmp` using the `googlenet-v1` model on a `GPU`, for example:
```
hello_classification googlenet-v1.xml car.bmp GPU
```
## Sample Output
@@ -70,29 +82,41 @@ python <path_to_omz_tools>/converter.py --name alexnet
The application outputs top-10 inference results.
```
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Loading model files: /models/googlenet-v1.xml
[ INFO ] model name: GoogleNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 224, 224}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
Top 10 results:
Image /opt/intel/openvino/samples/scripts/car.png
Image /images/car.bmp
classid probability
------- -----------
656 0.6664789
654 0.1129405
581 0.0684867
874 0.0333845
436 0.0261321
817 0.0167310
675 0.0109796
511 0.0105919
569 0.0081782
717 0.0063356
This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
656 0.8139648
654 0.0550537
468 0.0178375
436 0.0165405
705 0.0111694
817 0.0105820
581 0.0086823
575 0.0077515
734 0.0064468
785 0.0043983
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

2
samples/cpp/hello_classification/main.cpp
View File

@@ -28,7 +28,7 @@ int tmain(int argc, tchar* argv[]) {
// -------- Parsing and validation of input arguments --------
if (argc != 4) {
slog::info << "Usage : " << argc << " <path_to_model> <path_to_image> <device_name>" << slog::endl;
slog::info << "Usage : " << argv[0] << " <path_to_model> <path_to_image> <device_name>" << slog::endl;
return EXIT_FAILURE;
}

117
samples/cpp/hello_nv12_input_classification/README.md
View File

@@ -1,40 +1,42 @@
# Hello NV12 Input Classification C++ Sample {#openvino_inference_engine_samples_hello_nv12_input_classification_README}
This sample demonstrates how to execute an inference of image classification networks like AlexNet with images in NV12 color format using Synchronous Inference Request API and [input reshape feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
This sample demonstrates how to execute an inference of image classification models with images in NV12 color format using Synchronous Inference Request API.
Hello NV12 Input Classification C++ Sample demonstrates how to use the NV12 automatic input pre-processing API of the Inference Engine in your applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
|OpenVINO Runtime Core Operations| `ov::Core::get_metric` | Gets general runtime metric for dedicated hardware
| Tensor Operations | `ov::Tensor::get_element_type`, `ov::Tensor::get_shape`, `ov::Tensor::data` | Work with storing inputs, outputs of the model, weights and biases of the layers
| Input in N12 color format | `ov::preprocess::InputTensorInfo::set_color_format` | Change the color format of the input data
| Model Input Reshape | `ov::Model::get_output_shape`, `ov::Model::reshape`, `ov::get_batch` | Set the batch size equal to the number of input images
| Feature | API | Description |
| :--- | :--- | :--- |
| Node Operations | `ov::Output::get_any_name` | Get a layer name |
| Infer Request Operations | `ov::InferRequest::set_tensor`, `ov::InferRequest::get_tensor` | Operate with tensors |
| Preprocessing | `ov::preprocess::InputTensorInfo::set_color_format`, `ov::preprocess::PreProcessSteps::convert_element_type`, `ov::preprocess::PreProcessSteps::convert_color` | Change the color format of the input data |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | [alexnet](@ref omz_models_model_alexnet)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx)
| Validated images | An uncompressed image in the NV12 color format - \*.yuv
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C](../../../samples/c/hello_nv12_input_classification/README.md) |
| Options | Values |
| :--- |:--- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx) |
| Validated images | An uncompressed image in the NV12 color format - \*.yuv |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C](../../../samples/c/hello_nv12_input_classification/README.md) |
## How It Works
Upon the start-up, the sample application reads command-line parameters, loads specified network and an
image in the NV12 color format to an Inference Engine plugin. Then, the sample creates an synchronous inference request object. When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output.
At startup, the sample application reads command line parameters, loads the specified model and an
image in the NV12 color format to an OpenVINO™ Runtime plugin. Then, the sample creates an synchronous inference request object. When inference is done, the application outputs data to the standard output stream. You can place labels in .labels file near the model to get pretty output.
You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
```
hello_nv12_input_classification <path_to_model> <path_to_image> <image_size> <device_name>
```
To run the sample, you need specify a model and image:
- you can use [public](@ref omz_models_group_public) or [Intel's](@ref omz_models_group_intel) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader).
@@ -46,7 +48,7 @@ as FFmpeg\* or GStreamer\*. The following command shows how to convert an ordina
uncompressed NV12 image using FFmpeg:
```sh
ffmpeg -i cat.jpg -pix_fmt nv12 cat.yuv
ffmpeg -i cat.jpg -pix_fmt nv12 car.yuv
```
> **NOTES**:
@@ -55,31 +57,38 @@ ffmpeg -i cat.jpg -pix_fmt nv12 cat.yuv
> image path. The sample expects the logical size of the image, not the buffer size. For example,
> for 640x480 BGR/RGB image the corresponding NV12 logical image size is also 640x480, whereas the
> buffer size is 640x720.
> - By default, this sample expects that network input has BGR channels order. If you trained your
> - By default, this sample expects that model input has BGR channels order. If you trained your
> model to work with RGB order, you need to reconvert your model using the Model Optimizer tool
> with `--reverse_input_channels` argument specified. For more information about the argument,
> refer to **When to Reverse Input Channels** section of
> [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install openvino-dev python package if you don't have it to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of NV12 image using `alexnet` model on a `CPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name alexnet
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
<path_to_sample>/hello_nv12_input_classification <path_to_model>/alexnet.xml <path_to_image>/cat.yuv 300x300 CPU
omz_converter --name alexnet
```
4. Perform inference of NV12 image using `alexnet` model on a `CPU`, for example:
```
hello_nv12_input_classification alexnet.xml car.yuv 300x300 CPU
```
## Sample Output
@@ -87,33 +96,41 @@ python <path_to_omz_tools>/converter.py --name alexnet
The application outputs top-10 inference results.
```
[ INFO ] Files were added: 1
[ INFO ] ./cat.yuv
Batch size is 1
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Loading model files: \models\alexnet.xml
[ INFO ] model name: AlexNet
[ INFO ] inputs
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 227, 227}
[ INFO ] outputs
[ INFO ] output name: prob
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1000}
Top 10 results:
Image ./cat.yuv
Image \images\car.yuv
classid probability
------- -----------
435 0.0917327
876 0.0817254
999 0.0693054
587 0.0437265
666 0.0389570
419 0.0328923
285 0.0303094
700 0.0299405
696 0.0216280
855 0.0203389
This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
656 0.6668988
654 0.1125269
581 0.0679280
874 0.0340229
436 0.0257744
817 0.0169367
675 0.0110199
511 0.0106134
569 0.0083373
717 0.0061734
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

4
samples/cpp/hello_nv12_input_classification/main.cpp
View File

@@ -104,8 +104,8 @@ int main(int argc, char* argv[]) {
std::shared_ptr<ov::Model> model = core.read_model(model_path);
printInputAndOutputsInfo(*model);
OPENVINO_ASSERT(model->get_parameters().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->get_results().size() == 1, "Sample supports models with 1 output only");
OPENVINO_ASSERT(model->inputs().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->outputs().size() == 1, "Sample supports models with 1 output only");
std::string input_tensor_name = model->input().get_any_name();
std::string output_tensor_name = model->output().get_any_name();

127
samples/cpp/hello_query_device/README.md
View File

@@ -1,35 +1,34 @@
# Hello Query Device C++ Sample {#openvino_inference_engine_samples_hello_query_device_README}
This sample demonstrates how to execute an query Inference Engine devices, prints their metrics and default configuration values, using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
This sample demonstrates how to execute an query OpenVINO™ Runtime devices, prints their metrics and default configuration values, using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
Hello Query Device C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
|Available Devices|`ov::Core::get_available_devices`, `ov::Core::get_metric`, `ov::Core::get_config`| Get available devices information and configuration for inference
| Feature | API | Description |
| :--- | :--- | :--- |
| Available Devices | `ov::Core::get_available_devices`, `ov::Core::get_property` | Get available devices information and configuration for inference |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/hello_query_device/README.md) |
| :--- |:---
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/hello_query_device/README.md) |
## How It Works
The sample queries all available Inference Engine devices, prints their supported metrics and plugin configuration parameters.
The sample queries all available OpenVINO™ Runtime devices, prints their supported metrics and plugin configuration parameters.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
To see quired information, run the following:
```
<path_to_sample>/hello_query_device -h
Usage : hello_query_device
hello_query_device
```
## Sample Output
@@ -37,57 +36,59 @@ Usage : hello_query_device
The application prints all available devices with their supported metrics and default values for configuration parameters:
```
Available devices:
Device: CPU
Metrics:
SUPPORTED_METRICS : [ AVAILABLE_DEVICES SUPPORTED_METRICS FULL_DEVICE_NAME OPTIMIZATION_CAPABILITIES SUPPORTED_CONFIG_KEYS RANGE_FOR_ASYNC_INFER_REQUESTS RANGE_FOR_STREAMS ]
FULL_DEVICE_NAME : Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
OPTIMIZATION_CAPABILITIES : [ FP32 FP16 INT8 BIN ]
SUPPORTED_CONFIG_KEYS : [ CPU_BIND_THREAD CPU_THREADS_NUM CPU_THROUGHPUT_STREAMS DUMP_EXEC_GRAPH_AS_DOT DYN_BATCH_ENABLED DYN_BATCH_LIMIT ENFORCE_BF16 EXCLUSIVE_ASYNC_REQUESTS PERF_COUNT ]
RANGE_FOR_ASYNC_INFER_REQUESTS : { 1, 1, 1 }
RANGE_FOR_STREAMS : { 1, 8 }
Default values for device configuration keys:
CPU_BIND_THREAD : NUMA
CPU_THREADS_NUM : 0
CPU_THROUGHPUT_STREAMS : 1
DUMP_EXEC_GRAPH_AS_DOT : ""
DYN_BATCH_ENABLED : NO
DYN_BATCH_LIMIT : 0
ENFORCE_BF16 : NO
EXCLUSIVE_ASYNC_REQUESTS : NO
PERF_COUNT : NO
Device: GPU
Metrics:
SUPPORTED_METRICS : [ AVAILABLE_DEVICES SUPPORTED_METRICS FULL_DEVICE_NAME OPTIMIZATION_CAPABILITIES SUPPORTED_CONFIG_KEYS RANGE_FOR_ASYNC_INFER_REQUESTS RANGE_FOR_STREAMS ]
FULL_DEVICE_NAME : Intel(R) UHD Graphics 620 (iGPU)
OPTIMIZATION_CAPABILITIES : [ FP32 BIN FP16 ]
SUPPORTED_CONFIG_KEYS : [ CACHE_DIR CLDNN_ENABLE_FP16_FOR_QUANTIZED_MODELS CLDNN_GRAPH_DUMPS_DIR GPU_MAX_NUM_THREADS CLDNN_MEM_POOL CLDNN_NV12_TWO_INPUTS CLDNN_PLUGIN_PRIORITY CLDNN_PLUGIN_THROTTLE CLDNN_SOURCES_DUMPS_DIR GPU_ENABLE_LOOP_UNROLLING CONFIG_FILE DEVICE_ID DUMP_KERNELS DYN_BATCH_ENABLED EXCLUSIVE_ASYNC_REQUESTS GPU_THROUGHPUT_STREAMS PERF_COUNT TUNING_FILE TUNING_MODE ]
RANGE_FOR_ASYNC_INFER_REQUESTS : { 1, 2, 1 }
RANGE_FOR_STREAMS : { 1, 2 }
Default values for device configuration keys:
CACHE_DIR : ""
CLDNN_ENABLE_FP16_FOR_QUANTIZED_MODELS : YES
CLDNN_GRAPH_DUMPS_DIR : ""
CLDNN_MEM_POOL : YES
CLDNN_NV12_TWO_INPUTS : NO
CLDNN_PLUGIN_PRIORITY : 0
CLDNN_PLUGIN_THROTTLE : 0
CLDNN_SOURCES_DUMPS_DIR : ""
GPU_MAX_NUM_THREADS : 8
GPU_ENABLE_LOOP_UNROLLING : YES
CONFIG_FILE : ""
DEVICE_ID : ""
DUMP_KERNELS : NO
DYN_BATCH_ENABLED : NO
EXCLUSIVE_ASYNC_REQUESTS : NO
GPU_THROUGHPUT_STREAMS : 1
PERF_COUNT : NO
TUNING_FILE : ""
TUNING_MODE : TUNING_DISABLED
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Available devices:
[ INFO ] CPU
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES : [ ]
[ INFO ] FULL_DEVICE_NAME : Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
[ INFO ] OPTIMIZATION_CAPABILITIES : [ FP32 FP16 INT8 BIN ]
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS : { 1, 1, 1 }
[ INFO ] RANGE_FOR_STREAMS : { 1, 8 }
[ INFO ] IMPORT_EXPORT_SUPPORT : true
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CACHE_DIR : ""
[ INFO ] CPU_BIND_THREAD : NO
[ INFO ] CPU_THREADS_NUM : 0
[ INFO ] CPU_THROUGHPUT_STREAMS : 1
[ INFO ] DUMP_EXEC_GRAPH_AS_DOT : ""
[ INFO ] DYN_BATCH_ENABLED : NO
[ INFO ] DYN_BATCH_LIMIT : 0
[ INFO ] ENFORCE_BF16 : NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS : NO
[ INFO ] PERFORMANCE_HINT : ""
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS : 0
[ INFO ] PERF_COUNT : NO
[ INFO ]
[ INFO ] GNA
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES : [ GNA_SW_EXACT ]
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS : 1
[ INFO ] FULL_DEVICE_NAME : GNA_SW_EXACT
[ INFO ] GNA_LIBRARY_FULL_VERSION : 3.0.0.1455
[ INFO ] IMPORT_EXPORT_SUPPORT : true
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS : NO
[ INFO ] GNA_COMPACT_MODE : YES
[ INFO ] GNA_COMPILE_TARGET : ""
[ INFO ] GNA_DEVICE_MODE : GNA_SW_EXACT
[ INFO ] GNA_EXEC_TARGET : ""
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE : ""
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE_GENERATION : ""
[ INFO ] GNA_LIB_N_THREADS : 1
[ INFO ] GNA_PRECISION : I16
[ INFO ] GNA_PWL_MAX_ERROR_PERCENT : 1.000000
[ INFO ] GNA_PWL_UNIFORM_DESIGN : NO
[ INFO ] GNA_SCALE_FACTOR : 1.000000
[ INFO ] GNA_SCALE_FACTOR_0 : 1.000000
[ INFO ] LOG_LEVEL : LOG_NONE
[ INFO ] PERF_COUNT : NO
[ INFO ] SINGLE_THREAD : YES
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)

110
samples/cpp/hello_reshape_ssd/README.md
View File

@@ -1,24 +1,25 @@
# Hello Reshape SSD C++ Sample {#openvino_inference_engine_samples_hello_reshape_ssd_README}
This sample demonstrates how to execute an inference of object detection networks like SSD-VGG using Synchronous Inference Request API, [input reshape feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
This sample demonstrates how to do synchronous inference of object detection models using [input reshape feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
Models with only one input and output are supported.
Hello Reshape SSD C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
| Model Operations | `ov::Core::read_model`, `ov::Core::compile_model` | Managing of model
| Model Input Reshape | `ov::Model::reshape`| Resize model to match image sizes and given batch
| Tensor Operations | `ov::Tensor::get_element_type`, `ov::Tensor::get_shape`, `ov::Tensor::data` | Work with storing inputs, outputs of the model, weights and biases of the layers
| Feature | API | Description |
| :--- | :--- | :--- |
| Node operations | `ov::Node::get_type_info`, `ngraph::op::DetectionOutput::get_type_info_static`, `ov::Output::get_any_name`, `ov::Output::get_shape` | Get a node info |
| Model Operations | `ov::Model::get_ops`, `ov::Model::reshape` | Get model nodes, reshape input |
| Tensor Operations | `ov::Tensor::data` | Get a tensor data |
| Preprocessing | `ov::preprocess::PreProcessSteps::convert_element_type`, `ov::preprocess::PreProcessSteps::convert_layout` | Model input preprocessing |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | [person-detection-retail-0013](@ref omz_models_model_person_detection_retail_0013)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx)
| Validated images | The sample uses OpenCV\* to [read input image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png)
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/hello_reshape_ssd/README.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | [person-detection-retail-0013](@ref omz_models_model_person_detection_retail_0013) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin), ONNX (\*.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/hello_reshape_ssd/README.md) |
## How It Works
@@ -26,14 +27,18 @@ Upon the start-up the sample application reads command line parameters, loads sp
Engine plugin. Then, the sample creates an synchronous inference request object. When inference is done, the application creates output image and output data to the standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
```
hello_reshape_ssd <path_to_model> <path_to_image> <device>
```
To run the sample, you need specify a model and image:
- you can use [public](@ref omz_models_group_public) or [Intel's](@ref omz_models_group_intel) pre-trained models from the Open Model Zoo. The models can be downloaded using the [Model Downloader](@ref omz_tools_downloader).
@@ -41,34 +46,36 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (\*.onnx) that do not require preprocessing.
You can use the following command to do inference on CPU of an image using a trained SSD network:
```
<path_to_sample>/hello_reshape_ssd <path_to_model> <path_to_image> <device> <batch>
```
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name person-detection-retail-0013
```
2. `person-detection-retail-0013` model does not need to be converted, because it is already in necessary format, so you can skip this step. If you want to use a other model that is not in the Inference Engine IR or ONNX format, you can convert it using the model converter script:
1. Install openvino-dev python package if you don't have it to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name <model_name>
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `person_detection.png` using `person-detection-retail-0013` model on a `GPU`, for example:
2. Download a pre-trained model using:
```
<path_to_sample>/hello_reshape_ssd <path_to_model>/person-detection-retail-0013.xml <path_to_image>/person_detection.png GPU 1
omz_downloader --name person-detection-retail-0013
```
3. `person-detection-retail-0013` does not need to be converted, because it is already in necessary format, so you can skip this step. If you want to use another model that is not in the IR or ONNX format, you can convert it using the model converter script:
```
omz_converter --name <model_name>
```
4. Perform inference of `person_detection.bmp` using `person-detection-retail-0013` model on a `GPU`, for example:
```
hello_reshape_ssd person-detection-retail-0013.xml person_detection.bmp GPU
```
## Sample Output
@@ -78,33 +85,38 @@ of the detected objects along with the respective confidence values and the coor
rectangles to the standard output stream.
```
[ INFO ] Loading model files: C:\temp\models\public\ssd_mobilenet_v1_fpn_coco\FP16\ssd_mobilenet_v1_fpn_coco.xml
[ INFO ] model name: ssd_mobilenet_v1_fpn_coco
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Loading model files: \models\person-detection-retail-0013.xml
[ INFO ] model name: ResMobNet_v4 (LReLU) with single SSD head
[ INFO ] inputs
[ INFO ] input name: image_tensor
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 640, 640}
[ INFO ] input shape: {1, 3, 320, 544}
[ INFO ] outputs
[ INFO ] output name: DetectionOutput
[ INFO ] output name: detection_out
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1, 100, 7}
Reshape network to the image size = [512x512] with batch = 1
[ INFO ] model name: ssd_mobilenet_v1_fpn_coco
[ INFO ] output shape: {1, 1, 200, 7}
Reshape network to the image size = [960x1699]
[ INFO ] model name: ResMobNet_v4 (LReLU) with single SSD head
[ INFO ] inputs
[ INFO ] input name: image_tensor
[ INFO ] input name: data
[ INFO ] input type: f32
[ INFO ] input shape: {1, 3, 512, 512}
[ INFO ] input shape: {1, 3, 960, 1699}
[ INFO ] outputs
[ INFO ] output name: DetectionOutput
[ INFO ] output name: detection_out
[ INFO ] output type: f32
[ INFO ] output shape: {1, 1, 100, 7}
[0,18] element, prob = 0.781129 (109,52)-(342,441) batch id = 0
The resulting image was saved in the file: hello_reshape_ssd_batch_0.bmp
[ INFO ] output shape: {1, 1, 200, 7}
[0,1] element, prob = 0.716309, (852,187)-(983,520)
The resulting image was saved in the file: hello_reshape_ssd_output.bmp
This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

4
samples/cpp/hello_reshape_ssd/main.cpp
View File

@@ -46,8 +46,8 @@ int main(int argc, char* argv[]) {
printInputAndOutputsInfo(*model);
// Step 3. Validate model inputs and outputs
OPENVINO_ASSERT(model->get_parameters().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->get_results().size() == 1, "Sample supports models with 1 output only");
OPENVINO_ASSERT(model->inputs().size() == 1, "Sample supports models with 1 input only");
OPENVINO_ASSERT(model->outputs().size() == 1, "Sample supports models with 1 output only");
// SSD has an additional post-processing DetectionOutput layer that simplifies output filtering,
// try to find it.

210
samples/cpp/ngraph_function_creation_sample/README.md
View File

@@ -1,82 +1,62 @@
# nGraph Function Creation C++ Sample {#openvino_inference_engine_samples_ngraph_function_creation_sample_README}
This sample demonstrates how to execute an synchronous inference using [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) to create a network, which uses weights from LeNet classification network, which is known to work well on digit classification tasks.
This sample demonstrates how to execute an synchronous inference using [model](../../../docs/OV_Runtime_UG/model_representation.md) built on the fly which uses weights from LeNet classification model, which is known to work well on digit classification tasks.
The sample supports only single-channel [MNIST database](http://yann.lecun.com/exdb/mnist) images as an input.
You do not need an XML file to create a model. The API of ngraph::Function allows creating a model on the fly from the source code.
You do not need an XML file to create a network. The API of ngraph::Function allows to create a network on the fly from the source code.
The following C++ API is used in the application:
nGraph Function Creation C++ Sample demonstrates the following Inference Engine API in your applications:
| Feature | API | Description |
| :--- | :--- | :--- |
| OpenVINO Runtime Info | `ov::Core::get_versions` | Get device plugins versions |
| Shape Operations | `ov::Output::get_shape`, `ov::Shape::size`, `ov::shape_size`| Operate with shape |
| Tensor Operations | `ov::Tensor::get_byte_size`, `ov::Tensor:data` | Get tensor byte size and its data |
| Model Operations | `ov::set_batch` | Operate with model batch size |
| Infer Request Operations | `ov::InferRequest::get_input_tensor` | Get a input tensor |
| nGraph Functions | `ov::opset8::Parameter`, `ov::Node::output`, `ov::opset8::Constant`, `ov::opset8::Convolution`, `ov::opset8::Add`, `ov::opset1::MaxPool`, `ov::opset8::Reshape`, `ov::opset8::MatMul`, `ov::opset8::Relu`, `ov::opset8::Softmax`, `ov::descriptor::Tensor::set_names`, `ov::opset8::Result`, `ov::Model`, `ov::ParameterVector::vector` | Used to construct an nGraph function |
| Feature | API | Description |
|:--- |:--- |:---
|OpenVINO Runtime Version| `ov::get_openvino_versio` | Get Openvino API version
|Available Devices|`ov::Core::get_available_devices`| Get version information of the devices for inference
| Model Operations | `ov::set_batch`, `ov::get_batch` | Managing of model, operate with its batch size. Setting batch size using input image count.
|nGraph Functions| `ov::op`, `ov::Node`, `ov::Shape::Shape`, `ov::Strides::Strides`, `ov::CoordinateDiff::CoordinateDiff`, `ov::Node::set_friendly_name`, `ov::shape_size`, `ov::ParameterVector::vector` | Illustrates how to construct an nGraph function
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | LeNet
| Model Format | Network weights file (\*.bin)
| Validated images | single-channel `MNIST ubyte` images
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/ngraph_function_creation_sample/README.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | LeNet |
| Model Format | model weights file (\*.bin) |
| Validated images | single-channel `MNIST ubyte` images |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [Python](../../../samples/python/ngraph_function_creation_sample/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, creates a network using the [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file, loads the network and image(s) to the Inference Engine plugin, performs synchronous inference and processes output data, logging each step in a standard output stream. You can place labels in .labels file near the model to get pretty output.
At startup, the sample application does the following:
- Reads command line parameters
- [Build a Model](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file
- Loads the model and input data to the OpenVINO™ Runtime plugin
- Performs synchronous inference and processes output data, logging each step in a standard output stream
You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
You can see the explicit description of each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
To run the sample, you need specify a model wights and MNIST ubyte image:
- you can use LeNet model weights in the sample folder: `lenet.bin` with FP32 weights file
- you can use images from the media files collection available at https://storage.openvinotoolkit.org/data/test_data.
```
ngraph_function_creation_sample <path_to_lenet_weights> <device>
```
> **NOTES**:
>
> - you can use LeNet model weights in the sample folder: `lenet.bin` with FP32 weights file
> - The `lenet.bin` with FP32 weights file was generated by the [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) tool from the public LeNet model with the `--input_shape [64,1,28,28]` parameter specified.
>
> The original model is available in the [Caffe* repository](https://github.com/BVLC/caffe/tree/master/examples/mnist) on GitHub\*.
Running the application with the `-h` option yields the following usage message:
```
ngraph_function_creation_sample -h
[ INFO ] InferenceEngine:
API version ............<version>
Build ..................<build>
Description ....... API
[ INFO ] Parsing input parameters
ngraph_function_creation_sample [OPTION]
Options:
-h Print a usage message.
-m "<path>" Required. Path to a .bin file with weights for the trained model.
-i "<path>" Required. Path to a folder with images or path to image files. Support ubyte files only.
-d "<device>" Optional. Specify the target device to infer on (the list of available devices is shown below). Default value is CPU. Use "-d HETERO:<comma_separated_devices_list>" format to specify HETERO plugin. Sample will look for a suitable plugin for device specified.
-nt "<integer>" Number of top results. The default value is 10.
Available target devices: <devices>
```
Running the application with the empty list of options yields the usage message given above and an error message.
You can do inference of an image using a pre-trained model on a GPU using the following command:
```
<path_to_sample>/ngraph_function_creation_sample -m <path_to_weights_file>/lenet.bin -i <path_to_image> -d GPU
ngraph_function_creation_sample lenet.bin GPU
```
## Sample Output
@@ -84,46 +64,102 @@ You can do inference of an image using a pre-trained model on a GPU using the fo
The sample application logs each step in a standard output stream and outputs top-10 inference results.
```
ngraph_function_creation_sample.exe -m lenet.bin -i 7-ubyte
[ INFO ] InferenceEngine:
API version ............ <version>
Build .................. <build>
Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] 7-ubyte
[ INFO ] Loading Inference Engine
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Device info:
CPU
openvino_intel_cpu_plugin version ......... <version>
Build ........... <build>
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Checking that the outputs are as the sample expects
[ INFO ] Loading model to the device
[ INFO ] GPU
[ INFO ] Intel GPU plugin version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ]
[ INFO ] Create model from weights: lenet.bin
[ INFO ] model name: lenet
[ INFO ] inputs
[ INFO ] input name: NONE
[ INFO ] input type: f32
[ INFO ] input shape: {64, 1, 28, 28}
[ INFO ] outputs
[ INFO ] output name: output_tensor
[ INFO ] output type: f32
[ INFO ] output shape: {64, 10}
[ INFO ] Batch size is 10
[ INFO ] model name: lenet
[ INFO ] inputs
[ INFO ] input name: NONE
[ INFO ] input type: u8
[ INFO ] input shape: {10, 28, 28, 1}
[ INFO ] outputs
[ INFO ] output name: output_tensor
[ INFO ] output type: f32
[ INFO ] output shape: {10, 10}
[ INFO ] Compiling a model for the GPU device
[ INFO ] Create infer request
[ INFO ] Start inference
[ INFO ] Processing output blobs
[ INFO ] Combine images in batch and set to input tensor
[ INFO ] Start sync inference
[ INFO ] Processing output tensor
Top 10 results:
Top 1 results:
Image 7-ubyte
Image 0
classid probability
------- -----------
7 1.0000000
4 0.0000000
8 0.0000000
9 0.0000000
5 0.0000000
3 0.0000000
1 0.0000000
0 0.0000000
2 0.0000000
6 0.0000000
classid probability label
------- ----------- -----
0 1.0000000 0
[ INFO ] This sample is an API example, for performance measurements, use the dedicated benchmark_app tool
Image 1
classid probability label
------- ----------- -----
1 1.0000000 1
Image 2
classid probability label
------- ----------- -----
2 1.0000000 2
Image 3
classid probability label
------- ----------- -----
3 1.0000000 3
Image 4
classid probability label
------- ----------- -----
4 1.0000000 4
Image 5
classid probability label
------- ----------- -----
5 1.0000000 5
Image 6
classid probability label
------- ----------- -----
6 1.0000000 6
Image 7
classid probability label
------- ----------- -----
7 1.0000000 7
Image 8
classid probability label
------- ----------- -----
8 1.0000000 8
Image 9
classid probability label
------- ----------- -----
9 1.0000000 9
```
@@ -146,6 +182,6 @@ classid probability
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

211
samples/cpp/speech_sample/README.md
View File

@@ -4,53 +4,50 @@ This sample demonstrates how to execute an Asynchronous Inference of acoustic mo
The sample works with Kaldi ARK or Numpy* uncompressed NPZ files, so it does not cover an end-to-end speech recognition scenario (speech to text), requiring additional preprocessing (feature extraction) to get a feature vector from a speech signal, as well as postprocessing (decoding) to produce text from scores.
Automatic Speech Recognition C++ sample application demonstrates how to use the following Inference Engine C++ API in applications:
The following C++ API is used in the application:
| Feature | API | Description |
|:--- |:--- |:---
|OpenVINO Runtime Version| `ov::get_openvino_versio` | Get Openvino API version
|Available Devices|`ov::Core::get_available_devices`| Get version information of the devices for inference
| Model Operations | `ov::set_batch`, `ov::get_batch` | Managing of model, operate with its batch size. Setting batch size using input image count.
|Model Operations|`ov::Model::add_output`| Change names of output layers in the model
|Import Model|`ov::CompiledModel::export_model`,`ov::Core::import_model`| Creates a CompiledModel from a previously exported model
|Asynchronous Infer| `ov::InferRequest::start_async`, `ov::InferRequest::wait`| Do asynchronous inference and waits until inference result becomes available
|InferRequest Operations|`ov::InferRequest::query_state`, `ov::VariableState::reset`| Gets and resets state control interface for given CompiledModel
|InferRequest Operations|`ov::InferRequest::get_profiling_info`| Get profiling info for infer request
| Feature | API | Description |
| :--- | :--- | :--- |
| Available Devices | `ov::Core::get_available_devices`, `ov::Core::get_property` | Get information of the devices for inference |
| Import/Export Model | `ov::Core::import_model`, `ov::CompiledModel::export_model` | The GNA plugin supports loading and saving of the GNA-optimized model |
| Model Operations | `ov::set_batch`, `ov::Model::add_output`, `ov::CompiledModel::inputs`, `ov::CompiledModel::outputs` | Managing of model: configure batch_size, input and output tensors |
| Node Operations | `ov::OutputVector::size`, `ov::Output::get_shape` | Get node shape |
| Asynchronous Infer | `ov::InferRequest::start_async`, `ov::InferRequest::wait` | Do asynchronous inference and waits until inference result becomes available |
| InferRequest Operations | `ov::InferRequest::query_state`, `ov::VariableState::reset` | Gets and resets CompiledModel state control |
| Tensor Operations | `ov::Tensor::get_size`, `ov::Tensor::data`, `ov::InferRequest::get_tensor` | Get a tensor, its size and data |
| Profiling | `ov::InferRequest::get_profiling_info` | Get infer request profiling info |
Basic Inference Engine API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification C++ sample](../hello_classification/README.md).
| Options | Values |
|:--- |:---
| Validated Models | Acoustic model based on Kaldi\* neural networks (see [Model Preparation](#model-preparation) section)
| Model Format | Inference Engine Intermediate Representation (\*.xml + \*.bin)
| Supported devices | See [Execution Modes](#execution-modes) section below and [List Supported Devices](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Options | Values |
| :--- | :--- |
| Validated Models | Acoustic model based on Kaldi\* neural networks (see [Model Preparation](#model-preparation) section) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (\*.xml + \*.bin) |
| Supported devices | See [Execution Modes](#execution-modes) section below and [List Supported Devices](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
## How It Works
On startup, the application reads command line parameters, loads a specified model and input data to the Inference Engine plugin, performs synchronous inference on all speech utterances stored in the input file. Context-windowed speech frames are processed in batches of 1-8
frames according to the `-bs` parameter. Batching across utterances is not supported by this sample. When inference is done, the application creates an output file. If the `-r` option is given, error
statistics are provided for each speech utterance as shown above.
At startup, the sample application reads command-line parameters, loads a specified model and input data to the OpenVINO™ Runtime plugin, performs inference on all speech utterances stored in the input file(s), logging each step in a standard output stream.
If the `-r` option is given, error statistics are provided for each speech utterance as shown above.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
### GNA-specific details
#### Quantization
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA Inference Engine plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA OpenVINO™ Runtime plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
Several parameters control neural network quantization. The `-q` flag determines the quantization mode.
Three modes are supported:
- *static* - The first
utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used
for all subsequent inputs. The neural network is quantized to accommodate the scaled input dynamic range.
- *dynamic* - The scale factor for each input batch is computed
just before inference on that batch. The input and network are (re)quantized on the fly using an efficient procedure.
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural network is quantized to accommodate the scaled input dynamic range.
- *dynamic* - The scale factor for each input batch is computed just before inference on that batch. The input and network are (re)quantized on the fly using an efficient procedure.
- *user-defined* - The user may specify a scale factor via the `-sf` flag that will be used for static quantization.
The `-qb` flag provides a hint to the GNA plugin regarding the preferred target weight resolution for all layers. For example, when `-qb 8` is specified, the plugin will use 8-bit weights wherever possible in the
The `-qb` flag provides a hint to the GNA plugin regarding the preferred target weight resolution for all layers. For example, when `-qb 8` is specified, the plugin will use 8-bit weights wherever possible in the
network.
> **NOTE**:
>
> - It is not always possible to use 8-bit weights due to GNA hardware limitations. For example, convolutional layers always use 16-bit weights (GNA hardware version 1 and 2). This limitation will be removed in GNA hardware version 3 and higher.
@@ -80,131 +77,129 @@ In addition to performing inference directly from a GNA model file, these combin
## Building
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in Inference Engine Samples guide.
To build the sample, please use instructions available at [Build the Sample Applications](../../../docs/OV_Runtime_UG/Samples_Overview.md) section in OpenVINO™ Toolkit Samples guide.
## Running
Running the application with the `-h` option yields the following usage message:
Run the application with the -h option to see the usage message:
```sh
./speech_sample -h
[ INFO ] InferenceEngine:
API version ............ <version>
Build .................. <number>
Description ....... API
```
speech_sample -h
```
Usage message:
```
[ INFO ] OpenVINO Runtime version ......... <version>
[ INFO ] Build ........... <build>
[ INFO ]
[ INFO ] Parsing input parameters
speech_sample [OPTION]
Options:
-h Print a usage message.
-i "<path>" Required. Paths to input files. Example of usage: <file1.ark,file2.ark> or <file.ark> or <file.npz>.
-m "<path>" Required. Path to an .xml file with a trained model (required if -rg is missing).
-o "<path>" Optional. Output file name to save scores. Example of usage: <output.ark> or <output.npz>
-d "<device>" Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32, GNA_SW_EXACT and HETERO with combination of GNA
as the primary device and CPU as a secondary (e.g. HETERO:GNA,CPU) are supported. The list of available devices is shown below. The sample will look for a suitable plugin for device specified.
-pc Optional. Enables per-layer performance report.
-q "<mode>" Optional. Input quantization mode: static (default), dynamic, or user (use with -sf).
-qb "<integer>" Optional. Weight bits for quantization: 8 or 16 (default)
-sf "<double>" Optional. User-specified input scale factor for quantization (use with -q user). If the network contains multiple inputs, provide scale factors by separating them with commas.
-bs "<integer>" Optional. Batch size 1-8 (default 1)
-r "<path>" Optional. Read referefile and compare scores. Example of usage: <reference.ark> or <reference.npz>
-rg "<path>" Read GNA model from file using path/filename provided (required if -m is missing).
-wg "<path>" Optional. Write GNA model to file using path/filename provided.
-we "<path>" Optional. Write GNA embedded model to file using path/filename provided.
-nthreads "<integer>" Optional. Number of threads to use for concurrent async inference requests on the GNA.
-cw_l "<integer>" Optional. Number of frames for left context windows (default is 0). Works only with context window networks.
If you use the cw_l or cw_r flag, then batch size and nthreads arguments are ignored.
-cw_r "<integer>" Optional. Number of frames for right context windows (default is 0). Works only with context window networks.
If you use the cw_r or cw_l flag, then batch size and nthreads arguments are ignored.
-oname "<string>" Optional. Layer names for output blobs. The names are separated with "," Example: Output1:port,Output2:port
-iname "<string>" Optional. Layer names for input blobs. The names are separated with "," Example: Input1,Input2
-pwl_me "<double>" Optional. The maximum percent of error for PWL function.The value must be in <0, 100> range. The default value is 1.0.
Available target devices: <devices>
-h Print a usage message.
-i "<path>" Required. Paths to input files. Example of usage: <file1.ark,file2.ark> or <file.ark> or <file.npz>.
-m "<path>" Required. Path to an .xml file with a trained model (required if -rg is missing).
-o "<path>" Optional. Output file name to save scores. Example of usage: <output.ark> or <output.npz>
-d "<device>" Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_HW_WITH_SW_FBACK, GNA_SW_FP32, GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g. HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified.
-pc Optional. Enables per-layer performance report.
-q "<mode>" Optional. Input quantization mode: static (default), dynamic, or user (use with -sf).
-qb "<integer>" Optional. Weight bits for quantization: 8 or 16 (default)
-sf "<double>" Optional. User-specified input scale factor for quantization (use with -q user). If the network contains multiple inputs, provide scale factors by separating them with commas.
-bs "<integer>" Optional. Batch size 1-8 (default 1)
-r "<path>" Optional. Read reference score file and compare scores. Example of usage: <reference.ark> or <reference.npz>
-rg "<path>" Read GNA model from file using path/filename provided (required if -m is missing).
-wg "<path>" Optional. Write GNA model to file using path/filename provided.
-we "<path>" Optional. Write GNA embedded model to file using path/filename provided.
-cw_l "<integer>" Optional. Number of frames for left context windows (default is 0). Works only with context window networks. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-cw_r "<integer>" Optional. Number of frames for right context windows (default is 0). Works only with context window networks. If you use the cw_r or cw_l flag, then batch size argument is ignored.
-oname "<string>" Optional. Layer names for output blobs. The names are separated with "," Example: Output1:port,Output2:port
-iname "<string>" Optional. Layer names for input blobs. The names are separated with "," Example: Input1,Input2
-pwl_me "<double>" Optional. The maximum percent of error for PWL function.The value must be in <0, 100> range. The default value is 1.0.
-exec_target "<string>" Optional. Specify GNA execution target generation. May be one of GNA_TARGET_2_0, GNA_TARGET_3_0. By default, generation corresponds to the GNA HW available in the system or the latest fully supported generation by the software. See the GNA Plugin's GNA_EXEC_TARGET config option description.
-compile_target "<string>" Optional. Specify GNA compile target generation. May be one of GNA_TARGET_2_0, GNA_TARGET_3_0. By default, generation corresponds to the GNA HW available in the system or the latest fully supported generation by the software. See the GNA Plugin's GNA_COMPILE_TARGET config option description.
Available target devices: CPU GNA GPU
```
Running the application with the empty list of options yields the usage message given above and an error message.
### Model Preparation
Run Model Optimizer to convert a Kaldi nnet1 or nnet2 neural network to Inference Engine Intermediate Representation format:
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
```
mo --framework kaldi --input_model wsj_dnn5b.nnet --counts wsj_dnn5b.counts --remove_output_softmax --output_dir <OUTPUT_MODEL_DIR>
Assuming that the Kaldi-trained neural network, `wsj_dnn5b.nnet`, and Kaldi class counts file, `wsj_dnn5b.counts`, are in the working directory this produces the IR network consisting of `wsj_dnn5b.xml` and `wsj_dnn5b.bin`.
```
The following pre-trained models are available:
- wsj\_dnn5b\_smbr
- rm\_lstm4f
- rm\_cnn4a\_smbr
- wsj_dnn5b_smbr
- rm_lstm4f
- rm_cnn4a_smbr
All of them can be downloaded from [https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/](https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/).
All of them can be downloaded from [https://storage.openvinotoolkit.org/models_contrib/speech/2021.2](https://storage.openvinotoolkit.org/models_contrib/speech/2021.2).
### Speech Inference
Once the IR is created, you can use the following command to do inference on Intel® Processors with the GNA co-processor (or emulation library):
Once the IR is created, you can do inference on Intel® Processors with the GNA co-processor (or emulation library):
```sh
<path_to_sample>/speech_sample -d GNA_AUTO -bs 2 -i <path_to_ark>/dev93_10.ark -m <path_to_model>/wsj_dnn5b.xml -o scores.ark -r <path_to_ark>/dev93_scores_10.ark
```
speech_sample -m wsj_dnn5b.xml -i dev93_10.ark -r dev93_scores_10.ark -d GNA_AUTO -o result.ark
```
Here, the floating point Kaldi-generated reference neural network scores (`dev93_scores_10.ark`) corresponding to the input feature file (`dev93_10.ark`) are assumed to be available
for comparison.
All of them can be downloaded from [https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/wsj_dnn5b_smbr](https://storage.openvinotoolkit.org/models_contrib/speech/2021.2/wsj_dnn5b_smbr). Inference Engine Intermediate Representation `wsj_dnn5b.xml` file was generated in the previous [Model Preparation](#model-preparation) section.
Here, the floating point Kaldi-generated reference neural network scores (`dev93_scores_10.ark`) corresponding to the input feature file (`dev93_10.ark`) are assumed to be available for comparison.
> **NOTES**:
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample supports input and output in numpy file format (.npz)
## Sample Output
The acoustic log likelihood sequences for all utterances are stored in the file. Example `scores.ark` or `scores.npz`. If the `-r` option is used, a report on the statistical score error is generated for each utterance such as
the following:
The sample application logs each step in a standard output stream.
```sh
./speech_sample -d GNA_AUTO -bs 2 -i dev93_10.ark -m wsj_dnn5b.xml -o scores.ark -r dev93_scores_10.ark
[ INFO ] InferenceEngine:
API version ............ <version>
Build .................. <build>
Description ....... API
```
[ INFO ] OpenVINO runtime: OpenVINO Runtime version ......... 2022.1.0
[ INFO ] Build ........... 2022.1.0-6311-a90bb1ff017
[ INFO ]
[ INFO ] Parsing input parameters
[ INFO ] No extensions provided
[ INFO ] Loading Inference Engine
[ INFO ] Device info:
GNA
GNAPlugin version ......... <version>
Build ........... <build>
[ INFO ] Loading network files
[ INFO ] Batch size is 2
[ INFO ] Loading model files:
[ INFO ] \test_data\models\wsj_dnn5b_smbr_fp32\wsj_dnn5b_smbr_fp32.xml
[ INFO ] Using scale factor of 2175.43 calculated from first utterance.
[ INFO ] Model loading time 0.0034 ms
[ INFO ] Loading model to the device GNA_AUTO
[ INFO ] Loading model to the device
[ INFO ] Model loading time 124.21 ms
[ INFO ] Number scores per frame : 6850
[ INFO ] Number scores per frame : 3425
Utterance 0:
Total time in Infer (HW and SW): 2291.64 ms
Total time in Infer (HW and SW): 5687.53 ms
Frames in utterance: 1294 frames
Average Infer time per frame: 1.77097 ms
Average Infer time per frame: 4.39531 ms
max error: 0.705184
avg error: 0.0448388
avg rms error: 0.0287733
avg rms error: 0.0574098
stdev error: 0.0371649
End of Utterance 0
[ INFO ] Number scores per frame : 6850
Utterance X:
[ INFO ] Number scores per frame : 3425
Utterance 1:
Total time in Infer (HW and SW): 4341.34 ms
Frames in utterance: 1005 frames
Average Infer time per frame: 4.31974 ms
max error: 0.757597
avg error: 0.0452166
avg rms error: 0.0578436
stdev error: 0.0372769
End of Utterance 1
...
End of Utterance X
[ INFO ] Execution successful
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
[ INFO ] Execution successful
```
## Use of Sample in Kaldi* Speech Recognition Pipeline
@@ -223,7 +218,7 @@ nnet-forward --use-gpu=no final.feature_transform "ark,s,cs:copy-feats scp:feats
```sh
./speech_sample -d GNA_AUTO -bs 8 -i feat.ark -m wsj_dnn5b.xml -o scores.ark
```
Inference Engine Intermediate Representation `wsj_dnn5b.xml` file was generated in the previous [Model Preparation](#model-preparation) section.
OpenVINO™ toolkit Intermediate Representation `wsj_dnn5b.xml` file was generated in the previous [Model Preparation](#model-preparation) section.
3. Run the Kaldi decoder to produce n-best text hypotheses and select most likely text given the WFST (`HCLG.fst`), vocabulary (`words.txt`), and TID/PID mapping (`final.mdl`):
@@ -241,7 +236,7 @@ All of mentioned files can be downloaded from [https://storage.openvinotoolkit.o
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)

156
samples/python/classification_sample_async/README.md
View File

@@ -1,45 +1,43 @@
# Image Classification Async Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_classification_sample_async_README}
This sample demonstrates how to do inference of image classification networks using Asynchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Asynchronous Inference Request API.
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------------- | :-------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Asynchronous Infer | [InferRequest.async_infer], [InferRequest.wait], [Blob.buffer] | Do asynchronous inference |
| Custom Extension Kernels | [IECore.add_extension], [IECore.set_config] | Load extension library and config to the device |
| Feature | API | Description |
| :----------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------ |
| Asynchronous Infer | [openvino.runtime.AsyncInferQueue], [openvino.runtime.AsyncInferQueue.set_callback], [openvino.runtime.AsyncInferQueue.start_async], [openvino.runtime.AsyncInferQueue.wait_all], [openvino.runtime.InferRequest.results] | Do asynchronous inference |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/classification_sample_async/README.md) |
| Options | Values |
| :------------------------- | :----------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/classification_sample_async/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image(s) to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/classification_sample_async.py -h
python classification_sample_async.py -h
```
Usage message:
```
usage: classification_sample_async.py [-h] -m MODEL -i INPUT [INPUT ...]
[-l EXTENSION] [-c CONFIG] [-d DEVICE]
[--labels LABELS] [-nt NUMBER_TOP]
[-d DEVICE]
Options:
-h, --help Show this help message and exit.
@@ -48,22 +46,11 @@ Options:
model.
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Path to an image file(s).
-l EXTENSION, --extension EXTENSION
Optional. Required by the CPU Plugin for executing the
custom operation on a CPU. Absolute path to a shared
library with the kernels implementations.
-c CONFIG, --config CONFIG
Optional. Required by GPU or VPU Plugins for the
custom operation kernel. Absolute path to operation
description file (.xml).
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
```
To run the sample, you need specify a model and image:
@@ -73,28 +60,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` and `cat.jpg` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name alexnet
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/classification_sample_async.py -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp <path_to_image>/cat.jpg -d GPU
omz_converter --name alexnet
```
4. Perform inference of `banana.jpg` and `car.bmp` using the `alexnet` model on a `GPU`, for example:
```
python classification_sample_async.py -m alexnet.xml -i banana.jpg car.bmp -d GPU
```
## Sample Output
@@ -102,66 +96,52 @@ python <path_to_sample>/classification_sample_async.py -m <path_to_model>/alexne
The sample application logs each step in a standard output stream and outputs top-10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/alexnet.xml
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\car.bmp is resized from (637, 749) to (227, 227)
[ WARNING ] Image c:\images\cat.jpg is resized from (300, 300) to (227, 227)
[ INFO ] Starting inference in asynchronous mode
[ INFO ] Infer request 0 returned 0
[ INFO ] Image path: c:\images\car.bmp
[ INFO ] Image path: /test_data/images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 656 0.6645315
[ INFO ] 654 0.1121185
[ INFO ] 581 0.0698451
[ INFO ] 874 0.0334973
[ INFO ] 436 0.0259718
[ INFO ] 817 0.0173190
[ INFO ] 675 0.0109321
[ INFO ] 511 0.0109075
[ INFO ] 569 0.0083093
[ INFO ] 717 0.0063173
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9707602
[ INFO ] 666 0.0216788
[ INFO ] 659 0.0032558
[ INFO ] 435 0.0008082
[ INFO ] 809 0.0004359
[ INFO ] 502 0.0003860
[ INFO ] 618 0.0002867
[ INFO ] 910 0.0002866
[ INFO ] 951 0.0002410
[ INFO ] 961 0.0002193
[ INFO ]
[ INFO ] Infer request 1 returned 0
[ INFO ] Image path: c:\images\cat.jpg
[ INFO ] Image path: /test_data/images/car.bmp
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 876 0.1320105
[ INFO ] 435 0.1210389
[ INFO ] 285 0.0712640
[ INFO ] 282 0.0570528
[ INFO ] 281 0.0319335
[ INFO ] 999 0.0285931
[ INFO ] 94 0.0270323
[ INFO ] 36 0.0240510
[ INFO ] 335 0.0198461
[ INFO ] 186 0.0183939
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 656 0.5120340
[ INFO ] 874 0.1142275
[ INFO ] 654 0.0697167
[ INFO ] 436 0.0615163
[ INFO ] 581 0.0552262
[ INFO ] 705 0.0304179
[ INFO ] 675 0.0151660
[ INFO ] 734 0.0151582
[ INFO ] 627 0.0148493
[ INFO ] 757 0.0120964
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.add_extension]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a8a4b671a9928c7c059bd1e76d2333967
[IECore.set_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a2c738cee90fca27146e629825c039a05
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[InferRequest.async_infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InferRequest.html#a95ebe0368cdf4d5d64f9fddc8ee1cd0e
[InferRequest.wait]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InferRequest.html#a936fa50a7531e2f9a9e9c3d45afc9b43
<!-- TODO replace by python API link -->
[Blob.buffer]:https://docs.openvino.ai/latest/classInferenceEngine_1_1Blob.html#a0cad47b43204b115b4017b6b2564fa7e
<!-- [openvino.runtime.AsyncInferQueue]:
[openvino.runtime.AsyncInferQueue.set_callback]:
[openvino.runtime.AsyncInferQueue.start_async]:
[openvino.runtime.AsyncInferQueue.wait_all]:
[openvino.runtime.InferRequest.results]: -->

2
samples/python/classification_sample_async/classification_sample_async.py
View File

@@ -64,7 +64,7 @@ def main() -> int:
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {args.model}')
log.info(f'Reading the model: {args.model}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(args.model)

144
samples/python/hello_classification/README.md
View File

@@ -1,59 +1,35 @@
# Hello Classification Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_classification_README}
This sample demonstrates how to do inference of image classification networks using Synchronous Inference Request API.
This sample demonstrates how to do inference of image classification models using Synchronous Inference Request API.
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------- | :-------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Basic Infer Flow | [IECore], [IECore.read_network], [IECore.load_network] | Common API to do inference |
| Synchronous Infer | [ExecutableNetwork.infer] | Do synchronous inference |
| Network Operations | [IENetwork.input_info], [IENetwork.outputs], [InputInfoPtr.precision], [DataPtr.precision], [InputInfoPtr.input_data.shape] | Managing of network: configure input and output blobs |
| Feature | API | Description |
| :---------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Basic Infer Flow | [openvino.runtime.Core], [openvino.runtime.Core.read_model], [openvino.runtime.Core.compile_model] | Common API to do inference |
| Synchronous Infer | [openvino.runtime.CompiledModel.infer_new_request] | Do synchronous inference |
| Model Operations | [openvino.runtime.Model.inputs], [openvino.runtime.Model.outputs] | Managing of model |
| Preprocessing | [openvino.preprocess.PrePostProcessor], [openvino.preprocess.InputTensorInfo.set_element_type],[openvino.preprocess.InputTensorInfo.set_layout],[openvino.preprocess.InputTensorInfo.set_spatial_static_shape],[openvino.preprocess.PreProcessSteps.resize],[openvino.preprocess.InputModelInfo.set_layout],[openvino.preprocess.OutputTensorInfo.set_element_type],[openvino.preprocess.PrePostProcessor.build] | Set image of the original size as input for a model with other input size. Resize and layout conversions will be performed automatically by the corresponding plugin just before inference |
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------- |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Options | Values |
| :------------------------- | :------------------------------------------------------------------------------------------------------ |
| Validated Models | [alexnet](@ref omz_models_model_alexnet), [googlenet-v1](@ref omz_models_model_googlenet_v1) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_classification/README.md), [C](../../c/hello_classification/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/hello_classification.py -h
```
Usage message:
```
usage: hello_classification.py [-h] -m MODEL -i INPUT [-d DEVICE]
[--labels LABELS] [-nt NUMBER_TOP]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to an .xml or .onnx file with a trained
model.
-i INPUT, --input INPUT
Required. Path to an image file.
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
python hello_classification.py <path_to_model> <path_to_image> <device_name>
```
To run the sample, you need specify a model and image:
@@ -62,28 +38,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name alexnet
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name alexnet
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `alexnet` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name alexnet
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/hello_classification.py -m <path_to_model>/alexnet.xml -i <path_to_image>/car.bmp -d GPU
omz_converter --name alexnet
```
4. Perform inference of `banana.jpg` using the `alexnet` model on a `GPU`, for example:
```
python hello_classification.py alexnet.xml banana.jpg GPU
```
## Sample Output
@@ -91,43 +74,46 @@ python <path_to_sample>/hello_classification.py -m <path_to_model>/alexnet.xml -
The sample application logs each step in a standard output stream and outputs top-10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\alexnet\FP32\alexnet.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/alexnet/alexnet.xml
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\car.bmp is resized from (637, 749) to (227, 227)
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: c:\images\car.bmp
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 656 0.6645315
[ INFO ] 654 0.1121185
[ INFO ] 581 0.0698451
[ INFO ] 874 0.0334973
[ INFO ] 436 0.0259718
[ INFO ] 817 0.0173190
[ INFO ] 675 0.0109321
[ INFO ] 511 0.0109075
[ INFO ] 569 0.0083093
[ INFO ] 717 0.0063173
[ INFO ] Image path: /images/banana.jpg
[ INFO ] Top 10 results:
[ INFO ] class_id probability
[ INFO ] --------------------
[ INFO ] 954 0.9703885
[ INFO ] 666 0.0219518
[ INFO ] 659 0.0033120
[ INFO ] 435 0.0008246
[ INFO ] 809 0.0004433
[ INFO ] 502 0.0003852
[ INFO ] 618 0.0002906
[ INFO ] 910 0.0002848
[ INFO ] 951 0.0002427
[ INFO ] 961 0.0002213
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- [openvino.runtime.Core]:
[openvino.runtime.Core.read_model]:
[openvino.runtime.Core.compile_model]:
[openvino.runtime.CompiledModel.infer_new_request]:
[openvino.runtime.Model.inputs]:
[openvino.runtime.Model.outputs]:
[openvino.preprocess.PrePostProcessor]:
[openvino.preprocess.InputTensorInfo.set_element_type]:
[openvino.preprocess.InputTensorInfo.set_layout]:
[openvino.preprocess.InputTensorInfo.set_spatial_static_shape]:
[openvino.preprocess.PreProcessSteps.resize]:
[openvino.preprocess.InputModelInfo.set_layout]:
[openvino.preprocess.OutputTensorInfo.set_element_type]:
[openvino.preprocess.PrePostProcessor.build]: -->

4
samples/python/hello_classification/hello_classification.py
View File

@@ -17,7 +17,7 @@ def main():
# Parsing and validation of input arguments
if len(sys.argv) != 4:
log.info('Usage: <path_to_model> <path_to_image> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <path_to_image> <device_name>')
return 1
model_path = sys.argv[1]
@@ -29,7 +29,7 @@ def main():
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {model_path}')
log.info(f'Reading the model: {model_path}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(model_path)

79
samples/python/hello_query_device/README.md
View File

@@ -1,37 +1,37 @@
# Hello Query Device Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_query_device_README}
This sample demonstrates how to show Inference Engine devices and prints their metrics and default configuration values using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
This sample demonstrates how to show OpenVINO™ Runtime devices and prints their metrics and default configuration values using [Query Device API feature](../../../docs/OV_Runtime_UG/InferenceEngine_QueryAPI.md).
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------- | :--------------------------------------- | :-------------------- |
| Basic | [IECore] | Common API |
| Query Device | [IECore.get_metric], [IECore.get_config] | Get device properties |
| Feature | API | Description |
| :----------- | :---------------------------------------------------------------------------------------------------------------- | :-------------------- |
| Basic | [openvino.runtime.Core] | Common API |
| Query Device | [openvino.runtime.Core.available_devices], [openvino.runtime.Core.get_metric], [openvino.runtime.Core.get_config] | Get device properties |
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------- |
| Options | Values |
| :------------------------- | :---------------------------------------------------------------- |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_query_device/README.md) |
| Other language realization | [C++](../../../samples/cpp/hello_query_device/README.md) |
## How It Works
The sample queries all available Inference Engine devices and prints their supported metrics and plugin configuration parameters.
The sample queries all available OpenVINO™ Runtime devices and prints their supported metrics and plugin configuration parameters.
## Running
The sample has no command-line parameters. To see the report, run the following command:
```
python <path_to_sample>/hello_query_device.py
python hello_query_device.py
```
## Sample Output
The application prints all available devices with their supported metrics and default values for configuration parameters. (Some lines are not shown due to length.) For example:
The application prints all available devices with their supported metrics and default values for configuration parameters.
For example:
```
[ INFO ] Creating Inference Engine
[ INFO ] Available devices:
[ INFO ] CPU :
[ INFO ] SUPPORTED_METRICS:
@@ -40,9 +40,11 @@ The application prints all available devices with their supported metrics and de
[ INFO ] OPTIMIZATION_CAPABILITIES: FP32, FP16, INT8, BIN
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 1, 1
[ INFO ] RANGE_FOR_STREAMS: 1, 8
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CPU_BIND_THREAD: NUMA
[ INFO ] CACHE_DIR:
[ INFO ] CPU_BIND_THREAD: NO
[ INFO ] CPU_THREADS_NUM: 0
[ INFO ] CPU_THROUGHPUT_STREAMS: 1
[ INFO ] DUMP_EXEC_GRAPH_AS_DOT:
@@ -50,6 +52,8 @@ The application prints all available devices with their supported metrics and de
[ INFO ] DYN_BATCH_LIMIT: 0
[ INFO ] ENFORCE_BF16: NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] PERFORMANCE_HINT:
[ INFO ] PERFORMANCE_HINT_NUM_REQUESTS: 0
[ INFO ] PERF_COUNT: NO
[ INFO ]
[ INFO ] GNA :
@@ -57,54 +61,33 @@ The application prints all available devices with their supported metrics and de
[ INFO ] AVAILABLE_DEVICES: GNA_SW
[ INFO ] OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1
[ INFO ] FULL_DEVICE_NAME: GNA_SW
[ INFO ] GNA_LIBRARY_FULL_VERSION: 2.0.0.1047
[ INFO ] GNA_LIBRARY_FULL_VERSION: 3.0.0.1455
[ INFO ] IMPORT_EXPORT_SUPPORT: True
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] GNA_COMPACT_MODE: NO
[ INFO ] GNA_COMPACT_MODE: YES
[ INFO ] GNA_COMPILE_TARGET:
[ INFO ] GNA_DEVICE_MODE: GNA_SW_EXACT
[ INFO ] GNA_EXEC_TARGET:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE:
[ INFO ] GNA_FIRMWARE_MODEL_IMAGE_GENERATION:
[ INFO ] GNA_LIB_N_THREADS: 1
[ INFO ] GNA_PRECISION: I16
[ INFO ] GNA_PWL_MAX_ERROR_PERCENT: 1.000000
[ INFO ] GNA_PWL_UNIFORM_DESIGN: NO
[ INFO ] GNA_SCALE_FACTOR: 1.000000
[ INFO ] GNA_SCALE_FACTOR_0: 1.000000
[ INFO ] LOG_LEVEL: LOG_NONE
[ INFO ] PERF_COUNT: NO
[ INFO ] SINGLE_THREAD: YES
[ INFO ]
[ INFO ] GPU :
[ INFO ] SUPPORTED_METRICS:
[ INFO ] AVAILABLE_DEVICES: 0
[ INFO ] FULL_DEVICE_NAME: Intel(R) UHD Graphics 620 (iGPU)
[ INFO ] OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16
[ INFO ] RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1
[ INFO ] RANGE_FOR_STREAMS: 1, 2
[ INFO ]
[ INFO ] SUPPORTED_CONFIG_KEYS (default values):
[ INFO ] CACHE_DIR:
[ INFO ] CLDNN_ENABLE_FP16_FOR_QUANTIZED_MODELS: YES
[ INFO ] CLDNN_GRAPH_DUMPS_DIR:
[ INFO ] CLDNN_MEM_POOL: YES
[ INFO ] CLDNN_NV12_TWO_INPUTS: NO
[ INFO ] CLDNN_PLUGIN_PRIORITY: 0
[ INFO ] CLDNN_PLUGIN_THROTTLE: 0
[ INFO ] CLDNN_SOURCES_DUMPS_DIR:
[ INFO ] CONFIG_FILE:
[ INFO ] DEVICE_ID:
[ INFO ] DUMP_KERNELS: NO
[ INFO ] DYN_BATCH_ENABLED: NO
[ INFO ] EXCLUSIVE_ASYNC_REQUESTS: NO
[ INFO ] GPU_THROUGHPUT_STREAMS: 1
[ INFO ] PERF_COUNT: NO
[ INFO ] TUNING_FILE:
[ INFO ] TUNING_MODE: TUNING_DISABLED
[ INFO ]
```
## See Also
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.get_metric]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#af1cdf2ecbea6399c556957c2c2fdf8eb
[IECore.get_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a48764dec7c235d2374af8b8ef53c6363
<!-- [openvino.runtime.Core]:
[openvino.runtime.Core.available_devices]:
[openvino.runtime.Core.get_metric]:
[openvino.runtime.Core.get_config]: -->

119
samples/python/hello_reshape_ssd/README.md
View File

@@ -1,67 +1,35 @@
# Hello Reshape SSD Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_hello_reshape_ssd_README}
This sample demonstrates how to do synchronous inference of object detection networks using [Shape Inference feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
This sample demonstrates how to do synchronous inference of object detection models using [Shape Inference feature](../../../docs/OV_Runtime_UG/ShapeInference.md).
Models with only 1 input and output are supported.
The following Inference Engine Python API is used in the application:
The following Python API is used in the application:
| Feature | API | Description |
| :----------------------- | :-------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Network Operations | [IENetwork.reshape] | Managing of network: configure input and output blobs |
| Custom Extension Kernels | [IECore.add_extension], [IECore.set_config] | Load extension library and config to the device |
| Feature | API | Description |
| :--------------- | :---------------------------------------------------------------------------------------------------------------------------------------- | :---------------- |
| Model Operations | [openvino.runtime.Model.reshape], [openvino.runtime.Model.input], [openvino.runtime.Output.get_any_name], [openvino.runtime.PartialShape] | Managing of model |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------------------------------------------------------------- |
| Validated Models | [mobilenet-ssd](@ref omz_models_model_mobilenet_ssd) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_reshape_ssd/README.md) |
| Options | Values |
| :------------------------- | :----------------------------------------------------------------------- |
| Validated Models | [ssdlite_mobilenet_v2](@ref omz_models_model_ssdlite_mobilenet_v2) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin), ONNX (.onnx) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/hello_reshape_ssd/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the Inference Engine plugin, performs synchronous inference, and processes output data.
At startup, the sample application reads command-line parameters, prepares input data, loads a specified model and image to the OpenVINO™ Runtime plugin, performs synchronous inference, and processes output data.
As a result, the program creates an output image, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/hello_reshape_ssd.py -h
```
Usage message:
```
usage: hello_reshape_ssd.py [-h] -m MODEL -i INPUT [-l EXTENSION] [-c CONFIG]
[-d DEVICE] [--labels LABELS]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to an .xml or .onnx file with a trained
model.
-i INPUT, --input INPUT
Required. Path to an image file.
-l EXTENSION, --extension EXTENSION
Optional. Required by the CPU Plugin for executing the
custom operation on a CPU. Absolute path to a shared
library with the kernels implementations.
-c CONFIG, --config CONFIG
Optional. Required by GPU or VPU Plugins for the
custom operation kernel. Absolute path to operation
description file (.xml).
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
python hello_reshape_ssd.py <path_to_model> <path_to_image> <device_name>
```
To run the sample, you need specify a model and image:
@@ -70,28 +38,35 @@ To run the sample, you need specify a model and image:
> **NOTES**:
>
> - By default, Inference Engine samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
> - By default, OpenVINO™ Toolkit Samples and demos expect input with BGR channels order. If you trained your model to work with RGB order, you need to manually rearrange the default channels order in the sample or demo application or reconvert your model using the Model Optimizer tool with `--reverse_input_channels` argument specified. For more information about the argument, refer to **When to Reverse Input Channels** section of [Converting a Model](../../../docs/MO_DG/prepare_model/convert_model/Converting_Model.md).
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample accepts models in ONNX format (.onnx) that do not require preprocessing.
### Example
1. Download a pre-trained model using [Model Downloader](@ref omz_tools_downloader):
```
python <path_to_omz_tools>/downloader.py --name mobilenet-ssd
```
2. If a model is not in the Inference Engine IR or ONNX format, it must be converted. You can do this using the model converter script:
1. Install the `openvino-dev` Python package to use Open Model Zoo Tools:
```
python <path_to_omz_tools>/converter.py --name mobilenet-ssd
python -m pip install openvino-dev[caffe,onnx,tensorflow2,pytorch,mxnet]
```
3. Perform inference of `car.bmp` using `mobilenet-ssd` model on a `GPU`, for example:
2. Download a pre-trained model:
```
omz_downloader --name ssdlite_mobilenet_v2
```
3. If a model is not in the IR or ONNX format, it must be converted. You can do this using the model converter:
```
python <path_to_sample>/hello_reshape_ssd.py -m <path_to_model>/mobilenet-ssd.xml -i <path_to_image>/car.bmp -d GPU
omz_converter --name ssdlite_mobilenet_v2
```
4. Perform inference of `banana.jpg` using `ssdlite_mobilenet_v2` model on a `GPU`, for example:
```
python hello_reshape_ssd.py ssdlite_mobilenet_v2.xml banana.jpg GPU
```
## Sample Output
@@ -99,34 +74,24 @@ python <path_to_sample>/hello_reshape_ssd.py -m <path_to_model>/mobilenet-ssd.xm
The sample application logs each step in a standard output stream and creates an output image, drawing bounding boxes for inference results with an over 50% confidence.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: c:\openvino\deployment_tools\open_model_zoo\tools\downloader\public\mobilenet-ssd\FP32\mobilenet-ssd.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Reshaping the network to the height and width of the input image
[ INFO ] Input shape before reshape: [1, 3, 300, 300]
[ INFO ] Input shape after reshape: [1, 3, 637, 749]
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: C:/test_data/models/ssdlite_mobilenet_v2.xml
[ INFO ] Reshaping the model to the height and width of the input image
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Found: label = 7, confidence = 0.99, coords = (283, 166), (541, 472)
[ INFO ] Found: class_id = 52, confidence = 0.98, coords = (21, 98), (276, 210)
[ INFO ] Image out.bmp was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IECore.add_extension]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a8a4b671a9928c7c059bd1e76d2333967
[IECore.set_config]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a2c738cee90fca27146e629825c039a05
[IECore.read_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#a0d69c298618fab3a08b855442dca430f
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[IENetwork.reshape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a6683f0291db25f908f8d6720ab2f221a
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- [openvino.runtime.Model.reshape]:
[openvino.runtime.Model.input]:
[openvino.runtime.Output.get_any_name]:
[openvino.runtime.PartialShape]: -->

6
samples/python/hello_reshape_ssd/hello_reshape_ssd.py
View File

@@ -18,7 +18,7 @@ def main():
# Parsing and validation of input arguments
if len(sys.argv) != 4:
log.info('Usage: <path_to_model> <path_to_image> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <path_to_image> <device_name>')
return 1
model_path = sys.argv[1]
@@ -30,7 +30,7 @@ def main():
core = Core()
# --------------------------- Step 2. Read a model --------------------------------------------------------------------
log.info(f'Reading the network: {model_path}')
log.info(f'Reading the model: {model_path}')
# (.xml and .bin files) or (.onnx file)
model = core.read_model(model_path)
@@ -48,7 +48,7 @@ def main():
# Add N dimension
input_tensor = np.expand_dims(image, 0)
log.info('Reshaping the network to the height and width of the input image')
log.info('Reshaping the model to the height and width of the input image')
n, h, w, c = input_tensor.shape
model.reshape({model.input().get_any_name(): PartialShape((n, c, h, w))})

196
samples/python/ngraph_function_creation_sample/README.md
View File

@@ -1,66 +1,43 @@
# nGraph Function Creation Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_ngraph_function_creation_sample_README}
This sample demonstrates how to execute an inference using [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) to create a network that uses weights from LeNet classification network, which is known to work well on digit classification tasks. So you don't need an XML file, the model will be created from the source code on the fly.
This sample demonstrates how to run inference using a [model](../../../docs/OV_Runtime_UG/model_representation.md) built on the fly that uses weights from the LeNet classification model, which is known to work well on digit classification tasks. You do not need an XML file, the model is created from the source code on the fly.
In addition to regular grayscale images with a digit, the sample also supports single-channel `ubyte` images as an input.
The following Python API is used in the application:
The following Inference Engine Python API is used in the application:
| Feature | API | Description |
| :--------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------ |
| Model Operations | [openvino.runtime.Model], [openvino.runtime.set_batch], [openvino.runtime.Model.input] | Managing of model |
| nGraph Functions | [openvino.runtime.op.Parameter], [openvino.runtime.op.Constant], [openvino.runtime.opset8.convolution], [openvino.runtime.opset8.add], [openvino.runtime.opset1.max_pool], [openvino.runtime.opset8.reshape], [openvino.runtime.opset8.matmul], [openvino.runtime.opset8.relu], [openvino.runtime.opset8.softmax] | Description of a model topology using nGraph Python API |
| Feature | API | Description |
| :----------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------- |
| Network Operations | [IENetwork], [IENetwork.batch_size] | Managing of network |
| nGraph Functions | [ngraph.impl.Function], [ngraph.parameter], [ngraph.constant], [ngraph.convolution], [ngraph.add], [ngraph.max_pool], [ngraph.reshape], [ngraph.matmul], [ngraph.relu], [ngraph.softmax], [ngraph.result], ngraph.impl.Function.to_capsule | Description of a network using nGraph Python API |
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------- |
| Validated Models | LeNet |
| Model Format | Network weights file (\*.bin) |
| Validated images | The sample uses OpenCV\* to [read input grayscale image](https://docs.opencv.org/master/d4/da8/group__imgcodecs.html#ga288b8b3da0892bd651fce07b3bbd3a56) (\*.bmp, \*.png) or single-channel `ubyte` image |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Options | Values |
| :------------------------- | :-------------------------------------------------------------------- |
| Validated Models | LeNet |
| Model Format | Model weights file (\*.bin) |
| Supported devices | [All](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/ngraph_function_creation_sample/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, prepares input data, creates a network using [OpenVINO Model feature](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file, loads the network and image(s) to the Inference Engine plugin, performs synchronous inference, and processes output data, logging each step in a standard output stream.
At startup, the sample application does the following:
- Reads command line parameters
- [Build a Model](../../../docs/OV_Runtime_UG/model_representation.md) and passed weights file
- Loads the model and input data to the OpenVINO™ Runtime plugin
- Performs synchronous inference and processes output data, logging each step in a standard output stream
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## Running
Run the application with the `-h` option to see the usage message:
```sh
python <path_to_sample>/ngraph_function_creation_sample.py -h
```
Usage message:
To run the sample, you need to specify model weights and device.
```
usage: ngraph_function_creation_sample.py [-h] -m MODEL -i INPUT [INPUT ...]
[-d DEVICE] [--labels LABELS]
[-nt NUMBER_TOP]
Options:
-h, --help Show this help message and exit.
-m MODEL, --model MODEL
Required. Path to a file with network weights.
-i INPUT [INPUT ...], --input INPUT [INPUT ...]
Required. Path to an image file.
-d DEVICE, --device DEVICE
Optional. Specify the target device to infer on; CPU,
GPU, MYRIAD, HDDL or HETERO: is acceptable. The sample
will look for a suitable plugin for device specified.
Default value is CPU.
--labels LABELS Optional. Path to a labels mapping file.
-nt NUMBER_TOP, --number_top NUMBER_TOP
Optional. Number of top results.
python ngraph_function_creation_sample.py <path_to_model> <device_name>
```
To run the sample, you need specify a model weights and image. You can use images from the media files collection available at https://storage.openvinotoolkit.org/data/test_data.
> **NOTE**:
>
> - This sample supports models with FP32 weights only.
@@ -68,73 +45,102 @@ To run the sample, you need specify a model weights and image. You can use image
> - The `lenet.bin` weights file was generated by the [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) tool from the public LeNet model with the `--input_shape [64,1,28,28]` parameter specified.
>
> - The original model is available in the [Caffe* repository](https://github.com/BVLC/caffe/tree/master/examples/mnist) on GitHub\*.
>
> - The white over black images will be automatically inverted in color for a better predictions.
For example, you can do inference of [3.png](https://storage.openvinotoolkit.org/data/test_data/images/3.png) using the pre-trained model on a `GPU`:
For example:
```sh
python <path_to_sample>/ngraph_function_creation_sample.py -m <path_to_sample>/lenet.bin -i <path_to_image>/3.png -d GPU
```
python ngraph_function_creation_sample.py lenet.bin GPU
```
## Sample Output
The sample application logs each step in a standard output stream and outputs top-10 inference results.
The sample application logs each step in a standard output stream and outputs 10 inference results.
```
[ INFO ] Creating Inference Engine
[ INFO ] Loading the network using ngraph function with weights from c:\openvino\samples\python\ngraph_function_creation_sample\lenet.bin
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Loading the model using ngraph function with weights from lenet.bin
[ INFO ] Loading the model to the plugin
[ WARNING ] Image c:\images\3.png is inverted to white over black
[ WARNING ] Image c:\images\3.png is resized from (351, 353) to (28, 28)
[ INFO ] Starting inference in synchronous mode
[ INFO ] Image path: c:\images\3.png
[ INFO ] Top 10 results:
[ INFO ] classid probability
[ INFO ] -------------------
[ INFO ] 3 1.0000000
[ INFO ] 9 0.0000000
[ INFO ] 8 0.0000000
[ INFO ] 7 0.0000000
[ INFO ] 6 0.0000000
[ INFO ] 5 0.0000000
[ INFO ] 4 0.0000000
[ INFO ] 2 0.0000000
[ INFO ] 1 0.0000000
[ INFO ] 0 0.0000000
[ INFO ] Top 1 results:
[ INFO ] Image 0
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 0 1.0000000 0
[ INFO ]
[ INFO ] Image 1
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 1 1.0000000 1
[ INFO ]
[ INFO ] Image 2
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 2 1.0000000 2
[ INFO ]
[ INFO ] Image 3
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 3 1.0000000 3
[ INFO ]
[ INFO ] Image 4
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 4 1.0000000 4
[ INFO ]
[ INFO ] Image 5
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 5 1.0000000 5
[ INFO ]
[ INFO ] Image 6
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 6 1.0000000 6
[ INFO ]
[ INFO ] Image 7
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 7 1.0000000 7
[ INFO ]
[ INFO ] Image 8
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 8 1.0000000 8
[ INFO ]
[ INFO ] Image 9
[ INFO ]
[ INFO ] classid probability label
[ INFO ] -------------------------
[ INFO ] 9 1.0000000 9
[ INFO ]
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IECore]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html
[IENetwork]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html
[IENetwork.batch_size]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a79a647cb1b49645616eaeb2ca255ef2e
[IENetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[IENetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#data_fields
[InputInfoPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[DataPtr.precision]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1DataPtr.html#data_fields
[IECore.load_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc
[InputInfoPtr.input_data.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1InputInfoPtr.html#data_fields
[ExecutableNetwork.infer]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#aea96e8e534c8e23d8b257bad11063519
<!-- TODO: Replace the link by another one pointing to the Python API, if available -->
[ngraph.impl.Function]:https://docs.openvino.ai/latest/ngraph_cpp_api/classngraph_1_1Function.html
<!-- [ngraph.impl.Function.to_capsule]: -->
[ngraph.parameter]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a709acd09288f5a76ed8d07492efc3d13
[ngraph.constant]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a5b6c4e416026e007a4107b3f510d0c27
[ngraph.convolution]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a3143ff55f68428afc1b6c802ee9381e8
[ngraph.add]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#abfa0373c10ced1b1f129594d9bd8a159
[ngraph.max_pool]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#ac60b4459ad23b296086925abce6acd2d
[ngraph.reshape]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a38e1ead9435c4b75c1d891ba2dd6a62e
[ngraph.matmul]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a403b5e10e1f75aeb7569024237e85071
[ngraph.relu]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a70b9b3faf58d85e43d27fef5028117e3
[ngraph.softmax]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a632cc9a31ecaefa2a982d039ecad8d26
[ngraph.result]:https://docs.openvino.ai/latest/ngraph_python_api/namespacengraph_1_1opset1_1_1ops.html#a94f8bf6ab8910dfd461d09cb6c6edd11
<!-- [openvino.runtime.Model]:
[openvino.runtime.set_batch]:
[openvino.runtime.Model.input]:
[openvino.runtime.op.Parameter]:
[openvino.runtime.op.Constant]:
[openvino.runtime.opset8.convolution]:
[openvino.runtime.opset8.add]:
[openvino.runtime.opset1.max_pool]:
[openvino.runtime.opset8.reshape]:
[openvino.runtime.opset8.matmul]:
[openvino.runtime.opset8.relu]:
[openvino.runtime.opset8.softmax]: -->

90
samples/python/ngraph_function_creation_sample/ngraph_function_creation_sample.py
View File

@@ -9,13 +9,14 @@ from functools import reduce
import numpy as np
from openvino.preprocess import PrePostProcessor
from openvino.runtime import Core, Layout, Type, Model, Shape, PartialShape
import openvino
from openvino.runtime import (Core, Layout, Model, Shape, Type, op, opset1,
opset8, set_batch)
from data import digits
def create_ngraph_function(model_path: str) -> Model:
"""Create a network on the fly from the source code using ngraph"""
"""Create a model on the fly from the source code using ngraph"""
def shape_and_length(shape: list) -> typing.Tuple[list, int]:
length = reduce(lambda x, y: x * y, shape)
@@ -27,42 +28,42 @@ def create_ngraph_function(model_path: str) -> Model:
# input
input_shape = [64, 1, 28, 28]
param_node = openvino.runtime.op.Parameter(Type.f32, Shape(input_shape))
param_node = op.Parameter(Type.f32, Shape(input_shape))
# convolution 1
conv_1_kernel_shape, conv_1_kernel_length = shape_and_length([20, 1, 5, 5])
conv_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(conv_1_kernel_shape), weights[0:conv_1_kernel_length].tolist())
conv_1_kernel = op.Constant(Type.f32, Shape(conv_1_kernel_shape), weights[0:conv_1_kernel_length].tolist())
weights_offset += conv_1_kernel_length
conv_1_node = openvino.runtime.opset8.convolution(param_node, conv_1_kernel, [1, 1], padding_begin, padding_end, [1, 1])
conv_1_node = opset8.convolution(param_node, conv_1_kernel, [1, 1], padding_begin, padding_end, [1, 1])
# add 1
add_1_kernel_shape, add_1_kernel_length = shape_and_length([1, 20, 1, 1])
add_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_1_kernel_shape),
weights[weights_offset : weights_offset + add_1_kernel_length])
add_1_kernel = op.Constant(Type.f32, Shape(add_1_kernel_shape),
weights[weights_offset : weights_offset + add_1_kernel_length])
weights_offset += add_1_kernel_length
add_1_node = openvino.runtime.opset8.add(conv_1_node, add_1_kernel)
add_1_node = opset8.add(conv_1_node, add_1_kernel)
# maxpool 1
maxpool_1_node = openvino.runtime.opset1.max_pool(add_1_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
maxpool_1_node = opset1.max_pool(add_1_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
# convolution 2
conv_2_kernel_shape, conv_2_kernel_length = shape_and_length([50, 20, 5, 5])
conv_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(conv_2_kernel_shape),
weights[weights_offset : weights_offset + conv_2_kernel_length],
)
conv_2_kernel = op.Constant(Type.f32, Shape(conv_2_kernel_shape),
weights[weights_offset : weights_offset + conv_2_kernel_length],
)
weights_offset += conv_2_kernel_length
conv_2_node = openvino.runtime.opset8.convolution(maxpool_1_node, conv_2_kernel, [1, 1], padding_begin, padding_end, [1, 1])
conv_2_node = opset8.convolution(maxpool_1_node, conv_2_kernel, [1, 1], padding_begin, padding_end, [1, 1])
# add 2
add_2_kernel_shape, add_2_kernel_length = shape_and_length([1, 50, 1, 1])
add_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_2_kernel_shape),
weights[weights_offset : weights_offset + add_2_kernel_length],
)
add_2_kernel = op.Constant(Type.f32, Shape(add_2_kernel_shape),
weights[weights_offset : weights_offset + add_2_kernel_length],
)
weights_offset += add_2_kernel_length
add_2_node = openvino.runtime.opset8.add(conv_2_node, add_2_kernel)
add_2_node = opset8.add(conv_2_node, add_2_kernel)
# maxpool 2
maxpool_2_node = openvino.runtime.opset1.max_pool(add_2_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
maxpool_2_node = opset1.max_pool(add_2_node, [2, 2], padding_begin, padding_end, [2, 2], 'ceil')
# reshape 1
reshape_1_dims, reshape_1_length = shape_and_length([2])
@@ -71,52 +72,52 @@ def create_ngraph_function(model_path: str) -> Model:
weights[weights_offset : weights_offset + 2 * reshape_1_length],
dtype=np.int64,
)
reshape_1_kernel = openvino.runtime.op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_1_kernel = op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
weights_offset += 2 * reshape_1_length
reshape_1_node = openvino.runtime.opset8.reshape(maxpool_2_node, reshape_1_kernel, True)
reshape_1_node = opset8.reshape(maxpool_2_node, reshape_1_kernel, True)
# matmul 1
matmul_1_kernel_shape, matmul_1_kernel_length = shape_and_length([500, 800])
matmul_1_kernel = openvino.runtime.op.Constant(Type.f32, Shape(matmul_1_kernel_shape),
weights[weights_offset : weights_offset + matmul_1_kernel_length],
)
matmul_1_kernel = op.Constant(Type.f32, Shape(matmul_1_kernel_shape),
weights[weights_offset : weights_offset + matmul_1_kernel_length],
)
weights_offset += matmul_1_kernel_length
matmul_1_node = openvino.runtime.opset8.matmul(reshape_1_node, matmul_1_kernel, False, True)
matmul_1_node = opset8.matmul(reshape_1_node, matmul_1_kernel, False, True)
# add 3
add_3_kernel_shape, add_3_kernel_length = shape_and_length([1, 500])
add_3_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_3_kernel_shape),
weights[weights_offset : weights_offset + add_3_kernel_length],
)
add_3_kernel = op.Constant(Type.f32, Shape(add_3_kernel_shape),
weights[weights_offset : weights_offset + add_3_kernel_length],
)
weights_offset += add_3_kernel_length
add_3_node = openvino.runtime.opset8.add(matmul_1_node, add_3_kernel)
add_3_node = opset8.add(matmul_1_node, add_3_kernel)
# ReLU
relu_node = openvino.runtime.opset8.relu(add_3_node)
relu_node = opset8.relu(add_3_node)
# reshape 2
reshape_2_kernel = openvino.runtime.op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_2_node = openvino.runtime.opset8.reshape(relu_node, reshape_2_kernel, True)
reshape_2_kernel = op.Constant(Type.i64, Shape(list(dtype_weights.shape)), dtype_weights)
reshape_2_node = opset8.reshape(relu_node, reshape_2_kernel, True)
# matmul 2
matmul_2_kernel_shape, matmul_2_kernel_length = shape_and_length([10, 500])
matmul_2_kernel = openvino.runtime.op.Constant(Type.f32, Shape(matmul_2_kernel_shape),
weights[weights_offset : weights_offset + matmul_2_kernel_length],
)
matmul_2_kernel = op.Constant(Type.f32, Shape(matmul_2_kernel_shape),
weights[weights_offset : weights_offset + matmul_2_kernel_length],
)
weights_offset += matmul_2_kernel_length
matmul_2_node = openvino.runtime.opset8.matmul(reshape_2_node, matmul_2_kernel, False, True)
matmul_2_node = opset8.matmul(reshape_2_node, matmul_2_kernel, False, True)
# add 4
add_4_kernel_shape, add_4_kernel_length = shape_and_length([1, 10])
add_4_kernel = openvino.runtime.op.Constant(Type.f32, Shape(add_4_kernel_shape),
weights[weights_offset : weights_offset + add_4_kernel_length],
)
add_4_kernel = op.Constant(Type.f32, Shape(add_4_kernel_shape),
weights[weights_offset : weights_offset + add_4_kernel_length],
)
weights_offset += add_4_kernel_length
add_4_node = openvino.runtime.opset8.add(matmul_2_node, add_4_kernel)
add_4_node = opset8.add(matmul_2_node, add_4_kernel)
# softmax
softmax_axis = 1
softmax_node = openvino.runtime.opset8.softmax(add_4_node, softmax_axis)
softmax_node = opset8.softmax(add_4_node, softmax_axis)
return Model(softmax_node, [param_node], 'lenet')
@@ -125,7 +126,7 @@ def main():
log.basicConfig(format='[ %(levelname)s ] %(message)s', level=log.INFO, stream=sys.stdout)
# Parsing and validation of input arguments
if len(sys.argv) != 3:
log.info('Usage: <path_to_model> <device_name>')
log.info(f'Usage: {sys.argv[0]} <path_to_model> <device_name>')
return 1
model_path = sys.argv[1]
@@ -137,7 +138,7 @@ def main():
core = Core()
# ---------------------------Step 2. Read a model in OpenVINO Intermediate Representation------------------------------
log.info(f'Loading the network using ngraph function with weights from {model_path}')
log.info(f'Loading the model using ngraph function with weights from {model_path}')
model = create_ngraph_function(model_path)
# ---------------------------Step 3. Apply preprocessing----------------------------------------------------------
# Get names of input and output blobs
@@ -160,8 +161,7 @@ def main():
model = ppp.build()
# Set a batch size equal to number of input images
model.reshape({model.input().get_any_name(): PartialShape((digits.shape[0], model.input().shape[1], model.input().shape[2], model.input().shape[3]))})
set_batch(model, digits.shape[0])
# ---------------------------Step 4. Loading model to the device-------------------------------------------------------
log.info('Loading the model to the plugin')
compiled_model = core.compile_model(model, device_name)

307
samples/python/speech_sample/README.md
View File

@@ -1,48 +1,50 @@
# Automatic Speech Recognition Python* Sample {#openvino_inference_engine_ie_bridges_python_sample_speech_sample_README}
This sample demonstrates how to do a Synchronous Inference of acoustic model based on Kaldi\* neural networks and speech feature vectors.
This sample demonstrates how to do a Synchronous Inference of acoustic model based on Kaldi\* neural models and speech feature vectors.
The sample works with Kaldi ARK or Numpy* uncompressed NPZ files, so it does not cover an end-to-end speech recognition scenario (speech to text), requiring additional preprocessing (feature extraction) to get a feature vector from a speech signal, as well as postprocessing (decoding) to produce text from scores.
Automatic Speech Recognition Python sample application demonstrates how to use the following Inference Engine Python API in applications:
Automatic Speech Recognition Python sample application demonstrates how to use the following Python API in applications:
| Feature | API | Description |
| :------------------ | :---------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| Import/Export Model | [IECore.import_network], [ExecutableNetwork.export] | The GNA plugin supports loading and saving of the GNA-optimized model |
| Network Operations | [IENetwork.batch_size], [CDataPtr.shape], [ExecutableNetwork.input_info], [ExecutableNetwork.outputs] | Managing of network: configure input and output blobs |
| Network Operations | [IENetwork.add_outputs] | Managing of network: Change names of output layers in the network |
| InferRequest Operations|InferRequest.query_state, VariableState.reset| Gets and resets state control interface for given executable network |
| Feature | API | Description |
| :---------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| Import/Export Model | [openvino.runtime.Core.import_model], [openvino.runtime.CompiledModel.export_model] | The GNA plugin supports loading and saving of the GNA-optimized model |
| Model Operations | [openvino.runtime.Model.add_outputs], [openvino.runtime.set_batch], [openvino.runtime.CompiledModel.inputs], [openvino.runtime.CompiledModel.outputs], [openvino.runtime.ConstOutput.any_name] | Managing of model: configure batch_size, input and output tensors |
| Synchronous Infer | [openvino.runtime.CompiledModel.create_infer_request], [openvino.runtime.InferRequest.infer] | Do synchronous inference |
| InferRequest Operations | [openvino.runtime.InferRequest.get_input_tensor], [openvino.runtime.InferRequest.model_outputs], [openvino.runtime.InferRequest.model_inputs], | Get info about model using infer request API |
| InferRequest Operations | [openvino.runtime.InferRequest.query_state], [openvino.runtime.VariableState.reset] | Gets and resets CompiledModel state control |
| Profiling | [openvino.runtime.InferRequest.profiling_info], [openvino.runtime.ProfilingInfo.real_time] | Get infer request profiling info |
Basic Inference Engine API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
Basic OpenVINO™ Runtime API is covered by [Hello Classification Python* Sample](../hello_classification/README.md).
| Options | Values |
| :------------------------- | :---------------------------------------------------------------------------------------------------- |
| Validated Models | Acoustic model based on Kaldi* neural networks (see [Model Preparation](#model-preparation) section) |
| Model Format | Inference Engine Intermediate Representation (.xml + .bin) |
| Options | Values |
| :------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------- |
| Validated Models | Acoustic model based on Kaldi* neural models (see [Model Preparation](#model-preparation) section) |
| Model Format | OpenVINO™ toolkit Intermediate Representation (.xml + .bin) |
| Supported devices | See [Execution Modes](#execution-modes) section below and [List Supported Devices](../../../docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md) |
| Other language realization | [C++](../../../samples/cpp/speech_sample/README.md) |
| Other language realization | [C++](../../../samples/cpp/speech_sample/README.md) |
## How It Works
At startup, the sample application reads command-line parameters, loads a specified model and input data to the Inference Engine plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream.
At startup, the sample application reads command-line parameters, loads a specified model and input data to the OpenVINO™ Runtime plugin, performs synchronous inference on all speech utterances stored in the input file, logging each step in a standard output stream.
You can see the explicit description of
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the Inference Engine with Your Application" guide.
each sample step at [Integration Steps](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md) section of "Integrate the OpenVINO™ Runtime with Your Application" guide.
## GNA-specific details
### Quantization
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA Inference Engine plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
If the GNA device is selected (for example, using the `-d` GNA flag), the GNA OpenVINO™ Runtime plugin quantizes the model and input feature vector sequence to integer representation before performing inference.
Several neural network quantization modes:
Several neural model quantization modes:
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural network is quantized to accommodate the scaled input dynamic range.
- *static* - The first utterance in the input file is scanned for dynamic range. The scale factor (floating point scalar multiplier) required to scale the maximum input value of the first utterance to 16384 (15 bits) is used for all subsequent inputs. The neural model is quantized to accommodate the scaled input dynamic range.
- *user-defined* - The user may specify a scale factor via the `-sf` flag that will be used for static quantization.
The `-qb` flag provides a hint to the GNA plugin regarding the preferred target weight resolution for all layers.
For example, when `-qb 8` is specified, the plugin will use 8-bit weights wherever possible in the
network.
model.
> **NOTE**:
>
@@ -79,29 +81,23 @@ In addition to performing inference directly from a GNA model file, these option
Run the application with the `-h` option to see the usage message:
```
python <path_to_sample>/speech_sample.py -h
python speech_sample.py -h
```
Usage message:
```
usage: speech_sample.py [-h] (-m MODEL | -rg IMPORT_GNA_MODEL) -i INPUT
[-o OUTPUT] [-r REFERENCE] [-d DEVICE] [-bs [1-8]]
[-qb [8, 16]] [-sf SCALE_FACTOR]
[-wg EXPORT_GNA_MODEL] [-we EXPORT_EMBEDDED_GNA_MODEL]
[-we_gen [GNA1, GNA3]]
[--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]] [-pc]
[-a [CORE, ATOM]] [-iname INPUT_LAYERS]
[-oname OUTPUT_LAYERS] [-cw_l CONTEXT_WINDOW_LEFT]
[-cw_r CONTEXT_WINDOW_RIGHT]
usage: speech_sample.py [-h] (-m MODEL | -rg IMPORT_GNA_MODEL) -i INPUT [-o OUTPUT] [-r REFERENCE] [-d DEVICE] [-bs [1-8]]
[-qb [8, 16]] [-sf SCALE_FACTOR] [-wg EXPORT_GNA_MODEL] [-we EXPORT_EMBEDDED_GNA_MODEL]
[-we_gen [GNA1, GNA3]] [--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]] [-pc] [-a [CORE, ATOM]]
[-iname INPUT_LAYERS] [-oname OUTPUT_LAYERS] [-cw_l CONTEXT_WINDOW_LEFT] [-cw_r CONTEXT_WINDOW_RIGHT]
[-pwl_me PWL_ME]
optional arguments:
-m MODEL, --model MODEL
Path to an .xml file with a trained model (required if
-rg is missing).
Path to an .xml file with a trained model (required if -rg is missing).
-rg IMPORT_GNA_MODEL, --import_gna_model IMPORT_GNA_MODEL
Read GNA model from file using path/filename provided
(required if -m is missing).
Read GNA model from file using path/filename provided (required if -m is missing).
Options:
-h, --help Show this help message and exit.
@@ -110,70 +106,52 @@ Options:
-o OUTPUT, --output OUTPUT
Optional. Output file name to save inference results (.ark or .npz).
-r REFERENCE, --reference REFERENCE
Optional. Read reference score file and compare
scores.
Optional. Read reference score file and compare scores.
-d DEVICE, --device DEVICE
Optional. Specify a target device to infer on. CPU,
GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32,
GNA_SW_EXACT and HETERO with combination of GNA as the
primary device and CPU as a secondary (e.g.
HETERO:GNA,CPU) are supported. The sample will look
for a suitable plugin for device specified. Default
Optional. Specify a target device to infer on. CPU, GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32,
GNA_SW_EXACT and HETERO with combination of GNA as the primary device and CPU as a secondary (e.g.
HETERO:GNA,CPU) are supported. The sample will look for a suitable plugin for device specified. Default
value is CPU.
-bs [1-8], --batch_size [1-8]
Optional. Batch size 1-8 (default 1).
-qb [8, 16], --quantization_bits [8, 16]
Optional. Weight bits for quantization: 8 or 16
(default 16).
Optional. Weight bits for quantization: 8 or 16 (default 16).
-sf SCALE_FACTOR, --scale_factor SCALE_FACTOR
Optional. The user-specified input scale factor for
quantization. If the network contains multiple inputs,
provide scale factors by separating them with commas.
Optional. The user-specified input scale factor for quantization. If the model contains multiple
inputs, provide scale factors by separating them with commas.
-wg EXPORT_GNA_MODEL, --export_gna_model EXPORT_GNA_MODEL
Optional. Write GNA model to file using path/filename
provided.
Optional. Write GNA model to file using path/filename provided.
-we EXPORT_EMBEDDED_GNA_MODEL, --export_embedded_gna_model EXPORT_EMBEDDED_GNA_MODEL
Optional. Write GNA embedded model to file using
path/filename provided.
Optional. Write GNA embedded model to file using path/filename provided.
-we_gen [GNA1, GNA3], --embedded_gna_configuration [GNA1, GNA3]
Optional. GNA generation configuration string for
embedded export. Can be GNA1 (default) or GNA3.
Optional. GNA generation configuration string for embedded export. Can be GNA1 (default) or GNA3.
--exec_target [GNA_TARGET_2_0, GNA_TARGET_3_0]
Optional. Specify GNA execution target generation. By
default, generation corresponds to the GNA HW
available in the system or the latest fully supported
generation by the software. See the GNA Plugin's
Optional. Specify GNA execution target generation. By default, generation corresponds to the GNA HW
available in the system or the latest fully supported generation by the software. See the GNA Plugin's
GNA_EXEC_TARGET config option description.
-pc, --performance_counter
Optional. Enables performance report (specify -a to
ensure arch accurate results).
Optional. Enables performance report (specify -a to ensure arch accurate results).
-a [CORE, ATOM], --arch [CORE, ATOM]
Optional. Specify architecture. CORE, ATOM with the
combination of -pc.
Optional. Specify architecture. CORE, ATOM with the combination of -pc.
-iname INPUT_LAYERS, --input_layers INPUT_LAYERS
Optional. Layer names for input blobs. The names are
separated with ",". Allows to change the order of
Optional. Layer names for input blobs. The names are separated with ",". Allows to change the order of
input layers for -i flag. Example: Input1,Input2
-oname OUTPUT_LAYERS, --output_layers OUTPUT_LAYERS
Optional. Layer names for output blobs. The names are
separated with ",". Allows to change the order of
output layers for -o flag. Example:
Output1:port,Output2:port.
Optional. Layer names for output blobs. The names are separated with ",". Allows to change the order of
output layers for -o flag. Example: Output1:port,Output2:port.
-cw_l CONTEXT_WINDOW_LEFT, --context_window_left CONTEXT_WINDOW_LEFT
Optional. Number of frames for left context windows
(default is 0). Works only with context window
networks. If you use the cw_l or cw_r flag, then batch
size argument is ignored.
Optional. Number of frames for left context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-cw_r CONTEXT_WINDOW_RIGHT, --context_window_right CONTEXT_WINDOW_RIGHT
Optional. Number of frames for right context windows
(default is 0). Works only with context window
networks. If you use the cw_l or cw_r flag, then batch
size argument is ignored.
Optional. Number of frames for right context windows (default is 0). Works only with context window
models. If you use the cw_l or cw_r flag, then batch size argument is ignored.
-pwl_me PWL_ME Optional. The maximum percent of error for PWL function. The value must be in <0, 100> range. The
default value is 1.0.
```
## Model Preparation
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural network to Inference Engine Intermediate Representation format:
You can use the following model optimizer command to convert a Kaldi nnet1 or nnet2 neural model to OpenVINO™ toolkit Intermediate Representation format:
```sh
mo --framework kaldi --input_model wsj_dnn5b.nnet --counts wsj_dnn5b.counts --remove_output_softmax --output_dir <OUTPUT_MODEL_DIR>
@@ -192,12 +170,12 @@ All of them can be downloaded from [https://storage.openvinotoolkit.org/models_c
You can do inference on Intel® Processors with the GNA co-processor (or emulation library):
```
python <path_to_sample>/speech_sample.py -m <path_to_model>/wsj_dnn5b.xml -i <path_to_ark>/dev93_10.ark -r <path_to_ark>/dev93_scores_10.ark -d GNA_AUTO -o result.npz
python speech_sample.py -m wsj_dnn5b.xml -i dev93_10.ark -r dev93_scores_10.ark -d GNA_AUTO -o result.npz
```
> **NOTES**:
>
> - Before running the sample with a trained model, make sure the model is converted to the Inference Engine format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
> - Before running the sample with a trained model, make sure the model is converted to the intermediate representation (IR) format (\*.xml + \*.bin) using the [Model Optimizer tool](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
>
> - The sample supports input and output in numpy file format (.npz)
@@ -206,47 +184,168 @@ python <path_to_sample>/speech_sample.py -m <path_to_model>/wsj_dnn5b.xml -i <pa
The sample application logs each step in a standard output stream.
```
[ INFO ] Creating Inference Engine
[ INFO ] Reading the network: wsj_dnn5b.xml
[ INFO ] Configuring input and output blobs
[ INFO ] Creating OpenVINO Runtime Core
[ INFO ] Reading the model: /models/wsj_dnn5b_smbr_fp32.xml
[ INFO ] Using scale factor(s) calculated from first utterance
[ INFO ] For input 0 using scale factor of 2175.4322418
[ INFO ] Loading the model to the plugin
[ INFO ] Starting inference in synchronous mode
[ INFO ] Utterance 0 (4k0c0301)
[ INFO ] Output blob name: affinetransform14/Fused_Add_
[ INFO ]
[ INFO ] Utterance 0:
[ INFO ] Total time in Infer (HW and SW): 6326.06ms
[ INFO ] Frames in utterance: 1294
[ INFO ] Total time in Infer (HW and SW): 6211.45ms
[ INFO ] max error: 0.7051840
[ INFO ] avg error: 0.0448388
[ INFO ] avg rms error: 0.0582387
[ INFO ] stdev error: 0.0371650
[ INFO ] Average Infer time per frame: 4.89ms
[ INFO ]
[ INFO ] Utterance 1 (4k0c0302)
[ INFO ] Output blob name: affinetransform14/Fused_Add_
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7051840
[ INFO ] avg error: 0.0448388
[ INFO ] avg rms error: 0.0582387
[ INFO ] stdev error: 0.0371650
[ INFO ]
[ INFO ] Utterance 1:
[ INFO ] Total time in Infer (HW and SW): 4526.57ms
[ INFO ] Frames in utterance: 1005
[ INFO ] Total time in Infer (HW and SW): 4742.27ms
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7575974
[ INFO ] avg error: 0.0452166
[ INFO ] avg error: 0.0452166
[ INFO ] avg rms error: 0.0586013
[ INFO ] stdev error: 0.0372769
...
[ INFO ] Total sample time: 40219.99ms
[ INFO ] stdev error: 0.0372769
[ INFO ]
[ INFO ] Utterance 2:
[ INFO ] Total time in Infer (HW and SW): 6636.56ms
[ INFO ] Frames in utterance: 1471
[ INFO ] Average Infer time per frame: 4.51ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7191710
[ INFO ] avg error: 0.0472226
[ INFO ] avg rms error: 0.0612991
[ INFO ] stdev error: 0.0390846
[ INFO ]
[ INFO ] Utterance 3:
[ INFO ] Total time in Infer (HW and SW): 3927.01ms
[ INFO ] Frames in utterance: 845
[ INFO ] Average Infer time per frame: 4.65ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7436461
[ INFO ] avg error: 0.0477581
[ INFO ] avg rms error: 0.0621334
[ INFO ] stdev error: 0.0397457
[ INFO ]
[ INFO ] Utterance 4:
[ INFO ] Total time in Infer (HW and SW): 3891.49ms
[ INFO ] Frames in utterance: 855
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7071600
[ INFO ] avg error: 0.0449147
[ INFO ] avg rms error: 0.0585048
[ INFO ] stdev error: 0.0374897
[ INFO ]
[ INFO ] Utterance 5:
[ INFO ] Total time in Infer (HW and SW): 3378.61ms
[ INFO ] Frames in utterance: 699
[ INFO ] Average Infer time per frame: 4.83ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.8870468
[ INFO ] avg error: 0.0479243
[ INFO ] avg rms error: 0.0625490
[ INFO ] stdev error: 0.0401951
[ INFO ]
[ INFO ] Utterance 6:
[ INFO ] Total time in Infer (HW and SW): 4034.31ms
[ INFO ] Frames in utterance: 790
[ INFO ] Average Infer time per frame: 5.11ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7648273
[ INFO ] avg error: 0.0482702
[ INFO ] avg rms error: 0.0629734
[ INFO ] stdev error: 0.0404429
[ INFO ]
[ INFO ] Utterance 7:
[ INFO ] Total time in Infer (HW and SW): 2854.04ms
[ INFO ] Frames in utterance: 622
[ INFO ] Average Infer time per frame: 4.59ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.7389560
[ INFO ] avg error: 0.0465543
[ INFO ] avg rms error: 0.0604941
[ INFO ] stdev error: 0.0386294
[ INFO ]
[ INFO ] Utterance 8:
[ INFO ] Total time in Infer (HW and SW): 2493.28ms
[ INFO ] Frames in utterance: 548
[ INFO ] Average Infer time per frame: 4.55ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6680136
[ INFO ] avg error: 0.0439341
[ INFO ] avg rms error: 0.0574614
[ INFO ] stdev error: 0.0370353
[ INFO ]
[ INFO ] Utterance 9:
[ INFO ] Total time in Infer (HW and SW): 1654.67ms
[ INFO ] Frames in utterance: 368
[ INFO ] Average Infer time per frame: 4.50ms
[ INFO ]
[ INFO ] Output blob name: affinetransform14
[ INFO ] Number scores per frame: 3425
[ INFO ]
[ INFO ] max error: 0.6550579
[ INFO ] avg error: 0.0467643
[ INFO ] avg rms error: 0.0605045
[ INFO ] stdev error: 0.0383914
[ INFO ]
[ INFO ] Total sample time: 39722.60ms
[ INFO ] File result.npz was created!
[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool
```
## See Also
- [Integrate the Inference Engine with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using Inference Engine Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Integrate the OpenVINO™ Runtime with Your Application](../../../docs/OV_Runtime_UG/Integrate_with_customer_application_new_API.md)
- [Using OpenVINO™ Toolkit Samples](../../../docs/OV_Runtime_UG/Samples_Overview.md)
- [Model Downloader](@ref omz_tools_downloader)
- [Model Optimizer](../../../docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
[IENetwork.batch_size]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#a79a647cb1b49645616eaeb2ca255ef2e
[IENetwork.add_outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IENetwork.html#ae8024b07f3301d6d5de5c0d153e2e6e6
[CDataPtr.shape]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1CDataPtr.html#aa6fd459edb323d1c6215dc7a970ebf7f
[ExecutableNetwork.input_info]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#ac76a04c2918607874018d2e15a2f274f
[ExecutableNetwork.outputs]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#a4a631776df195004b1523e6ae91a65c1
[IECore.import_network]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1IECore.html#afdeac5192bb1d9e64722f1071fb0a64a
[ExecutableNetwork.export]:https://docs.openvino.ai/latest/ie_python_api/classie__api_1_1ExecutableNetwork.html#afa78158252f0d8070181bafec4318413
<!-- [openvino.runtime.Core.import_model]:
[openvino.runtime.CompiledModel.export_model]:
[openvino.runtime.Model.add_outputs]:
[openvino.runtime.set_batch]:
[openvino.runtime.CompiledModel.inputs]:
[openvino.runtime.CompiledModel.outputs]:
[openvino.runtime.ConstOutput.any_name]:
[openvino.runtime.CompiledModel.create_infer_request]:
[openvino.runtime.InferRequest.infer]:
[openvino.runtime.InferRequest.get_input_tensor]:
[openvino.runtime.InferRequest.model_outputs]:
[openvino.runtime.InferRequest.model_inputs]:
[openvino.runtime.InferRequest.query_state]:
[openvino.runtime.VariableState.reset]:
[openvino.runtime.InferRequest.profiling_info]:
[openvino.runtime.ProfilingInfo.real_time]: -->

22
samples/python/speech_sample/arg_parser.py
View File

@@ -4,8 +4,8 @@
import argparse
def parse_args() -> argparse.Namespace:
"""Parse and return command line arguments"""
def build_arg_parser() -> argparse.ArgumentParser:
"""Create and return argument parser"""
parser = argparse.ArgumentParser(add_help=False)
args = parser.add_argument_group('Options')
model = parser.add_mutually_exclusive_group(required=True)
@@ -25,13 +25,15 @@ def parse_args() -> argparse.Namespace:
'CPU, GPU, MYRIAD, GNA_AUTO, GNA_HW, GNA_SW_FP32, GNA_SW_EXACT and HETERO with combination of GNA'
' as the primary device and CPU as a secondary (e.g. HETERO:GNA,CPU) are supported. '
'The sample will look for a suitable plugin for device specified. Default value is CPU.')
args.add_argument('-bs', '--batch_size', default=1, type=int, choices=range(1, 9), metavar='[1-8]',
help='Optional. Batch size 1-8 (default 1).')
args.add_argument('-bs', '--batch_size', type=int, choices=range(1, 9), metavar='[1-8]',
help='Optional. Batch size 1-8.')
args.add_argument('-layout', type=str,
help='Optional. Custom layout in format: "input0[value0],input1[value1]" or "[value]" (applied to all inputs)')
args.add_argument('-qb', '--quantization_bits', default=16, type=int, choices=(8, 16), metavar='[8, 16]',
help='Optional. Weight bits for quantization: 8 or 16 (default 16).')
args.add_argument('-sf', '--scale_factor', type=str,
help='Optional. The user-specified input scale factor for quantization. '
'If the network contains multiple inputs, provide scale factors by separating them with commas.')
'If the model contains multiple inputs, provide scale factors by separating them with commas.')
args.add_argument('-wg', '--export_gna_model', type=str,
help='Optional. Write GNA model to file using path/filename provided.')
args.add_argument('-we', '--export_embedded_gna_model', type=str,
@@ -57,16 +59,22 @@ def parse_args() -> argparse.Namespace:
'Allows to change the order of output layers for -o flag. Example: Output1:port,Output2:port.')
args.add_argument('-cw_l', '--context_window_left', type=int, default=0,
help='Optional. Number of frames for left context windows (default is 0). '
'Works only with context window networks. '
'Works only with context window models. '
'If you use the cw_l or cw_r flag, then batch size argument is ignored.')
args.add_argument('-cw_r', '--context_window_right', type=int, default=0,
help='Optional. Number of frames for right context windows (default is 0). '
'Works only with context window networks. '
'Works only with context window models. '
'If you use the cw_l or cw_r flag, then batch size argument is ignored.')
args.add_argument('-pwl_me', type=float, default=1.0,
help='Optional. The maximum percent of error for PWL function. '
'The value must be in <0, 100> range. The default value is 1.0.')
return parser
def parse_args() -> argparse.Namespace:
"""Parse and validate command-line arguments"""
parser = build_arg_parser()
args = parser.parse_args()
if args.context_window_left < 0:

36
samples/python/speech_sample/speech_sample.py
View File

@@ -17,8 +17,8 @@ from arg_parser import parse_args
from file_options import read_utterance_file, write_utterance_file
from utils import (GNA_ATOM_FREQUENCY, GNA_CORE_FREQUENCY,
compare_with_reference, get_scale_factor, log,
parse_outputs_from_args, parse_scale_factors,
set_scale_factors)
parse_input_layouts, parse_outputs_from_args,
parse_scale_factors, set_scale_factors)
def do_inference(data: Dict[str, np.ndarray], infer_request: InferRequest, cw_l: int = 0, cw_r: int = 0) -> np.ndarray:
@@ -26,7 +26,7 @@ def do_inference(data: Dict[str, np.ndarray], infer_request: InferRequest, cw_l:
frames_to_infer = {}
result = {}
batch_size = infer_request.get_input_tensor(0).shape[0]
batch_size = infer_request.model_inputs[0].shape[0]
num_of_frames = next(iter(data.values())).shape[0]
for output in infer_request.model_outputs:
@@ -83,24 +83,32 @@ def main():
output_layer_names, output_layer_ports = parse_outputs_from_args(args)
model.add_outputs(list(zip(output_layer_names, output_layer_ports)))
if args.layout:
layouts = parse_input_layouts(args, model.inputs)
ppp = PrePostProcessor(model)
for i in range(len(model.inputs)):
ppp.input(i).tensor() \
.set_element_type(Type.f32) \
.set_layout(Layout('NC')) # noqa: N400
ppp.input(i).tensor().set_element_type(Type.f32)
ppp.input(i).model().set_layout(Layout('NC'))
input_name = model.input(i).get_any_name()
if args.layout and input_name in layouts.keys():
ppp.input(i).tensor().set_layout(Layout(layouts[input_name]))
ppp.input(i).model().set_layout(Layout(layouts[input_name]))
for i in range(len(model.outputs)):
ppp.output(i).tensor().set_element_type(Type.f32)
model = ppp.build()
if args.context_window_left == args.context_window_right == 0:
set_batch(model, args.batch_size)
else:
set_batch(model, 1)
if args.batch_size:
batch_size = args.batch_size if args.context_window_left == args.context_window_right == 0 else 1
if any([not _input.node.layout.empty for _input in model.inputs]):
set_batch(model, batch_size)
else:
log.warning('Layout is not set for any input, so custom batch size is not set')
# ---------------------------Step 4. Configure plugin ---------------------------------------------------------
devices = args.device.replace('HETERO:', '').split(',')
@@ -175,7 +183,7 @@ def main():
input_file_names = re.split(', |,', args.input)
if len(input_layer_names) != len(input_file_names):
log.error(f'Number of network inputs ({len(compiled_model.inputs)}) is not equal '
log.error(f'Number of model inputs ({len(compiled_model.inputs)}) is not equal '
f'to number of ark files ({len(input_file_names)})')
sys.exit(-3)
@@ -201,14 +209,14 @@ def main():
output_file_names = re.split(', |,', args.output)
if len(output_layer_names) != len(output_file_names):
log.error('The number of output files is not equal to the number of network outputs.')
log.error('The number of output files is not equal to the number of model outputs.')
sys.exit(-6)
if args.reference:
reference_file_names = re.split(', |,', args.reference)
if len(output_layer_names) != len(reference_file_names):
log.error('The number of reference files is not equal to the number of network outputs.')
log.error('The number of reference files is not equal to the number of model outputs.')
sys.exit(-5)
reference_file_data = [read_utterance_file(file_name) for file_name in reference_file_names]

10
samples/python/speech_sample/utils.py
View File

@@ -6,9 +6,10 @@ import argparse
import logging as log
import re
import sys
from typing import List, Tuple
from typing import Dict, List, Tuple
import numpy as np
from openvino.runtime import Output
# Operating Frequency for GNA HW devices for Core and Atom architecture
GNA_CORE_FREQUENCY = 400
@@ -78,3 +79,10 @@ def parse_outputs_from_args(args: argparse.Namespace) -> Tuple[List[str], List[i
except ValueError:
log.error('Incorrect value for -oname/--output_layers option, please specify a port for each output layer.')
sys.exit(-4)
def parse_input_layouts(args: argparse.Namespace, inputs: List[Output]) -> Dict[str, str]:
if ',' in args.layout:
return dict([_input.split('[') for _input in args.layout[:-1].split('],')])
else:
return {_input.get_any_name(): args.layout[1:-1] for _input in inputs}

1
src/bindings/python/CMakeLists.txt
View File

@@ -16,6 +16,7 @@ endif()
set(LIBRARY_OUTPUT_DIRECTORY_BIN ${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
set(PYTHON_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/src)
set(PYTHON_BINDINGS_ROOT_DIR ${CMAKE_CURRENT_SOURCE_DIR})
# Remove this condition as 76672 will be fixed
if (CMAKE_CROSSCOMPILING AND DEFINED ENV{OE_CMAKE_TOOLCHAIN_FILE})

1
src/bindings/python/src/compatibility/ngraph/__init__.py
View File

@@ -17,7 +17,6 @@ from ngraph.impl import Node
from ngraph.impl import PartialShape
from ngraph.helpers import function_from_cnn
from ngraph.helpers import function_to_cnn
from ngraph.helpers import partial_shape_from_data
from ngraph.opset8 import absolute
from ngraph.opset8 import absolute as abs
from ngraph.opset8 import acos

10
src/bindings/python/src/compatibility/ngraph/helpers.py
View File

@@ -5,8 +5,8 @@
from typing import Union
from ngraph.impl import Function, PartialShape
from openvino.inference_engine import IENetwork, DataPtr, CDataPtr # type: ignore
from ngraph.impl import Function
from openvino.inference_engine import IENetwork # type: ignore
def function_from_cnn(cnn_network: IENetwork) -> Function:
@@ -20,9 +20,3 @@ def function_to_cnn(ng_function: Function) -> Function:
"""Get Inference Engine CNN network from nGraph function."""
capsule = Function.to_capsule(ng_function)
return IENetwork(capsule)
def partial_shape_from_data(data: Union[DataPtr, CDataPtr]) -> PartialShape:
"""Get nGraph PartialShape from Inference Engine Data."""
capsule = data._get_partial_shape_capsule()
return PartialShape.from_capsule(capsule)

266
src/bindings/python/src/compatibility/ngraph/opset8/ops.py
View File

@@ -51,22 +51,22 @@ def deformable_convolution(
) -> Node:
"""Return a node which performs deformable convolution operation.
@param data: The node providing data batch tensor.
@param offsets: The node providing offset tensor.
@param filters: The node providing filters tensor.
@param strides: The distance (in pixels) to slide the filter on the feature map over the axes.
@param pads_begin: The number of pixels to add to the beginning along each axis.
@param pads_end: The number of pixels to add to the end along each axis.
@param dilations: The distance in width and height between elements (weights) in the filter.
@param mask: The node providing modulation scalar (mask) tensor.
@param auto_pad: The type of padding. Range of values: explicit, same_upper, same_lower, valid.
@param group: The number of groups which both output and input should be split into.
@param deformable_group: The number of groups which deformable values and output should be split
:param data: The node providing data batch tensor.
:param offsets: The node providing offset tensor.
:param filters: The node providing filters tensor.
:param strides: The distance (in pixels) to slide the filter on the feature map over the axes.
:param pads_begin: The number of pixels to add to the beginning along each axis.
:param pads_end: The number of pixels to add to the end along each axis.
:param dilations: The distance in width and height between elements (weights) in the filter.
:param mask: The node providing modulation scalar (mask) tensor.
:param auto_pad: The type of padding. Range of values: explicit, same_upper, same_lower, valid.
:param group: The number of groups which both output and input should be split into.
:param deformable_group: The number of groups which deformable values and output should be split
into along the channel axis.
@param bilinear_interpolation_pad: The flag that determines the mode of bilinear interpolation
:param bilinear_interpolation_pad: The flag that determines the mode of bilinear interpolation
execution.
@param name: The optional new name for output node.
@return New node performing deformable convolution operation.
:param name: The optional new name for output node.
:return: New node performing deformable convolution operation.
"""
if mask is None:
inputs = as_nodes(data, offsets, filters)
@@ -96,9 +96,9 @@ def adaptive_avg_pool(
) -> Node:
"""Return a node which performs AdaptiveAvgPool operation.
@param data: The list of input nodes
@param output_shape: the shape of spatial dimentions after operation
@return: The new node performing AdaptiveAvgPool operation on the data
:param data: The list of input nodes
:param output_shape: the shape of spatial dimentions after operation
:return: The new node performing AdaptiveAvgPool operation on the data
"""
inputs = as_nodes(data, output_shape)
return _get_node_factory_opset8().create("AdaptiveAvgPool", inputs)
@@ -112,10 +112,10 @@ def adaptive_max_pool(
) -> Node:
"""Return a node which performs AdaptiveMaxPool operation.
@param data: The list of input nodes
@param output_shape: the shape of spatial dimentions after operation
@param index_element_type: Type of indices output.
@return: The new node performing AdaptiveMaxPool operation on the data
:param data: The list of input nodes
:param output_shape: the shape of spatial dimentions after operation
:param index_element_type: Type of indices output.
:return: The new node performing AdaptiveMaxPool operation on the data
"""
inputs = as_nodes(data, output_shape)
@@ -143,26 +143,26 @@ def multiclass_nms(
) -> Node:
"""Return a node which performs MulticlassNms.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param sort_result_type: Specifies order of output elements, possible values:
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param sort_result_type: Specifies order of output elements, possible values:
'class': sort selected boxes by class id (ascending)
'score': sort selected boxes by score (descending)
'none': do not guarantee the order.
@param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
:param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
across batches or not
@param output_type: Specifies the output tensor type, possible values:
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32'
@param iou_threshold: Specifies intersection over union threshold
@param score_threshold: Specifies minimum score to consider box for the processing
@param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
:param iou_threshold: Specifies intersection over union threshold
:param score_threshold: Specifies minimum score to consider box for the processing
:param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
to keep all boxes
@param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
:param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
meaning to keep all boxes
@param background_class: Specifies the background class id, -1 meaning to keep all classes
@param nms_eta: Specifies eta parameter for adpative NMS, in close range [0, 1.0]
@param normalized: Specifies whether boxes are normalized or not
@return: The new node which performs MuticlassNms
:param background_class: Specifies the background class id, -1 meaning to keep all classes
:param nms_eta: Specifies eta parameter for adpative NMS, in close range [0, 1.0]
:param normalized: Specifies whether boxes are normalized or not
:return: The new node which performs MuticlassNms
"""
inputs = as_nodes(boxes, scores)
@@ -200,29 +200,29 @@ def matrix_nms(
) -> Node:
"""Return a node which performs MatrixNms.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param sort_result_type: Specifies order of output elements, possible values:
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param sort_result_type: Specifies order of output elements, possible values:
'class': sort selected boxes by class id (ascending)
'score': sort selected boxes by score (descending)
'none': do not guarantee the order.
@param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
:param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
across batches or not
@param output_type: Specifies the output tensor type, possible values:
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32'
@param score_threshold: Specifies minimum score to consider box for the processing
@param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
:param score_threshold: Specifies minimum score to consider box for the processing
:param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
to keep all boxes
@param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
:param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
meaning to keep all boxes
@param background_class: Specifies the background class id, -1 meaning to keep all classes
@param decay_function: Specifies decay function used to decay scores, possible values:
:param background_class: Specifies the background class id, -1 meaning to keep all classes
:param decay_function: Specifies decay function used to decay scores, possible values:
'gaussian', 'linear'
@param gaussian_sigma: Specifies gaussian_sigma parameter for gaussian decay_function
@param post_threshold: Specifies threshold to filter out boxes with low confidence score
:param gaussian_sigma: Specifies gaussian_sigma parameter for gaussian decay_function
:param post_threshold: Specifies threshold to filter out boxes with low confidence score
after decaying
@param normalized: Specifies whether boxes are normalized or not
@return: The new node which performs MatrixNms
:param normalized: Specifies whether boxes are normalized or not
:return: The new node which performs MatrixNms
"""
inputs = as_nodes(boxes, scores)
@@ -252,12 +252,12 @@ def gather(
) -> Node:
"""Return a node which performs Gather with support of negative indices.
@param data: N-D tensor with data for gathering
@param indices: N-D tensor with indices by which data is gathered. Negative indices
:param data: N-D tensor with data for gathering
:param indices: N-D tensor with indices by which data is gathered. Negative indices
indicate reverse indexing from the end
@param axis: axis along which elements are gathered
@param batch_dims: number of batch dimensions
@return: The new node which performs Gather
:param axis: axis along which elements are gathered
:param batch_dims: number of batch dimensions
:return: The new node which performs Gather
"""
inputs = as_nodes(data, indices, axis)
attributes = {
@@ -282,25 +282,25 @@ def max_pool(
) -> Node:
"""Perform max pooling operation and return both values and indices of the selected elements.
@param data: The node providing input data.
@param strides: The distance (in pixels) to slide the filter on the feature map
:param data: The node providing input data.
:param strides: The distance (in pixels) to slide the filter on the feature map
over the axes.
@param dilations: The dilation of filter elements(distance between elements).
@param pads_begin: The number of pixels to add at the beginning along each axis.
@param pads_end: The number of pixels to add at the end along each axis.
@param kernel_shape: The pooling operation kernel shape.
@param rounding_type: Determines used rounding schema when computing output shape.
:param dilations: The dilation of filter elements(distance between elements).
:param pads_begin: The number of pixels to add at the beginning along each axis.
:param pads_end: The number of pixels to add at the end along each axis.
:param kernel_shape: The pooling operation kernel shape.
:param rounding_type: Determines used rounding schema when computing output shape.
Acceptable values are: ['floor', 'ceil']. Defaults to 'floor'.
@param auto_pad: Determines how the padding is calculated. Acceptable values:
:param auto_pad: Determines how the padding is calculated. Acceptable values:
[None, 'same_upper', 'same_lower', 'valid']. Defaults to None.
@param index_element_type: The data type used for the indices output of this operator.
:param index_element_type: The data type used for the indices output of this operator.
Defaults to i64.
@param axis: The first dimension in the data shape used to determine the maximum
:param axis: The first dimension in the data shape used to determine the maximum
returned index value. The value is the product of all dimensions
starting at the provided axis. Defaults to 0.
@param name: The optional name for the created output node.
:param name: The optional name for the created output node.
@return The new node performing max pooling operation.
:return: The new node performing max pooling operation.
"""
if auto_pad is None:
auto_pad = "explicit"
@@ -332,14 +332,14 @@ def random_uniform(
) -> Node:
"""Return a node which generates sequence of random values from uniform distribution.
@param output_shape: Tensor with shape of the output tensor.
@param min_val: Tensor with the lower bound on the range of random values to generate.
@param max_val: Tensor with the upper bound on the range of random values to generate.
@param output_type: Specifies the output tensor type, possible values:
:param output_shape: Tensor with shape of the output tensor.
:param min_val: Tensor with the lower bound on the range of random values to generate.
:param max_val: Tensor with the upper bound on the range of random values to generate.
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32', 'f64', 'f32', 'f16', 'bf16'.
@param global_seed: Specifies global seed value. Required to be a positive integer or 0.
@param op_seed: Specifies operational seed value. Required to be a positive integer or 0.
@return The new node which performs generation of random values from uniform distribution.
:param global_seed: Specifies global seed value. Required to be a positive integer or 0.
:param op_seed: Specifies operational seed value. Required to be a positive integer or 0.
:return: The new node which performs generation of random values from uniform distribution.
"""
inputs = as_nodes(output_shape, min_val, max_val)
@@ -368,18 +368,18 @@ def if_op(
) -> Node:
"""Execute one of the bodies depending on condtion value.
@param condition: A scalar or 1D tensor with 1 element specifying body will be executed.
:param condition: A scalar or 1D tensor with 1 element specifying body will be executed.
If condition is True, then body will be executed, False - else_body.
@param inputs: The provided inputs to If operation.
@param bodies: Two graphs (then_body, else_body) which will be executed depending on
:param inputs: The provided inputs to If operation.
:param bodies: Two graphs (then_body, else_body) which will be executed depending on
condition value.
@param input_desc Two lists (for then_body and else_body) which contain rules how If
:param input_desc Two lists (for then_body and else_body) which contain rules how If
inputs are connected with body parameters.
@param output_desc: Two lists (for then_body and else_body) which contain rules how If
:param output_desc: Two lists (for then_body and else_body) which contain rules how If
outputs are connected with body results.
@param name: The optional name for the created output node.
:param name: The optional name for the created output node.
@return: The new node which performs If operation.
:return: The new node which performs If operation.
"""
attributes = {
"then_body": bodies[0].serialize(),
@@ -404,13 +404,13 @@ def slice(
) -> Node:
"""Return a node which generates Slice operation.
@param data: The node providing input data.
@param start: The node providing start indices (inclusively).
@param stop: The node providing stop indices (exclusively).
@param step: The node providing step values.
@param axes: The optional node providing axes to slice, default [0, 1, ..., len(start)-1].
@param name: The optional name for the created output node.
@return The new node performing Slice operation.
:param data: The node providing input data.
:param start: The node providing start indices (inclusively).
:param stop: The node providing stop indices (exclusively).
:param step: The node providing step values.
:param axes: The optional node providing axes to slice, default [0, 1, ..., len(start)-1].
:param name: The optional name for the created output node.
:return: The new node performing Slice operation.
"""
if axes is None:
inputs = as_nodes(data, start, stop, step)
@@ -429,10 +429,10 @@ def gather_nd(
) -> Node:
"""Return a node which performs GatherND.
@param data: N-D tensor with data for gathering
@param indices: K-D tensor of tuples with indices by which data is gathered
@param batch_dims: Scalar value of batch dimensions
@return: The new node which performs GatherND
:param data: N-D tensor with data for gathering
:param indices: K-D tensor of tuples with indices by which data is gathered
:param batch_dims: Scalar value of batch dimensions
:return: The new node which performs GatherND
"""
inputs = as_nodes(data, indices)
@@ -448,12 +448,7 @@ def prior_box(
) -> Node:
"""Generate prior boxes of specified sizes and aspect ratios across all dimensions.
@param layer_shape: Shape of layer for which prior boxes are computed.
@param image_shape: Shape of image to which prior boxes are scaled.
@param attrs: The dictionary containing key, value pairs for attributes.
@param name: Optional name for the output node.
@return Node representing prior box operation.
Available attributes are:
Available attributes are:
* min_size The minimum box size (in pixels).
Range of values: positive floating point numbers
Default value: []
@@ -513,7 +508,9 @@ def prior_box(
Default value: True
Required: no
Example of attribute dictionary:
@code{.py}
.. code-block:: python
# just required ones
attrs = {
'offset': 85,
@@ -524,8 +521,14 @@ def prior_box(
'clip': True,
'fixed_size': [32, 64, 128]
}
@endcode
Optional attributes which are absent from dictionary will be set with corresponding default.
:param layer_shape: Shape of layer for which prior boxes are computed.
:param image_shape: Shape of image to which prior boxes are scaled.
:param attrs: The dictionary containing key, value pairs for attributes.
:param name: Optional name for the output node.
:return: Node representing prior box operation.
"""
requirements = [
("offset", True, np.floating, is_non_negative_value),
@@ -557,11 +560,11 @@ def i420_to_bgr(
) -> Node:
"""Return a node which performs I420toBGR operation.
@param arg: The node providing single or Y plane data.
@param arg_u: The node providing U plane data. Required for separate planes.
@param arg_v: The node providing V plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing I420toBGR operation.
:param arg: The node providing single or Y plane data.
:param arg_u: The node providing U plane data. Required for separate planes.
:param arg_v: The node providing V plane data. Required for separate planes.
:param name: The optional name for the created output node.
:return: The new node performing I420toBGR operation.
"""
if arg_u is None and arg_v is None:
inputs = as_nodes(arg)
@@ -584,11 +587,11 @@ def i420_to_rgb(
) -> Node:
"""Return a node which performs I420toRGB operation.
@param arg: The node providing single or Y plane data.
@param arg_u: The node providing U plane data. Required for separate planes.
@param arg_v: The node providing V plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing I420toRGB operation.
:param arg: The node providing single or Y plane data.
:param arg_u: The node providing U plane data. Required for separate planes.
:param arg_v: The node providing V plane data. Required for separate planes.
:param name: The optional name for the created output node.
:return: The new node performing I420toRGB operation.
"""
if arg_u is None and arg_v is None:
inputs = as_nodes(arg)
@@ -610,10 +613,10 @@ def nv12_to_bgr(
) -> Node:
"""Return a node which performs NV12toBGR operation.
@param arg: The node providing single or Y plane data.
@param arg_uv: The node providing UV plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing NV12toBGR operation.
:param arg: The node providing single or Y plane data.
:param arg_uv: The node providing UV plane data. Required for separate planes.
:param name: The optional name for the created output node.
:return: The new node performing NV12toBGR operation.
"""
if arg_uv is None:
inputs = as_nodes(arg)
@@ -631,10 +634,10 @@ def nv12_to_rgb(
) -> Node:
"""Return a node which performs NV12toRGB operation.
@param arg: The node providing single or Y plane data.
@param arg_uv: The node providing UV plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing NV12toRGB operation.
:param arg: The node providing single or Y plane data.
:param arg_uv: The node providing UV plane data. Required for separate planes.
:param name: The optional name for the created output node.
:return: The new node performing NV12toRGB operation.
"""
if arg_uv is None:
inputs = as_nodes(arg)
@@ -656,14 +659,6 @@ def detection_output(
) -> Node:
"""Generate the detection output using information on location and confidence predictions.
@param box_logits: The 2D input tensor with box logits.
@param class_preds: The 2D input tensor with class predictions.
@param proposals: The 3D input tensor with proposals.
@param attrs: The dictionary containing key, value pairs for attributes.
@param aux_class_preds: The 2D input tensor with additional class predictions information.
@param aux_box_preds: The 2D input tensor with additional box predictions information.
@param name: Optional name for the output node.
@return Node representing DetectionOutput operation.
Available attributes are:
* background_label_id The background label id.
Range of values: integer value
@@ -732,7 +727,9 @@ def detection_output(
Default value: 0
Required: no
Example of attribute dictionary:
@code{.py}
.. code-block:: python
# just required ones
attrs = {
'keep_top_k': [1, 2, 3],
@@ -746,8 +743,17 @@ def detection_output(
'input_height': [32],
'input_width': [32],
}
@endcode
Optional attributes which are absent from dictionary will be set with corresponding default.
:param box_logits: The 2D input tensor with box logits.
:param class_preds: The 2D input tensor with class predictions.
:param proposals: The 3D input tensor with proposals.
:param attrs: The dictionary containing key, value pairs for attributes.
:param aux_class_preds: The 2D input tensor with additional class predictions information.
:param aux_box_preds: The 2D input tensor with additional box predictions information.
:param name: Optional name for the output node.
:return: Node representing DetectionOutput operation.
"""
requirements = [
("background_label_id", False, np.integer, None),
@@ -783,9 +789,9 @@ def detection_output(
def softmax(data: NodeInput, axis: int, name: Optional[str] = None) -> Node:
"""Apply softmax operation on each element of input tensor.
@param data: The tensor providing input data.
@param axis: An axis along which Softmax should be calculated. Can be positive or negative.
@param name: Optional name for the node
@return The new node with softmax operation applied on each element.
:param data: The tensor providing input data.
:param axis: An axis along which Softmax should be calculated. Can be positive or negative.
:param name: Optional name for the node
:return: The new node with softmax operation applied on each element.
"""
return _get_node_factory_opset8().create("Softmax", [as_node(data)], {"axis": axis})

36
src/bindings/python/src/compatibility/ngraph/utils/node_factory.py
View File

@@ -21,7 +21,7 @@ class NodeFactory(object):
def __init__(self, opset_version: str = DEFAULT_OPSET) -> None:
"""Create the NodeFactory object.
@param opset_version: The opset version the factory will use to produce ops from.
:param opset_version: The opset version the factory will use to produce ops from.
"""
self.factory = _NodeFactory(opset_version)
@@ -35,11 +35,11 @@ class NodeFactory(object):
The user does not have to provide all node's attributes, but only required ones.
@param op_type_name: The operator type name.
@param arguments: The operator arguments.
@param attributes: The operator attributes.
:param op_type_name: The operator type name.
:param arguments: The operator arguments.
:param attributes: The operator attributes.
@return Node object representing requested operator with attributes set.
returns Node object representing requested operator with attributes set.
"""
if arguments is None and attributes is None:
node = self.factory.create(op_type_name)
@@ -111,10 +111,10 @@ class NodeFactory(object):
def _normalize_attr_name(attr_name: str, prefix: str) -> str:
"""Normalize attribute name.
@param attr_name: The attribute name.
@param prefix: The prefix to attach to attribute name.
:param attr_name: The attribute name.
:param prefix: The prefix to attach to attribute name.
@return The modified attribute name.
returns The modified attribute name.
"""
# Trim first part of the name if there is only one level of attribute hierarchy.
if attr_name.count(".") == 1:
@@ -125,9 +125,9 @@ class NodeFactory(object):
def _normalize_attr_name_getter(cls, attr_name: str) -> str:
"""Normalize atr name to be suitable for getter function name.
@param attr_name: The attribute name to normalize
:param attr_name: The attribute name to normalize
@return The appropriate getter function name.
returns The appropriate getter function name.
"""
return cls._normalize_attr_name(attr_name, "get_")
@@ -135,9 +135,9 @@ class NodeFactory(object):
def _normalize_attr_name_setter(cls, attr_name: str) -> str:
"""Normalize attribute name to be suitable for setter function name.
@param attr_name: The attribute name to normalize
:param attr_name: The attribute name to normalize
@return The appropriate setter function name.
returns The appropriate setter function name.
"""
return cls._normalize_attr_name(attr_name, "set_")
@@ -145,10 +145,10 @@ class NodeFactory(object):
def _get_node_attr_value(node: Node, attr_name: str) -> Any:
"""Get provided node attribute value.
@param node: The node we retrieve attribute value from.
@param attr_name: The attribute name.
:param node: The node we retrieve attribute value from.
:param attr_name: The attribute name.
@return The node attribute value.
returns The node attribute value.
"""
if not node._attr_cache_valid:
node._attr_cache = node.get_attributes()
@@ -159,9 +159,9 @@ class NodeFactory(object):
def _set_node_attr_value(node: Node, attr_name: str, value: Any) -> None:
"""Set the node attribute value.
@param node: The node we change attribute value for.
@param attr_name: The attribute name.
@param value: The new attribute value.
:param node: The node we change attribute value for.
:param attr_name: The attribute name.
:param value: The new attribute value.
"""
node.set_attribute(attr_name, value)
node._attr_cache[attr_name] = value

2
src/bindings/python/src/compatibility/openvino/inference_engine/ie_api.pxd
View File

@@ -39,7 +39,7 @@ cdef class InferRequest:
cpdef get_perf_counts(self)
cdef void user_callback(self, int status) with gil
cdef public:
_inputs_list, _outputs_list, _py_callback, _py_data, _user_blobs, _inputs_is_dynamic
_inputs_list, _outputs_list, _py_callback, _py_data, _user_blobs
cdef class IENetwork:
cdef C.IENetwork impl

44
src/bindings/python/src/compatibility/openvino/inference_engine/ie_api.pyx
View File

@@ -52,11 +52,6 @@ cdef c_map_to_dict(map[string, string] c_map):
return py_dict
cdef expand_dims_to_corresponding_layout(shape, layout):
single_axes = [1] * (len(layout) - len(shape))
return single_axes + list(shape)
def get_version():
return C.get_version().decode()
@@ -312,6 +307,7 @@ cdef class Blob:
self._initial_shape = new_shape
deref(self._ptr).setShape(new_shape)
## This class represents an Inference Engine entity and allows you to manipulate with plugins using unified interfaces.
cdef class IECore:
"""
@@ -1002,14 +998,6 @@ cdef class DataPtr:
"""
return deref(self._ptr).isInitialized()
@property
def is_dynamic(self):
return deref(self._ptr).isDynamic()
## get capsule with ngraph::PartialShape
def _get_partial_shape_capsule(self):
return C.getPartialShape_capsule(self._ptr)
cdef class CDataPtr:
"""
@@ -1051,14 +1039,6 @@ cdef class CDataPtr:
"""
return deref(self._ptr).isInitialized()
@property
def is_dynamic(self):
return deref(self._ptr).isDynamic()
## get capsule with ngraph::PartialShape
def _get_partial_shape_capsule(self):
return C.getPartialShape_capsule(self._ptr)
cdef class ExecutableNetwork:
"""
@@ -1140,8 +1120,6 @@ cdef class ExecutableNetwork:
infer_request.impl = &(deref(self.impl).infer_requests[i])
infer_request._inputs_list = list(self.input_info.keys())
infer_request._outputs_list = list(self.outputs.keys())
for input_name in infer_request._inputs_list:
infer_request._inputs_is_dynamic[input_name] = self.input_info[input_name].input_data.is_dynamic
self._infer_requests.append(infer_request)
if len(self._infer_requests) != c_infer_requests_size:
@@ -1330,7 +1308,6 @@ cdef class InferRequest:
self._outputs_list = []
self._py_callback = lambda *args, **kwargs: None
self._py_data = None
self._inputs_is_dynamic = {}
cdef void user_callback(self, int status) with gil:
if self._py_callback:
@@ -1598,9 +1575,6 @@ cdef class InferRequest:
def _fill_inputs(self, inputs):
for k, v in inputs.items():
assert k in self._inputs_list, f"No input with name {k} found in network"
if self._inputs_is_dynamic[k]:
shape = expand_dims_to_corresponding_layout(v.shape, self.input_blobs[k].tensor_desc.layout)
self.input_blobs[k].set_shape(shape)
if self.input_blobs[k].tensor_desc.precision == "FP16":
self.input_blobs[k].buffer[:] = v.view(dtype=np.int16)
else:
@@ -1758,25 +1732,15 @@ cdef class IENetwork:
n, c, h, w = net.input_info[input_layer].input_data.shape
net.reshape({input_layer: (n, c, h*2, w*2)})
"""
cdef map[string, vector[vector[int64_t]]] c_input_shapes
cdef vector[vector[int64_t]] c_shape
cdef vector[int64_t] dim
cdef map[string, vector[size_t]] c_input_shapes
cdef vector[size_t] c_shape
net_inputs = self.input_info
for input, shape in input_shapes.items():
c_shape = []
if input not in net_inputs:
raise AttributeError(f"Specified '{input}' layer not in network inputs '{net_inputs}'! ")
for v in shape:
if isinstance(v, list) or isinstance(v, tuple):
if len(v) < 1 or len(v) > 2:
raise ValueError(f"Incorrect PartialShape dimension definition '{v}' "
f"in shape '{shape}', expected one or two values for a dimension! ")
for d in v:
dim.push_back(d)
else:
dim.push_back(v)
c_shape.push_back(dim)
dim.clear()
c_shape.push_back(v)
c_input_shapes[input.encode()] = c_shape
self.impl.reshape(c_input_shapes)

38
src/bindings/python/src/compatibility/openvino/inference_engine/ie_api_impl.cpp
View File

@@ -5,7 +5,6 @@
#include "ie_api_impl.hpp"
#include "ie_plugin_config.hpp"
#include "ngraph/partial_shape.hpp"
#include "openvino/op/util/framework_node.hpp"
const std::string EXPORTED_NETWORK_NAME = "undefined";
@@ -215,26 +214,6 @@ InferenceEnginePython::IENetwork InferenceEnginePython::read_network(std::string
return InferenceEnginePython::IENetwork(std::make_shared<InferenceEngine::CNNNetwork>(net));
}
PyObject* InferenceEnginePython::getPartialShape_capsule(InferenceEngine::CDataPtr data) {
const char* py_capsule_name = "ngraph_partial_shape";
IE_SUPPRESS_DEPRECATED_START
auto ngraph_pShape_ptr = std::make_shared<ngraph::PartialShape>(data->getPartialShape());
IE_SUPPRESS_DEPRECATED_END
auto* sp_copy = new std::shared_ptr<const ngraph::PartialShape>(ngraph_pShape_ptr);
auto sp_deleter = [](PyObject* capsule) {
auto* capsule_ptr = PyCapsule_GetPointer(capsule, "ngraph_partial_shape");
auto* function_sp = static_cast<std::shared_ptr<ngraph::PartialShape>*>(capsule_ptr);
if (function_sp) {
delete function_sp;
}
};
if (ngraph_pShape_ptr) {
return PyCapsule_New(sp_copy, py_capsule_name, sp_deleter);
} else {
return nullptr;
}
}
InferenceEnginePython::IENetwork::IENetwork(const std::shared_ptr<InferenceEngine::CNNNetwork>& cnn_network)
: actual(cnn_network) {
if (actual == nullptr)
@@ -318,21 +297,8 @@ size_t InferenceEnginePython::IENetwork::getBatch() {
return actual->getBatchSize();
}
void InferenceEnginePython::IENetwork::reshape(
const std::map<std::string, std::vector<std::vector<int64_t>>>& input_shapes) {
std::map<std::string, ngraph::PartialShape> inputShapes;
for (auto const& input : input_shapes) {
using ngraph::Dimension;
std::vector<Dimension> dims;
for (auto const& d : input.second) {
if (d.size() == 1)
dims.push_back(Dimension(d[0]));
else if (d.size() == 2)
dims.push_back(Dimension(d[0], d[1]));
}
inputShapes[input.first] = ngraph::PartialShape(dims);
}
actual->reshape(inputShapes);
void InferenceEnginePython::IENetwork::reshape(const std::map<std::string, std::vector<size_t>>& input_shapes) {
actual->reshape(input_shapes);
}
InferenceEnginePython::IEExecNetwork::IEExecNetwork(const std::string& name, size_t num_requests)

4
src/bindings/python/src/compatibility/openvino/inference_engine/ie_api_impl.hpp
View File

@@ -62,7 +62,7 @@ struct IENetwork {
const std::map<std::string, InferenceEngine::DataPtr> getOutputs();
void reshape(const std::map<std::string, std::vector<std::vector<int64_t>>>& input_shapes);
void reshape(const std::map<std::string, std::vector<size_t>>& input_shapes);
void serialize(const std::string& path_to_xml, const std::string& path_to_bin);
@@ -211,8 +211,6 @@ std::string get_version();
InferenceEnginePython::IENetwork read_network(std::string path_to_xml, std::string path_to_bin);
PyObject* getPartialShape_capsule(InferenceEngine::CDataPtr data);
const size_t product(const InferenceEngine::SizeVector& dims);
}; // namespace InferenceEnginePython

5
src/bindings/python/src/compatibility/openvino/inference_engine/ie_api_impl_defs.pxd
View File

@@ -48,7 +48,6 @@ cdef extern from "<inference_engine.hpp>" namespace "InferenceEngine":
const Layout getLayout() except +
void setLayout(Layout layout) except +
const bool isInitialized() except +
bool isDynamic() except +
ctypedef shared_ptr[Data] DataPtr
ctypedef weak_ptr[Data] DataWeakPtr
@@ -181,7 +180,7 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
size_t getBatch() except +
void setLayerParams(map[string, map[string, string]] params_map) except +
void serialize(const string& path_to_xml, const string& path_to_bin) except +
void reshape(map[string, vector[vector[int64_t]]] input_shapes) except +
void reshape(map[string, vector[size_t]] input_shapes) except +
object getFunction() except +
void convertToOldRepresentation() except +
string getOVNameForTensor(const string &) except +
@@ -234,6 +233,4 @@ cdef extern from "ie_api_impl.hpp" namespace "InferenceEnginePython":
cdef IENetwork read_network(string path_to_xml, string path_to_bin)
cdef object getPartialShape_capsule(DataPtr)
cdef const size_t product(const SizeVector& dims)

2
src/bindings/python/src/compatibility/pyngraph/CMakeLists.txt
View File

@@ -79,7 +79,7 @@ if(OpenVINO_SOURCE_DIR OR InferenceEngineDeveloperPackage_FOUND)
COMPONENT ${OV_COMP_PYTHON_NGRAPH}_${PYTHON_VERSION}
USE_SOURCE_PERMISSIONS)
install(DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}/../../../tests
install(DIRECTORY ${PYTHON_BINDINGS_ROOT_DIR}/tests_compatibility
DESTINATION tests/${PROJECT_NAME}
COMPONENT tests EXCLUDE_FROM_ALL)
endif()

101
src/bindings/python/src/compatibility/pyngraph/dimension.cpp
View File

@@ -26,10 +26,8 @@ void regclass_pyngraph_Dimension(py::module m) {
R"(
Construct a static dimension.
Parameters
----------
dimension : int
Value of the dimension.
:param dimension: Value of the dimension.
:type dimension: int
)");
dim.def(py::init<value_type&, value_type&>(),
py::arg("min_dimension"),
@@ -37,13 +35,10 @@ void regclass_pyngraph_Dimension(py::module m) {
R"(
Construct a dynamic dimension with bounded range.
Parameters
----------
min_dimension : int
The lower inclusive limit for the dimension.
max_dimension : int
The upper inclusive limit for the dimension.
:param min_dimension: The lower inclusive limit for the dimension.
:type min_dimension: int
:param max_dimension: inclusive limit for the dimension.
:type max_dimension: The upper inclusive limit for the dimension.
)");
dim.def_static("dynamic", &ngraph::Dimension::dynamic);
@@ -53,20 +48,16 @@ void regclass_pyngraph_Dimension(py::module m) {
R"(
Check if Dimension is dynamic.
Returns
----------
is_dynamic : bool
True if dynamic, else False.
:return: True if dynamic, else False.
:rtype: bool
)");
dim.def_property_readonly("is_static",
&ngraph::Dimension::is_static,
R"(
Check if Dimension is static.
Returns
----------
is_static : bool
True if static, else False.
:return: True if static, else False.
:rtype: bool
)");
dim.def(
@@ -89,10 +80,8 @@ void regclass_pyngraph_Dimension(py::module m) {
Return this dimension as integer.
This dimension must be static and non-negative.
Returns
----------
get_length : int
Value of the dimension.
:return Value of the dimension.
:rtype: int
)");
dim.def("get_min_length",
&ngraph::Dimension::get_min_length,
@@ -100,10 +89,8 @@ void regclass_pyngraph_Dimension(py::module m) {
Return this dimension's min_dimension as integer.
This dimension must be dynamic and non-negative.
Returns
----------
get_min_length : int
Value of the dimension.
:return: Value of the dimension.
:rtype: int
)");
dim.def("get_max_length",
&ngraph::Dimension::get_max_length,
@@ -111,10 +98,8 @@ void regclass_pyngraph_Dimension(py::module m) {
Return this dimension's max_dimension as integer.
This dimension must be dynamic and non-negative.
Returns
----------
get_max_length : int
Value of the dimension.
:return: Value of the dimension.
:rtype: int
)");
dim.def("same_scheme",
@@ -124,16 +109,11 @@ void regclass_pyngraph_Dimension(py::module m) {
Return this dimension's max_dimension as integer.
This dimension must be dynamic and non-negative.
Parameters
----------
dim : Dimension
The other dimension to compare this dimension to.
Returns
----------
same_scheme : bool
True if this dimension and dim are both dynamic,
or if they are both static and equal, otherwise False.
:param dim: The other dimension to compare this dimension to.
:type dim: Dimension
:return: True if this dimension and dim are both dynamic,
or if they are both static and equal, otherwise False.
:rtype: bool
)");
dim.def("compatible",
&ngraph::Dimension::compatible,
@@ -142,15 +122,10 @@ void regclass_pyngraph_Dimension(py::module m) {
Check whether this dimension is capable of being merged
with the argument dimension.
Parameters
----------
d : Dimension
The dimension to compare this dimension with.
Returns
----------
compatible : bool
True if this dimension is compatible with d, else False.
:param d: The dimension to compare this dimension with.
:type d: Dimension
:return: True if this dimension is compatible with d, else False.
:rtype: bool
)");
dim.def("relaxes",
&ngraph::Dimension::relaxes,
@@ -164,15 +139,10 @@ void regclass_pyngraph_Dimension(py::module m) {
this.relaxes(d) is equivalent to d.refines(this).
Parameters
----------
d : Dimension
The dimension to compare this dimension with.
Returns
----------
relaxes : bool
True if this dimension relaxes d, else False.
:param d: The dimension to compare this dimension with.
:type d: Dimension
:return: True if this dimension relaxes d, else False.
:rtype: bool
)");
dim.def("refines",
&ngraph::Dimension::refines,
@@ -186,15 +156,10 @@ void regclass_pyngraph_Dimension(py::module m) {
this.refines(d) is equivalent to d.relaxes(this).
Parameters
----------
d : Dimension
The dimension to compare this dimension with.
Returns
----------
relaxes : bool
True if this dimension refines d, else False.
:param d: The dimension to compare this dimension with.
:type d: Dimension
:return: True if this dimension refines d, else False.
:rtype: bool
)");
dim.def("__str__", [](const ngraph::Dimension& self) -> std::string {

2
src/bindings/python/src/openvino/frontend/__init__.py
View File

@@ -3,7 +3,7 @@
"""
Package: openvino
Low level wrappers for the FrontEnd c++ api.
Low level wrappers for the FrontEnd C++ API.
"""
# flake8: noqa

2
src/bindings/python/src/openvino/frontend/onnx/__init__.py
View File

@@ -3,7 +3,7 @@
"""
Package: openvino
Low level wrappers for the FrontEnd c++ api.
Low level wrappers for the FrontEnd C++ API.
"""
# flake8: noqa

2
src/bindings/python/src/openvino/frontend/paddle/__init__.py
View File

@@ -3,7 +3,7 @@
"""
Package: openvino
Low level wrappers for the FrontEnd c++ api.
Low level wrappers for the FrontEnd C++ API.
"""
# flake8: noqa

2
src/bindings/python/src/openvino/frontend/tensorflow/__init__.py
View File

@@ -3,7 +3,7 @@
"""
Package: openvino
Low level wrappers for the FrontEnd c++ api.
Low level wrappers for the FrontEnd C++ API.
"""
# flake8: noqa

4
src/bindings/python/src/openvino/preprocess/__init__.py
View File

@@ -2,8 +2,8 @@
# SPDX-License-Identifier: Apache-2.0
"""
Package: ngraph
Low level wrappers for the PrePostProcessing c++ api.
Package: openvino
Low level wrappers for the PrePostProcessing C++ API.
"""
# flake8: noqa

171
src/bindings/python/src/openvino/runtime/ie_api.py
View File

@@ -2,8 +2,7 @@
# SPDX-License-Identifier: Apache-2.0
import numpy as np
import copy
from typing import Any, List, Type, Union
from typing import Any, Union
from openvino.pyopenvino import Model
from openvino.pyopenvino import Core as CoreBase
@@ -39,7 +38,7 @@ def convert_dict_items(inputs: dict, py_types: dict) -> dict:
new_inputs[k] = (
val
if isinstance(val, Tensor)
else Tensor(np.array(val, get_dtype(ov_type)))
else Tensor(np.array(val, get_dtype(ov_type), copy=False))
)
return new_inputs
@@ -83,10 +82,30 @@ def get_input_types(obj: Union[InferRequestBase, CompiledModelBase]) -> dict:
class InferRequest(InferRequestBase):
"""InferRequest wrapper."""
"""InferRequest class represents infer request which can be run in asynchronous or synchronous manners."""
def infer(self, inputs: Union[dict, list] = None) -> dict:
"""Infer wrapper for InferRequest."""
"""Infers specified input(s) in synchronous mode.
Blocks all methods of InferRequest while request is running.
Calling any method will lead to throwing exceptions.
The allowed types of keys in the `inputs` dictionary are:
(1) `int`
(2) `str`
(3) `openvino.runtime.ConstOutput`
The allowed types of values in the `inputs` are:
(1) `numpy.array`
(2) `openvino.runtime.Tensor`
:param inputs: Data to be set on input tensors.
:type inputs: Union[Dict[keys, values], List[values]], optional
:return: Dictionary of results from output tensors with ports as keys.
:rtype: Dict[openvino.runtime.ConstOutput, numpy.array]
"""
return super().infer(
{} if inputs is None else normalize_inputs(inputs, get_input_types(self))
)
@@ -94,7 +113,28 @@ class InferRequest(InferRequestBase):
def start_async(
self, inputs: Union[dict, list] = None, userdata: Any = None
) -> None:
"""Asynchronous infer wrapper for InferRequest."""
"""Starts inference of specified input(s) in asynchronous mode.
Returns immediately. Inference starts also immediately.
Calling any method on the `InferRequest` object while the request is running
will lead to throwing exceptions.
The allowed types of keys in the `inputs` dictionary are:
(1) `int`
(2) `str`
(3) `openvino.runtime.ConstOutput`
The allowed types of values in the `inputs` are:
(1) `numpy.array`
(2) `openvino.runtime.Tensor`
:param inputs: Data to be set on input tensors.
:type inputs: Union[Dict[keys, values], List[values]], optional
:param userdata: Any data that will be passed inside callback call.
:type userdata: Any
"""
super().start_async(
{} if inputs is None else normalize_inputs(inputs, get_input_types(self)),
userdata,
@@ -102,34 +142,95 @@ class InferRequest(InferRequestBase):
class CompiledModel(CompiledModelBase):
"""CompiledModel wrapper."""
"""CompiledModel class.
CompiledModel represents Model that is compiled for a specific device by applying
multiple optimization transformations, then mapping to compute kernels.
"""
def create_infer_request(self) -> InferRequest:
"""Create new InferRequest object."""
"""Creates an inference request object used to infer the compiled model.
The created request has allocated input and output tensors.
:return: New InferRequest object.
:rtype: openvino.runtime.InferRequest
"""
return InferRequest(super().create_infer_request())
def infer_new_request(self, inputs: Union[dict, list] = None) -> dict:
"""Infer wrapper for CompiledModel."""
"""Infers specified input(s) in synchronous mode.
Blocks all methods of CompiledModel while request is running.
Method creates new temporary InferRequest and run inference on it.
It is advised to use dedicated InferRequest class for performance,
optimizing workflows and creating advanced pipelines.
The allowed types of keys in the `inputs` dictionary are:
(1) `int`
(2) `str`
(3) `openvino.runtime.ConstOutput`
The allowed types of values in the `inputs` are:
(1) `numpy.array`
(2) `openvino.runtime.Tensor`
:param inputs: Data to be set on input tensors.
:type inputs: Union[Dict[keys, values], List[values]], optional
:return: Dictionary of results from output tensors with ports as keys.
:rtype: Dict[openvino.runtime.ConstOutput, numpy.array]
"""
return super().infer_new_request(
{} if inputs is None else normalize_inputs(inputs, get_input_types(self))
)
def __call__(self, inputs: Union[dict, list] = None) -> dict:
"""Callable infer wrapper for CompiledModel."""
"""Callable infer wrapper for CompiledModel. Look at `infer_new_request` for reference."""
return self.infer_new_request(inputs)
class AsyncInferQueue(AsyncInferQueueBase):
"""AsyncInferQueue wrapper."""
"""AsyncInferQueue with pool of asynchronous requests.
AsyncInferQueue represents helper that creates a pool of asynchronous
InferRequests and provides synchronization functions to control flow of
a simple pipeline.
"""
def __getitem__(self, i: int) -> InferRequest:
"""Return i-th InferRequest from AsyncInferQueue."""
"""Gets InferRequest from the pool with given i id.
:param i: InferRequest id.
:type i: int
:return: InferRequests from the pool with given id.
:rtype: openvino.runtime.InferRequest
"""
return InferRequest(super().__getitem__(i))
def start_async(
self, inputs: Union[dict, list] = None, userdata: Any = None
) -> None:
"""Asynchronous infer wrapper for AsyncInferQueue."""
"""Run asynchronous inference using the next available InferRequest from the pool.
The allowed types of keys in the `inputs` dictionary are:
(1) `int`
(2) `str`
(3) `openvino.runtime.ConstOutput`
The allowed types of values in the `inputs` are:
(1) `numpy.array`
(2) `openvino.runtime.Tensor`
:param inputs: Data to be set on input tensors of the next available InferRequest.
:type inputs: Union[Dict[keys, values], List[values]], optional
:param userdata: Any data that will be passed to a callback.
:type userdata: Any, optional
"""
super().start_async(
{}
if inputs is None
@@ -141,7 +242,13 @@ class AsyncInferQueue(AsyncInferQueueBase):
class Core(CoreBase):
"""Core wrapper."""
"""Core class represents OpenVINO runtime Core entity.
User applications can create several Core class instances, but in this
case the underlying plugins are created multiple times and not shared
between several Core instances. The recommended way is to have a single
Core instance per application.
"""
def compile_model(
self, model: Union[Model, str], device_name: str = None, config: dict = None
@@ -157,17 +264,17 @@ class Core(CoreBase):
)
def import_model(
self, model_file: str, device_name: str, config: dict = None
self, model_stream: str, device_name: str, config: dict = None
) -> CompiledModel:
"""Compile a model from given model file path."""
return CompiledModel(
super().import_model(
model_file, device_name, {} if config is None else config
model_stream, device_name, {} if config is None else config
)
)
class ExtendedNetwork(CompiledModel):
class ExtendedModel(CompiledModel):
"""CompiledModel that additionally holds Core object."""
def __init__(self, core: Core, net: CompiledModel):
@@ -176,9 +283,14 @@ class ExtendedNetwork(CompiledModel):
def compile_model(model_path: str) -> CompiledModel:
"""Compact method to compile model with AUTO plugin."""
"""Compact method to compile model with AUTO plugin.
:param model_path: Path to file with model.
:type model_path: str
:return: Extended version of `CompiledModel` that holds and keeps alive `Core` object.
"""
core = Core()
return ExtendedNetwork(core, core.compile_model(model_path, "AUTO"))
return ExtendedModel(core, core.compile_model(model_path, "AUTO"))
class OVAny(OVAnyBase):
@@ -186,27 +298,28 @@ class OVAny(OVAnyBase):
Wrapper provides some useful overloads for simple built-in Python types.
Access to the OVAny value is direct if it is a built-in Python data type.
Example:
@code{.py}
Access to the `OVAny` value is direct if it is a built-in Python data type.
:Example:
.. code-block:: ipython
any = OVAny([1, 2])
print(any[0])
Output: 2
@endcode
Otherwise if OVAny value is a custom data type (for example user class),
access to the value is possible by 'get()' method or property 'value'.
Example:
@code{.py}
Otherwise if `OVAny` value is a custom data type (for example user class),
access to the value is possible by `get()` method or property 'value'.
:Example:
.. code-block:: python
class Test:
def __init__(self):
self.data = "test"
any = OVAny(Test())
print(any.value.data)
@endcode
"""
def __getitem__(self, key: Union[str, int]) -> Any:

1121
src/bindings/python/src/openvino/runtime/opset1/ops.py
View File

File diff suppressed because it is too large Load Diff

62
src/bindings/python/src/openvino/runtime/opset2/ops.py
View File

@@ -49,12 +49,12 @@ def batch_to_space(
BatchToSpace permutes data from the batch dimension of the data tensor into spatial dimensions.
@param data: Node producing the data tensor.
@param block_shape: The sizes of the block of values to be moved.
@param crops_begin: Specifies the amount to crop from the beginning along each axis of `data`.
@param crops_end: Specifies the amount to crop from the end along each axis of `data`.
@param name: Optional output node name.
@return The new node performing a BatchToSpace operation.
:param data: Node producing the data tensor.
:param block_shape: The sizes of the block of values to be moved.
:param crops_begin: Specifies the amount to crop from the beginning along each axis of `data`.
:param crops_end: Specifies the amount to crop from the end along each axis of `data`.
:param name: Optional output node name.
returns The new node performing a BatchToSpace operation.
"""
return _get_node_factory_opset2().create(
"BatchToSpace", as_nodes(data, block_shape, crops_begin, crops_end)
@@ -71,9 +71,9 @@ def gelu(node: NodeInput, name: Optional[str] = None) -> Node:
For more information refer to [Gaussian Error Linear Unit (GELU)](https://arxiv.org/pdf/1606.08415.pdf>)
@param node: Input tensor. One of: input node, array or scalar.
@param name: Optional output node name.
@return The new node performing a GELU operation on its input data element-wise.
:param node: Input tensor. One of: input node, array or scalar.
:param name: Optional output node name.
returns The new node performing a GELU operation on its input data element-wise.
"""
return _get_node_factory_opset2().create("Gelu", [node])
@@ -92,13 +92,13 @@ def mvn(
\f[ Y = \dfrac{X-EX}{\sqrt{E(X-EX)^2}} \f]
@param data: The node with data tensor.
@param across_channels: Denotes if mean values are shared across channels.
@param normalize_variance: Denotes whether to perform variance normalization.
@param eps: The number added to the variance to avoid division by zero
:param data: The node with data tensor.
:param across_channels: Denotes if mean values are shared across channels.
:param normalize_variance: Denotes whether to perform variance normalization.
:param eps: The number added to the variance to avoid division by zero
when normalizing the value. Scalar value.
@param name: Optional output node name.
@return The new node performing a MVN operation on input tensor.
:param name: Optional output node name.
returns The new node performing a MVN operation on input tensor.
"""
return _get_node_factory_opset2().create(
"MVN",
@@ -111,10 +111,10 @@ def mvn(
def reorg_yolo(input: Node, stride: List[int], name: Optional[str] = None) -> Node:
"""Return a node which produces the ReorgYolo operation.
@param input: Input data
@param stride: Stride to reorganize input by
@param name: Optional name for output node.
@return ReorgYolo node
:param input: Input data
:param stride: Stride to reorganize input by
:param name: Optional name for output node.
returns ReorgYolo node
"""
return _get_node_factory_opset2().create("ReorgYolo", [input], {"stride": stride})
@@ -130,12 +130,12 @@ def roi_pooling(
) -> Node:
"""Return a node which produces an ROIPooling operation.
@param input: Input feature map {N, C, ...}
@param coords: Coordinates of bounding boxes
@param output_size: Height/Width of ROI output features (shape)
@param spatial_scale: Ratio of input feature map over input image size (float)
@param method: Method of pooling - string: "max" or "bilinear"
@return ROIPooling node
:param input: Input feature map {N, C, ...}
:param coords: Coordinates of bounding boxes
:param output_size: Height/Width of ROI output features (shape)
:param spatial_scale: Ratio of input feature map over input image size (float)
:param method: Method of pooling - string: "max" or "bilinear"
returns ROIPooling node
"""
method = method.lower()
return _get_node_factory_opset2().create(
@@ -159,12 +159,12 @@ def space_to_batch(
The operator returns a copy of the input tensor where values from spatial blocks dimensions
are moved in the batch dimension
@param data: Node producing the data tensor.
@param block_shape: The sizes of the block of values to be moved.
@param pads_begin: Specifies the padding for the beginning along each axis of `data`.
@param pads_end: Specifies the padding for the ending along each axis of `data`.
@param name: Optional output node name.
@return The new node performing a SpaceToBatch operation.
:param data: Node producing the data tensor.
:param block_shape: The sizes of the block of values to be moved.
:param pads_begin: Specifies the padding for the beginning along each axis of `data`.
:param pads_end: Specifies the padding for the ending along each axis of `data`.
:param name: Optional output node name.
returns The new node performing a SpaceToBatch operation.
"""
return _get_node_factory_opset2().create(
"SpaceToBatch", as_nodes(data, block_shape, pads_begin, pads_end)

250
src/bindings/python/src/openvino/runtime/opset3/ops.py
View File

@@ -41,10 +41,10 @@ _get_node_factory_opset3 = partial(_get_node_factory, "opset3")
def assign(new_value: NodeInput, variable_id: str, name: Optional[str] = None) -> Node:
"""Return a node which produces the Assign operation.
@param new_value: Node producing a value to be assigned to a variable.
@param variable_id: Id of a variable to be updated.
@param name: Optional name for output node.
@return Assign node
:param new_value: Node producing a value to be assigned to a variable.
:param variable_id: Id of a variable to be updated.
:param name: Optional name for output node.
returns Assign node
"""
return _get_node_factory_opset3().create(
"Assign",
@@ -63,14 +63,14 @@ def broadcast(
) -> Node:
"""Create a node which broadcasts the input node's values along specified axes to a desired shape.
@param data: The node with input tensor data.
@param target_shape: The node with a new shape we want to broadcast tensor to.
@param axes_mapping: The node with a axis positions (0-based) in the result
:param data: The node with input tensor data.
:param target_shape: The node with a new shape we want to broadcast tensor to.
:param axes_mapping: The node with a axis positions (0-based) in the result
that are being broadcast.
@param broadcast_spec: The type of broadcasting that specifies mapping of input tensor axes
:param broadcast_spec: The type of broadcasting that specifies mapping of input tensor axes
to output shape axes. Range of values: NUMPY, EXPLICIT, BIDIRECTIONAL.
@param name: Optional new name for output node.
@return New node with broadcast shape.
:param name: Optional new name for output node.
returns New node with broadcast shape.
"""
inputs = as_nodes(data, target_shape)
if broadcast_spec.upper() == "EXPLICIT":
@@ -90,13 +90,13 @@ def bucketize(
) -> Node:
"""Return a node which produces the Bucketize operation.
@param data: Input data to bucketize
@param buckets: 1-D of sorted unique boundaries for buckets
@param output_type: Output tensor type, "i64" or "i32", defaults to i64
@param with_right_bound: indicates whether bucket includes the right or left
:param data: Input data to bucketize
:param buckets: 1-D of sorted unique boundaries for buckets
:param output_type: Output tensor type, "i64" or "i32", defaults to i64
:param with_right_bound: indicates whether bucket includes the right or left
edge of interval. default true = includes right edge
@param name: Optional name for output node.
@return Bucketize node
:param name: Optional name for output node.
returns Bucketize node
"""
return _get_node_factory_opset3().create(
"Bucketize",
@@ -115,11 +115,11 @@ def cum_sum(
) -> Node:
"""Construct a cumulative summation operation.
@param arg: The tensor to be summed.
@param axis: zero dimension tensor specifying axis position along which sum will be performed.
@param exclusive: if set to true, the top element is not included
@param reverse: if set to true, will perform the sums in reverse direction
@return New node performing the operation
:param arg: The tensor to be summed.
:param axis: zero dimension tensor specifying axis position along which sum will be performed.
:param exclusive: if set to true, the top element is not included
:param reverse: if set to true, will perform the sums in reverse direction
returns New node performing the operation
"""
return _get_node_factory_opset3().create(
"CumSum", as_nodes(arg, axis), {"exclusive": exclusive, "reverse": reverse}
@@ -137,13 +137,13 @@ def embedding_bag_offsets_sum(
) -> Node:
"""Return a node which performs sums of bags of embeddings without the intermediate embeddings.
@param emb_table: Tensor containing the embedding lookup table.
@param indices: Tensor with indices.
@param offsets: Tensor containing the starting index positions of each bag in indices.
@param per_sample_weights: Tensor with weights for each sample.
@param default_index: Scalar containing default index in embedding table to fill empty bags.
@param name: Optional name for output node.
@return The new node which performs EmbeddingBagOffsetsSum
:param emb_table: Tensor containing the embedding lookup table.
:param indices: Tensor with indices.
:param offsets: Tensor containing the starting index positions of each bag in indices.
:param per_sample_weights: Tensor with weights for each sample.
:param default_index: Scalar containing default index in embedding table to fill empty bags.
:param name: Optional name for output node.
returns The new node which performs EmbeddingBagOffsetsSum
"""
inputs = [emb_table, as_node(indices), as_node(offsets)]
if per_sample_weights is not None:
@@ -167,11 +167,11 @@ def embedding_bag_packed_sum(
EmbeddingSegmentsSum constructs an output tensor by replacing every index in a given
input tensor with a row (from the weights matrix) at that index
@param emb_table: Tensor containing the embedding lookup table.
@param indices: Tensor with indices.
@param per_sample_weights: Weights to be multiplied with embedding table.
@param name: Optional name for output node.
@return EmbeddingBagPackedSum node
:param emb_table: Tensor containing the embedding lookup table.
:param indices: Tensor with indices.
:param per_sample_weights: Weights to be multiplied with embedding table.
:param name: Optional name for output node.
returns EmbeddingBagPackedSum node
"""
inputs = [as_node(emb_table), as_node(indices)]
if per_sample_weights is not None:
@@ -195,14 +195,14 @@ def embedding_segments_sum(
EmbeddingSegmentsSum constructs an output tensor by replacing every index in a given
input tensor with a row (from the weights matrix) at that index
@param emb_table: Tensor containing the embedding lookup table.
@param indices: Tensor with indices.
@param segment_ids: Tensor with indices into the output Tensor
@param num_segments: Tensor with number of segments.
@param default_index: Scalar containing default index in embedding table to fill empty bags.
@param per_sample_weights: Weights to be multiplied with embedding table.
@param name: Optional name for output node.
@return EmbeddingSegmentsSum node
:param emb_table: Tensor containing the embedding lookup table.
:param indices: Tensor with indices.
:param segment_ids: Tensor with indices into the output Tensor
:param num_segments: Tensor with number of segments.
:param default_index: Scalar containing default index in embedding table to fill empty bags.
:param per_sample_weights: Weights to be multiplied with embedding table.
:param name: Optional name for output node.
returns EmbeddingSegmentsSum node
"""
inputs = [as_node(emb_table), as_node(indices), as_node(segment_ids)]
if per_sample_weights is not None:
@@ -229,13 +229,13 @@ def extract_image_patches(
) -> Node:
"""Return a node which produces the ExtractImagePatches operation.
@param image: 4-D Input data to extract image patches.
@param sizes: Patch size in the format of [size_rows, size_cols].
@param strides: Patch movement stride in the format of [stride_rows, stride_cols]
@param rates: Element seleciton rate for creating a patch.
@param auto_pad: Padding type.
@param name: Optional name for output node.
@return ExtractImagePatches node
:param image: 4-D Input data to extract image patches.
:param sizes: Patch size in the format of [size_rows, size_cols].
:param strides: Patch movement stride in the format of [stride_rows, stride_cols]
:param rates: Element seleciton rate for creating a patch.
:param auto_pad: Padding type.
:param name: Optional name for output node.
returns ExtractImagePatches node
"""
return _get_node_factory_opset3().create(
"ExtractImagePatches",
@@ -266,29 +266,29 @@ def gru_cell(
Note this class represents only single *cell* and not whole *layer*.
@param X: The input tensor with shape: [batch_size, input_size].
@param initial_hidden_state: The hidden state tensor at current time step with shape:
:param X: The input tensor with shape: [batch_size, input_size].
:param initial_hidden_state: The hidden state tensor at current time step with shape:
[batch_size, hidden_size].
@param W: The weights for matrix multiplication, gate order: zrh.
:param W: The weights for matrix multiplication, gate order: zrh.
Shape: [3*hidden_size, input_size].
@param R: The recurrence weights for matrix multiplication.
:param R: The recurrence weights for matrix multiplication.
Shape: [3*hidden_size, hidden_size].
@param B: The sum of biases (weight and recurrence).
:param B: The sum of biases (weight and recurrence).
For linear_before_reset set True the shape is [4*hidden_size].
Otherwise the shape is [3*hidden_size].
@param hidden_size: The number of hidden units for recurrent cell.
:param hidden_size: The number of hidden units for recurrent cell.
Specifies hidden state size.
@param activations: The vector of activation functions used inside recurrent cell.
@param activation_alpha: The vector of alpha parameters for activation functions in
:param activations: The vector of activation functions used inside recurrent cell.
:param activation_alpha: The vector of alpha parameters for activation functions in
order respective to activation list.
@param activation_beta: The vector of beta parameters for activation functions in order
:param activation_beta: The vector of beta parameters for activation functions in order
respective to activation list.
@param clip: The value defining clipping range [-clip, clip] on input of
:param clip: The value defining clipping range [-clip, clip] on input of
activation functions.
@param linear_before_reset: Flag denotes if the layer behaves according to the modification
:param linear_before_reset: Flag denotes if the layer behaves according to the modification
of GRUCell described in the formula in the ONNX documentation.
@param name: Optional output node name.
@return The new node performing a GRUCell operation on tensor from input node.
:param name: Optional output node name.
returns The new node performing a GRUCell operation on tensor from input node.
"""
if activations is None:
activations = ["sigmoid", "tanh"]
@@ -323,17 +323,17 @@ def non_max_suppression(
) -> Node:
"""Return a node which performs NonMaxSuppression.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
to be selected per class.
@param iou_threshold: Tensor specifying intersection over union threshold
@param score_threshold: Tensor specifying minimum score to consider box for the processing.
@param box_encoding: Format of boxes data encoding.
@param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
:param iou_threshold: Tensor specifying intersection over union threshold
:param score_threshold: Tensor specifying minimum score to consider box for the processing.
:param box_encoding: Format of boxes data encoding.
:param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
boxes across batches or not.
@param output_type: Output element type.
@return The new node which performs NonMaxSuppression
:param output_type: Output element type.
returns The new node which performs NonMaxSuppression
"""
if max_output_boxes_per_class is None:
max_output_boxes_per_class = make_constant_node(0, np.int64)
@@ -356,10 +356,10 @@ def non_max_suppression(
def non_zero(data: NodeInput, output_type: str = "i64", name: Optional[str] = None,) -> Node:
"""Return the indices of the elements that are non-zero.
@param data: Input data.
@param output_type: Output tensor type.
:param data: Input data.
:param output_type: Output tensor type.
@return The new node which performs NonZero
returns The new node which performs NonZero
"""
return _get_node_factory_opset3().create(
"NonZero",
@@ -372,10 +372,10 @@ def non_zero(data: NodeInput, output_type: str = "i64", name: Optional[str] = No
def read_value(init_value: NodeInput, variable_id: str, name: Optional[str] = None) -> Node:
"""Return a node which produces the Assign operation.
@param init_value: Node producing a value to be returned instead of an unassigned variable.
@param variable_id: Id of a variable to be read.
@param name: Optional name for output node.
@return ReadValue node
:param init_value: Node producing a value to be returned instead of an unassigned variable.
:param variable_id: Id of a variable to be read.
:param name: Optional name for output node.
returns ReadValue node
"""
return _get_node_factory_opset3().create(
"ReadValue",
@@ -405,24 +405,24 @@ def rnn_cell(
Note this class represents only single *cell* and not whole RNN *layer*.
@param X: The input tensor with shape: [batch_size, input_size].
@param initial_hidden_state: The hidden state tensor at current time step with shape:
:param X: The input tensor with shape: [batch_size, input_size].
:param initial_hidden_state: The hidden state tensor at current time step with shape:
[batch_size, hidden_size].
@param W: The weight tensor with shape: [hidden_size, input_size].
@param R: The recurrence weight tensor with shape: [hidden_size,
:param W: The weight tensor with shape: [hidden_size, input_size].
:param R: The recurrence weight tensor with shape: [hidden_size,
hidden_size].
@param B: The sum of biases (weight and recurrence) with shape: [hidden_size].
@param hidden_size: The number of hidden units for recurrent cell.
:param B: The sum of biases (weight and recurrence) with shape: [hidden_size].
:param hidden_size: The number of hidden units for recurrent cell.
Specifies hidden state size.
@param activations: The vector of activation functions used inside recurrent cell.
@param activation_alpha: The vector of alpha parameters for activation functions in
:param activations: The vector of activation functions used inside recurrent cell.
:param activation_alpha: The vector of alpha parameters for activation functions in
order respective to activation list.
@param activation_beta: The vector of beta parameters for activation functions in order
:param activation_beta: The vector of beta parameters for activation functions in order
respective to activation list.
@param clip: The value defining clipping range [-clip, clip] on input of
:param clip: The value defining clipping range [-clip, clip] on input of
activation functions.
@param name: Optional output node name.
@return The new node performing a RNNCell operation on tensor from input node.
:param name: Optional output node name.
returns The new node performing a RNNCell operation on tensor from input node.
"""
if activations is None:
activations = ["tanh"]
@@ -456,18 +456,18 @@ def roi_align(
) -> Node:
"""Return a node which performs ROIAlign.
@param data: Input data.
@param rois: RoIs (Regions of Interest) to pool over.
@param batch_indices: Tensor with each element denoting the index of
:param data: Input data.
:param rois: RoIs (Regions of Interest) to pool over.
:param batch_indices: Tensor with each element denoting the index of
the corresponding image in the batch.
@param pooled_h: Height of the ROI output feature map.
@param pooled_w: Width of the ROI output feature map.
@param sampling_ratio: Number of bins over height and width to use to calculate
:param pooled_h: Height of the ROI output feature map.
:param pooled_w: Width of the ROI output feature map.
:param sampling_ratio: Number of bins over height and width to use to calculate
each output feature map element.
@param spatial_scale: Multiplicative spatial scale factor to translate ROI coordinates.
@param mode: Method to perform pooling to produce output feature map elements.
:param spatial_scale: Multiplicative spatial scale factor to translate ROI coordinates.
:param mode: Method to perform pooling to produce output feature map elements.
@return The new node which performs ROIAlign
returns The new node which performs ROIAlign
"""
inputs = as_nodes(data, rois, batch_indices)
attributes = {
@@ -490,11 +490,11 @@ def scatter_elements_update(
) -> Node:
"""Return a node which produces a ScatterElementsUpdate operation.
@param data: The input tensor to be updated.
@param indices: The tensor with indexes which will be updated.
@param updates: The tensor with update values.
@param axis: The axis for scatter.
@return ScatterElementsUpdate node
:param data: The input tensor to be updated.
:param indices: The tensor with indexes which will be updated.
:param updates: The tensor with update values.
:param axis: The axis for scatter.
returns ScatterElementsUpdate node
ScatterElementsUpdate creates a copy of the first input tensor with updated elements
specified with second and third input tensors.
@@ -519,11 +519,11 @@ def scatter_update(
ScatterUpdate sets new values to slices from data addressed by indices.
@param data: The input tensor to be updated.
@param indices: The tensor with indexes which will be updated.
@param updates: The tensor with update values.
@param axis: The axis at which elements will be updated.
@return ScatterUpdate node
:param data: The input tensor to be updated.
:param indices: The tensor with indexes which will be updated.
:param updates: The tensor with update values.
:param axis: The axis at which elements will be updated.
returns ScatterUpdate node
"""
return _get_node_factory_opset3().create(
"ScatterUpdate",
@@ -535,9 +535,9 @@ def scatter_update(
def shape_of(data: NodeInput, output_type: str = "i64", name: Optional[str] = None) -> Node:
"""Return a node which produces a tensor containing the shape of its input data.
@param data: The tensor containing the input data.
@param output_type: Output element type.
@return ShapeOf node
:param data: The tensor containing the input data.
:param output_type: Output element type.
returns ShapeOf node
"""
return _get_node_factory_opset3().create(
"ShapeOf",
@@ -550,14 +550,14 @@ def shape_of(data: NodeInput, output_type: str = "i64", name: Optional[str] = No
def shuffle_channels(data: Node, axis: int, group: int, name: Optional[str] = None) -> Node:
"""Perform permutation on data in the channel dimension of the input tensor.
@param data: The node with input tensor.
@param axis: Channel dimension index in the data tensor.
:param data: The node with input tensor.
:param axis: Channel dimension index in the data tensor.
A negative value means that the index should be calculated
from the back of the input data shape.
@param group: The channel dimension specified by the axis parameter
:param group: The channel dimension specified by the axis parameter
should be split into this number of groups.
@param name: Optional output node name.
@return The new node performing a permutation on data in the channel dimension
:param name: Optional output node name.
returns The new node performing a permutation on data in the channel dimension
of the input tensor.
The operation is the equivalent with the following transformation of the input tensor
@@ -571,7 +571,8 @@ def shuffle_channels(data: Node, axis: int, group: int, name: Optional[str] = No
For example:
@code{.py}
.. code-block:: ipython
Inputs: tensor of shape [1, 6, 2, 2]
data = [[[[ 0., 1.], [ 2., 3.]],
@@ -592,7 +593,6 @@ def shuffle_channels(data: Node, axis: int, group: int, name: Optional[str] = No
[[ 4., 5.], [ 6., 7.]],
[[12., 13.], [14., 15.]],
[[20., 21.], [22., 23.]]]]
@endcode
"""
return _get_node_factory_opset3().create(
"ShuffleChannels", [as_node(data)], {"axis": axis, "group": group}
@@ -611,13 +611,13 @@ def topk(
) -> Node:
"""Return a node which performs TopK.
@param data: Input data.
@param k: K.
@param axis: TopK Axis.
@param mode: Compute TopK largest ('max') or smallest ('min')
@param sort: Order of output elements (sort by: 'none', 'index' or 'value')
@param index_element_type: Type of output tensor with indices.
@return The new node which performs TopK (both indices and values)
:param data: Input data.
:param k: K.
:param axis: TopK Axis.
:param mode: Compute TopK largest ('max') or smallest ('min')
:param sort: Order of output elements (sort by: 'none', 'index' or 'value')
:param index_element_type: Type of output tensor with indices.
returns The new node which performs TopK (both indices and values)
"""
return _get_node_factory_opset3().create(
"TopK",

134
src/bindings/python/src/openvino/runtime/opset4/ops.py
View File

@@ -51,15 +51,15 @@ def ctc_loss(
) -> Node:
"""Return a node which performs CTCLoss.
@param logits: 3-D tensor of logits.
@param logit_length: 1-D tensor of lengths for each object from a batch.
@param labels: 2-D tensor of labels for which likelihood is estimated using logits.
@param label_length: 1-D tensor of length for each label sequence.
@param blank_index: Scalar used to mark a blank index.
@param preprocess_collapse_repeated: Flag for preprocessing labels before loss calculation.
@param ctc_merge_repeated: Flag for merging repeated characters in a potential alignment.
@param unique: Flag to find unique elements in a target.
@return The new node which performs CTCLoss
:param logits: 3-D tensor of logits.
:param logit_length: 1-D tensor of lengths for each object from a batch.
:param labels: 2-D tensor of labels for which likelihood is estimated using logits.
:param label_length: 1-D tensor of length for each label sequence.
:param blank_index: Scalar used to mark a blank index.
:param preprocess_collapse_repeated: Flag for preprocessing labels before loss calculation.
:param ctc_merge_repeated: Flag for merging repeated characters in a potential alignment.
:param unique: Flag to find unique elements in a target.
returns The new node which performs CTCLoss
"""
if blank_index is not None:
inputs = as_nodes(logits, logit_length, labels, label_length, blank_index)
@@ -89,17 +89,17 @@ def non_max_suppression(
) -> Node:
"""Return a node which performs NonMaxSuppression.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
to be selected per class.
@param iou_threshold: Tensor specifying intersection over union threshold
@param score_threshold: Tensor specifying minimum score to consider box for the processing.
@param box_encoding: Format of boxes data encoding.
@param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
:param iou_threshold: Tensor specifying intersection over union threshold
:param score_threshold: Tensor specifying minimum score to consider box for the processing.
:param box_encoding: Format of boxes data encoding.
:param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
boxes across batches or not.
@param output_type: Output element type.
@return The new node which performs NonMaxSuppression
:param output_type: Output element type.
returns The new node which performs NonMaxSuppression
"""
if max_output_boxes_per_class is None:
max_output_boxes_per_class = make_constant_node(0, np.int64)
@@ -122,8 +122,8 @@ def non_max_suppression(
def softplus(data: NodeInput, name: Optional[str] = None) -> Node:
"""Apply SoftPlus operation on each element of input tensor.
@param data: The tensor providing input data.
@return The new node with SoftPlus operation applied on each element.
:param data: The tensor providing input data.
returns The new node with SoftPlus operation applied on each element.
"""
return _get_node_factory_opset4().create("SoftPlus", as_nodes(data), {})
@@ -132,8 +132,8 @@ def softplus(data: NodeInput, name: Optional[str] = None) -> Node:
def mish(data: NodeInput, name: Optional[str] = None,) -> Node:
"""Return a node which performs Mish.
@param data: Tensor with input data floating point type.
@return The new node which performs Mish
:param data: Tensor with input data floating point type.
returns The new node which performs Mish
"""
return _get_node_factory_opset4().create("Mish", as_nodes(data), {})
@@ -142,8 +142,8 @@ def mish(data: NodeInput, name: Optional[str] = None,) -> Node:
def hswish(data: NodeInput, name: Optional[str] = None,) -> Node:
"""Return a node which performs HSwish (hard version of Swish).
@param data: Tensor with input data floating point type.
@return The new node which performs HSwish
:param data: Tensor with input data floating point type.
returns The new node which performs HSwish
"""
return _get_node_factory_opset4().create("HSwish", as_nodes(data), {})
@@ -156,8 +156,8 @@ def swish(
) -> Node:
"""Return a node which performing Swish activation function Swish(x, beta=1.0) = x * sigmoid(x * beta)).
@param data: Tensor with input data floating point type.
@return The new node which performs Swish
:param data: Tensor with input data floating point type.
returns The new node which performs Swish
"""
if beta is None:
beta = make_constant_node(1.0, np.float32)
@@ -168,9 +168,9 @@ def swish(
def acosh(node: NodeInput, name: Optional[str] = None) -> Node:
"""Apply hyperbolic inverse cosine function on the input node element-wise.
@param node: One of: input node, array or scalar.
@param name: Optional new name for output node.
@return New node with arccosh operation applied on it.
:param node: One of: input node, array or scalar.
:param name: Optional new name for output node.
returns New node with arccosh operation applied on it.
"""
return _get_node_factory_opset4().create("Acosh", [node])
@@ -179,9 +179,9 @@ def acosh(node: NodeInput, name: Optional[str] = None) -> Node:
def asinh(node: NodeInput, name: Optional[str] = None) -> Node:
"""Apply hyperbolic inverse sinus function on the input node element-wise.
@param node: One of: input node, array or scalar.
@param name: Optional new name for output node.
@return New node with arcsinh operation applied on it.
:param node: One of: input node, array or scalar.
:param name: Optional new name for output node.
returns New node with arcsinh operation applied on it.
"""
return _get_node_factory_opset4().create("Asinh", [node])
@@ -190,9 +190,9 @@ def asinh(node: NodeInput, name: Optional[str] = None) -> Node:
def atanh(node: NodeInput, name: Optional[str] = None) -> Node:
"""Apply hyperbolic inverse tangent function on the input node element-wise.
@param node: One of: input node, array or scalar.
@param name: Optional new name for output node.
@return New node with arctanh operation applied on it.
:param node: One of: input node, array or scalar.
:param name: Optional new name for output node.
returns New node with arctanh operation applied on it.
"""
return _get_node_factory_opset4().create("Atanh", [node])
@@ -207,11 +207,11 @@ def proposal(
) -> Node:
"""Filter bounding boxes and outputs only those with the highest prediction confidence.
@param class_probs: 4D input floating point tensor with class prediction scores.
@param bbox_deltas: 4D input floating point tensor with corrected predictions of bounding boxes
@param image_shape: The 1D input tensor with 3 or 4 elements describing image shape.
@param attrs: The dictionary containing key, value pairs for attributes.
@param name: Optional name for the output node.
:param class_probs: 4D input floating point tensor with class prediction scores.
:param bbox_deltas: 4D input floating point tensor with corrected predictions of bounding boxes
:param image_shape: The 1D input tensor with 3 or 4 elements describing image shape.
:param attrs: The dictionary containing key, value pairs for attributes.
:param name: Optional name for the output node.
* base_size The size of the anchor to which scale and ratio attributes are applied.
Range of values: a positive unsigned integer number
Default value: None
@@ -276,7 +276,9 @@ def proposal(
Required: no
Example of attribute dictionary:
@code{.py}
.. code-block:: python
# just required ones
attrs = {
'base_size': 85,
@@ -288,9 +290,9 @@ def proposal(
'ratio': [0.1, 1.5, 2.0, 2.5],
'scale': [2, 3, 3, 4],
}
@endcode
Optional attributes which are absent from dictionary will be set with corresponding default.
@return Node representing Proposal operation.
returns Node representing Proposal operation.
"""
requirements = [
("base_size", True, np.unsignedinteger, is_positive_value),
@@ -322,11 +324,11 @@ def reduce_l1(
) -> Node:
"""L1-reduction operation on input tensor, eliminating the specified reduction axes.
@param node: The tensor we want to mean-reduce.
@param reduction_axes: The axes to eliminate through mean operation.
@param keep_dims: If set to True it holds axes that are used for reduction
@param name: Optional name for output node.
@return The new node performing mean-reduction operation.
:param node: The tensor we want to mean-reduce.
:param reduction_axes: The axes to eliminate through mean operation.
:param keep_dims: If set to True it holds axes that are used for reduction
:param name: Optional name for output node.
returns The new node performing mean-reduction operation.
"""
return _get_node_factory_opset4().create(
"ReduceL1", as_nodes(node, reduction_axes), {"keep_dims": keep_dims}
@@ -339,11 +341,11 @@ def reduce_l2(
) -> Node:
"""L2-reduction operation on input tensor, eliminating the specified reduction axes.
@param node: The tensor we want to mean-reduce.
@param reduction_axes: The axes to eliminate through mean operation.
@param keep_dims: If set to True it holds axes that are used for reduction
@param name: Optional name for output node.
@return The new node performing mean-reduction operation.
:param node: The tensor we want to mean-reduce.
:param reduction_axes: The axes to eliminate through mean operation.
:param keep_dims: If set to True it holds axes that are used for reduction
:param name: Optional name for output node.
returns The new node performing mean-reduction operation.
"""
return _get_node_factory_opset4().create(
"ReduceL2", as_nodes(node, reduction_axes), {"keep_dims": keep_dims}
@@ -367,20 +369,20 @@ def lstm_cell(
) -> Node:
"""Return a node which performs LSTMCell operation.
@param X: The input tensor with shape: [batch_size, input_size].
@param initial_hidden_state: The hidden state tensor with shape: [batch_size, hidden_size].
@param initial_cell_state: The cell state tensor with shape: [batch_size, hidden_size].
@param W: The weight tensor with shape: [4*hidden_size, input_size].
@param R: The recurrence weight tensor with shape: [4*hidden_size, hidden_size].
@param B: The bias tensor for gates with shape: [4*hidden_size].
@param hidden_size: Specifies hidden state size.
@param activations: The list of three activation functions for gates.
@param activations_alpha: The list of alpha parameters for activation functions.
@param activations_beta: The list of beta parameters for activation functions.
@param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
@param name: An optional name of the output node.
:param X: The input tensor with shape: [batch_size, input_size].
:param initial_hidden_state: The hidden state tensor with shape: [batch_size, hidden_size].
:param initial_cell_state: The cell state tensor with shape: [batch_size, hidden_size].
:param W: The weight tensor with shape: [4*hidden_size, input_size].
:param R: The recurrence weight tensor with shape: [4*hidden_size, hidden_size].
:param B: The bias tensor for gates with shape: [4*hidden_size].
:param hidden_size: Specifies hidden state size.
:param activations: The list of three activation functions for gates.
:param activations_alpha: The list of alpha parameters for activation functions.
:param activations_beta: The list of beta parameters for activation functions.
:param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
:param name: An optional name of the output node.
@return The new node represents LSTMCell. Node outputs count: 2.
returns The new node represents LSTMCell. Node outputs count: 2.
"""
if activations is None:
activations = ["sigmoid", "tanh", "tanh"]

132
src/bindings/python/src/openvino/runtime/opset5/ops.py
View File

@@ -49,14 +49,14 @@ def batch_norm_inference(
) -> Node:
"""Perform layer normalizes a input tensor by mean and variance with appling scale and offset.
@param data: The input tensor with data for normalization.
@param gamma: The scalar scaling for normalized value.
@param beta: The bias added to the scaled normalized value.
@param mean: The value for mean normalization.
@param variance: The value for variance normalization.
@param epsilon: The number to be added to the variance to avoid division
:param data: The input tensor with data for normalization.
:param gamma: The scalar scaling for normalized value.
:param beta: The bias added to the scaled normalized value.
:param mean: The value for mean normalization.
:param variance: The value for variance normalization.
:param epsilon: The number to be added to the variance to avoid division
by zero when normalizing a value.
@param name: The optional name of the output node.
:param name: The optional name of the output node.
@return: The new node which performs BatchNormInference.
"""
inputs = as_nodes(data, gamma, beta, mean, variance)
@@ -72,9 +72,9 @@ def gather_nd(
) -> Node:
"""Return a node which performs GatherND.
@param data: N-D tensor with data for gathering
@param indices: K-D tensor of tuples with indices by which data is gathered
@param batch_dims: Scalar value of batch dimensions
:param data: N-D tensor with data for gathering
:param indices: K-D tensor of tuples with indices by which data is gathered
:param batch_dims: Scalar value of batch dimensions
@return: The new node which performs GatherND
"""
inputs = as_nodes(data, indices)
@@ -90,8 +90,8 @@ def gather_nd(
def log_softmax(data: NodeInput, axis: int, name: Optional[str] = None) -> Node:
"""Apply LogSoftmax operation on each element of input tensor.
@param data: The tensor providing input data.
@param axis: An axis along which LogSoftmax should be calculated
:param data: The tensor providing input data.
:param axis: An axis along which LogSoftmax should be calculated
@return: The new node with LogSoftmax operation applied on each element.
"""
return _get_node_factory_opset5().create("LogSoftmax", [as_node(data)], {"axis": axis})
@@ -112,17 +112,17 @@ def non_max_suppression(
) -> Node:
"""Return a node which performs NonMaxSuppression.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param max_output_boxes_per_class: Tensor Specifying maximum number of boxes
to be selected per class.
@param iou_threshold: Tensor specifying intersection over union threshold
@param score_threshold: Tensor specifying minimum score to consider box for the processing.
@param soft_nms_sigma: Tensor specifying the sigma parameter for Soft-NMS.
@param box_encoding: Format of boxes data encoding.
@param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
:param iou_threshold: Tensor specifying intersection over union threshold
:param score_threshold: Tensor specifying minimum score to consider box for the processing.
:param soft_nms_sigma: Tensor specifying the sigma parameter for Soft-NMS.
:param box_encoding: Format of boxes data encoding.
:param sort_result_descending: Flag that specifies whenever it is necessary to sort selected
boxes across batches or not.
@param output_type: Output element type.
:param output_type: Output element type.
@return: The new node which performs NonMaxSuppression
"""
if max_output_boxes_per_class is None:
@@ -153,11 +153,11 @@ def non_max_suppression(
def round(data: NodeInput, mode: str = "half_to_even", name: Optional[str] = None) -> Node:
"""Apply Round operation on each element of input tensor.
@param data: The tensor providing input data.
@param mode: Rule to round halfway cases. If set to 'half_to_even' then halfs round to the nearest even
:param data: The tensor providing input data.
:param mode: Rule to round halfway cases. If set to 'half_to_even' then halfs round to the nearest even
integer or rounding in such a way that the result heads away from zero if `mode` attribute is
'half_away_from_zero`.
@param name: An optional name of the output node.
:param name: An optional name of the output node.
@return: The new node with Round operation applied on each element.
"""
return _get_node_factory_opset5().create("Round", as_nodes(data), {"mode": mode.upper()})
@@ -182,28 +182,28 @@ def lstm_sequence(
) -> Node:
"""Return a node which performs LSTMSequence operation.
@param X: The input tensor. Shape: [batch_size, seq_length, input_size].
@param initial_hidden_state: The hidden state tensor.
:param X: The input tensor. Shape: [batch_size, seq_length, input_size].
:param initial_hidden_state: The hidden state tensor.
Shape: [batch_size, num_directions, hidden_size].
@param initial_cell_state: The cell state tensor.
:param initial_cell_state: The cell state tensor.
Shape: [batch_size, num_directions, hidden_size].
@param sequence_lengths: Specifies real sequence lengths for each batch element.
:param sequence_lengths: Specifies real sequence lengths for each batch element.
Shape: [batch_size]. Integer type.
@param W: Tensor with weights for matrix multiplication operation with input portion of data.
:param W: Tensor with weights for matrix multiplication operation with input portion of data.
Expected format: fico
Shape: [num_directions, 4*hidden_size, input_size].
@param R: The tensor with weights for matrix multiplication operation with hidden state.
:param R: The tensor with weights for matrix multiplication operation with hidden state.
Expected format: fico
Shape: [num_directions, 4*hidden_size, hidden_size].
@param B: The sum of biases (weight and recurrence). Expected format: fico
:param B: The sum of biases (weight and recurrence). Expected format: fico
Shape: [num_directions, 4*hidden_size].
@param hidden_size: Specifies hidden state size.
@param direction: Specifies if the RNN is forward, reverse, or bidirectional.
@param activations: The list of three activation functions for gates.
@param activations_alpha: The list of alpha parameters for activation functions.
@param activations_beta: The list of beta parameters for activation functions.
@param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
@param name: An optional name of the output node.
:param hidden_size: Specifies hidden state size.
:param direction: Specifies if the RNN is forward, reverse, or bidirectional.
:param activations: The list of three activation functions for gates.
:param activations_alpha: The list of alpha parameters for activation functions.
:param activations_beta: The list of beta parameters for activation functions.
:param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
:param name: An optional name of the output node.
@return: The new node represents LSTMSequence. Node outputs count: 3.
"""
@@ -230,7 +230,7 @@ def lstm_sequence(
def hsigmoid(data: NodeInput, name: Optional[str] = None,) -> Node:
"""Return a node which performs HSigmoid.
@param data: Tensor with input data floating point type.
:param data: Tensor with input data floating point type.
@return: The new node which performs HSigmoid
"""
return _get_node_factory_opset5().create("HSigmoid", as_nodes(data), {})
@@ -255,27 +255,27 @@ def gru_sequence(
) -> Node:
"""Return a node which performs GRUSequence operation.
@param X: The input tensor. Shape: [batch_size, seq_length, input_size].
@param initial_hidden_state: The hidden state tensor.
:param X: The input tensor. Shape: [batch_size, seq_length, input_size].
:param initial_hidden_state: The hidden state tensor.
Shape: [batch_size, num_directions, hidden_size].
@param sequence_lengths: Specifies real sequence lengths for each batch element.
:param sequence_lengths: Specifies real sequence lengths for each batch element.
Shape: [batch_size]. Integer type.
@param W: Tensor with weights for matrix multiplication operation with input portion of data.
:param W: Tensor with weights for matrix multiplication operation with input portion of data.
Shape: [num_directions, 3*hidden_size, input_size].
@param R: The tensor with weights for matrix multiplication operation with hidden state.
:param R: The tensor with weights for matrix multiplication operation with hidden state.
Shape: [num_directions, 3*hidden_size, hidden_size].
@param B: The sum of biases (weight and recurrence).
:param B: The sum of biases (weight and recurrence).
For linear_before_reset set True the shape is [num_directions, 4*hidden_size].
Otherwise the shape is [num_directions, 3*hidden_size].
@param hidden_size: Specifies hidden state size.
@param direction: Specifies if the RNN is forward, reverse, or bidirectional.
@param activations: The list of three activation functions for gates.
@param activations_alpha: The list of alpha parameters for activation functions.
@param activations_beta: The list of beta parameters for activation functions.
@param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
@param linear_before_reset: Flag denotes if the layer behaves according to the modification
:param hidden_size: Specifies hidden state size.
:param direction: Specifies if the RNN is forward, reverse, or bidirectional.
:param activations: The list of three activation functions for gates.
:param activations_alpha: The list of alpha parameters for activation functions.
:param activations_beta: The list of beta parameters for activation functions.
:param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
:param linear_before_reset: Flag denotes if the layer behaves according to the modification
of GRU described in the formula in the ONNX documentation.
@param name: An optional name of the output node.
:param name: An optional name of the output node.
@return: The new node represents GRUSequence. Node outputs count: 2.
"""
@@ -318,24 +318,24 @@ def rnn_sequence(
) -> Node:
"""Return a node which performs RNNSequence operation.
@param X: The input tensor. Shape: [batch_size, seq_length, input_size].
@param initial_hidden_state: The hidden state tensor.
:param X: The input tensor. Shape: [batch_size, seq_length, input_size].
:param initial_hidden_state: The hidden state tensor.
Shape: [batch_size, num_directions, hidden_size].
@param sequence_lengths: Specifies real sequence lengths for each batch element.
:param sequence_lengths: Specifies real sequence lengths for each batch element.
Shape: [batch_size]. Integer type.
@param W: Tensor with weights for matrix multiplication operation with input portion of data.
:param W: Tensor with weights for matrix multiplication operation with input portion of data.
Shape: [num_directions, hidden_size, input_size].
@param R: The tensor with weights for matrix multiplication operation with hidden state.
:param R: The tensor with weights for matrix multiplication operation with hidden state.
Shape: [num_directions, hidden_size, hidden_size].
@param B: The sum of biases (weight and recurrence).
:param B: The sum of biases (weight and recurrence).
Shape: [num_directions, hidden_size].
@param hidden_size: Specifies hidden state size.
@param direction: Specifies if the RNN is forward, reverse, or bidirectional.
@param activations: The list of three activation functions for gates.
@param activations_alpha: The list of alpha parameters for activation functions.
@param activations_beta: The list of beta parameters for activation functions.
@param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
@param name: An optional name of the output node.
:param hidden_size: Specifies hidden state size.
:param direction: Specifies if the RNN is forward, reverse, or bidirectional.
:param activations: The list of three activation functions for gates.
:param activations_alpha: The list of alpha parameters for activation functions.
:param activations_beta: The list of beta parameters for activation functions.
:param clip: Specifies bound values [-C, C] for tensor clipping performed before activations.
:param name: An optional name of the output node.
@return: The new node represents RNNSequence. Node outputs count: 2.
"""

42
src/bindings/python/src/openvino/runtime/opset6/ops.py
View File

@@ -49,9 +49,9 @@ def ctc_greedy_decoder_seq_len(
) -> Node:
"""Return a node which performs CTCGreedyDecoderSeqLen.
@param data: The input 3D tensor. Shape: [batch_size, seq_length, num_classes]
@param sequence_length: Input 1D tensor with sequence length. Shape: [batch_size]
@param blank_index: Scalar or 1D tensor with specifies the class index to use for the blank class.
:param data: The input 3D tensor. Shape: [batch_size, seq_length, num_classes]
:param sequence_length: Input 1D tensor with sequence length. Shape: [batch_size]
:param blank_index: Scalar or 1D tensor with specifies the class index to use for the blank class.
Optional parameter. Default value is num_classes-1.
@return: The new node which performs CTCGreedyDecoderSeqLen.
"""
@@ -78,9 +78,9 @@ def gather_elements(
) -> Node:
"""Return a node which performs GatherElements.
@param data: N-D tensor with data for gathering
@param indices: N-D tensor with indices by which data is gathered
@param axis: axis along which elements are gathered
:param data: N-D tensor with data for gathering
:param indices: N-D tensor with indices by which data is gathered
:param axis: axis along which elements are gathered
@return: The new node which performs GatherElements
"""
inputs = as_nodes(data, indices)
@@ -103,14 +103,14 @@ def mvn(
) -> Node:
"""Return a node which performs MeanVarianceNormalization (MVN).
@param data: The node with data tensor.
@param axes: The node with axes to reduce on.
@param normalize_variance: Denotes whether to perform variance normalization.
@param eps: The number added to the variance to avoid division by zero
:param data: The node with data tensor.
:param axes: The node with axes to reduce on.
:param normalize_variance: Denotes whether to perform variance normalization.
:param eps: The number added to the variance to avoid division by zero
when normalizing the value. Scalar value.
@param eps_mode: how eps is applied (`inside_sqrt` or `outside_sqrt`)
@param name: Optional output node name.
@return The new node performing a MVN operation on input tensor.
:param eps_mode: how eps is applied (`inside_sqrt` or `outside_sqrt`)
:param name: Optional output node name.
returns The new node performing a MVN operation on input tensor.
"""
inputs = as_nodes(data, axes)
@@ -127,10 +127,10 @@ def mvn(
def assign(new_value: NodeInput, variable_id: str, name: Optional[str] = None) -> Node:
"""Return a node which produces the Assign operation.
@param new_value: Node producing a value to be assigned to a variable.
@param variable_id: Id of a variable to be updated.
@param name: Optional name for output node.
@return Assign node
:param new_value: Node producing a value to be assigned to a variable.
:param variable_id: Id of a variable to be updated.
:param name: Optional name for output node.
returns Assign node
"""
return _get_node_factory_opset6().create(
"Assign",
@@ -143,10 +143,10 @@ def assign(new_value: NodeInput, variable_id: str, name: Optional[str] = None) -
def read_value(init_value: NodeInput, variable_id: str, name: Optional[str] = None) -> Node:
"""Return a node which produces the Assign operation.
@param init_value: Node producing a value to be returned instead of an unassigned variable.
@param variable_id: Id of a variable to be read.
@param name: Optional name for output node.
@return ReadValue node
:param init_value: Node producing a value to be returned instead of an unassigned variable.
:param variable_id: Id of a variable to be read.
:param name: Optional name for output node.
returns ReadValue node
"""
return _get_node_factory_opset6().create(
"ReadValue",

40
src/bindings/python/src/openvino/runtime/opset7/ops.py
View File

@@ -44,8 +44,8 @@ def einsum(
) -> Node:
"""Return a node which performs Einsum operation.
@param inputs: The list of input nodes
@param equation: Einsum equation
:param inputs: The list of input nodes
:param equation: Einsum equation
@return: The new node performing Einsum operation on the inputs
"""
attributes = {
@@ -63,10 +63,10 @@ def gelu(
) -> Node:
"""Return a node which performs Gelu activation function.
@param data: The node with data tensor.
@param approximation_mode: defines which approximation to use ('tanh' or 'erf')
@param name: Optional output node name.
@return The new node performing a Gelu activation with the input tensor.
:param data: The node with data tensor.
:param approximation_mode: defines which approximation to use ('tanh' or 'erf')
:param name: Optional output node name.
returns The new node performing a Gelu activation with the input tensor.
"""
inputs = as_nodes(data)
@@ -85,10 +85,10 @@ def roll(
) -> Node:
"""Return a node which performs Roll operation.
@param data: The node with data tensor.
@param shift: The node with the tensor with numbers of places by which elements are shifted.
@param axes: The node with the tensor with axes along which elements are shifted.
@return The new node performing a Roll operation on the input tensor.
:param data: The node with data tensor.
:param shift: The node with the tensor with numbers of places by which elements are shifted.
:param axes: The node with the tensor with axes along which elements are shifted.
returns The new node performing a Roll operation on the input tensor.
"""
inputs = as_nodes(data, shift, axes)
@@ -104,10 +104,10 @@ def gather(
) -> Node:
"""Return a node which performs Gather.
@param data: N-D tensor with data for gathering
@param indices: N-D tensor with indices by which data is gathered
@param axis: axis along which elements are gathered
@param batch_dims: number of batch dimensions
:param data: N-D tensor with data for gathering
:param indices: N-D tensor with indices by which data is gathered
:param axis: axis along which elements are gathered
:param batch_dims: number of batch dimensions
@return: The new node which performs Gather
"""
inputs = as_nodes(data, indices, axis)
@@ -124,9 +124,9 @@ def dft(
) -> Node:
"""Return a node which performs DFT operation.
@param data: Tensor with transformed data.
@param axes: Tensor with axes to transform.
@param signal_size: Tensor specifying signal size with respect to axes from the input 'axes'.
:param data: Tensor with transformed data.
:param axes: Tensor with axes to transform.
:param signal_size: Tensor specifying signal size with respect to axes from the input 'axes'.
@return: The new node which performs DFT operation on the input data tensor.
"""
if signal_size is None:
@@ -145,9 +145,9 @@ def idft(
) -> Node:
"""Return a node which performs IDFT operation.
@param data: Tensor with transformed data.
@param axes: Tensor with axes to transform.
@param signal_size: Tensor specifying signal size with respect to axes from the input 'axes'.
:param data: Tensor with transformed data.
:param axes: Tensor with axes to transform.
:param signal_size: Tensor specifying signal size with respect to axes from the input 'axes'.
@return: The new node which performs IDFT operation on the input data tensor.
"""
if signal_size is None:

236
src/bindings/python/src/openvino/runtime/opset8/ops.py
View File

@@ -47,22 +47,22 @@ def deformable_convolution(
) -> Node:
"""Return a node which performs deformable convolution operation.
@param data: The node providing data batch tensor.
@param offsets: The node providing offset tensor.
@param filters: The node providing filters tensor.
@param strides: The distance (in pixels) to slide the filter on the feature map over the axes.
@param pads_begin: The number of pixels to add to the beginning along each axis.
@param pads_end: The number of pixels to add to the end along each axis.
@param dilations: The distance in width and height between elements (weights) in the filter.
@param mask: The node providing modulation scalar (mask) tensor.
@param auto_pad: The type of padding. Range of values: explicit, same_upper, same_lower, valid.
@param group: The number of groups which both output and input should be split into.
@param deformable_group: The number of groups which deformable values and output should be split
:param data: The node providing data batch tensor.
:param offsets: The node providing offset tensor.
:param filters: The node providing filters tensor.
:param strides: The distance (in pixels) to slide the filter on the feature map over the axes.
:param pads_begin: The number of pixels to add to the beginning along each axis.
:param pads_end: The number of pixels to add to the end along each axis.
:param dilations: The distance in width and height between elements (weights) in the filter.
:param mask: The node providing modulation scalar (mask) tensor.
:param auto_pad: The type of padding. Range of values: explicit, same_upper, same_lower, valid.
:param group: The number of groups which both output and input should be split into.
:param deformable_group: The number of groups which deformable values and output should be split
into along the channel axis.
@param bilinear_interpolation_pad: The flag that determines the mode of bilinear interpolation
:param bilinear_interpolation_pad: The flag that determines the mode of bilinear interpolation
execution.
@param name: The optional new name for output node.
@return New node performing deformable convolution operation.
:param name: The optional new name for output node.
returns New node performing deformable convolution operation.
"""
if mask is None:
inputs = as_nodes(data, offsets, filters)
@@ -92,8 +92,8 @@ def adaptive_avg_pool(
) -> Node:
"""Return a node which performs AdaptiveAvgPool operation.
@param data: The list of input nodes
@param output_shape: the shape of spatial dimentions after operation
:param data: The list of input nodes
:param output_shape: the shape of spatial dimentions after operation
@return: The new node performing AdaptiveAvgPool operation on the data
"""
inputs = as_nodes(data, output_shape)
@@ -108,9 +108,9 @@ def adaptive_max_pool(
) -> Node:
"""Return a node which performs AdaptiveMaxPool operation.
@param data: The list of input nodes
@param output_shape: the shape of spatial dimentions after operation
@param index_element_type: Type of indices output.
:param data: The list of input nodes
:param output_shape: the shape of spatial dimentions after operation
:param index_element_type: Type of indices output.
@return: The new node performing AdaptiveMaxPool operation on the data
"""
inputs = as_nodes(data, output_shape)
@@ -139,25 +139,25 @@ def multiclass_nms(
) -> Node:
"""Return a node which performs MulticlassNms.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param sort_result_type: Specifies order of output elements, possible values:
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param sort_result_type: Specifies order of output elements, possible values:
'class': sort selected boxes by class id (ascending)
'score': sort selected boxes by score (descending)
'none': do not guarantee the order.
@param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
:param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
across batches or not
@param output_type: Specifies the output tensor type, possible values:
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32'
@param iou_threshold: Specifies intersection over union threshold
@param score_threshold: Specifies minimum score to consider box for the processing
@param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
:param iou_threshold: Specifies intersection over union threshold
:param score_threshold: Specifies minimum score to consider box for the processing
:param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
to keep all boxes
@param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
:param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
meaning to keep all boxes
@param background_class: Specifies the background class id, -1 meaning to keep all classes
@param nms_eta: Specifies eta parameter for adpative NMS, in close range [0, 1.0]
@param normalized: Specifies whether boxes are normalized or not
:param background_class: Specifies the background class id, -1 meaning to keep all classes
:param nms_eta: Specifies eta parameter for adpative NMS, in close range [0, 1.0]
:param normalized: Specifies whether boxes are normalized or not
@return: The new node which performs MuticlassNms
"""
inputs = as_nodes(boxes, scores)
@@ -196,28 +196,28 @@ def matrix_nms(
) -> Node:
"""Return a node which performs MatrixNms.
@param boxes: Tensor with box coordinates.
@param scores: Tensor with box scores.
@param sort_result_type: Specifies order of output elements, possible values:
:param boxes: Tensor with box coordinates.
:param scores: Tensor with box scores.
:param sort_result_type: Specifies order of output elements, possible values:
'class': sort selected boxes by class id (ascending)
'score': sort selected boxes by score (descending)
'none': do not guarantee the order.
@param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
:param sort_result_across_batch: Specifies whenever it is necessary to sort selected boxes
across batches or not
@param output_type: Specifies the output tensor type, possible values:
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32'
@param score_threshold: Specifies minimum score to consider box for the processing
@param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
:param score_threshold: Specifies minimum score to consider box for the processing
:param nms_top_k: Specifies maximum number of boxes to be selected per class, -1 meaning
to keep all boxes
@param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
:param keep_top_k: Specifies maximum number of boxes to be selected per batch element, -1
meaning to keep all boxes
@param background_class: Specifies the background class id, -1 meaning to keep all classes
@param decay_function: Specifies decay function used to decay scores, possible values:
:param background_class: Specifies the background class id, -1 meaning to keep all classes
:param decay_function: Specifies decay function used to decay scores, possible values:
'gaussian', 'linear'
@param gaussian_sigma: Specifies gaussian_sigma parameter for gaussian decay_function
@param post_threshold: Specifies threshold to filter out boxes with low confidence score
:param gaussian_sigma: Specifies gaussian_sigma parameter for gaussian decay_function
:param post_threshold: Specifies threshold to filter out boxes with low confidence score
after decaying
@param normalized: Specifies whether boxes are normalized or not
:param normalized: Specifies whether boxes are normalized or not
@return: The new node which performs MatrixNms
"""
inputs = as_nodes(boxes, scores)
@@ -248,11 +248,11 @@ def gather(
) -> Node:
"""Return a node which performs Gather with support of negative indices.
@param data: N-D tensor with data for gathering
@param indices: N-D tensor with indices by which data is gathered. Negative indices
:param data: N-D tensor with data for gathering
:param indices: N-D tensor with indices by which data is gathered. Negative indices
indicate reverse indexing from the end
@param axis: axis along which elements are gathered
@param batch_dims: number of batch dimensions
:param axis: axis along which elements are gathered
:param batch_dims: number of batch dimensions
@return: The new node which performs Gather
"""
inputs = as_nodes(data, indices, axis)
@@ -278,25 +278,25 @@ def max_pool(
) -> Node:
"""Perform max pooling operation and return both values and indices of the selected elements.
@param data: The node providing input data.
@param strides: The distance (in pixels) to slide the filter on the feature map
:param data: The node providing input data.
:param strides: The distance (in pixels) to slide the filter on the feature map
over the axes.
@param dilations: The dilation of filter elements(distance between elements).
@param pads_begin: The number of pixels to add at the beginning along each axis.
@param pads_end: The number of pixels to add at the end along each axis.
@param kernel_shape: The pooling operation kernel shape.
@param rounding_type: Determines used rounding schema when computing output shape.
:param dilations: The dilation of filter elements(distance between elements).
:param pads_begin: The number of pixels to add at the beginning along each axis.
:param pads_end: The number of pixels to add at the end along each axis.
:param kernel_shape: The pooling operation kernel shape.
:param rounding_type: Determines used rounding schema when computing output shape.
Acceptable values are: ['floor', 'ceil']. Defaults to 'floor'.
@param auto_pad: Determines how the padding is calculated. Acceptable values:
:param auto_pad: Determines how the padding is calculated. Acceptable values:
[None, 'same_upper', 'same_lower', 'valid']. Defaults to None.
@param index_element_type: The data type used for the indices output of this operator.
:param index_element_type: The data type used for the indices output of this operator.
Defaults to i64.
@param axis: The first dimension in the data shape used to determine the maximum
:param axis: The first dimension in the data shape used to determine the maximum
returned index value. The value is the product of all dimensions
starting at the provided axis. Defaults to 0.
@param name: The optional name for the created output node.
:param name: The optional name for the created output node.
@return The new node performing max pooling operation.
returns The new node performing max pooling operation.
"""
if auto_pad is None:
auto_pad = "explicit"
@@ -328,14 +328,14 @@ def random_uniform(
) -> Node:
"""Return a node which generates sequence of random values from uniform distribution.
@param output_shape: Tensor with shape of the output tensor.
@param min_val: Tensor with the lower bound on the range of random values to generate.
@param max_val: Tensor with the upper bound on the range of random values to generate.
@param output_type: Specifies the output tensor type, possible values:
:param output_shape: Tensor with shape of the output tensor.
:param min_val: Tensor with the lower bound on the range of random values to generate.
:param max_val: Tensor with the upper bound on the range of random values to generate.
:param output_type: Specifies the output tensor type, possible values:
'i64', 'i32', 'f64', 'f32', 'f16', 'bf16'.
@param global_seed: Specifies global seed value. Required to be a positive integer or 0.
@param op_seed: Specifies operational seed value. Required to be a positive integer or 0.
@return The new node which performs generation of random values from uniform distribution.
:param global_seed: Specifies global seed value. Required to be a positive integer or 0.
:param op_seed: Specifies operational seed value. Required to be a positive integer or 0.
returns The new node which performs generation of random values from uniform distribution.
"""
inputs = as_nodes(output_shape, min_val, max_val)
@@ -364,13 +364,13 @@ def slice(
) -> Node:
"""Return a node which generates Slice operation.
@param data: The node providing input data.
@param start: The node providing start indices (inclusively).
@param stop: The node providing stop indices (exclusively).
@param step: The node providing step values.
@param axes: The optional node providing axes to slice, default [0, 1, ..., len(start)-1].
@param name: The optional name for the created output node.
@return The new node performing Slice operation.
:param data: The node providing input data.
:param start: The node providing start indices (inclusively).
:param stop: The node providing stop indices (exclusively).
:param step: The node providing step values.
:param axes: The optional node providing axes to slice, default [0, 1, ..., len(start)-1].
:param name: The optional name for the created output node.
returns The new node performing Slice operation.
"""
if axes is None:
inputs = as_nodes(data, start, stop, step)
@@ -389,9 +389,9 @@ def gather_nd(
) -> Node:
"""Return a node which performs GatherND.
@param data: N-D tensor with data for gathering
@param indices: K-D tensor of tuples with indices by which data is gathered
@param batch_dims: Scalar value of batch dimensions
:param data: N-D tensor with data for gathering
:param indices: K-D tensor of tuples with indices by which data is gathered
:param batch_dims: Scalar value of batch dimensions
@return: The new node which performs GatherND
"""
inputs = as_nodes(data, indices)
@@ -409,11 +409,11 @@ def prior_box(
) -> Node:
"""Generate prior boxes of specified sizes and aspect ratios across all dimensions.
@param layer_shape: Shape of layer for which prior boxes are computed.
@param image_shape: Shape of image to which prior boxes are scaled.
@param attrs: The dictionary containing key, value pairs for attributes.
@param name: Optional name for the output node.
@return Node representing prior box operation.
:param layer_shape: Shape of layer for which prior boxes are computed.
:param image_shape: Shape of image to which prior boxes are scaled.
:param attrs: The dictionary containing key, value pairs for attributes.
:param name: Optional name for the output node.
returns Node representing prior box operation.
Available attributes are:
* min_size The minimum box size (in pixels).
Range of values: positive floating point numbers
@@ -474,7 +474,9 @@ def prior_box(
Default value: True
Required: no
Example of attribute dictionary:
@code{.py}
.. code-block:: python
# just required ones
attrs = {
'offset': 85,
@@ -485,7 +487,7 @@ def prior_box(
'clip': True,
'fixed_size': [32, 64, 128]
}
@endcode
Optional attributes which are absent from dictionary will be set with corresponding default.
"""
requirements = [
@@ -518,11 +520,11 @@ def i420_to_bgr(
) -> Node:
"""Return a node which performs I420toBGR operation.
@param arg: The node providing single or Y plane data.
@param arg_u: The node providing U plane data. Required for separate planes.
@param arg_v: The node providing V plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing I420toBGR operation.
:param arg: The node providing single or Y plane data.
:param arg_u: The node providing U plane data. Required for separate planes.
:param arg_v: The node providing V plane data. Required for separate planes.
:param name: The optional name for the created output node.
returns The new node performing I420toBGR operation.
"""
if arg_u is None and arg_v is None:
inputs = as_nodes(arg)
@@ -545,11 +547,11 @@ def i420_to_rgb(
) -> Node:
"""Return a node which performs I420toRGB operation.
@param arg: The node providing single or Y plane data.
@param arg_u: The node providing U plane data. Required for separate planes.
@param arg_v: The node providing V plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing I420toRGB operation.
:param arg: The node providing single or Y plane data.
:param arg_u: The node providing U plane data. Required for separate planes.
:param arg_v: The node providing V plane data. Required for separate planes.
:param name: The optional name for the created output node.
returns The new node performing I420toRGB operation.
"""
if arg_u is None and arg_v is None:
inputs = as_nodes(arg)
@@ -571,10 +573,10 @@ def nv12_to_bgr(
) -> Node:
"""Return a node which performs NV12toBGR operation.
@param arg: The node providing single or Y plane data.
@param arg_uv: The node providing UV plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing NV12toBGR operation.
:param arg: The node providing single or Y plane data.
:param arg_uv: The node providing UV plane data. Required for separate planes.
:param name: The optional name for the created output node.
returns The new node performing NV12toBGR operation.
"""
if arg_uv is None:
inputs = as_nodes(arg)
@@ -592,10 +594,10 @@ def nv12_to_rgb(
) -> Node:
"""Return a node which performs NV12toRGB operation.
@param arg: The node providing single or Y plane data.
@param arg_uv: The node providing UV plane data. Required for separate planes.
@param name: The optional name for the created output node.
@return The new node performing NV12toRGB operation.
:param arg: The node providing single or Y plane data.
:param arg_uv: The node providing UV plane data. Required for separate planes.
:param name: The optional name for the created output node.
returns The new node performing NV12toRGB operation.
"""
if arg_uv is None:
inputs = as_nodes(arg)
@@ -617,14 +619,14 @@ def detection_output(
) -> Node:
"""Generate the detection output using information on location and confidence predictions.
@param box_logits: The 2D input tensor with box logits.
@param class_preds: The 2D input tensor with class predictions.
@param proposals: The 3D input tensor with proposals.
@param attrs: The dictionary containing key, value pairs for attributes.
@param aux_class_preds: The 2D input tensor with additional class predictions information.
@param aux_box_preds: The 2D input tensor with additional box predictions information.
@param name: Optional name for the output node.
@return Node representing DetectionOutput operation.
:param box_logits: The 2D input tensor with box logits.
:param class_preds: The 2D input tensor with class predictions.
:param proposals: The 3D input tensor with proposals.
:param attrs: The dictionary containing key, value pairs for attributes.
:param aux_class_preds: The 2D input tensor with additional class predictions information.
:param aux_box_preds: The 2D input tensor with additional box predictions information.
:param name: Optional name for the output node.
returns Node representing DetectionOutput operation.
Available attributes are:
* background_label_id The background label id.
Range of values: integer value
@@ -693,7 +695,9 @@ def detection_output(
Default value: 0
Required: no
Example of attribute dictionary:
@code{.py}
.. code-block:: python
# just required ones
attrs = {
'keep_top_k': [1, 2, 3],
@@ -707,7 +711,7 @@ def detection_output(
'input_height': [32],
'input_width': [32],
}
@endcode
Optional attributes which are absent from dictionary will be set with corresponding default.
"""
requirements = [
@@ -744,9 +748,9 @@ def detection_output(
def softmax(data: NodeInput, axis: int, name: Optional[str] = None) -> Node:
"""Apply softmax operation on each element of input tensor.
@param data: The tensor providing input data.
@param axis: An axis along which Softmax should be calculated. Can be positive or negative.
@param name: Optional name for the node.
@return The new node with softmax operation applied on each element.
:param data: The tensor providing input data.
:param axis: An axis along which Softmax should be calculated. Can be positive or negative.
:param name: Optional name for the node.
returns The new node with softmax operation applied on each element.
"""
return _get_node_factory_opset8().create("Softmax", [as_node(data)], {"axis": axis})

8
src/bindings/python/src/openvino/runtime/utils/broadcasting.py
View File

@@ -19,10 +19,10 @@ def get_broadcast_axes(
replicating elements from the input tensor as needed to fill the new dimensions.
Function calculate which of the output axes are added in this way.
@param output_shape: The new shape for the output tensor.
@param input_shape: The shape of input tensor.
@param axis: The axis along which we want to replicate elements.
@return The indices of added axes.
:param output_shape: The new shape for the output tensor.
:param input_shape: The shape of input tensor.
:param axis: The axis along which we want to replicate elements.
returns The indices of added axes.
"""
axes_indexes = list(range(0, len(output_shape)))
if axis is None:

42
src/bindings/python/src/openvino/runtime/utils/input_validation.py
View File

@@ -28,14 +28,14 @@ def _check_value(op_name, attr_key, value, val_type, cond=None):
# type: (str, str, Any, Type, Optional[Callable[[Any], bool]]) -> bool
"""Check whether provided value satisfies specified criteria.
@param op_name: The operator name which attributes are checked.
@param attr_key: The attribute name.
@param value: The value to check.
@param val_type: Required value type.
@param cond: The optional function running additional checks.
:param op_name: The operator name which attributes are checked.
:param attr_key: The attribute name.
:param value: The value to check.
:param val_type: Required value type.
:param cond: The optional function running additional checks.
:raises UserInputError:
@return True if attribute satisfies all criterias. Otherwise False.
returns True if attribute satisfies all criterias. Otherwise False.
"""
if not np.issubdtype(type(value), val_type):
raise UserInputError(
@@ -56,17 +56,17 @@ def check_valid_attribute(op_name, attr_dict, attr_key, val_type, cond=None, req
# type: (str, dict, str, Type, Optional[Callable[[Any], bool]], Optional[bool]) -> bool
"""Check whether specified attribute satisfies given criteria.
@param op_name: The operator name which attributes are checked.
@param attr_dict: Dictionary containing key-value attributes to check.
@param attr_key: Key value for validated attribute.
@param val_type: Value type for validated attribute.
@param cond: Any callable wich accept attribute value and returns True or False.
@param required: Whether provided attribute key is not required. This mean it may be missing
:param op_name: The operator name which attributes are checked.
:param attr_dict: Dictionary containing key-value attributes to check.
:param attr_key: Key value for validated attribute.
:param val_type: Value type for validated attribute.
:param cond: Any callable wich accept attribute value and returns True or False.
:param required: Whether provided attribute key is not required. This mean it may be missing
from provided dictionary.
:raises UserInputError:
@return True if attribute satisfies all criterias. Otherwise False.
returns True if attribute satisfies all criterias. Otherwise False.
"""
result = True
@@ -99,9 +99,9 @@ def check_valid_attributes(
# type: (...) -> bool
"""Perform attributes validation according to specified type, value criteria.
@param op_name: The operator name which attributes are checked.
@param attributes: The dictionary with user provided attributes to check.
@param requirements: The list of tuples describing attributes' requirements. The tuple should
:param op_name: The operator name which attributes are checked.
:param attributes: The dictionary with user provided attributes to check.
:param requirements: The list of tuples describing attributes' requirements. The tuple should
contain following values:
(attr_name: str,
is_required: bool,
@@ -109,7 +109,7 @@ def check_valid_attributes(
value_condition: Callable)
:raises UserInputError:
@return True if all attributes satisfies criterias. Otherwise False.
returns True if all attributes satisfies criterias. Otherwise False.
"""
for attr, required, val_type, cond in requirements:
check_valid_attribute(op_name, attributes, attr, val_type, cond, required)
@@ -119,9 +119,9 @@ def check_valid_attributes(
def is_positive_value(x): # type: (Any) -> bool
"""Determine whether the specified x is positive value.
@param x: The value to check.
:param x: The value to check.
@return True if the specified x is positive value, False otherwise.
returns True if the specified x is positive value, False otherwise.
"""
return x > 0
@@ -129,8 +129,8 @@ def is_positive_value(x): # type: (Any) -> bool
def is_non_negative_value(x): # type: (Any) -> bool
"""Determine whether the specified x is non-negative value.
@param x: The value to check.
:param x: The value to check.
@return True if the specified x is non-negative value, False otherwise.
returns True if the specified x is non-negative value, False otherwise.
"""
return x >= 0

6
src/bindings/python/src/openvino/runtime/utils/reduction.py
View File

@@ -11,9 +11,9 @@ def get_reduction_axes(node: Node, reduction_axes: Optional[Iterable[int]]) -> I
If reduction_axes is None we default to reduce all axes.
@param node: The node we fill reduction axes for.
@param reduction_axes: The collection of indices of axes to reduce. May be None.
@return Set filled with indices of axes we want to reduce.
:param node: The node we fill reduction axes for.
:param reduction_axes: The collection of indices of axes to reduce. May be None.
returns Set filled with indices of axes we want to reduce.
"""
if reduction_axes is None:
reduction_axes = set(range(len(node.shape)))

2
src/bindings/python/src/openvino/utils.py
View File

@@ -6,7 +6,7 @@ import sys
def add_openvino_libs_to_path() -> None:
"""Adds OpenVINO libs to path on Win OS."""
"""Adds OpenVINO libraries to the PATH environment variable on Windows."""
if sys.platform == "win32":
# Installer, yum, pip installs openvino dlls to the different directories
# and those paths need to be visible to the openvino modules

6
src/bindings/python/src/pyopenvino/CMakeLists.txt
View File

@@ -92,7 +92,7 @@ if(OpenVINO_SOURCE_DIR OR InferenceEngineDeveloperPackage_FOUND)
ie_cpack_add_component(${OV_COMP_PYTHON_OPENVINO}_${PYTHON_VERSION})
install(DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/../openvino
install(DIRECTORY ${PYTHON_SOURCE_DIR}/openvino
DESTINATION ${IE_CPACK_PLUGIN_PATH}/${PYTHON_VERSION}
COMPONENT ${OV_COMP_PYTHON_OPENVINO}_${PYTHON_VERSION}
USE_SOURCE_PERMISSIONS
@@ -101,6 +101,10 @@ if(OpenVINO_SOURCE_DIR OR InferenceEngineDeveloperPackage_FOUND)
install(TARGETS ${PROJECT_NAME}
DESTINATION ${IE_CPACK_PLUGIN_PATH}/${PYTHON_VERSION}/openvino
COMPONENT ${OV_COMP_PYTHON_OPENVINO}_${PYTHON_VERSION})
install(DIRECTORY ${PYTHON_BINDINGS_ROOT_DIR}/tests
DESTINATION tests/${PROJECT_NAME}
COMPONENT tests EXCLUDE_FROM_ALL)
endif()
if(TARGET ie_wheel)

153
src/bindings/python/src/pyopenvino/core/async_infer_queue.cpp
View File

@@ -140,10 +140,12 @@ public:
void regclass_AsyncInferQueue(py::module m) {
py::class_<AsyncInferQueue, std::shared_ptr<AsyncInferQueue>> cls(m, "AsyncInferQueue");
cls.doc() = "openvino.runtime.AsyncInferQueue represents helper that creates a pool of asynchronous"
"InferRequests and provides synchronization functions to control flow of a simple pipeline.";
cls.def(py::init([](ov::CompiledModel& net, size_t jobs) {
cls.def(py::init([](ov::CompiledModel& model, size_t jobs) {
if (jobs == 0) {
jobs = (size_t)Common::get_optimal_number_of_requests(net);
jobs = (size_t)Common::get_optimal_number_of_requests(model);
}
std::vector<InferRequestWrapper> requests;
@@ -151,10 +153,10 @@ void regclass_AsyncInferQueue(py::module m) {
std::vector<py::object> user_ids(jobs);
for (size_t handle = 0; handle < jobs; handle++) {
auto request = InferRequestWrapper(net.create_infer_request());
// Get Inputs and Outputs info from executable network
request._inputs = net.inputs();
request._outputs = net.outputs();
auto request = InferRequestWrapper(model.create_infer_request());
// Get Inputs and Outputs info from compiled model
request._inputs = model.inputs();
request._outputs = model.outputs();
requests.push_back(request);
idle_handles.push(handle);
@@ -162,8 +164,18 @@ void regclass_AsyncInferQueue(py::module m) {
return new AsyncInferQueue(requests, idle_handles, user_ids);
}),
py::arg("network"),
py::arg("jobs") = 0);
py::arg("model"),
py::arg("jobs") = 0,
R"(
Creates AsyncInferQueue.
:param model: Model to be used to create InferRequests in a pool.
:type model: openvino.runtime.CompiledModel
:param jobs: Number of InferRequests objects in a pool. If 0, jobs number
will be set automatically to the optimal number. Default: 0
:type jobs: int
:rtype: openvino.runtime.AsyncInferQueue
)");
cls.def(
"start_async",
@@ -185,27 +197,94 @@ void regclass_AsyncInferQueue(py::module m) {
}
},
py::arg("inputs"),
py::arg("userdata"));
py::arg("userdata"),
R"(
Run asynchronous inference using next available InferRequest.
cls.def("is_ready", [](AsyncInferQueue& self) {
return self._is_ready();
});
This function releases the GIL, so another Python thread can
work while this function runs in the background.
cls.def("wait_all", [](AsyncInferQueue& self) {
return self.wait_all();
});
:param inputs: Data to set on input tensors of next available InferRequest from
AsyncInferQueue's pool.
:type inputs: dict[Union[int, str, openvino.runtime.ConstOutput] : openvino.runtime.Tensor]
:param userdata: Any data that will be passed to a callback
:rtype: None
)");
cls.def("get_idle_request_id", [](AsyncInferQueue& self) {
return self.get_idle_request_id();
});
cls.def(
"is_ready",
[](AsyncInferQueue& self) {
return self._is_ready();
},
R"(
One of 'flow control' functions.
Returns True if any free request in the pool, otherwise False.
cls.def("set_callback", [](AsyncInferQueue& self, py::function f_callback) {
self.set_custom_callbacks(f_callback);
});
Function releases GIL, other threads can work while this function waits.
cls.def("__len__", [](AsyncInferQueue& self) {
return self._requests.size();
});
:return: If there is at least one free InferRequest in a pool, returns True.
:rtype: bool
)");
cls.def(
"wait_all",
[](AsyncInferQueue& self) {
return self.wait_all();
},
R"(
One of 'flow control' functions. Blocking call.
Waits for all InferRequests in a pool to finish scheduled work.
Function releases GIL, other threads can work while this function waits.
)");
cls.def(
"get_idle_request_id",
[](AsyncInferQueue& self) {
return self.get_idle_request_id();
},
R"(
Returns next free id of InferRequest from queue's pool.
Function waits for any request to complete and then returns this request's id.
Function releases GIL, other threads can work while this function waits.
:rtype: int
)");
cls.def(
"set_callback",
[](AsyncInferQueue& self, py::function callback) {
self.set_custom_callbacks(callback);
},
R"(
Sets unified callback on all InferRequests from queue's pool.
Signature of such function should have two arguments, where
first one is InferRequest object and second one is userdata
connected to InferRequest from the AsyncInferQueue's pool.
.. code-block:: python
def f(request, userdata):
result = request.output_tensors[0]
print(result + userdata)
async_infer_queue.set_callback(f)
:param callback: Any Python defined function that matches callback's requirements.
:type callback: function
)");
cls.def(
"__len__",
[](AsyncInferQueue& self) {
return self._requests.size();
},
R"(
Number of InferRequests in the pool.
:rtype: int
)");
cls.def(
"__iter__",
@@ -214,11 +293,25 @@ void regclass_AsyncInferQueue(py::module m) {
},
py::keep_alive<0, 1>()); /* Keep set alive while iterator is used */
cls.def("__getitem__", [](AsyncInferQueue& self, size_t i) {
return self._requests[i];
});
cls.def(
"__getitem__",
[](AsyncInferQueue& self, size_t i) {
return self._requests[i];
},
R"(
:param i: InferRequest id
:type i: int
:return: InferRequests from the pool with given id.
:rtype: openvino.runtime.InferRequest
)");
cls.def_property_readonly("userdata", [](AsyncInferQueue& self) {
return self._user_ids;
});
cls.def_property_readonly(
"userdata",
[](AsyncInferQueue& self) {
return self._user_ids;
},
R"(
:return: List of all passed userdata. None if the data wasn't passed yet.
:rtype: List[Any]
)");
}

11
src/bindings/python/src/pyopenvino/core/common.cpp
View File

@@ -47,6 +47,17 @@ const std::map<std::string, ov::element::Type>& dtype_to_ov_type() {
return dtype_to_ov_type_mapping;
}
ov::Tensor tensor_from_pointer(py::array& array, const ov::Shape& shape) {
bool is_contiguous = C_CONTIGUOUS == (array.flags() & C_CONTIGUOUS);
auto type = Common::dtype_to_ov_type().at(py::str(array.dtype()));
if (is_contiguous) {
return ov::Tensor(type, shape, const_cast<void*>(array.data(0)), {});
} else {
throw ov::Exception("Tensor with shared memory must be C contiguous!");
}
}
ov::Tensor tensor_from_numpy(py::array& array, bool shared_memory) {
// Check if passed array has C-style contiguous memory layout.
bool is_contiguous = C_CONTIGUOUS == (array.flags() & C_CONTIGUOUS);

2
src/bindings/python/src/pyopenvino/core/common.hpp
View File

@@ -27,6 +27,8 @@ const std::map<ov::element::Type, py::dtype>& ov_type_to_dtype();
const std::map<std::string, ov::element::Type>& dtype_to_ov_type();
ov::Tensor tensor_from_pointer(py::array& array, const ov::Shape& shape);
ov::Tensor tensor_from_numpy(py::array& array, bool shared_memory);
py::array as_contiguous(py::array& array, ov::element::Type type);

198
src/bindings/python/src/pyopenvino/core/compiled_model.cpp
View File

@@ -4,6 +4,7 @@
#include "openvino/runtime/compiled_model.hpp"
#include <pybind11/iostream.h>
#include <pybind11/stl.h>
#include <pyopenvino/graph/any.hpp>
@@ -19,15 +20,26 @@ namespace py = pybind11;
void regclass_CompiledModel(py::module m) {
py::class_<ov::CompiledModel, std::shared_ptr<ov::CompiledModel>> cls(m, "CompiledModel");
cls.doc() = "openvino.runtime.CompiledModel represents Model that is compiled for a specific device by applying "
"multiple optimization transformations, then mapping to compute kernels.";
cls.def(py::init([](ov::CompiledModel& other) {
return other;
}),
py::arg("other"));
cls.def("create_infer_request", [](ov::CompiledModel& self) {
return std::make_shared<InferRequestWrapper>(self.create_infer_request(), self.inputs(), self.outputs());
});
cls.def(
"create_infer_request",
[](ov::CompiledModel& self) {
return std::make_shared<InferRequestWrapper>(self.create_infer_request(), self.inputs(), self.outputs());
},
R"(
Creates an inference request object used to infer the compiled model.
The created request has allocated input and output tensors.
:return: New InferRequest object.
:rtype: openvino.runtime.InferRequest
)");
cls.def(
"infer_new_request",
@@ -38,7 +50,20 @@ void regclass_CompiledModel(py::module m) {
request.infer();
return Common::outputs_to_dict(self.outputs(), request);
},
py::arg("inputs"));
py::arg("inputs"),
R"(
Infers specified input(s) in synchronous mode.
Blocks all methods of CompiledModel while request is running.
Method creates new temporary InferRequest and run inference on it.
It is advised to use dedicated InferRequest class for performance,
optimizing workflows and creating advanced pipelines.
:param inputs: Data to set on input tensors.
:type inputs: Dict[Union[int, str, openvino.runtime.ConstOutput], openvino.runtime.Tensor]
:return: Dictionary of results from output tensors with ports as keys.
:rtype: Dict[openvino.runtime.ConstOutput, numpy.array]
)");
cls.def(
"export_model",
@@ -50,25 +75,19 @@ void regclass_CompiledModel(py::module m) {
R"(
Exports the compiled model to bytes/output stream.
Parameters
----------
None
:return: Bytes object that contains this compiled model.
:rtype: bytes
Returns
----------
export_model : bytes
Bytes object that contains this compiled model.
.. code-block:: python
Examples
----------
user_stream = compiled.export_model()
user_stream = compiled.export_model()
with open('./my_model', 'wb') as f:
f.write(user_stream)
with open('./my_model', 'wb') as f:
f.write(user_stream)
# ...
# ...
new_compiled = core.import_model(user_stream, "CPU")
new_compiled = core.import_model(user_stream, "CPU")
)");
cls.def(
@@ -95,26 +114,21 @@ void regclass_CompiledModel(py::module m) {
Function performs flushing of the stream, writes to it and then rewinds
the stream to the beginning (using seek(0)).
Parameters
----------
model_stream : io.BytesIO
A stream object to which the model will be serialized.
:param model_stream: A stream object to which the model will be serialized.
:type model_stream: io.BytesIO
:rtype: None
Returns
----------
export_model : None
.. code-block:: python
Examples
----------
user_stream = io.BytesIO()
compiled.export_model(user_stream)
user_stream = io.BytesIO()
compiled.export_model(user_stream)
with open('./my_model', 'wb') as f:
f.write(user_stream.getvalue()) # or read() if seek(0) was applied before
with open('./my_model', 'wb') as f:
f.write(user_stream.getvalue()) # or read() if seek(0) was applied before
# ...
# ...
new_compiled = core.import_model(user_stream, "CPU")
new_compiled = core.import_model(user_stream, "CPU")
)");
cls.def(
@@ -129,13 +143,10 @@ void regclass_CompiledModel(py::module m) {
py::arg("properties"),
R"(
Sets properties for current compiled model.
Parameters
----------
properties : dict
Dict of pairs: (property name, property value)
Returns
----------
set_property : None
:param properties: Dict of pairs: (property name, property value)
:type properties: dict
:rtype: None
)");
// todo: remove after Accuracy Checker migration to set/get_property API
@@ -155,13 +166,10 @@ void regclass_CompiledModel(py::module m) {
py::arg("name"),
R"(
Gets properties for current compiled model.
Parameters
----------
name : str
Property name.
Returns
----------
get_property : Any
:param name: Property name.
:type name: str
:rtype: Any
)");
// todo: remove after Accuracy Checker migration to set/get_property API
@@ -173,29 +181,107 @@ void regclass_CompiledModel(py::module m) {
},
py::arg("name"));
cls.def("get_runtime_model", &ov::CompiledModel::get_runtime_model);
cls.def("get_runtime_model",
&ov::CompiledModel::get_runtime_model,
R"(
Gets runtime model information from a device.
cls.def_property_readonly("inputs", &ov::CompiledModel::inputs);
This object (returned model) represents the internal device specific model
which is optimized for particular accelerator. It contains device specific nodes,
runtime information and can be used only to understand how the source model
is optimized and which kernels, element types and layouts are selected.
cls.def("input", (ov::Output<const ov::Node>(ov::CompiledModel::*)() const) & ov::CompiledModel::input);
:return: Model containing Executable Graph information.
:rtype: openvino.runtime.Model
)");
cls.def_property_readonly("inputs",
&ov::CompiledModel::inputs,
R"(
Gets all inputs of a compiled model.
:return: Inputs of a compiled model.
:rtype: List[openvino.runtime.ConstOutput]
)");
cls.def("input",
(ov::Output<const ov::Node>(ov::CompiledModel::*)() const) & ov::CompiledModel::input,
R"(
Gets a single input of a compiled model.
If a model has more than one input, this method throws an exception.
:return: A compiled model input.
:rtype: openvino.runtime.ConstOutput
)");
cls.def("input",
(ov::Output<const ov::Node>(ov::CompiledModel::*)(size_t) const) & ov::CompiledModel::input,
py::arg("index"));
py::arg("index"),
R"(
Gets input of a compiled model identified by an index.
If an input with given index is not found, this method throws an exception.
:param index: An input index.
:type index: int
:return: A compiled model input.
:rtype: openvino.runtime.ConstOutput
)");
cls.def("input",
(ov::Output<const ov::Node>(ov::CompiledModel::*)(const std::string&) const) & ov::CompiledModel::input,
py::arg("tensor_name"));
py::arg("tensor_name"),
R"(
Gets input of a compiled model identified by a tensor_name.
If an input with given tensor name is not found, this method throws an exception.
cls.def_property_readonly("outputs", &ov::CompiledModel::outputs);
:param tensor_name: An input tensor's name.
:type tensor_name: str
:return: A compiled model input.
:rtype: openvino.runtime.ConstOutput
)");
cls.def("output", (ov::Output<const ov::Node>(ov::CompiledModel::*)() const) & ov::CompiledModel::output);
cls.def_property_readonly("outputs",
&ov::CompiledModel::outputs,
R"(
Gets all outputs of a compiled model.
:return: Outputs of a compiled model.
:rtype: List[openvino.runtime.ConstOutput]
)");
cls.def("output",
(ov::Output<const ov::Node>(ov::CompiledModel::*)() const) & ov::CompiledModel::output,
R"(
Gets a single output of a compiled model.
If a model has more than one output, this method throws an exception.
:return: A compiled model output.
:rtype: openvino.runtime.ConstOutput
)");
cls.def("output",
(ov::Output<const ov::Node>(ov::CompiledModel::*)(size_t) const) & ov::CompiledModel::output,
py::arg("index"));
py::arg("index"),
R"(
Gets output of a compiled model identified by an index.
If an output with given index is not found, this method throws an exception.
:param index: An output index.
:type index: int
:return: A compiled model output.
:rtype: openvino.runtime.ConstOutput
)");
cls.def("output",
(ov::Output<const ov::Node>(ov::CompiledModel::*)(const std::string&) const) & ov::CompiledModel::output,
py::arg("tensor_name"));
py::arg("tensor_name"),
R"(
Gets output of a compiled model identified by a tensor_name.
If an output with given tensor name is not found, this method throws an exception.
:param tensor_name: An output tensor's name.
:type tensor_name: str
:return: A compiled model output.
:rtype: openvino.runtime.ConstOutput
)");
}

355
src/bindings/python/src/pyopenvino/core/core.cpp
View File

@@ -25,6 +25,10 @@ std::string to_string(py::handle handle) {
void regclass_Core(py::module m) {
py::class_<ov::Core, std::shared_ptr<ov::Core>> cls(m, "Core");
cls.doc() =
"openvino.runtime.Core class represents OpenVINO runtime Core entity. User applications can create several "
"Core class instances, but in this case the underlying plugins are created multiple times and not shared "
"between several Core instances. The recommended way is to have a single Core instance per application.";
cls.def(py::init<const std::string&>(), py::arg("xml_config_file") = "");
@@ -35,8 +39,11 @@ void regclass_Core(py::module m) {
PyErr_WarnEx(PyExc_DeprecationWarning, "set_config() is deprecated, use set_property() instead.", 1);
self.set_property(device_name, {config.begin(), config.end()});
},
py::arg("config"),
py::arg("device_name") = "");
py::arg("device_name") = "",
py::arg("properties"),
R"(
Sets properties for the device.
)");
cls.def(
"set_property",
@@ -50,13 +57,9 @@ void regclass_Core(py::module m) {
py::arg("properties"),
R"(
Sets properties.
Parameters
----------
properties : dict
Dict of pairs: (property name, property value)
Returns
----------
set_property : None
:param properties: Optional dict of pairs: (property name, property value).
:type properties: dict
)");
cls.def(
@@ -72,15 +75,11 @@ void regclass_Core(py::module m) {
py::arg("properties"),
R"(
Sets properties for the device.
Parameters
----------
device_name : str
Name of the device to load the model to.
properties : dict
Dict of pairs: (property name, property value)
Returns
----------
set_property : None
:param device_name: Name of the device.
:type device_name: str
:param properties: Optional dict of pairs: (property name, property value).
:type properties: dict
)");
cls.def(
@@ -88,12 +87,26 @@ void regclass_Core(py::module m) {
[](ov::Core& self,
const std::shared_ptr<const ov::Model>& model,
const std::string& device_name,
const std::map<std::string, std::string>& config) {
return self.compile_model(model, device_name, {config.begin(), config.end()});
const std::map<std::string, std::string>& properties) {
return self.compile_model(model, device_name, {properties.begin(), properties.end()});
},
py::arg("model"),
py::arg("device_name"),
py::arg("config") = py::dict());
py::arg("config") = py::dict(),
R"(
Creates a compiled model from a source model object.
Users can create as many compiled models as they need and use them simultaneously
(up to the limitation of the hardware resources).
:param model: Model acquired from read_model function.
:type model: openvino.runtime.Model
:param device_name: Name of the device to load the model to.
:type device_name: str
:param properties: Optional dict of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
)");
cls.def(
"compile_model",
@@ -103,7 +116,19 @@ void regclass_Core(py::module m) {
return self.compile_model(model, ov::AnyMap{config.begin(), config.end()});
},
py::arg("model"),
py::arg("config") = py::dict());
py::arg("config") = py::dict(),
R"(
Creates and loads a compiled model from a source model to the default OpenVINO device
selected by AUTO plugin. Users can create as many compiled models as they need and use
them simultaneously (up to the limitation of the hardware resources).
:param model: Model acquired from read_model function.
:type model: openvino.runtime.Model
:param properties: Optional dict of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
)");
cls.def(
"compile_model",
@@ -115,17 +140,53 @@ void regclass_Core(py::module m) {
},
py::arg("model_path"),
py::arg("device_name"),
py::arg("config") = py::dict());
py::arg("properties") = py::dict(),
R"(
Reads model and creates a compiled model from IR / ONNX / PDPD file.
This can be more efficient than using read_model + compile_model(model_in_memory_object) flow,
especially for cases when caching is enabled and cached model is available.
:param model_path: A path to a model in IR / ONNX / PDPD format.
:type model_path: str
:param device_name: Name of the device to load the model to.
:type device_name: str
:param properties: Optional dict of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
)");
cls.def(
"compile_model",
[](ov::Core& self, const std::string& model_path, const std::map<std::string, std::string>& config) {
return self.compile_model(model_path, ov::AnyMap{config.begin(), config.end()});
[](ov::Core& self, const std::string& model_path, const std::map<std::string, std::string>& properties) {
return self.compile_model(model_path, ov::AnyMap{properties.begin(), properties.end()});
},
py::arg("model_path"),
py::arg("config") = py::dict());
py::arg("config") = py::dict(),
R"(
Reads model and creates a compiled model from IR / ONNX / PDPD file with device selected by AUTO plugin.
This can be more efficient than using read_model + compile_model(model_in_memory_object) flow,
especially for cases when caching is enabled and cached model is available.
cls.def("get_versions", &ov::Core::get_versions, py::arg("device_name"));
:param model_path: A path to a model in IR / ONNX / PDPD format.
:type model_path: str
:param properties: Optional dict of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
)");
cls.def("get_versions",
&ov::Core::get_versions,
py::arg("device_name"),
R"(
Returns device plugins version information.
:param device_name: Device name to identify a plugin.
:type device_name: str
:return: Plugin version information.
:rtype: Dict[str, openvino.runtime.Version]
)");
cls.def(
"read_model",
@@ -145,19 +206,54 @@ void regclass_Core(py::module m) {
return self.read_model(model, tensor);
},
py::arg("model"),
py::arg("weights") = py::bytes());
py::arg("weights") = py::bytes(),
R"(
Reads models from IR / ONNX / PDPD formats.
:param model: Bytes with model in IR / ONNX / PDPD format.
:type model: bytes
:param weights: Bytes with tensor's data.
:type weights: bytes
:return: A model.
:rtype: openvino.runtime.Model
)");
cls.def(
"read_model",
(std::shared_ptr<ov::Model>(ov::Core::*)(const std::string&, const std::string&) const) & ov::Core::read_model,
py::arg("model"),
py::arg("weights") = "");
py::arg("weights") = "",
R"(
Reads models from IR / ONNX / PDPD formats.
:param model: A path to a model in IR / ONNX / PDPD format.
:type model: str
:param weights: A path to a data file For IR format (*.bin): if path is empty,
will try to read bin file with the same name as xml and if bin
file with the same name was not found, will load IR without weights.
For ONNX format (*.onnx): weights parameter is not used.
For PDPD format (*.pdmodel) weights parameter is not used.
:type weights: str
:return: A model.
:rtype: openvino.runtime.Model
)");
cls.def(
"read_model",
(std::shared_ptr<ov::Model>(ov::Core::*)(const std::string&, const ov::Tensor&) const) & ov::Core::read_model,
py::arg("model"),
py::arg("weights"));
py::arg("weights"),
R"(
Reads models from IR / ONNX / PDPD formats.
:param model: A string with model in IR / ONNX / PDPD format.
:type model: str
:param weights: Tensor with weights. Reading ONNX / PDPD models doesn't support
loading weights from weights tensors.
:type weights: openvino.runtime.Tensor
:return: A model.
:rtype: openvino.runtime.Model
)");
cls.def(
"read_model",
@@ -165,7 +261,21 @@ void regclass_Core(py::module m) {
return self.read_model(py::str(model), py::str(weights));
},
py::arg("model"),
py::arg("weights") = "");
py::arg("weights") = "",
R"(
Reads models from IR / ONNX / PDPD formats.
:param model: A string with model in IR / ONNX / PDPD format.
:type model: str
:param weights: A path to a data file For IR format (*.bin): if path is empty,
will try to read bin file with the same name as xml and if bin
file with the same name was not found, will load IR without weights.
For ONNX format (*.onnx): weights parameter is not used.
For PDPD format (*.pdmodel) weights parameter is not used.
:type weights: str
:return: A model.
:rtype: openvino.runtime.Model
)");
cls.def(
"import_model",
@@ -183,32 +293,27 @@ void regclass_Core(py::module m) {
R"(
Imports a compiled model from a previously exported one.
Parameters
----------
model_stream : bytes
Input stream containing a model previously exported using export_model method.
:param model_stream: Input stream containing a model previously exported using export_model method.
:type model_stream: bytes
:param device_name: Name of device to import compiled model for.
Note, if device_name device was not used to compile the original mode, an exception is thrown.
:type device_name: str
:param properties: Optional map of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict, optional
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
device_name : str
Name of device to import compiled model for.
Note, if device_name device was not used to compile the original mode, an exception is thrown.
:Example:
.. code-block:: python
properties : dict
Optional map of pairs: (property name, property value) relevant only for this load operation.
user_stream = compiled.export_model()
Returns
----------
import_model : openvino.runtime.CompiledModel
with open('./my_model', 'wb') as f:
f.write(user_stream)
Examples
----------
user_stream = compiled.export_model()
# ...
with open('./my_model', 'wb') as f:
f.write(user_stream)
# ...
new_compiled = core.import_model(user_stream, "CPU")
new_compiled = core.import_model(user_stream, "CPU")
)");
// keep as second one to solve overload resolution problem
@@ -239,33 +344,29 @@ void regclass_Core(py::module m) {
Advanced version of `import_model`. It utilizes, streams from standard
Python library `io`.
Parameters
----------
model_stream : bytes
Input stream containing a model previously exported using export_model method.
device_name : str
Name of device to import compiled model for.
Note, if device_name device was not used to compile the original mode, an exception is thrown.
:param model_stream: Input stream containing a model previously exported using export_model method.
:type model_stream: io.BytesIO
:param device_name: Name of device to import compiled model for.
Note, if device_name device was not used to compile the original mode, an exception is thrown.
:type device_name: str
:param properties: Optional map of pairs: (property name, property value) relevant only for this load operation.
:type properties: dict, optional
:return: A compiled model.
:rtype: openvino.runtime.CompiledModel
properties : dict
Optional map of pairs: (property name, property value) relevant only for this load operation.
:Example:
.. code-block:: python
Returns
----------
import_model : openvino.runtime.CompiledModel
user_stream = io.BytesIO()
compiled.export_model(user_stream)
Examples
----------
user_stream = io.BytesIO()
compiled.export_model(user_stream)
with open('./my_model', 'wb') as f:
f.write(user_stream.getvalue()) # or read() if seek(0) was applied before
with open('./my_model', 'wb') as f:
f.write(user_stream.getvalue()) # or read() if seek(0) was applied before
# ...
# ...
new_compiled = core.import_model(user_stream, "CPU")
new_compiled = core.import_model(user_stream, "CPU")
)");
// todo: remove after Accuracy Checker migration to set/get_property API
@@ -287,15 +388,13 @@ void regclass_Core(py::module m) {
py::arg("name"),
R"(
Gets properties dedicated to device behaviour.
Parameters
----------
device_name : str
A name of a device to get a properties value.
name : str
Property name.
Returns
----------
get_property : Any
:param device_name: A name of a device to get a properties value.
:type device_name: str
:param name: Property name.
:type name: str
:return: Extracted information from property.
:rtype: object
)");
// todo: remove after Accuracy Checker migration to set/get_property API
@@ -308,36 +407,108 @@ void regclass_Core(py::module m) {
py::arg("device_name"),
py::arg("name"));
cls.def("register_plugin", &ov::Core::register_plugin, py::arg("plugin_name"), py::arg("device_name"));
cls.def("register_plugin",
&ov::Core::register_plugin,
py::arg("plugin_name"),
py::arg("device_name"),
R"(
Register a new device and plugin which enable this device inside OpenVINO Runtime.
cls.def("register_plugins", &ov::Core::register_plugins, py::arg("xml_config_file"));
:param plugin_name: A name of plugin. Depending on platform `plugin_name` is wrapped with shared library
suffix and prefix to identify library full name E.g. on Linux platform plugin name
specified as `plugin_name` will be wrapped as `libplugin_name.so`.
:type plugin_name: str
:param device_name: A device name to register plugin for.
:type device_name: str
)");
cls.def("unload_plugin", &ov::Core::unload_plugin, py::arg("device_name"));
cls.def("register_plugins",
&ov::Core::register_plugins,
py::arg("xml_config_file"),
R"(
Registers a device plugin to OpenVINO Runtime Core instance using XML configuration
file with plugins description.
:param xml_config_file: A path to .xml file with plugins to register.
:type xml_config_file: str
)");
cls.def("unload_plugin",
&ov::Core::unload_plugin,
py::arg("device_name"),
R"(
Unloads the previously loaded plugin identified by device_name from OpenVINO Runtime.
The method is needed to remove loaded plugin instance and free its resources.
If plugin for a specified device has not been created before, the method throws an exception.
:param device_name: A device name identifying plugin to remove from OpenVINO.
:type device_name: str
)");
cls.def(
"query_model",
[](ov::Core& self,
const std::shared_ptr<const ov::Model>& model,
const std::string& device_name,
const std::map<std::string, std::string>& config) {
return self.query_model(model, device_name, {config.begin(), config.end()});
const std::map<std::string, std::string>& properties) {
return self.query_model(model, device_name, {properties.begin(), properties.end()});
},
py::arg("model"),
py::arg("device_name"),
py::arg("config") = py::dict());
py::arg("properties") = py::dict(),
R"(
Query device if it supports specified model with specified properties.
:param model: Model object to query.
:type model: openvino.runtime.Model
:param device_name: A name of a device to query.
:type device_name: str
:param properties: Optional dict of pairs: (property name, property value)
:type properties: dict
:return: Pairs a operation name -> a device name supporting this operation.
:rtype: dict
)");
cls.def("add_extension",
static_cast<void (ov::Core::*)(const std::string&)>(&ov::Core::add_extension),
py::arg("library_path"));
py::arg("library_path"),
R"(
Registers an extension to a Core object.
:param library_path: Path to library with ov::Extension
:type library_path: str
)");
cls.def("add_extension",
static_cast<void (ov::Core::*)(const std::shared_ptr<ov::Extension>&)>(&ov::Core::add_extension),
py::arg("extension"));
py::arg("extension"),
R"(
Registers an extension to a Core object.
:param extension: Extension object.
:type extension: openvino.runtime.Extension
)");
cls.def(
"add_extension",
static_cast<void (ov::Core::*)(const std::vector<std::shared_ptr<ov::Extension>>&)>(&ov::Core::add_extension),
py::arg("extensions"));
py::arg("extensions"),
R"(
Registers extensions to a Core object.
cls.def_property_readonly("available_devices", &ov::Core::get_available_devices);
:param extensions: List of Extension objects.
:type extensions: list[openvino.runtime.Extension]
)");
cls.def_property_readonly("available_devices",
&ov::Core::get_available_devices,
R"(
Returns devices available for inference Core objects goes over all registered plugins.
:returns: A list of devices. The devices are returned as: CPU, GPU.0, GPU.1, MYRIAD...
If there more than one device of specific type, they are enumerated with .# suffix.
Such enumerated device can later be used as a device name in all Core methods like:
compile_model, query_model, set_property and so on.
:rtype: list
)");
}

2
src/bindings/python/src/pyopenvino/core/extension.cpp
View File

@@ -13,5 +13,7 @@ namespace py = pybind11;
void regclass_Extension(py::module m) {
py::class_<ov::Extension, std::shared_ptr<ov::Extension>> ext(m, "Extension", py::dynamic_attr());
ext.doc() = "openvino.runtime.Extension provides the base interface for OpenVINO extensions.";
ext.def(py::init<>());
}

507
src/bindings/python/src/pyopenvino/core/infer_request.cpp
View File

@@ -20,22 +20,75 @@ namespace py = pybind11;
void regclass_InferRequest(py::module m) {
py::class_<InferRequestWrapper, std::shared_ptr<InferRequestWrapper>> cls(m, "InferRequest");
cls.doc() = "openvino.runtime.InferRequest represents infer request which can be run in asynchronous or "
"synchronous manners.";
cls.def(py::init([](InferRequestWrapper& other) {
return other;
}),
py::arg("other"));
// Python API exclusive function
cls.def(
"set_tensors",
[](InferRequestWrapper& self, const py::dict& inputs) {
auto tensor_map = Common::cast_to_tensor_name_map(inputs);
for (auto&& input : tensor_map) {
self._request.set_tensor(input.first, input.second);
}
Common::set_request_tensors(self._request, inputs);
},
py::arg("inputs"));
py::arg("inputs"),
R"(
Set tensors using given keys.
:param inputs: Data to set on tensors.
:type inputs: Dict[Union[int, str, openvino.runtime.ConstOutput], openvino.runtime.Tensor]
)");
cls.def(
"set_tensors",
[](InferRequestWrapper& self, const std::string& tensor_name, const std::vector<ov::Tensor>& tensors) {
self._request.set_tensors(tensor_name, tensors);
},
py::arg("tensor_name"),
py::arg("tensors"),
R"(
Sets batch of tensors for input data to infer by tensor name.
Model input shall have batch dimension and number of tensors shall
match with batch size. Current version supports set tensors to model inputs only.
In case if `tensor_name` is associated with output (or any other non-input node),
an exception will be thrown.
:param tensor_name: Name of input tensor.
:type tensor_name: str
:param tensors: Input tensors for batched infer request. The type of each tensor
must match the model input element type and shape (except batch dimension).
Total size of tensors shall match with input's size.
:type tensors: List[openvino.runtime.Tensor]
)");
cls.def(
"set_tensors",
[](InferRequestWrapper& self, const ov::Output<const ov::Node>& port, const std::vector<ov::Tensor>& tensors) {
self._request.set_tensors(port, tensors);
},
py::arg("port"),
py::arg("tensors"),
R"(
Sets batch of tensors for input data to infer by tensor name.
Model input shall have batch dimension and number of tensors shall
match with batch size. Current version supports set tensors to model inputs only.
In case if `port` is associated with output (or any other non-input node),
an exception will be thrown.
:param port: Port of input tensor.
:type port: openvino.runtime.ConstOutput
:param tensors: Input tensors for batched infer request. The type of each tensor
must match the model input element type and shape (except batch dimension).
Total size of tensors shall match with input's size.
:type tensors: List[openvino.runtime.Tensor]
:rtype: None
)");
// Python API exclusive function
cls.def(
"set_output_tensors",
[](InferRequestWrapper& self, const py::dict& outputs) {
@@ -44,8 +97,15 @@ void regclass_InferRequest(py::module m) {
self._request.set_output_tensor(output.first, output.second);
}
},
py::arg("outputs"));
py::arg("outputs"),
R"(
Set output tensors using given indexes.
:param inputs: Data to set on output tensors.
:type inputs: Dict[int, openvino.runtime.Tensor]
)");
// Python API exclusive function
cls.def(
"set_input_tensors",
[](InferRequestWrapper& self, const py::dict& inputs) {
@@ -54,7 +114,49 @@ void regclass_InferRequest(py::module m) {
self._request.set_input_tensor(input.first, input.second);
}
},
py::arg("inputs"));
py::arg("inputs"),
R"(
Set input tensors using given indexes.
:param inputs: Data to set on output tensors.
:type inputs: Dict[int, openvino.runtime.Tensor]
)");
cls.def(
"set_input_tensors",
[](InferRequestWrapper& self, const std::vector<ov::Tensor>& tensors) {
self._request.set_input_tensors(tensors);
},
py::arg("tensors"),
R"(
Sets batch of tensors for single input data.
Model input shall have batch dimension and number of `tensors`
shall match with batch size.
:param tensors: Input tensors for batched infer request. The type of each tensor
must match the model input element type and shape (except batch dimension).
Total size of tensors shall match with input's size.
:type tensors: List[openvino.runtime.Tensor]
)");
cls.def(
"set_input_tensors",
[](InferRequestWrapper& self, size_t idx, const std::vector<ov::Tensor>& tensors) {
self._request.set_input_tensors(idx, tensors);
},
py::arg("idx"),
py::arg("tensors"),
R"(
Sets batch of tensors for single input data to infer by index.
Model input shall have batch dimension and number of `tensors`
shall match with batch size.
:param idx: Index of input tensor.
:type idx: int
:param tensors: Input tensors for batched infer request. The type of each tensor
must match the model input element type and shape (except batch dimension).
Total size of tensors shall match with input's size.
)");
cls.def(
"infer",
@@ -67,7 +169,17 @@ void regclass_InferRequest(py::module m) {
self._end_time = Time::now();
return Common::outputs_to_dict(self._outputs, self._request);
},
py::arg("inputs"));
py::arg("inputs"),
R"(
Infers specified input(s) in synchronous mode.
Blocks all methods of InferRequest while request is running.
Calling any method will lead to throwing exceptions.
:param inputs: Data to set on input tensors.
:type inputs: Dict[Union[int, str, openvino.runtime.ConstOutput], openvino.runtime.Tensor]
:return: Dictionary of results from output tensors with ports as keys.
:rtype: Dict[openvino.runtime.ConstOutput, numpy.array]
)");
cls.def(
"start_async",
@@ -86,16 +198,44 @@ void regclass_InferRequest(py::module m) {
self._request.start_async();
},
py::arg("inputs"),
py::arg("userdata"));
py::arg("userdata"),
R"(
Starts inference of specified input(s) in asynchronous mode.
Returns immediately. Inference starts also immediately.
cls.def("cancel", [](InferRequestWrapper& self) {
self._request.cancel();
});
This function releases the GIL, so another Python thread can
work while this function runs in the background.
cls.def("wait", [](InferRequestWrapper& self) {
py::gil_scoped_release release;
self._request.wait();
});
Calling any method on this InferRequest while the request is
running will lead to throwing exceptions.
:param inputs: Data to set on input tensors.
:type inputs: Dict[Union[int, str, openvino.runtime.ConstOutput], openvino.runtime.Tensor]
:param userdata: Any data that will be passed inside callback call.
:type userdata: Any
)");
cls.def(
"cancel",
[](InferRequestWrapper& self) {
self._request.cancel();
},
R"(
Cancels inference request.
)");
cls.def(
"wait",
[](InferRequestWrapper& self) {
py::gil_scoped_release release;
self._request.wait();
},
R"(
Waits for the result to become available.
Blocks until the result becomes available.
Function releases GIL, other threads can work while this function waits.
)");
cls.def(
"wait_for",
@@ -103,14 +243,26 @@ void regclass_InferRequest(py::module m) {
py::gil_scoped_release release;
return self._request.wait_for(std::chrono::milliseconds(timeout));
},
py::arg("timeout"));
py::arg("timeout"),
R"(
Waits for the result to become available.
Blocks until specified timeout has elapsed or
the result becomes available, whichever comes first.
Function releases GIL, other threads can work while this function waits.
:param timeout: Maximum duration in milliseconds (ms) of blocking call.
:type timeout: int
:return: True if InferRequest is ready, False otherwise.
:rtype: bool
)");
cls.def(
"set_callback",
[](InferRequestWrapper& self, py::function f_callback, py::object& userdata) {
[](InferRequestWrapper& self, py::function callback, py::object& userdata) {
self.userdata = userdata;
self.user_callback_defined = true;
self._request.set_callback([&self, f_callback](std::exception_ptr exception_ptr) {
self._request.set_callback([&self, callback](std::exception_ptr exception_ptr) {
self._end_time = Time::now();
try {
if (exception_ptr) {
@@ -121,54 +273,123 @@ void regclass_InferRequest(py::module m) {
}
// Acquire GIL, execute Python function
py::gil_scoped_acquire acquire;
f_callback(self.userdata);
callback(self.userdata);
});
},
py::arg("f_callback"),
py::arg("userdata"));
py::arg("callback"),
py::arg("userdata"),
R"(
Sets a callback function that will be called on success or failure of asynchronous InferRequest.
:param callback: Function defined in Python.
:type callback: function
:param userdata: Any data that will be passed inside callback call.
:type userdata: Any
)");
cls.def(
"get_tensor",
[](InferRequestWrapper& self, const std::string& name) {
return self._request.get_tensor(name);
},
py::arg("name"));
py::arg("name"),
R"(
Gets input/output tensor of InferRequest.
:param name: Name of tensor to get.
:type name: str
:return: A Tensor object with given name.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_tensor",
[](InferRequestWrapper& self, const ov::Output<const ov::Node>& port) {
return self._request.get_tensor(port);
},
py::arg("port"));
py::arg("port"),
R"(
Gets input/output tensor of InferRequest.
:param port: Port of tensor to get.
:type port: openvino.runtime.ConstOutput
:return: A Tensor object for the port.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_tensor",
[](InferRequestWrapper& self, const ov::Output<ov::Node>& port) {
return self._request.get_tensor(port);
},
py::arg("port"));
py::arg("port"),
R"(
Gets input/output tensor of InferRequest.
:param port: Port of tensor to get.
:type port: openvino.runtime.Output
:return: A Tensor object for the port.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_input_tensor",
[](InferRequestWrapper& self, size_t idx) {
return self._request.get_input_tensor(idx);
},
py::arg("index"));
py::arg("index"),
R"(
Gets input tensor of InferRequest.
cls.def("get_input_tensor", [](InferRequestWrapper& self) {
return self._request.get_input_tensor();
});
:param idx: An index of tensor to get.
:type idx: int
:return: An input Tensor with index idx for the model.
If a tensor with specified idx is not found,
an exception is thrown.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_input_tensor",
[](InferRequestWrapper& self) {
return self._request.get_input_tensor();
},
R"(
Gets input tensor of InferRequest.
:return: An input Tensor for the model.
If model has several inputs, an exception is thrown.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_output_tensor",
[](InferRequestWrapper& self, size_t idx) {
return self._request.get_output_tensor(idx);
},
py::arg("index"));
py::arg("index"),
R"(
Gets output tensor of InferRequest.
cls.def("get_output_tensor", [](InferRequestWrapper& self) {
return self._request.get_output_tensor();
});
:param idx: An index of tensor to get.
:type idx: int
:return: An output Tensor with index idx for the model.
If a tensor with specified idx is not found, an exception is thrown.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"get_output_tensor",
[](InferRequestWrapper& self) {
return self._request.get_output_tensor();
},
R"(
Gets output tensor of InferRequest.
:return: An output Tensor for the model.
If model has several outputs, an exception is thrown.
:rtype: openvino.runtime.Tensor
)");
cls.def(
"set_tensor",
@@ -176,7 +397,16 @@ void regclass_InferRequest(py::module m) {
self._request.set_tensor(name, tensor);
},
py::arg("name"),
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets input/output tensor of InferRequest.
:param name: Name of input/output tensor.
:type name: str
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's input/output element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_tensor",
@@ -184,7 +414,16 @@ void regclass_InferRequest(py::module m) {
self._request.set_tensor(port, tensor);
},
py::arg("port"),
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets input/output tensor of InferRequest.
:param port: Port of input/output tensor.
:type port: openvino.runtime.ConstOutput
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's input/output element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_tensor",
@@ -192,7 +431,16 @@ void regclass_InferRequest(py::module m) {
self._request.set_tensor(port, tensor);
},
py::arg("port"),
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets input/output tensor of InferRequest.
:param port: Port of input/output tensor.
:type port: openvino.runtime.Output
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's input/output element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_input_tensor",
@@ -200,14 +448,32 @@ void regclass_InferRequest(py::module m) {
self._request.set_input_tensor(idx, tensor);
},
py::arg("index"),
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets input tensor of InferRequest.
:param idx: Index of input tensor. If idx is greater than number of model's inputs,
an exception is thrown.
:type idx: int
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's input element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_input_tensor",
[](InferRequestWrapper& self, const ov::Tensor& tensor) {
self._request.set_input_tensor(tensor);
},
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets input tensor of InferRequest with single input.
If model has several inputs, an exception is thrown.
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's input element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_output_tensor",
@@ -215,52 +481,157 @@ void regclass_InferRequest(py::module m) {
self._request.set_output_tensor(idx, tensor);
},
py::arg("index"),
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets output tensor of InferRequest.
:param idx: Index of output tensor.
:type idx: int
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's output element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def(
"set_output_tensor",
[](InferRequestWrapper& self, const ov::Tensor& tensor) {
self._request.set_output_tensor(tensor);
},
py::arg("tensor"));
py::arg("tensor"),
R"(
Sets output tensor of InferRequest with single output.
If model has several outputs, an exception is thrown.
cls.def("get_profiling_info", [](InferRequestWrapper& self) {
return self._request.get_profiling_info();
});
:param tensor: Tensor object. The element_type and shape of a tensor
must match the model's output element_type and shape.
:type tensor: openvino.runtime.Tensor
)");
cls.def("query_state", [](InferRequestWrapper& self) {
return self._request.query_state();
});
cls.def(
"get_profiling_info",
[](InferRequestWrapper& self) {
return self._request.get_profiling_info();
},
R"(
Queries performance measures per layer to get feedback of what
is the most time consuming operation, not all plugins provide
meaningful data.
cls.def_property_readonly("userdata", [](InferRequestWrapper& self) {
return self.userdata;
});
:return: List of profiling information for operations in model.
:rtype: List[openvino.runtime.ProfilingInfo]
)");
cls.def_property_readonly("model_inputs", [](InferRequestWrapper& self) {
return self._inputs;
});
cls.def(
"query_state",
[](InferRequestWrapper& self) {
return self._request.query_state();
},
R"(
Gets state control interface for given infer request.
cls.def_property_readonly("model_outputs", [](InferRequestWrapper& self) {
return self._outputs;
});
:return: List of VariableState objects.
:rtype: List[openvino.runtime.VariableState]
)");
cls.def_property_readonly("inputs", &InferRequestWrapper::get_input_tensors);
cls.def_property_readonly(
"userdata",
[](InferRequestWrapper& self) {
return self.userdata;
},
R"(
Gets currently held userdata.
cls.def_property_readonly("outputs", &InferRequestWrapper::get_output_tensors);
:rtype: Any
)");
cls.def_property_readonly("input_tensors", &InferRequestWrapper::get_input_tensors);
cls.def_property_readonly(
"model_inputs",
[](InferRequestWrapper& self) {
return self._inputs;
},
R"(
Gets all inputs of a compiled model which was used to create this InferRequest.
cls.def_property_readonly("output_tensors", &InferRequestWrapper::get_output_tensors);
:rtype: List[openvino.runtime.ConstOutput]
)");
cls.def_property_readonly("latency", [](InferRequestWrapper& self) {
return self.get_latency();
});
cls.def_property_readonly(
"model_outputs",
[](InferRequestWrapper& self) {
return self._outputs;
},
R"(
Gets all outputs of a compiled model which was used to create this InferRequest.
cls.def_property_readonly("profiling_info", [](InferRequestWrapper& self) {
return self._request.get_profiling_info();
});
:rtype: List[openvino.runtime.ConstOutput]
)");
cls.def_property_readonly("results", [](InferRequestWrapper& self) {
return Common::outputs_to_dict(self._outputs, self._request);
});
cls.def_property_readonly("inputs",
&InferRequestWrapper::get_input_tensors,
R"(
Gets all input tensors of this InferRequest.
:rtype: List[openvino.runtime.Tensor]
)");
cls.def_property_readonly("outputs",
&InferRequestWrapper::get_output_tensors,
R"(
Gets all output tensors of this InferRequest.
:rtype: List[openvino.runtime.Tensor]
)");
cls.def_property_readonly("input_tensors",
&InferRequestWrapper::get_input_tensors,
R"(
Gets all input tensors of this InferRequest.
:rtype: List[openvino.runtime.Tensor]
)");
cls.def_property_readonly("output_tensors",
&InferRequestWrapper::get_output_tensors,
R"(
Gets all output tensors of this InferRequest.
:rtype: List[openvino.runtime.Tensor]
)");
cls.def_property_readonly(
"latency",
[](InferRequestWrapper& self) {
return self.get_latency();
},
R"(
Gets latency of this InferRequest.
:rtype: float
)");
cls.def_property_readonly(
"profiling_info",
[](InferRequestWrapper& self) {
return self._request.get_profiling_info();
},
R"(
Performance measures per layer to get feedback of what is the most time consuming operation.
Not all plugins provide meaningful data!
:return: Inference time.
:rtype: List[openvino.runtime.ProfilingInfo]
)");
cls.def_property_readonly(
"results",
[](InferRequestWrapper& self) {
return Common::outputs_to_dict(self._outputs, self._request);
},
R"(
Gets all outputs tensors of this InferRequest.
:return: Dictionary of results from output tensors with ports as keys.
:rtype: Dict[openvino.runtime.ConstOutput, numpy.array]
)");
}

138
src/bindings/python/src/pyopenvino/core/offline_transformations.cpp
View File

@@ -38,150 +38,154 @@ namespace py = pybind11;
void regmodule_offline_transformations(py::module m) {
py::module m_offline_transformations = m.def_submodule("offline_transformations", "Offline transformations module");
m_offline_transformations.doc() = "openvino.offline_transformations module contains different offline passes.";
m_offline_transformations.def(
"apply_moc_transformations",
[](std::shared_ptr<ov::Model> function, bool cf) {
[](std::shared_ptr<ov::Model> model, bool cf) {
ov::pass::Manager manager;
manager.register_pass<ngraph::pass::MOCTransformations>(cf);
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("cf"));
m_offline_transformations.def(
"apply_moc_legacy_transformations",
[](std::shared_ptr<ov::Model> function, const std::vector<std::string>& params_with_custom_types) {
[](std::shared_ptr<ov::Model> model, const std::vector<std::string>& params_with_custom_types) {
ov::pass::Manager manager;
manager.register_pass<ov::pass::MOCLegacyTransformations>(params_with_custom_types);
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("params_with_custom_types"));
m_offline_transformations.def(
"apply_pot_transformations",
[](std::shared_ptr<ov::Model> function, std::string device) {
[](std::shared_ptr<ov::Model> model, std::string device) {
ov::pass::Manager manager;
manager.register_pass<ngraph::pass::POTTransformations>(std::move(device));
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("device"));
m_offline_transformations.def(
"apply_low_latency_transformation",
[](std::shared_ptr<ov::Model> function, bool use_const_initializer = true) {
[](std::shared_ptr<ov::Model> model, bool use_const_initializer = true) {
ov::pass::Manager manager;
manager.register_pass<ov::pass::LowLatency2>(use_const_initializer);
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("use_const_initializer") = true);
m_offline_transformations.def(
"apply_pruning_transformation",
[](std::shared_ptr<ov::Model> function) {
[](std::shared_ptr<ov::Model> model) {
ov::pass::Manager manager;
manager.register_pass<ngraph::pass::Pruning>();
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"));
py::arg("model"));
m_offline_transformations.def(
"generate_mapping_file",
[](std::shared_ptr<ov::Model> function, std::string path, bool extract_names) {
[](std::shared_ptr<ov::Model> model, std::string path, bool extract_names) {
ov::pass::Manager manager;
manager.register_pass<ngraph::pass::GenerateMappingFile>(path, extract_names);
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("path"),
py::arg("extract_names"));
m_offline_transformations.def(
"apply_make_stateful_transformation",
[](std::shared_ptr<ov::Model> function, const std::map<std::string, std::string>& param_res_names) {
[](std::shared_ptr<ov::Model> model, const std::map<std::string, std::string>& param_res_names) {
ngraph::pass::Manager manager;
manager.register_pass<ov::pass::MakeStateful>(param_res_names);
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("param_res_names"));
m_offline_transformations.def(
"compress_model_transformation",
[](std::shared_ptr<ov::Model> function) {
[](std::shared_ptr<ov::Model> model) {
ov::pass::Manager manager;
manager.register_pass<ov::pass::MarkPrecisionSensitiveSubgraphs>();
manager.register_pass<ov::pass::CompressFloatConstants>();
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"));
py::arg("model"));
m_offline_transformations.def(
"compress_quantize_weights_transformation",
[](std::shared_ptr<ov::Model> function) {
[](std::shared_ptr<ov::Model> model) {
ov::pass::Manager manager;
manager.register_pass<ngraph::pass::CompressQuantizeWeights>();
manager.register_pass<ngraph::pass::ZeroPointOptimizer>();
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"));
py::arg("model"));
// todo: remove as serialize as part of passManager api will be merged
m_offline_transformations.def(
"serialize",
[](std::shared_ptr<ov::Model> function,
[](std::shared_ptr<ov::Model> model,
const std::string& path_to_xml,
const std::string& path_to_bin,
const std::string& version) {
ov::pass::Manager manager;
manager.register_pass<ov::pass::Serialize>(path_to_xml, path_to_bin, convert_to_version(version));
manager.run_passes(function);
manager.run_passes(model);
},
py::arg("function"),
py::arg("model"),
py::arg("model_path"),
py::arg("weights_path"),
py::arg("version") = "UNSPECIFIED",
R"(
Serialize given function into IR. The generated .xml and .bin files will be save
into provided paths.
Parameters
----------
function : ov.Model
function which will be converted to IR representation
xml_path : str
path where .xml file will be saved
bin_path : str
path where .bin file will be saved
version : str
sets the version of the IR which will be generated.
Supported versions are:
- "UNSPECIFIED" (default) : Use the latest or function version
- "IR_V10" : v10 IR
- "IR_V11" : v11 IR
Serialize given model into IR. The generated .xml and .bin files will be save
into provided paths.
Examples:
----------
1. Default IR version:
shape = [2, 2]
parameter_a = ov.parameter(shape, dtype=np.float32, name="A")
parameter_b = ov.parameter(shape, dtype=np.float32, name="B")
parameter_c = ov.parameter(shape, dtype=np.float32, name="C")
model = (parameter_a + parameter_b) * parameter_c
func = Model(model, [parameter_a, parameter_b, parameter_c], "Model")
# IR generated with default version
serialize(func, model_path="./serialized.xml", weights_path="./serialized.bin")
:param model: model which will be converted to IR representation
:type model: openvino.runtime.Model
:param xml_path: path where .xml file will be saved
:type xml_path: str
:param bin_path: path where .bin file will be saved
:type bin_path: str
:param version: sets the version of the IR which will be generated.
Supported versions are:
- "UNSPECIFIED" (default) : Use the latest or model version
- "IR_V10" : v10 IR
- "IR_V11" : v11 IR
:Examples:
1. Default IR version:
.. code-block:: python
shape = [2, 2]
parameter_a = ov.parameter(shape, dtype=np.float32, name="A")
parameter_b = ov.parameter(shape, dtype=np.float32, name="B")
parameter_c = ov.parameter(shape, dtype=np.float32, name="C")
model = (parameter_a + parameter_b) * parameter_c
func = Model(model, [parameter_a, parameter_b, parameter_c], "Model")
# IR generated with default version
serialize(func, model_path="./serialized.xml", weights_path="./serialized.bin")
2. IR version 11:
shape = [2, 2]
parameter_a = ov.parameter(shape, dtype=np.float32, name="A")
parameter_b = ov.parameter(shape, dtype=np.float32, name="B")
parameter_c = ov.parameter(shape, dtype=np.float32, name="C")
model = (parameter_a + parameter_b) * parameter_c
func = Model(model, [parameter_a, parameter_b, parameter_c], "Model")
# IR generated with default version
serialize(func, model_path="./serialized.xml", "./serialized.bin", version="IR_V11")
// )");
2. IR version 11:
.. code-block:: python
parameter_a = ov.parameter(shape, dtype=np.float32, name="A")
parameter_b = ov.parameter(shape, dtype=np.float32, name="B")
parameter_c = ov.parameter(shape, dtype=np.float32, name="C")
model = (parameter_a + parameter_b) * parameter_c
func = Model(model, [parameter_a, parameter_b, parameter_c], "Model")
# IR generated with default version
serialize(func, model_path="./serialized.xml", "./serialized.bin", version="IR_V11")
)");
}

2
src/bindings/python/src/pyopenvino/core/profiling_info.cpp
View File

@@ -12,6 +12,8 @@ namespace py = pybind11;
void regclass_ProfilingInfo(py::module m) {
py::class_<ov::ProfilingInfo, std::shared_ptr<ov::ProfilingInfo>> cls(m, "ProfilingInfo");
cls.doc() = "openvino.runtime.ProfilingInfo contains performance metrics for single node.";
cls.def(py::init<>())
.def_readwrite("status", &ov::ProfilingInfo::status)
.def_readwrite("real_time", &ov::ProfilingInfo::real_time)

183
src/bindings/python/src/pyopenvino/core/tensor.cpp
View File

@@ -14,12 +14,66 @@ namespace py = pybind11;
void regclass_Tensor(py::module m) {
py::class_<ov::Tensor, std::shared_ptr<ov::Tensor>> cls(m, "Tensor");
cls.doc() = "openvino.runtime.Tensor holding either copy of memory or shared host memory.";
cls.def(py::init([](py::array& array, bool shared_memory) {
return Common::tensor_from_numpy(array, shared_memory);
}),
py::arg("array"),
py::arg("shared_memory") = false);
py::arg("shared_memory") = false,
R"(
Tensor's special constructor.
:param array: Array to create tensor from.
:type array: numpy.array
:param shared_memory: If `True` this Tensor memory is being shared with a host,
that means the responsibility of keeping host memory is
on the side of a user. Any action performed on the host
memory will be reflected on this Tensor's memory!
If `False`, data is being copied to this Tensor.
Requires data to be C_CONTIGUOUS if `True`.
:type shared_memory: bool
)");
cls.def(py::init([](py::array& array, const ov::Shape& shape) {
return Common::tensor_from_pointer(array, shape);
}),
py::arg("array"),
py::arg("shape"),
R"(
Another Tensor's special constructor.
It take an array or slice of it and shape that will be
selected starting from the first element of given array/slice.
Please use it only in advanced cases if necessary!
:param array: Underlaying methods will retrieve pointer on first element
from it, which is simulating `host_ptr` from C++ API.
Tensor memory is being shared with a host,
that means the responsibility of keeping host memory is
on the side of a user. Any action performed on the host
memory will be reflected on this Tensor's memory!
Data is required to be C_CONTIGUOUS.
:type array: numpy.array
:param shape: Shape of the new tensor.
:type shape: openvino.runtime.Shape
:Example:
.. code-block:: python
import openvino.runtime as ov
import numpy as np
arr = np.array([[1, 2, 3], [4, 5, 6]])
t = ov.Tensor(arr[1][0:1], ov.Shape([3]))
t.data[0] = 9
print(arr)
>>> [[1 2 3]
>>> [9 5 6]]
)");
cls.def(py::init<const ov::element::Type, const ov::Shape>(), py::arg("type"), py::arg("shape"));
@@ -56,23 +110,122 @@ void regclass_Tensor(py::module m) {
py::arg("begin"),
py::arg("end"));
cls.def_property_readonly("element_type", &ov::Tensor::get_element_type);
cls.def("get_element_type",
&ov::Tensor::get_element_type,
R"(
Gets Tensor's element type.
cls.def_property_readonly("size", &ov::Tensor::get_size);
:rtype: openvino.runtime.Type
)");
cls.def_property_readonly("byte_size", &ov::Tensor::get_byte_size);
cls.def_property_readonly("element_type",
&ov::Tensor::get_element_type,
R"(
Tensor's element type.
cls.def_property_readonly("data", [](ov::Tensor& self) {
return py::array(Common::ov_type_to_dtype().at(self.get_element_type()),
self.get_shape(),
self.get_strides(),
self.data(),
py::cast(self));
});
:rtype: openvino.runtime.Type
)");
cls.def_property("shape", &ov::Tensor::get_shape, &ov::Tensor::set_shape);
cls.def("get_size",
&ov::Tensor::get_size,
R"(
Gets Tensor's size as total number of elements.
cls.def_property("shape", &ov::Tensor::get_shape, [](ov::Tensor& self, std::vector<size_t>& shape) {
self.set_shape(shape);
});
:rtype: int
)");
cls.def_property_readonly("size",
&ov::Tensor::get_size,
R"(
Tensor's size as total number of elements.
:rtype: int
)");
cls.def("get_byte_size",
&ov::Tensor::get_byte_size,
R"(
Gets Tensor's size in bytes.
:rtype: int
)");
cls.def_property_readonly("byte_size",
&ov::Tensor::get_byte_size,
R"(
Tensor's size in bytes.
:rtype: int
)");
cls.def("get_strides",
&ov::Tensor::get_strides,
R"(
Gets Tensor's strides in bytes.
:rtype: openvino.runtime.Strides
)");
cls.def_property_readonly("strides",
&ov::Tensor::get_strides,
R"(
Tensor's strides in bytes.
:rtype: openvino.runtime.Strides
)");
cls.def_property_readonly(
"data",
[](ov::Tensor& self) {
return py::array(Common::ov_type_to_dtype().at(self.get_element_type()),
self.get_shape(),
self.get_strides(),
self.data(),
py::cast(self));
},
R"(
Access to Tensor's data.
:rtype: numpy.array
)");
cls.def("get_shape",
&ov::Tensor::get_shape,
R"(
Gets Tensor's shape.
:rtype: openvino.runtime.Shape
)");
cls.def("set_shape",
&ov::Tensor::set_shape,
R"(
Sets Tensor's shape.
)");
cls.def(
"set_shape",
[](ov::Tensor& self, std::vector<size_t>& shape) {
self.set_shape(shape);
},
R"(
Sets Tensor's shape.
)");
cls.def_property("shape",
&ov::Tensor::get_shape,
&ov::Tensor::set_shape,
R"(
Tensor's shape get/set.
)");
cls.def_property(
"shape",
&ov::Tensor::get_shape,
[](ov::Tensor& self, std::vector<size_t>& shape) {
self.set_shape(shape);
},
R"(
Tensor's shape get/set.
)");
}

44
src/bindings/python/src/pyopenvino/core/variable_state.cpp
View File

@@ -12,10 +12,48 @@ namespace py = pybind11;
void regclass_VariableState(py::module m) {
py::class_<ov::VariableState, std::shared_ptr<ov::VariableState>> variable_st(m, "VariableState");
variable_st.doc() = "openvino.runtime.VariableState class.";
variable_st.def("reset", &ov::VariableState::reset);
variable_st.def("reset",
&ov::VariableState::reset,
R"(
Reset internal variable state for relevant infer request,
to a value specified as default for according node.
variable_st.def_property_readonly("name", &ov::VariableState::get_name);
Parameters
----------
None
variable_st.def_property("state", &ov::VariableState::get_state, &ov::VariableState::set_state);
Returns
----------
reset : None
)");
variable_st.def_property_readonly("name",
&ov::VariableState::get_name,
R"(
Gets name of current variable state.
Returns
----------
reset : str
A string representing a state name.
)");
variable_st.def_property("state",
&ov::VariableState::get_state,
&ov::VariableState::set_state,
R"(
Gets/sets variable state.
Parameters
----------
state : openvino.runtime.Tensor
The current state to set.
Returns
----------
state : openvino.runtime.Tensor
A tensor representing a state.
)");
}

41
src/bindings/python/src/pyopenvino/core/version.cpp
View File

@@ -10,15 +10,40 @@ namespace py = pybind11;
void regclass_Version(py::module m) {
py::class_<ov::Version> cls(m, "Version");
cls.doc() =
"openvino.runtime.Version represents version information that describes plugins and the OpenVINO library.";
cls.def_readonly("build_number", &ov::Version::buildNumber);
cls.def_readonly("description", &ov::Version::description);
cls.def_readonly("build_number",
&ov::Version::buildNumber,
R"(
:return: String with build number.
:rtype: str
)");
cls.def_property_readonly("major", [](ov::Version& self) {
return OPENVINO_VERSION_MAJOR;
});
cls.def_readonly("description",
&ov::Version::description,
R"(
:return: Description string.
:rtype: str
)");
cls.def_property_readonly("minor", [](ov::Version& self) {
return OPENVINO_VERSION_MINOR;
});
cls.def_property_readonly(
"major",
[](ov::Version& self) {
return OPENVINO_VERSION_MAJOR;
},
R"(
:return: OpenVINO's major version.
:rtype: int
)");
cls.def_property_readonly(
"minor",
[](ov::Version& self) {
return OPENVINO_VERSION_MINOR;
},
R"(
:return: OpenVINO's minor version.
:rtype: int
)");
}

12
src/bindings/python/src/pyopenvino/frontend/frontend.cpp
View File

@@ -142,10 +142,8 @@ void regclass_frontend_FrontEnd(py::module m) {
Add extension defined by an object inheriting from Extension
used in order to extend capabilities of Frontend.
Parameters
----------
extension : Extension
Provided extension object.
:param extension: Provided extension object.
:type extension: Extension
)");
fem.def("add_extension",
@@ -154,10 +152,8 @@ void regclass_frontend_FrontEnd(py::module m) {
Add extension defined in external library indicated by a extension_path
used in order to extend capabilities of Frontend.
Parameters
----------
extension_path : str
A path to extension.
:param extension_path: A path to extension.
:type extension_path: str
)");
fem.def("__repr__", [](const FrontEnd& self) -> std::string {

30
src/bindings/python/src/pyopenvino/graph/any.cpp
View File

@@ -13,7 +13,10 @@ namespace py = pybind11;
void regclass_graph_Any(py::module m) {
py::class_<PyAny, std::shared_ptr<PyAny>> ov_any(m, "OVAny", py::module_local());
ov_any.doc() = "openvino.runtime.OVAny wraps ov::Any";
ov_any.doc() = "openvino.runtime.OVAny provides object wrapper for OpenVINO"
"ov::Any class. It allows to pass different types of objects"
"into C++ based core of the project.";
ov_any.def(py::init<py::object>());
ov_any.def("__repr__", [](const PyAny& self) {
@@ -36,10 +39,8 @@ void regclass_graph_Any(py::module m) {
return self.as<py::object>();
},
R"(
Returns
----------
get : Any
Value of ov::Any.
:return: Value of this OVAny.
:rtype: Any
)");
ov_any.def(
"set",
@@ -47,13 +48,16 @@ void regclass_graph_Any(py::module m) {
self = PyAny(value);
},
R"(
Parameters
----------
set : Any
Value to be set in ov::Any.
:param: Value to be set in OVAny.
:type: Any
)");
ov_any.def_property_readonly(
"value",
[](const PyAny& self) {
return self.as<py::object>();
},
R"(
:return: Value of this OVAny.
:rtype: Any
)");
ov_any.def_property_readonly("value", [](const PyAny& self) {
return self.as<py::object>();
});
}

12
src/bindings/python/src/pyopenvino/graph/descriptors/tensor.cpp
View File

@@ -101,6 +101,18 @@ void regclass_graph_descriptor_Tensor(py::module m) {
Set of names
)");
tensor.def("set_names",
&ov::descriptor::Tensor::add_names,
py::arg("names"),
R"(
Adds names for tensor.
Parameters
----------
names : set
Set of names
)");
tensor.def("get_any_name",
&ov::descriptor::Tensor::get_any_name,
R"(

104
src/bindings/python/src/pyopenvino/graph/dimension.cpp
View File

@@ -26,10 +26,8 @@ void regclass_graph_Dimension(py::module m) {
R"(
Construct a static dimension.
Parameters
----------
dimension : int
Value of the dimension.
:param dimension: Value of the dimension.
:type dimension: int
)");
dim.def(py::init<value_type&, value_type&>(),
py::arg("min_dimension"),
@@ -37,13 +35,10 @@ void regclass_graph_Dimension(py::module m) {
R"(
Construct a dynamic dimension with bounded range.
Parameters
----------
min_dimension : int
The lower inclusive limit for the dimension.
max_dimension : int
The upper inclusive limit for the dimension.
:param min_dimension: The lower inclusive limit for the dimension.
:type min_dimension: int
:param max_dimension: The upper inclusive limit for the dimension.
:type max_dimension: int
)");
dim.def_static("dynamic", &ov::Dimension::dynamic);
@@ -52,21 +47,16 @@ void regclass_graph_Dimension(py::module m) {
&ov::Dimension::is_dynamic,
R"(
Check if Dimension is dynamic.
Returns
----------
is_dynamic : bool
True if dynamic, else False.
:return: True if dynamic, else False.
:rtype: bool
)");
dim.def_property_readonly("is_static",
&ov::Dimension::is_static,
R"(
Check if Dimension is static.
Returns
----------
is_static : bool
True if static, else False.
:return: True if static, else False.
:rtype: bool
)");
dim.def(
@@ -88,11 +78,9 @@ void regclass_graph_Dimension(py::module m) {
R"(
Return this dimension as integer.
This dimension must be static and non-negative.
Returns
----------
get_length : int
Value of the dimension.
:return: Value of the dimension.
:rtype: int
)");
dim.def("get_min_length",
&ov::Dimension::get_min_length,
@@ -100,10 +88,8 @@ void regclass_graph_Dimension(py::module m) {
Return this dimension's min_dimension as integer.
This dimension must be dynamic and non-negative.
Returns
----------
get_min_length : int
Value of the dimension.
:return: Value of the dimension.
:rtype: int
)");
dim.def("get_max_length",
&ov::Dimension::get_max_length,
@@ -111,10 +97,8 @@ void regclass_graph_Dimension(py::module m) {
Return this dimension's max_dimension as integer.
This dimension must be dynamic and non-negative.
Returns
----------
get_max_length : int
Value of the dimension.
:return: Value of the dimension.
:rtype: int
)");
dim.def("same_scheme",
@@ -124,16 +108,11 @@ void regclass_graph_Dimension(py::module m) {
Return this dimension's max_dimension as integer.
This dimension must be dynamic and non-negative.
Parameters
----------
dim : Dimension
The other dimension to compare this dimension to.
Returns
----------
same_scheme : bool
True if this dimension and dim are both dynamic,
or if they are both static and equal, otherwise False.
:param dim: The other dimension to compare this dimension to.
:type dim: Dimension
:return: True if this dimension and dim are both dynamic,
or if they are both static and equal, otherwise False.
:rtype: bool
)");
dim.def("compatible",
&ov::Dimension::compatible,
@@ -142,15 +121,10 @@ void regclass_graph_Dimension(py::module m) {
Check whether this dimension is capable of being merged
with the argument dimension.
Parameters
----------
dim : Dimension
The dimension to compare this dimension with.
Returns
----------
compatible : bool
True if this dimension is compatible with d, else False.
:param dim: The dimension to compare this dimension with.
:type dim: Dimension
:return: True if this dimension is compatible with d, else False.
:rtype: bool
)");
dim.def("relaxes",
&ov::Dimension::relaxes,
@@ -164,15 +138,10 @@ void regclass_graph_Dimension(py::module m) {
this.relaxes(d) is equivalent to d.refines(this).
Parameters
----------
dim : Dimension
The dimension to compare this dimension with.
Returns
----------
relaxes : bool
True if this dimension relaxes d, else False.
:param dim: The dimension to compare this dimension with.
:type dim: Dimension
:return: True if this dimension relaxes d, else False.
:rtype: bool
)");
dim.def("refines",
&ov::Dimension::refines,
@@ -186,15 +155,10 @@ void regclass_graph_Dimension(py::module m) {
this.refines(d) is equivalent to d.relaxes(this).
Parameters
----------
dim : Dimension
The dimension to compare this dimension with.
Returns
----------
relaxes : bool
True if this dimension refines d, else False.
:param dim: The dimension to compare this dimension with.
:type dim: Dimension
:return: True if this dimension refines d, else False.
:rtype: bool
)");
dim.def("__str__", [](const ov::Dimension& self) -> std::string {

412
src/bindings/python/src/pyopenvino/graph/model.cpp
View File

@@ -63,19 +63,14 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model.
Parameters
----------
results : List[op.Result]
List of results.
sinks : List[Node]
List of Nodes to be used as Sinks (e.g. Assign ops).
parameters : List[op.Parameter]
List of parameters.
name : str
String to set as function's friendly name.
:param results: List of results.
:type results: List[op.Result]
:param sinks: List of Nodes to be used as Sinks (e.g. Assign ops).
:type sinks: List[Node]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const std::vector<std::shared_ptr<ov::Node>>& results,
@@ -90,16 +85,12 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model.
Parameters
----------
results : List[Node]
List of Nodes to be used as results.
parameters : List[op.Parameter]
List of parameters.
name : str
String to set as function's friendly name.
:param results: List of Nodes to be used as results.
:type results: List[Node]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const std::shared_ptr<ov::Node>& result,
@@ -114,16 +105,12 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model.
Parameters
----------
result : Node
Node to be used as result.
parameters : List[op.Parameter]
List of parameters.
name : str
String to set as function's friendly name.
:param result: Node to be used as result.
:type result: Node
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(
@@ -137,16 +124,12 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[Output]
List of outputs.
parameters : List[op.Parameter]
List of parameters.
name : str
String to set as function's friendly name.
:param results: List of outputs.
:type results: List[Output]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const ov::OutputVector& results,
@@ -164,19 +147,12 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[Output]
List of outputs.
sinks : List[Node]
List of Nodes to be used as Sinks (e.g. Assign ops).
parameters : List[op.Parameter]
List of parameters.
name : str
String to set as function's friendly name.
:param results: List of outputs.
:type results: List[Output]
:param sinks: List of Nodes to be used as Sinks (e.g. Assign ops).
:type sinks: List[Node]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const ov::ResultVector& results,
const std::vector<std::shared_ptr<ov::Node>>& nodes,
@@ -195,22 +171,16 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[op.Result]
List of results.
sinks : List[Node]
List of Nodes to be used as Sinks (e.g. Assign ops).
parameters : List[op.Parameter]
List of parameters.
variables : List[op.util.Variable]
List of variables.
name : str
String to set as function's friendly name.
:param results: List of results.
:type results: List[op.Result]
:param sinks: List of Nodes to be used as Sinks (e.g. Assign ops).
:type sinks: List[Node]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param variables: List of variables.
:type variables: List[op.util.Variable]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const ov::OutputVector& results,
@@ -230,22 +200,14 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[Output]
List of results.
sinks : List[Node]
List of Nodes to be used as Sinks (e.g. Assign ops).
parameters : List[op.Parameter]
List of parameters.
variables : List[op.util.Variable]
List of variables.
name : str
String to set as function's friendly name.
:param results: List of results.
:type results: List[Output]
:param sinks: List of Nodes to be used as Sinks (e.g. Assign ops).
:type sinks: List[Node]
:param variables: List of variables.
:type variables: List[op.util.Variable]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const ov::ResultVector& results,
@@ -262,19 +224,14 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[op.Result]
List of results.
parameters : List[op.Parameter]
List of parameters.
variables : List[op.util.Variable]
List of variables.
name : str
String to set as function's friendly name.
:param results: List of results.
:type results: List[op.Result]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param variables: List of variables.
:type variables: List[op.util.Variable]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def(py::init([](const ov::OutputVector& results,
@@ -291,19 +248,12 @@ void regclass_graph_Model(py::module m) {
R"(
Create user-defined Model which is a representation of a model
Parameters
----------
results : List[Output]
List of results.
parameters : List[op.Parameter]
List of parameters.
variables : List[op.util.Variable]
List of variables.
name : str
String to set as function's friendly name.
:param results: List of results.
:type results: List[Output]
:param parameters: List of parameters.
:type parameters: List[op.Parameter]
:param name: String to set as function's friendly name.
:type name: str
)");
function.def("validate_nodes_and_infer_types", &ov::Model::validate_nodes_and_infer_types);
@@ -315,14 +265,9 @@ void regclass_graph_Model(py::module m) {
},
py::arg("partial_shapes"),
R"(
Parameters
----------
partial_shapes : PartialShape
Index of Output.
Returns
----------
reshape : void
:param partial_shapes: Index of Output.
:type partial_shapes: PartialShape
:return : void
)");
function.def(
@@ -332,14 +277,10 @@ void regclass_graph_Model(py::module m) {
},
py::arg("partial_shapes"),
R"(
Parameters
----------
partial_shapes : Dict[int, PartialShape]
Index of Output.
Returns
----------
reshape : void
:param partial_shapes: Index of Output.
:type partial_shapes: Dict[int, PartialShape]
:return: void
)");
function.def(
@@ -349,14 +290,9 @@ void regclass_graph_Model(py::module m) {
},
py::arg("partial_shapes"),
R"(
Parameters
----------
partial_shapes : Dict[string, PartialShape]
Index of Output.
Returns
----------
reshape : void
:param partial_shapes: Index of Output.
:type partial_shapes: Dict[string, PartialShape]
:return: void
)");
function.def(
@@ -366,14 +302,9 @@ void regclass_graph_Model(py::module m) {
},
py::arg("partial_shapes"),
R"(
Parameters
----------
partial_shapes : Dict[Output, PartialShape]
Index of Output.
Returns
----------
reshape : void
:param partial_shapes: Index of Output.
:type partial_shapes: Dict[Output, PartialShape]
:return: void
)");
function.def("get_output_size",
@@ -381,30 +312,24 @@ void regclass_graph_Model(py::module m) {
R"(
Return the number of outputs for the function.
Returns
----------
get_output_size : int
Number of outputs.
:return: Number of outputs.
:rtype: int
)");
function.def("get_ops",
&ov::Model::get_ops,
R"(
Return ops used in the function.
Returns
----------
get_ops : List[Node]
List of Nodes representing ops used in function.
:return: List of Nodes representing ops used in function.
:rtype: List[Node]
)");
function.def("get_ordered_ops",
&ov::Model::get_ordered_ops,
R"(
Return ops used in the function in topological order.
Returns
----------
get_ordered_ops : List[Node]
List of sorted Nodes representing ops used in function.
:return: List of sorted Nodes representing ops used in function.
:rtype: List[Node]
)");
function.def("get_output_op",
&ov::Model::get_output_op,
@@ -412,15 +337,10 @@ void regclass_graph_Model(py::module m) {
R"(
Return the op that generates output i
Parameters
----------
index : int
output index
Returns
----------
get_output_op : Node
Node object that generates output i
:param index: output index
:type index: output index
:return: Node object that generates output i
:rtype: Node
)");
function.def("get_output_element_type",
&ov::Model::get_output_element_type,
@@ -428,15 +348,10 @@ void regclass_graph_Model(py::module m) {
R"(
Return the element type of output i
Parameters
----------
index : int
output index
Returns
----------
get_output_op : Type
Type object of output i
:param index: output index
:type index: int
:return: Type object of output i
:rtype: Type
)");
function.def("get_output_shape",
&ov::Model::get_output_shape,
@@ -444,15 +359,10 @@ void regclass_graph_Model(py::module m) {
R"(
Return the shape of element i
Parameters
----------
index : int
element index
Returns
----------
get_output_shape : Shape
Shape object of element i
:param index: element index
:type index: int
:return: Shape object of element i
:rtype: Shape
)");
function.def("get_output_partial_shape",
&ov::Model::get_output_partial_shape,
@@ -460,45 +370,34 @@ void regclass_graph_Model(py::module m) {
R"(
Return the partial shape of element i
Parameters
----------
index : int
element index
Returns
----------
get_output_partial_shape : PartialShape
PartialShape object of element i
:param index: element index
:type index: int
:return: PartialShape object of element i
:rtype: PartialShape
)");
function.def("get_parameters",
&ov::Model::get_parameters,
R"(
Return the function parameters.
Returns
----------
get_parameters : ParameterVector
ParameterVector containing function parameters.
:return: ParameterVector containing function parameters.
:rtype: ParameterVector
)");
function.def("get_results",
&ov::Model::get_results,
R"(
Return a list of function outputs.
Returns
----------
get_results : ResultVector
ResultVector containing function parameters.
:return: ResultVector containing function parameters.
:rtype: ResultVector
)");
function.def("get_result",
&ov::Model::get_result,
R"(
Return single result.
Returns
----------
get_result : Node
Node object representing result.
:return: Node object representing result.
:rtype: Node
)");
function.def("get_result_index",
(int64_t(ov::Model::*)(const ov::Output<ov::Node>&) const) & ov::Model::get_result_index,
@@ -508,15 +407,10 @@ void regclass_graph_Model(py::module m) {
Return -1 if `value` not matched.
Parameters
----------
value : Output
Output containing Node
Returns
----------
get_result_index : int
Index for value referencing it.
:param value: Output containing Node
:type value: Output
:return: Index for value referencing it.
:rtype: int
)");
function.def("get_result_index",
(int64_t(ov::Model::*)(const ov::Output<const ov::Node>&) const) & ov::Model::get_result_index,
@@ -526,15 +420,10 @@ void regclass_graph_Model(py::module m) {
Return -1 if `value` not matched.
Parameters
----------
value : Output
Output containing Node
Returns
----------
get_result_index : int
Index for value referencing it.
:param value: Output containing Node
:type value: Output
:return: Index for value referencing it.
:rtype: int
)");
function.def("get_name",
@@ -542,10 +431,8 @@ void regclass_graph_Model(py::module m) {
R"(
Get the unique name of the function.
Returns
----------
get_name : str
String with a name of the function.
:return: String with a name of the function.
:rtype: str
)");
function.def("get_friendly_name",
&ov::Model::get_friendly_name,
@@ -554,10 +441,8 @@ void regclass_graph_Model(py::module m) {
friendly name has been set via set_friendly_name
then the function's unique name is returned.
Returns
----------
get_friendly_name : str
String with a friendly name of the function.
:return: String with a friendly name of the function.
:rtype: str
)");
function.def("set_friendly_name",
&ov::Model::set_friendly_name,
@@ -568,10 +453,8 @@ void regclass_graph_Model(py::module m) {
is retrieved via get_friendly_name(). Used mainly
for debugging.
Parameters
----------
name : str
String to set as the friendly name.
:param name: String to set as the friendly name.
:type name: str
)");
function.def("is_dynamic",
&ov::Model::is_dynamic,
@@ -579,9 +462,7 @@ void regclass_graph_Model(py::module m) {
Returns true if any of the op's defined in the function
contains partial shape.
Returns
----------
is_dynamic : bool
:rtype: bool
)");
function.def("input", (ov::Output<ov::Node>(ov::Model::*)()) & ov::Model::input);
@@ -664,17 +545,15 @@ void regclass_graph_Model(py::module m) {
py::arg("parameter_index"),
py::arg("parameter"),
R"(
Replace the `parameter_index`th parameter of the function with `parameter`.
Replace the `parameter_index` parameter of the function with `parameter`
All users of the `parameter_index`th parameter are redirected to `parameter`, and the
`parameter_index`th entry in the function parameter list is replaced with `parameter`.
All users of the `parameter_index` parameter are redirected to `parameter` , and the
`parameter_index` entry in the function parameter list is replaced with `parameter`
Parameters
----------
parameter_index : int
The index of the parameter to replace.
parameter: op.Parameter
The parameter to substitute for the `parameter_index`th parameter.
:param parameter_index: The index of the parameter to replace.
:type parameter_index: int
:param parameter: The parameter to substitute for the `parameter_index` parameter.
:type parameter: op.Parameter
)");
function.def(
@@ -682,18 +561,14 @@ void regclass_graph_Model(py::module m) {
(int64_t(ov::Model::*)(const std::shared_ptr<ov::op::v0::Parameter>&) const) & ov::Model::get_parameter_index,
py::arg("parameter"),
R"(
Return the index position of `parameter`.
Return the index position of `parameter`
Return -1 if parameter not matched.
Parameters
----------
parameter : op.Parameter
Returns
----------
get_parameter_index : int
Index for parameter
:param parameter: Parameter which index is to be found.
:type parameter: op.Parameter
:return: Index for parameter
:rtype: int
)");
function.def(
@@ -710,20 +585,15 @@ void regclass_graph_Model(py::module m) {
R"(
Evaluate the function on inputs, putting results in outputs
Parameters
----------
output_tensors : List[op.Tensor]
Tensors for the outputs to compute. One for each result
input_tensors : List[op.Tensor]
Tensors for the inputs. One for each inputs.
evaluation_context: PyRTMap
Storage of additional settings and attributes that can be used
when evaluating the function. This additional information can be shared across nodes.
Returns
----------
evaluate : bool
:param output_tensors: Tensors for the outputs to compute. One for each result
:type output_tensors: List[op.Tensor]
:param input_tensors: Tensors for the inputs. One for each inputs.
:type input_tensors: List[op.Tensor]
:param evaluation_context: Storage of additional settings and attributes that can be used
when evaluating the function. This additional information can be
shared across nodes.
:type evaluation_context: PyRTMap
:rtype: bool
)");
function.def("__repr__", [](const ov::Model& self) {
std::string class_name = py::cast(self).get_type().attr("__name__").cast<std::string>();

210
src/bindings/python/src/pyopenvino/graph/node.cpp
View File

@@ -93,18 +93,15 @@ void regclass_graph_Node(py::module m) {
py::arg("evaluationContext"),
R"(
Evaluate the node on inputs, putting results in outputs
Parameters
----------
output_tensors : List[op.Tensor]
Tensors for the outputs to compute. One for each result.
input_tensors : List[op.Tensor]
Tensors for the inputs. One for each inputs.
evaluation_context: PyRTMap
Storage of additional settings and attributes that can be used
when evaluating the function. This additional information can be shared across nodes.
Returns
----------
evaluate : bool
:param output_tensors: Tensors for the outputs to compute. One for each result.
:type output_tensors: List[op.Tensor]
:param input_tensors: Tensors for the inputs. One for each inputs.
:type input_tensors: List[op.Tensor]
:param evaluation_context: Storage of additional settings and attributes that can be used
when evaluating the function. This additional information can be shared across nodes.
:type evaluation_context: PyRTMap
:rtype: bool
)");
node.def(
"evaluate",
@@ -115,15 +112,12 @@ void regclass_graph_Node(py::module m) {
py::arg("input_values"),
R"(
Evaluate the function on inputs, putting results in outputs
Parameters
----------
output_tensors : List[op.Tensor]
Tensors for the outputs to compute. One for each result.
input_tensors : List[op.Tensor]
Tensors for the inputs. One for each inputs.
Returns
----------
evaluate : bool
:param output_tensors: Tensors for the outputs to compute. One for each result.
:type output_tensors: List[op.Tensor]
:param input_tensors: Tensors for the inputs. One for each inputs.
:type input_tensors: List[op.Tensor]
:rtype: bool
)");
node.def("get_input_tensor",
&ov::Node::get_input_tensor,
@@ -132,35 +126,26 @@ void regclass_graph_Node(py::module m) {
R"(
Returns the tensor for the node's input with index i
Parameters
----------
index : int
Index of Input.
Returns
----------
input_tensor : descriptor.Tensor
Tensor of the input i
:param index: Index of Input.
:type index: int
:return: Tensor of the input i
:rtype: descriptor.Tensor
)");
node.def("get_element_type",
&ov::Node::get_element_type,
R"(
Checks that there is exactly one output and returns it's element type.
Returns
----------
get_element_type : Type
Type of the output.
:return: Type of the output.
:rtype: Type
)");
node.def("input_values",
&ov::Node::input_values,
R"(
Returns list of node's inputs, in order.
Returns
----------
inputs : List[Input]
List of node's inputs
:return: List of node's inputs
:rtype: List[Input]
)");
node.def("input_value",
&ov::Node::input_value,
@@ -168,35 +153,26 @@ void regclass_graph_Node(py::module m) {
R"(
Returns input of the node with index i
Parameters
----------
index : int
Index of Input.
Returns
----------
input : Input
Input of this node.
:param index: Index of Input.
:type index: int
:return: Input of this node.
:rtype: Input
)");
node.def("get_input_size",
&ov::Node::get_input_size,
R"(
Returns the number of inputs to the node.
Returns
----------
input_size : int
Number of inputs.
:return: Number of inputs.
:rtype: int
)");
node.def("get_output_size",
&ov::Node::get_output_size,
R"(
Returns the number of outputs from the node.
Returns
----------
output_size : int
Number of outputs.
:return: Number of outputs.
:rtype: int
)");
node.def("get_output_element_type",
&ov::Node::get_output_element_type,
@@ -204,15 +180,10 @@ void regclass_graph_Node(py::module m) {
R"(
Returns the element type for output i
Parameters
----------
index : int
Index of the output.
Returns
----------
get_output_element_type : Type
Type of the output i
:param index: Index of the output.
:type index: int
:return: Type of the output i
:rtype: Type
)");
node.def("get_output_shape",
&ov::Node::get_output_shape,
@@ -220,15 +191,10 @@ void regclass_graph_Node(py::module m) {
R"(
Returns the shape for output i
Parameters
----------
index : int
Index of the output.
Returns
----------
get_output_shape : Shape
Shape of the output i
:param index: Index of the output.
:return: Shape of the output i
:rtype: Shape
)");
node.def("get_output_partial_shape",
&ov::Node::get_output_partial_shape,
@@ -236,15 +202,10 @@ void regclass_graph_Node(py::module m) {
R"(
Returns the partial shape for output i
Parameters
----------
index : int
Index of the output.
Returns
----------
get_output_partial_shape : PartialShape
PartialShape of the output i
:param index: Index of the output.
:type index: int
:return: PartialShape of the output i
:rtype: PartialShape
)");
node.def("get_output_tensor",
&ov::Node::get_output_tensor,
@@ -253,35 +214,26 @@ void regclass_graph_Node(py::module m) {
R"(
Returns the tensor for output i
Parameters
----------
index : int
Index of the output.
Returns
----------
get_output_tensor : descriptor.Tensor
Tensor of the output i
:param index: Index of the output.
:type index: int
:return: Tensor of the output i
:rtype: descriptor.Tensor
)");
node.def("get_type_name",
&ov::Node::get_type_name,
R"(
Returns Type's name from the node.
Returns
----------
get_type_name : str
String repesenting Type's name.
:return: String repesenting Type's name.
:rtype: str
)");
node.def("get_name",
&ov::Node::get_name,
R"(
Get the unique name of the node
Returns
----------
get_name : str
Unique name of the node.
:return: Unique name of the node.
:rtype: str
)");
node.def("get_friendly_name",
&ov::Node::get_friendly_name,
@@ -290,10 +242,8 @@ void regclass_graph_Node(py::module m) {
been set via set_friendly_name then the node's unique name
is returned.
Returns
----------
get_name : str
Friendly name of the node.
:return: Friendly name of the node.
:rtype: str
)");
node.def("get_type_info", &ov::Node::get_type_info);
node.def("set_friendly_name",
@@ -304,10 +254,8 @@ void regclass_graph_Node(py::module m) {
of the node and is retrieved via get_friendly_name(). Used mainly for
debugging. The friendly name may be set exactly once.
Parameters
----------
name : str
Friendly name to set.
:param name: Friendly name to set.
:type name: str
)");
node.def("input",
(ov::Input<ov::Node>(ov::Node::*)(size_t)) & ov::Node::input,
@@ -315,25 +263,18 @@ void regclass_graph_Node(py::module m) {
R"(
A handle to the input_index input of this node.
Parameters
----------
input_index : int
Index of Input.
Returns
----------
input : Input
Input of this node.
:param input_index: Index of Input.
:type input_index: int
:return: Input of this node.
:rtype: Input
)");
node.def("inputs",
(std::vector<ov::Input<ov::Node>>(ov::Node::*)()) & ov::Node::inputs,
R"(
A list containing a handle for each of this node's inputs, in order.
Returns
----------
inputs : List[Input]
List of node's inputs.
:return: List of node's inputs.
:rtype: List[Input]
)");
node.def("output",
(ov::Output<ov::Node>(ov::Node::*)(size_t)) & ov::Node::output,
@@ -341,25 +282,18 @@ void regclass_graph_Node(py::module m) {
R"(
A handle to the output_index output of this node.
Parameters
----------
output_index : int
Index of Output.
Returns
----------
input : Output
Output of this node.
:param output_index: Index of Output.
:type output_index: int
:return: Output of this node.
:rtype: Output
)");
node.def("outputs",
(std::vector<ov::Output<ov::Node>>(ov::Node::*)()) & ov::Node::outputs,
R"(
A list containing a handle for each of this node's outputs, in order.
Returns
----------
inputs : List[Output]
List of node's outputs.
:return: List of node's outputs.
:rtype: List[Output]
)");
node.def("get_rt_info",
(PyRTMap & (ov::Node::*)()) & ov::Node::get_rt_info,
@@ -367,20 +301,16 @@ void regclass_graph_Node(py::module m) {
R"(
Returns PyRTMap which is a dictionary of user defined runtime info.
Returns
----------
get_rt_info : PyRTMap
A dictionary of user defined data.
:return: A dictionary of user defined data.
:rtype: PyRTMap
)");
node.def("get_version",
&ov::Node::get_version,
R"(
Returns operation's version of the node.
Returns
----------
get_version : int
Operation version.
:return: Operation version.
:rtype: int
)");
node.def("set_argument", &ov::Node::set_argument);

64
src/bindings/python/src/pyopenvino/graph/node_output.hpp
View File

@@ -20,7 +20,7 @@ template <typename VT>
void regclass_graph_Output(py::module m, std::string typestring)
{
auto pyclass_name = py::detail::c_str((typestring + std::string("Output")));
auto docs = py::detail::c_str((std::string("openvino.runtime.") + typestring + std::string("Output wraps ov::Output<") + typestring + std::string(" ov::Node >")));
auto docs = py::detail::c_str(std::string("openvino.runtime.") + typestring + std::string("Output represents port/node output."));
py::class_<ov::Output<VT>, std::shared_ptr<ov::Output<VT>>> output(m,
pyclass_name,
py::dynamic_attr());
@@ -43,20 +43,16 @@ void regclass_graph_Output(py::module m, std::string typestring)
R"(
Get node referenced by this output handle.
Returns
----------
get_node : Node or const Node
Node object referenced by this output handle.
:return: Node object referenced by this output handle.
:rtype: openvino.runtime.Node
)");
output.def("get_index",
&ov::Output<VT>::get_index,
R"(
The index of the output referred to by this output handle.
Returns
----------
get_index : int
Index value as integer.
:return: Index value as integer.
:rtype: int
)");
output.def("get_any_name",
&ov::Output<VT>::get_any_name,
@@ -64,30 +60,24 @@ void regclass_graph_Output(py::module m, std::string typestring)
One of the tensor names associated with this output.
Note: first name in lexicographical order.
Returns
----------
get_any_name : str
Tensor name as string.
:return: Tensor name as string.
:rtype: str
)");
output.def("get_names",
&ov::Output<VT>::get_names,
R"(
The tensor names associated with this output.
Returns
----------
get_names : set
Set of tensor names.
:return: Set of tensor names.
:rtype: Set[str]
)");
output.def("get_element_type",
&ov::Output<VT>::get_element_type,
R"(
The element type of the output referred to by this output handle.
Returns
----------
get_element_type : Type
Type of the output.
:return: Type of the output.
:rtype: openvino.runtime.Type
)");
output.def("get_shape",
&ov::Output<VT>::get_shape,
@@ -95,10 +85,8 @@ void regclass_graph_Output(py::module m, std::string typestring)
R"(
The shape of the output referred to by this output handle.
Returns
----------
get_shape : Shape
Copy of Shape of the output.
:return: Copy of Shape of the output.
:rtype: openvino.runtime.Shape
)");
output.def("get_partial_shape",
&ov::Output<VT>::get_partial_shape,
@@ -106,20 +94,17 @@ void regclass_graph_Output(py::module m, std::string typestring)
R"(
The partial shape of the output referred to by this output handle.
Returns
----------
get_partial_shape : PartialShape
Copy of PartialShape of the output.
:return: Copy of PartialShape of the output.
:rtype: openvino.runtime.PartialShape
)");
output.def("get_target_inputs",
&ov::Output<VT>::get_target_inputs,
R"(
A set containing handles for all inputs targeted by the output
referenced by this output handle.
Returns
----------
get_target_inputs : Set[Input]
Set of Inputs.
:return: Set of Inputs.
:rtype: Set[openvino.runtime.Input]
)");
output.def("_from_node", [](const std::shared_ptr<ov::Node>& node) {
return ov::Output<ov::Node>(node);
@@ -129,10 +114,9 @@ void regclass_graph_Output(py::module m, std::string typestring)
py::return_value_policy::reference_internal,
R"(
A reference to the tensor descriptor for this output.
Returns
----------
get_tensor : descriptor.Tensor
Tensor of the output.
:return: Tensor of the output.
:rtype: openvino.pyopenvino.DescriptorTensor
)");
output.def("get_rt_info",
(ov::RTMap & (ov::Output<VT>::*)()) & ov::Output<VT>::get_rt_info,
@@ -140,10 +124,8 @@ void regclass_graph_Output(py::module m, std::string typestring)
R"(
Returns RTMap which is a dictionary of user defined runtime info.
Returns
----------
get_rt_info : RTMap
A dictionary of user defined data.
:return: A dictionary of user defined data.
:rtype: openvino.runtime.RTMap
)");

94
src/bindings/python/src/pyopenvino/graph/partial_shape.cpp
View File

@@ -68,16 +68,10 @@ void regclass_graph_PartialShape(py::module m) {
Check whether this shape is compatible with the argument, i.e.,
whether it is possible to merge them.
Parameters
----------
shape : PartialShape
The shape to be checked for compatibility with this shape.
Returns
----------
compatible : bool
True if this shape is compatible with s, else False.
:param shape: The shape to be checked for compatibility with this shape.
:type shape: openvino.runtime.PartialShape
:return: True if this shape is compatible with s, else False.
:rtype: bool
)");
shape.def("refines",
&ov::PartialShape::refines,
@@ -85,15 +79,10 @@ void regclass_graph_PartialShape(py::module m) {
R"(
Check whether this shape is a refinement of the argument.
Parameters
----------
shape : PartialShape
The shape which is being compared against this shape.
Returns
----------
refines : bool
True if this shape refines s, else False.
:param shape: The shape which is being compared against this shape.
:type shape: openvino.runtime.PartialShape
:return: True if this shape refines s, else False.
:rtype: bool
)");
shape.def("relaxes",
&ov::PartialShape::relaxes,
@@ -101,15 +90,10 @@ void regclass_graph_PartialShape(py::module m) {
R"(
Check whether this shape is a relaxation of the argument.
Parameters
----------
shape : PartialShape
The shape which is being compared against this shape.
Returns
----------
relaxes : bool
True if this shape relaxes s, else False.
:param shape: The shape which is being compared against this shape.
:type shape: openvino.runtime.PartialShape
:return: True if this shape relaxes s, else False.
:rtype: bool
)");
shape.def("same_scheme",
&ov::PartialShape::same_scheme,
@@ -117,47 +101,34 @@ void regclass_graph_PartialShape(py::module m) {
R"(
Check whether this shape represents the same scheme as the argument.
Parameters
----------
shape : PartialShape
The shape which is being compared against this shape.
Returns
----------
same_scheme : bool
True if shape represents the same scheme as s, else False.
:param shape: The shape which is being compared against this shape.
:type shape: openvino.runtime.PartialShape
:return: True if shape represents the same scheme as s, else False.
:rtype: bool
)");
shape.def("get_max_shape",
&ov::PartialShape::get_max_shape,
R"(
Returns
----------
get_max_shape : Shape
Get the max bounding shape.
:return: Get the max bounding shape.
:rtype: openvino.runtime.Shape
)");
shape.def("get_min_shape",
&ov::PartialShape::get_min_shape,
R"(
Returns
----------
get_min_shape : Shape
Get the min bounding shape.
:return: Get the min bounding shape.
:rtype: openvino.runtime.Shape
)");
shape.def("get_shape",
&ov::PartialShape::get_shape,
R"(
Returns
----------
get_shape : Shape
Get the unique shape.
:return: Get the unique shape.
:rtype: openvino.runtime.Shape
)");
shape.def("to_shape",
&ov::PartialShape::to_shape,
R"(
Returns
----------
to_shapess : Shape
Get the unique shape.
:return: Get the unique shape.
:rtype: openvino.runtime.Shape
)");
shape.def(
"get_dimension",
@@ -166,18 +137,13 @@ void regclass_graph_PartialShape(py::module m) {
},
py::arg("index"),
R"(
Get the dimension at specified index of a partial shape.
Get the dimension at specified index of a partial shape.
Parameters
----------
index : int
The index of dimension
Returns
----------
get_dimension : Dimension
Get the particular dimension of a partial shape.
)");
:param index: The index of dimension.
:type index: int
:return: Get the particular dimension of a partial shape.
:rtype: openvino.runtime.Dimension
)");
shape.def(
"__eq__",

476
src/bindings/python/src/pyopenvino/graph/preprocess/pre_post_process.cpp
View File

@@ -25,147 +25,170 @@ static void regclass_graph_PreProcessSteps(py::module m) {
steps.def(
"mean",
[](ov::preprocess::PreProcessSteps& me, float value) {
return &me.mean(value);
[](ov::preprocess::PreProcessSteps& self, float value) {
return &self.mean(value);
},
py::arg("value"),
R"(
Subtracts single float value from each element in input tensor.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : float
Value to subtract.
Returns
----------
mean : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Subtracts single float value from each element in input tensor.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : float
Value to subtract.
Returns
----------
selfan : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"mean",
[](ov::preprocess::PreProcessSteps& me, const std::vector<float>& values) {
return &me.mean(values);
[](ov::preprocess::PreProcessSteps& self, const std::vector<float>& values) {
return &self.mean(values);
},
py::arg("values"),
R"(
Subtracts a given single float value from each element in a given channel from input tensor.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
values : List
Values to subtract.
Returns
----------
mean : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Subtracts a given single float value from each element in a given channel from input tensor.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
values : List[float]
Values to subtract.
Returns
----------
selfan : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"scale",
[](ov::preprocess::PreProcessSteps& me, float value) {
return &me.scale(value);
[](ov::preprocess::PreProcessSteps& self, float value) {
return &self.scale(value);
},
py::arg("value"),
R"(
Divides each element in input tensor by specified constant float value.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : float
Value to divide.
Returns
----------
scale : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Divides each element in input tensor by specified constant float value.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : float
Value to divide.
Returns
----------
scale : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"scale",
[](ov::preprocess::PreProcessSteps& me, const std::vector<float>& values) {
return &me.scale(values);
[](ov::preprocess::PreProcessSteps& self, const std::vector<float>& values) {
return &self.scale(values);
},
py::arg("values"),
R"(
Divides each element in a given channel from input tensor by a given single float value.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : List
Value to divide.
Returns
----------
scale : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Divides each element in a given channel from input tensor by a given single float value.
Input tensor must have ov.Type.f32 data type.
Parameters
----------
value : List[float]
Value to divide.
Returns
----------
scale : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"convert_element_type",
[](ov::preprocess::PreProcessSteps& me, ov::element::Type type) {
return &me.convert_element_type(type);
[](ov::preprocess::PreProcessSteps& self, ov::element::Type type) {
return &self.convert_element_type(type);
},
py::arg("type"),
R"(
Converts input tensor element type to specified type.
Input tensor must have openvino.Type.f32 data type.
Parameters
----------
type : Type
Destination type.
Returns
----------
convert_element_type : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Converts input tensor element type to specified type.
Input tensor must have openvino.Type.f32 data type.
Parameters
----------
type : openvino.runtime.Type
Destination type.
Returns
----------
convert_element_type : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"custom",
[](ov::preprocess::PreProcessSteps& me, py::function op) {
return &me.custom(op.cast<const ov::preprocess::PreProcessSteps::CustomPreprocessOp>());
[](ov::preprocess::PreProcessSteps& self, py::function op) {
return &self.custom(op.cast<const ov::preprocess::PreProcessSteps::CustomPreprocessOp>());
},
py::arg("operation"),
R"(
Adds custom preprocessing operation.
Parameters
----------
operation : function taking Output<Node> as input argument and returning Output<Node> after preprocessing.
Returns
----------
custom : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Adds custom preprocessing operation.
Parameters
----------
operation : function taking Output<Node> as input argument and returning Output<Node> after preprocessing.
Returns
----------
custom : PreProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"convert_color",
[](ov::preprocess::PreProcessSteps& me, const ov::preprocess::ColorFormat& dst_format) {
return &me.convert_color(dst_format);
[](ov::preprocess::PreProcessSteps& self, const ov::preprocess::ColorFormat& dst_format) {
return &self.convert_color(dst_format);
},
py::arg("dst_format"));
steps.def(
"resize",
[](ov::preprocess::PreProcessSteps& me,
[](ov::preprocess::PreProcessSteps& self,
ov::preprocess::ResizeAlgorithm alg,
size_t dst_height,
size_t dst_width) {
return &me.resize(alg, dst_height, dst_width);
return &self.resize(alg, dst_height, dst_width);
},
py::arg("alg"),
py::arg("dst_height"),
py::arg("dst_width"));
steps.def(
"resize",
[](ov::preprocess::PreProcessSteps& me, ov::preprocess::ResizeAlgorithm alg) {
return &me.resize(alg);
[](ov::preprocess::PreProcessSteps& self, ov::preprocess::ResizeAlgorithm alg) {
return &self.resize(alg);
},
py::arg("alg"));
steps.def(
"convert_layout",
[](ov::preprocess::PreProcessSteps& me, const ov::Layout& layout = {}) {
return &me.convert_layout(layout);
[](ov::preprocess::PreProcessSteps& self, const ov::Layout& layout = {}) {
return &self.convert_layout(layout);
},
py::arg("dst_layout"));
steps.def(
"convert_layout",
[](ov::preprocess::PreProcessSteps& me, const std::vector<uint64_t>& dims) {
return &me.convert_layout(dims);
[](ov::preprocess::PreProcessSteps& self, const std::vector<uint64_t>& dims) {
return &self.convert_layout(dims);
},
py::arg("dims"));
steps.def("reverse_channels", [](ov::preprocess::PreProcessSteps& me) {
return &me.reverse_channels();
steps.def("reverse_channels", [](ov::preprocess::PreProcessSteps& self) {
return &self.reverse_channels();
});
}
@@ -177,50 +200,57 @@ static void regclass_graph_PostProcessSteps(py::module m) {
steps.def(
"convert_element_type",
[](ov::preprocess::PostProcessSteps& me, ov::element::Type type) {
return &me.convert_element_type(type);
[](ov::preprocess::PostProcessSteps& self, ov::element::Type type) {
return &self.convert_element_type(type);
},
py::arg("type"),
R"(
Converts tensor element type to specified type.
Tensor must have openvino.Type.f32 data type.
Parameters
----------
type : Type
Destination type.
Returns
----------
convert_element_type : PostProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Converts tensor element type to specified type.
Tensor must have openvino.Type.f32 data type.
Parameters
----------
type : Type
Destination type.
Returns
----------
convert_element_type : PostProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
steps.def(
"convert_layout",
[](ov::preprocess::PostProcessSteps& me, const ov::Layout& layout = {}) {
return &me.convert_layout(layout);
[](ov::preprocess::PostProcessSteps& self, const ov::Layout& layout = {}) {
return &self.convert_layout(layout);
},
py::arg("dst_layout"));
steps.def(
"convert_layout",
[](ov::preprocess::PostProcessSteps& me, const std::vector<uint64_t>& dims) {
return &me.convert_layout(dims);
[](ov::preprocess::PostProcessSteps& self, const std::vector<uint64_t>& dims) {
return &self.convert_layout(dims);
},
py::arg("dims"));
steps.def(
"custom",
[](ov::preprocess::PostProcessSteps& me, py::function op) {
return &me.custom(op.cast<const ov::preprocess::PostProcessSteps::CustomPostprocessOp>());
[](ov::preprocess::PostProcessSteps& self, py::function op) {
return &self.custom(op.cast<const ov::preprocess::PostProcessSteps::CustomPostprocessOp>());
},
py::arg("operation"),
R"(
Adds custom postprocessing operation.
Parameters
----------
operation : function taking Output<Node> as input argument and returning Output<Node> after postprocessing.
Returns
----------
custom : PostProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
Adds custom postprocessing operation.
Parameters
----------
operation : function taking Output<Node> as input argument and returning Output<Node> after postprocessing.
Returns
----------
custom : PostProcessSteps
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
}
static void regclass_graph_InputTensorInfo(py::module m) {
@@ -231,47 +261,75 @@ static void regclass_graph_InputTensorInfo(py::module m) {
info.def(
"set_element_type",
[](ov::preprocess::InputTensorInfo& me, const ov::element::Type& type) {
return &me.set_element_type(type);
[](ov::preprocess::InputTensorInfo& self, const ov::element::Type& type) {
return &self.set_element_type(type);
},
py::arg("type"),
R"(
Set initial client's tensor element type. If type is not the same as model's element type,
conversion of element type will be done automatically.
Parameters
----------
type : Type
Client's input tensor element type.
Returns
----------
tensor : InputTensorInfo
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
info.def("set_layout", [](ov::preprocess::InputTensorInfo& me, const ov::Layout& layout) {
return &me.set_layout(layout);
});
info.def("set_spatial_dynamic_shape", [](ov::preprocess::InputTensorInfo& me) {
return &me.set_spatial_dynamic_shape();
});
info.def("set_spatial_static_shape", [](ov::preprocess::InputTensorInfo& me, size_t height, size_t width) {
return &me.set_spatial_static_shape(height, width);
});
info.def("set_shape", [](ov::preprocess::InputTensorInfo& me, const ov::PartialShape& shape) {
return &me.set_shape(shape);
Set initial client's tensor element type. If type is not the same as model's element type,
conversion of element type will be done automatically.
Parameters
----------
type : Type
Client's input tensor element type.
Returns
----------
tensor : InputTensorInfo
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
info.def(
"set_layout",
[](ov::preprocess::InputTensorInfo& self, const ov::Layout& layout) {
return &self.set_layout(layout);
},
py::arg("layout"));
info.def("set_spatial_dynamic_shape", [](ov::preprocess::InputTensorInfo& self) {
return &self.set_spatial_dynamic_shape();
});
info.def(
"set_spatial_static_shape",
[](ov::preprocess::InputTensorInfo& self, size_t height, size_t width) {
return &self.set_spatial_static_shape(height, width);
},
py::arg("height"),
py::arg("width"));
info.def(
"set_shape",
[](ov::preprocess::InputTensorInfo& self, const ov::PartialShape& shape) {
return &self.set_shape(shape);
},
py::arg("shape"));
// Allow to use set_shape([1,2,3]) in Python code, not set_shape(PartialShape([1,2,3]))
info.def("set_shape", [](ov::preprocess::InputTensorInfo& me, const std::vector<int64_t>& shape) {
return &me.set_shape(shape);
});
info.def("set_color_format",
[](ov::preprocess::InputTensorInfo& me,
const ov::preprocess::ColorFormat& format,
const std::vector<std::string>& sub_names = {}) {
return &me.set_color_format(format, sub_names);
});
info.def("set_memory_type", [](ov::preprocess::InputTensorInfo& me, const std::string& memory_type) {
return &me.set_memory_type(memory_type);
});
info.def(
"set_shape",
[](ov::preprocess::InputTensorInfo& self, const std::vector<int64_t>& shape) {
return &self.set_shape(shape);
},
py::arg("shape"));
info.def(
"set_color_format",
[](ov::preprocess::InputTensorInfo& self,
const ov::preprocess::ColorFormat& format,
const std::vector<std::string>& sub_names = {}) {
return &self.set_color_format(format, sub_names);
},
py::arg("format"),
py::arg("sub_names"));
info.def(
"set_memory_type",
[](ov::preprocess::InputTensorInfo& self, const std::string& memory_type) {
return &self.set_memory_type(memory_type);
},
py::arg("memory_type"));
}
static void regclass_graph_OutputTensorInfo(py::module m) {
@@ -282,39 +340,45 @@ static void regclass_graph_OutputTensorInfo(py::module m) {
info.def(
"set_element_type",
[](ov::preprocess::OutputTensorInfo& me, const ov::element::Type& type) {
return &me.set_element_type(type);
[](ov::preprocess::OutputTensorInfo& self, const ov::element::Type& type) {
return &self.set_element_type(type);
},
py::arg("type"),
R"(
Set client's output tensor element type. If type is not the same as model's element type,
conversion of element type will be done automatically.
Parameters
----------
type : Type
Client's output tensor element type.
Returns
----------
tensor : OutputTensorInfo
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
info.def("set_layout", [](ov::preprocess::OutputTensorInfo& me, const ov::Layout& layout) {
return &me.set_layout(layout);
});
Set client's output tensor element type. If type is not the same as model's element type,
conversion of element type will be done automatically.
Parameters
----------
type : Type
Client's output tensor element type.
Returns
----------
tensor : OutputTensorInfo
Reference to itself to allow chaining of calls in client's code in a builder-like manner.
)");
info.def(
"set_layout",
[](ov::preprocess::OutputTensorInfo& self, const ov::Layout& layout) {
return &self.set_layout(layout);
},
py::arg("layout"));
}
static void regclass_graph_InputInfo(py::module m) {
py::class_<ov::preprocess::InputInfo, Common::ref_wrapper<ov::preprocess::InputInfo>> inp(m, "InputInfo");
inp.doc() = "openvino.runtime.preprocess.InputInfo wraps ov::preprocess::InputInfo";
inp.def("tensor", [](ov::preprocess::InputInfo& me) {
return &me.tensor();
inp.def("tensor", [](ov::preprocess::InputInfo& self) {
return &self.tensor();
});
inp.def("preprocess", [](ov::preprocess::InputInfo& me) {
return &me.preprocess();
inp.def("preprocess", [](ov::preprocess::InputInfo& self) {
return &self.preprocess();
});
inp.def("model", [](ov::preprocess::InputInfo& me) {
return &me.model();
inp.def("model", [](ov::preprocess::InputInfo& self) {
return &self.model();
});
}
@@ -322,14 +386,16 @@ static void regclass_graph_OutputInfo(py::module m) {
py::class_<ov::preprocess::OutputInfo, Common::ref_wrapper<ov::preprocess::OutputInfo>> out(m, "OutputInfo");
out.doc() = "openvino.runtime.preprocess.OutputInfo wraps ov::preprocess::OutputInfo";
out.def("tensor", [](ov::preprocess::OutputInfo& me) {
return &me.tensor();
out.def("tensor", [](ov::preprocess::OutputInfo& self) {
return &self.tensor();
});
out.def("postprocess", [](ov::preprocess::OutputInfo& me) {
return &me.postprocess();
out.def("postprocess", [](ov::preprocess::OutputInfo& self) {
return &self.postprocess();
});
out.def("model", [](ov::preprocess::OutputInfo& me) {
return &me.model();
out.def("model", [](ov::preprocess::OutputInfo& self) {
return &self.model();
});
}
@@ -339,9 +405,12 @@ static void regclass_graph_OutputModelInfo(py::module m) {
"OutputModelInfo");
info.doc() = "openvino.runtime.preprocess.OutputModelInfo wraps ov::preprocess::OutputModelInfo";
info.def("set_layout", [](ov::preprocess::OutputModelInfo& me, const ov::Layout& layout) {
return &me.set_layout(layout);
});
info.def(
"set_layout",
[](ov::preprocess::OutputModelInfo& self, const ov::Layout& layout) {
return &self.set_layout(layout);
},
py::arg("layout"));
}
static void regclass_graph_InputModelInfo(py::module m) {
@@ -350,9 +419,12 @@ static void regclass_graph_InputModelInfo(py::module m) {
"InputModelInfo");
info.doc() = "openvino.runtime.preprocess.InputModelInfo wraps ov::preprocess::InputModelInfo";
info.def("set_layout", [](ov::preprocess::InputModelInfo& me, const ov::Layout& layout) {
return &me.set_layout(layout);
});
info.def(
"set_layout",
[](ov::preprocess::InputModelInfo& self, const ov::Layout& layout) {
return &self.set_layout(layout);
},
py::arg("layout"));
}
static void regenum_graph_ColorFormat(py::module m) {
@@ -393,38 +465,44 @@ void regclass_graph_PrePostProcessor(py::module m) {
"PrePostProcessor");
proc.doc() = "openvino.runtime.preprocess.PrePostProcessor wraps ov::preprocess::PrePostProcessor";
proc.def(py::init<const std::shared_ptr<ov::Model>&>());
proc.def(py::init<const std::shared_ptr<ov::Model>&>(), py::arg("model"));
proc.def("input", [](ov::preprocess::PrePostProcessor& me) {
return &me.input();
proc.def("input", [](ov::preprocess::PrePostProcessor& self) {
return &self.input();
});
proc.def(
"input",
[](ov::preprocess::PrePostProcessor& me, const std::string& tensor_name) {
return &me.input(tensor_name);
[](ov::preprocess::PrePostProcessor& self, const std::string& tensor_name) {
return &self.input(tensor_name);
},
py::arg("tensor_name"));
proc.def(
"input",
[](ov::preprocess::PrePostProcessor& me, size_t input_index) {
return &me.input(input_index);
[](ov::preprocess::PrePostProcessor& self, size_t input_index) {
return &self.input(input_index);
},
py::arg("input_index"));
proc.def("output", [](ov::preprocess::PrePostProcessor& me) {
return &me.output();
proc.def("output", [](ov::preprocess::PrePostProcessor& self) {
return &self.output();
});
proc.def(
"output",
[](ov::preprocess::PrePostProcessor& me, const std::string& tensor_name) {
return &me.output(tensor_name);
[](ov::preprocess::PrePostProcessor& self, const std::string& tensor_name) {
return &self.output(tensor_name);
},
py::arg("tensor_name"));
proc.def(
"output",
[](ov::preprocess::PrePostProcessor& me, size_t output_index) {
return &me.output(output_index);
[](ov::preprocess::PrePostProcessor& self, size_t output_index) {
return &self.output(output_index);
},
py::arg("output_index"));
proc.def("build", &ov::preprocess::PrePostProcessor::build);
proc.def("__str__", [](const ov::preprocess::PrePostProcessor& self) -> std::string {

2
src/bindings/python/src/pyopenvino/graph/rt_map.cpp
View File

@@ -28,7 +28,7 @@ PYBIND11_MAKE_OPAQUE(PyRTMap);
void regclass_graph_PyRTMap(py::module m) {
auto py_map = py::class_<PyRTMap>(m, "RTMap");
py_map.doc() = "openvino.runtime.RTMap makes bindings for std::map<std::string, "
"ov::Any, which can later be used as ov::Node::RTMap";
"ov::Any>, which can later be used as ov::Node::RTMap";
py_map.def("__setitem__", [](PyRTMap& m, const std::string& k, const std::string v) {
m[k] = v;

21
src/bindings/python/src/pyopenvino/graph/util.cpp
View File

@@ -22,18 +22,13 @@ void regmodule_graph_util(py::module m) {
&ov::get_constant_from_source,
py::arg("output"),
R"(
Runs an estimation of source tensor.
Runs an estimation of source tensor.
Parameters
----------
output : Output
output node
Returns
----------
get_constant_from_source : Constant or None
If it succeeded to calculate both bounds and
they are the same returns Constant operation
from the resulting bound, otherwise Null.
)");
:param index: Output node.
:type index: openvino.runtime.Output
:return: If it succeeded to calculate both bounds and
they are the same returns Constant operation
from the resulting bound, otherwise Null.
:rtype: openvino.runtime.op.Constant or openvino.runtime.Node
)");
}

2
src/bindings/python/tests/test_inference_engine/test_compiled_model.py
View File

@@ -238,7 +238,7 @@ def test_inputs_docs(device):
exec_net = core.compile_model(func, device)
inputs = exec_net.inputs
input_0 = inputs[0]
expected_string = "openvino.runtime.ConstOutput wraps ov::Output<Const ov::Node >"
expected_string = "openvino.runtime.ConstOutput represents port/node output."
assert input_0.__doc__ == expected_string

64
src/bindings/python/tests/test_inference_engine/test_infer_request.py
View File

@@ -9,7 +9,8 @@ import datetime
import time
import openvino.runtime.opset8 as ops
from openvino.runtime import Core, AsyncInferQueue, Tensor, ProfilingInfo, Model, Type
from openvino.runtime import Core, AsyncInferQueue, Tensor, ProfilingInfo, Model
from openvino.runtime import Type, Shape, Layout
from openvino.preprocess import PrePostProcessor
from ..conftest import model_path, read_image
@@ -160,6 +161,67 @@ def test_set_tensors(device):
assert np.allclose(tensor4.data, t9.data, atol=1e-2, rtol=1e-2)
@pytest.mark.dynamic_library
@pytest.mark.template_extension
def test_batched_tensors(device):
batch = 4
one_shape = Shape([1, 2, 2, 2])
batch_shape = Shape([batch, 2, 2, 2])
one_shape_size = np.prod(one_shape)
core = Core()
core.register_plugin("openvino_template_plugin", "TEMPLATE")
data1 = ops.parameter(batch_shape, np.float32)
data1.set_friendly_name("input0")
data1.get_output_tensor(0).set_names({"tensor_input0"})
data1.set_layout(Layout("N..."))
constant = ops.constant([1], np.float32)
op1 = ops.add(data1, constant)
op1.set_friendly_name("Add0")
res1 = ops.result(op1)
res1.set_friendly_name("Result0")
res1.get_output_tensor(0).set_names({"tensor_output0"})
model = Model([res1], [data1])
compiled = core.compile_model(model, "TEMPLATE")
buffer = np.zeros([one_shape_size * batch * 2], dtype=np.float32)
req = compiled.create_infer_request()
tensors = []
for i in range(0, batch):
_start = i * one_shape_size * 2
# Use of special constructor for Tensor.
# It creates a Tensor from pointer, thus it requires only
# one element from original buffer, and shape to "crop".
tensor = Tensor(buffer[_start:(_start + 1)], one_shape)
tensors.append(tensor)
req.set_input_tensors(tensors) # using list overload!
actual_tensor = req.get_tensor("tensor_output0")
actual = actual_tensor.data
for test_num in range(0, 5):
for i in range(0, batch):
tensors[i].data[:] = test_num + 10
req.infer() # Adds '1' to each element
# Reference values for each batch:
_tmp = np.array([test_num + 11] * one_shape_size, dtype=np.float32).reshape([2, 2, 2])
for j in range(0, batch):
assert np.array_equal(actual[j], _tmp)
def test_inputs_outputs_property(device):
num_inputs = 10
input_shape = [1]

2
src/bindings/python/tests/test_inference_engine/test_output_const_node.py
View File

@@ -38,7 +38,7 @@ def test_const_output_docs(device):
func = core.read_model(model=test_net_xml, weights=test_net_bin)
exec_net = core.compile_model(func, device)
node = exec_net.input(0)
exptected_string = "openvino.runtime.ConstOutput wraps ov::Output<Const ov::Node >"
exptected_string = "openvino.runtime.ConstOutput represents port/node output."
assert node.__doc__ == exptected_string

15
src/bindings/python/tests/test_inference_engine/test_tensor.py
View File

@@ -107,6 +107,13 @@ def test_init_with_numpy_shared_memory(ov_type, numpy_dtype):
assert ov_tensor.size == arr.size
assert ov_tensor.byte_size == arr.nbytes
assert tuple(ov_tensor.get_shape()) == shape
assert ov_tensor.get_element_type() == ov_type
assert ov_tensor.data.dtype == numpy_dtype
assert ov_tensor.data.shape == shape
assert ov_tensor.get_size() == arr.size
assert ov_tensor.get_byte_size() == arr.nbytes
@pytest.mark.parametrize("ov_type, numpy_dtype", [
(ov.Type.f32, np.float32),
@@ -198,13 +205,21 @@ def test_set_shape(ov_type, numpy_dtype):
ref_shape = ov.Shape([1, 3, 48, 48])
ref_shape_np = [1, 3, 28, 28]
ov_tensor = Tensor(ov_type, shape)
ov_tensor.set_shape(ref_shape)
assert list(ov_tensor.shape) == list(ref_shape)
ov_tensor.shape = ref_shape
assert list(ov_tensor.shape) == list(ref_shape)
ones_arr = np.ones(list(ov_tensor.shape), numpy_dtype)
ov_tensor.data[:] = ones_arr
assert np.array_equal(ov_tensor.data, ones_arr)
ov_tensor.set_shape(ref_shape_np)
assert list(ov_tensor.shape) == ref_shape_np
ov_tensor.shape = ref_shape_np
assert list(ov_tensor.shape) == ref_shape_np
zeros = np.zeros(ref_shape_np, numpy_dtype)
ov_tensor.data[:] = zeros
assert np.array_equal(ov_tensor.data, zeros)

1
src/bindings/python/tests_compatibility/__init__.py
View File

@@ -138,3 +138,4 @@ xfail_issue_63138 = xfail_test(reason="Missing ONNX Shape-15 support")
xfail_issue_77668 = xfail_test(reason="Accuracy issue related to Gather-8.")
xfail_issue_78843 = xfail_test(reason="Missing reference output files for ssd mobilenet models")
xfail_issue_78741 = xfail_test(reason="Cannot get dims for non static shape")

16
src/bindings/python/tests_compatibility/test_inference_engine/test_Blob.py
View File

@@ -143,19 +143,3 @@ def test_cannot_set_shape_preallocated_memory():
with pytest.raises(RuntimeError) as e:
blob.set_shape([1, 4, 128, 128])
assert "Cannot call setShape for Blobs created on top of preallocated memory" in str(e.value)
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_blob_set_shape_after_async_infer():
function = create_encoder([1, 4, 20, 20])
net = ng.function_to_cnn(function)
net.reshape({"data": [(1, 5), 4, 20, 20]})
ie_core = IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
request = exec_net.requests[0]
request.async_infer({"data": np.ones([4, 4, 20, 20])})
with pytest.raises(RuntimeError) as e:
request.input_blobs['data'].set_shape([3, 4, 20, 20])
assert "REQUEST_BUSY" in str(e.value)
request.wait()

15
src/bindings/python/tests_compatibility/test_inference_engine/test_CDataPtr.py
View File

@@ -57,18 +57,3 @@ def test_initialized(device):
net = ie.read_network(model=test_net_xml, weights=test_net_bin)
exec_net = ie.load_network(net, device, num_requests=5)
assert exec_net.outputs['fc_out'].initialized, "Incorrect value for initialized property for layer 'fc_out"
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_is_dynamic():
function = create_relu([-1, 3, 20, 20])
net = ng.function_to_cnn(function)
ie = IECore()
ie.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie.load_network(net, "TEMPLATE")
assert exec_net.outputs["out"].is_dynamic
p_shape = ng.partial_shape_from_data(exec_net.outputs["out"])
assert isinstance(p_shape, ng.impl.PartialShape)
with pytest.raises(RuntimeError) as e:
exec_net.outputs["out"].shape
assert "Cannot return dims for Data with dynamic shapes!" in str(e.value)

21
src/bindings/python/tests_compatibility/test_inference_engine/test_DataPtr.py
View File

@@ -44,24 +44,3 @@ def test_layout():
def test_initialized():
assert layer_out_data().initialized, "Incorrect value for initialized property for layer 'fc_out'"
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_is_dynamic():
function = create_relu([-1, 3, 20, 20])
net = ng.function_to_cnn(function)
assert net.input_info["data"].input_data.is_dynamic
assert net.outputs["out"].is_dynamic
p_shape = ng.partial_shape_from_data(net.input_info["data"].input_data)
assert isinstance(p_shape, ng.impl.PartialShape)
p_shape = ng.partial_shape_from_data(net.outputs["out"])
assert isinstance(p_shape, ng.impl.PartialShape)
with pytest.raises(RuntimeError) as e:
net.input_info["data"].input_data.shape
assert "Cannot return dims for Data with dynamic shapes!" in str(e.value)
ie = IECore()
ie.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie.load_network(net, "TEMPLATE")
assert exec_net.input_info["data"].input_data.is_dynamic
p_shape = ng.partial_shape_from_data(exec_net.input_info["data"].input_data)
assert isinstance(p_shape, ng.impl.PartialShape)

44
src/bindings/python/tests_compatibility/test_inference_engine/test_IENetwork.py
View File

@@ -157,38 +157,6 @@ def test_reshape():
assert net.input_info["data"].input_data.shape == [2, 3, 32, 32]
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
@pytest.mark.parametrize("shape, p_shape", [
([1, 3, 22, 22], [1, 3, -1, 25]),
([1, 3, 22, 22], [-1, -1, -1, -1]),
([1, 3, -1, 25], [1, 3, 22, -1])
])
def test_reshape_with_partial_shape(device, shape, p_shape):
function = create_relu(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
changedFunction = ng.function_from_cnn(net)
p_shape = ng.impl.PartialShape(p_shape)
assert changedFunction.get_parameters()[0].get_partial_shape().is_dynamic
assert changedFunction.get_results()[0].get_output_partial_shape(0).is_dynamic
assert function.get_parameters()[0].get_partial_shape().is_dynamic
assert function.get_results()[0].get_output_partial_shape(0).is_dynamic
assert changedFunction.get_parameters()[0].get_partial_shape() == p_shape
assert changedFunction.get_results()[0].get_output_partial_shape(0) == p_shape
assert function.get_parameters()[0].get_partial_shape() == p_shape
assert function.get_results()[0].get_output_partial_shape(0) == p_shape
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_incorrect_reshape():
function = create_relu([1, 3, 22, 22])
net = ng.function_to_cnn(function)
with pytest.raises(ValueError) as e:
net.reshape({"data": [(2, 4, 6), 3, 22, 22]})
assert "Incorrect PartialShape dimension definition '(2, 4, 6)' " \
"in shape '[(2, 4, 6), 3, 22, 22]', expected one or two values for a dimension! " in str(e.value)
def test_net_from_buffer_valid():
ie = IECore()
with open(test_net_bin, 'rb') as f:
@@ -277,15 +245,3 @@ def test_tensor_names():
assert net.get_ov_name_for_tensor("relu_t") == "activation"
assert net.get_ov_name_for_tensor("identity_t") == "activation"
assert net.get_ov_name_for_tensor("input") == "in1"
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_create_two_exec_net():
function = create_relu([ng.Dimension(0,5), ng.Dimension(4), ng.Dimension(20), ng.Dimension(20)])
net = ng.function_to_cnn(function)
ie_core = IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net1 = ie_core.load_network(net, "TEMPLATE", num_requests=2)
assert ng.function_from_cnn(net) != None
exec_net2 = ie_core.load_network(net, "TEMPLATE", num_requests=2)
assert ng.function_from_cnn(net) != None

159
src/bindings/python/tests_compatibility/test_inference_engine/test_InferRequest.py
View File

@@ -580,146 +580,6 @@ def test_query_state_write_buffer(device, input_shape, data_type, mode):
"Expected values: {} \n Actual values: {} \n".format(expected_res, res)
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
@pytest.mark.parametrize("shape, p_shape, ref_shape", [
([1, 4, 20, 20], [-1, 4, 20, 20], [5, 4, 20, 20]),
([1, 4, 20, 20], [(0,5), 4, 20, 20], [3, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [2, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [6, 4, 20, 20]),
])
def test_infer_dynamic_network_with_set_shape(shape, p_shape, ref_shape):
function = create_encoder(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
exec_net.requests[0].input_blobs["data"].set_shape(ref_shape)
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape
exec_net.infer({"data": np.ones(ref_shape)})
request = exec_net.requests[0]
request.async_infer({"data": np.ones(ref_shape)})
status = request.wait(ie.WaitMode.RESULT_READY)
assert status == ie.StatusCode.OK
assert request.output_blobs['out'].tensor_desc.dims == ref_shape
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
@pytest.mark.parametrize("shape, p_shape, ref_shape", [
([1, 4, 20, 20], [-1, 4, 20, 20], [5, 4, 20, 20]),
([1, 4, 20, 20], [(0,5), 4, 20, 20], [3, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [2, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [6, 4, 20, 20]),
])
def test_infer_dynamic_network_without_set_shape(shape, p_shape, ref_shape):
function = create_encoder(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
exec_net.infer({"data": np.ones(ref_shape)})
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape
request = exec_net.requests[0]
request.async_infer({"data": np.ones(ref_shape)})
status = request.wait(ie.WaitMode.RESULT_READY)
assert status == ie.StatusCode.OK
assert request.output_blobs['out'].tensor_desc.dims == ref_shape
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
@pytest.mark.parametrize("shape, p_shape, ref_shape", [
([1, 4, 20, 20], [-1, 4, 20, 20], [5, 4, 20, 20]),
([1, 4, 20, 20], [(0,5), 4, 20, 20], [3, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [2, 4, 20, 20]),
([1, 4, 20, 20], [(3,5), 4, 20, 20], [6, 4, 20, 20]),
])
def test_infer_dynamic_network_with_set_blob(shape, p_shape, ref_shape):
function = create_encoder(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
tensor_desc = exec_net.requests[0].input_blobs["data"].tensor_desc
tensor_desc.dims = ref_shape
blob = ie.Blob(tensor_desc)
exec_net.requests[0].set_blob("data", blob)
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape
request = exec_net.requests[0]
request.infer({"data": np.ones(ref_shape)})
request.async_infer({"data": np.ones(ref_shape)})
status = request.wait(ie.WaitMode.RESULT_READY)
assert status == ie.StatusCode.OK
assert request.output_blobs["out"].tensor_desc.dims == ref_shape
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_infer_dynamic_network_twice():
shape, p_shape = [1, 4, 20, 20], [(0,5), 4, 20, 20]
ref_shape1, ref_shape2 = [2, 4, 20, 20], [3, 4, 20, 20]
function = create_encoder(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
request = exec_net.requests[0]
request.infer({"data": np.ones(ref_shape1)})
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape1
assert request.output_blobs['out'].tensor_desc.dims == ref_shape1
request.infer({"data": np.ones(ref_shape2)})
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape2
assert request.output_blobs['out'].tensor_desc.dims == ref_shape2
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_infer_dynamic_network_with_set_blob_twice():
shape, p_shape = [1, 4, 20, 20], [(0,5), 4, 20, 20]
ref_shape1, ref_shape2 = [2, 4, 20, 20], [3, 4, 20, 20]
function = create_encoder(shape)
net = ng.function_to_cnn(function)
net.reshape({"data": p_shape})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
request = exec_net.requests[0]
td = request.input_blobs['data'].tensor_desc
td.dims = ref_shape1
blob = ie.Blob(td)
request.set_blob("data", blob)
request.infer({"data": np.ones(ref_shape1)})
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape1
assert request.output_blobs['out'].tensor_desc.dims == ref_shape1
td = request.input_blobs['data'].tensor_desc
td.dims = ref_shape2
blob = ie.Blob(td)
request.set_blob("data", blob)
request.infer({"data": np.ones(ref_shape2)})
assert exec_net.requests[0].input_blobs["data"].tensor_desc.dims == ref_shape2
assert request.output_blobs['out'].tensor_desc.dims == ref_shape2
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
@pytest.mark.parametrize("shapes", [
([3, 4, 20, 20], [3, 4, 20, 20], [3, 4, 20, 20]),
([3, 4, 20, 20], [3, 4, 28, 28], [3, 4, 45, 45]),
])
def test_async_infer_dynamic_network_3_requests(shapes):
function = create_encoder([3, 4, 20, 20])
net = ng.function_to_cnn(function)
net.reshape({"data": [3, 4, (20, 50), (20, 50)]})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE", num_requests=3)
for i,request in enumerate(exec_net.requests):
request.async_infer({"data": np.ones(shapes[i])})
for i,request in enumerate(exec_net.requests):
status = request.wait(ie.WaitMode.RESULT_READY)
assert status == ie.StatusCode.OK
assert request.output_blobs['out'].tensor_desc.dims == shapes[i]
@pytest.mark.template_plugin
def test_set_blob_with_incorrect_name():
function = create_encoder([4, 4, 20, 20])
@@ -752,22 +612,3 @@ def test_set_blob_with_incorrect_size():
with pytest.raises(RuntimeError) as e:
exec_net.requests[0].set_blob("out", blob)
assert f"Output blob size is not equal network output size" in str(e.value)
@pytest.mark.skip(reason="Old Python API seg faults during dynamic shape inference")
def test_set_blob_after_async_infer():
function = create_encoder([1, 4, 20, 20])
net = ng.function_to_cnn(function)
net.reshape({"data": [(0, 5), 4, 20, 20]})
ie_core = ie.IECore()
ie_core.register_plugin("openvino_template_plugin", "TEMPLATE")
exec_net = ie_core.load_network(net, "TEMPLATE")
request = exec_net.requests[0]
tensor_desc = request.input_blobs['data'].tensor_desc
tensor_desc.dims = [2, 4, 20, 20]
blob = ie.Blob(tensor_desc)
request.async_infer({"data": np.ones([4, 4, 20, 20])})
with pytest.raises(RuntimeError) as e:
request.set_blob("data", blob)
assert "REQUEST_BUSY" in str(e.value)
request.wait()

Some files were not shown because too many files have changed in this diff Show More