DOCS shift to rst - Model representation doc (#17320)
* model representation to rst * fix indentation * fix cide tabs * fix indentation * change doc * fix snippets * fix snippet
This commit is contained in:
Tatiana Savina
GitHub
@@ -1,210 +1,226 @@
|
||||
# Model Representation in OpenVINO™ Runtime {#openvino_docs_OV_UG_Model_Representation}
|
||||
|
||||
In OpenVINO™ Runtime, a model is represented by the `ov::Model` class.
|
||||
@sphinxdirective
|
||||
|
||||
The `ov::Model` object stores shared pointers to `ov::op::v0::Parameter`, `ov::op::v0::Result`, and `ov::op::Sink` operations, which are inputs, outputs, and sinks of the graph.
|
||||
Sinks of the graph have no consumers and are not included in the results vector. All other operations hold each other via shared pointers, in which a child operation holds its parent via a hard link. If an operation has no consumers and is neither the `Result` nor the `Sink` operation
|
||||
whose shared pointer counter is zero, the operation will be destructed and not be accessible anymore.
|
||||
In OpenVINO™ Runtime, a model is represented by the ``:ref:`ov::Model <doxid-classov_1_1_model>``` class.
|
||||
|
||||
Each operation in `ov::Model` has the `std::shared_ptr<ov::Node>` type.
|
||||
The ``:ref:`ov::Model <doxid-classov_1_1_model>``` object stores shared pointers to ``:ref:`ov::op::v0::Parameter <doxid-classov_1_1op_1_1v0_1_1_parameter>```, ``:ref:`ov::op::v0::Result <doxid-classov_1_1op_1_1v0_1_1_result>```, and ``:ref:`ov::op::Sink <doxid-classov_1_1op_1_1_sink>``` operations, which are inputs, outputs, and sinks of the graph. Sinks of the graph have no consumers and are not included in the results vector. All other operations hold each other via shared pointers, in which a child operation holds its parent via a hard link. If an operation has no consumers and is neither the ``Result`` nor the ``Sink`` operation whose shared pointer counter is zero, the operation will be destructed and not be accessible anymore.
|
||||
|
||||
## How OpenVINO Runtime Works with Models
|
||||
Each operation in ``:ref:`ov::Model <doxid-classov_1_1_model>``` has the ``std::shared_ptr<:ref:`ov::Node <doxid-classov_1_1_node>`>`` type.
|
||||
|
||||
How OpenVINO Runtime Works with Models
|
||||
#########################################
|
||||
|
||||
OpenVINO™ Runtime enables you to use different approaches to work with model inputs/outputs:
|
||||
- The `ov::Model::inputs()`/`ov::Model::outputs()` methods are used to get vectors of all input/output ports.
|
||||
- For a model that has only one input or output, you can use the `ov::Model::input()` or `ov::Model::output()` methods without any arguments to get input or output port respectively.
|
||||
- The `ov::Model::input()` and `ov::Model::output()` methods can be used with the index of inputs or outputs from the framework model to get specific ports by index.
|
||||
- You can use the tensor name of input or output from the original framework model together with the `ov::Model::input()` or `ov::Model::output()` methods to get specific ports. It means that you do not need to have any additional mapping of names from framework to OpenVINO as it was before. OpenVINO™ Runtime allows the usage of native framework tensor names, for example:
|
||||
|
||||
@sphinxtabset
|
||||
* The ``:ref:`ov::Model::inputs() <doxid-classov_1_1_model_1ac28a4c66071e165c4f98906ab489e5d5>``` / ``:ref:`ov::Model::outputs() <doxid-classov_1_1_model_1af6e381008712ce22d6f4b93b87303dd8>``` methods are used to get vectors of all input/output ports.
|
||||
|
||||
@sphinxtab{C++}
|
||||
* For a model that has only one input or output, you can use the ``:ref:`ov::Model::input() <doxid-classov_1_1_model_1a5deeced6688795bc6cdad9ce74d972e7>``` or ``:ref:`ov::Model::output() <doxid-classov_1_1_model_1a54c76c98bc7dd8fb04e866d06134efc7>``` methods without any arguments to get input or output port respectively.
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp all_inputs_ouputs
|
||||
* The ``:ref:`ov::Model::input() <doxid-classov_1_1_model_1a5deeced6688795bc6cdad9ce74d972e7>``` and ``:ref:`ov::Model::output() <doxid-classov_1_1_model_1a54c76c98bc7dd8fb04e866d06134efc7>``` methods can be used with the index of inputs or outputs from the framework model to get specific ports by index.
|
||||
|
||||
@endsphinxtab
|
||||
* You can use the tensor name of input or output from the original framework model together with the ``:ref:`ov::Model::input() <doxid-classov_1_1_model_1a5deeced6688795bc6cdad9ce74d972e7>``` or ``:ref:`ov::Model::output() <doxid-classov_1_1_model_1a54c76c98bc7dd8fb04e866d06134efc7>``` methods to get specific ports. It means that you do not need to have any additional mapping of names from framework to OpenVINO as it was before. OpenVINO Runtime allows the usage of native framework tensor names, for example:
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. tab-set::
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py all_inputs_ouputs
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@endsphinxtab
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [all_inputs_ouputs]
|
||||
|
||||
@endsphinxtabset
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [all_inputs_ouputs]
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
For details on how to build a model in OpenVINO™ Runtime, see the :ref:`Build a Model in OpenVINO Runtime <ov_ug_build_model>` section.
|
||||
|
||||
@endsphinxdirective
|
||||
OpenVINO™ Runtime model representation uses special classes to work with model data types and shapes. The ``:ref:`ov::element::Type <doxid-classov_1_1element_1_1_type>``` is used for data types. See the section below for representation of shapes.
|
||||
|
||||
OpenVINO™ Runtime model representation uses special classes to work with model data types and shapes. The `ov::element::Type` is used for data types. See the section below for representation of shapes.
|
||||
|
||||
## Representation of Shapes
|
||||
Representation of Shapes
|
||||
###########################
|
||||
|
||||
OpenVINO™ Runtime provides two types for shape representation:
|
||||
|
||||
* `ov::Shape` - Represents static (fully defined) shapes.
|
||||
* ``:ref:`ov::Shape <doxid-classov_1_1_shape>``` - Represents static (fully defined) shapes.
|
||||
|
||||
* `ov::PartialShape` - Represents dynamic shapes. This means that the rank or some of dimensions are dynamic (dimension defines an interval or undefined).
|
||||
* ``:ref:`ov::PartialShape <doxid-classov_1_1_partial_shape>``` - Represents dynamic shapes. This means that the rank or some of dimensions are dynamic (dimension defines an interval or undefined).
|
||||
|
||||
`ov::PartialShape` can be converted to `ov::Shape` by using the `get_shape()` method if all dimensions are static; otherwise, the conversion will throw an exception. For example:
|
||||
``:ref:`ov::PartialShape <doxid-classov_1_1_partial_shape>``` can be converted to ``:ref:`ov::Shape <doxid-classov_1_1_shape>``` by using the ``get_shape()`` method if all dimensions are static; otherwise, the conversion will throw an exception. For example:
|
||||
|
||||
@sphinxtabset
|
||||
.. tab-set::
|
||||
|
||||
@sphinxtab{C++}
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:partial_shape
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:partial_shape]
|
||||
|
||||
@endsphinxtab
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [ov:partial_shape]
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py ov:partial_shape
|
||||
|
||||
@endsphinxtab
|
||||
However, in most cases, before getting static shape using the ``get_shape()`` method, you need to check if that shape is static.
|
||||
|
||||
@endsphinxtabset
|
||||
Representation of Operations
|
||||
################################
|
||||
|
||||
However, in most cases, before getting static shape using the `get_shape()` method, you need to check if that shape is static.
|
||||
The ``ov::Op`` class represents any abstract operation in the model representation. Use this class to create :doc:`custom operations <openvino_docs_Extensibility_UG_add_openvino_ops>`.
|
||||
|
||||
## Representation of Operations
|
||||
Representation of Operation Sets
|
||||
######################################
|
||||
|
||||
The `ov::Op` class represents any abstract operation in the model representation. Use this class to create [custom operations](../Extensibility_UG/add_openvino_ops.md).
|
||||
|
||||
## Representation of Operation Sets
|
||||
|
||||
An operation set (opset) is a collection of operations that can be used to construct a model. The `ov::OpSet` class provides the functionality to work with operation sets.
|
||||
For each operation set, OpenVINO™ Runtime provides a separate namespace, for example `opset8`.
|
||||
An operation set (opset) is a collection of operations that can be used to construct a model. The ``:ref:`ov::OpSet <doxid-classov_1_1_op_set>``` class provides the functionality to work with operation sets.
|
||||
For each operation set, OpenVINO™ Runtime provides a separate namespace, for example ``opset8``.
|
||||
|
||||
Each OpenVINO™ Release introduces new operations and adds them to new operation sets, within which the new operations would change the behavior of previous operations. Using operation sets helps you avoid changing your application when new operations are introduced.
|
||||
For a complete list of operation sets supported in OpenVINO™ toolkit, see the [Available Operations Sets](../ops/opset.md).
|
||||
To add the support for custom operations, see [OpenVINO Extensibility Mechanism](../Extensibility_UG/Intro.md).
|
||||
|
||||
@sphinxdirective
|
||||
For a complete list of operation sets supported in OpenVINO™ toolkit, see the :doc:`Available Operations Sets <openvino_docs_ops_opset>`.
|
||||
To add the support for custom operations, see :doc:`OpenVINO Extensibility Mechanism <openvino_docs_Extensibility_UG_Intro>`.
|
||||
|
||||
.. _ov_ug_build_model:
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
## Building a Model in OpenVINO™ Runtime
|
||||
Building a Model in OpenVINO™ Runtime
|
||||
###########################################
|
||||
|
||||
You can create a model from source. This section illustrates how to construct a model composed of operations from an available operation set.
|
||||
|
||||
Operation set `opsetX` integrates a list of pre-compiled operations that work for this purpose. In other words, `opsetX` defines a set of operations for building a graph.
|
||||
Operation set ``opsetX`` integrates a list of pre-compiled operations that work for this purpose. In other words, ``opsetX`` defines a set of operations for building a graph.
|
||||
|
||||
To build an `ov::Model` instance from `opset8` operations, include the following files:
|
||||
To build an ``:ref:`ov::Model <doxid-classov_1_1_model>``` instance from ``opset8`` operations, include the following files:
|
||||
|
||||
@sphinxtabset
|
||||
.. tab-set::
|
||||
|
||||
@sphinxtab{C++}
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:include
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:include]
|
||||
|
||||
@endsphinxtab
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [import]
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py import
|
||||
|
||||
@endsphinxtab
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
The following code demonstrates how to create a simple model:
|
||||
|
||||
@sphinxtabset
|
||||
.. tab-set::
|
||||
|
||||
@sphinxtab{C++}
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:create_simple_model
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_simple_model]
|
||||
|
||||
@endsphinxtab
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [ov:create_simple_model]
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py ov:create_simple_model
|
||||
|
||||
@endsphinxtab
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
The following code creates a model with several outputs:
|
||||
|
||||
@sphinxtabset
|
||||
.. tab-set::
|
||||
|
||||
@sphinxtab{C++}
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:create_advanced_model
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:create_advanced_model]
|
||||
|
||||
@endsphinxtab
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [ov:create_advanced_model]
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py ov:create_advanced_model
|
||||
|
||||
@endsphinxtab
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
## Model Debugging Capabilities
|
||||
Model Debugging Capabilities
|
||||
###########################################
|
||||
|
||||
OpenVINO™ provides several debug capabilities:
|
||||
- To receive additional messages about applied model modifications, rebuild the OpenVINO™ Runtime library with the `-DENABLE_OPENVINO_DEBUG=ON` option.
|
||||
- Model can be visualized to image from the xDot format:
|
||||
|
||||
@sphinxtabset
|
||||
* To receive additional messages about applied model modifications, rebuild the OpenVINO™ Runtime library with the ``-DENABLE_OPENVINO_DEBUG=ON`` option.
|
||||
|
||||
@sphinxtab{C++}
|
||||
* Model can be visualized to image from the xDot format:
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:visualize
|
||||
.. tab-set::
|
||||
|
||||
@endsphinxtab
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
@sphinxtab{Python}
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:visualize]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py ov:visualize
|
||||
|
||||
@endsphinxtab
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
|
||||
`ov::pass::VisualizeTree` can be parametrized via environment variables:
|
||||
|
||||
OV_VISUALIZE_TREE_OUTPUT_SHAPES=1 - visualize shapes
|
||||
|
||||
OV_VISUALIZE_TREE_OUTPUT_TYPES=1 - visualize types
|
||||
|
||||
OV_VISUALIZE_TREE_MIN_MAX_DENORMAL=1 - pretty denormal values
|
||||
|
||||
OV_VISUALIZE_TREE_RUNTIME_INFO=1 - print runtime information
|
||||
|
||||
OV_VISUALIZE_TREE_IO=1 - print I/O ports
|
||||
|
||||
OV_VISUALIZE_TREE_MEMBERS_NAME=1 - print member names
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [ov:visualize]
|
||||
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
`ov::pass::VisualizeTree` can be parametrized via environment variables:
|
||||
|
||||
- Also model can be serialized to IR:
|
||||
OV_VISUALIZE_TREE_OUTPUT_SHAPES=1 - visualize shapes
|
||||
|
||||
@sphinxtabset
|
||||
OV_VISUALIZE_TREE_OUTPUT_TYPES=1 - visualize types
|
||||
|
||||
@sphinxtab{C++}
|
||||
OV_VISUALIZE_TREE_MIN_MAX_DENORMAL=1 - pretty denormal values
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.cpp ov:serialize
|
||||
OV_VISUALIZE_TREE_RUNTIME_INFO=1 - print runtime information
|
||||
|
||||
@endsphinxtab
|
||||
OV_VISUALIZE_TREE_IO=1 - print I/O ports
|
||||
|
||||
@sphinxtab{Python}
|
||||
OV_VISUALIZE_TREE_MEMBERS_NAME=1 - print member names
|
||||
|
||||
@snippet docs/snippets/ov_model_snippets.py ov:serialize
|
||||
|
||||
@endsphinxtab
|
||||
* Also model can be serialized to IR:
|
||||
|
||||
## Additional Resources
|
||||
.. tab-set::
|
||||
|
||||
.. tab-item:: C++
|
||||
:sync: cpp
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
|
||||
:language: cpp
|
||||
:fragment: [ov:serialize]
|
||||
|
||||
.. tab-item:: Python
|
||||
:sync: python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_model_snippets.py
|
||||
:language: cpp
|
||||
:fragment: [ov:serialize]
|
||||
|
||||
|
||||
Additional Resources
|
||||
########################
|
||||
|
||||
* :doc:`Available Operation Sets <openvino_docs_ops_opset>`.
|
||||
* :doc:`OpenVINO™ Runtime Extensibility Developer Guide <openvino_docs_Extensibility_UG_Intro>`.
|
||||
* :doc:`Transformations Developer Guide <openvino_docs_transformations>`.
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
* [Available Operation Sets](../ops/opset.md)
|
||||
* [OpenVINO™ Runtime Extensibility Developer Guide](../Extensibility_UG/Intro.md)
|
||||
* [Transformations Developer Guide](../Extensibility_UG/ov_transformations.md).
|
||||
|
||||
Reference in New Issue
Block a user