IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

DOCS shift to rst transformations (#16269)

Browse Source 
* transformations to rst

* fix snippets

* fix links

* add sphinx directive

* change img path

* fix snippet path

* fix link

* fix anchor

* fix transformation image

* fix reference

* fix reference anchor

* fix matcher pass link
This commit is contained in:
Tatiana Savina 2023-03-16 13:57:18 +01:00 committed by GitHub
parent 91b9675bed
commit 8a7956e3cb
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 269 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
docs/Extensibility_UG/graph_rewrite_pass.md
View File

@ -1,28 +1,37 @@
# OpenVINO Graph Rewrite Pass {#openvino_docs_Extensibility_UG_graph_rewrite_pass}
`ov::pass::GraphRewrite` serves for running multiple matcher passes on `ov::Model` in a single graph traversal.
@sphinxdirective
``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>``` serves for running multiple matcher passes on ``:ref:`ov::Model <doxid-classov_1_1_model>``` in a single graph traversal.
Example:
@snippet template_pattern_transformation.cpp matcher_pass:graph_rewrite
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:graph_rewrite]
In addition, GraphRewrite handles nodes that were registered by MatcherPasses during their execution. This nodes will be added to the beginning of the sequence with nodes for pattern matching.
> **NOTE**: when using `ov::pass::Manager` temporary GraphRewrite is used to execute single MatcherPass.
.. note::
When using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` temporary GraphRewrite is used to execute single MatcherPass.
GraphRewrite has two algorithms for MatcherPasses execution. First algorithm is straightforward. It applies each MatcherPass in registration order to current node.
![graph_rewrite_execution]
.. image:: ./_static/images/graph_rewrite_execution.png
But it is not really efficient when you have a lot of registered passes. So first of all GraphRewrite checks that all MatcherPass patterns has type-based root node (it means that type of this node is not hidden into predicate).
And then creates map from registered MatcherPasses. That helps to avoid additional cost of applying each MatcherPass for each node.
![graph_rewrite_efficient_search]
.. image:: ./_static/images/graph_rewrite_efficient_search.png
> **NOTE**: GraphRewrite execution algorithm cannot be set manually and depends only on root nodes registered inside MatcherPasses.
.. note::
## See Also
GraphRewrite execution algorithm cannot be set manually and depends only on root nodes registered inside MatcherPasses.
* [OpenVINO™ Transformations](./ov_transformations.md)
See Also
########
* :doc:`OpenVINO™ Transformations <openvino_docs_transformations>`
@endsphinxdirective
[graph_rewrite_execution]: ./img/graph_rewrite_execution.png
[graph_rewrite_efficient_search]: ./img/graph_rewrite_efficient_search.png

141
docs/Extensibility_UG/matcher_pass.md
View File

@ -1,13 +1,22 @@
# OpenVINO Matcher Pass {#openvino_docs_Extensibility_UG_matcher_pass}
`ov::pass::MatcherPass` is used for pattern-based transformations.
@sphinxdirective
``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>``` is used for pattern-based transformations.
Template for MatcherPass transformation class
@snippet template_pattern_transformation.hpp graph_rewrite:template_transformation_hpp
@snippet template_pattern_transformation.cpp graph_rewrite:template_transformation_cpp
.. doxygensnippet:: docs/snippets/template_pattern_transformation.hpp
:language: cpp
:fragment: [graph_rewrite:template_transformation_hpp]
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [graph_rewrite:template_transformation_cpp]
To use ``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>```, you need to complete these steps:
To use `ov::pass::MatcherPass`, you need to complete these steps:
1. Create a pattern
2. Implement a callback
3. Register the pattern and Matcher
@ -15,87 +24,135 @@ To use `ov::pass::MatcherPass`, you need to complete these steps:
So let's go through each of these steps.
## Create a pattern
Create a pattern
################
Pattern is a single root `ov::Model`. But the only difference is that you do not need to create a model object, you just need to create and connect opset or special pattern operations.
Pattern is a single root ``:ref:`ov::Model <doxid-classov_1_1_model>```. But the only difference is that you do not need to create a model object, you just need to create and connect opset or special pattern operations.
Then you need to take the last created operation and put it as a root of the pattern. This root node will be used as a root node in pattern matching.
> **NOTE**: Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching.
@snippet ov_model_snippets.cpp pattern:simple_example
.. note::
Any nodes in a pattern that have no consumers and are not registered as root will not be used in pattern matching.
The `Parameter` operation in the example above has type and shape specified. These attributes are needed only to create Parameter operation class and will not be used in pattern matching.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [pattern:simple_example]
For more pattern examples, refer to the [pattern matching](#pattern_matching) section.
The ``Parameter`` operation in the example above has type and shape specified. These attributes are needed only to create Parameter operation class and will not be used in pattern matching.
## Implement callback
For more pattern examples, refer to the `pattern matching section <#pattern-matching>`__.
Implement callback
##################
Callback is an action applied to every pattern entrance. In general, callback is the lambda function that takes Matcher object with detected subgraph.
@snippet ov_model_snippets.cpp pattern:callback_example
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [pattern:callback_example]
The example above shows the callback structure and how Matcher can be used for accessing nodes detected by pattern.
Callback return value is `true` if root node was replaced and another pattern cannot be applied to the same root node; otherwise, it is `false`.
> **NOTE**: It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching.
Callback return value is ``true`` if root node was replaced and another pattern cannot be applied to the same root node; otherwise, it is ``false``.
.. note::
It is not recommended to manipulate with nodes that are under root node. This may affect GraphRewrite execution as it is expected that all nodes that come after root node in topological order are valid and can be used in pattern matching.
MatcherPass also provides functionality that allows reporting of the newly created nodes that can be used in additional pattern matching.
If MatcherPass was registered in `ov::pass::Manager` or `ov::pass::GraphRewrite`, these registered nodes will be added for additional pattern matching.
That means that matcher passes registered in `ov::pass::GraphRewrite` will be applied to these nodes.
If MatcherPass was registered in ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` or ``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>```, these registered nodes will be added for additional pattern matching.
That means that matcher passes registered in ``:ref:`ov::pass::GraphRewrite <doxid-classov_1_1pass_1_1_graph_rewrite>``` will be applied to these nodes.
The example below shows how single MatcherPass can fuse sequence of operations using the `register_new_node` method.
The example below shows how single MatcherPass can fuse sequence of operations using the ``register_new_node`` method.
@snippet template_pattern_transformation.cpp matcher_pass:relu_fusion
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:relu_fusion]
> **NOTE**: If you register multiple nodes, please add them in topological order. We do not topologically sort these nodes as it is a time-consuming operation.
.. note::
If you register multiple nodes, please add them in topological order. We do not topologically sort these nodes as it is a time-consuming operation.
## Register pattern and Matcher
Register pattern and Matcher
############################
The last step is to register Matcher and callback inside the MatcherPass pass. To do this, call the `register_matcher` method.
> **NOTE**: Only one matcher can be registered for a single MatcherPass class.
The last step is to register Matcher and callback inside the MatcherPass pass. To do this, call the ``register_matcher`` method.
.. note::
Only one matcher can be registered for a single MatcherPass class.
.. code-block:: cpp
```cpp
// Register matcher and callback
register_matcher(m, callback);
```
## Execute MatcherPass
Execute MatcherPass
###################
MatcherPass has multiple ways to be executed:
* Run on a single node - it can be useful if you want to run MatcherPass inside another transformation.
@snippet template_pattern_transformation.cpp matcher_pass:run_on_node
* Run on `ov::Model` using GraphRewrite - this approach gives ability to run MatcherPass on whole `ov::Model`. Moreover, multiple MatcherPass transformation can be registered in a single GraphRewite to be executed in a single graph traversal.
@snippet template_pattern_transformation.cpp matcher_pass:graph_rewrite
* Run on `ov::Model` using `ov::pass::Manager` - this approach helps you to register MatcherPass for execution on `ov::Model` as another transformation types.
@snippet template_pattern_transformation.cpp matcher_pass:manager
## Pattern Matching <a name="pattern_matching"></a>
* Run on a single node - it can be useful if you want to run MatcherPass inside another transformation.
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:run_on_node]
* Run on ``:ref:`ov::Model <doxid-classov_1_1_model>``` using GraphRewrite - this approach gives ability to run MatcherPass on whole ``:ref:`ov::Model <doxid-classov_1_1_model>```. Moreover, multiple MatcherPass transformation can be registered in a single GraphRewite to be executed in a single graph traversal.
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:graph_rewrite]
* Run on ``:ref:`ov::Model <doxid-classov_1_1_model>``` using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` - this approach helps you to register MatcherPass for execution on ``:ref:`ov::Model <doxid-classov_1_1_model>``` as another transformation types.
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:manager]
Pattern Matching
################
Sometimes patterns cannot be expressed via regular operations or it is too complicated.
For example, if you want to detect **Convolution->Add** sub-graph without specifying particular input type for Convolution operation or you want to create a pattern where some of operations can have different types.
And for these cases OpenVINO™ provides additional helpers to construct patterns for GraphRewrite transformations.
There are two main helpers:
1. `ov::pass::pattern::any_input` - helps to express inputs if their types are undefined.
2. `ov::pass::pattern::wrap_type<T>` - helps to express nodes of pattern without specifying node attributes.
1. ``:ref:`ov::pass::pattern::any_input <doxid-namespaceov_1_1pass_1_1pattern_1a8ed84c3eed4610f117ee10d86d500e02>``` - helps to express inputs if their types are undefined.
2. ``:ref:`ov::pass::pattern::wrap_type <doxid-namespaceov_1_1pass_1_1pattern_1adfcd6031c95d7bace5f084e2aa105af8>`<T>`` - helps to express nodes of pattern without specifying node attributes.
Let's go through the example to have better understanding of how it works:
> **NOTE**: Node attributes do not participate in pattern matching and are needed only for operations creation. Only operation types participate in pattern matching.
.. note::
Node attributes do not participate in pattern matching and are needed only for operations creation. Only operation types participate in pattern matching.
The example below shows basic usage of `ov::passpattern::any_input`.
The example below shows basic usage of ``ov::passpattern::any_input``.
Here we construct Multiply pattern with arbitrary first input and Constant as a second input.
Also as Multiply is commutative operation, it does not matter in which order we set inputs (any_input/Constant or Constant/any_input) because both cases will be matched.
@snippet ov_model_snippets.cpp pattern:label_example
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [pattern:label_example]
This example shows how we can construct a pattern when operation has arbitrary number of inputs.
@snippet ov_model_snippets.cpp pattern:concat_example
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [pattern:concat_example]
This example shows how to use predicate to construct a pattern. Also it shows how to match pattern manually on given node.
@snippet ov_model_snippets.cpp pattern:predicate_example
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [pattern:predicate_example]
> **NOTE**: Be careful with manual matching because Matcher object holds matched nodes. To clear a match, use the m->clear_state() method.
.. note::
## See Also
Be careful with manual matching because Matcher object holds matched nodes. To clear a match, use the m->clear_state() method.
* [OpenVINO™ Transformations](./ov_transformations.md)
See Also
########
* :doc:`OpenVINO™ Transformations <openvino_docs_transformations>`
@endsphinxdirective

25
docs/Extensibility_UG/model_pass.md
View File

@ -1,17 +1,26 @@
# OpenVINO Model Pass {#openvino_docs_Extensibility_UG_model_pass}
`ov::pass::ModelPass` is used for transformations that take entire `ov::Model` as an input and process it.
@sphinxdirective
``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>``` is used for transformations that take entire ``:ref:`ov::Model <doxid-classov_1_1_model>``` as an input and process it.
Template for ModelPass transformation class
@snippet template_model_transformation.hpp model_pass:template_transformation_hpp
.. doxygensnippet:: docs/snippets/template_model_transformation.hpp
:language: cpp
:fragment: [model_pass:template_transformation_hpp]
@snippet template_model_transformation.cpp model_pass:template_transformation_cpp
.. doxygensnippet:: docs/snippets/template_model_transformation.cpp
:language: cpp
:fragment: [model_pass:template_transformation_cpp]
Using `ov::pass::ModelPass`, you need to override the `run_on_model` method where you will write the transformation code.
Return value is `true` if the original model has changed during transformation (new operation was added, or operations replacement was made, or node attributes were changed); otherwise, it is `false`.
Also `ov::pass::ModelPass` based transformations can be executed via `ov::pass::Manager`.
Using ``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>```, you need to override the ``run_on_model`` method where you will write the transformation code.
Return value is ``true`` if the original model has changed during transformation (new operation was added, or operations replacement was made, or node attributes were changed); otherwise, it is ``false``.
Also ``:ref:`ov::pass::ModelPass <doxid-classov_1_1pass_1_1_model_pass>``` based transformations can be executed via ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>```.
## See Also
See Also
########
* [OpenVINO™ Transformations](./ov_transformations.md)
* :doc:`OpenVINO™ Transformations <openvino_docs_transformations>`
@endsphinxdirective

186
docs/Extensibility_UG/ov_transformations.md
View File

@ -10,164 +10,208 @@
openvino_docs_Extensibility_UG_matcher_pass
openvino_docs_Extensibility_UG_graph_rewrite_pass
@endsphinxdirective
OpenVINO Transformation mechanism allows to develop transformation passes to modify `ov::Model`. You can use this mechanism to apply additional optimizations to the original Model or transform unsupported subgraphs and operations to new operations which are supported by the plugin.
OpenVINO Transformation mechanism allows to develop transformation passes to modify ``:ref:`ov::Model <doxid-classov_1_1_model>```. You can use this mechanism to apply additional optimizations to the original Model or transform unsupported subgraphs and operations to new operations which are supported by the plugin.
This guide contains all necessary information that you need to start implementing OpenVINO™ transformations.
## Working with Model
Working with Model
##################
Before the moving to transformation part it is needed to say several words about functions which allow to modify `ov::Model`.
This chapter extends the [model representation guide](../OV_Runtime_UG/model_representation.md) and shows an API that allows us to manipulate with `ov::Model`.
Before the moving to transformation part it is needed to say several words about functions which allow to modify ``:ref:`ov::Model <doxid-classov_1_1_model>```.
This chapter extends the :doc:`model representation guide <openvino_docs_OV_UG_Model_Representation>` and shows an API that allows us to manipulate with ``:ref:`ov::Model <doxid-classov_1_1_model>```.
### Working with node input and output ports
Working with node input and output ports
++++++++++++++++++++++++++++++++++++++++
First of all let's talk about `ov::Node` input/output ports. Each OpenVINO™ operation has input and output ports except cases when operation has `Parameter` or `Constant` type.
First of all let's talk about ``:ref:`ov::Node <doxid-classov_1_1_node>``` input/output ports. Each OpenVINO™ operation has input and output ports except cases when operation has ``Parameter`` or ``Constant`` type.
Every port belongs to its node, so using a port we can access parent node, get shape and type for particular input/output, get all consumers in case of output port, and get producer node in case of input port.
With output port we can set inputs for newly created operations.
Lets look at the code example.
@snippet ov_model_snippets.cpp ov:ports_example
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:ports_example]
### Node replacement
Node replacement
++++++++++++++++
OpenVINO™ provides two ways for node replacement: via OpenVINO™ helper function and directly via port methods. We are going to review both of them.
Let's start with OpenVINO™ helper functions. The most popular function is `ov::replace_node(old_node, new_node)`.
Let's start with OpenVINO™ helper functions. The most popular function is ``ov::replace_node(old_node, new_node)``.
We will review real replacement case where Negative operation is replaced with Multiply.
![ngraph_replace_node]
.. image:: ./_static/images/ngraph_replace_node.png
@snippet ov_model_snippets.cpp ov:replace_node
`ov::replace_node` has a constraint that number of output ports for both of ops must be the same; otherwise, it raises an exception.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:replace_node]
``:ref:`ov::replace_node <doxid-namespaceov_1a75d84ee654edb73fe4fb18936a5dca6d>``` has a constraint that number of output ports for both of ops must be the same; otherwise, it raises an exception.
The alternative way to do the same replacement is the following:
@snippet ov_model_snippets.cpp ov:manual_replace
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:manual_replace]
Another transformation example is insertion.
![ngraph_insert_node]
.. image:: ./_static/images/ngraph_insert_node.png
@snippet ov_model_snippets.cpp ov:insert_node
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:insert_node]
The alternative way to the insert operation is to make a node copy and use `ov::replace_node()`:
The alternative way to the insert operation is to make a node copy and use ``:ref:`ov::replace_node() <doxid-namespaceov_1a75d84ee654edb73fe4fb18936a5dca6d>```:
@snippet ov_model_snippets.cpp ov:insert_node_with_copy
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:insert_node_with_copy]
### Node elimination
Node elimination
++++++++++++++++
Another type of node replacement is its elimination.
To eliminate operation, OpenVINO™ has special method that considers all limitations related to OpenVINO™ Runtime.
@snippet ov_model_snippets.cpp ov:eliminate_node
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:eliminate_node]
`ov::replace_output_update_name()` in case of successful replacement it automatically preserves friendly name and runtime info.
``:ref:`ov::replace_output_update_name() <doxid-namespaceov_1a75ba2120e573883bd96bb19c887c6a1d>``` in case of successful replacement it automatically preserves friendly name and runtime info.
## Transformations types <a name="transformations-types"></a>
.. _transformations_types:
Transformations types
#####################
OpenVINO™ Runtime has three main transformation types:
* [Model pass](./model_pass.md) - straightforward way to work with `ov::Model` directly
* [Matcher pass](./matcher_pass.md) - pattern-based transformation approach
* [Graph rewrite pass](./graph_rewrite_pass.md) - container for matcher passes needed for efficient execution
* :doc:`Model pass <openvino_docs_Extensibility_UG_model_pass>` - straightforward way to work with ``:ref:`ov::Model <doxid-classov_1_1_model>``` directly
* :doc:`Matcher pass <openvino_docs_Extensibility_UG_matcher_pass>` - pattern-based transformation approach
* :doc:`Graph rewrite pass <openvino_docs_Extensibility_UG_graph_rewrite_pass>` - container for matcher passes needed for efficient execution
![transformations_structure]
.. image:: ./_static/images/transformations_structure.png
## Transformation conditional compilation
Transformation conditional compilation
######################################
Transformation library has two internal macros to support conditional compilation feature.
* `MATCHER_SCOPE(region)` - allows to disable the MatcherPass if matcher isn't used. The region name should be unique. This macro creates a local variable `matcher_name` which you should use as a matcher name.
* `RUN_ON_MODEL_SCOPE(region)` - allows to disable run_on_model pass if it isn't used. The region name should be unique.
* ``:ref:`MATCHER_SCOPE(region) <doxid-conditional__compilation_2include_2openvino_2cc_2pass_2itt_8hpp_1a3d1377542bcf3e305c33a1b683cc77df>``` - allows to disable the MatcherPass if matcher isn't used. The region name should be unique. This macro creates a local variable ``matcher_name`` which you should use as a matcher name.
* ``:ref:`RUN_ON_MODEL_SCOPE(region) <doxid-conditional__compilation_2include_2openvino_2cc_2pass_2itt_8hpp_1ab308561b849d47b9c820506ec73c4a30>``` - allows to disable run_on_model pass if it isn't used. The region name should be unique.
## Transformation writing essentials <a name="transformation_writing_essentials"></a>
.. _transformation_writing_essentials:
Transformation writing essentials
#################################
When developing a transformation, you need to follow these transformation rules:
### 1. Friendly Names
1. Friendly Names
+++++++++++++++++
Each `ov::Node` has an unique name and a friendly name. In transformations we care only about friendly name because it represents the name from the model.
Each ``:ref:`ov::Node <doxid-classov_1_1_node>``` has an unique name and a friendly name. In transformations we care only about friendly name because it represents the name from the model.
To avoid losing friendly name when replacing node with other node or subgraph, set the original friendly name to the latest node in replacing subgraph. See the example below.
@snippet ov_model_snippets.cpp ov:replace_friendly_name
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:replace_friendly_name]
In more advanced cases, when replaced operation has several outputs and we add additional consumers to its outputs, we make a decision how to set friendly name by arrangement.
### 2. Runtime Info
2. Runtime Info
+++++++++++++++
Runtime info is a map `std::map<std::string, ov::Any>` located inside `ov::Node` class. It represents additional attributes in `ov::Node`.
These attributes can be set by users or by plugins and when executing transformation that changes `ov::Model` we need to preserve these attributes as they will not be automatically propagated.
Runtime info is a map ``std::map<std::string, :ref:`ov::Any <doxid-classov_1_1_any>`>`` located inside ``:ref:`ov::Node <doxid-classov_1_1_node>``` class. It represents additional attributes in ``:ref:`ov::Node <doxid-classov_1_1_node>```.
These attributes can be set by users or by plugins and when executing transformation that changes ``:ref:`ov::Model <doxid-classov_1_1_model>``` we need to preserve these attributes as they will not be automatically propagated.
In most cases, transformations have the following types: 1:1 (replace node with another node), 1:N (replace node with a sub-graph), N:1 (fuse sub-graph into a single node), N:M (any other transformation).
Currently, there is no mechanism that automatically detects transformation types, so we need to propagate this runtime information manually. See the examples below.
@snippet ov_model_snippets.cpp ov:copy_runtime_info
When transformation has multiple fusions or decompositions, `ov::copy_runtime_info` must be called multiple times for each case.
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:copy_runtime_info]
> **NOTE**: `copy_runtime_info` removes `rt_info` from destination nodes. If you want to keep it, you need to specify them in source nodes like this: `copy_runtime_info({a, b, c}, {a, b})`
When transformation has multiple fusions or decompositions, ``:ref:`ov::copy_runtime_info <doxid-namespaceov_1a3bb5969a95703b4b4fd77f6f58837207>``` must be called multiple times for each case.
### 3. Constant Folding
.. note:: ``copy_runtime_info`` removes ``rt_info`` from destination nodes. If you want to keep it, you need to specify them in source nodes like this: ``copy_runtime_info({a, b, c}, {a, b})``
If your transformation inserts constant sub-graphs that need to be folded, do not forget to use `ov::pass::ConstantFolding()` after your transformation or call constant folding directly for operation.
3. Constant Folding
+++++++++++++++++++
If your transformation inserts constant sub-graphs that need to be folded, do not forget to use ``:ref:`ov::pass::ConstantFolding() <doxid-classov_1_1pass_1_1_constant_folding>``` after your transformation or call constant folding directly for operation.
The example below shows how constant subgraph can be constructed.
@snippet ov_model_snippets.cpp ov:constant_subgraph
.. doxygensnippet:: docs/snippets/ov_model_snippets.cpp
:language: cpp
:fragment: [ov:constant_subgraph]
Manual constant folding is more preferable than `ov::pass::ConstantFolding()` because it is much faster.
Manual constant folding is more preferable than ``:ref:`ov::pass::ConstantFolding() <doxid-classov_1_1pass_1_1_constant_folding>``` because it is much faster.
Below you can find an example of manual constant folding:
@snippet template_pattern_transformation.cpp manual_constant_folding
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [manual_constant_folding]
## Common mistakes in transformations <a name="common_mistakes"></a>
.. _common_mistakes:
Common mistakes in transformations
##################################
In transformation development process:
* Do not use deprecated OpenVINO™ API. Deprecated methods has the `OPENVINO_DEPRECATED` macros in its definition.
* Do not pass `shared_ptr<Node>` as an input for other node if type of node is unknown or it has multiple outputs. Use explicit output port.
* If you replace node with another node that produces different shape, remember that new shape will not be propagated until the first `validate_nodes_and_infer_types` call for `ov::Model`. If you are using `ov::pass::Manager`, it will automatically call this method after each transformation execution.
* Do not forget to call the `ov::pass::ConstantFolding` pass if your transformation creates constant subgraphs.
* Do not use deprecated OpenVINO™ API. Deprecated methods has the ``OPENVINO_DEPRECATED`` macros in its definition.
* Do not pass ``shared_ptr<Node>`` as an input for other node if type of node is unknown or it has multiple outputs. Use explicit output port.
* If you replace node with another node that produces different shape, remember that new shape will not be propagated until the first ``validate_nodes_and_infer_types`` call for ``:ref:`ov::Model <doxid-classov_1_1_model>```. If you are using ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>```, it will automatically call this method after each transformation execution.
* Do not forget to call the ``:ref:`ov::pass::ConstantFolding <doxid-classov_1_1pass_1_1_constant_folding>``` pass if your transformation creates constant subgraphs.
* Use latest OpSet if you are not developing downgrade transformation pass.
* When developing a callback for `ov::pass::MatcherPass`, do not change nodes that come after the root node in topological order.
* When developing a callback for ``:ref:`ov::pass::MatcherPass <doxid-classov_1_1pass_1_1_matcher_pass>```, do not change nodes that come after the root node in topological order.
## Using pass manager <a name="using_pass_manager"></a>
.. _using_pass_manager:
`ov::pass::Manager` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations.
It can register and apply any [transformation pass](#transformations-types) on model.
In addition, `ov::pass::Manager` has extended debug capabilities (find more information in the [how to debug transformations](#how-to-debug-transformations) section).
Using pass manager
##################
The example below shows basic usage of `ov::pass::Manager`
``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` is a container class that can store the list of transformations and execute them. The main idea of this class is to have high-level representation for grouped list of transformations.
It can register and apply any `transformation pass <#transformations_types>`__ on model.
In addition, ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>``` has extended debug capabilities (find more information in the `how to debug transformations <#how_to_debug_transformations>`__ section).
@snippet template_pattern_transformation.cpp matcher_pass:manager3
The example below shows basic usage of ``:ref:`ov::pass::Manager <doxid-classov_1_1pass_1_1_manager>```
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:manager3]
Another example shows how multiple matcher passes can be united into single GraphRewrite.
@snippet template_pattern_transformation.cpp matcher_pass:manager2
.. doxygensnippet:: docs/snippets/template_pattern_transformation.cpp
:language: cpp
:fragment: [matcher_pass:manager2]
## How to debug transformations <a name="how-to-debug-transformations"></a>
.. _how_to_debug_transformations:
If you are using `ngraph::pass::Manager` to run sequence of transformations, you can get additional debug capabilities by using the following environment variables:
How to debug transformations
############################
If you are using ``ngraph::pass::Manager`` to run sequence of transformations, you can get additional debug capabilities by using the following environment variables:
.. code-block:: cpp
```
OV_PROFILE_PASS_ENABLE=1 - enables performance measurement for each transformation and prints execution status
OV_ENABLE_VISUALIZE_TRACING=1 - enables visualization after each transformation. By default, it saves dot and svg files.
```
> **NOTE**: Make sure that you have dot installed on your machine; otherwise, it will silently save only dot file without svg file.
## See Also
.. note:: Make sure that you have dot installed on your machine; otherwise, it will silently save only dot file without svg file.
* [OpenVINO™ Model Representation](../OV_Runtime_UG/model_representation.md)
* [OpenVINO™ Extensions](./Intro.md)
See Also
########
[ngraph_replace_node]: ./img/ngraph_replace_node.png
[ngraph_insert_node]: ./img/ngraph_insert_node.png
[transformations_structure]: ./img/transformations_structure.png
[register_new_node]: ./img/register_new_node.png
* :doc:`OpenVINO™ Model Representation <openvino_docs_OV_UG_Model_Representation>`
* :doc:`OpenVINO™ Extensions <openvino_docs_Extensibility_UG_Intro>`
@endsphinxdirective

3
docs/_static/images/graph_rewrite_efficient_search.png vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:05eb8600d2c905975674f3a0a5dc676107d22f65f2a1f78ee1cfabc1771721ea
size 41307

3
docs/_static/images/graph_rewrite_execution.png vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:17cd470c6d04d7aabbdb4a08e31f9c97eab960cf7ef5bbd3a541df92db38f26b
size 40458

3
docs/_static/images/ngraph_insert_node.png vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0838f4046b7f135a2dcd251a0bac9ae445801cf2e23535ec085bb2da2818b352
size 23310

3
docs/_static/images/ngraph_replace_node.png vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4793780a48ad63936a046c7c1b87b16d3867676191e10794475630e70169cfa2
size 44911

3
docs/_static/images/transformations_structure.png vendored Normal file
View File

@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b206c602626f17ba5787810b9a28f9cde511448c3e63a5c7ba976cee7868bdb
size 14907