Changed location of extensibility guide (#10433)
* Changed location of extensibility guide * Removed hardware kernels legacy documentation * Changed all extension guild to new API * Removed Custom_Layers_Guide * Fixed build * Fixed some moments * Update docs/Extensibility_UG/Intro.md * Fixed build * Added more examples * Fixed typo * Fixed comments * Extend library topic * Fixed typo
This commit is contained in:
Ilya Churaev authored and 
GitHub committed
1 parent
a2f9963045
commit
1c18733ade
30 files changed
+268
-1632
No files matched your search
@@ -0,0 +1,114 @@
|
||||
# OpenVINO Extensibility Mechanism {#openvino_docs_Extensibility_UG_Intro}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
openvino_docs_Extensibility_UG_add_openvino_ops
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
The Intel® Distribution of OpenVINO™ toolkit supports neural network models trained with multiple frameworks including
|
||||
TensorFlow, Caffe, MXNet, Kaldi, PaddlePaddle, and ONNX. The list of supported operations (layers) is different for
|
||||
each of the supported frameworks. To see the operations supported by your framework, refer to
|
||||
[Supported Framework Operations](../MO_DG/prepare_model/Supported_Frameworks_Layers.md).
|
||||
|
||||
Custom operations, that is those not included in the list, are not recognized by OpenVINO™ out-of-the-box. Therefore, creating Intermediate Representation (IR) for a model using them requires additional steps. This guide illustrates the workflow for running inference on topologies featuring custom operations, allowing you to plug in your own implementation for existing or completely new operations.
|
||||
|
||||
If your model contains operations not normally supported by OpenVINO™, the OpenVINO™ Extensibility API lets you add support for those custom operations and use one implementation for Model Optimizer and OpenVINO™ Runtime.
|
||||
|
||||
There are two steps to support inference of a model with custom operation(s):
|
||||
1. Add support for a [custom operation in the Model Optimizer](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) so
|
||||
the Model Optimizer can generate the IR with the operation.
|
||||
2. Create a custom operation in it as described in the [Custom Operation](add_openvino_ops.md).
|
||||
|
||||
## OpenVINO™ Extensions
|
||||
|
||||
An OpenVINO™ provides extensions for:
|
||||
|
||||
* [Custom OpenVINO™ Operation](add_openvino_ops.md):
|
||||
- Enables the creation of unsupported operations
|
||||
- Enables the use of `ov::Core::read_model` to read models with unsupported operations
|
||||
- Provides a shape inference mechanism for custom operations
|
||||
- Provides an evaluate method which allow to support the operation on CPU or perform constant folding
|
||||
|
||||
> **NOTE**: This documentation is written based on the [Template extension](https://github.com/openvinotoolkit/openvino/tree/master/docs/template_extension/new), which demonstrates extension development details. You can review the complete code, which is fully compilable and up-to-date, to see how it works.
|
||||
|
||||
## Load extensions to OpenVINO™ Runtime
|
||||
|
||||
To load the extensions to the `ov::Core` object, use the `ov::Core::add_extension` method, this method allows to load library with extensions or extensions from the code.
|
||||
|
||||
### Load extensions to core
|
||||
|
||||
Extensions can be loaded from code with `ov::Core::add_extension` method:
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: add_extension
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: add_extension
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
### Create library with extensions
|
||||
|
||||
You need to create extension library in following cases:
|
||||
- Load extensions to Model Optimizer
|
||||
- Load extensions to Python application
|
||||
|
||||
If you want to create an extension library, for example in order to load these extensions to the Model Optimizer, you need to do next steps:
|
||||
Create an entry point for extension library. OpenVINO™ provides an `OPENVINO_CREATE_EXTENSIONS()` macro, which allows to define an entry point to a library with OpenVINO™ Extensions.
|
||||
This macro should have a vector of all OpenVINO™ Extensions as an argument.
|
||||
|
||||
Based on that, the declaration of an extension class can look as follows:
|
||||
|
||||
@snippet template_extension/new/ov_extension.cpp ov_extension:entry_point
|
||||
|
||||
To configure the build of your extension library, use the following CMake script:
|
||||
|
||||
@snippet template_extension/new/CMakeLists.txt cmake:extension
|
||||
|
||||
This CMake script finds the OpenVINO™ using the `find_package` CMake command.
|
||||
|
||||
To build the extension library, run the commands below:
|
||||
|
||||
```sh
|
||||
$ cd docs/template_extension/new
|
||||
$ mkdir build
|
||||
$ cd build
|
||||
$ cmake -DOpenVINO_DIR=<OpenVINO_DIR> ../
|
||||
$ cmake --build .
|
||||
```
|
||||
|
||||
After the build you can use path to your extension library to load your extensions to OpenVINO™ Runtime:
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. tab:: C++
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.cpp
|
||||
:language: cpp
|
||||
:fragment: add_extension_lib
|
||||
|
||||
.. tab:: Python
|
||||
|
||||
.. doxygensnippet:: docs/snippets/ov_extensions.py
|
||||
:language: python
|
||||
:fragment: add_extension_lib
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
## See Also
|
||||
|
||||
* [Using Inference Engine Samples](../OV_Runtime_UG/Samples_Overview.md)
|
||||
* [Hello Shape Infer SSD sample](../../samples/cpp/hello_reshape_ssd/README.md)
|
||||
@@ -0,0 +1,62 @@
|
||||
# Custom OpenVINO™ Operations {#openvino_docs_Extensibility_UG_add_openvino_ops}
|
||||
|
||||
OpenVINO™ Extension API allows you to register custom operations to support models with operations which OpenVINO™ does not support out-of-the-box.
|
||||
|
||||
## Operation Class
|
||||
|
||||
To add your custom operation, create a new class that extends `ov::Op`, which is in turn derived from `ov::Node`, the base class for all graph operations in OpenVINO™. To add `ov::Op` please include next file:
|
||||
|
||||
@snippet template_extension/new/identity.hpp op:common_include
|
||||
|
||||
Follow the steps below to add a custom operation:
|
||||
|
||||
1. Add the `OPENVINO_OP` macro which defines a `NodeTypeInfo` object that identifies the type of the operation to the graph users and helps with dynamic type resolution. The type info of an operation currently consists of a string operation identifier and a string for operation version.
|
||||
|
||||
2. Implement default constructor and constructors that optionally take the operation inputs and attributes as parameters.
|
||||
|
||||
3. Override the shape inference method `validate_and_infer_types`. This method is called multiple times during graph manipulations to determine the shapes and element types of the operations outputs. To access the input shapes and input element types, use the `get_input_partial_shape()` and `get_input_element_type()` methods of `ov::Node`. Set the inferred shape and element type of the output using `set_output_type`.
|
||||
|
||||
4. Override the `clone_with_new_inputs` method, which enables graph manipulation routines to create copies of this operation and connect it to different nodes during optimization.
|
||||
|
||||
5. Override the `visit_attributes` method, which enables serialization and deserialization of operation attributes. An `AttributeVisitor` is passed to the method, and the implementation is expected to walk over all the attributes in the op using the type-aware `on_attribute` helper. Helpers are already implemented for standard C++ types like `int64_t`, `float`, `bool`, `vector`, and for existing OpenVINO defined types.
|
||||
|
||||
6. Override `evaluate`, which is an optional method that enables fallback of some devices to this implementation and the application of constant folding if there is a custom operation on the constant branch. If your operation contains `evaluate` method you also need to override the `has_evaluate` method, this method allow to get information about availability of `evaluate` method for the operation.
|
||||
|
||||
7. Add the `OPENVINO_FRAMEWORK_MAP` macro if you want to map custom operation to framework operation with the same name. It is an optional macro which can be used for one to one mapping. In order to use this macro please include frontend specific headers:
|
||||
@snippet template_extension/new/identity.hpp op:frontend_include
|
||||
|
||||
Based on that, declaration of an operation class can look as follows:
|
||||
|
||||
@snippet template_extension/new/identity.hpp op:header
|
||||
|
||||
### Operation Constructors
|
||||
|
||||
OpenVINO™ operation contains two constructors:
|
||||
* Default constructor, which enables you to create an operation without attributes
|
||||
* Constructor that creates and validates an operation with specified inputs and attributes
|
||||
|
||||
@snippet template_extension/new/identity.cpp op:ctor
|
||||
|
||||
### `validate_and_infer_types()`
|
||||
|
||||
`ov::Node::validate_and_infer_types` method validates operation attributes and calculates output shapes using attributes of the operation.
|
||||
|
||||
@snippet template_extension/new/identity.cpp op:validate
|
||||
|
||||
### `clone_with_new_inputs()`
|
||||
|
||||
`ov::Node::clone_with_new_inputs` method creates a copy of the operation with new inputs.
|
||||
|
||||
@snippet template_extension/new/identity.cpp op:copy
|
||||
|
||||
### `visit_attributes()`
|
||||
|
||||
`ov::Node::visit_attributes` method enables you to visit all operation attributes.
|
||||
|
||||
@snippet template_extension/new/identity.cpp op:visit_attributes
|
||||
|
||||
### `evaluate()` and `has_evaluate()`
|
||||
|
||||
`ov::Node::evaluate` method enables you to apply constant folding to an operation.
|
||||
|
||||
@snippet template_extension/new/identity.cpp op:evaluate
|
||||
Reference in new issue
Block a user