dbad8809bf
* Release mo dev guide refactoring (#3266) * Updated MO extension guide * Minor change and adding svg images * Added additional information about operation extractors. Fixed links and markdown issues * Added missing file with information about Caffe Python layers and image for MO transformations dependencies graph * Added section with common graph transformations attributes and diagram with anchor transformations. Added list of available front phase transformations * Added description of front-phase transformations except the scope-defined and points defined. Removed legacy document and examples for such transformations. * Added sections about node name pattern defined front phase transformations. Copy-pasted the old one for the points defined front transformation * Added description of the rest of front transformations and and all middle and back phase transformations * Refactored Legacy_Mode_for_Caffe_Custom_Layers and updated the Customize_Model_Optimizer with information about extractors order * Added TOC for the MO Dev guide document and updated SVG images with PNG ones * Fixed broken link. Removed redundant image * Fixed broken links * Added information about attributes 'run_not_recursively', 'force_clean_up' and 'force_shape_inference' of the transformation * Code review comments * Added a section about `Port`s * Extended Ports description with examples * Added information about Connections * Updated MO README.md and removed a lot of redundant and misleading information * Updates to the Customize_Model_Optimizer.md * More updates to the Customize_Model_Optimizer.md * Final updates for the Customize_Model_Optimizer.md * Fixed some broken links * More fixed links * Refactored Custom Layers Guide: removed legacy and incorrect text, added up-to-date. * Draft implementation of the Custom layer guide example for the MO part * Fixed broken links using #. Change layer->operation in extensibility documents * Updated Custom operation guide with IE part * Fixed broken links and minor updates to the Custom Operations Guide * Updating links * Layer->Operation * Moved FFTOp implementation to the template extension * Update the CMake for template_extension to build the FFT op conditionally * Fixed template extension compilation * Fixed CMake for template extension * Fixed broken snippet * Added mri_demo script and updated documentation * One more compilation error fix * Added missing header for a demo file * Added reference to OpenCV * Fixed unit test for the template extension * Fixed typos in the template extension * Fixed compilation of template extension for case when ONNX importer is disabled Co-authored-by: Alexander Zhogov <alexander.zhogov@intel.com>
2.6 KiB
Inference Engine Extensibility Mechanism
Inference Engine Extensibility API allows to add support of custom operations to the Inference Engine.
Extension should contain operation sets with custom operations and execution kernels for custom operations.
Physically, an extension library can be represented as a dynamic library exporting the single CreateExtension function
that allows to create a new extension instance.
Extensibility library can be loaded to the InferenceEngine::Core object using the
InferenceEngine::Core::AddExtension method.
Inference Engine Extension Library
Inference Engine Extension dynamic library contains several components:
- Extension Library:
- Contains custom operation sets
- Provides CPU implementations for custom operations
- Custom nGraph Operation:
- Allows to use
InferenceEngine::Core::ReadNetworkto read Intermediate Representation (IR) with unsupported operations - Allows to create
ngraph::Functionwith unsupported operations - Provides shape inference mechanism for custom operations
- Allows to use
Note
: This documentation is written based on the
Template extension, which demonstrates extension development details. Find the complete code of theTemplate extension, which is fully compilable and up-to-date, at<dldt source tree>/docs/template_extension.
Execution Kernels
The Inference Engine workflow involves the creation of custom kernels and either custom or existing operations.
An Operation is a network building block implemented in the training framework, for example, Convolution in Caffe*.
A Kernel is defined as the corresponding implementation in the Inference Engine.
Refer to the Model Optimizer Extensibility for details on how a mapping between framework operations and Inference Engine kernels is registered.
In short, you can plug your own kernel implementations into the Inference Engine and map them to the operations in the original framework.
The following pages describe how to integrate custom kernels into the Inference Engine:
- Introduction to development of custom CPU kernels
- Introduction to development of custom GPU kernels
- Introduction to development of custom VPU kernels