ca6f8fc9cf
* Updated MO extension guide * Minor change and adding svg images * Added additional information about operation extractors. Fixed links and markdown issues * Added missing file with information about Caffe Python layers and image for MO transformations dependencies graph * Added section with common graph transformations attributes and diagram with anchor transformations. Added list of available front phase transformations * Added description of front-phase transformations except the scope-defined and points defined. Removed legacy document and examples for such transformations. * Added sections about node name pattern defined front phase transformations. Copy-pasted the old one for the points defined front transformation * Added description of the rest of front transformations and and all middle and back phase transformations * Refactored Legacy_Mode_for_Caffe_Custom_Layers and updated the Customize_Model_Optimizer with information about extractors order * Added TOC for the MO Dev guide document and updated SVG images with PNG ones * Fixed broken link. Removed redundant image * Fixed broken links * Added information about attributes 'run_not_recursively', 'force_clean_up' and 'force_shape_inference' of the transformation * Code review comments * Added a section about `Port`s * Extended Ports description with examples * Added information about Connections * Updated MO README.md and removed a lot of redundant and misleading information * Updates to the Customize_Model_Optimizer.md * More updates to the Customize_Model_Optimizer.md * Final updates for the Customize_Model_Optimizer.md * Fixed some broken links * More fixed links * Refactored Custom Layers Guide: removed legacy and incorrect text, added up-to-date. * Draft implementation of the Custom layer guide example for the MO part * Fixed broken links using #. Change layer->operation in extensibility documents * Updated Custom operation guide with IE part * Fixed broken links and minor updates to the Custom Operations Guide * Updating links * Layer->Operation
3.2 KiB
How to Implement Custom CPU Operations
The primary vehicle for the performance of the CPU codepath in the Inference Engine is the Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN), and new CPU kernels extend the Inference Engine plugin for the Intel MKL-DNN. Implementing the InferenceEngine::ILayerExecImpl defines a general CPU-side extension. There are no Intel MKL-DNN specifics in the way you need to implement a kernel.
Implementation Class
All custom kernels for the CPU plugin should be inherited from the InferenceEngine::ILayerExecImpl interface. Based on that, declaration of a kernel implementation class can look as follows:
@snippet template_extension/cpu_kernel.hpp cpu_implementation:header
Class Fields
The provided implementation has several fields:
addof the typeint64_tis an attribute of a custom operationinShapeof the typengraph::Shapeis an input shapeoutShapeof the typengraph::Shapeis an output shapeerrorof the typestd::stringis a field to handle errors from a constructor
Constructor of Implementation
An implementation constructor checks parameters of nGraph operation, stores needed attributes, and stores an error message in the case of an error.
@snippet template_extension/cpu_kernel.cpp cpu_implementation:ctor
getSupportedConfigurations
InferenceEngine::ILayerExecImpl::getSupportedConfigurations method returns all supported configuration formats (input/output tensor layouts) for your implementation. To specify formats of data, use InferenceEngine::TensorDesc. Refer to the Memory Primitives section for instructions on how to do it.
@snippet template_extension/cpu_kernel.cpp cpu_implementation:getSupportedConfigurations
init
InferenceEngine::ILayerExecImpl::init method gets a runtime-selected configuration from a vector that is populated from the getSupportedConfigurations method and checks the parameters:
@snippet template_extension/cpu_kernel.cpp cpu_implementation:init
execute
InferenceEngine::ILayerExecImpl::execute method accepts and processes the actual tenors as input/output blobs:
@snippet template_extension/cpu_kernel.cpp cpu_implementation:execute
Register Implementation in Extension Class
To register custom kernel implementation in the Extension class, implement the following methods:
getImplTypes
InferenceEngine::IExtension::getImplTypes returns a vector of implementation types for an operation.
@snippet template_extension/extension.cpp extension:getImplTypes
getImplementation
InferenceEngine::IExtension::getImplementation returns the kernel implementation with a specified type for an operation.
@snippet template_extension/extension.cpp extension:getImplementation
Load Extension with Executable Kernels to Plugin
Use the AddExtension method of the general plugin interface to load your primitives:
@snippet snippets/CPU_Kernel.cpp part0