Feature/azaytsev/from 2021 4 (#9247)

* Added info on DockerHub CI Framework * Feature/azaytsev/change layout (#3295) * Changes according to feedback comments * Replaced @ref's with html links * Fixed links, added a title page for installing from repos and images, fixed formatting issues * Added links * minor fix * Added DL Streamer to the list of components installed by default * Link fixes * Link fixes * ovms doc fix (#2988) * added OpenVINO Model Server * ovms doc fixes Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com> * Updated openvino_docs.xml * Updated the link to software license agreements * Revert "Updated the link to software license agreements" This reverts commit 706dac500e. * Docs to Sphinx (#8151) * docs to sphinx * Update GPU.md * Update CPU.md * Update AUTO.md * Update performance_int8_vs_fp32.md * update * update md * updates * disable doc ci * disable ci * fix index.rst Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> # Conflicts: # .gitignore # docs/CMakeLists.txt # docs/IE_DG/Deep_Learning_Inference_Engine_DevGuide.md # docs/IE_DG/Extensibility_DG/Custom_ONNX_Ops.md # docs/IE_DG/Extensibility_DG/VPU_Kernel.md # docs/IE_DG/InferenceEngine_QueryAPI.md # docs/IE_DG/Int8Inference.md # docs/IE_DG/Integrate_with_customer_application_new_API.md # docs/IE_DG/Model_caching_overview.md # docs/IE_DG/supported_plugins/GPU_RemoteBlob_API.md # docs/IE_DG/supported_plugins/HETERO.md # docs/IE_DG/supported_plugins/MULTI.md # docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md # docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md # docs/MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md # docs/MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md # docs/MO_DG/prepare_model/convert_model/Converting_Model.md # docs/MO_DG/prepare_model/convert_model/Converting_Model_General.md # docs/MO_DG/prepare_model/convert_model/Cutting_Model.md # docs/MO_DG/prepare_model/convert_model/pytorch_specific/Convert_RNNT.md # docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_EfficientDet_Models.md # docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_WideAndDeep_Family_Models.md # docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_YOLO_From_Tensorflow.md # docs/doxygen/Doxyfile.config # docs/doxygen/ie_docs.xml # docs/doxygen/ie_plugin_api.config # docs/doxygen/ngraph_cpp_api.config # docs/doxygen/openvino_docs.xml # docs/get_started/get_started_macos.md # docs/get_started/get_started_raspbian.md # docs/get_started/get_started_windows.md # docs/img/cpu_int8_flow.png # docs/index.md # docs/install_guides/VisionAcceleratorFPGA_Configure.md # docs/install_guides/VisionAcceleratorFPGA_Configure_Windows.md # docs/install_guides/deployment-manager-tool.md # docs/install_guides/installing-openvino-linux.md # docs/install_guides/installing-openvino-macos.md # docs/install_guides/installing-openvino-windows.md # docs/optimization_guide/dldt_optimization_guide.md # inference-engine/ie_bridges/c/include/c_api/ie_c_api.h # inference-engine/ie_bridges/python/docs/api_overview.md # inference-engine/ie_bridges/python/sample/ngraph_function_creation_sample/README.md # inference-engine/ie_bridges/python/sample/speech_sample/README.md # inference-engine/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx # inference-engine/include/ie_api.h # inference-engine/include/ie_core.hpp # inference-engine/include/ie_version.hpp # inference-engine/samples/benchmark_app/README.md # inference-engine/samples/speech_sample/README.md # inference-engine/src/plugin_api/exec_graph_info.hpp # inference-engine/src/plugin_api/file_utils.h # inference-engine/src/transformations/include/transformations_visibility.hpp # inference-engine/tools/benchmark_tool/README.md # ngraph/core/include/ngraph/ngraph.hpp # ngraph/frontend/onnx_common/include/onnx_common/parser.hpp # ngraph/python/src/ngraph/utils/node_factory.py # openvino/itt/include/openvino/itt.hpp # thirdparty/ade # tools/benchmark/README.md * Cherry-picked remove font-family (#8211) * Cherry-picked: Update get_started_scripts.md (#8338) * doc updates (#8268) * Various doc changes * theme changes * remove font-family (#8211) * fix css * Update uninstalling-openvino.md * fix css * fix * Fixes for Installation Guides Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> Co-authored-by: kblaszczak-intel <karol.blaszczak@intel.com> # Conflicts: # docs/IE_DG/Bfloat16Inference.md # docs/IE_DG/InferenceEngine_QueryAPI.md # docs/IE_DG/OnnxImporterTutorial.md # docs/IE_DG/supported_plugins/AUTO.md # docs/IE_DG/supported_plugins/HETERO.md # docs/IE_DG/supported_plugins/MULTI.md # docs/MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md # docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_YOLO_From_Tensorflow.md # docs/install_guides/installing-openvino-macos.md # docs/install_guides/installing-openvino-windows.md # docs/ops/opset.md # inference-engine/samples/benchmark_app/README.md # inference-engine/tools/benchmark_tool/README.md # thirdparty/ade * Cherry-picked: doc script changes (#8568) * fix openvino-sphinx-theme * add linkcheck target * fix * change version * add doxygen-xfail.txt * fix * AA * fix * fix * fix * fix * fix # Conflicts: # thirdparty/ade * Cherry-pick: Feature/azaytsev/doc updates gna 2021 4 2 (#8567) * Various doc changes * Reformatted C++/Pythob sections. Updated with info from PR8490 * additional fix * Gemini Lake replaced with Elkhart Lake * Fixed links in IGs, Added 12th Gen # Conflicts: # docs/IE_DG/supported_plugins/GNA.md # thirdparty/ade * Cherry-pick: Feature/azaytsev/doc fixes (#8897) * Various doc changes * Removed the empty Learning path topic * Restored the Gemini Lake CPIU list # Conflicts: # docs/IE_DG/supported_plugins/GNA.md # thirdparty/ade * Cherry-pick: sphinx copybutton doxyrest code blocks (#8992) # Conflicts: # thirdparty/ade * Cherry-pick: iframe video enable fullscreen (#9041) # Conflicts: # thirdparty/ade * Cherry-pick: fix untitled titles (#9213) # Conflicts: # thirdparty/ade * Cherry-pick: perf bench graph animation (#9045) * animation * fix # Conflicts: # thirdparty/ade * Cherry-pick: doc pytest (#8888) * docs pytest * fixes # Conflicts: # docs/doxygen/doxygen-ignore.txt # docs/scripts/ie_docs.xml # thirdparty/ade * Cherry-pick: restore deleted files (#9215) * Added new operations to the doc structure (from removed ie_docs.xml) * Additional fixes * Update docs/IE_DG/InferenceEngine_QueryAPI.md Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com> * Update docs/IE_DG/Int8Inference.md Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com> * Update Custom_Layers_Guide.md * Changes according to review comments * doc scripts fixes * Update docs/IE_DG/Int8Inference.md Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com> * Update Int8Inference.md * update xfail * clang format * updated xfail Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com> Co-authored-by: kblaszczak-intel <karol.blaszczak@intel.com> Co-authored-by: Yury Gorbachev <yury.gorbachev@intel.com> Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
2021-12-21 20:26:37 +03:00
1 parent 0c7089acc6
commit 4ae6258bed
@@ -5,28 +5,22 @@ TensorFlow*, Caffe*, MXNet*, Kaldi* and ONNX* file format. The list of supported
 each of the supported frameworks. To see the operations supported by your framework, refer to
 [Supported Framework Layers](../MO_DG/prepare_model/Supported_Frameworks_Layers.md).

-Custom operations are operations that are not included in the list of known operations. If your model contains any
-operation that is not in the list of known operations, the Model Optimizer is not able to generate an Intermediate
-Representation (IR) for this model.
+Custom operations, that is those not included in the list, are not recognized by Model Optimizer out-of-the-box. Therefore, creating Intermediate Representation (IR) for a model using them requires additional steps. This guide illustrates the workflow for running inference on topologies featuring custom operations, allowing you to plug in your own implementation for existing or completely new operations.

-This guide illustrates the workflow for running inference on topologies featuring custom operations, allowing you to
-plug in your own implementation for existing or completely new operation.
-
-> **NOTE:** *Layer* — The legacy term for an *operation* which came from Caffe\* framework. Currently it is not used.
+> **NOTE**: *Layer* is a legacy term for *operation* which came from Caffe\* framework. Currently it is not used.
 > Refer to the [Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™](../MO_DG/IR_and_opsets.md)
 > for more information on the topic.

 ## Terms Used in This Guide

- *Intermediate Representation (IR)* — Neural Network used only by the Inference Engine in OpenVINO abstracting the
-  different frameworks and describing the model topology, operations parameters and weights.
+- *Intermediate Representation (IR)* — OpenVINO's Neural Network format used by Inference Engine. It abstracts different frameworks and describs model topology, operations parameters, and weights.

- *Operation* — The abstract concept of a math function that is selected for a specific purpose. Operations supported by
+- *Operation* — an abstract concept of a math function selected for a specific purpose. Operations supported by
  OpenVINO™ are listed in the supported operation set provided in the [Available Operations Sets](../ops/opset.md).
  Examples of the operations are: [ReLU](../ops/activation/ReLU_1.md), [Convolution](../ops/convolution/Convolution_1.md),
  [Add](../ops/arithmetic/Add_1.md), etc.

- *Kernel* — The implementation of a operation function in the OpenVINO™ plugin, in this case, the math programmed (in
+- *Kernel* — The implementation of an operation function in the OpenVINO™ plugin, in this case, the math programmed (in
  C++ and OpenCL) to perform the operation for a target hardware (CPU or GPU).

 - *Inference Engine Extension* — Device-specific module implementing custom operations (a set of kernels).
@@ -44,7 +38,7 @@ plugins to support inference of this operation using a particular target hardwar
 To see the operations that are supported by each device plugin for the Inference Engine, refer to the
 [Supported Devices](../IE_DG/supported_plugins/Supported_Devices.md).

-> **NOTE:** If a device doesn't support a particular operation, an alternative to creating a new operation is to target
+> **NOTE**: If a device doesn't support a particular operation, an alternative to creating a new operation is to target
 > an additional device using the HETERO plugin. The [Heterogeneous Plugin](../IE_DG/supported_plugins/HETERO.md) may be
 > used to run an inference model on multiple devices allowing the unsupported operations on one device to "fallback" to
 > run on another device (e.g., CPU) that does support those operations.
@@ -56,18 +50,18 @@ Model Optimizer model conversion pipeline is described in detail in "Model Conve
 Model Optimizer provides an extensions mechanism to support new operations and implement custom model transformations to generate optimized IR. This mechanism is described in the "Model Optimizer Extensions" section of 
 [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).

-Two types of the Model Optimizer extensions should be implemented to support custom operations, at a minimum:
+Two types of Model Optimizer extensions should be implemented to support custom operations, at a minimum:
 1. Operation class for a new operation. This class stores information about the operation, its attributes, shape inference function, attributes to be saved to an IR and some others internally used attributes. Refer to the "Model Optimizer Operation" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.
 2. Operation attributes extractor. The extractor is responsible for parsing framework-specific representation of the
 operation and uses corresponding operation class to update graph node attributes with necessary attributes of the
 operation. Refer to the "Operation Extractor" section of
 [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.

-> **NOTE:** In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
+> **NOTE**: In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).

 ## Custom Operations Extensions for the Inference Engine

-Inference Engine provides extensions mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
+Inference Engine provides an extension mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).

 Each device plugin includes a library of optimized implementations to execute known operations which must be extended to execute a custom operation. The custom operation extension is implemented according to the target device:

@@ -127,19 +121,18 @@ As a result the TensorFlow\* frozen model file "wnet_20.pb" is generated.

 ### Convert the Frozen TensorFlow\* Model to Intermediate Representation

-Firstly, open the model in the TensorBoard or other TensorFlow* model visualization tool. The model supports dynamic
+Firstly, open the model in TensorBoard or other TensorFlow* model visualization tool. The model supports dynamic
 batch dimension because the value for the batch dimension is not hardcoded in the model. Model Optimizer need to set all
 dynamic dimensions to some specific value to create the IR, therefore specify the command line parameter `-b 1` to set
 the batch dimension equal to 1. The actual batch size dimension can be changed at runtime using the Inference Engine API
-described in the [Using Shape Inference](../IE_DG/ShapeInference.md). Also refer to
-[Converting a Model Using General Conversion Parameters](../MO_DG/prepare_model/convert_model/Converting_Model_General.md)
-and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
+described in the [Using Shape Inference](../IE_DG/ShapeInference.md). Also refer to the General Conversion Parameters section in [Converting a Model to Intermediate Representation (IR)](../MO_DG/prepare_model/convert_model/Converting_Model.md) and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
 for more details and command line parameters used for the model conversion.

-```bash
+```sh
 mo --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1
 ```
-> **NOTE:** This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
+
+> **NOTE**: This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
 > the OpenVINO supports this model out of the box.

 Model Optimizer produces the following error:
@@ -221,7 +214,7 @@ following snippet provides two extractors: one for "IFFT2D", another one for "FF

@snippet FFT_ext.py fft_ext:extractor

-> **NOTE:** The graph is in inconsistent state after extracting node attributes because according to original operation
+> **NOTE**: The graph is in inconsistent state after extracting node attributes because according to original operation
 > "IFFT2D" semantic it should have an input consuming a tensor of complex numbers, but the extractor instantiated an
 > operation "FFT" which expects a real tensor with specific layout. But the inconsistency will be resolved during
 > applying front phase transformations discussed below.
@@ -239,7 +232,7 @@ information on how this type of transformation works. The code snippet should be

@snippet Complex.py complex:transformation

-> **NOTE:** The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
+> **NOTE**: The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
 >  "FFT" produces real value tensor.

 Now lets implement a transformation which replace a "ComplexAbs" operation with a sub-graph of primitive operations
@@ -257,15 +250,17 @@ The implementation should be saved to the file `mo_extensions/front/tf/ComplexAb
@snippet ComplexAbs.py complex_abs:transformation

 Now it is possible to convert the model using the following command line:
-```bash
+```sh
 mo --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1 --extensions mo_extensions/
 ```

+@endsphinxdirective
+
 The sub-graph corresponding to the originally non-supported one is depicted in the image below:

 ![Converted sub-graph](img/converted_subgraph.png)

-> **NOTE:** Model Optimizer performed conversion of the model from NHWC to NCHW layout that is why the dimension with
+> **NOTE**: Model Optimizer performed conversion of the model from NHWC to NCHW layout that is why the dimension with
 > the value 2 moved to another position.

 ### Inference Engine Extension Implementation
@@ -279,7 +274,7 @@ file is the following:
@snippet ../template_extension/CMakeLists.txt cmake:extension

 The CPU FFT kernel implementation uses OpenCV to perform the FFT that is why the extension library is linked with
-"opencv_core" which comes with the OpenVINO.
+`opencv_core` which comes with the OpenVINO.

 #### Custom nGraph Operation "FFT" Implementation
 The next step is to create the nGraph operation FFT. The header file "fft_op.hpp" has the following content:
@@ -287,7 +282,7 @@ The next step is to create the nGraph operation FFT. The header file "fft_op.hpp
@snippet ../template_extension/fft_op.hpp fft_op:header

 The operation has just one boolean attribute `inverse`. Implementation of the necessary nGraph operation functions are
-in the "fft_op.cpp" file with the following content:
+in the `fft_op.cpp` file with the following content:

@snippet ../template_extension/fft_op.cpp fft_op:implementation

@@ -350,7 +345,7 @@ python3 mri_reconstruction_demo.py \
 ## Converting Models:

 - [Convert Your Caffe* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md)
- [Convert Your Kaldi* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md)
 - [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
 - [Convert Your MXNet* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md)
+- [Convert Your Kaldi* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md)
 - [Convert Your ONNX* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md)