IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[DOCS] ARM CPU plugin docs (#10885)

Browse Source 
* initial commit

ARM_CPU.md added
ARM CPU is added to the list of supported devices

* Update the list of supported properties

* Update Device_Plugins.md

* Update CODEOWNERS

* Removed quotes in limitations section

* NVIDIA and Android are added to the list of supported devices

* Added See Also section and reg sign to arm

* Added Preprocessing acceleration section

* Update the list of supported layers

* updated list of supported layers

* fix typos

* Added support disclaimer

* update trade and reg symbols

* fixed typos

* fix typos

* reg fix

* add reg symbol back

Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com>
This commit is contained in:
Aleksandr Voron
2022-03-15 17:10:14 +03:00
committed by GitHub
parent a7f1710edf
commit 6cf81ad6a3
4 changed files with 288 additions and 182 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
CODEOWNERS
View File

@@ -66,6 +66,9 @@ Jenkinsfile @openvinotoolkit/openvino-admins
/src/plugins/intel_gna/ @openvinotoolkit/openvino-ie-gna-maintainers /src/plugins/intel_gna/ @openvinotoolkit/openvino-ie-gna-maintainers
/src/inference/include/ie/gna/ @openvinotoolkit/openvino-ie-gna-maintainers /src/inference/include/ie/gna/ @openvinotoolkit/openvino-ie-gna-maintainers
# IE ARM CPU:
/docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md @openvinotoolkit/openvino_contrib-arm_plugin-maintainers
# IE Auto (MULTI) plugin: # IE Auto (MULTI) plugin:
/src/plugins/auto/ @openvinotoolkit/openvino-ie-auto-multi-maintainers /src/plugins/auto/ @openvinotoolkit/openvino-ie-auto-multi-maintainers
/src/inference/include/ie/multi-device/ @openvinotoolkit/openvino-ie-auto-multi-maintainers /src/inference/include/ie/multi-device/ @openvinotoolkit/openvino-ie-auto-multi-maintainers

91
docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Normal file
View File

@@ -0,0 +1,91 @@
# Arm® CPU device {#openvino_docs_OV_UG_supported_plugins_ARM_CPU}
## Introducing the Arm® CPU Plugin
The Arm® CPU plugin is developed in order to enable deep neural networks inference on Arm® CPU, using [Compute Library](https://github.com/ARM-software/ComputeLibrary) as a backend.
> **NOTE**: Note that this is a community-level add-on to OpenVINO™. Intel® welcomes community participation in the OpenVINO™ ecosystem and technical questions on community forums as well as code contributions are welcome. However, this component has not undergone full release validation or qualification from Intel®, and no official support is offered.
The Arm® CPU plugin is not a part of the Intel® Distribution of OpenVINO™ toolkit and is not distributed in pre-built form. To use the plugin, it should be built from source code. Plugin build procedure is described on page [How to build Arm® CPU plugin](https://github.com/openvinotoolkit/openvino_contrib/wiki/How-to-build-ARM-CPU-plugin).
The set of supported layers is defined on [Operation set specification](https://github.com/openvinotoolkit/openvino_contrib/wiki/ARM-plugin-operation-set-specification).
## Supported inference data types
The Arm® CPU plugin supports the following data types as inference precision of internal primitives:
- Floating-point data types:
- f32
- f16
- Quantized data types:
- i8
> **NOTE**: i8 support is experimental.
[Hello Query Device C++ Sample](../../../samples/cpp/hello_query_device/README.md) can be used to print out supported data types for all detected devices.
## Supported features
### Preprocessing acceleration
The Arm® CPU plugin supports the following accelerated preprocessing operations:
- Precision conversion:
- u8 -> u16, s16, s32
- u16 -> u8, u32
- s16 -> u8, s32
- f16 -> f32
- Transposion of tensors with dims < 5
- Interpolation of 4D tensors with no padding (`pads_begin` and `pads_end` equal 0).
The Arm® CPU plugin supports the following preprocessing operations, however they are not accelerated:
- Precision conversion that are not mentioned above
- Color conversion:
- NV12 to RGB
- NV12 to BGR
- i420 to RGB
- i420 to BGR
See [preprocessing API guide](../preprocessing_overview.md) for more details.
## Supported properties
The plugin supports the properties listed below.
### Read-write properties
All parameters must be set before calling `ov::Core::compile_model()` in order to take effect or passed as additional argument to `ov::Core::compile_model()`
- ov::enable_profiling
### Read-only properties
- ov::supported_properties
- ov::available_devices
- ov::range_for_async_infer_requests
- ov::range_for_streams
- ov::device::full_name
- ov::device::capabilities
## Known Layers Limitation
* `AvgPool` layer is supported via arm_compute library for 4D input tensor and via reference implementation for another cases.
* `BatchToSpace` layer is supported 4D tensors only and constant nodes: `block_shape` with `N` = 1 and `C`= 1, `crops_begin` with zero values and `crops_end` with zero values.
* `ConvertLike` layer is supported configuration like `Convert`.
* `DepthToSpace` layer is supported 4D tensors only and for `BLOCKS_FIRST` of `mode` attribute.
* `Equal` does not support `broadcast` for inputs.
* `Gather` layer is supported constant scalar or 1D indices axes only. Layer is supported as via arm_compute library for non negative indices and via reference implementation otherwise.
* `Less` does not support `broadcast` for inputs.
* `LessEqual` does not support `broadcast` for inputs.
* `LRN` layer is supported `axes = {1}` or `axes = {2, 3}` only.
* `MaxPool-1` layer is supported via arm_compute library for 4D input tensor and via reference implementation for another cases.
* `Mod` layer is supported for f32 only.
* `MVN` layer is supported via arm_compute library for 2D inputs and `false` value of `normalize_variance` and `false` value of `across_channels`, for another cases layer is implemented via runtime reference.
* `Normalize` layer is supported via arm_compute library with `MAX` value of `eps_mode` and `axes = {2 | 3}`, and for `ADD` value of `eps_mode` layer uses `DecomposeNormalizeL2Add`, for another cases layer is implemented via runtime reference.
* `NotEqual` does not support `broadcast` for inputs.
* `Pad` layer works with `pad_mode = {REFLECT | CONSTANT | SYMMETRIC}` parameters only.
* `Round` layer is supported via arm_compute library with `RoundMode::HALF_AWAY_FROM_ZERO` value of `mode`, for another cases layer is implemented via runtime reference.
* `SpaceToBatch` layer is supported 4D tensors only and constant nodes: `shapes`, `pads_begin` or `pads_end` with zero paddings for batch or channels and one values `shapes` for batch and channels.
* `SpaceToDepth` layer is supported 4D tensors only and for `BLOCKS_FIRST` of `mode` attribute.
* `StridedSlice` layer is supported via arm_compute library for tensors with dims < 5 and zero values of `ellipsis_mask` or zero values of `new_axis_mask` and `shrink_axis_mask`, for another cases layer is implemented via runtime reference.
* `FakeQuantize` layer is supported via arm_compute library in Low Precision evaluation mode for suitable models and via runtime reference otherwise.
## See Also
* [How to run YOLOv4 model inference using OpenVINO™ and OpenCV on Arm®](https://opencv.org/how-to-run-yolov4-using-openvino-and-opencv-on-arm/)
* [Face recognition on Android™ using OpenVINO™ toolkit with Arm® plugin](https://opencv.org/face-recognition-on-android-using-openvino-toolkit-with-arm-plugin/)

26
docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md
View File

@@ -11,6 +11,7 @@
openvino_docs_OV_UG_supported_plugins_GPU openvino_docs_OV_UG_supported_plugins_GPU
openvino_docs_IE_DG_supported_plugins_VPU openvino_docs_IE_DG_supported_plugins_VPU
openvino_docs_OV_UG_supported_plugins_GNA openvino_docs_OV_UG_supported_plugins_GNA
openvino_docs_OV_UG_supported_plugins_ARM_CPU
@endsphinxdirective @endsphinxdirective
@@ -22,6 +23,7 @@ The OpenVINO Runtime provides capabilities to infer deep learning models on the
|[GPU](GPU.md) |Intel® Graphics, including Intel® HD Graphics, Intel® UHD Graphics, Intel® Iris® Graphics, Intel® Xe Graphics, Intel® Xe MAX Graphics | |[GPU](GPU.md) |Intel® Graphics, including Intel® HD Graphics, Intel® UHD Graphics, Intel® Iris® Graphics, Intel® Xe Graphics, Intel® Xe MAX Graphics |
|[VPUs](VPU.md) |Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs | |[VPUs](VPU.md) |Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs |
|[GNA](GNA.md) |[Intel® Speech Enabling Developer Kit](https://www.intel.com/content/www/us/en/support/articles/000026156/boards-and-kits/smart-home.html); [Amazon Alexa\* Premium Far-Field Developer Kit](https://developer.amazon.com/en-US/alexa/alexa-voice-service/dev-kits/amazon-premium-voice); [Intel® Pentium® Silver Processors N5xxx, J5xxx and Intel® Celeron® Processors N4xxx, J4xxx (formerly codenamed Gemini Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/83915/gemini-lake.html): [Intel® Pentium® Silver J5005 Processor](https://ark.intel.com/content/www/us/en/ark/products/128984/intel-pentium-silver-j5005-processor-4m-cache-up-to-2-80-ghz.html), [Intel® Pentium® Silver N5000 Processor](https://ark.intel.com/content/www/us/en/ark/products/128990/intel-pentium-silver-n5000-processor-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® J4005 Processor](https://ark.intel.com/content/www/us/en/ark/products/128992/intel-celeron-j4005-processor-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® J4105 Processor](https://ark.intel.com/content/www/us/en/ark/products/128989/intel-celeron-j4105-processor-4m-cache-up-to-2-50-ghz.html), [Intel® Celeron® J4125 Processor](https://ark.intel.com/content/www/us/en/ark/products/197305/intel-celeron-processor-j4125-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® Processor N4100](https://ark.intel.com/content/www/us/en/ark/products/128983/intel-celeron-processor-n4100-4m-cache-up-to-2-40-ghz.html), [Intel® Celeron® Processor N4000](https://ark.intel.com/content/www/us/en/ark/products/128988/intel-celeron-processor-n4000-4m-cache-up-to-2-60-ghz.html); [Intel® Pentium® Processors N6xxx, J6xxx, Intel® Celeron® Processors N6xxx, J6xxx and Intel Atom® x6xxxxx (formerly codenamed Elkhart Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/128825/products-formerly-elkhart-lake.html); [Intel® Core™ Processors (formerly codenamed Cannon Lake)](https://ark.intel.com/content/www/us/en/ark/products/136863/intel-core-i3-8121u-processor-4m-cache-up-to-3-20-ghz.html); [10th Generation Intel® Core™ Processors (formerly codenamed Ice Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/74979/ice-lake.html): [Intel® Core™ i7-1065G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/196597/intel-core-i71065g7-processor-8m-cache-up-to-3-90-ghz.html), [Intel® Core™ i7-1060G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/197120/intel-core-i71060g7-processor-8m-cache-up-to-3-80-ghz.html), [Intel® Core™ i5-1035G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/196591/intel-core-i51035g4-processor-6m-cache-up-to-3-70-ghz.html), [Intel® Core™ i5-1035G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/196592/intel-core-i51035g7-processor-6m-cache-up-to-3-70-ghz.html), [Intel® Core™ i5-1035G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/196603/intel-core-i51035g1-processor-6m-cache-up-to-3-60-ghz.html), [Intel® Core™ i5-1030G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/197119/intel-core-i51030g7-processor-6m-cache-up-to-3-50-ghz.html), [Intel® Core™ i5-1030G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/197121/intel-core-i51030g4-processor-6m-cache-up-to-3-50-ghz.html), [Intel® Core™ i3-1005G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/196588/intel-core-i31005g1-processor-4m-cache-up-to-3-40-ghz.html), [Intel® Core™ i3-1000G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/197122/intel-core-i31000g1-processor-4m-cache-up-to-3-20-ghz.html), [Intel® Core™ i3-1000G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/197123/intel-core-i31000g4-processor-4m-cache-up-to-3-20-ghz.html); [11th Generation Intel® Core™ Processors (formerly codenamed Tiger Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/88759/tiger-lake.html); [12th Generation Intel® Core™ Processors (formerly codenamed Alder Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/147470/products-formerly-alder-lake.html)| |[GNA](GNA.md) |[Intel® Speech Enabling Developer Kit](https://www.intel.com/content/www/us/en/support/articles/000026156/boards-and-kits/smart-home.html); [Amazon Alexa\* Premium Far-Field Developer Kit](https://developer.amazon.com/en-US/alexa/alexa-voice-service/dev-kits/amazon-premium-voice); [Intel® Pentium® Silver Processors N5xxx, J5xxx and Intel® Celeron® Processors N4xxx, J4xxx (formerly codenamed Gemini Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/83915/gemini-lake.html): [Intel® Pentium® Silver J5005 Processor](https://ark.intel.com/content/www/us/en/ark/products/128984/intel-pentium-silver-j5005-processor-4m-cache-up-to-2-80-ghz.html), [Intel® Pentium® Silver N5000 Processor](https://ark.intel.com/content/www/us/en/ark/products/128990/intel-pentium-silver-n5000-processor-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® J4005 Processor](https://ark.intel.com/content/www/us/en/ark/products/128992/intel-celeron-j4005-processor-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® J4105 Processor](https://ark.intel.com/content/www/us/en/ark/products/128989/intel-celeron-j4105-processor-4m-cache-up-to-2-50-ghz.html), [Intel® Celeron® J4125 Processor](https://ark.intel.com/content/www/us/en/ark/products/197305/intel-celeron-processor-j4125-4m-cache-up-to-2-70-ghz.html), [Intel® Celeron® Processor N4100](https://ark.intel.com/content/www/us/en/ark/products/128983/intel-celeron-processor-n4100-4m-cache-up-to-2-40-ghz.html), [Intel® Celeron® Processor N4000](https://ark.intel.com/content/www/us/en/ark/products/128988/intel-celeron-processor-n4000-4m-cache-up-to-2-60-ghz.html); [Intel® Pentium® Processors N6xxx, J6xxx, Intel® Celeron® Processors N6xxx, J6xxx and Intel Atom® x6xxxxx (formerly codenamed Elkhart Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/128825/products-formerly-elkhart-lake.html); [Intel® Core™ Processors (formerly codenamed Cannon Lake)](https://ark.intel.com/content/www/us/en/ark/products/136863/intel-core-i3-8121u-processor-4m-cache-up-to-3-20-ghz.html); [10th Generation Intel® Core™ Processors (formerly codenamed Ice Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/74979/ice-lake.html): [Intel® Core™ i7-1065G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/196597/intel-core-i71065g7-processor-8m-cache-up-to-3-90-ghz.html), [Intel® Core™ i7-1060G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/197120/intel-core-i71060g7-processor-8m-cache-up-to-3-80-ghz.html), [Intel® Core™ i5-1035G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/196591/intel-core-i51035g4-processor-6m-cache-up-to-3-70-ghz.html), [Intel® Core™ i5-1035G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/196592/intel-core-i51035g7-processor-6m-cache-up-to-3-70-ghz.html), [Intel® Core™ i5-1035G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/196603/intel-core-i51035g1-processor-6m-cache-up-to-3-60-ghz.html), [Intel® Core™ i5-1030G7 Processor](https://ark.intel.com/content/www/us/en/ark/products/197119/intel-core-i51030g7-processor-6m-cache-up-to-3-50-ghz.html), [Intel® Core™ i5-1030G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/197121/intel-core-i51030g4-processor-6m-cache-up-to-3-50-ghz.html), [Intel® Core™ i3-1005G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/196588/intel-core-i31005g1-processor-4m-cache-up-to-3-40-ghz.html), [Intel® Core™ i3-1000G1 Processor](https://ark.intel.com/content/www/us/en/ark/products/197122/intel-core-i31000g1-processor-4m-cache-up-to-3-20-ghz.html), [Intel® Core™ i3-1000G4 Processor](https://ark.intel.com/content/www/us/en/ark/products/197123/intel-core-i31000g4-processor-4m-cache-up-to-3-20-ghz.html); [11th Generation Intel® Core™ Processors (formerly codenamed Tiger Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/88759/tiger-lake.html); [12th Generation Intel® Core™ Processors (formerly codenamed Alder Lake)](https://ark.intel.com/content/www/us/en/ark/products/codename/147470/products-formerly-alder-lake.html)|
|[Arm® CPU](ARM_CPU.md) |Raspberry Pi™ 4 Model B, Apple® Mac mini with M1 chip, NVIDIA® Jetson Nano™, Android™ devices |
OpenVINO runtime also has several execution capabilities which work on top of other devices: OpenVINO runtime also has several execution capabilities which work on top of other devices:
@@ -38,17 +40,17 @@ Devices similar to the ones we have used for benchmarking can be accessed using
## Features support matrix ## Features support matrix
The table below demonstrates support of key features by OpenVINO device plugins. The table below demonstrates support of key features by OpenVINO device plugins.
| Capability | [CPU](CPU.md) | [GPU](GPU.md) | [GNA](GNA.md) | [VPU](VPU.md) | | Capability | [CPU](CPU.md) | [GPU](GPU.md) | [GNA](GNA.md) | [VPU](VPU.md) | [Arm® CPU](ARM_CPU.md) |
| ---------- | --- | --- | --- | --- | | ---------- | --- | --- | --- | --- | --- |
| [Heterogeneous execution](../hetero_execution.md)| Yes | Yes | No | ? | | [Heterogeneous execution](../hetero_execution.md)| Yes | Yes | No | ? | Yes |
| [Multi-device execution](../multi_device.md) | Yes | Yes | Partial | ? | | [Multi-device execution](../multi_device.md) | Yes | Yes | Partial | ? | Yes |
| [Automatic batching](../automatic_batching.md) | No | Yes | No | ? | | [Automatic batching](../automatic_batching.md) | No | Yes | No | ? | No |
| [Multi-stream execution](@ref openvino_docs_optimization_guide_dldt_optimization_guide) | Yes | Yes | No | ? | | [Multi-stream execution](@ref openvino_docs_optimization_guide_dldt_optimization_guide) | Yes | Yes | No | ? | Yes |
| [Models caching](../Model_caching_overview.md) | Yes | Partial | Yes | ? | | [Models caching](../Model_caching_overview.md) | Yes | Partial | Yes | ? | No |
| [Dynamic shapes](../ov_dynamic_shapes.md) | Yes | Partial | No | ? | | [Dynamic shapes](../ov_dynamic_shapes.md) | Yes | Partial | No | ? | No |
| Import/Export | Yes | No | Yes | ? | | Import/Export | Yes | No | Yes | ? | No |
| [Preprocessing acceleration](../preprocessing_overview.md) | Yes | Yes | No | ? | | [Preprocessing acceleration](../preprocessing_overview.md) | Yes | Yes | No | ? | Partial |
| [Stateful models](../network_state_intro.md) | Yes | No | Yes | ? | | [Stateful models](../network_state_intro.md) | Yes | No | Yes | ? | No |
| [Extensibility](@ref openvino_docs_Extensibility_UG_Intro) | Yes | Yes | No | ? | | [Extensibility](@ref openvino_docs_Extensibility_UG_Intro) | Yes | Yes | No | ? | No |
For more details on plugin specific feature limitation see corresponding plugin pages. For more details on plugin specific feature limitation see corresponding plugin pages.

350
docs/OV_Runtime_UG/supported_plugins/Supported_Devices.md
View File

@@ -13,6 +13,7 @@ The OpenVINO Runtime provides unique capabilities to infer deep learning models
|[CPU plugin](CPU.md) |Intel&reg; Xeon&reg; with Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512), and AVX512_BF16, Intel&reg; Core&trade; Processors with Intel&reg; AVX2, Intel&reg; Atom&reg; Processors with Intel® Streaming SIMD Extensions (Intel® SSE) | |[CPU plugin](CPU.md) |Intel&reg; Xeon&reg; with Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512), and AVX512_BF16, Intel&reg; Core&trade; Processors with Intel&reg; AVX2, Intel&reg; Atom&reg; Processors with Intel® Streaming SIMD Extensions (Intel® SSE) |
|[VPU plugins](VPU.md) (available in the Intel® Distribution of OpenVINO™ toolkit) |Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs | |[VPU plugins](VPU.md) (available in the Intel® Distribution of OpenVINO™ toolkit) |Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X, Intel® Vision Accelerator Design with Intel® Movidius™ VPUs |
|[GNA plugin](GNA.md) (available in the Intel® Distribution of OpenVINO™ toolkit) |Intel&reg; Speech Enabling Developer Kit, Amazon Alexa* Premium Far-Field Developer Kit, Intel&reg; Pentium&reg; Silver J5005 Processor, Intel&reg; Pentium&reg; Silver N5000 Processor, Intel&reg; Celeron&reg; J4005 Processor, Intel&reg; Celeron&reg; J4105 Processor, Intel&reg; Celeron&reg; Processor N4100, Intel&reg; Celeron&reg; Processor N4000, Intel&reg; Core&trade; i3-8121U Processor, Intel&reg; Core&trade; i7-1065G7 Processor, Intel&reg; Core&trade; i7-1060G7 Processor, Intel&reg; Core&trade; i5-1035G4 Processor, Intel&reg; Core&trade; i5-1035G7 Processor, Intel&reg; Core&trade; i5-1035G1 Processor, Intel&reg; Core&trade; i5-1030G7 Processor, Intel&reg; Core&trade; i5-1030G4 Processor, Intel&reg; Core&trade; i3-1005G1 Processor, Intel&reg; Core&trade; i3-1000G1 Processor, Intel&reg; Core&trade; i3-1000G4 Processor| |[GNA plugin](GNA.md) (available in the Intel® Distribution of OpenVINO™ toolkit) |Intel&reg; Speech Enabling Developer Kit, Amazon Alexa* Premium Far-Field Developer Kit, Intel&reg; Pentium&reg; Silver J5005 Processor, Intel&reg; Pentium&reg; Silver N5000 Processor, Intel&reg; Celeron&reg; J4005 Processor, Intel&reg; Celeron&reg; J4105 Processor, Intel&reg; Celeron&reg; Processor N4100, Intel&reg; Celeron&reg; Processor N4000, Intel&reg; Core&trade; i3-8121U Processor, Intel&reg; Core&trade; i7-1065G7 Processor, Intel&reg; Core&trade; i7-1060G7 Processor, Intel&reg; Core&trade; i5-1035G4 Processor, Intel&reg; Core&trade; i5-1035G7 Processor, Intel&reg; Core&trade; i5-1035G1 Processor, Intel&reg; Core&trade; i5-1030G7 Processor, Intel&reg; Core&trade; i5-1030G4 Processor, Intel&reg; Core&trade; i3-1005G1 Processor, Intel&reg; Core&trade; i3-1000G1 Processor, Intel&reg; Core&trade; i3-1000G4 Processor|
|[Arm® CPU plugin](ARM_CPU.md) (unavailable in the Intel® Distribution of OpenVINO™ toolkit) |Raspberry Pi™ 4 Model B, Apple® Mac mini with M1 chip, NVIDIA® Jetson Nano™, Android™ devices |
|[Multi-Device execution](../multi_device.md) |Multi-Device execution enables simultaneous inference of the same model on several devices in parallel | |[Multi-Device execution](../multi_device.md) |Multi-Device execution enables simultaneous inference of the same model on several devices in parallel |
|[Auto-Device plugin](../auto_device_selection.md) |Auto-Device plugin enables selecting Intel&reg; device for inference automatically | |[Auto-Device plugin](../auto_device_selection.md) |Auto-Device plugin enables selecting Intel&reg; device for inference automatically |
|[Heterogeneous plugin](../hetero_execution.md) |Heterogeneous execution enables automatic inference splitting between several devices (for example if a device doesn't [support certain operation](#supported-layers)). | |[Heterogeneous plugin](../hetero_execution.md) |Heterogeneous execution enables automatic inference splitting between several devices (for example if a device doesn't [support certain operation](#supported-layers)). |
@@ -28,6 +29,7 @@ The table below shows the plugin libraries and additional dependencies for Linux
| MYRIAD | `libopenvino_intel_myriad_plugin.so` | `libusb.so` | `openvino_intel_myriad_plugin.dll`| `usb.dll` | `libopenvino_intel_myriad_plugin.so` | `libusb.dylib` | | MYRIAD | `libopenvino_intel_myriad_plugin.so` | `libusb.so` | `openvino_intel_myriad_plugin.dll`| `usb.dll` | `libopenvino_intel_myriad_plugin.so` | `libusb.dylib` |
| HDDL | `libintel_hddl_plugin.so` | `libbsl.so`, `libhddlapi.so`, `libmvnc-hddl.so` | `intel_hddl_plugin.dll` | `bsl.dll`, `hddlapi.dll`, `json-c.dll`, `libcrypto-1_1-x64.dll`, `libssl-1_1-x64.dll`, `mvnc-hddl.dll` | Is not supported | - | | HDDL | `libintel_hddl_plugin.so` | `libbsl.so`, `libhddlapi.so`, `libmvnc-hddl.so` | `intel_hddl_plugin.dll` | `bsl.dll`, `hddlapi.dll`, `json-c.dll`, `libcrypto-1_1-x64.dll`, `libssl-1_1-x64.dll`, `mvnc-hddl.dll` | Is not supported | - |
| GNA | `libopenvino_intel_gna_plugin.so` | `libgna.so`, | `openvino_intel_gna_plugin.dll` | `gna.dll` | Is not supported | - | | GNA | `libopenvino_intel_gna_plugin.so` | `libgna.so`, | `openvino_intel_gna_plugin.dll` | `gna.dll` | Is not supported | - |
| Arm® CPU | `libopenvino_arm_cpu_plugin.so` | | Is not supported | - | `libopenvino_arm_cpu_plugin.so` | - |
| HETERO | `libopenvino_hetero_plugin.so` | Same as for selected plugins | `openvino_hetero_plugin.dll` | Same as for selected plugins | `libopenvino_hetero_plugin.so` | Same as for selected plugins | | HETERO | `libopenvino_hetero_plugin.so` | Same as for selected plugins | `openvino_hetero_plugin.dll` | Same as for selected plugins | `libopenvino_hetero_plugin.so` | Same as for selected plugins |
| MULTI | `libopenvino_auto_plugin.so` | Same as for selected plugins | `openvino_auto_plugin.dll` | Same as for selected plugins | `libopenvino_auto_plugin.so` | Same as for selected plugins | | MULTI | `libopenvino_auto_plugin.so` | Same as for selected plugins | `openvino_auto_plugin.dll` | Same as for selected plugins | `libopenvino_auto_plugin.so` | Same as for selected plugins |
| AUTO | `libopenvino_auto_plugin.so` | Same as for selected plugins | `openvino_auto_plugin.dll` | Same as for selected plugins | `libopenvino_auto_plugin.so` | Same as for selected plugins | | AUTO | `libopenvino_auto_plugin.so` | Same as for selected plugins | `openvino_auto_plugin.dll` | Same as for selected plugins | `libopenvino_auto_plugin.so` | Same as for selected plugins |
@@ -66,23 +68,26 @@ For example, the CHW value at index (c,h,w) is physically located at index (c\*H
### Supported Model Formats ### Supported Model Formats
|Plugin |FP32 |FP16 |I8 | |Plugin |FP32 |FP16 |I8 |
|:-------------|:----------------------:|:----------------------:|:----------------------:| |:------------------|:----------------------:|:----------------------:|:----------------------:|
|CPU plugin |Supported and preferred |Supported |Supported | |CPU plugin |Supported and preferred |Supported |Supported |
|GPU plugin |Supported |Supported and preferred |Supported | |GPU plugin |Supported |Supported and preferred |Supported |
|VPU plugins |Not supported |Supported |Not supported | |VPU plugins |Not supported |Supported |Not supported |
|GNA plugin |Supported |Supported |Not supported | |GNA plugin |Supported |Supported |Not supported |
|Arm® CPU plugin |Supported and preferred |Supported |Supported (partially) |
For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions
the supported models formats depends on the actual underlying devices. _Generally, FP16 is preferable as it is most ubiquitous and performant_. the supported models formats depends on the actual underlying devices. _Generally, FP16 is preferable as it is most ubiquitous and performant_.
### Supported Input Precision ### Supported Input Precision
|Plugin |FP32 |FP16 |U8 |U16 |I8 |I16 | |Plugin |FP32 |FP16 |U8 |U16 |I8 |I16 |
|:-------------|:--------:|:-------------:|:-------------:|:-------------:|:------------:|:-------------:| |:------------------|:--------:|:-------------:|:-------------:|:-------------:|:------------:|:-------------:|
|CPU plugin |Supported |Not supported |Supported |Supported |Not supported |Supported | |CPU plugin |Supported |Not supported |Supported |Supported |Not supported |Supported |
|GPU plugin |Supported |Supported\* |Supported\* |Supported\* |Not supported |Supported\* | |GPU plugin |Supported |Supported\* |Supported\* |Supported\* |Not supported |Supported\* |
|VPU plugins |Supported |Supported |Supported |Not supported |Not supported |Not supported | |VPU plugins |Supported |Supported |Supported |Not supported |Not supported |Not supported |
|GNA plugin |Supported |Not supported |Supported |Not supported |Supported |Supported | |GNA plugin |Supported |Not supported |Supported |Not supported |Supported |Supported |
|Arm® CPU plugin |Supported |Supported |Supported |Supported |Not supported |Not supported |
<br>\* - Supported via `SetBlob` only, `GetBlob` returns FP32<br> <br>\* - Supported via `SetBlob` only, `GetBlob` returns FP32<br>
For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions
@@ -90,23 +95,26 @@ the supported input precision depends on the actual underlying devices. _Genera
### Supported Output Precision ### Supported Output Precision
|Plugin |FP32 |FP16 | |Plugin |FP32 |FP16 |
|:-------------|:--------:|:------------:| |:------------------|:--------:|:------------:|
|CPU plugin |Supported |Not supported | |CPU plugin |Supported |Not supported |
|GPU plugin |Supported |Supported | |GPU plugin |Supported |Supported |
|VPU plugins |Supported |Supported | |VPU plugins |Supported |Supported |
|GNA plugin |Supported |Not supported | |GNA plugin |Supported |Not supported |
|Arm® CPU plugin |Supported |Supported |
For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions For [Multi-Device](../multi_device.md) and [Heterogeneous](../hetero_execution.md) executions
the supported output precision depends on the actual underlying devices. _Generally, FP32 is preferable as it is most ubiquitous_. the supported output precision depends on the actual underlying devices. _Generally, FP32 is preferable as it is most ubiquitous_.
### Supported Input Layout ### Supported Input Layout
|Plugin |NCDHW |NCHW |NHWC |NC | |Plugin |NCDHW |NCHW |NHWC |NC |
|:-------------|:------------:|:------------:|:------------:|:------------:| |:------------------|:------------:|:------------:|:------------:|:------------:|
|CPU plugin |Supported |Supported |Supported |Supported | |CPU plugin |Supported |Supported |Supported |Supported |
|GPU plugin |Supported |Supported |Supported |Supported | |GPU plugin |Supported |Supported |Supported |Supported |
|VPU plugins |Supported |Supported |Supported |Supported | |VPU plugins |Supported |Supported |Supported |Supported |
|GNA plugin |Not supported |Supported |Supported |Supported | |GNA plugin |Not supported |Supported |Supported |Supported |
|Arm® CPU plugin |Not supported |Supported |Supported |Supported |
### Supported Output Layout ### Supported Output Layout
@@ -121,155 +129,157 @@ For setting relevant configuration, refer to the
### Supported Layers ### Supported Layers
The following layers are supported by the plugins and by [Shape Inference feature](../ShapeInference.md): The following layers are supported by the plugins and by [Shape Inference feature](../ShapeInference.md):
| Layers | GPU | CPU | VPU | GNA | ShapeInfer | | Layers | GPU | CPU | VPU | GNA | Arm® CPU | ShapeInfer |
|:-------------------------------|:-------------:|:-------------:|:-------------:|:-------------:|:-------------:| |:-------------------------------|:-------------:|:-------------:|:-------------:|:-------------:|:---------------:|:-------------:|
| Abs | Supported | Supported\*\* | Supported | Not Supported | Supported | | Abs | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Acos | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Acos | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*|Supported |
| Acosh | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Acosh | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*|Supported |
| Activation-Clamp | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-Clamp | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Activation-ELU | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Activation-ELU | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Activation-Exp | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-Exp | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Activation-Leaky ReLU | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-Leaky ReLU | Supported |Supported\*\*\*| Supported | Supported | Not Supported | Supported |
| Activation-Not | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Activation-Not | Supported |Supported\*\*\*| Supported | Not Supported | Not Supported | Supported |
| Activation-PReLU | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Activation-PReLU | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Activation-ReLU | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-ReLU | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Activation-ReLU6 | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Activation-ReLU6 | Supported |Supported\*\*\*| Supported | Not Supported | Not Supported | Supported |
| Activation-Sigmoid/Logistic | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-Sigmoid/Logistic | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Activation-TanH | Supported |Supported\*\*\*| Supported | Supported | Supported | | Activation-TanH | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| ArgMax | Supported | Supported\*\* | Supported | Not Supported | Supported | | ArgMax | Supported | Supported\*\* | Supported | Not Supported | Not Supported | Supported |
| Asin | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Asin | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Asinh | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Asinh | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Atan | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Atan | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Atanh | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Atanh | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| BatchNormalization | Supported | Supported | Supported | Not Supported | Supported | | BatchNormalization | Supported | Supported | Supported | Not Supported | Supported | Supported |
| BinaryConvolution | Supported | Supported | Not Supported | Not Supported | Supported | | BinaryConvolution | Supported | Supported | Not Supported | Not Supported | Not Supported | Supported |
| Broadcast | Supported | Supported\*\* | Supported | Not Supported | Supported | | Broadcast | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Ceil | Supported | Supported\*\* | Supported | Not Supported | Supported | | Ceil | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Concat | Supported |Supported\*\*\*| Supported | Supported | Supported | | Concat | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Const | Supported | Supported | Supported | Supported | Not Supported | | Const | Supported | Supported | Supported | Supported | Supported | Not Supported |
| Convolution-Dilated | Supported | Supported | Supported | Not Supported | Supported | | Convolution-Dilated | Supported | Supported | Supported | Not Supported | Supported | Supported |
| Convolution-Dilated 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | | Convolution-Dilated 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | Not Supported |
| Convolution-Grouped | Supported | Supported | Supported | Not Supported | Supported | | Convolution-Grouped | Supported | Supported | Supported | Not Supported | Supported | Supported |
| Convolution-Grouped 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | | Convolution-Grouped 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | Not Supported |
| Convolution-Ordinary | Supported | Supported | Supported | Supported\* | Supported | | Convolution-Ordinary | Supported | Supported | Supported | Supported\* | Supported | Supported |
| Convolution-Ordinary 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | | Convolution-Ordinary 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | Not Supported |
| Cos | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Cos | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Cosh | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Cosh | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Crop | Supported | Supported | Supported | Supported | Supported | | Crop | Supported | Supported | Supported | Supported | Not Supported | Supported |
| CTCGreedyDecoder | Supported\*\* | Supported\*\* | Supported\* | Not Supported | Supported | | CTCGreedyDecoder | Supported\*\* | Supported\*\* | Supported\* | Not Supported |Supported\*\*\*\*| Supported |
| Deconvolution | Supported | Supported | Supported | Not Supported | Supported | | Deconvolution | Supported | Supported | Supported | Not Supported | Not Supported | Supported |
| Deconvolution 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | | Deconvolution 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | Not Supported |
| DeformableConvolution | Supported | Supported | Not Supported | Not Supported | Supported | | DeformableConvolution | Supported | Supported | Not Supported | Not Supported | Not Supported | Supported |
| DepthToSpace | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | DepthToSpace | Supported | Supported\*\* | Not Supported | Not Supported | Supported\* | Supported |
| DetectionOutput | Supported | Supported\*\* | Supported\* | Not Supported | Supported | | DetectionOutput | Supported | Supported\*\* | Supported\* | Not Supported |Supported\*\*\*\*| Supported |
| Eltwise-And | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-And | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Add | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Add | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Div | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Div | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Equal | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Equal | Supported |Supported\*\*\*| Supported | Not Supported | Supported\* | Supported |
| Eltwise-FloorMod | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-FloorMod | Supported |Supported\*\*\*| Supported | Not Supported |Supported\*\*\*\*| Supported |
| Eltwise-Greater | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Greater | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-GreaterEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-GreaterEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Less | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Less | Supported |Supported\*\*\*| Supported | Not Supported | Supported\* | Supported |
| Eltwise-LessEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-LessEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported\* | Supported |
| Eltwise-LogicalAnd | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-LogicalAnd | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-LogicalOr | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-LogicalOr | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-LogicalXor | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-LogicalXor | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Max | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Max | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Min | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Min | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Mul | Supported |Supported\*\*\*| Supported | Supported | Supported | | Eltwise-Mul | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Eltwise-NotEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-NotEqual | Supported |Supported\*\*\*| Supported | Not Supported | Supported\* | Supported |
| Eltwise-Pow | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-Pow | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Prod | Supported |Supported\*\*\*| Supported | Supported | Supported | | Eltwise-Prod | Supported |Supported\*\*\*| Supported | Supported | Not Supported | Supported |
| Eltwise-SquaredDiff | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | Eltwise-SquaredDiff | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Eltwise-Sub | Supported |Supported\*\*\*| Supported | Supported | Supported | | Eltwise-Sub | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Eltwise-Sum | Supported |Supported\*\*\*| Supported | Supported | Supported | | Eltwise-Sum | Supported |Supported\*\*\*| Supported | Supported |Supported\*\*\*\*| Supported |
| Erf | Supported | Supported\*\* | Supported | Not Supported | Supported | | Erf | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| Exp | Supported | Supported | Supported | Supported | Supported | | Exp | Supported | Supported | Supported | Supported | Supported | Supported |
| FakeQuantize | Not Supported | Supported | Not Supported | Not Supported | Supported | | FakeQuantize | Not Supported | Supported | Not Supported | Not Supported | Supported\* | Supported |
| Fill | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Fill | Not Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| Flatten | Supported | Supported | Supported | Not Supported | Supported | | Flatten | Supported | Supported | Supported | Not Supported | Not Supported | Supported |
| Floor | Supported | Supported\*\* | Supported | Not Supported | Supported | | Floor | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| FullyConnected (Inner Product) | Supported |Supported\*\*\*| Supported | Supported | Supported | | FullyConnected (Inner Product) | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Gather | Supported | Supported\*\* | Supported | Not Supported | Supported | | Gather | Supported | Supported\*\* | Supported | Not Supported | Supported\* | Supported |
| GatherTree | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported | | GatherTree | Not Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Gemm | Supported | Supported | Supported | Not Supported | Supported | | Gemm | Supported | Supported | Supported | Not Supported | Not Supported | Supported |
| GRN | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported | | GRN | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported | Supported |
| HardSigmoid | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | HardSigmoid | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| Interp | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported\* | | Interp | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported\* | Supported\* |
| Log | Supported | Supported\*\* | Supported | Supported | Supported | | Log | Supported | Supported\*\* | Supported | Supported | Supported | Supported |
| LRN (Norm) | Supported | Supported | Supported | Not Supported | Supported | | LRN (Norm) | Supported | Supported | Supported | Not Supported | Supported\* | Supported |
| LSTMCell | Supported | Supported | Supported | Supported | Not Supported | | LSTMCell | Supported | Supported | Supported | Supported | Supported | Not Supported |
| GRUCell | Supported | Supported | Not Supported | Not Supported | Not Supported | | GRUCell | Supported | Supported | Not Supported | Not Supported | Supported | Not Supported |
| RNNCell | Supported | Supported | Not Supported | Not Supported | Not Supported | | RNNCell | Supported | Supported | Not Supported | Not Supported | Supported | Not Supported |
| LSTMSequence | Supported | Supported | Supported | Not Supported | Not Supported | | LSTMSequence | Supported | Supported | Supported | Not Supported |Supported\*\*\*\*| Not Supported |
| GRUSequence | Supported | Supported | Not Supported | Not Supported | Not Supported | | GRUSequence | Supported | Supported | Not Supported | Not Supported |Supported\*\*\*\*| Not Supported |
| RNNSequence | Supported | Supported | Not Supported | Not Supported | Not Supported | | RNNSequence | Supported | Supported | Not Supported | Not Supported |Supported\*\*\*\*| Not Supported |
| LogSoftmax | Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | | LogSoftmax | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Not Supported |
| Memory | Not Supported | Supported | Not Supported | Supported | Supported | | Memory | Not Supported | Supported | Not Supported | Supported | Not Supported | Supported |
| MVN | Supported | Supported\*\* | Supported\* | Not Supported | Supported | | MVN | Supported | Supported\*\* | Supported\* | Not Supported | Supported\* | Supported |
| Neg | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Neg | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| NonMaxSuppression | Not Supported | Supported\*\* | Supported | Not Supported | Supported | | NonMaxSuppression | Not Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| Normalize | Supported | Supported\*\* | Supported\* | Not Supported | Supported | | Normalize | Supported | Supported\*\* | Supported\* | Not Supported | Supported\* | Supported |
| OneHot | Supported | Supported\*\* | Supported | Not Supported | Supported | | OneHot | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| Pad | Supported | Supported\*\* | Supported\* | Not Supported | Supported | | Pad | Supported | Supported\*\* | Supported\* | Not Supported | Supported\* | Supported |
| Permute | Supported | Supported | Supported | Supported\* | Supported | | Permute | Supported | Supported | Supported | Supported\* | Not Supported | Supported |
| Pooling(AVG,MAX) | Supported | Supported | Supported | Supported | Supported | | Pooling(AVG,MAX) | Supported | Supported | Supported | Supported | Supported | Supported |
| Pooling(AVG,MAX) 3D | Supported | Supported | Not Supported | Not Supported | Not Supported | | Pooling(AVG,MAX) 3D | Supported | Supported | Not Supported | Not Supported | Supported\* | Not Supported |
| Power | Supported | Supported\*\* | Supported | Supported\* | Supported | | Power | Supported | Supported\*\* | Supported | Supported\* | Supported | Supported |
| PowerFile | Not Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | | PowerFile | Not Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Not Supported |
| PriorBox | Supported | Supported\*\* | Supported | Not Supported | Supported | | PriorBox | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| PriorBoxClustered | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported | | PriorBoxClustered | Supported\*\* | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Proposal | Supported | Supported\*\* | Supported | Not Supported | Supported | | Proposal | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| PSROIPooling | Supported | Supported\*\* | Supported | Not Supported | Supported | | PSROIPooling | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| Range | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Range | Not Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| Reciprocal | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Reciprocal | Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| ReduceAnd | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReduceAnd | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| ReduceL1 | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceL1 | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| ReduceL2 | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceL2 | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| ReduceLogSum | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceLogSum | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| ReduceLogSumExp | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceLogSumExp | Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| ReduceMax | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReduceMax | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| ReduceMean | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReduceMean | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| ReduceMin | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReduceMin | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| ReduceOr | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceOr | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| ReduceProd | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceProd | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| ReduceSum | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReduceSum | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| ReduceSumSquare | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ReduceSumSquare | Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| RegionYolo | Supported | Supported\*\* | Supported | Not Supported | Supported | | RegionYolo | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| ReorgYolo | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReorgYolo | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Resample | Supported | Supported\*\* | Supported | Not Supported | Supported | | Resample | Supported | Supported\*\* | Supported | Not Supported | Not Supported | Supported |
| Reshape | Supported |Supported\*\*\*| Supported | Supported | Supported\* | | Reshape | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported\* |
| ReverseSequence | Supported | Supported\*\* | Supported | Not Supported | Supported | | ReverseSequence | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| RNN | Not Supported | Supported | Supported | Not Supported | Not Supported | | RNN | Not Supported | Supported | Supported | Not Supported | Supported | Not Supported |
| ROIPooling | Supported\* | Supported | Supported | Not Supported | Supported | | ROIPooling | Supported\* | Supported | Supported | Not Supported |Supported\*\*\*\*| Supported |
| ScaleShift | Supported |Supported\*\*\*| Supported\* | Supported | Supported | | ScaleShift | Supported |Supported\*\*\*| Supported\* | Supported | Not Supported | Supported |
| ScatterUpdate | Not Supported | Supported\*\* | Supported | Not Supported | Supported | | ScatterUpdate | Not Supported | Supported\*\* | Supported | Not Supported | Not Supported | Supported |
| Select | Supported | Supported | Supported | Not Supported | Supported | | Select | Supported | Supported | Supported | Not Supported | Supported | Supported |
| Selu | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Selu | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| ShuffleChannels | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | ShuffleChannels | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| Sign | Supported | Supported\*\* | Supported | Not Supported | Supported | | Sign | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Sin | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Sin | Supported | Supported\*\* | Not Supported | Not Supported | Supported | Supported |
| Sinh | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Sinh | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| SimplerNMS | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | SimplerNMS | Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| Slice | Supported |Supported\*\*\*| Supported | Supported | Supported | | Slice | Supported |Supported\*\*\*| Supported | Supported | Not Supported | Supported |
| SoftMax | Supported |Supported\*\*\*| Supported | Not Supported | Supported | | SoftMax | Supported |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| Softplus | Supported | Supported\*\* | Supported | Not Supported | Supported | | Softplus | Supported | Supported\*\* | Supported | Not Supported | Supported | Supported |
| Softsign | Supported | Supported\*\* | Not Supported | Supported | Supported | | Softsign | Supported | Supported\*\* | Not Supported | Supported | Not Supported | Supported |
| SpaceToDepth | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported | | SpaceToDepth | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported\* | Supported |
| SpatialTransformer | Not Supported | Supported\*\* | Not Supported | Not Supported | Supported | | SpatialTransformer | Not Supported | Supported\*\* | Not Supported | Not Supported | Not Supported | Supported |
| Split | Supported |Supported\*\*\*| Supported | Supported | Supported | | Split | Supported |Supported\*\*\*| Supported | Supported | Supported | Supported |
| Squeeze | Supported | Supported\*\* | Supported | Supported | Supported | | Squeeze | Supported | Supported\*\* | Supported | Supported | Supported | Supported |
| StridedSlice | Supported | Supported\*\* | Supported | Not Supported | Supported | | StridedSlice | Supported | Supported\*\* | Supported | Not Supported | Supported\* | Supported |
| Tan | Supported | Supported\*\* | Not Supported | Not Supported | Supported | | Tan | Supported | Supported\*\* | Not Supported | Not Supported |Supported\*\*\*\*| Supported |
| TensorIterator | Not Supported | Supported | Supported | Supported | Not Supported | | TensorIterator | Not Supported | Supported | Supported | Supported | Supported | Not Supported |
| Tile | Supported\*\* |Supported\*\*\*| Supported | Not Supported | Supported | | Tile | Supported\*\* |Supported\*\*\*| Supported | Not Supported | Supported | Supported |
| TopK | Supported | Supported\*\* | Supported | Not Supported | Supported | | TopK | Supported | Supported\*\* | Supported | Not Supported |Supported\*\*\*\*| Supported |
| Unpooling | Supported | Not Supported | Not Supported | Not Supported | Not Supported | | Unpooling | Supported | Not Supported | Not Supported | Not Supported | Not Supported | Not Supported |
| Unsqueeze | Supported | Supported\*\* | Supported | Supported | Supported | | Unsqueeze | Supported | Supported\*\* | Supported | Supported | Supported | Supported |
| Upsampling | Supported | Not Supported | Not Supported | Not Supported | Not Supported | | Upsampling | Supported | Not Supported | Not Supported | Not Supported | Not Supported | Not Supported |
\*- support is limited to the specific parameters. Refer to "Known Layers Limitation" section for the device [from the list of supported](Supported_Devices.md). \*- support is limited to the specific parameters. Refer to "Known Layers Limitation" section for the device [from the list of supported](Supported_Devices.md).
\*\*- support is implemented via [Extensibility mechanism](../../Extensibility_UG/Intro.md). \*\*- support is implemented via [Extensibility mechanism](../../Extensibility_UG/Intro.md).
\*\*\*- supports NCDHW layout. \*\*\*- supports NCDHW layout.
\*\*\*\*- support is implemented via runtime reference.