Merge remote-tracking branch 'upstream/master' into debian-packages
This commit is contained in:
Ilya Lavrenov
commit
1930daa580
@ -42,7 +42,7 @@ Please report questions, issues and suggestions using:
|
||||
\* Other names and brands may be claimed as the property of others.
|
||||
|
||||
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
|
||||
[OpenVINO™ Runtime]:https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_User_Guide.html
|
||||
[OpenVINO™ Runtime]:https://docs.openvino.ai/latest/openvino_docs_OV_UG_OV_Runtime_User_Guide.html
|
||||
[Model Optimizer]:https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html
|
||||
[Post-Training Optimization Tool]:https://docs.openvino.ai/latest/pot_introduction.html
|
||||
[Samples]:https://github.com/openvinotoolkit/openvino/tree/master/samples
|
||||
|
||||
@ -37,7 +37,7 @@ The implementation `CompileNetwork` is fully device-specific.
|
||||
|
||||
The function accepts a const shared pointer to `ngraph::Function` object and performs the following steps:
|
||||
|
||||
1. Applies ngraph passes using `TransformNetwork` function, which defines plugin-specific conversion pipeline. To support low precision inference, the pipeline can include Low Precision Transformations. These transformations are usually hardware specific. You can find how to use and configure Low Precisions Transformations in [Low Precision Transformations](@ref openvino_docs_IE_DG_lpt) guide.
|
||||
1. Applies nGraph passes using `TransformNetwork` function, which defines plugin-specific conversion pipeline. To support low precision inference, the pipeline can include Low Precision Transformations. These transformations are usually hardware specific. You can find how to use and configure Low Precisions Transformations in [Low Precision Transformations](@ref openvino_docs_OV_UG_lpt) guide.
|
||||
2. Maps the transformed graph to a backend specific graph representation (for example, to CPU plugin internal graph representation).
|
||||
3. Allocates and fills memory for graph weights, backend specific memory handles and so on.
|
||||
|
||||
|
||||
@ -56,7 +56,7 @@ Detailed guides
|
||||
* [Build](@ref openvino_docs_ie_plugin_dg_plugin_build) a plugin library using CMake\*
|
||||
* Plugin and its components [testing](@ref openvino_docs_ie_plugin_dg_plugin_testing)
|
||||
* [Quantized networks](@ref openvino_docs_ie_plugin_dg_quantized_networks)
|
||||
* [Low precision transformations](@ref openvino_docs_IE_DG_lpt) guide
|
||||
* [Low precision transformations](@ref openvino_docs_OV_UG_lpt) guide
|
||||
* [Writing OpenVINO™ transformations](@ref openvino_docs_transformations) guide
|
||||
|
||||
API References
|
||||
|
||||
@ -7,12 +7,12 @@
|
||||
:hidden:
|
||||
|
||||
openvino_docs_ie_plugin_dg_quantized_networks
|
||||
openvino_docs_IE_DG_lpt
|
||||
openvino_docs_OV_UG_lpt
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
The guides below provides extra information about specific features of OpenVINO needed for understanding during OpenVINO plugin development:
|
||||
|
||||
* [Quantized networks](@ref openvino_docs_ie_plugin_dg_quantized_networks)
|
||||
* [Low precision transformations](@ref openvino_docs_IE_DG_lpt) guide
|
||||
* [Low precision transformations](@ref openvino_docs_OV_UG_lpt) guide
|
||||
* [Writing OpenVINO™ transformations](@ref openvino_docs_transformations) guide
|
||||
|
||||
@ -5,74 +5,74 @@
|
||||
<tab type="usergroup" url="index.html" title="Developer Guide for Inference Engine Plugin Library">
|
||||
<tab type="user" url="@ref plugin" visibile="yes" title="Implement Plugin Functionality"/>
|
||||
<tab type="user" url="@ref executable_network" visibile="yes" title="Implement Executable Network Functionality">
|
||||
<tab type="usergroup" title="Low Precision Transformations" url="@ref openvino_docs_IE_DG_lpt">
|
||||
<tab type="user" title="Attributes" url="@ref openvino_docs_IE_DG_lpt_attributes">
|
||||
<tab type="user" title="AvgPoolPrecisionPreserved" url="@ref openvino_docs_IE_DG_lpt_AvgPoolPrecisionPreserved"/>
|
||||
<tab type="user" title="IntervalsAlignment" url="@ref openvino_docs_IE_DG_lpt_IntervalsAlignment"/>
|
||||
<tab type="user" title="PerTensorQuantization" url="@ref openvino_docs_IE_DG_lpt_PerTensorQuantization"/>
|
||||
<tab type="user" title="PrecisionPreserved" url="@ref openvino_docs_IE_DG_lpt_PrecisionPreserved"/>
|
||||
<tab type="user" title="Precisions" url="@ref openvino_docs_IE_DG_lpt_Precisions"/>
|
||||
<tab type="user" title="QuantizationAlignment" url="@ref openvino_docs_IE_DG_lpt_QuantizationAlignment"/>
|
||||
<tab type="usergroup" title="Low Precision Transformations" url="@ref openvino_docs_OV_UG_lpt">
|
||||
<tab type="user" title="Attributes" url="@ref openvino_docs_OV_UG_lpt_attributes">
|
||||
<tab type="user" title="AvgPoolPrecisionPreserved" url="@ref openvino_docs_OV_UG_lpt_AvgPoolPrecisionPreserved"/>
|
||||
<tab type="user" title="IntervalsAlignment" url="@ref openvino_docs_OV_UG_lpt_IntervalsAlignment"/>
|
||||
<tab type="user" title="PerTensorQuantization" url="@ref openvino_docs_OV_UG_lpt_PerTensorQuantization"/>
|
||||
<tab type="user" title="PrecisionPreserved" url="@ref openvino_docs_OV_UG_lpt_PrecisionPreserved"/>
|
||||
<tab type="user" title="Precisions" url="@ref openvino_docs_OV_UG_lpt_Precisions"/>
|
||||
<tab type="user" title="QuantizationAlignment" url="@ref openvino_docs_OV_UG_lpt_QuantizationAlignment"/>
|
||||
</tab>
|
||||
<tab type="user" title="Step 1. Prerequisites transformations" url="@ref openvino_docs_IE_DG_lpt_step1_prerequisites">
|
||||
<tab type="user" title="LinOpSequenceFusion" url="@ref openvino_docs_IE_DG_lpt_LinOpSequenceFusion"/>
|
||||
<tab type="user" title="PullReshapeThroughDequantization" url="@ref openvino_docs_IE_DG_lpt_PullReshapeThroughDequantization"/>
|
||||
<tab type="user" title="PullTransposeThroughDequantization" url="@ref openvino_docs_IE_DG_lpt_PullTransposeThroughDequantization"/>
|
||||
<tab type="user" title="Step 1. Prerequisites transformations" url="@ref openvino_docs_OV_UG_lpt_step1_prerequisites">
|
||||
<tab type="user" title="LinOpSequenceFusion" url="@ref openvino_docs_OV_UG_lpt_LinOpSequenceFusion"/>
|
||||
<tab type="user" title="PullReshapeThroughDequantization" url="@ref openvino_docs_OV_UG_lpt_PullReshapeThroughDequantization"/>
|
||||
<tab type="user" title="PullTransposeThroughDequantization" url="@ref openvino_docs_OV_UG_lpt_PullTransposeThroughDequantization"/>
|
||||
</tab>
|
||||
<tab type="user" title="Step 2. Markup transformations" url="@ref openvino_docs_IE_DG_lpt_step2_markup">
|
||||
<tab type="user" title="AlignQuantizationIntervals" url="@ref openvino_docs_IE_DG_lpt_AlignQuantizationIntervals"/>
|
||||
<tab type="user" title="AlignQuantizationParameters" url="@ref openvino_docs_IE_DG_lpt_AlignQuantizationParameters"/>
|
||||
<tab type="user" title="CreateAttribute" url="@ref openvino_docs_IE_DG_lpt_CreateAttribute"/>
|
||||
<tab type="user" title="CreatePrecisionsDependentAttribute" url="@ref openvino_docs_IE_DG_lpt_CreatePrecisionsDependentAttribute"/>
|
||||
<tab type="user" title="MarkupAvgPoolPrecisionPreserved" url="@ref openvino_docs_IE_DG_lpt_MarkupAvgPoolPrecisionPreserved"/>
|
||||
<tab type="user" title="MarkupCanBeQuantized" url="@ref openvino_docs_IE_DG_lpt_MarkupCanBeQuantized"/>
|
||||
<tab type="user" title="MarkupPerTensorQuantization" url="@ref openvino_docs_IE_DG_lpt_MarkupPerTensorQuantization"/>
|
||||
<tab type="user" title="MarkupPrecisions" url="@ref openvino_docs_IE_DG_lpt_MarkupPrecisions"/>
|
||||
<tab type="user" title="PropagatePrecisions" url="@ref openvino_docs_IE_DG_lpt_PropagatePrecisions"/>
|
||||
<tab type="user" title="PropagateThroughPrecisionPreserved" url="@ref openvino_docs_IE_DG_lpt_PropagateThroughPrecisionPreserved"/>
|
||||
<tab type="user" title="PropagateToInput" url="@ref openvino_docs_IE_DG_lpt_PropagateToInput"/>
|
||||
<tab type="user" title="UpdateSharedPrecisionPreserved" url="@ref openvino_docs_IE_DG_lpt_UpdateSharedPrecisionPreserved"/>
|
||||
<tab type="user" title="Step 2. Markup transformations" url="@ref openvino_docs_OV_UG_lpt_step2_markup">
|
||||
<tab type="user" title="AlignQuantizationIntervals" url="@ref openvino_docs_OV_UG_lpt_AlignQuantizationIntervals"/>
|
||||
<tab type="user" title="AlignQuantizationParameters" url="@ref openvino_docs_OV_UG_lpt_AlignQuantizationParameters"/>
|
||||
<tab type="user" title="CreateAttribute" url="@ref openvino_docs_OV_UG_lpt_CreateAttribute"/>
|
||||
<tab type="user" title="CreatePrecisionsDependentAttribute" url="@ref openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute"/>
|
||||
<tab type="user" title="MarkupAvgPoolPrecisionPreserved" url="@ref openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved"/>
|
||||
<tab type="user" title="MarkupCanBeQuantized" url="@ref openvino_docs_OV_UG_lpt_MarkupCanBeQuantized"/>
|
||||
<tab type="user" title="MarkupPerTensorQuantization" url="@ref openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization"/>
|
||||
<tab type="user" title="MarkupPrecisions" url="@ref openvino_docs_OV_UG_lpt_MarkupPrecisions"/>
|
||||
<tab type="user" title="PropagatePrecisions" url="@ref openvino_docs_OV_UG_lpt_PropagatePrecisions"/>
|
||||
<tab type="user" title="PropagateThroughPrecisionPreserved" url="@ref openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved"/>
|
||||
<tab type="user" title="PropagateToInput" url="@ref openvino_docs_OV_UG_lpt_PropagateToInput"/>
|
||||
<tab type="user" title="UpdateSharedPrecisionPreserved" url="@ref openvino_docs_OV_UG_lpt_UpdateSharedPrecisionPreserved"/>
|
||||
</tab>
|
||||
<tab type="user" title="Step 3. Main transformations" url="@ref openvino_docs_IE_DG_lpt_step3_main">
|
||||
<tab type="user" title="AddTransformation" url="@ref openvino_docs_IE_DG_lpt_AddTransformation"/>
|
||||
<tab type="user" title="AvgPoolTransformation" url="@ref openvino_docs_IE_DG_lpt_AvgPoolTransformation"/>
|
||||
<tab type="user" title="ClampTransformation" url="@ref openvino_docs_IE_DG_lpt_ClampTransformation"/>
|
||||
<tab type="user" title="ConcatTransformation" url="@ref openvino_docs_IE_DG_lpt_ConcatTransformation"/>
|
||||
<tab type="user" title="ConvolutionTransformation" url="@ref openvino_docs_IE_DG_lpt_ConvolutionTransformation"/>
|
||||
<tab type="user" title="ConvolutionBackpropDataTransformation" url="@ref openvino_docs_IE_DG_lpt_ConvolutionBackpropDataTransformation"/>
|
||||
<tab type="user" title="DepthToSpaceTransformation" url="@ref openvino_docs_IE_DG_lpt_DepthToSpaceTransformation"/>
|
||||
<tab type="user" title="FakeQuantizeDecompositionTransformation" url="@ref openvino_docs_IE_DG_lpt_FakeQuantizeDecompositionTransformation"/>
|
||||
<tab type="user" title="FakeQuantizeTransformation" url="@ref openvino_docs_IE_DG_lpt_FakeQuantizeTransformation"/>
|
||||
<tab type="user" title="InterpolateTransformation" url="@ref openvino_docs_IE_DG_lpt_InterpolateTransformation"/>
|
||||
<tab type="user" title="GroupConvolutionTransformation" url="@ref openvino_docs_IE_DG_lpt_GroupConvolutionTransformation"/>
|
||||
<tab type="user" title="MatMulTransformation" url="@ref openvino_docs_IE_DG_lpt_MatMulTransformation"/>
|
||||
<tab type="user" title="MaxPoolTransformation" url="@ref openvino_docs_IE_DG_lpt_MaxPoolTransformation"/>
|
||||
<tab type="user" title="MultiplyTransformation" url="@ref openvino_docs_IE_DG_lpt_MultiplyTransformation"/>
|
||||
<tab type="user" title="MVNTransformation" url="@ref openvino_docs_IE_DG_lpt_MVNTransformation"/>
|
||||
<tab type="user" title="NormalizeL2Transformation" url="@ref openvino_docs_IE_DG_lpt_NormalizeL2Transformation"/>
|
||||
<tab type="user" title="PadTransformation" url="@ref openvino_docs_IE_DG_lpt_PadTransformation"/>
|
||||
<tab type="user" title="PReluTransformation" url="@ref openvino_docs_IE_DG_lpt_PReluTransformation"/>
|
||||
<tab type="user" title="ReduceMaxTransformation" url="@ref openvino_docs_IE_DG_lpt_ReduceMaxTransformation"/>
|
||||
<tab type="user" title="ReduceMeanTransformation" url="@ref openvino_docs_IE_DG_lpt_ReduceMeanTransformation"/>
|
||||
<tab type="user" title="ReduceMinTransformation" url="@ref openvino_docs_IE_DG_lpt_ReduceMinTransformation"/>
|
||||
<tab type="user" title="ReduceSumTransformation" url="@ref openvino_docs_IE_DG_lpt_ReduceSumTransformation"/>
|
||||
<tab type="user" title="ReluTransformation" url="@ref openvino_docs_IE_DG_lpt_ReluTransformation"/>
|
||||
<tab type="user" title="ReshapeTransformation" url="@ref openvino_docs_IE_DG_lpt_ReshapeTransformation"/>
|
||||
<tab type="user" title="SqueezeTransformation" url="@ref openvino_docs_IE_DG_lpt_SqueezeTransformation"/>
|
||||
<tab type="user" title="ShuffleChannelsTransformation" url="@ref openvino_docs_IE_DG_lpt_ShuffleChannelsTransformation"/>
|
||||
<tab type="user" title="SplitTransformation" url="@ref openvino_docs_IE_DG_lpt_SplitTransformation"/>
|
||||
<tab type="user" title="StridedSliceTransformation" url="@ref openvino_docs_IE_DG_lpt_StridedSliceTransformation"/>
|
||||
<tab type="user" title="TransposeTransformation" url="@ref openvino_docs_IE_DG_lpt_TransposeTransformation"/>
|
||||
<tab type="user" title="UnsqueezeTransformation" url="@ref openvino_docs_IE_DG_lpt_UnsqueezeTransformation"/>
|
||||
<tab type="user" title="VariadicSplitTransformation" url="@ref openvino_docs_IE_DG_lpt_VariadicSplitTransformation"/>
|
||||
<tab type="user" title="Step 3. Main transformations" url="@ref openvino_docs_OV_UG_lpt_step3_main">
|
||||
<tab type="user" title="AddTransformation" url="@ref openvino_docs_OV_UG_lpt_AddTransformation"/>
|
||||
<tab type="user" title="AvgPoolTransformation" url="@ref openvino_docs_OV_UG_lpt_AvgPoolTransformation"/>
|
||||
<tab type="user" title="ClampTransformation" url="@ref openvino_docs_OV_UG_lpt_ClampTransformation"/>
|
||||
<tab type="user" title="ConcatTransformation" url="@ref openvino_docs_OV_UG_lpt_ConcatTransformation"/>
|
||||
<tab type="user" title="ConvolutionTransformation" url="@ref openvino_docs_OV_UG_lpt_ConvolutionTransformation"/>
|
||||
<tab type="user" title="ConvolutionBackpropDataTransformation" url="@ref openvino_docs_OV_UG_lpt_ConvolutionBackpropDataTransformation"/>
|
||||
<tab type="user" title="DepthToSpaceTransformation" url="@ref openvino_docs_OV_UG_lpt_DepthToSpaceTransformation"/>
|
||||
<tab type="user" title="FakeQuantizeDecompositionTransformation" url="@ref openvino_docs_OV_UG_lpt_FakeQuantizeDecompositionTransformation"/>
|
||||
<tab type="user" title="FakeQuantizeTransformation" url="@ref openvino_docs_OV_UG_lpt_FakeQuantizeTransformation"/>
|
||||
<tab type="user" title="InterpolateTransformation" url="@ref openvino_docs_OV_UG_lpt_InterpolateTransformation"/>
|
||||
<tab type="user" title="GroupConvolutionTransformation" url="@ref openvino_docs_OV_UG_lpt_GroupConvolutionTransformation"/>
|
||||
<tab type="user" title="MatMulTransformation" url="@ref openvino_docs_OV_UG_lpt_MatMulTransformation"/>
|
||||
<tab type="user" title="MaxPoolTransformation" url="@ref openvino_docs_OV_UG_lpt_MaxPoolTransformation"/>
|
||||
<tab type="user" title="MultiplyTransformation" url="@ref openvino_docs_OV_UG_lpt_MultiplyTransformation"/>
|
||||
<tab type="user" title="MVNTransformation" url="@ref openvino_docs_OV_UG_lpt_MVNTransformation"/>
|
||||
<tab type="user" title="NormalizeL2Transformation" url="@ref openvino_docs_OV_UG_lpt_NormalizeL2Transformation"/>
|
||||
<tab type="user" title="PadTransformation" url="@ref openvino_docs_OV_UG_lpt_PadTransformation"/>
|
||||
<tab type="user" title="PReluTransformation" url="@ref openvino_docs_OV_UG_lpt_PReluTransformation"/>
|
||||
<tab type="user" title="ReduceMaxTransformation" url="@ref openvino_docs_OV_UG_lpt_ReduceMaxTransformation"/>
|
||||
<tab type="user" title="ReduceMeanTransformation" url="@ref openvino_docs_OV_UG_lpt_ReduceMeanTransformation"/>
|
||||
<tab type="user" title="ReduceMinTransformation" url="@ref openvino_docs_OV_UG_lpt_ReduceMinTransformation"/>
|
||||
<tab type="user" title="ReduceSumTransformation" url="@ref openvino_docs_OV_UG_lpt_ReduceSumTransformation"/>
|
||||
<tab type="user" title="ReluTransformation" url="@ref openvino_docs_OV_UG_lpt_ReluTransformation"/>
|
||||
<tab type="user" title="ReshapeTransformation" url="@ref openvino_docs_OV_UG_lpt_ReshapeTransformation"/>
|
||||
<tab type="user" title="SqueezeTransformation" url="@ref openvino_docs_OV_UG_lpt_SqueezeTransformation"/>
|
||||
<tab type="user" title="ShuffleChannelsTransformation" url="@ref openvino_docs_OV_UG_lpt_ShuffleChannelsTransformation"/>
|
||||
<tab type="user" title="SplitTransformation" url="@ref openvino_docs_OV_UG_lpt_SplitTransformation"/>
|
||||
<tab type="user" title="StridedSliceTransformation" url="@ref openvino_docs_OV_UG_lpt_StridedSliceTransformation"/>
|
||||
<tab type="user" title="TransposeTransformation" url="@ref openvino_docs_OV_UG_lpt_TransposeTransformation"/>
|
||||
<tab type="user" title="UnsqueezeTransformation" url="@ref openvino_docs_OV_UG_lpt_UnsqueezeTransformation"/>
|
||||
<tab type="user" title="VariadicSplitTransformation" url="@ref openvino_docs_OV_UG_lpt_VariadicSplitTransformation"/>
|
||||
</tab>
|
||||
<tab type="user" title="Step 4. Cleanup transformations" url="@ref openvino_docs_IE_DG_lpt_step4_cleanup">
|
||||
<tab type="user" title="FoldConvertTransformation" url="@ref openvino_docs_IE_DG_lpt_FoldConvertTransformation"/>
|
||||
<tab type="user" title="FoldFakeQuantizeTransformation" url="@ref openvino_docs_IE_DG_lpt_FoldFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="FuseConvertTransformation" url="@ref openvino_docs_IE_DG_lpt_FuseConvertTransformation"/>
|
||||
<tab type="user" title="FuseMultiplyToFakeQuantizeTransformation" url="@ref openvino_docs_IE_DG_lpt_FuseMultiplyToFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="FuseSubtractToFakeQuantizeTransformation" url="@ref openvino_docs_IE_DG_lpt_FuseSubtractToFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="MultiplyToGroupConvolutionTransformation" url="@ref openvino_docs_IE_DG_lpt_MultiplyToGroupConvolutionTransformation"/>
|
||||
<tab type="user" title="Step 4. Cleanup transformations" url="@ref openvino_docs_OV_UG_lpt_step4_cleanup">
|
||||
<tab type="user" title="FoldConvertTransformation" url="@ref openvino_docs_OV_UG_lpt_FoldConvertTransformation"/>
|
||||
<tab type="user" title="FoldFakeQuantizeTransformation" url="@ref openvino_docs_OV_UG_lpt_FoldFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="FuseConvertTransformation" url="@ref openvino_docs_OV_UG_lpt_FuseConvertTransformation"/>
|
||||
<tab type="user" title="FuseMultiplyToFakeQuantizeTransformation" url="@ref openvino_docs_OV_UG_lpt_FuseMultiplyToFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="FuseSubtractToFakeQuantizeTransformation" url="@ref openvino_docs_OV_UG_lpt_FuseSubtractToFakeQuantizeTransformation"/>
|
||||
<tab type="user" title="MultiplyToGroupConvolutionTransformation" url="@ref openvino_docs_OV_UG_lpt_MultiplyToGroupConvolutionTransformation"/>
|
||||
</tab>
|
||||
</tab>
|
||||
</tab>
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# AvgPoolPrecisionPreserved attribute {#openvino_docs_IE_DG_lpt_AvgPoolPrecisionPreserved}
|
||||
# AvgPoolPrecisionPreserved attribute {#openvino_docs_OV_UG_lpt_AvgPoolPrecisionPreserved}
|
||||
|
||||
ngraph::AvgPoolPrecisionPreservedAttribute class represents the `AvgPoolPrecisionPreserved` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# IntervalsAlignment attribute {#openvino_docs_IE_DG_lpt_IntervalsAlignment}
|
||||
# IntervalsAlignment attribute {#openvino_docs_OV_UG_lpt_IntervalsAlignment}
|
||||
|
||||
ngraph::IntervalsAlignmentAttribute class represents the `IntervalsAlignment` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# PerTensorQuantization attribute {#openvino_docs_IE_DG_lpt_PerTensorQuantization}
|
||||
# PerTensorQuantization attribute {#openvino_docs_OV_UG_lpt_PerTensorQuantization}
|
||||
|
||||
ngraph::PerTensorQuantizationAttribute class represents the `PerTensorQuantization` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# PrecisionPreserved attribute {#openvino_docs_IE_DG_lpt_PrecisionPreserved}
|
||||
# PrecisionPreserved attribute {#openvino_docs_OV_UG_lpt_PrecisionPreserved}
|
||||
|
||||
ngraph::PrecisionPreservedAttribute class represents the `PrecisionPreserved` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Precisions attribute {#openvino_docs_IE_DG_lpt_Precisions}
|
||||
# Precisions attribute {#openvino_docs_OV_UG_lpt_Precisions}
|
||||
|
||||
ngraph::PrecisionsAttribute class represents the `Precisions` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# QuantizationAlignment attribute {#openvino_docs_IE_DG_lpt_QuantizationAlignment}
|
||||
# QuantizationAlignment attribute {#openvino_docs_OV_UG_lpt_QuantizationAlignment}
|
||||
|
||||
ngraph::QuantizationAlignmentAttribute class represents the `QuantizationAlignment` attribute.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# OpenVINO™ Low Precision Transformations {#openvino_docs_IE_DG_lpt}
|
||||
# OpenVINO™ Low Precision Transformations {#openvino_docs_OV_UG_lpt}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -7,13 +7,13 @@
|
||||
:caption: Low Precision Transformations
|
||||
:hidden:
|
||||
|
||||
Low Precision Transformations <openvino_docs_IE_DG_lpt>
|
||||
Low Precision Transformations <openvino_docs_OV_UG_lpt>
|
||||
|
||||
Attributes <openvino_docs_IE_DG_lpt_attributes>
|
||||
Step 1. Prerequisites transformations <openvino_docs_IE_DG_lpt_step1_prerequisites>
|
||||
Step 2. Markup transformations <openvino_docs_IE_DG_lpt_step2_markup>
|
||||
Step 3. Main transformations <openvino_docs_IE_DG_lpt_step3_main>
|
||||
Step 4. Cleanup transformations <openvino_docs_IE_DG_lpt_step4_cleanup>
|
||||
Attributes <openvino_docs_OV_UG_lpt_attributes>
|
||||
Step 1. Prerequisites transformations <openvino_docs_OV_UG_lpt_step1_prerequisites>
|
||||
Step 2. Markup transformations <openvino_docs_OV_UG_lpt_step2_markup>
|
||||
Step 3. Main transformations <openvino_docs_OV_UG_lpt_step3_main>
|
||||
Step 4. Cleanup transformations <openvino_docs_OV_UG_lpt_step4_cleanup>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
@ -111,63 +111,63 @@ Inside each step LPT transformations handle input model operation by operation,
|
||||
As result, usually all operations are inferred by plugin in low precision. If plugin doesn't support an operation inference in low precision, then corresponding LPT transformation can be disabled, and input tensor precisions for the operation will not be changed. In this case the operation is inferred in the original precision.
|
||||
|
||||
Low precision transformations pipeline includes four steps:
|
||||
* [Step #1: Prerequisites](@ref openvino_docs_IE_DG_lpt_step1_prerequisites)
|
||||
* [Step #2: Markup transformations](@ref openvino_docs_IE_DG_lpt_step2_markup)
|
||||
* [Step #3: Main transformations](@ref openvino_docs_IE_DG_lpt_step3_main)
|
||||
* [Step #4: Cleanup transformations](@ref openvino_docs_IE_DG_lpt_step4_cleanup)
|
||||
* [Step #1: Prerequisites](@ref openvino_docs_OV_UG_lpt_step1_prerequisites)
|
||||
* [Step #2: Markup transformations](@ref openvino_docs_OV_UG_lpt_step2_markup)
|
||||
* [Step #3: Main transformations](@ref openvino_docs_OV_UG_lpt_step3_main)
|
||||
* [Step #4: Cleanup transformations](@ref openvino_docs_OV_UG_lpt_step4_cleanup)
|
||||
|
||||
### Step 1. Prerequisites
|
||||
This step fuses and propagates some operations in the model to prepare for the next step. It is required for OpenVINO plugins. Transformations:
|
||||
* [PullReshapeThroughDequantization](@ref openvino_docs_IE_DG_lpt_PullReshapeThroughDequantization)
|
||||
* [PullTransposeThroughDequantization](@ref openvino_docs_IE_DG_lpt_PullTransposeThroughDequantization)
|
||||
* [LinOpSequenceFusion](@ref openvino_docs_IE_DG_lpt_LinOpSequenceFusion)
|
||||
* [PullReshapeThroughDequantization](@ref openvino_docs_OV_UG_lpt_PullReshapeThroughDequantization)
|
||||
* [PullTransposeThroughDequantization](@ref openvino_docs_OV_UG_lpt_PullTransposeThroughDequantization)
|
||||
* [LinOpSequenceFusion](@ref openvino_docs_OV_UG_lpt_LinOpSequenceFusion)
|
||||
|
||||
The model on this step is changed. There are more details in developer guide [Prerequisites transformations](@ref openvino_docs_IE_DG_lpt_step1_prerequisites).
|
||||
The model on this step is changed. There are more details in developer guide [Prerequisites transformations](@ref openvino_docs_OV_UG_lpt_step1_prerequisites).
|
||||
|
||||
### Step 2. Markup
|
||||
This step creates runtime attributes for operations. These attributes will be used in next step. Transformations:
|
||||
* [MarkupCanBeQuantized](@ref openvino_docs_IE_DG_lpt_MarkupCanBeQuantized)
|
||||
* [MarkupPrecisions](@ref openvino_docs_IE_DG_lpt_MarkupPrecisions)
|
||||
* [MarkupPerTensorQuantization](@ref openvino_docs_IE_DG_lpt_MarkupPerTensorQuantization)
|
||||
* [MarkupAvgPoolPrecisionPreserved](@ref openvino_docs_IE_DG_lpt_MarkupAvgPoolPrecisionPreserved)
|
||||
* [PropagatePrecisions](@ref openvino_docs_IE_DG_lpt_PropagatePrecisions)
|
||||
* [AlignQuantizationIntervals](@ref openvino_docs_IE_DG_lpt_AlignQuantizationIntervals)
|
||||
* [AlignQuantizationParameters](@ref openvino_docs_IE_DG_lpt_AlignQuantizationParameters)
|
||||
* [MarkupCanBeQuantized](@ref openvino_docs_OV_UG_lpt_MarkupCanBeQuantized)
|
||||
* [MarkupPrecisions](@ref openvino_docs_OV_UG_lpt_MarkupPrecisions)
|
||||
* [MarkupPerTensorQuantization](@ref openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization)
|
||||
* [MarkupAvgPoolPrecisionPreserved](@ref openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved)
|
||||
* [PropagatePrecisions](@ref openvino_docs_OV_UG_lpt_PropagatePrecisions)
|
||||
* [AlignQuantizationIntervals](@ref openvino_docs_OV_UG_lpt_AlignQuantizationIntervals)
|
||||
* [AlignQuantizationParameters](@ref openvino_docs_OV_UG_lpt_AlignQuantizationParameters)
|
||||
|
||||
The model on this step is changed: only new attributes are added to some operations. There are more details in developer guide [Markup transformations](@ref openvino_docs_IE_DG_lpt_step2_markup).
|
||||
The model on this step is changed: only new attributes are added to some operations. There are more details in developer guide [Markup transformations](@ref openvino_docs_OV_UG_lpt_step2_markup).
|
||||
|
||||
### Step 3. Main transformations, FakeQuantize decomposition and dequantization operations handling
|
||||
This step has the most transformations. These transformations can be separated in two groups: decomposition transformation and dequantization operations handling. There are more details in developer guide [Main transformations](@ref openvino_docs_IE_DG_lpt_step3_main). Transformations:
|
||||
* [AddTransformation](@ref openvino_docs_IE_DG_lpt_AddTransformation)
|
||||
* [AvgPoolTransformation](@ref openvino_docs_IE_DG_lpt_AvgPoolTransformation)
|
||||
* [ClampTransformation](@ref openvino_docs_IE_DG_lpt_AvgPoolTransformation)
|
||||
* [ConcatTransformation](@ref openvino_docs_IE_DG_lpt_ConcatTransformation)
|
||||
* [ConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_ConvolutionTransformation)
|
||||
* [ConvolutionBackpropDataTransformation](@ref openvino_docs_IE_DG_lpt_ConvolutionBackpropDataTransformation)
|
||||
* [DepthToSpaceTransformation](@ref openvino_docs_IE_DG_lpt_DepthToSpaceTransformation)
|
||||
* [FakeQuantizeDecompositionTransformation](@ref openvino_docs_IE_DG_lpt_FakeQuantizeDecompositionTransformation)
|
||||
* [FakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FakeQuantizeTransformation)
|
||||
* [InterpolateTransformation](@ref openvino_docs_IE_DG_lpt_InterpolateTransformation)
|
||||
* [GroupConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_GroupConvolutionTransformation)
|
||||
* [MatMulTransformation](@ref openvino_docs_IE_DG_lpt_MatMulTransformation)
|
||||
* [MaxPoolTransformation](@ref openvino_docs_IE_DG_lpt_MaxPoolTransformation)
|
||||
* [MultiplyTransformation](@ref openvino_docs_IE_DG_lpt_MultiplyTransformation)
|
||||
* [MVNTransformation](@ref openvino_docs_IE_DG_lpt_MVNTransformation)
|
||||
* [NormalizeL2Transformation](@ref openvino_docs_IE_DG_lpt_NormalizeL2Transformation)
|
||||
* [PReluTransformation](@ref openvino_docs_IE_DG_lpt_PReluTransformation)
|
||||
* [ReduceMaxTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMaxTransformation)
|
||||
* [ReduceMeanTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMeanTransformation)
|
||||
* [ReduceMinTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMinTransformation)
|
||||
* [ReduceSumTransformation](@ref openvino_docs_IE_DG_lpt_ReduceSumTransformation)
|
||||
* [ReluTransformation](@ref openvino_docs_IE_DG_lpt_ReluTransformation)
|
||||
* [ReshapeTransformation](@ref openvino_docs_IE_DG_lpt_ReshapeTransformation)
|
||||
* [SqueezeTransformation](@ref openvino_docs_IE_DG_lpt_SqueezeTransformation)
|
||||
* [ShuffleChannelsTransformation](@ref openvino_docs_IE_DG_lpt_ShuffleChannelsTransformation)
|
||||
* [SplitTransformation](@ref openvino_docs_IE_DG_lpt_SplitTransformation)
|
||||
* [StridedSliceTransformation](@ref openvino_docs_IE_DG_lpt_StridedSliceTransformation)
|
||||
* [TransposeTransformation](@ref openvino_docs_IE_DG_lpt_TransposeTransformation)
|
||||
* [UnsqueezeTransformation](@ref openvino_docs_IE_DG_lpt_UnsqueezeTransformation)
|
||||
* [VariadicSplitTransformation](@ref openvino_docs_IE_DG_lpt_VariadicSplitTransformation)
|
||||
This step has the most transformations. These transformations can be separated in two groups: decomposition transformation and dequantization operations handling. There are more details in developer guide [Main transformations](@ref openvino_docs_OV_UG_lpt_step3_main). Transformations:
|
||||
* [AddTransformation](@ref openvino_docs_OV_UG_lpt_AddTransformation)
|
||||
* [AvgPoolTransformation](@ref openvino_docs_OV_UG_lpt_AvgPoolTransformation)
|
||||
* [ClampTransformation](@ref openvino_docs_OV_UG_lpt_AvgPoolTransformation)
|
||||
* [ConcatTransformation](@ref openvino_docs_OV_UG_lpt_ConcatTransformation)
|
||||
* [ConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_ConvolutionTransformation)
|
||||
* [ConvolutionBackpropDataTransformation](@ref openvino_docs_OV_UG_lpt_ConvolutionBackpropDataTransformation)
|
||||
* [DepthToSpaceTransformation](@ref openvino_docs_OV_UG_lpt_DepthToSpaceTransformation)
|
||||
* [FakeQuantizeDecompositionTransformation](@ref openvino_docs_OV_UG_lpt_FakeQuantizeDecompositionTransformation)
|
||||
* [FakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FakeQuantizeTransformation)
|
||||
* [InterpolateTransformation](@ref openvino_docs_OV_UG_lpt_InterpolateTransformation)
|
||||
* [GroupConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_GroupConvolutionTransformation)
|
||||
* [MatMulTransformation](@ref openvino_docs_OV_UG_lpt_MatMulTransformation)
|
||||
* [MaxPoolTransformation](@ref openvino_docs_OV_UG_lpt_MaxPoolTransformation)
|
||||
* [MultiplyTransformation](@ref openvino_docs_OV_UG_lpt_MultiplyTransformation)
|
||||
* [MVNTransformation](@ref openvino_docs_OV_UG_lpt_MVNTransformation)
|
||||
* [NormalizeL2Transformation](@ref openvino_docs_OV_UG_lpt_NormalizeL2Transformation)
|
||||
* [PReluTransformation](@ref openvino_docs_OV_UG_lpt_PReluTransformation)
|
||||
* [ReduceMaxTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMaxTransformation)
|
||||
* [ReduceMeanTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMeanTransformation)
|
||||
* [ReduceMinTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMinTransformation)
|
||||
* [ReduceSumTransformation](@ref openvino_docs_OV_UG_lpt_ReduceSumTransformation)
|
||||
* [ReluTransformation](@ref openvino_docs_OV_UG_lpt_ReluTransformation)
|
||||
* [ReshapeTransformation](@ref openvino_docs_OV_UG_lpt_ReshapeTransformation)
|
||||
* [SqueezeTransformation](@ref openvino_docs_OV_UG_lpt_SqueezeTransformation)
|
||||
* [ShuffleChannelsTransformation](@ref openvino_docs_OV_UG_lpt_ShuffleChannelsTransformation)
|
||||
* [SplitTransformation](@ref openvino_docs_OV_UG_lpt_SplitTransformation)
|
||||
* [StridedSliceTransformation](@ref openvino_docs_OV_UG_lpt_StridedSliceTransformation)
|
||||
* [TransposeTransformation](@ref openvino_docs_OV_UG_lpt_TransposeTransformation)
|
||||
* [UnsqueezeTransformation](@ref openvino_docs_OV_UG_lpt_UnsqueezeTransformation)
|
||||
* [VariadicSplitTransformation](@ref openvino_docs_OV_UG_lpt_VariadicSplitTransformation)
|
||||
|
||||
#### Decomposition transformations
|
||||
Decomposition transformations decompose the `FakeQuantize` operation to: quantize (`FakeQuantize` with low precision output) and dequantization operations (opposite to quantize, with low precision input and the original precision output). For dequantization operations LPT uses three operations: `Convert`, `Subtract` and `Multiply`. Element-wise operations `Subtract` and `Multiply` have constants on the second branches. If dequantization operations are not handled at the end of LPT pipeline, then they will be fused back to the `FakeQuantize`.
|
||||
@ -193,14 +193,14 @@ Original `Convolution` operation in FP32 with dequantization operations before:
|
||||
|
||||
### Step 4: Cleanup of the result model
|
||||
LPT cleanup transformations is final stage in LPT pipeline. In this step LPT transformations clean up the result model to avoid not handled dequantization operations: fuse dequantization operations if possible (fuse at least `Convert` operations if not) to other model operations to cleanup result model. Transformations:
|
||||
* [FoldConvertTransformation](@ref openvino_docs_IE_DG_lpt_FoldConvertTransformation)
|
||||
* [FoldFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FoldFakeQuantizeTransformation)
|
||||
* [FuseConvertTransformation](@ref openvino_docs_IE_DG_lpt_FuseConvertTransformation)
|
||||
* [FuseMultiplyToFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FuseMultiplyToFakeQuantizeTransformation)
|
||||
* [FuseSubtractToFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FuseSubtractToFakeQuantizeTransformation)
|
||||
* [MultiplyToGroupConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_MultiplyToGroupConvolutionTransformation)
|
||||
* [FoldConvertTransformation](@ref openvino_docs_OV_UG_lpt_FoldConvertTransformation)
|
||||
* [FoldFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FoldFakeQuantizeTransformation)
|
||||
* [FuseConvertTransformation](@ref openvino_docs_OV_UG_lpt_FuseConvertTransformation)
|
||||
* [FuseMultiplyToFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FuseMultiplyToFakeQuantizeTransformation)
|
||||
* [FuseSubtractToFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FuseSubtractToFakeQuantizeTransformation)
|
||||
* [MultiplyToGroupConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_MultiplyToGroupConvolutionTransformation)
|
||||
|
||||
There are more details in developer guide [Cleanup transformations](@ref openvino_docs_IE_DG_lpt_step4_cleanup).
|
||||
There are more details in developer guide [Cleanup transformations](@ref openvino_docs_OV_UG_lpt_step4_cleanup).
|
||||
|
||||
`FakeQuantize` operation with not handled dequantization operations:
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Attributes {#openvino_docs_IE_DG_lpt_attributes}
|
||||
# Attributes {#openvino_docs_OV_UG_lpt_attributes}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -7,12 +7,12 @@
|
||||
:caption: Attributes
|
||||
:hidden:
|
||||
|
||||
AvgPoolPrecisionPreserved <openvino_docs_IE_DG_lpt_AvgPoolPrecisionPreserved>
|
||||
IntervalsAlignment <openvino_docs_IE_DG_lpt_IntervalsAlignment>
|
||||
PerTensorQuantization <openvino_docs_IE_DG_lpt_PerTensorQuantization>
|
||||
PrecisionPreserved <openvino_docs_IE_DG_lpt_PrecisionPreserved>
|
||||
Precisions <openvino_docs_IE_DG_lpt_Precisions>
|
||||
QuantizationAlignment <openvino_docs_IE_DG_lpt_QuantizationAlignment>
|
||||
AvgPoolPrecisionPreserved <openvino_docs_OV_UG_lpt_AvgPoolPrecisionPreserved>
|
||||
IntervalsAlignment <openvino_docs_OV_UG_lpt_IntervalsAlignment>
|
||||
PerTensorQuantization <openvino_docs_OV_UG_lpt_PerTensorQuantization>
|
||||
PrecisionPreserved <openvino_docs_OV_UG_lpt_PrecisionPreserved>
|
||||
Precisions <openvino_docs_OV_UG_lpt_Precisions>
|
||||
QuantizationAlignment <openvino_docs_OV_UG_lpt_QuantizationAlignment>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
@ -20,12 +20,12 @@
|
||||
|
||||
| Name | Target | Required | Mutable |
|
||||
|-------------------------------------------------------------------------------------|------------------------|----------|---------|
|
||||
| [AvgPoolPrecisionPreserved](@ref openvino_docs_IE_DG_lpt_AvgPoolPrecisionPreserved) | Precision | No | Yes |
|
||||
| [IntervalsAlignment](@ref openvino_docs_IE_DG_lpt_IntervalsAlignment) | Quantization interval | Yes | Yes |
|
||||
| [PerTensorQuantization](@ref openvino_docs_IE_DG_lpt_PerTensorQuantization) | Precision | Yes | No |
|
||||
| [PrecisionPreserved](@ref openvino_docs_IE_DG_lpt_PrecisionPreserved) | Precision | Yes | Yes |
|
||||
| [Precisions](@ref openvino_docs_IE_DG_lpt_Precisions) | Precision | Yes | Yes |
|
||||
| [QuantizationAlignment](@ref openvino_docs_IE_DG_lpt_QuantizationAlignment) | Quantization alignment | Yes | Yes |
|
||||
| [AvgPoolPrecisionPreserved](@ref openvino_docs_OV_UG_lpt_AvgPoolPrecisionPreserved) | Precision | No | Yes |
|
||||
| [IntervalsAlignment](@ref openvino_docs_OV_UG_lpt_IntervalsAlignment) | Quantization interval | Yes | Yes |
|
||||
| [PerTensorQuantization](@ref openvino_docs_OV_UG_lpt_PerTensorQuantization) | Precision | Yes | No |
|
||||
| [PrecisionPreserved](@ref openvino_docs_OV_UG_lpt_PrecisionPreserved) | Precision | Yes | Yes |
|
||||
| [Precisions](@ref openvino_docs_OV_UG_lpt_Precisions) | Precision | Yes | Yes |
|
||||
| [QuantizationAlignment](@ref openvino_docs_OV_UG_lpt_QuantizationAlignment) | Quantization alignment | Yes | Yes |
|
||||
|
||||
> `Target` attribute group defines attribute usage during model transformation for the best performance:
|
||||
> - `Precision` - the attribute defines the most optimal output port precision.
|
||||
|
||||
@ -1,6 +1,6 @@
|
||||
# Step 1. Prerequisites Transformations {#openvino_docs_IE_DG_lpt_step1_prerequisites}
|
||||
# Step 1. Prerequisites Transformations {#openvino_docs_OV_UG_lpt_step1_prerequisites}
|
||||
|
||||
Prerequisites transformations are optional. The transformations prepare a model before running other low precision transformations. The transformations do not operate with dequantization operations or update precisions. Prerequisites transformations include:
|
||||
* [PullReshapeThroughDequantization](@ref openvino_docs_IE_DG_lpt_PullReshapeThroughDequantization)
|
||||
* [PullTransposeThroughDequantization](@ref openvino_docs_IE_DG_lpt_PullTransposeThroughDequantization)
|
||||
* [LinOpSequenceFusion](@ref openvino_docs_IE_DG_lpt_LinOpSequenceFusion)
|
||||
* [PullReshapeThroughDequantization](@ref openvino_docs_OV_UG_lpt_PullReshapeThroughDequantization)
|
||||
* [PullTransposeThroughDequantization](@ref openvino_docs_OV_UG_lpt_PullTransposeThroughDequantization)
|
||||
* [LinOpSequenceFusion](@ref openvino_docs_OV_UG_lpt_LinOpSequenceFusion)
|
||||
@ -1,14 +1,14 @@
|
||||
# Step 2. Markup Transformations {#openvino_docs_IE_DG_lpt_step2_markup}
|
||||
# Step 2. Markup Transformations {#openvino_docs_OV_UG_lpt_step2_markup}
|
||||
|
||||
This step defines the optimal `FakeQuantize` decomposition precisions for the best inference performance via operations markup with runtime attribute instances. Attributes are created for input and output ports and operations. Transformations do not change the operation output port precisions. A model markup low precision logic is decomposed and implemented into the following common markup transformations. The order of transformations is important:
|
||||
|
||||
1. [MarkupCanBeQuantized](@ref openvino_docs_IE_DG_lpt_MarkupCanBeQuantized)
|
||||
2. [MarkupPrecisions](@ref openvino_docs_IE_DG_lpt_MarkupPrecisions)
|
||||
3. [MarkupPerTensorQuantization](@ref openvino_docs_IE_DG_lpt_MarkupPerTensorQuantization)
|
||||
4. [MarkupAvgPoolPrecisionPreserved](@ref openvino_docs_IE_DG_lpt_MarkupAvgPoolPrecisionPreserved)
|
||||
5. [PropagatePrecisions](@ref openvino_docs_IE_DG_lpt_PropagatePrecisions)
|
||||
6. [AlignQuantizationIntervals](@ref openvino_docs_IE_DG_lpt_AlignQuantizationIntervals)
|
||||
7. [AlignQuantizationParameters](@ref openvino_docs_IE_DG_lpt_AlignQuantizationParameters)
|
||||
1. [MarkupCanBeQuantized](@ref openvino_docs_OV_UG_lpt_MarkupCanBeQuantized)
|
||||
2. [MarkupPrecisions](@ref openvino_docs_OV_UG_lpt_MarkupPrecisions)
|
||||
3. [MarkupPerTensorQuantization](@ref openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization)
|
||||
4. [MarkupAvgPoolPrecisionPreserved](@ref openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved)
|
||||
5. [PropagatePrecisions](@ref openvino_docs_OV_UG_lpt_PropagatePrecisions)
|
||||
6. [AlignQuantizationIntervals](@ref openvino_docs_OV_UG_lpt_AlignQuantizationIntervals)
|
||||
7. [AlignQuantizationParameters](@ref openvino_docs_OV_UG_lpt_AlignQuantizationParameters)
|
||||
|
||||
The table of transformations and used attributes:
|
||||
|
||||
@ -25,11 +25,11 @@ The table of transformations and used attributes:
|
||||
> **Note:** the same type of attribute instances can be created in different transformations. This approach is the result of the transformation single-responsibility principle. For example, `Precision` attribute instances are created in `MarkupCanBeQuantized` and `MarkupPrecisions` transformations, but the reasons for their creation are different
|
||||
|
||||
Common markup transformations can be decomposed into simpler utility markup transformations. The order of Markup utility transformations is not important:
|
||||
* [CreateAttribute](@ref openvino_docs_IE_DG_lpt_CreateAttribute)
|
||||
* [CreatePrecisionsDependentAttribute](@ref openvino_docs_IE_DG_lpt_CreatePrecisionsDependentAttribute)
|
||||
* [PropagateThroughPrecisionPreserved](@ref openvino_docs_IE_DG_lpt_PropagateThroughPrecisionPreserved)
|
||||
* [PropagateToInput](@ref openvino_docs_IE_DG_lpt_PropagateToInput)
|
||||
* [UpdateSharedPrecisionPreserved](@ref openvino_docs_IE_DG_lpt_UpdateSharedPrecisionPreserved)
|
||||
* [CreateAttribute](@ref openvino_docs_OV_UG_lpt_CreateAttribute)
|
||||
* [CreatePrecisionsDependentAttribute](@ref openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute)
|
||||
* [PropagateThroughPrecisionPreserved](@ref openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved)
|
||||
* [PropagateToInput](@ref openvino_docs_OV_UG_lpt_PropagateToInput)
|
||||
* [UpdateSharedPrecisionPreserved](@ref openvino_docs_OV_UG_lpt_UpdateSharedPrecisionPreserved)
|
||||
|
||||
Let's explore all transformations and their relations in detail, using one and the same model:
|
||||
|
||||
|
||||
@ -1,36 +1,36 @@
|
||||
# Step 3. Main Transformations {#openvino_docs_IE_DG_lpt_step3_main}
|
||||
# Step 3. Main Transformations {#openvino_docs_OV_UG_lpt_step3_main}
|
||||
|
||||
Main transformations are the majority of low precision transformations. Transformations operate with dequantization operations. Main transformations include:
|
||||
* [AddTransformation](@ref openvino_docs_IE_DG_lpt_AddTransformation)
|
||||
* [AvgPoolTransformation](@ref openvino_docs_IE_DG_lpt_AvgPoolTransformation)
|
||||
* [ClampTransformation](@ref openvino_docs_IE_DG_lpt_AvgPoolTransformation)
|
||||
* [ConcatTransformation](@ref openvino_docs_IE_DG_lpt_ConcatTransformation)
|
||||
* [ConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_ConvolutionTransformation)
|
||||
* [ConvolutionBackpropDataTransformation](@ref openvino_docs_IE_DG_lpt_ConvolutionBackpropDataTransformation)
|
||||
* [DepthToSpaceTransformation](@ref openvino_docs_IE_DG_lpt_DepthToSpaceTransformation)
|
||||
* [FakeQuantizeDecompositionTransformation](@ref openvino_docs_IE_DG_lpt_FakeQuantizeDecompositionTransformation)
|
||||
* [FakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FakeQuantizeTransformation)
|
||||
* [InterpolateTransformation](@ref openvino_docs_IE_DG_lpt_InterpolateTransformation)
|
||||
* [GroupConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_GroupConvolutionTransformation)
|
||||
* [MatMulTransformation](@ref openvino_docs_IE_DG_lpt_MatMulTransformation)
|
||||
* [MaxPoolTransformation](@ref openvino_docs_IE_DG_lpt_MaxPoolTransformation)
|
||||
* [MultiplyTransformation](@ref openvino_docs_IE_DG_lpt_MultiplyTransformation)
|
||||
* [MVNTransformation](@ref openvino_docs_IE_DG_lpt_MVNTransformation)
|
||||
* [NormalizeL2Transformation](@ref openvino_docs_IE_DG_lpt_NormalizeL2Transformation)
|
||||
* [PReluTransformation](@ref openvino_docs_IE_DG_lpt_PReluTransformation)
|
||||
* [ReduceMaxTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMaxTransformation)
|
||||
* [ReduceMeanTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMeanTransformation)
|
||||
* [ReduceMinTransformation](@ref openvino_docs_IE_DG_lpt_ReduceMinTransformation)
|
||||
* [ReduceSumTransformation](@ref openvino_docs_IE_DG_lpt_ReduceSumTransformation)
|
||||
* [ReluTransformation](@ref openvino_docs_IE_DG_lpt_ReluTransformation)
|
||||
* [ReshapeTransformation](@ref openvino_docs_IE_DG_lpt_ReshapeTransformation)
|
||||
* [SqueezeTransformation](@ref openvino_docs_IE_DG_lpt_SqueezeTransformation)
|
||||
* [ShuffleChannelsTransformation](@ref openvino_docs_IE_DG_lpt_ShuffleChannelsTransformation)
|
||||
* [SplitTransformation](@ref openvino_docs_IE_DG_lpt_SplitTransformation)
|
||||
* [StridedSliceTransformation](@ref openvino_docs_IE_DG_lpt_StridedSliceTransformation)
|
||||
* [TransposeTransformation](@ref openvino_docs_IE_DG_lpt_TransposeTransformation)
|
||||
* [UnsqueezeTransformation](@ref openvino_docs_IE_DG_lpt_UnsqueezeTransformation)
|
||||
* [VariadicSplitTransformation](@ref openvino_docs_IE_DG_lpt_VariadicSplitTransformation)
|
||||
* [AddTransformation](@ref openvino_docs_OV_UG_lpt_AddTransformation)
|
||||
* [AvgPoolTransformation](@ref openvino_docs_OV_UG_lpt_AvgPoolTransformation)
|
||||
* [ClampTransformation](@ref openvino_docs_OV_UG_lpt_AvgPoolTransformation)
|
||||
* [ConcatTransformation](@ref openvino_docs_OV_UG_lpt_ConcatTransformation)
|
||||
* [ConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_ConvolutionTransformation)
|
||||
* [ConvolutionBackpropDataTransformation](@ref openvino_docs_OV_UG_lpt_ConvolutionBackpropDataTransformation)
|
||||
* [DepthToSpaceTransformation](@ref openvino_docs_OV_UG_lpt_DepthToSpaceTransformation)
|
||||
* [FakeQuantizeDecompositionTransformation](@ref openvino_docs_OV_UG_lpt_FakeQuantizeDecompositionTransformation)
|
||||
* [FakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FakeQuantizeTransformation)
|
||||
* [InterpolateTransformation](@ref openvino_docs_OV_UG_lpt_InterpolateTransformation)
|
||||
* [GroupConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_GroupConvolutionTransformation)
|
||||
* [MatMulTransformation](@ref openvino_docs_OV_UG_lpt_MatMulTransformation)
|
||||
* [MaxPoolTransformation](@ref openvino_docs_OV_UG_lpt_MaxPoolTransformation)
|
||||
* [MultiplyTransformation](@ref openvino_docs_OV_UG_lpt_MultiplyTransformation)
|
||||
* [MVNTransformation](@ref openvino_docs_OV_UG_lpt_MVNTransformation)
|
||||
* [NormalizeL2Transformation](@ref openvino_docs_OV_UG_lpt_NormalizeL2Transformation)
|
||||
* [PReluTransformation](@ref openvino_docs_OV_UG_lpt_PReluTransformation)
|
||||
* [ReduceMaxTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMaxTransformation)
|
||||
* [ReduceMeanTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMeanTransformation)
|
||||
* [ReduceMinTransformation](@ref openvino_docs_OV_UG_lpt_ReduceMinTransformation)
|
||||
* [ReduceSumTransformation](@ref openvino_docs_OV_UG_lpt_ReduceSumTransformation)
|
||||
* [ReluTransformation](@ref openvino_docs_OV_UG_lpt_ReluTransformation)
|
||||
* [ReshapeTransformation](@ref openvino_docs_OV_UG_lpt_ReshapeTransformation)
|
||||
* [SqueezeTransformation](@ref openvino_docs_OV_UG_lpt_SqueezeTransformation)
|
||||
* [ShuffleChannelsTransformation](@ref openvino_docs_OV_UG_lpt_ShuffleChannelsTransformation)
|
||||
* [SplitTransformation](@ref openvino_docs_OV_UG_lpt_SplitTransformation)
|
||||
* [StridedSliceTransformation](@ref openvino_docs_OV_UG_lpt_StridedSliceTransformation)
|
||||
* [TransposeTransformation](@ref openvino_docs_OV_UG_lpt_TransposeTransformation)
|
||||
* [UnsqueezeTransformation](@ref openvino_docs_OV_UG_lpt_UnsqueezeTransformation)
|
||||
* [VariadicSplitTransformation](@ref openvino_docs_OV_UG_lpt_VariadicSplitTransformation)
|
||||
|
||||
Let's explore some main transformations on the example model. Original model:
|
||||
|
||||
|
||||
@ -1,8 +1,8 @@
|
||||
# Step 4. Cleanup Transformations {#openvino_docs_IE_DG_lpt_step4_cleanup}
|
||||
# Step 4. Cleanup Transformations {#openvino_docs_OV_UG_lpt_step4_cleanup}
|
||||
|
||||
* [FoldConvertTransformation](@ref openvino_docs_IE_DG_lpt_FoldConvertTransformation)
|
||||
* [FoldFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FoldFakeQuantizeTransformation)
|
||||
* [FuseConvertTransformation](@ref openvino_docs_IE_DG_lpt_FuseConvertTransformation)
|
||||
* [FuseMultiplyToFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FuseMultiplyToFakeQuantizeTransformation)
|
||||
* [FuseSubtractToFakeQuantizeTransformation](@ref openvino_docs_IE_DG_lpt_FuseSubtractToFakeQuantizeTransformation)
|
||||
* [MultiplyToGroupConvolutionTransformation](@ref openvino_docs_IE_DG_lpt_MultiplyToGroupConvolutionTransformation)
|
||||
* [FoldConvertTransformation](@ref openvino_docs_OV_UG_lpt_FoldConvertTransformation)
|
||||
* [FoldFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FoldFakeQuantizeTransformation)
|
||||
* [FuseConvertTransformation](@ref openvino_docs_OV_UG_lpt_FuseConvertTransformation)
|
||||
* [FuseMultiplyToFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FuseMultiplyToFakeQuantizeTransformation)
|
||||
* [FuseSubtractToFakeQuantizeTransformation](@ref openvino_docs_OV_UG_lpt_FuseSubtractToFakeQuantizeTransformation)
|
||||
* [MultiplyToGroupConvolutionTransformation](@ref openvino_docs_OV_UG_lpt_MultiplyToGroupConvolutionTransformation)
|
||||
@ -1,3 +1,3 @@
|
||||
# ConvertSubtractConstant transformation {#openvino_docs_IE_DG_lpt_ConvertSubtractConstant}
|
||||
# ConvertSubtractConstant transformation {#openvino_docs_OV_UG_lpt_ConvertSubtractConstant}
|
||||
|
||||
ngraph::pass::low_precision::ConvertSubtractConstant class represents the `ConvertSubtractConstant` transformation.
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# LinOpSequenceFusion transformation {#openvino_docs_IE_DG_lpt_LinOpSequenceFusion}
|
||||
# LinOpSequenceFusion transformation {#openvino_docs_OV_UG_lpt_LinOpSequenceFusion}
|
||||
|
||||
ngraph::pass::LinOpSequenceFusion class represents the `LinOpSequenceFusion` transformation.
|
||||
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PullReshapeThroughDequantization transformation {#openvino_docs_IE_DG_lpt_PullReshapeThroughDequantization}
|
||||
# PullReshapeThroughDequantization transformation {#openvino_docs_OV_UG_lpt_PullReshapeThroughDequantization}
|
||||
|
||||
ngraph::pass::low_precision::PullReshapeThroughDequantization class represents the `PullReshapeThroughDequantization` transformation.
|
||||
@ -1,3 +1,3 @@
|
||||
# PullTransposeThroughDequantization transformation {#openvino_docs_IE_DG_lpt_PullTransposeThroughDequantization}
|
||||
# PullTransposeThroughDequantization transformation {#openvino_docs_OV_UG_lpt_PullTransposeThroughDequantization}
|
||||
|
||||
ngraph::pass::low_precision::PullTransposeThroughDequantization class represents the `PullTransposeThroughDequantization` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# AlignQuantizationIntervals transformation {#openvino_docs_IE_DG_lpt_AlignQuantizationIntervals}
|
||||
# AlignQuantizationIntervals transformation {#openvino_docs_OV_UG_lpt_AlignQuantizationIntervals}
|
||||
|
||||
ngraph::pass::low_precision::AlignQuantizationIntervals class represents the `AlignQuantizationIntervals` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# AlignQuantizationParameters transformation {#openvino_docs_IE_DG_lpt_AlignQuantizationParameters}
|
||||
# AlignQuantizationParameters transformation {#openvino_docs_OV_UG_lpt_AlignQuantizationParameters}
|
||||
|
||||
ngraph::pass::low_precision::AlignQuantizationParameters class represents the `AlignQuantizationParameters` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# CreateAttribute transformation {#openvino_docs_IE_DG_lpt_CreateAttribute}
|
||||
# CreateAttribute transformation {#openvino_docs_OV_UG_lpt_CreateAttribute}
|
||||
|
||||
ngraph::pass::low_precision::CreateAttribute class represents the `CreateAttribute` transformation.
|
||||
@ -1,3 +1,3 @@
|
||||
# CreatePrecisionsDependentAttribute transformation {#openvino_docs_IE_DG_lpt_CreatePrecisionsDependentAttribute}
|
||||
# CreatePrecisionsDependentAttribute transformation {#openvino_docs_OV_UG_lpt_CreatePrecisionsDependentAttribute}
|
||||
|
||||
ngraph::pass::low_precision::CreatePrecisionsDependentAttribute class represents the `CreatePrecisionsDependentAttribute` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MarkupAvgPoolPrecisionPreserved transformation {#openvino_docs_IE_DG_lpt_MarkupAvgPoolPrecisionPreserved}
|
||||
# MarkupAvgPoolPrecisionPreserved transformation {#openvino_docs_OV_UG_lpt_MarkupAvgPoolPrecisionPreserved}
|
||||
|
||||
ngraph::pass::low_precision::MarkupAvgPoolPrecisionPreserved class represents the `MarkupAvgPoolPrecisionPreserved` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MarkupCanBeQuantized transformation {#openvino_docs_IE_DG_lpt_MarkupCanBeQuantized}
|
||||
# MarkupCanBeQuantized transformation {#openvino_docs_OV_UG_lpt_MarkupCanBeQuantized}
|
||||
|
||||
ngraph::pass::low_precision::MarkupCanBeQuantized class represents the `MarkupCanBeQuantized` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MarkupPerTensorQuantization transformation {#openvino_docs_IE_DG_lpt_MarkupPerTensorQuantization}
|
||||
# MarkupPerTensorQuantization transformation {#openvino_docs_OV_UG_lpt_MarkupPerTensorQuantization}
|
||||
|
||||
ngraph::pass::low_precision::MarkupPerTensorQuantization class represents the `MarkupPerTensorQuantization` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MarkupPrecisions transformation {#openvino_docs_IE_DG_lpt_MarkupPrecisions}
|
||||
# MarkupPrecisions transformation {#openvino_docs_OV_UG_lpt_MarkupPrecisions}
|
||||
|
||||
ngraph::pass::low_precision::MarkupPrecisions class represents the `MarkupPrecisions` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PropagatePrecisions transformation {#openvino_docs_IE_DG_lpt_PropagatePrecisions}
|
||||
# PropagatePrecisions transformation {#openvino_docs_OV_UG_lpt_PropagatePrecisions}
|
||||
|
||||
ngraph::pass::low_precision::PropagatePrecisions class represents the `PropagatePrecisions` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PropagateSharedValue transformation {#openvino_docs_IE_DG_lpt_PropagateSharedValue}
|
||||
# PropagateSharedValue transformation {#openvino_docs_OV_UG_lpt_PropagateSharedValue}
|
||||
|
||||
ngraph::pass::low_precision::PropagateSharedValue class represents the `PropagateSharedValue` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PropagateThroughPrecisionPreserved transformation {#openvino_docs_IE_DG_lpt_PropagateThroughPrecisionPreserved}
|
||||
# PropagateThroughPrecisionPreserved transformation {#openvino_docs_OV_UG_lpt_PropagateThroughPrecisionPreserved}
|
||||
|
||||
ngraph::pass::low_precision::PropagateThroughPrecisionPreserved class represents the `PropagateThroughPrecisionPreserved` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PropagateToInput transformation {#openvino_docs_IE_DG_lpt_PropagateToInput}
|
||||
# PropagateToInput transformation {#openvino_docs_OV_UG_lpt_PropagateToInput}
|
||||
|
||||
ngraph::pass::low_precision::PropagateToInput class represents the `PropagateToInput` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# UpdateSharedPrecisionPreserved transformation {#openvino_docs_IE_DG_lpt_UpdateSharedPrecisionPreserved}
|
||||
# UpdateSharedPrecisionPreserved transformation {#openvino_docs_OV_UG_lpt_UpdateSharedPrecisionPreserved}
|
||||
|
||||
ngraph::pass::low_precision::UpdateSharedPrecisionPreserved class represents the `UpdateSharedPrecisionPreserved` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ClampTransformation transformation {#openvino_docs_IE_DG_lpt_ClampTransformation}
|
||||
# ClampTransformation transformation {#openvino_docs_OV_UG_lpt_ClampTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ClampTransformation class represents the `Clamp` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PReluTransformation transformation {#openvino_docs_IE_DG_lpt_PReluTransformation}
|
||||
# PReluTransformation transformation {#openvino_docs_OV_UG_lpt_PReluTransformation}
|
||||
|
||||
ngraph::pass::low_precision::PReluTransformation class represents the `PRelu` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReluTransformation transformation {#openvino_docs_IE_DG_lpt_ReluTransformation}
|
||||
# ReluTransformation transformation {#openvino_docs_OV_UG_lpt_ReluTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReluTransformation class represents the `Relu` operation transformation.
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# AddTransformation transformation {#openvino_docs_IE_DG_lpt_AddTransformation}
|
||||
# AddTransformation transformation {#openvino_docs_OV_UG_lpt_AddTransformation}
|
||||
|
||||
ngraph::pass::low_precision::AddTransformation class represents the `Add` operation transformation.
|
||||
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MultiplyTransformation transformation {#openvino_docs_IE_DG_lpt_MultiplyTransformation}
|
||||
# MultiplyTransformation transformation {#openvino_docs_OV_UG_lpt_MultiplyTransformation}
|
||||
|
||||
ngraph::pass::low_precision::MultiplyTransformation class represents the `Multiply` operation transformation.
|
||||
@ -1,3 +1,3 @@
|
||||
# SubtractTransformation transformation {#openvino_docs_IE_DG_lpt_SubtractTransformation}
|
||||
# SubtractTransformation transformation {#openvino_docs_OV_UG_lpt_SubtractTransformation}
|
||||
|
||||
ngraph::pass::low_precision::SubtractTransformation class represents the `Subtract` operation transformation.
|
||||
@ -1,4 +1,4 @@
|
||||
# ConvolutionTransformation transformation {#openvino_docs_IE_DG_lpt_ConvolutionTransformation}
|
||||
# ConvolutionTransformation transformation {#openvino_docs_OV_UG_lpt_ConvolutionTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ConvolutionTransformation class represents the `Convolution` operation transformation.
|
||||
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ConvolutionBackpropDataTransformation transformation {#openvino_docs_IE_DG_lpt_ConvolutionBackpropDataTransformation}
|
||||
# ConvolutionBackpropDataTransformation transformation {#openvino_docs_OV_UG_lpt_ConvolutionBackpropDataTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ConvolutionBackpropDataTransformation class represents the `ConvolutionBackpropData` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# GroupConvolutionTransformation transformation {#openvino_docs_IE_DG_lpt_GroupConvolutionTransformation}
|
||||
# GroupConvolutionTransformation transformation {#openvino_docs_OV_UG_lpt_GroupConvolutionTransformation}
|
||||
|
||||
ngraph::pass::low_precision::GroupConvolutionTransformation class represents the `GroupConvolution` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# InterpolateTransformation transformation {#openvino_docs_IE_DG_lpt_InterpolateTransformation}
|
||||
# InterpolateTransformation transformation {#openvino_docs_OV_UG_lpt_InterpolateTransformation}
|
||||
|
||||
ngraph::pass::low_precision::InterpolateTransformation class represents the `Interpolate` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MatMulTransformation transformation {#openvino_docs_IE_DG_lpt_MatMulTransformation}
|
||||
# MatMulTransformation transformation {#openvino_docs_OV_UG_lpt_MatMulTransformation}
|
||||
|
||||
ngraph::pass::low_precision::MatMulTransformation class represents the `MatMul` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ConcatTransformation transformation {#openvino_docs_IE_DG_lpt_ConcatTransformation}
|
||||
# ConcatTransformation transformation {#openvino_docs_OV_UG_lpt_ConcatTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ConcatTransformation class represents the `Concat` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# DepthToSpaceTransformation transformation {#openvino_docs_IE_DG_lpt_DepthToSpaceTransformation}
|
||||
# DepthToSpaceTransformation transformation {#openvino_docs_OV_UG_lpt_DepthToSpaceTransformation}
|
||||
|
||||
ngraph::pass::low_precision::DepthToSpaceTransformation class represents the `DepthToSpace` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# PadTransformation transformation {#openvino_docs_IE_DG_lpt_PadTransformation}
|
||||
# PadTransformation transformation {#openvino_docs_OV_UG_lpt_PadTransformation}
|
||||
|
||||
ngraph::pass::low_precision::PadTransformation class represents the `Pad` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ShuffleChannelsTransformation transformation {#openvino_docs_IE_DG_lpt_ShuffleChannelsTransformation}
|
||||
# ShuffleChannelsTransformation transformation {#openvino_docs_OV_UG_lpt_ShuffleChannelsTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ShuffleChannelsTransformation class represents the `ShuffleChannels` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# SplitTransformation transformation {#openvino_docs_IE_DG_lpt_SplitTransformation}
|
||||
# SplitTransformation transformation {#openvino_docs_OV_UG_lpt_SplitTransformation}
|
||||
|
||||
ngraph::pass::low_precision::SplitTransformation class represents the `Split` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# StridedSliceTransformation transformation {#openvino_docs_IE_DG_lpt_StridedSliceTransformation}
|
||||
# StridedSliceTransformation transformation {#openvino_docs_OV_UG_lpt_StridedSliceTransformation}
|
||||
|
||||
ngraph::pass::low_precision::StridedSliceTransformation class represents the `StridedSlice` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# TransposeTransformation transformation {#openvino_docs_IE_DG_lpt_TransposeTransformation}
|
||||
# TransposeTransformation transformation {#openvino_docs_OV_UG_lpt_TransposeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::TransposeTransformation class represents the `Transpose` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# VariadicSplitTransformation transformation {#openvino_docs_IE_DG_lpt_VariadicSplitTransformation}
|
||||
# VariadicSplitTransformation transformation {#openvino_docs_OV_UG_lpt_VariadicSplitTransformation}
|
||||
|
||||
ngraph::pass::low_precision::VariadicSplitTransformation class represents the `VariadicSplit` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MVNTransformation transformation {#openvino_docs_IE_DG_lpt_MVNTransformation}
|
||||
# MVNTransformation transformation {#openvino_docs_OV_UG_lpt_MVNTransformation}
|
||||
|
||||
ngraph::pass::low_precision::MVNTransformation class represents the `MVN` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# NormalizeL2Transformation transformation {#openvino_docs_IE_DG_lpt_NormalizeL2Transformation}
|
||||
# NormalizeL2Transformation transformation {#openvino_docs_OV_UG_lpt_NormalizeL2Transformation}
|
||||
|
||||
ngraph::pass::low_precision::NormalizeL2Transformation class represents the `NormalizeL2` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# AvgPoolTransformation transformation {#openvino_docs_IE_DG_lpt_AvgPoolTransformation}
|
||||
# AvgPoolTransformation transformation {#openvino_docs_OV_UG_lpt_AvgPoolTransformation}
|
||||
|
||||
ngraph::pass::low_precision::AvgPoolTransformation class represents the `AvgPool` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MaxPoolTransformation transformation {#openvino_docs_IE_DG_lpt_MaxPoolTransformation}
|
||||
# MaxPoolTransformation transformation {#openvino_docs_OV_UG_lpt_MaxPoolTransformation}
|
||||
|
||||
ngraph::pass::low_precision::MaxPoolTransformation class represents the `MaxPool` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FakeQuantizeTransformation transformation {#openvino_docs_IE_DG_lpt_FakeQuantizeTransformation}
|
||||
# FakeQuantizeTransformation transformation {#openvino_docs_OV_UG_lpt_FakeQuantizeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FakeQuantizeTransformation class represents the `FakeQuantize` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FoldFakeQuantizeTransformation transformation {#openvino_docs_IE_DG_lpt_FoldFakeQuantizeTransformation}
|
||||
# FoldFakeQuantizeTransformation transformation {#openvino_docs_OV_UG_lpt_FoldFakeQuantizeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FoldFakeQuantizeTransformation class represents the `FoldFakeQuantize` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReduceMaxTransformation transformation {#openvino_docs_IE_DG_lpt_ReduceMaxTransformation}
|
||||
# ReduceMaxTransformation transformation {#openvino_docs_OV_UG_lpt_ReduceMaxTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReduceMaxTransformation class represents the `ReduceMax` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReduceMeanTransformation transformation {#openvino_docs_IE_DG_lpt_ReduceMeanTransformation}
|
||||
# ReduceMeanTransformation transformation {#openvino_docs_OV_UG_lpt_ReduceMeanTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReduceMeanTransformation class represents the `ReduceMean` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReduceMinTransformation transformation {#openvino_docs_IE_DG_lpt_ReduceMinTransformation}
|
||||
# ReduceMinTransformation transformation {#openvino_docs_OV_UG_lpt_ReduceMinTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReduceMinTransformation class represents the `ReduceMin` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReduceSumTransformation transformation {#openvino_docs_IE_DG_lpt_ReduceSumTransformation}
|
||||
# ReduceSumTransformation transformation {#openvino_docs_OV_UG_lpt_ReduceSumTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReduceSumTransformation class represents the `ReduceSum` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# ReshapeTransformation transformation {#openvino_docs_IE_DG_lpt_ReshapeTransformation}
|
||||
# ReshapeTransformation transformation {#openvino_docs_OV_UG_lpt_ReshapeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::ReshapeTransformation class represents the `Reshape` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# SqueezeTransformation transformation {#openvino_docs_IE_DG_lpt_SqueezeTransformation}
|
||||
# SqueezeTransformation transformation {#openvino_docs_OV_UG_lpt_SqueezeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::SqueezeTransformation class represents the `Squeeze` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# UnsqueezeTransformation transformation {#openvino_docs_IE_DG_lpt_UnsqueezeTransformation}
|
||||
# UnsqueezeTransformation transformation {#openvino_docs_OV_UG_lpt_UnsqueezeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::UnsqueezeTransformation class represents the `Unsqueeze` operation transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FakeQuantizeDecompositionTransformation transformation {#openvino_docs_IE_DG_lpt_FakeQuantizeDecompositionTransformation}
|
||||
# FakeQuantizeDecompositionTransformation transformation {#openvino_docs_OV_UG_lpt_FakeQuantizeDecompositionTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FakeQuantizeDecompositionTransformation class represents the `FakeQuantizeDecompositionTransformation` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FoldConvertTransformation transformation {#openvino_docs_IE_DG_lpt_FoldConvertTransformation}
|
||||
# FoldConvertTransformation transformation {#openvino_docs_OV_UG_lpt_FoldConvertTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FoldConvertTransformation class represents the `FoldConvertTransformation` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FuseConvertTransformation transformation {#openvino_docs_IE_DG_lpt_FuseConvertTransformation}
|
||||
# FuseConvertTransformation transformation {#openvino_docs_OV_UG_lpt_FuseConvertTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FuseConvertTransformation class represents the `FuseConvertTransformation` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FuseMultiplyToFakeQuantizeTransformation transformation {#openvino_docs_IE_DG_lpt_FuseMultiplyToFakeQuantizeTransformation}
|
||||
# FuseMultiplyToFakeQuantizeTransformation transformation {#openvino_docs_OV_UG_lpt_FuseMultiplyToFakeQuantizeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FuseMultiplyToFakeQuantizeTransformation class represents the `FuseMultiplyToFakeQuantizeTransformation` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# FuseSubtractToFakeQuantizeTransformation transformation {#openvino_docs_IE_DG_lpt_FuseSubtractToFakeQuantizeTransformation}
|
||||
# FuseSubtractToFakeQuantizeTransformation transformation {#openvino_docs_OV_UG_lpt_FuseSubtractToFakeQuantizeTransformation}
|
||||
|
||||
ngraph::pass::low_precision::FuseSubtractToFakeQuantizeTransformation class represents the `FuseSubtractToFakeQuantizeTransformation` transformation.
|
||||
|
||||
@ -1,3 +1,3 @@
|
||||
# MultiplyToGroupConvolutionTransformation transformation {#openvino_docs_IE_DG_lpt_MultiplyToGroupConvolutionTransformation}
|
||||
# MultiplyToGroupConvolutionTransformation transformation {#openvino_docs_OV_UG_lpt_MultiplyToGroupConvolutionTransformation}
|
||||
|
||||
ngraph::pass::low_precision::MultiplyToGroupConvolutionTransformation class represents the `MultiplyToGroupConvolutionTransformation` transformation.
|
||||
|
||||
@ -509,7 +509,7 @@ There are a number of common attributes used in the operations. Here is the list
|
||||
Model Optimizer operations this attribute should be set to `None`. The model conversion fails if an operation with
|
||||
`type` equal to `None` comes to the IR emitting phase. **Mandatory**.
|
||||
* `version` — the operation set (opset) name the operation belongs to. If not specified, the Model Optimizer sets it
|
||||
equal to `experimental`. Refer to [OpenVINO Model Representation](@ref openvino_docs_OV_Runtime_UG_Model_Representation) for more
|
||||
equal to `experimental`. Refer to [OpenVINO Model Representation](@ref openvino_docs_OV_UG_Model_Representation) for more
|
||||
information about operation sets. **Mandatory**.
|
||||
* `op` — Model Optimizer type of the operation. In many cases, the value of `type` is equal to the value of `op`. But
|
||||
when the Model Optimizer cannot instantiate the opset operation during model loading, it creates an instance of an internal
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Model Caching Overview {#openvino_docs_IE_DG_Model_caching_overview}
|
||||
# Model Caching Overview {#openvino_docs_OV_UG_Model_caching_overview}
|
||||
|
||||
## Introduction
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# OpenVINO™ Python API exclusives {#openvino_docs_OV_Runtime_UG_Python_API_exclusives}
|
||||
# OpenVINO™ Python API exclusives {#openvino_docs_OV_UG_Python_API_exclusives}
|
||||
|
||||
OpenVINO™ Runtime Python API is exposing additional features and helpers to elevate user experience. Main goal of Python API is to provide user-friendly and simple, still powerful, tool for Python users.
|
||||
|
||||
@ -28,12 +28,6 @@ Python API allows passing data as tensors. `Tensor` object holds a copy of the d
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py tensor_shared_mode
|
||||
|
||||
### Slices of array's memory
|
||||
|
||||
One of the `Tensor` class constructors allows to share the slice of array's memory. When `shape` is specified in the constructor that has the numpy array as first argument, it triggers the special shared memory mode.
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py tensor_slice_mode
|
||||
|
||||
## Running inference
|
||||
|
||||
Python API supports extra calling methods to synchronous and asynchronous modes for inference.
|
||||
@ -76,6 +70,14 @@ The callback of `AsyncInferQueue` is uniform for every job. When executed, GIL i
|
||||
|
||||
@snippet docs/snippets/ov_python_exclusives.py asyncinferqueue_set_callback
|
||||
|
||||
### Working with u1, u4 and i4 element types
|
||||
|
||||
Since openvino supports low precision element types there are few ways how to handle them in python.
|
||||
To create an input tensor with such element types you may need to pack your data in new numpy array which byte size matches original input size:
|
||||
@snippet docs/snippets/ov_python_exclusives.py packing_data
|
||||
|
||||
To extract low precision values from tensor into numpy array you can use next helper:
|
||||
@snippet docs/snippets/ov_python_exclusives.py unpacking
|
||||
|
||||
### Releasing the GIL
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# OpenVINO Samples {#openvino_docs_IE_DG_Samples_Overview}
|
||||
# OpenVINO Samples {#openvino_docs_OV_UG_Samples_Overview}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Changing input shapes {#openvino_docs_IE_DG_ShapeInference}
|
||||
# Changing input shapes {#openvino_docs_OV_UG_ShapeInference}
|
||||
|
||||
## Introduction (C++)
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Automatic device selection {#openvino_docs_IE_DG_supported_plugins_AUTO}
|
||||
# Automatic device selection {#openvino_docs_OV_UG_supported_plugins_AUTO}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -6,7 +6,7 @@
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
Debugging Auto-Device Plugin <openvino_docs_IE_DG_supported_plugins_AUTO_debugging>
|
||||
Debugging Auto-Device Plugin <openvino_docs_OV_UG_supported_plugins_AUTO_debugging>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Integrate OpenVINO™ with Your Application {#openvino_docs_Integrate_OV_with_your_application}
|
||||
# Integrate OpenVINO™ with Your Application {#openvino_docs_OV_UG_Integrate_OV_with_your_application}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -6,9 +6,9 @@
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
openvino_docs_OV_Runtime_UG_Model_Representation
|
||||
openvino_docs_OV_Runtime_UG_Infer_request
|
||||
openvino_docs_OV_Runtime_UG_Python_API_exclusives
|
||||
openvino_docs_OV_UG_Model_Representation
|
||||
openvino_docs_OV_UG_Infer_request
|
||||
openvino_docs_OV_UG_Python_API_exclusives
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Layout API overview {#openvino_docs_OV_Runtime_UG_Layout_Overview}
|
||||
# Layout API overview {#openvino_docs_OV_UG_Layout_Overview}
|
||||
|
||||
## Introduction
|
||||
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
# Inference Pipeline {#openvino_2_0_inference_pipeline}
|
||||
|
||||
Usually to infer models with OpenVINO™ Runtime, you need to do the following steps in the application pipeline:
|
||||
Usually, to infer models with OpenVINO™ Runtime, you need to make the following steps in the application pipeline:
|
||||
- 1. Create Core object
|
||||
- 1.1. (Optional) Load extensions
|
||||
- 2. Read model from the disk
|
||||
- 2.1. (Optional) Model preprocessing
|
||||
- 2. Read a model from a drive
|
||||
- 2.1. (Optional) Perform model preprocessing
|
||||
- 3. Load the model to the device
|
||||
- 4. Create an inference request
|
||||
- 5. Fill input tensors with data
|
||||
@ -45,7 +45,7 @@ OpenVINO™ Runtime API 2.0:
|
||||
|
||||
### 1.1 (Optional) Load extensions
|
||||
|
||||
To load model with custom operation, you need to add extensions for these operations. We highly recommend to use [OpenVINO Extensibility API](../../Extensibility_UG/Intro.md) to write extensions, but if you already have old extensions you can load it to new OpenVINO™ Runtime:
|
||||
To load a model with custom operations, you need to add extensions for these operations. We highly recommend using [OpenVINO Extensibility API](../../Extensibility_UG/Intro.md) to write extensions, but if you already have old extensions you can also load them to the new OpenVINO™ Runtime:
|
||||
|
||||
Inference Engine API:
|
||||
|
||||
@ -75,7 +75,7 @@ OpenVINO™ Runtime API 2.0:
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
## 2. Read model from the disk
|
||||
## 2. Read a model from a drive
|
||||
|
||||
Inference Engine API:
|
||||
|
||||
@ -109,10 +109,10 @@ Read model has the same structure as in the example from [Model Creation](./grap
|
||||
|
||||
Note, you can combine read and compile model stages into a single call `ov::Core::compile_model(filename, devicename)`.
|
||||
|
||||
### 2.1 (Optional) Model preprocessing
|
||||
### 2.1 (Optional) Perform model preprocessing
|
||||
|
||||
When application's input data doesn't perfectly match with model's input format, preprocessing steps may need to be added.
|
||||
See detailed guide [how to migrate preprocessing in OpenVINO Runtime API 2.0](./preprocessing.md)
|
||||
When application's input data doesn't perfectly match the model's input format, preprocessing steps may be necessary.
|
||||
See a detailed guide on [how to migrate preprocessing in OpenVINO Runtime API 2.0](./preprocessing.md)
|
||||
|
||||
## 3. Load the Model to the Device
|
||||
|
||||
@ -144,7 +144,7 @@ OpenVINO™ Runtime API 2.0:
|
||||
|
||||
@endsphinxtabset
|
||||
|
||||
If you need to configure OpenVINO Runtime devices with additional configuration parameters, please, refer to the migration [Configure devices](./configure_devices.md) guide.
|
||||
If you need to configure OpenVINO Runtime devices with additional configuration parameters, refer to the [Configure devices](./configure_devices.md) guide.
|
||||
|
||||
## 4. Create an Inference Request
|
||||
|
||||
@ -178,7 +178,7 @@ OpenVINO™ Runtime API 2.0:
|
||||
|
||||
## 5. Fill input tensors
|
||||
|
||||
Inference Engine API fills inputs as `I32` precision (**not** aligned with the original model):
|
||||
The Inference Engine API fills inputs as `I32` precision (**not** aligned with the original model):
|
||||
|
||||
@sphinxtabset
|
||||
|
||||
@ -398,7 +398,7 @@ OpenVINO™ Runtime API 2.0:
|
||||
|
||||
## 7. Process the Inference Results
|
||||
|
||||
Inference Engine API processes outputs as `I32` precision (**not** aligned with the original model):
|
||||
The Inference Engine API processes outputs as `I32` precision (**not** aligned with the original model):
|
||||
|
||||
@sphinxtabset
|
||||
|
||||
@ -469,8 +469,8 @@ Inference Engine API processes outputs as `I32` precision (**not** aligned with
|
||||
@endsphinxtabset
|
||||
|
||||
OpenVINO™ Runtime API 2.0 processes outputs:
|
||||
- For IR v10 as `I32` precision (**not** aligned with the original model) to match **old** behavior
|
||||
- For IR v11, ONNX, ov::Model, Paddle as `I64` precision (aligned with the original model) to match **new** behavior
|
||||
- For IR v10 as `I32` precision (**not** aligned with the original model) to match the **old** behavior.
|
||||
- For IR v11, ONNX, ov::Model, Paddle as `I64` precision (aligned with the original model) to match the **new** behavior.
|
||||
|
||||
@sphinxtabset
|
||||
|
||||
|
||||
@ -2,9 +2,9 @@
|
||||
|
||||
### Introduction
|
||||
|
||||
Inference Engine API provides an [ability to configure devices](https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_InferenceEngine_QueryAPI.html) via configuration keys and [get device specific metrics](https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_InferenceEngine_QueryAPI.html#getmetric). The values taken from `InferenceEngine::Core::GetConfig` are requested by its string name, while return type is `InferenceEngine::Parameter` and users don't know what is the actual type is stored in this parameter.
|
||||
The Inference Engine API provides the [ability to configure devices](https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_InferenceEngine_QueryAPI.html) via configuration keys and [get device specific metrics](https://docs.openvino.ai/2021.4/openvino_docs_IE_DG_InferenceEngine_QueryAPI.html#getmetric). The values taken from `InferenceEngine::Core::GetConfig` are requested by the string name, while the return type is `InferenceEngine::Parameter`, making users lost on what the actual type stored in this parameter is.
|
||||
|
||||
OpenVINO Runtime API 2.0 solves these issues by introducing [properties](../supported_plugins/config_properties.md), which unify metrics and configuration key concepts, but the main advantage of properties - they have C++ type:
|
||||
The OpenVINO Runtime API 2.0 solves these issues by introducing [properties](../supported_plugins/config_properties.md), which unify metrics and configuration key concepts. Their main advantage is that they have the C++ type:
|
||||
|
||||
```
|
||||
static constexpr Property<std::string> full_name{"FULL_DEVICE_NAME"};
|
||||
@ -14,7 +14,7 @@ And the property can be requested from an inference device as:
|
||||
|
||||
@snippet ov_properties_migration.cpp core_get_ro_property
|
||||
|
||||
The snippets below show how to migrate from Inference Engine device configuration to OpenVINO Runtime API 2.0 steps.
|
||||
The snippets below show how to migrate from an Inference Engine device configuration to OpenVINO Runtime API 2.0 steps.
|
||||
|
||||
### Set configuration values
|
||||
|
||||
|
||||
@ -1,29 +1,29 @@
|
||||
# Installation & Deployment {#openvino_2_0_deployment}
|
||||
|
||||
"Easy to use" is one of the main concepts for OpenVINO™ API 2.0. It includes not only simplifying the migration from frameworks to OpenVINO, but also how OpenVINO is organized, how the development tools are used, and how to develop and deploy OpenVINO-based applications.
|
||||
"Easy to use" is one of the main concepts for OpenVINO™ API 2.0. It is about simplifying migration from different frameworks to OpenVINO, but also touches on how OpenVINO is organized, how its development tools are used, and how OpenVINO-based applications are developed and deployed.
|
||||
|
||||
To accomplish that, we have made some changes on the installation and deployment of OpenVINO in the 2022.1 release. This guide will walk you through them.
|
||||
To accomplish that, we made some changes to the installation and deployment processes of OpenVINO in the 2022.1 release. This guide will walk you through them.
|
||||
|
||||
## Installer Package Contains OpenVINO™ Runtime Only
|
||||
## The Installer Package Contains OpenVINO™ Runtime Only
|
||||
|
||||
Starting from OpenVINO 2022.1, Model Optimizer, Post-Training Optimization tool and Python-based Development tools such as Open Model Zoo tools are distributed via [PyPI](https://pypi.org/project/openvino-dev/) only, and are no longer included in the OpenVINO installer package. This change has several benefits as it:
|
||||
Starting from OpenVINO 2022.1, development tools are distributed via [PyPI](https://pypi.org/project/openvino-dev/) only and are no longer included in the OpenVINO installer package. For a list of these components, refer to the [installation overview](../../../install_guides/installing-openvino-overview.md). This approach has several benefits:
|
||||
|
||||
* Simplifies the user experience. In previous versions, the installation and usage of OpenVINO Development Tools differ according to the distribution type (via an OpenVINO installer or PyPI).
|
||||
* Ensures that dependencies are handled properly via the PIP package manager and support virtual environments of development tools.
|
||||
* simplifies the user experience - in previous versions, installation and usage of OpenVINO Development Tools differed from one distribution type to another (the OpenVINO installer vs. PyPI),
|
||||
* ensures that dependencies are handled properly via the PIP package manager and support virtual environments of development tools.
|
||||
|
||||
The structure of OpenVINO 2022.1 installer package has been organized as below:
|
||||
The structure of the OpenVINO 2022.1 installer package has been organized as follows:
|
||||
|
||||
- The `runtime` folder includes headers, libraries and CMake interfaces.
|
||||
- The `tools` folder contains [the compile tool](../../../tools/compile_tool/README.md), [deployment manager](../../OV_Runtime_UG/deployment/deployment-manager-tool.md) and a set of `requirements.txt` files with links to the corresponding versions of the `openvino-dev` package.
|
||||
- The `tools` folder contains [the compile tool](../../../tools/compile_tool/README.md), [deployment manager](../../OV_Runtime_UG/deployment/deployment-manager-tool.md), and a set of `requirements.txt` files with links to the corresponding versions of the `openvino-dev` package.
|
||||
- The `python` folder contains the Python version for OpenVINO Runtime.
|
||||
|
||||
## Installing OpenVINO Development Tools via PyPI
|
||||
|
||||
Since OpenVINO Development Tools is no longer in the installer package, the installation process has changed too. This section describes it through a comparison with previous versions.
|
||||
Since OpenVINO Development Tools is no longer in the installer package, the installation process has also changed. This section describes it through a comparison with previous versions.
|
||||
|
||||
### For Versions Prior to 2022.1
|
||||
|
||||
In previous versions, OpenVINO Development Tools is a part of main package. After the package is installed, to convert models (for example, TensorFlow), you need to install additional dependencies by using the requirements files such as `requirements_tf.txt`, install Post-Training Optimization tool and Accuracy Checker tool via the `setup.py` scripts, and then use the `setupvars` scripts to make the tools available to the following command:
|
||||
In previous versions, OpenVINO Development Tools was a part of the main package. After the package was installed, to convert models (for example, TensorFlow), you needed to install additional dependencies by using the requirement files, such as `requirements_tf.txt`, install Post-Training Optimization tool and Accuracy Checker tool via the `setup.py` scripts, and then use the `setupvars` scripts to make the tools available to the following command:
|
||||
|
||||
```sh
|
||||
$ mo.py -h
|
||||
@ -31,13 +31,13 @@ $ mo.py -h
|
||||
|
||||
### For 2022.1 and After
|
||||
|
||||
Starting from OpenVINO 2022.1, you can install the development tools from [PyPI](https://pypi.org/project/openvino-dev/) repository only, using the following command (taking TensorFlow as an example):
|
||||
In OpenVINO 2022.1 and later, you can install the development tools from a [PyPI](https://pypi.org/project/openvino-dev/) repository only, using the following command (taking TensorFlow as an example):
|
||||
|
||||
```sh
|
||||
$ python3 -m pip install -r <INSTALL_DIR>/tools/requirements_tf.txt
|
||||
```
|
||||
|
||||
This will install all the development tools and additional necessary components to work with TensorFlow via the `openvino-dev` package (see **Step 4. Install the Package** on the [PyPI page](https://pypi.org/project/openvino-dev/) for parameters of other frameworks).
|
||||
This will install all the development tools and additional components necessary to work with TensorFlow via the `openvino-dev` package (see **Step 4. Install the Package** on the [PyPI page](https://pypi.org/project/openvino-dev/) for parameters of other frameworks).
|
||||
|
||||
Then, the tools can be used by commands like:
|
||||
|
||||
@ -50,11 +50,11 @@ You don't have to install any other dependencies. For more details on the instal
|
||||
|
||||
## Interface Changes for Building C/C++ Applications
|
||||
|
||||
The new OpenVINO Runtime with API 2.0 has also brought some changes for builiding your C/C++ applications.
|
||||
The new OpenVINO Runtime with its API 2.0 has also brought some changes for building C/C++ applications.
|
||||
|
||||
### CMake Interface
|
||||
|
||||
The CMake interface has been changed as below:
|
||||
The CMake interface has been changed as follows:
|
||||
|
||||
**With Inference Engine of previous versions**:
|
||||
|
||||
@ -78,7 +78,7 @@ target_link_libraries(ov_c_app PRIVATE openvino::runtime::c)
|
||||
|
||||
### Native Interfaces
|
||||
|
||||
To build applications without CMake interface, you can also use MSVC IDE, UNIX makefiles and any other interfaces, which have been changed as below:
|
||||
To build applications without the CMake interface, you can also use MSVC IDE, UNIX makefiles, and any other interface, which has been changed as shown here:
|
||||
|
||||
**With Inference Engine of previous versions**:
|
||||
|
||||
@ -153,19 +153,19 @@ To build applications without CMake interface, you can also use MSVC IDE, UNIX m
|
||||
|
||||
## Clearer Library Structure for Deployment
|
||||
|
||||
OpenVINO 2022.1 has reorganized the libraries to make it easier for deployment. In previous versions, to perform deployment steps, you have to use several libraries. Now you can just use `openvino` or `openvino_c` based on your developing language plus necessary plugins to complete your task. For example, `openvino_intel_cpu_plugin` and `openvino_ir_frontend` plugins will enable you to load OpenVINO IRs and perform inference on CPU device (see [Local distribution with OpenVINO](../deployment/local-distribution.md) for more details).
|
||||
OpenVINO 2022.1 introduced a reorganization of the libraries, to make deployment easier. In the previous versions, to perform deployment steps, you had to use several libraries. Now you can just use `openvino` or `openvino_c` based on your developing language, together with the necessary plugins to complete your task. For example, `openvino_intel_cpu_plugin` and `openvino_ir_frontend` plugins will enable you to load OpenVINO IRs and perform inference on the CPU device (see [Local distribution with OpenVINO](../deployment/local-distribution.md) for more details).
|
||||
|
||||
Here you can find some detailed comparisons on library structure between OpenVINO 2022.1 and previous versions:
|
||||
Here you can find detailed comparisons on the library structure between OpenVINO 2022.1 and the previous versions:
|
||||
|
||||
* A single core library with all the functionalities (`openvino` for C++ Runtime, `openvino_c` for Inference Engine API C interface) is used in 2022.1, instead of the previous core libraries which contain `inference_engine`, `ngraph`, `inference_engine_transformations` and `inference_engine_lp_transformations`.
|
||||
* The optional `inference_engine_preproc` preprocessing library (if `InferenceEngine::PreProcessInfo::setColorFormat` or `InferenceEngine::PreProcessInfo::setResizeAlgorithm` is used) is renamed as `openvino_gapi_preproc` and deprecated in 2022.1. See more details on [Preprocessing capabilities of OpenVINO API 2.0](preprocessing.md).
|
||||
* The libraries of plugins are renamed as below:
|
||||
* `openvino_intel_cpu_plugin` is used for [CPU](../supported_plugins/CPU.md) device instead of `MKLDNNPlugin` in previous versions.
|
||||
* `openvino_intel_gpu_plugin` is used for [GPU](../supported_plugins/GPU.md) device instead of `clDNNPlugin` in previous versions.
|
||||
* `openvino_auto_plugin` is used for [Auto-Device Plugin](../auto_device_selection.md) in 2022.1.
|
||||
* The plugins for reading and converting models have been changed as below:
|
||||
* `openvino_ir_frontend` is used to read IRs instead of `inference_engine_ir_reader` in previous versions.
|
||||
* `openvino_onnx_frontend` is used to read ONNX models instead of `inference_engine_onnx_reader` (with its dependencies) in previous versions.
|
||||
* A single core library with all the functionalities (`openvino` for C++ Runtime, `openvino_c` for Inference Engine API C interface) is used in 2022.1, instead of the previous core libraries which contained `inference_engine`, `ngraph`, `inference_engine_transformations` and `inference_engine_lp_transformations`.
|
||||
* The optional `inference_engine_preproc` preprocessing library (if `InferenceEngine::PreProcessInfo::setColorFormat` or `InferenceEngine::PreProcessInfo::setResizeAlgorithm` is used) has been renamed to `openvino_gapi_preproc` and deprecated in 2022.1. See more details on [Preprocessing capabilities of OpenVINO API 2.0](preprocessing.md).
|
||||
* The libraries of plugins have been renamed as follows:
|
||||
* `openvino_intel_cpu_plugin` is used for [CPU](../supported_plugins/CPU.md) device instead of `MKLDNNPlugin`.
|
||||
* `openvino_intel_gpu_plugin` is used for [GPU](../supported_plugins/GPU.md) device instead of `clDNNPlugin`.
|
||||
* `openvino_auto_plugin` is used for [Auto-Device Plugin](../auto_device_selection.md).
|
||||
* The plugins for reading and converting models have been changed as follows:
|
||||
* `openvino_ir_frontend` is used to read IRs instead of `inference_engine_ir_reader`.
|
||||
* `openvino_onnx_frontend` is used to read ONNX models instead of `inference_engine_onnx_reader` (with its dependencies).
|
||||
* `openvino_paddle_frontend` is added in 2022.1 to read PaddlePaddle models.
|
||||
|
||||
<!-----
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
# Model Creation in Runtime {#openvino_2_0_model_creation}
|
||||
|
||||
OpenVINO™ Runtime API 2.0 includes nGraph engine as a common part. The `ngraph` namespace was changed to `ov`, all other ngraph API is preserved as is.
|
||||
Code snippets below show how application code should be changed for migration to OpenVINO™ Runtime API 2.0.
|
||||
OpenVINO™ Runtime API 2.0 includes the nGraph engine as a common part. The `ngraph` namespace has been changed to `ov` but all other parts of the ngraph API have been preserved.
|
||||
The code snippets below show how to change application code for migration to OpenVINO™ Runtime API 2.0.
|
||||
|
||||
### nGraph API
|
||||
|
||||
|
||||
@ -2,26 +2,26 @@
|
||||
|
||||
### Introduction
|
||||
|
||||
Inference Engine API has preprocessing capabilities in `InferenceEngine::CNNNetwork` class. Such preprocessing information is not a part of the main inference graph executed by the [OpenVINO devices](../supported_plugins/Device_Plugins.md), so it is stored and executed separately before an inference stage:
|
||||
- Preprocessing operations are executed on CPU processor for most of the OpenVINO inference plugins. So, instead of occupying of acceleators, CPU processor is also busy with computational tasks.
|
||||
- Preprocessing information stored in `InferenceEngine::CNNNetwork` is lost during saving back to IR file format.
|
||||
The Inference Engine API contains preprocessing capabilities in the `InferenceEngine::CNNNetwork` class. Such preprocessing information is not part of the main inference graph executed by [OpenVINO devices](../supported_plugins/Device_Plugins.md), so it is stored and executed separately before the inference stage.
|
||||
- Preprocessing operations are executed on the CPU for most OpenVINO inference plugins. So, instead of occupying accelerators, they make CPU busy with computational tasks.
|
||||
- Preprocessing information stored in `InferenceEngine::CNNNetwork` is lost when saving back to the IR file format.
|
||||
|
||||
OpenVINO Runtime API 2.0 introduces [new way of adding preprocessing operations to the model](../preprocessing_overview.md) - each preprocessing or postprocessing operation is integrated directly to the model and compiled together with inference graph:
|
||||
OpenVINO Runtime API 2.0 introduces a [new way of adding preprocessing operations to the model](../preprocessing_overview.md) - each preprocessing or postprocessing operation is integrated directly into the model and compiled together with the inference graph.
|
||||
- Add preprocessing operations first using `ov::preprocess::PrePostProcessor`
|
||||
- Compile model on the target then using `ov::Core::compile_model`
|
||||
- Then, compile the model on the target using `ov::Core::compile_model`
|
||||
|
||||
Having preprocessing operations as a part of OpenVINO opset allows to read and serialize preprocessed model as the IR file format.
|
||||
Having preprocessing operations as part of an OpenVINO opset makes it possible to read and serialize a preprocessed model as the IR file format.
|
||||
|
||||
It's also important to mention that since OpenVINO 2.0, the Runtime API does not assume any default layouts like Inference Engine did, for example both `{ 1, 224, 224, 3 }` and `{ 1, 3, 224, 224 }` shapes are supposed to have `NCHW` layout while only the last shape has `NCHW`. So, some preprocessing capabilities in OpenVINO Runtime API 2.0 requires explicitly set layouts, see [Layout overview](../layout_overview.md) how to do it. For example, to perform image scaling by partial dimensions `H` and `W`, preprocessing needs to know what dimensions are `H` and `W`.
|
||||
It is also important to mention that the OpenVINO Runtime API 2.0 does not assume any default layouts, like Inference Engine did. For example, both `{ 1, 224, 224, 3 }` and `{ 1, 3, 224, 224 }` shapes are supposed to be in the `NCHW` layout, while only the latter one is. So, some preprocessing capabilities in the API require layouts to be set explicitly. To learn how to do it, refer to [Layout overview](../layout_overview.md). For example, to perform image scaling by partial dimensions `H` and `W`, preprocessing needs to know what dimensions `H` and `W` are.
|
||||
|
||||
> **NOTE**: Use Model Optimizer preprocessing capabilities to insert and optimize preprocessing operations to the model. In this case you don't need to read model in runtime application and set preprocessing, you can use [model caching feature](../Model_caching_overview.md) to improve time to inference stage.
|
||||
> **NOTE**: Use Model Optimizer preprocessing capabilities to insert preprocessing operations in you model for optimization. This way the application does not need to read the model and set preprocessing repeatedly, you can use the [model caching feature](../Model_caching_overview.md) to improve the time-to-inference.
|
||||
|
||||
The steps below demonstrates how to migrate preprocessing scenarios from Inference Engine API to OpenVINO Runtime API 2.0.
|
||||
The snippets suppose we need to preprocess a model input with tensor name `tensor_name`, in Inferenece Engine API using operation names to address the data, it's called `operation_name`.
|
||||
The steps below demonstrate how to migrate preprocessing scenarios from the Inference Engine API to the OpenVINO Runtime API 2.0.
|
||||
The snippets assume we need to preprocess a model input with the tensor name of `tensor_name` in the Inferenece Engine API, using operation names to address the data, called `operation_name`.
|
||||
|
||||
#### Importing preprocessing in Python
|
||||
|
||||
In order to utilize preprocessing following imports must be added.
|
||||
In order to utilize preprocessing, the following imports must be added.
|
||||
|
||||
Inference Engine API:
|
||||
|
||||
@ -31,7 +31,7 @@ OpenVINO Runtime API 2.0:
|
||||
|
||||
@snippet docs/snippets/ov_preprocessing_migration.py ov_imports
|
||||
|
||||
There are two different namespaces `runtime`, which contains OpenVINO Runtime API classes and `preprocess` which provides Preprocessing API.
|
||||
There are two different namespaces: `runtime`, which contains OpenVINO Runtime API classes; and `preprocess`, which provides the Preprocessing API.
|
||||
|
||||
|
||||
### Mean and scale values
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Model Representation in OpenVINO™ Runtime {#openvino_docs_OV_Runtime_UG_Model_Representation}
|
||||
# Model Representation in OpenVINO™ Runtime {#openvino_docs_OV_UG_Model_Representation}
|
||||
|
||||
In OpenVINO™ Runtime a model is represented by the `ov::Model` class.
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
Stateful models {#openvino_docs_IE_DG_network_state_intro}
|
||||
Stateful models {#openvino_docs_OV_UG_network_state_intro}
|
||||
==============================
|
||||
|
||||
This section describes how to work with stateful networks in OpenVINO toolkit, specifically:
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Performing inference with OpenVINO Runtime {#openvino_docs_OV_Runtime_User_Guide}
|
||||
# Performing inference with OpenVINO Runtime {#openvino_docs_OV_UG_OV_Runtime_User_Guide}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -8,17 +8,17 @@
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
openvino_docs_Integrate_OV_with_your_application
|
||||
openvino_docs_IE_DG_ShapeInference
|
||||
openvino_docs_OV_UG_Integrate_OV_with_your_application
|
||||
openvino_docs_OV_UG_ShapeInference
|
||||
openvino_docs_OV_UG_Working_with_devices
|
||||
openvino_docs_OV_Runtime_UG_Preprocessing_Overview
|
||||
openvino_docs_OV_UG_Preprocessing_Overview
|
||||
openvino_docs_OV_UG_DynamicShapes
|
||||
openvino_docs_IE_DG_supported_plugins_AUTO
|
||||
openvino_docs_OV_UG_supported_plugins_AUTO
|
||||
openvino_docs_OV_UG_Running_on_multiple_devices
|
||||
openvino_docs_OV_UG_Hetero_execution
|
||||
openvino_docs_OV_UG_Performance_Hints
|
||||
openvino_docs_OV_UG_Automatic_Batching
|
||||
openvino_docs_IE_DG_network_state_intro
|
||||
openvino_docs_OV_UG_network_state_intro
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# OpenVINO™ Inference Request {#openvino_docs_OV_Runtime_UG_Infer_request}
|
||||
# OpenVINO™ Inference Request {#openvino_docs_OV_UG_Infer_request}
|
||||
|
||||
OpenVINO™ Runtime uses Infer Request mechanism which allows to run models on different devices in asynchronous or synchronous manners.
|
||||
`ov::InferRequest` class is used for this purpose inside the OpenVINO™ Runtime.
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Preprocessing API - details {#openvino_docs_OV_Runtime_UG_Preprocessing_Details}
|
||||
# Preprocessing API - details {#openvino_docs_OV_UG_Preprocessing_Details}
|
||||
|
||||
## Preprocessing capabilities
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Optimize Preprocessing {#openvino_docs_OV_Runtime_UG_Preprocessing_Overview}
|
||||
# Optimize Preprocessing {#openvino_docs_OV_UG_Preprocessing_Overview}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@ -6,9 +6,9 @@
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
openvino_docs_OV_Runtime_UG_Preprocessing_Details
|
||||
openvino_docs_OV_Runtime_UG_Layout_Overview
|
||||
openvino_docs_OV_Runtime_UG_Preprocess_Usecase_save
|
||||
openvino_docs_OV_UG_Preprocessing_Details
|
||||
openvino_docs_OV_UG_Layout_Overview
|
||||
openvino_docs_OV_UG_Preprocess_Usecase_save
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Use Case - Integrate and Save Preprocessing Steps Into IR {#openvino_docs_OV_Runtime_UG_Preprocess_Usecase_save}
|
||||
# Use Case - Integrate and Save Preprocessing Steps Into IR {#openvino_docs_OV_UG_Preprocess_Usecase_save}
|
||||
|
||||
## Introduction
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Using Encrypted Models with OpenVINO™ {#openvino_docs_IE_DG_protecting_model_guide}
|
||||
# Using Encrypted Models with OpenVINO™ {#openvino_docs_OV_UG_protecting_model_guide}
|
||||
|
||||
Deploying deep-learning capabilities to edge devices can present security
|
||||
challenges, for example, ensuring inference integrity or providing copyright
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# Debugging Auto-Device Plugin {#openvino_docs_IE_DG_supported_plugins_AUTO_debugging}
|
||||
# Debugging Auto-Device Plugin {#openvino_docs_OV_UG_supported_plugins_AUTO_debugging}
|
||||
|
||||
## Using Debug Log
|
||||
In case of execution problems, just like all other plugins, Auto-Device provides the user with information on exceptions and error values. If the returned data is not enough for debugging purposes, more information may be acquired by means of `ov::log::Level`.
|
||||
@ -89,27 +89,27 @@ With Intel® VTune™ Profiler installed you can configure your analysis with th
|
||||
2. select **Configure Analysis**
|
||||
3. In the **where** pane, select **Local Host**
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/IE_DG_supported_plugins_AUTO_debugging-img01-localhost.png
|
||||
.. image:: _static/images/OV_UG_supported_plugins_AUTO_debugging-img01-localhost.png
|
||||
:align: center
|
||||
@endsphinxdirective
|
||||
4. In the **what** pane, specify your target application/script on the local system.
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/IE_DG_supported_plugins_AUTO_debugging-img02-launch.png
|
||||
.. image:: _static/images/OV_UG_supported_plugins_AUTO_debugging-img02-launch.png
|
||||
:align: center
|
||||
@endsphinxdirective
|
||||
5. In the **how** pane, choose and configure the analysis type you want to perform, for example, **Hotspots Analysis**:
|
||||
identify the most time-consuming functions and drill down to see time spent on each line of source code. Focus optimization efforts on hot code for the greatest performance impact.
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/IE_DG_supported_plugins_AUTO_debugging-img03-hotspots.png
|
||||
.. image:: _static/images/OV_UG_supported_plugins_AUTO_debugging-img03-hotspots.png
|
||||
:align: center
|
||||
@endsphinxdirective
|
||||
6. Start the analysis by clicking the start button. When it is done, you will get a summary of the run, including top hotspots and top tasks in your application:
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/IE_DG_supported_plugins_AUTO_debugging-img04-vtunesummary.png
|
||||
.. image:: _static/images/OV_UG_supported_plugins_AUTO_debugging-img04-vtunesummary.png
|
||||
:align: center
|
||||
@endsphinxdirective
|
||||
7. To analyze ITT info related to the Auto plugin, click on the **Bottom-up** tab, choose the **Task Domain/Task Type/Function/Call Stack** from the dropdown list - Auto plugin-related ITT info is under the MULTIPlugin task domain:
|
||||
@sphinxdirective
|
||||
.. image:: _static/images/IE_DG_supported_plugins_AUTO_debugging-img05-vtunebottomup.png
|
||||
.. image:: _static/images/OV_UG_supported_plugins_AUTO_debugging-img05-vtunebottomup.png
|
||||
:align: center
|
||||
@endsphinxdirective
|
||||
|
||||
@ -205,7 +205,7 @@ The next time the model is compiled, the cached representation will be loaded to
|
||||
These transformations take a significant amount of time during model compilation, so caching this representation reduces time spent for subsequent compilations of the model,
|
||||
thereby reducing first inference latency (FIL).
|
||||
|
||||
See [model caching overview](@ref openvino_docs_IE_DG_Model_caching_overview) for more details.
|
||||
See [model caching overview](@ref openvino_docs_OV_UG_Model_caching_overview) for more details.
|
||||
|
||||
### Extensibility
|
||||
CPU plugin supports fallback on `ov::Op` reference implementation if the plugin do not have its own implementation for such operation.
|
||||
@ -217,7 +217,7 @@ To enable fallback on a custom operation implementation, one have to override `o
|
||||
### Stateful models
|
||||
CPU plugin supports stateful models without any limitations.
|
||||
|
||||
See [stateful models guide](@ref openvino_docs_IE_DG_network_state_intro) for details.
|
||||
See [stateful models guide](@ref openvino_docs_OV_UG_network_state_intro) for details.
|
||||
|
||||
## Supported properties
|
||||
The plugin supports the properties listed below.
|
||||
|
||||
@ -9,7 +9,7 @@
|
||||
openvino_docs_OV_UG_query_api
|
||||
openvino_docs_OV_UG_supported_plugins_CPU
|
||||
openvino_docs_OV_UG_supported_plugins_GPU
|
||||
openvino_docs_IE_DG_supported_plugins_VPU
|
||||
openvino_docs_OV_UG_supported_plugins_VPU
|
||||
openvino_docs_OV_UG_supported_plugins_GNA
|
||||
openvino_docs_OV_UG_supported_plugins_ARM_CPU
|
||||
|
||||
|
||||
@ -112,7 +112,7 @@ For POT quantized model `ov::hint::inference_precision` property has no effect e
|
||||
### Models caching
|
||||
Cache for GNA plugin may be enabled via common OpenVINO `ov::cache_dir` property due to import/export functionality support (see below).
|
||||
|
||||
See [Model caching overview page](@ref openvino_docs_IE_DG_Model_caching_overview) for more details.
|
||||
See [Model caching overview page](@ref openvino_docs_OV_UG_Model_caching_overview) for more details.
|
||||
|
||||
### Import/Export
|
||||
|
||||
@ -159,7 +159,7 @@ Import model:
|
||||
### Stateful models
|
||||
GNA plugin natively supports stateful models.
|
||||
|
||||
Please refer to [Stateful models] (@ref openvino_docs_IE_DG_network_state_intro) for more details about such models.
|
||||
Please refer to [Stateful models] (@ref openvino_docs_OV_UG_network_state_intro) for more details about such models.
|
||||
|
||||
> **NOTE**: Typically, GNA is used in streaming scenarios, when minimizing the latency is important. Taking into account that POT does not support the `TensorIterator` operation, the recommendation is to use the `--transform` option of the Model Optimizer to apply `LowLatency2` transformation when converting an original model.
|
||||
|
||||
@ -282,9 +282,9 @@ For POT to successfully work with the models including GNA3.0 2D convolutions, t
|
||||
|
||||
Intel® GNA plugin supports the processing of context-windowed speech frames in batches of 1-8 frames.
|
||||
|
||||
Please refer to [Layout API overview](@ref openvino_docs_OV_Runtime_UG_Layout_Overview) to determine batch dimension.
|
||||
Please refer to [Layout API overview](@ref openvino_docs_OV_UG_Layout_Overview) to determine batch dimension.
|
||||
|
||||
To set layout of model inputs in runtime use [Optimize Preprocessing](@ref openvino_docs_OV_Runtime_UG_Preprocessing_Overview) guide:
|
||||
To set layout of model inputs in runtime use [Optimize Preprocessing](@ref openvino_docs_OV_UG_Preprocessing_Overview) guide:
|
||||
|
||||
@sphinxtabset
|
||||
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
Supported Devices {#openvino_docs_IE_DG_supported_plugins_Supported_Devices}
|
||||
Supported Devices {#openvino_docs_OV_UG_supported_plugins_Supported_Devices}
|
||||
==================
|
||||
|
||||
The OpenVINO Runtime can infer models in different formats with various input and output formats. This section provides supported and optimal configurations per device. In OpenVINO™ documentation, "device" refers to an Intel® processors used for inference, which can be a supported CPU, GPU, VPU (vision processing unit), or GNA (Gaussian neural accelerator coprocessor), or a combination of those devices.
|
||||
|
||||
@ -1,4 +1,4 @@
|
||||
# VPU devices {#openvino_docs_IE_DG_supported_plugins_VPU}
|
||||
# VPU devices {#openvino_docs_OV_UG_supported_plugins_VPU}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue
Block a user