* Revert "Fix experimental detectron do ref impl (#10621)"
This reverts commit d87233863d.
* Disabled Experimental Detectron per agreement with GPU team. Ticket to fix it: 90209
* Add Overview page
* Revert "Add Overview page"
* init (#11985)
* [GPU] Pass convolution unit tests on DG2 (#12056)
* scale -> eltwise
* Proofreading-OV-Runtime (#11658)
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/protecting_model_guide.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/optimization_guide/dldt_deployment_optimization_common.md
Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/HDDL.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/HDDL.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/HDDL.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/ov_dynamic_shapes.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/config_properties.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/config_properties.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/performance_hints.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Apply suggestions from code review
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/performance_hints.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/preprocessing_details.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/performance_hints.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Apply suggestions from code review
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Apply suggestions from code review
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Apply suggestions from code review
* Update ref links
* Update Getting_performance_numbers.md
* Update deployment_intro.md
* Update preprocessing_details.md
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update automatic_batching.md
* Update docs/OV_Runtime_UG/automatic_batching.md
* Update docs/OV_Runtime_UG/ShapeInference.md
* Update deployment-manager-tool.md
* Update deployment-manager-tool.md
* Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md
* Update automatic_batching.md
* Update automatic_batching.md
* Update docs/OV_Runtime_UG/ShapeInference.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update integrate_with_your_application.md
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/model_representation.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/model_representation.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update integrate_with_your_application.md
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update model_representation.md
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update integrate_with_your_application.md
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update Additional_Optimizations.md
Removing redundant information.
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update Additional_Optimizations.md
* Apply suggestions from code review
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update Additional_Optimizations.md
* Update docs/OV_Runtime_UG/model_representation.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/OV_Runtime_UG/layout_overview.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update model_representation.md
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/SaturationIssue.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/SaturationIssue.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/SaturationIssue.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/SaturationIssue.md
* Update tools/pot/docs/SaturationIssue.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update README.md
* Update README.md
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/Introduction.md
* Update tools/pot/docs/AccuracyAwareQuantizationUsage.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Removing one-liners
Removing introductory sentences from 'Supported Features' sections.
* Update docs/OV_Runtime_UG/openvino_intro.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Update docs/benchmarks/performance_benchmarks_ovms.md
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/Introduction.md
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* Update tools/pot/docs/DefaultQuantizationUsage.md
* Update tools/pot/docs/BestPractices.md
* Update tools/pot/docs/BestPractices.md
* Update tools/pot/docs/AccuracyAwareQuantizationUsage.md
* Update docs/optimization_guide/model_optimization_guide.md
* Update docs/optimization_guide/dldt_deployment_optimization_guide.md
* Update docs/OV_Runtime_UG/supported_plugins/config_properties.md
* Update docs/OV_Runtime_UG/supported_plugins/GNA.md
* Update docs/OV_Runtime_UG/supported_plugins/CPU.md
* Update docs/OV_Runtime_UG/preprocessing_usecase_save.md
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com>
Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
* updated to fuse activation in eltwise_vload8 (#12084)
* [GPU] Fix gather data type issue (#12085) (#12085)
* setting tput as the default performance mode only for AUTO, excluding MULTI plugin. (#12083)
Signed-off-by: ywang2 <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* [C API][COVERITY SCAN]Fix the TAINTED_SCALAR and DEADCODE in Coverity Scan (#12087)
* Fix the Coverity scan issues
* Fix the insecure data handling (TAINTED_SCALAR) issue found in coverity scan
* [hotfix] pytest error of act_act example (#12093)
* [hotfix] pytest error of act_act example
* remove needless import
* NonZero operation: uncomment tests since they can be passed now (#11548)
* NonZero operation: uncomment tests since they can be passed now
# Conflicts:
# src/tests/functional/plugin/cpu/shared_tests_instances/skip_tests_config.cpp
* Unbreak tests once more by changing base class from LayerTestsCommon to SubgraphBaseTest
* Unbreak compilation / style
* Add test case for cache
Co-authored-by: Chenhu Wang <chenhu.wang@intel.com>
* Increase zeroes count for NonZero tests
* Correct the change
* Remove my previous changes and add dynamic shapes / repeatable shapes into the correct file
Co-authored-by: Chenhu Wang <chenhu.wang@intel.com>
* [SAMPLES] Remove unused commandline arguments for speech_sample (#11892)
* GNA SF propagation fix (#11806)
* Fix the uninitialized value issue found in Coverity Scan (#12098)
* [GPU] Assign-6 and ReadValue-6 (#11780)
* Add methods for access to varables information in Program class
* add ReadValue and Assign primitives
* ReadValue and Assign implementations
* Implementation of memory states allocation
* Add output existance check in primitive_inst to avoid crashes if output is set during execution
* Add memory states management functionality in network component
* Integration of memory states feature in inference request component
* Exclude constant path for read_value and assign nodes in cldnn transformations
* Improve memory states test to run on a single inference request
* unit tests for ReadValue and Assign
* single-layer test for ReadValue and Assign
* Add QueryState API implementation
* Add memory state test which covers dynamic batch case
Co-authored-by: Oleksii Khovan <okhovan@lohika.com>
* [GNA] Add automatic model splitting for compiled graphs (#12001)
* DOCS-code-reference-css-style-change (#12109)
code formatting changed from blue to black, to distinguish from links
* Virtual destructor for the base class (#12102)
* [GPU] Pass Resample unit tests on DG2 (#12052)
* fix validate_fusings_gpu error
* fix biased scale testcase
* [GPU] Pass lrn unit tests on DG2 (#11986)
* [GPU] Pass reduce unit tests on DG2 (#12086)
* scale to eltwise
* [CPU] Move cpu_dump_check into CPU plugin's tools folder (#12100)
* Move cpu_dump_check into CPU plugin's tools folder
* remove cpu from names
* Update README
* Zlib update to 1.12.2 (#12128)
* [GNA] Reduce impact of sf propagation fix (#12115)
* [GPU] Simplify namespaces in the plugin part (#12121)
* [GNA] Add support for future devices with relaxed capabilities (#12000)
* [GPU] Pass eltwise unit tests on DG2 (#12113)
* check fusion in onednn too
* [GPU] modify fusing condition for reduce (#12119)
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* Enable tensor offset to GemmKernelRef for input padding support (#12133)
Signed-off-by: Andrew Park <andrew.park@intel.com>
* [PYTHON][BENCHMARK_APP] Add BGR covert to Gray function (#12118)
* Fix the JIRA 80700 issue. Add BGR covert to Gray function
* Support NCHW and NHWC
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* [CPU] revert pr 11990 and enable brgconv avx512 on SPR by default (#12105)
* polish onednn cc readme (#12114)
* [ONNX] Add operator com.microsoft.Fusedgemm support into frontend/onnx (#11878)
* [GPU] Implement NMS-9 operation (#11890)
* Fix GPU NonMaxSuppression implementation
* Introduce Nms9 single layer tests
* Adapt internal NMS and GPU implementation for NMS9 implementation
* Adapt CPU implementation in GPU for NMS9
* Add blocked layouts support to NMS
* Add unit tests for blocked formats for NMS
* Fix boxes groups size for the small shapes
* Use ocl implementation for blocked layout input
* Fix templates typedefs to pass win build
* Fix second output to set data in correct format
* [POT] optimizer - update usage of IndexSampler (#12146)
* Revert "[GPU] Pass activation unit tests on DG2 (#11969)" (#12167)
This reverts commit 3334e8933c.
* Fix IRDFT for case when axes are in reversed order (#12155)
* [MO] Fix output shape bug in GatherNDDecomposition (#12110)
* [GPU] Add reorder from i32 to f32 for max-pooling/conv/fc which doesn't support i32 (#12137)
* Update pypi.org pages (#12170)
* fix references
* update links
* update the wording to be more clear
* add the error message about Visual studio back
* update links to static html links of 2022.2
* Ubuntu 22.04 support (#11472)
* Ubuntu 22.04 support
* Try to fix setuptools
* Try to fix arm
* Try to add more packages
* Test 2
* test 3
* Turn dependnecies download off
* Fix
* Fix
* Fix
* Fix
* Fix
* test
* Fix
* restore eveything
* Try to restore
* Restore install_openvino_dependencies.sh
* clean-up raspbian
* New if conditions
* Removed excess dependencies
* COsmetic chnages
* Removed autools
* Removed libgkt-2
* Added HDLDL libs
* Test
* Removed some dependnecies
* Better fixes
* Removed some dependencies
* Fixed compilation
* Removed all extra
* [GPU] optimize permute_ref (#12159)
* change memory access pattern of fsv layout for permute
* Fix permute_ref to process F first only when (bf...) => (b...f)
* Refactor
Co-authored-by: si-eun-kim <sieun.kim@intel.com>
* Update of naming of the last operators in the graph (#12139)
* Update opset.md with opset9 (#12169)
* [GPU] integrate persistent caching for onednn (#12094)
* integrate persistant caching for onednn
* add api to save/load binary file.
* Check memory allocation size of network graph (#11911)
+ Add exception handling for out of resource
* TI repetative shape inference (#12178)
* Fixes for system libraries pugixml, tbb (#12206)
* Fixes for system libraries pugixml, tbb
* Added more dependencies for core
* Debian packages: base version (#11387)
* Xp/benchmark app ocl (#12112)
* Add some tip description about enable OpenCL for benchmark_app.
Signed-off-by: xipingya <xiping.yan@intel.com>
* Export doesn't work, we need to add -Dopencl_root_hints=[PATH]/OpenCL-CLHPP/include to cmake command.
Signed-off-by: xipingya <xiping.yan@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* ONNX: Pass name to the InputEdge (#12177)
* [IE TESTS][CONFORMANCE] Fix OpImplCheck Precision (#12148)
* add new article for using binaries
* [PyOV][DOCS] Python API contribution and developer guide (#12145)
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* [DOC][CPU] Denormals optimization doc (#12127)
* Use system pugixml where it's possible (#12218)
* Restore FEM to be static instance (#12219)
* Restore FEM to be static instance
* Restore frontend manager in ie_read_network.cpp
* [MO] Fix TopK partial shape inference with dynamic K (#12212)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [CPU] Fixed heap sort bug regarding heapifying (#12221)
* [CPU] Explicitly enable DNNL_VERBOSE only in case of CPU_DEBUG_CAPS (#12108)
* [GNA] Fixed convolutions with shared transpose and un-fuse-able activations after Convolution filter (Renew PR11373) (#12152)
* Commits from PR11373:
Fixed handling of transpose after convolution
[GNA] Fixed calculation of dimensions for ConvolutionFilter and PWL primitives
[GNA] Fixed coverity error and failed tests
* Apply comments
* Update src/plugins/intel_gna/gna_graph_compiler.cpp
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* Update src/plugins/intel_gna/gna_graph_compiler.cpp
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* Rollback names
* Separate test data
* Move coverity issue to separate request
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
* [GNA] Fix accuracy degradation in compact mode (#12150)
* [TF FE] Handle optional attributes for Convolutional operations (#12230)
* [TF FE] Handle optional attributes for Convolutional operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* update the information for pypi.org pages
* [GPU] ROIAlign v9 support (#11899)
* ROIAlign v9 support
* Code changes after review1
* Code changes after review2
* fix of single layer test for Windows
* Since PR #12043 we don't need strong include order of primitive_base.hpp and
impls/implementation map.hpp anymore
* Code changes after review3
* Code changes after review4
* update the verifying checksum step
* Fixed WIndows backslash paths (#12250)
* update install_dir info
* Move GNU build flag to "cmake/developer_package/compile_flags/sdl.cmake" (#12143)
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* [MO] Fix Mul fusion with dynamic dimension (#12253)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* updates
* update wording for pypi.org
* Fixed newAPI for case if core was removed (#12207)
* Fixed newAPI for case if core was removed
* Fixed code style
* Fixed typo
* Use new API by default
* Create core with template plugin
* Added doxygen comment
* Install user provided TBB as well (#12260)
* Disable loading of v7 reader for new IR versions (#12252)
* Disable loading of v7 reader for new IR versions
* Try to fix CI
* Fixed PDPD frontend
* Fixed error message creation
* Fixes for cases when TBB_DIR env var is set (#12266)
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* [GPU] Get rid of direct layout::size field usages (#12172)
* [GPU] Get rid of direct layout::size field usages to simplify further replacement
* [GPU] Enabled -Wall and resolved compiler complaints
* Update summarize.py (#12175)
* [CPU] Add RDFT and IRDFT operators (#12099)
* [CPU] Add RDFT and IRDFT operators
Tickets: 79178 and 79192
Co-authored-by: Mateusz Bencer <mateusz.bencer@intel.com>
* Remove Interpolate Transposes as it does nothing (#12205)
* [TF FE] Implement LinSpace and BatchMatMul translators (#12271)
* [TF FE] Implement LinSpace and BatchMatMul translators
It helps to convert STN model (from e2e testing) using TensorFlow frontend
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix BatchMatMul translator
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix LinSpace operation translator
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update error message on pypi.org (#12243)
* Add Overview page
* Revert "Add Overview page"
* fix references
* update links
* update the wording to be more clear
* add the error message about Visual studio back
* update links to static html links of 2022.2
* port changes to master
* update description
* update commands and uninstallation
* Add const fold check in operators instead pass (#12189)
* Add const fold check in operators instead pass
- refactor constant fold pass to using ov instead of ngraph
- add constant_folding_is_disabled overload for raw pointer
* Remove Reshape from skip const inferences
in legacy graph transformer
* Const fold test for modified operators
* [GPU] Use int64_t type for axis in softmax (#12287)
* remove obsolete info from source files to avoid confusion
* [DOC] [CPU] Proofreading for grammatical and stylistic corrections (#12288)
* Porting to master - update -readme for CPP and Python benchmark (#12245)
Porting #11961
* Fixed build_samples.sh not to call setupvars.sh for Debian package case (#12309)
* Investigate GNA tests (#12267)
* Test commit
* Revert "Disable loading of v7 reader for new IR versions (#12252)"
This reverts commit cb6ca7bb89.
* Revert "Test commit"
This reverts commit 977b83f2ba.
* [PyOV] Test refactoring (#12248)
* [GNA] Add missing support for batch normalization with weights broadcasting. Add unit tests. (#12301)
* Xiaoxia/onetbb old version (#12303)
* support oneTBB old version
* fix oneTBB version mismatch issues
* fix clang issue
* add 'tbb' path to setupvars.sh and OpenVINOConfig.cmake.in
* Update scripts/setupvars/setupvars.sh
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* simple Windows installer POC (#12308)
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* First version of Windows installer
* WIndows NSIS installer
* [GPU] Fix get_default_params & choose_impl not to dependent on program_node (#12239)
* Getting rid of dependency from get_default_param for typed_program_node
* Fix bug
* Enable two pathes to call choose_impl / does_possible_impl_exists / does_an_impl_exists to be able to use given layout
* Replaced impl factory API to get kernel_impl_param's pointer
* Update for recently added primitives
* Add and apply optional_layout
* fix kernel_param_impl to be handled as unique_ptr
* Applied review comments
* Fix rebase conflict
* Fix CI error
* [CC]Fix CC issue for transformation (#12292)
* Revert "Fixed 3 naming issue"
This reverts commit a92d3cfff5.
* Revert "Fix CC issues for transformation and snippets"
This reverts commit d08a3f5aac.
* Fix NGRAPH_PASS_CALLBACK issue to make it can work
* Fix matcher name missing issue
* [TF FE] Fix conversion of NetVLAD model (#12328)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [MO] Fix broken port numbering for Constant operations (#12318)
* Restore inputs order in IR Reader
* Fix broken port numbering for Constant operations
Co-authored-by: Chetverikov <anton.chetverikov@intel.com>
* [GPU] Align TopK parameters with ngraph (#12278)
* [GPU] Use int64_t type for axis in CumSum (#12306)
* [GPU] Use int64_t type for axis in ScatterElementsUpdate (#12323)
* Bump OMZ submodule to fix pip-conflicts ssues (#12320)
* [PyOV] Enable type casters (#12204)
* add type caster for ov::Layout, enable load method to take pathlibs.Path as arugment
* fix typo
* fix style
* add missing blank line
* add common function to check if py::object is either Path or string
* fix style
* Update src/bindings/python/src/pyopenvino/graph/preprocess/pre_post_process.hpp
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* add tests, fix style, remove pointer argument overload
* fix style
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
* [GNA] Replace GNA SoftSign by opset9 SoftSign (#12302)
* Replace GNA SoftSign by opset9 SoftSign
* v9 -> opset9
* [GPU] ScatterUpdate axis alignment (#12233)
* [GPU] added is_dynamic methods to program_node and primitive_inst. Minor refactoring (#12322)
* updates
* [GPU] Remove dependency to typed_program_node from calc_output_layout (#12378)
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Use static pointers to frontend libraries (#12235)
* Add static shared_objects map in FEM
- add unit tests for frontend lib close
- not use static FEM in ie network reader
- add main for gtest which can use manifest file to filter tests
* Move library pointers map to manger impl
- add to manger impl method to make frontend from loaded plugin
* Add shutdown function to ov namespace
it cleans the static resources
* Revert changes related to linking mian for tests
* Add python binding to ov::openvino_shutdown
* Renamed shutdown method and added to legacy C++ API
(cherry picked from commit a8395bd207)
* Added C bindings
(cherry picked from commit d2c9ddc263)
* Move frontend lib close test to ieFunctTest
- moved to not introduced new test binary and modification on CI
the frontend tests use dynamic linked frontend lib which is load
on test application start and mask lib close tests
- remove gtest_main_manifest as not required now
- add ov::shutdown test to expect application crash
* Fix lib_close test
- remove not get_disabled_tests from utils
- revert CMake file formating
* Fix get model path in lib close tests
* Skip frontend lib close tests if static lib build
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* Decompose NormalizeL2 on GPU (#12361)
* [TF FE] Implement translators for TensorFlow ConvBackpropInput operations (#12356)
* [TF FE] Implement ConvBackPropInput translators
Now the translators supports dynamic input_sizes attribute and different padding modes
including EXPLICIT mode
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix clang-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback and fix build issues
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: check for input size
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix retrieving explicit_padding attribute
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code style
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add debug log showing the result transformation callback (#12365)
* [AUGRU] AUGRUCell/Sequence op specification (#12162)
* [GPU] Add exception handling for calc_output_layout (#12393)
* Add exception handling for calc_output_layout
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Apply comment to error handler
Signed-off-by: Andrew Park <andrew.park@intel.com>
* [GPU]get data type of conv weights from node.weights() when network is internal (#12232)
* get data type of convolution weights from node.weights() when network is internal
* use only instance.node.weights().get_output_layout().data_type
* fix typo
* add unit test for the case
* Update pre_replace_deconv to support output_shape for transposed conv (#12335)
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Improved OpenVINO debian packages (#12385)
* [GPU] implement lru_cache(#12349) (#12349)
* Fix memory leak issue
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
* DOCS-fix_maths_formatting (#12402)
mathematical equation formatting issue fixed in POT readme for range supervision
* [GPU] Pass concat unit tests on DG2 (#12142)
* check optimized
* skip kernel compile when optimized
* GroupedGatherElimination short circuit (#12380)
* Disable GroupedGatherElimination in case of scalar inputs containing indices
* clang format
* [MO, POT] Top up upper bounds for TensorFlow and NumPy modules in all requirement files (#12191)
* [MO] Relax MO upper-bound requirements for TensorFlow and NumPy
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Just debug numpy version
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Pin upper-bounds for NumPy and TensorFlow modules in all reqs files
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update submodule dependency for open_model_zoo
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Install numpy module first
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy version in POT setup.py
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Extend telemetry tests with a set of possible solutions for events
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Update NumPy module version for layer tests
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [GPU] Added common impl for optionals (#12366)
* [LPT] Correct a check for whether model is quantized (#12364)
Look inside subgraph operations, such as TensorIterator, Loop, If, etc
* Update doc for AUTO and AUTO_BATCH (#12265)
* Update doc for AUTO and AUTO_BATCH
Signed-off-by: Chen Peter <peter.chen@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* fix: incorrect fq type (#12234)
Co-authored-by: Wonju Lee <wonju.lee@intel.com>
* Implement workaround to convert non-frozen models using new TensorFlow frontend (#12386)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert "Merge branch 'master' into add-install-binaries-22/2"
This reverts commit f4d6f04636, reversing
changes made to e505e739e2.
* update comments
* update comments
* Update docs/install_guides/installing-openvino-from-archive-windows.md
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* update OpenCV installation
* Update docs/install_guides/uninstalling-openvino.md
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* Update docs/install_guides/uninstalling-openvino.md
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* Update docs/install_guides/uninstalling-openvino.md
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* update uninstall wording
* add C++ redistributable to pypi.org pages
* update pypi.org pages and opencv for macOS
* update whats next
* add a note about long paths on Windows
* fix errors
* update CMake dependency
* fix formatting
* apply the same changes from Ilya's comments
* update uninstall, remove dev from pkg names
* update C++ requirements according to Ilya's requests
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Signed-off-by: Andrew Park <andrew.park@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Co-authored-by: Felix Dohyun Kim <tuxedcat@gmail.com>
Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com>
Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com>
Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
Co-authored-by: Eddy Kim <eddy.kim@intel.com>
Co-authored-by: Paul Youngsoo Ahn <paul.y.ahn@intel.com>
Co-authored-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
Co-authored-by: RICKIE777 <ruiqi.yang@intel.com>
Co-authored-by: Bonhun Koo <bonhun.koo@intel.com>
Co-authored-by: avoskoboinyk-lohika <avoskoboinyk@lohika.com>
Co-authored-by: Chenhu Wang <chenhu.wang@intel.com>
Co-authored-by: Marcin Kusmierski <marcin.kusmierski@intel.com>
Co-authored-by: Szymon Irzabek <szymon.jakub.irzabek@intel.com>
Co-authored-by: Yaroslav Torzuk <yaroslav.torzuk2@altran.com>
Co-authored-by: Oleksii Khovan <okhovan@lohika.com>
Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>
Co-authored-by: Tingqian Li <tingqian.li@intel.com>
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
Co-authored-by: Min, Byungil <byungil.min@intel.com>
Co-authored-by: Andrew Kwangwoong Park <andrew.park@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
Co-authored-by: Luo Cheng <cheng.luo@intel.com>
Co-authored-by: zihan wu <zihan.wu@intel.com>
Co-authored-by: sheng.gui@intel.com <guisheng315@sina.com>
Co-authored-by: Tetiana Gubanova <tgubanova@lohika.com>
Co-authored-by: Mateusz Bencer <mateusz.bencer@intel.com>
Co-authored-by: Przemyslaw Wysocki <przemyslaw.wysocki@intel.com>
Co-authored-by: Kelvin Choi <kelvin.choi@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
Co-authored-by: si-eun-kim <sieun.kim@intel.com>
Co-authored-by: Katarzyna Mitrus <katarzyna.mitrus@intel.com>
Co-authored-by: Sungeun Kim <sungeun.kim@intel.com>
Co-authored-by: Jade Cho <jade.cho@intel.com>
Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
Co-authored-by: Xiping Yan <xiping.yan@intel.com>
Co-authored-by: Artur Kulikowski <artur.kulikowski@intel.com>
Co-authored-by: Irina Efode <irina.efode@intel.com>
Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com>
Co-authored-by: River Li <river.li@intel.com>
Co-authored-by: Roman Kazantsev <roman.kazantsev@intel.com>
Co-authored-by: Chen Xu <chen.xu@intel.com>
Co-authored-by: Egor Duplenskii <egor.duplensky@gmail.com>
Co-authored-by: Nadezhda Ageeva <nadezhda.ageeva@intel.com>
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
Co-authored-by: Konstantin Beluchenko <kostiantyn.bieliuchenko@altran.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Mateusz Tabaka <mateusz.tabaka@intel.com>
Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>
Co-authored-by: Roman Lyamin <Roman.Lyamin@intel.com>
Co-authored-by: almilosz <108654258+almilosz@users.noreply.github.com>
Co-authored-by: Sun Xiaoxia <xiaoxia.sun@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Co-authored-by: Chetverikov <anton.chetverikov@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
Co-authored-by: Bartek Szmelczynski <bartosz.szmelczynski@intel.com>
Co-authored-by: Wilson Seok <wilson.seok@intel.com>
Co-authored-by: Inhyuk Jo <andy.inhyuk.jo@intel.com>
Co-authored-by: Wonju Lee <wonju.lee@intel.com>
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* [TF FE] Add Transpose Sinking for Prelu operation
Now it covers a case with a scalar slope.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add unit-tests for Transpose sinking of Prelu
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix non-scalar slope case
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
According to the specification we must have the same type for block_shape and crops inputs
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Make model reshape and track batch (#12736)
* CVS-89672 Make model reshape and track batch
* Minor refactoring
* Changed mechanism of constant replacement to more mature
* Update src/common/transformations/include/transformations/smart_reshape/lstm_states_broadcast.hpp
* Update src/common/transformations/src/transformations/smart_reshape/lstm_states_broadcast.cpp
* Comments resolving
* Style and getting rid of asserts
* style
* Apply suggestions from code review
* Fix GPU NonMaxSuppression implementation
* Introduce Nms9 single layer tests
* Adapt internal NMS and GPU implementation for NMS9 implementation
* Adapt CPU implementation in GPU for NMS9
* Add blocked layouts support to NMS
* Add unit tests for blocked formats for NMS
* Fix boxes groups size for the small shapes
* Use ocl implementation for blocked layout input
* Fix templates typedefs to pass win build
* Fix second output to set data in correct format
Co-authored-by: Tetiana Gubanova <tgubanova@lohika.com>
* [TF FE] Add Transpose Sinking for additional unary-wise Operations
It helps to fix performance degradation for MobileNet models
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Add LogicalNot for Transpose sinking
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Support dynamic rank support for Convolutional and Pooling operations (#12661)
* [TF FE] Add dynamic rank support for Convolutional and Pooling operations
Refactor DepthwiseConv2D, AvgPool, and FusedBatchNorm operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue with rvalue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix build issue with climit
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Skip duplication of Parameter nodes
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert changes in StridedSlice and add check for AvgPool operation type
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Revert the rest of changes for StridedSlice
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix translator for AvgPool: add pad mode
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Introduce helper default_op_checks
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Refactor translators for Resize operations and correct Pooling (#12721)
* [TF FE] Refactor translators for Resize operations and correct Pooling
It allows to convert magenta_arbitrary-image-stylization model
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Align TF FE tranlator for Resize with legacy frontend
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Do minor fix for MaxPool
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Disable GroupedGatherElimination in case of scalar inputs containing indices
* clang format
Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>
* Add overrided method to generating vector of strings
* Trim the value from the the left and right
* Add test to verify that output names are correctly read from IR
* Use spaces instead of tabs
* Add C++ tests for read model contains outputs with whitespaces
* Fix test for add output
* Remove python test
* Cherry-pick U22 adoption in github actions
* More fixes for shellcheck
* More fixes for shellcheck
* Update .github/workflows/py_checks.yml
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* [TF FE] Handle optional attributes for Convolutional operations (#12230)
* [TF FE] Handle optional attributes for Convolutional operations
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Implement LinSpace and BatchMatMul translators (#12271)
* [TF FE] Implement LinSpace and BatchMatMul translators
It helps to convert STN model (from e2e testing) using TensorFlow frontend
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix BatchMatMul translator
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix LinSpace operation translator
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code style rules
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Fix conversion of NetVLAD model (#12328)
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Implement translators for TensorFlow ConvBackpropInput operations (#12356)
* [TF FE] Implement ConvBackPropInput translators
Now the translators supports dynamic input_sizes attribute and different padding modes
including EXPLICIT mode
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix clang-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code-style issue
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback and fix build issues
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Apply code-review feedback: check for input size
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix retrieving explicit_padding attribute
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Fix code style
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Fix StridedSlice translator for new_axis vector size longer input rank (#12442)
* [TF FE] Fix StridedSlice translator for new_axis vector longer input rank
Currently, new_axis vector is cut by input rank that is correct and leads to the loss of new axes.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Use int64 type in mask_to_vector function
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Refactor translators for Conv2d and Conv3d (#12444)
It allows to convert CNN-Transformer model. Padding was previously incorrect.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Implement conversion for Attention OCR model (#12428)
* [TF FE] Implement conversion for Attention OCR model
The following scope of work is done to make Attention OCR convertable:
1. Refactored translators for BiasAdd, Slice, and ArgMax operations. Add translation for StopGradient operation.
2. The previous traversing algorithm to compute topological sorted nodes list was incorrect. Now it is implemented based on topologically_sorted function from core/graph_util.hpp.
3. The unsupported data types are now preliminary converted to undefined type for the purpose of to have them cut off.
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* [TF FE] Refactor MaxPool operation translator for xj_feature model (#12485)
* [TF FE] Refactor MaxPool operation translator for xj_feature model
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
* Correct MaxPoolV2 since it has three inputs
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
Signed-off-by: Kazantsev, Roman <roman.kazantsev@intel.com>
+ benchmark cache_dir option takes longer than cl_cache_dir env in loading network.
+ For clDNN execution, benchmark cache_dir created onednn_engine if just ONEDNN_ENABLE config is ON.
+ Creation of onednn_engine in ocl_engine is changed to on-demand.
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Signed-off-by: Min, Byungil <byungil.min@intel.com>
* Revert "[C API] Enable hello_nv12_input_classification samples for C APIs of OV API 2.0 (#12031)"
This reverts commit 70d967ffb6.
* Revert "Add hello_classification_ov_c test (#11933)"
This reverts commit ebeb0a3802.
* Revert "Refine ov_partial_shape for OV API 2.0 C interface (#11891)"
This reverts commit ce5b2c6a45.
* Revert "Enable unit test for OV 2.0 C API (#11828)"
This reverts commit c4fdcafa70.
* Revert "OV 2.0 C API (#11700)"
This reverts commit 8faf8f2d89.
* Revert "Fixed 3 naming issue"
This reverts commit a92d3cfff5.
* Revert "Fix CC issues for transformation and snippets"
This reverts commit d08a3f5aac.
* Fix NGRAPH_PASS_CALLBACK issue to make it can work
* Fix matcher name missing issue
* Add static shared_objects map in FEM
- add unit tests for frontend lib close
- not use static FEM in ie network reader
- add main for gtest which can use manifest file to filter tests
* Move library pointers map to manger impl
- add to manger impl method to make frontend from loaded plugin
* Add shutdown function to ov namespace
it cleans the static resources
* Revert changes related to linking mian for tests
* Add python binding to ov::openvino_shutdown
* Renamed shutdown method and added to legacy C++ API
(cherry picked from commit a8395bd207)
* Added C bindings
(cherry picked from commit d2c9ddc263)
* Move frontend lib close test to ieFunctTest
- moved to not introduced new test binary and modification on CI
the frontend tests use dynamic linked frontend lib which is load
on test application start and mask lib close tests
- remove gtest_main_manifest as not required now
- add ov::shutdown test to expect application crash
* Fix lib_close test
- remove not get_disabled_tests from utils
- revert CMake file formating
* Fix get model path in lib close tests
* Skip frontend lib close tests if static lib build
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Pawel Raasz <pawel.raasz@intel.com>
* Fixed WIndows backslash paths (#12250)
* Install user provided TBB as well (#12260)
* Fixes for cases when TBB_DIR env var is set (#12266)
* Fixes for cases when TBB_DIR env var is set
* Don't use make in build_samples.sh script
* Xiaoxia/onetbb old version (#12303)
* support oneTBB old version
* fix oneTBB version mismatch issues
* fix clang issue
* add 'tbb' path to setupvars.sh and OpenVINOConfig.cmake.in
* Update scripts/setupvars/setupvars.sh
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* Trying to fix CVS-85530 (#12455)
Co-authored-by: Sun Xiaoxia <xiaoxia.sun@intel.com>
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* Update doc for AUTO and AUTO_BATCH
Signed-off-by: Chen Peter <peter.chen@intel.com>
* Update per the comments
Signed-off-by: Chen Peter <peter.chen@intel.com>
* Move default hint to THROUGHPUT section
Signed-off-by: Chen Peter <peter.chen@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Fixed newAPI for case if core was removed
* Fixed code style
* Fixed typo
* Use new API by default
* Create core with template plugin
* Added doxygen comment
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* fix references
* update links
* update the wording to be more clear
* add the error message about Visual studio back
* update links to static html links of 2022.2
* change memory access pattern of fsv layout for permute
* Fix permute_ref to process F first only when (bf...) => (b...f)
* Refactor
Co-authored-by: si-eun-kim <sieun.kim@intel.com>
* add auto_batch_timeout for MULTI and AUTO
* fix clang-format for ie_core.cpp
* fix coredump
* simplify insert key to deviceConfig logic and parseDeviceNameIntoConfig() check "AUTO" and "AUTO:" only
* check config auto_batch_timeout
* add CleanUpInIECore()
* fix clang-format for ie_core.cpp
* Fix the deconv fused issue on AVX2 and AVX512 and enable deconv test
* Keep GroupDeconv BF16 test cases still disabled.
* Update to also excluding nightly
* Update onednn submodule.
* Update onednn submodule
* Update onednn submodule.
* Update the ONDENN submodule
* Update the ONEDNN commit.
* Update with merged onednn commit.
* Define new ppp API for nv12
* Add new ppp API function
* Add new ppp API unit test
* Add hello nv12 input classification ov
* Define new ppp API for nv12
* Add new ppp API function
* Add new ppp API unit test
* Add hello nv12 input classification ov
* Fix the clang -formate issue
* Modify the function called is_supported_image_size
* Update code as suggested
* Add hello_nv12_input_classification e2e test
* clang-format openvinotoolkit
* Fix the doc error in CI
Co-authored-by: River Li <river.li@intel.com>
Some compiler flags restrict the compiler from making arbitrary decisions while handling undefined C/C++ behaviors.
Therefore they can be used to fix some issues caused by undefined behavior.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* FTZ_and_DAZ_set_for_cpu
* remove DAZ
* fix
* extract to utils
* ie core part changes to add do as property and benchmark_app enable do
* enable brgcov from Luocheng patch
* add debug info
* enable_brgemm_on_avx512
* add python binding
* dlb test
* revert test code
* revert test code
* Handle in-place failure cases in reshape node
* Disable inplace when non-const reshape connected to constant
* Add comment to reshape_inplace test
* move copy WA into execute() to cover more general in-place failure cases
* enable brgconv f32
* use config to enable brgconv f32
* when brg disabled not init bin-postops
* change prop name for extensive
* use more general field
* fix review comments.
* Add FORCE_TBB_TERMINATE to legacy API
* Put this config into proper place
* fix issue in property test
Co-authored-by: Shen, Wanglei <wanglei.shen@intel.com>
* [CPU] Optimize NonZero operation
# Conflicts:
# src/plugins/intel_cpu/src/nodes/non_zero.cpp
* [CPU] Rewrite NonZero implementation, so it will use generic ie_parallel API
* [CPU] NonZero operation: apply an additional optimization
* NonZero operation: add fallback code for inRank >= 6
* NonZero operation: apply review modifications
# Conflicts:
# src/plugins/intel_cpu/src/nodes/non_zero.cpp
* NonZero operation: inShape.getDims().size() -> inRank
* NonZero operation: eliminate input array index calculation by slight modification of ie_parallel API
* Adjust ie_parallel.hpp style for clang-format
* Try to unbreak the build
* Move to parallel_nt and add a cache for nd loops to optimize more
* Add minimal size threshold for threading and reduce warning count
* Try to workaround linter errors
* One more try to unbreak cpplint build
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* Remove vmaxps in store_vector.
This instruction is not needed for dst_prc int8.
And it may lead to wrong result with denormals optimization is on.
* Add vpmaxsd if dst_prc is u8 or u16.
* Enable hint to tput if no property is specified for both AUTO device and target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Update logic.
2. Add test cases.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update. Set hints to default for target device if no hints setting for AUTO plugin and no specific properties setting for target device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
This extra semicolon creates an output as example below. The extra
'::' is equivalent to add '.' as part of the LD_LIBRARY_PATH. This
breaks glibc build, and very often creates weird issue when launch
commands from different path.
...inference_engine/external/tbb/lib::/opt/intel/openvino_2021/...
We also noticed that :${parameter:+:$parameter} is widely used in
this file. Please review the code and fix as needed.
* ExperimentalDetectronDetectionOutput: refine sorting criteria for NMS stage
This is to ensure the operation produces stable predictable results across
the possible sorting algorithm implementaions.
This property is useful for the operation testing.
* [GPU] Implement ExperimentalDetectronDetectionOutput operation
* [GPU] ExperimentalDetectronDetectionOutput: use vector types and operations in kernel
* Reformat changed files to make clang format checker happy
* [GPU] ExperimentalDetectronDetectionOutput: add another test case to the unit test
* [GPU] ExperimentalDetectronDetectionOutput: Add f16 test
* ExperimentalDetectronDetectionOutput: single-layer test: use all three outputs
* [GPU] ExperimentalDetectronDetectionOutput: increase single layer test coverage
More attribute permutations were added.
* add testcase for plugin properties should not be revised by compile_model
* rename smoke_cpuCompileModelBehaviorTests to smoke_gpuCompileModelBehaviorTests
* remove property EXCLUSIVE_ASYNC_REQUESTS in ov2.0 test
* add testcase for plugin properties should not be revised by loadNetwork
* 1. Enable IE Core filter to promote the secondary properties to first level for hardware device.
2. Enable IE Core filter to pass the secondary properties to AUTO plguin.
3. Enable AUTO Plugin to parse secondary properties to first level and pass them to corresponding target hardware device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable MULTI Plugin to support secondary properties.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. Enable HETERO Plugin to support secondary priorities.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Catch the EXPECT_CALL with AVAILABLE_DEVICES argument inputting to GetMetric.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Revert the logic of handling secondary properties for MULTI and HETERO device.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove the secondary property flattening logic because this logic has been implemented within AUTO plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* 1. update flatten logic when secondary properties is specified.
2. add the test case with secondary properties for CPU.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* add the test case with secondary properties for GPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add debug message to fix the test case failure issue.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add more debug info.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
1. For IE Core, 1st level property overides the 2nd level property.
2. For AUTO plugin, add available device list to check if the secondary properties is vaild.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add CUDA and ARM.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update device name for ARM Plugin and add device name for HPU plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
Co-authored-by: Chen Peter <peter.chen@intel.com>
* 1. Enable OPTIMIZATION_CAPABILITIES for AUTO plugin.
2. Add corresponding test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Remove EXPORT_IMPORT as Export is not implemented in the AUTO/MULTI.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* DOCS-structure_workflow
workflow diagram files and formatting
added overview articles on models and deployment
added the ecosystem page and changed the header from addons
* DOCS-structure_dlworkbench
* DOCS-structure_ovtf
* fixed FakeOutputResolver to avoid renaming correctly named nodes
* fixed failed mo_args test: process reverse_input_channels through eltwise with constant with shape=[]
* changed fix to more accuarate to avoid possible issues
* Remove unnecessary iterating over producer outputs
Co-authored-by: sadolini <svetlana.a.dolinina@intel.com>
* Property to force terminate tbb threads
During inference done, tbb threads cannot be closed by itself, which cause memory leak and unload/lingering threads.
Sometimes the tbb threads need to be terminate for resource(memory, thread) consumption
This PR contains:
1. Add a new property to control whether force to terminate tbb threads.
2. Property key is "FORCE_TBB_TERMINATE", default value is false.
3. Explicitly to terminate tbb task scheduler during unload openvino dll if this property is set true.
e.g: core.set_property(device, ov::force_tbb_terminate(true));
4. If not set FORCE_TBB_TERMINATE, there will be no any additional tbb operations.
Change-Id: I32dc0ba122bb19a9dbf3ba12fdd596aad9ac54b4
* Fix executorManager test case
Change executorManager from static to be dynamic, the test case should fit this change.
* Change frontendManger to be non-static instance
Make frontendManger to be non-static instance.
We should guard it is not released before Model, due to Model will use the mem allocated by frontendManger.
So put frontendManager reference in ov::Model to make it work.
* Fix race condition between executor and executorManger
* Add test case for tbb property
1. Add basic test case for ov::force_tbb_terminate property
2. set ov::force_tbb_terminate to be false
* Avoid terminate tbb in case of no tbb thread created
* Fix Constant ops segmentfault issue
There is segmentfault issue during Constant destruction, which is caused by some shared memory is double free
Test case is:
ie = IECore()
net = ie.read_network(model=test_net_xml, weights=test_net_bin)
query_res = ie.query_network(net, device)
func_net = ng.function_from_cnn(net)
ops_net = func_net.get_ordered_ops()
ie and net will be released before ops_net destruction, so Constant will free the shared memory that has been freed
* Make constant::m_data is released before frontendmanager
* tiny format change
* change tbb blocking_terminate to terminate
Tbb blocking_terminate calling will cause some segmentfault during run some special models,
the reason may comes from block_terminate cause current thread block here to wait for tbb exit,
but cannot handle some resource dependencies.
After adopt terminate(), the dependencies can be resolved and no segmentfault any more.
Change-Id: I0b920630a25cd3fd2747c57ec71ca749ba35573b
* Remove unnecessary dependencies
* Disable dynamic lib test case in static library compilation version
As CVS-68982 description, we should disable the test case which will load
dynamic library in openvino static library compilation.
* Fix nested-namespace-definition issue
* Address reviewer's comments
* Refine ov_partial_shape for OV 2.0 C interface
To avoid potential string security problem, remove string pointer from ov_partial_shape structure.
* Remove redundant code
* fix typo issue
* fix shape test issue
* fix some minor issues
* Address reviewing comments
Use Dimension to represent rank of parital shape.
* Appy safer method to parse partialShape string
1. adopt ov::Dimension::value_type to construct ov::Dimension
2. safter method to convert string to dimension value
3. apply std::vector<std::string> to replace std::vector<char *> during pasrsing partialShape string
Change-Id: I0e0b70a915fc5c5fefad51de51f167798854f55e
* Convolution concat sum inplace conflict fix
* Minor refactoring.
* Rebase to OV2.0, build pass.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Remove old file.
Rebase introduce this file by mistake.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Move functional test for subgraph.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Disable some crash test for continue to test others.
* Rename ConcatConvSumInPlaceTest to ReLuConcatConvSumInPlaceTest
fix ci crash issue.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* Revert "Disable some crash test for continue to test others."
This reverts commit f7a8677c002747b45e84f74672f76e2fdfc7ab22.
* Add const for inPlace.
Signed-off-by: Yan, Xiping <xiping.yan@intel.com>
* fix build issue, missing braces;
Co-authored-by: Maksim Kutakov <maksim.kutakov@intel.com>
* Add signal stack management for AMX in linux python API
* fix wording
* fix empty line
* add AT_MINSIGSTKSZ definition
* Fix misspelling and conditional compiling on __linux__
* Change read_image() into generate_image()
* Move test utils from testdata repo to local files
* Minor changes
* Remove unnecessary code
* Minor changes
* Fix compatibility tests
* Fix imports for Azure pipeline
* Move model generation into test_utils
* Minor changes
* Minor changes
* Update linux.yml CI
* Remove testdata repo from .ci/linux.yml
* Remove testdata repo from pipelines
* Fix Azure compatibility tests
* Reset linux.yml
* Remove testdata repo from linux CI
* Try eliminating one of configs
* Attempt at fixing Azure tests
* Add separate utils for compatibility
* xfail comp if op tests
* Minor changes
* Revert changes to .ci files
* minor changes
* Remove xfails
* Remove unecessary import
* Skip if op tests
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* add paddle op top_k_v2
* rebase
* fix variable support issue for paddle top_k_v2
* Update src/frontends/paddle/src/op/top_k_v2.cpp
Co-authored-by: Bo Liu <bo4.liu@intel.com>
* Update src/frontends/paddle/src/op/top_k_v2.cpp
Co-authored-by: Bo Liu <bo4.liu@intel.com>
* Update src/frontends/paddle/src/op/top_k_v2.cpp
Co-authored-by: Bo Liu <bo4.liu@intel.com>
* format the top_k_v2.cpp
Co-authored-by: meiyang-intel <yang.mei@intel.com>
Co-authored-by: Bo Liu <bo4.liu@intel.com>
They sporadically impact CI... possible reason is the order of paddle and openvino is not guaranteed when more than
one bboxes have equal scores.
Actually there is no need for these random tests as the remainding cases have covered them.
* draft pr for planar and fsv16
* draft pr for general test
* update fusion test (failing)
* update fusing test (pass)
* update fusing test (include exception)
* clean gpu unit test
* review comment applied
* unit test cases added & cpplint applied
* cpplint error fixed
* change gpu test cases for fp16
* fusing test fix generate_unique_indices
* fix typo
* revise cl kernel for occasions when updates shape is altered
* Initial files & cmakefiles for ov 2.0 c api development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add all ov 2.0 C APIs define
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix review comments
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Disable test of OV 2.0 C APIs test for tmp
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add related property key for ov 2.0 C-API
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add description for ov_property_key_e
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add EXECEPTION handling
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* compiledModel add interface
* add inferrequest interface
* solve cpplint problem
* Finished OV 2.0 C-APIs PPP related development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix code review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add ov::tensor API
* add compiled model func
* Finished C-API funs about core, model, node development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] add const to ov_output_node
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] Using define GET_OV_ELEMENT_TYPE & GET_CAPI_ELEMENT_TYPE in tensor APIs
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C-API] add string initialize
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* add inferrequest func
* add move construction to runtime_model
* supplement two infer request interface functions
* [OV 2.0 C-API] Add the common framwork of unit test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* modify ov_infer_request_get_profiling_info
* add tests dir
* restore CMakeLists.txt
* Fix the bug of COPY in Tensor
* [OV 2.0 C API] Finished core related function unite test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Add ov:Tensor API test
* [OV 2.0 C API] fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* add some infer request test
* add compiled model test
* [OV 2.0 C API] Finished preprocess related function unite test
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Fix review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Modify to use default model
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* transfer device_name from fix value to parameter
* add some infer request test
* remove compiled model get_property test
* add infer request tests
* Add ov::model Test and modify Tensor Test name
* Determine whether partial shape meets the standard
* Add get tensor name function and Modify reshape test case
* modify fixed tensor name,remove unnecessary comparison
* add ov_model_get_nodes_info, modify according to comments
* Update reshape test
* extract common function, modify interface about get tensor name,shape and type
* modify according comments
* [OV 2.0 C API] Finished hello classification with ov 2.0 c-api development
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] Fixed hello classification with ov 2.0 c-api review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [OV 2.0 C API] delete inactive code hello classification with ov 2.0 c-api
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix clang format issue
* [OV 2.0 C API] rename
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix windows build erre
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Apply qsort for sorting data
Apply qsort for sarting data
Fix issues of "potentially uninitialized local pointer variable"
* Not use deprecated INSTANTIATE_TEST_CASE_P for c api gtest
INSTANTIATE_TEST_CASE_P is deprecated, should use INSTANTIATE_TEST_SUITE_P.
* Fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* [Ov 2.0 C API] Add error info
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix some review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* Fix review issues
Signed-off-by: xuejun <Xuejun.Zhai@intel.com>
* polish error message for ov c api
* Redefined ov_shape_t, ov_partial_shape_t and ov_layout_t. Modified functions and test cases involving these variables
* Added the conversion between char* and partial_shape
* Add partial_shape_to_shape
* prune code
* modify split
* Use regex to split and search pattern
* Modify str_to_char_array delete
* Add the judgment of rank
* Fix compiling error
Fix issue: address of array 'shape.dims' will always evaluate to 'true' if -Wpointer-bool-conversion
Co-authored-by: xuejun <Xuejun.Zhai@intel.com>
Co-authored-by: sunxiaoxia2022 <xiaoxia.sun@intel.com>
Co-authored-by: ruiqi <ruiqi.yang@intel.com>
* gather blocked format
* enable double blocked
* 5d test
* support cross dimension
* Add some disabled test for later use
* Support non-default planar formats
* It has better performance by using reduction kernel instead of pooling kernel in oneDNN for reduction layer.
* Stop using global pooling instead of reduce primitive
* Use oneDNN reduction if its mode is supported by optimized onedNN kernel
* activation pow is supported
* Use clDNN reduce if 3d or redundant reduce, tensor size mismatch
* Updated thirdparty onednn_gpu
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Co-authored-by: Wei Tang <wei1.tang@intel.com>
Co-authored-by: Chen Kurt <kurt.chen@intel.com>
* [GPU] Implement Roll kernel
* [GPU] Add Roll kernel selector
* [GPU] Add Roll primitive
* [GPU] Add Roll helpers
* [GPU] Implement unit tests for the Roll operation
* [GPU] Add Roll operation to GPU plugin
* [GPU] Add single layer tests for the Roll operation
* [GPU] Add changes after review
* [GPU] Improve cldnn unit test
* Dynamic shape memory reuse solution
* Fix Split node to properly work with dyn mem
* Fix race condition for Memory mgrHandle
* Avoid Memory race condition between GetData and SetDataHandle
Add a lock for race condition between ov::intel_cpu::Memory::GetData() and ov::intel_cpu::Memory::SetDataHandle() is not a good solution,
which will impact the inference performance. We found that it is unnecessary get edge DataPtr in inferRequest::SetBlob or GetBlob, which
only need the tensorDesc, so we can only get tensorDesc to replace get dataPtr to avoid this race condition.
* Resolve reviewer's comments
* Avoid performance impact due to frenquent reset MemMngrHandle
If MemMngrHandle already has been assigned an external buffer, it can be reused.
Else it need create a new one.
* multiclass_nms opset9 spec, api, reference, paddle fe mapper, paddle fe unittest.
* multiclass_nms opset9 cpu node impl.
* multiclass_nms opset9 shape infer fix.
* multiclass_nms opset9: add transform ConvertMulticlassNms8ToMulticlassNms9.
* ConvertMulticlassNmsToMulticlassNmsIE: to MulticlassNmsIEInternal
* add test dependency package paddledet==2.1.0
* 1. fix for roisnum overflow. 2. common shape_infer private function.
Signed-off-by: jialipen <cecilia.peng@intel.com>
* 1. use common infer_shape helper. 2. fix roisnum overflow issue. 3. fix for nmsWithEta.
* test suite for opset9 multiclass_nms smoke tests pass, with both static and dynamic shapes.
code clean for unit test.
* decouple specification from this PR.
* op fuzzy: dynamic input/output
* reference impl refactor
* multiclass_nms_base no need clone_inputs.
* code clean
* restrict ppdet import
* fix clang format error
* change ppdet import to resolve CI fail issue related to its dependency.
* fix CI
* refactor: multiclass_nms_shape_inference for opset9 and reference impl.
TODO: could be applied to opset8 and even matrix_nms.
* fix CI build failure.
* CI fix for ambiguous namespace reference issue when
building static libs.
* update nms save_model python scripts.
* dynamic inputs for NMS with CPU plugin.
* copyright header for test scripts.
* op comformance test for multiclass_nms_9.
* minor update: is_type
* python opset9 and multiclass_nms
* flake8 CI fix
flake8 CI fix
flake8 CI fix
* remove NmsBase. stage1.
flake8 CI fix
remove NmsBase. stage 1 fix.
* rm NmsBase. stage2.
* more multiclass_nms prop tests and fix.
* remove unchanged ops from binding opset9.
* dependcy of paddle_tests.
* fix: add MulticlassNms to op mapper.
* clang format fix
* fix merge error.
* add formats for 3d conv
data formats
-bs_fs_zyx_bsv32_fsv32
-bs_fs_zyx_bsv32_fsv16
-bs_fs_zyx_bsv8_fsv4
-bs_fs_zyx_bsv8_fsv2
-bs_fs_zyx_bsv16_fsv32
-b_fs_zyx_fsv2, b_fs_zyx_fsv4
weight formats
-os_is_zyx_osa2_isa8_osv8_isv2
-os_is_zyx_osv8_isv4
-os_is_zyx_osv8_isv2
-gs_oizyx_gsv32
* add supported formats for primitives
* choose onednn convolution impl for 3d conv
* optimize layout of shallow depth convolution
* remove reorder for conv
* Don't remove reorder between bs_fs_zyx_b32_f16/f32 and bfyx.
* add formats to SetDefault() to optimize gws/lws for quantize/eltwise
* fallback cldnn if onednn pooling's layout is b_fs_zyx_fsv32 and i8.
* fixed wrong position for new weight formats
* restore imad_case()
* This func is used to choose format for fallbacked cldnn
* [GPU] add debug flag: OV_GPU_SerialCompile
0(default): parallel compile
1: serial compile
* add is_mixed_layout
* remove format::bs_fs_zyx_bsv8_fsv4 in needs_onednn_small_ic_to_blocked
* prevent to fuse the reorder which is between quantize and conv
* shallow feature first conv
* Revert "[MO args][ONNX FE]fix cutting graph with input, output or both (#9698)"
This reverts commit 2b03d5fe66.
* Fix cutting the graph when inputs/outputs are passed to the MO
* Check that port exists
* Simplification of getting node port
* Reducing amount of nesting inside searching of node by operation name
* Refactoring
- remove mutable default arg
- changes in code style
- change variables name
* Check that user input data type is dictionary
Co-authored-by: Michal Lukaszewski <michal.lukaszewski@intel.com>
* [GPU] Modify Softmax single layer tests to check Softmax-8 is supported with axes in [-rank, rank) interval
* [GPU] Fix cldnn::softmax::dimension_t documentation
* [GPU] Fix ParamsKey::EnableSoftmaxDim
Support Z dimension.
* [GPU] Add Softmax single layer test that checks 5D case
Since some Softmax kernel code contains ifdef on 5-dimensional case,
a test case is needed that covers this functionality.
* [GPU] Support axis 0 in Softmax
* [GPU] Modify Softmax single layer tests to check axis 0
* [GPU] Modify Softmax items class optimized kernel to handle axis 0 correctly
Modify single layer test accordingly.
* [GPU] Modify Softmax unit-test to check softmax::normalize_b
* Split SoftMaxLayerTest into opset1 and opset8 versions
Use SoftMax8LayerTest in the tests throughout repository.
SoftMaxLayerTest now defaults to SoftMax1LayerTest for compatibility.
* [GPU] Add f16 test-case for Softmax single-layer test
Co-authored-by: tgubanova-lohika <tgubanova@lohika.com>
* dft with single layer test
* idft with single layer test
* fix output param usage in dft
* update dft according to the clang-format
* move output layout setup to calc_output_layout
* add support for other dimensions
* add clDNN unit test for DFT/IDFT
* remove unnecessary original rank
* use defined formats in kernel
* fix dft docs
* changes after review
* Revert "fix dft docs"
This reverts commit 45b05172dfd161d92dae6d26e0f1b74748e56fd5.
Co-authored-by: Serhii Pavlovskyi <spavlovskyi@lohika.com>
Co-authored-by: Mykhailo Hnap <mhnap@lohika.com>
With new networkx release (2.8.1) some of MO tests started to fail
with following error:
```
def __setstate__(self, state):
self._graph = G = state["_graph"]
self._adjdict = G._pred if hasattr(G, "pred") else G._adj
AttributeError: 'Graph' object has no attribute '_adj'
```
Seems like regression that was introduced in
f50fc70b8c
convolution_gpu_yxfb_yxio_b16 for fp16 has hardcoded reqd_work_group_size
to (16, 1, 1). On devices where CL_DEVICE_MAX_WORK_GROUP_SIZE is 512
GetOptimalLocalWorkGroupSizes picks (16, 2, 1) for LWS.
That causes issues during clEnqueueNDRangeKernel since LWS doesn't match
with reqd_work_group_size in the kernel.
* Add single layer tests for GPU
* Add GPU primitive for ExperimentalDetectronGenerateProposalsSingleImage
* Add kernel for ExperimentalDetectronGenerateProposalsSingleImage
* Add unit test
* rename abbreviation edgpsi to the full name experimental_detectron_generate_proposal_single_image
* Add f16 support to operation
* Add f16 support to the unit test
* Add notification about the second output in primitive
Co-authored-by: Oleksii Khovan <okhovan@lohika.com>
* Added shell for Eye-9
* Updated spec for Eye-9
* Added reference for Eye-9
* eye cpu
* Added op impl check for Eye-9
* Fix unallowed dynamic to static dim conversion in eye shape_infer
* Add template plugin tests for dynamic shapes
* Add template plugin tests for dynamic shapes batch input
* Enable batch shape input dynamic rank
* Uncomment 3D batch cpu Eye tests
* Update assertions and messages
* use ov::element type
* Remove redundant evaluate from eval map
* Style fix
* Add static_cast<T>(1) to cpu eye
* Add defaults to eye cpu class members
* Reuse out_ptr and checks
* Reutrn if onesPerBatchNum == 0
* Add Eye CPU Dynamic shape tests with 2D batch
* Additional test cases for CPU and reference
* Disable 3D batch eye cpu tests
* Fix CPU implementation for matrix with not equal cols and rows
* Update CPU test name
* Disable CPU Eye 3D batch static shapes tests
Co-authored-by: Alexandra Sidorova <alexandra.sidorova@intel.com>
Co-authored-by: Yury Gaydaychuk <yury.gaydaychuk@intel.com>
* Update oneDNN rls-v2.6
* Support weight tag for oneDNN v2.6
* Fix first conv selection issue in oneDNN
* oneDNN v2.6 required specific tags to run jit:ir primitives.
* any_tag can find optimized primitives in oneDNN.
* Enable aBcd2b src tag for oneDNN v2.6
* Add create_memory_desc from format string.
* Apply group depthwise separable conv uses jit:ir in oneDNN v2.6
* Use byxf format.
* Update only use acdb format in shallow group conv
* Fix refconv selection in shallow conv with post operations.
* Enable reshape int8
* Fixed quantize fusing through reorder+reshape : Fixed the condition to check per_tensor_input_shift only when need_input_shift is true
* minor change
* Allow FP quant to be fused to FC/gemm
* Disable reshape tranform for onednn until onednn FC is optimized
* [GPU] Support implicit crop in input transposition.
+ Make the crop in front of quantize implicit by changing output format to bfyx.
+ Use implicit concat after quantize nodes.
* Add unit test for implicit crop and concat.
+ remove unnecessary code.
+ Modified jitter Load for planar input of fused eltwise
+ Bugfix in jitter if planar input has LT_ALIGNED_READ
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Update the branch to be used for 2022.1 and remove reference to
-staticdev package which isn't generated anymore.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* roi_align_9: ov_core, transformations, template_plugin
* roi_align_9: CPU Plugin
* keep only constructor with enums which is aligned with spec
* remove evaluate function for ROIAlign_9
* Add op check test for operation ROIAlign-9
* Apply suggestions from code review
* fix version name from 'v0' to 'v3' in transform part
* use common shape_infer function for v3 and v9
* remove'tf_' prefix for ROIAlign::AlignedMode to avoid misleading for models from different platforms
* Update Convert_Model_From_TensorFlow.md (#11425)
* Apply suggestions by Yuan
The changes are made in the port PR, so will be published with the 22.2 version.
Co-authored-by: Evan <evan.juras@gmail.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* Docs: Add links to specific examples (#11618)
* Update docs/OV_Runtime_UG/integrate_with_your_application.md
* Add links to specific examples
This edit adds links to more example applications, making it easier for users to discover how to build an OpenVINO application around their specific model.
* Add links to MO installation and ONNX examples (#11617)
These edits help make it easier for a new user to find more information on how to convert ONNX models.
* Apply suggestions by Yuan
The changes are made in the port PR, so will be published with the 22.2 version.
Co-authored-by: Evan <evan.juras@gmail.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
* selectdevice returns MULTI:device in cumulative_throughput
* load multi with throughput and disable cpu helper in cumulative
* disable cpu helper in cumulative_throughput
* add cumulative to bechmark_app help message
* modify benchmark_app.hpp clang-format
- Add TC for decrease_label_id=true to cover MXNet-style NMS models
- Fix segfault issue that occurs when data precision is fp16
Signed-off-by: Andrew Kwangwoong Park <andrew.kwangwoong.park@intel.com>
Signed-off-by: Andrew Park <andrew.park@intel.com>
* Einsum test helper
* Einsum single layer tests
* Add Einsum decomposition with repeated labels and ellipsis support
to GPU transformations pipeline
Co-authored-by: Oleksii Khovan <okhovan@lohika.com>
Check first whether the path specified by --input_dirs is a directory.
Otherwise the argument is always treated as a .lst file,
and in case it is a directory it silently fails,
which causes the test runner to not execute any tests intended.
porting from 22.1 as per Andrey's request from 04.08
* sphinx google search
* fixes
* fixes
* fix version tabs
Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
* DOCS-benchmarktool_python_correction
add info on tool installation
* Update docs/OV_Runtime_UG/Samples_Overview.md
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
* Try to improve gflags
* Try to improve gflags: part 2
* Tried to use dependencies on system
* Use nlohmann_jsonConfig from system
* Enabled nlohmann_json from system
* Improvements
* handle system gflags in developer package
* Simplifications
* Simplify dependency management
* Corrected package names
* Fixed subgraphsDumper configure stage
* Try to fix rhel8
* Try to fix macosx
* Fixed VPUX build
* Fixed aliasing issues
* Suppress some wanrings
* export gflags when build it
* Fixed some LTO
* Try to fix Mac
* revert
* use gflags as private dependency
* Aligned targets in developer package
* Fixed frontends tests build on U20 with LTO
* PAssed
* Don't use pkg_search_module(zlib ..) during cross-compilation
* Removed unused variables
* Fixed finding of zlib during cross-compilation
* CVS-83529
* Use nothreads_static
* Fixed python
* Moving PWL to ngraph
* improving the running time of php_search; refactoring the pwl operation
* fixed erros & refactored code
* moved PWL op to GNA
* Update src/plugins/intel_gna/ops/pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/ops/reference/pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/ops/pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/transformations/transpose_to_pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/transformations/transpose_to_pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* fixed compilation error
* Update inference-engine/tests/unit/gna/ngraph/transformations/gna_pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* added some tests; changed algorithm of checking accuracy of pwl; refactoring
* added first and last segments; added fq and fixed errors
* fixed after review & rewrote some tests on ngraph
* removed debug logs & fixed code style check error
* s/ngraph_helper/ngraph_util
* removed TRANSFORMATIONS_API in PWLApproximation class declaration
* removed OPENVINO_API in Pwl class declaration
* replaced the deprecated version of evaluate() with a new one
* fixed some problems after reviewing
* fixed a problem when a value of function of left point of segment is less than minimum of function
* corrected a value of the right point of last segments
* [GNA] Moved pwl func tests
* Deleted deprecated test
* s/OPENVINO_RTTI/OPENVINO_OP
* Deleted conflicted test file
* fixed after review
Co-authored-by: Dmitrii Khurtin <dmitrii.khurtin@intel.com>
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* [IE Samples] Activating new parameter is compact mode(memory_reuse) in speech sample
* changed format
* renamed the option to memory_reuse
* renamed the option
* DynamicShapeResolver is able to save information about dynamic output in order to pass it in INFER_DYNAMIC_SHAPE mode. Previously, it propagated fully dynamic output shape (however ranks were equal) and dynamic Convolutions and Poolings were performed incorrectly. Now in the case of dynamic batch, DSR propagates only dynamic batch and Convolutions and Poolings are performed properly as a Loop of single-batch operations.
* Fixed dynamicToStaticShapeTranspose transformation. There was a bug: transposition indices could not be applied with Scatter because the formula is not applicable for this. Replaced with Gather.
i.e. Shape of output tensor of Transpose with transition [0,3,1,2] indices (NHWC [1, 224, 224, 3]->NCHW [1, 3, 224, 224]) was calculated by ScatterElementsUpdate. So output_shape[transposition[i]] = input_shape[i] and the result was output_shape=[1, 224, 3, 224] which was wrong. Vise-versa Gather does output_shape[i] = input_shape[transposition[i]] and the result is [1, 3, 224, 224] which is right.
* MaxPool and AvgPool can be sliced for loop in case of dynamic batch
* Convert stage for inputs is not inserted in the VPU model in the case of OV API 2.0. It did not cause a problem with non-dynamic functions because Graph Transformer has a pass to eliminate redundant converts (u8->f16, ~f16->f16~). In the case of dynamic inputs, yet another inserted Convert breaks data<->shape relations.
* Try to improve gflags
* Try to improve gflags: part 2
* Tried to use dependencies on system
* Use nlohmann_jsonConfig from system
* Enabled nlohmann_json from system
* Improvements
* handle system gflags in developer package
* Simplifications
* Simplify dependency management
* Corrected package names
* Fixed subgraphsDumper configure stage
* Try to fix rhel8
* Try to fix macosx
* Fixed VPUX build
* Fixed aliasing issues
* Suppress some wanrings
* export gflags when build it
* Fixed some LTO
* Try to fix Mac
* revert
* use gflags as private dependency
* Aligned targets in developer package
* Fixed frontends tests build on U20 with LTO
* PAssed
* Don't use pkg_search_module(zlib ..) during cross-compilation
* Removed unused variables
* Fixed finding of zlib during cross-compilation
* added recursive run for transformation to fix fp16 IR with Interpolate inside If
* added test for interpolate inside If
* remove useless variable
* fixed transformaion for divide
* fix code style
* commit auto change
* review fix
* add test for recursive call of divide marks
* removed empty line
* [MO] Support TensorFlow Grouped Conv2DBackpropInput
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Correct computation of group number for ConvBackpropInput operation
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix get_conv_backprop_groups function
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Add unit-tests for Deconvolution shape inference
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
Compilation with ENABLE_CPU_DEBUG_CAPS was fixed.
Previous to this change it failed due to undefined dnnl::impl::md2dim_str
(since DNNL_VERBOSE was disabled in the scope of PR #11244).
* Removed a redundant image
* Fixed ops specifications and other issues
* converted html links to anchor links
* converted html links to anchor links
* Fixed a link
* Fixed a link
* Changed anchor links according to dev review
# Conflicts:
# docs/OV_Runtime_UG/Operations_specifications.md
* Right fill in the values of the inputs
* Using create_and_fill_tensor_unique_sequence() instead of create_and_fill_tensor()
* Fixing a problem with a missing parameter when calling the create_and_fill_tensor method
* Fix Bucketize Conformance tests inputs generation for Template plugin
* Correct filling of the first port (data)
* Correct the order of passing arguments to the InputGenerateData constructor
* Full range correction for random numbers
* Refactoring the argument sequence of the InputGenerateData class constructor
* A small imperfection
* Rollback changes that are related to range
PR for 22.1 made, now porting to release...
some discrepancy between this version and the 22.1 branch seems to exist, so I adjusted the conflicting link to avoid build check errors...
the overview has been merged, the remaining articles are reviewed here
* Paddle FasterRCNN Ops Conversion: roi_align, strided_slice, where
* add check for 'aligned' feature of 'roi_align' op; use common function for idx_node in 'striede_slice' op
* Apply suggestions from code review
* use common funciton for stride_slice and slice, OP_CHECK for 'where' op conversion
* Apply suggestions from code review
* Fix batchability check of MAX_BATCH_SIZE
* Applied review comment
* clonenetwork in auto
Signed-off-by: fishbell <bell.song@intel.com>
* clone in correct way
Signed-off-by: fishbell <bell.song@intel.com>
Co-authored-by: Taylor Yeonbok Lee <taylor.lee@intel.com>
* Frontend exception safety
Every call to frontend's API (except Places) can throw exception. If during exception handling, FrontEndManager is destroyed and calls 'dlclose' for plugin - call stack will be corrupted and crash will occur.
Solution is to wrap 'plugins' calls with try/catch and throw new exception in 'openvino' context
TODO: currently "Place" objects don't have 'actual' wrappers, so exception in 'place' objects will potentially cause such crash (if exception handler destroys FrontEndManager). Workaround for user would be to try/catch any calls of Place API on their side.
We're not expecting users to use Place API directly, so this workaround looks acceptable
* Add check for exception message
* Keep type of frontend exception during rethrow
* IR FE tests: don't expect InferenceEngine::exception as it be not propagated as is by FrontEndManager
* [Python API] Remove old api class from the new api
* start working on refactoring of OVAny
* fix tests
* fix code-style
* remove tuple test
* fix test
* fix omz hash
* one more overload
* fix pyfloat
* move from_ov_any to utils
* code-style
* move function from common to utils
* Build with system TBB
* Fixes
* Check whether system TBB is available
* Try to fix ONNX Runtime build with system TBB
* Test
* Fixed compilation of threading.cpp
* Fixed unset of cache dirs
* Limit dearch paths of TBB
* Try to enable pip packages with custom TBB
* Fix for TBB 2021.2
* Install only needed TBB libraries
* Install TBB from system to pip package
* Reverted usage of TBBROOT
* Fixed oneTBB case
* Try to fix Android
* Escape some paths
* Added samples path
* Fixed TBBBind usage for case of system TBB
* Added specification for EyeLike-9
* Update docs/ops/generation/EyeLike_9.md
* removed batch from TF
* minor fix
* Applied comment by Anton
* Added new example with dynamic output, added corner case
* Fixed corner case description
* Rename matrix
* applied comments by Yuan
* Added diag_idx as input, minor fixes, renaming
* added support of batch_shape from TF
Co-authored-by: Andrei Kochin <andrei.kochin@intel.com>
* [GNA] Fuse all FakeQuantize layers with their previous layers
* [GNA] Fuse FQ with previous layer if it's not required for precision change
* [GNA] Fixed MatMulOverloadCorrectionTest
* New command line parameters format for speech sample
* fixed notes
* changed format for scale factor
* changed format for scale factor in tests
* added more variants, when name is directy specified for i/o/r like it is done for sf
* removed nthreads flag
* fixed notes
* changed output params
* updated tests with new format
Co-authored-by: Alexander Zhogov <alexander.zhogov@intel.com>
* Fix for str_to_container if string value has whitespaces
* Add test
* Add trim for leading and trailing whitespaces
* Apply comments
* Apply comments 2
* Apply comments 3
* Enable explicit TBlob declaration in all compilers
This fixes problems when linking gcc compiled IE with clang compiled
applications.
Previous to this change, only clang compilers would consider TBlob<T>
templated types as declared externally. When *declared* explictly (with
the `extern template` syntax), the C++ spec says
that any inline methods of the templated class (such as TBlob<T>
constructors) should be ignored in favor of the externally instantiated
version of that templated type:
"An explicit instantiation declaration (an extern template) skips
implicit instantiation step: the code that would otherwise cause an
implicit instantiation instead uses the explicit instantiation
definition provided elsewhere (resulting in link errors if no such
instantiation exists)."
However, when IE is compiled with gcc, it does not see the explicit
`extern template` declarations of TBlob<T> (due to the `#ifdef
__clang__` guards in `ie_blob.h`). As an end result, presumably due to
link-time-optimizations during IE library compilation(?), none of the
TBlob<T> implementations are actually included in the IE dynamic
libraries.
* Fix warnings for windows
* Fix typo
* revert previous version of convert_seq_to_ti transformation
* try to check that outputs of TI are connected to Result nodes
* add unit tests
* fix codestyle
* fix Memory tests
* revert local change
* revert local change
* replace duplicated code with lambda
* Written nGraph reference for the operation RDFT.
* Used std::reverse() algorithm to simplify the function reverse_shape() from fft_common.cpp.
* Added assert into the function offset_from_coords_and_strides().
* Deleted redundant variable.
* Deleted redundant functions from the reference implementation of (I)DFT.
* Renamed the method reverse_shape() in fft_common.hpp.
* Code style fix.
* Paddle FasterRCNN Ops Conversion: greater_than, less_than, gather, floor
* Apply suggestions from code review
* fix 'gather' testcase failure issue on CI
* implement 'axis' input for 'Gather' Op conversion with testcase comment;use common function for all elementwise Ops
* Fix setupvars.bat patching
setupvars.bat shoudl not be patched for regular Debug and Release
configurations.
* Use SRTEQUAL for cmake string comparison
* Improve performance for 'ov::Model::add_output'
On first call of `add_output(tensor_name)` all available tensor names are cached.
Next calls take nodes from cache which significantly reduces complexity.
Cache is invalidated if topological cache is not valid or cache points to incorrect output (no tensor name of this node anymore)
The same caching is done for 'add_output(op_name, output_index)'
Tests:
- Verifies that adding outputs to all nodes has linear complexity O(N), not O(N^2)
- Verifies cache invalidation scenarios
* Fix python tests
* Update topological cache after add_output(Output<Node>) by adding result to the end of cached ops
* Add 'm_shared_rt_info' to 'result node just for consistency (there is actually no scenario which may fail due to absence of this info for Result
* Added test cases to verify that names cache should be cleared on refresh of 'get_ordered_ops'
* wip remote tests2, fixed smoke_canInferOnUserContext
* completed the OV 1.0 tests for remote blobs
* updated OV 2.0 tests for remote blobs with auto-batching (using the ngraph func that is reshape-able by the batch)
* re-using the DetectionOutput-based ngraph func that is 100% batch-reshapeble
* Add test case for the loadNetwork with Auto Batching.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Enable logic test case for GPU.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Enable property for config key 'AUTO_BATCH_DEVICE_CONFIG'.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Omit {}.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add commont test for the property ALLOW_AUTO_BATCHING.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add commont test for AUTO Batching plugin.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Moving PWL to ngraph
* improving the running time of php_search; refactoring the pwl operation
* fixed erros & refactored code
* moved PWL op to GNA
* Update src/plugins/intel_gna/ops/pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/ops/reference/pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/ops/pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/transformations/transpose_to_pwl.hpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* Update src/plugins/intel_gna/transformations/transpose_to_pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* fixed compilation error
* Update inference-engine/tests/unit/gna/ngraph/transformations/gna_pwl.cpp
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
* added some tests; changed algorithm of checking accuracy of pwl; refactoring
* added first and last segments; added fq and fixed errors
* fixed after review & rewrote some tests on ngraph
* removed debug logs & fixed code style check error
* s/ngraph_helper/ngraph_util
* removed TRANSFORMATIONS_API in PWLApproximation class declaration
* removed OPENVINO_API in Pwl class declaration
* replaced the deprecated version of evaluate() with a new one
* fixed some problems after reviewing
* fixed a problem when a value of function of left point of segment is less than minimum of function
* corrected a value of the right point of last segments
* s/OPENVINO_RTTI/OPENVINO_OP
Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
+ Fix colorization-sig accuracy issue using oneDNN
Memory crash in case reuse_eltwise_sum_post in oneDNN and memory_pool
And print node in/out gpu_usm_mem addr at OV_GPU_Verbose >= 1
+ Check the size of z spatial axis for checking fulltensor.
+ Remove program_helpers's functions.
Co-authored-by: hyunback <hyunback.kim@intel.com>
Scenario:
- Node "Split" with multiple outputs (e.g. 3). All outputs are connected to "Result"s
- Add post-processing step (e.g. convert element type, can be also implicit)
Issue: after post-processing, 3 new results will be created, each will have "Split" friendly name - inconsistency with IRv10 rules
Fix:
- For nodes with multiple outputs, add '.<idx>' suffix to new output's friendly name
- If no post-processing is applied, return immediately, keeping original results as is
Tests:
- Split with 3 outputs where 2 outputs have post-processing.
- Split with 3 outputs, post-processing doesn't create any nodes
* [XLink] - tests to smoke scope
* [XLink] - small change in XLink related file to trigger ie-tests-windows-myriadx
* [XLink] - azure windows and linux
* [XLink] - azure windows and linux
* [XLink] - azure windows and linux - change dir?
* [XLink] - azure windows and linux - change dir?
* [XLink] - azure windows and linux - install?
* [XLink] - azure windows and linux - xlink cmake
* [XLink] - azure windows and linux - XLinkTests because another target with the same name already exists
* [XLink] - azure windows and linux - XLinkTests because another target with the same name already exists
* [XLink] - azure windows and linux - install TARGETS given target XLinkTests which does not exist
* [XLink] - azure windows and linux - remove smoke
Inserting padding into oneDNN primitive has issue with implicit concat behavior.
Deconv onedNN initialized output buffer to 0 including padding area. Padding area should be reserved.
Use oneDNN offset from program_node in/out lower_padding instead of oneDNN memory desc.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* add 3D shape to test and rename crop4d to strided_slice
* remove ConvertStridedSliceToCropNegative2 since 3D is now supported
* add myriad functional tests to skip-list
* update Auto docs
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* update python snippets
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* remove vpu, fix a mistaken in python code
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* update MYRIAD device full name
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* update API name
old API use name Inference Engine API
NEW API usen name OpenVINO Runtime API 2.0
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* update tab name, and code format
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix AUTO4 format issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* update set_property code
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* auto draft
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* mv code into .cpp and .py
modify the devicelist part accoding to the review
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* remove priority list in code and document
modify the begning of the document
remove perfomance data
remove old API
use compile_model instead of set_property
add a image about cpu accelerate
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix mis print and code is not match document
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* try to fix doc build issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* fix snippets code compile issue
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* Added migration for deployment (#10800)
* Added migration for deployment
* Addressed comments
* more info after the What's new Sessions' questions (#10803)
* more info after the What's new Sessions' questions
* generalizing the optimal_batch_size vs explicit value message
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Update docs/OV_Runtime_UG/automatic_batching.md
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Perf Hints docs and General Opt Guide refactoring (#10815)
* Brushed the general optimization page
* Opt GUIDE, WIP
* perf hints doc placeholder
* WIP
* WIP2
* WIP 3
* added streams and few other details
* fixed titles, misprints etc
* Perf hints
* movin the runtime optimizations intro
* fixed link
* Apply suggestions from code review
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* some details on the FIL and other means when pure inference time is not the only factor
* shuffled according to general->use-case->device-specifics flow, minor brushing
* next iter
* section on optimizing for tput and latency
* couple of links to the features support matrix
* Links, brushing, dedicated subsections for Latency/FIL/Tput
* had to make the link less specific (otherwise docs compilations fails)
* removing the Temp/Should be moved to the Opt Guide
* shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins
- openvino_docs_IE_DG_Model_caching_overview
- openvino_docs_IE_DG_Int8Inference
- openvino_docs_IE_DG_Bfloat16Inference
- openvino_docs_OV_UG_NoDynamicShapes
* fixed toc for ov_dynamic_shapes.md
* referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors
* fixed main product TOC, removed ref from the second-level items
* reviewers remarks
* reverted the openvino_docs_OV_UG_NoDynamicShapes
* reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference
* "No dynamic shapes" to the "Dynamic shapes" as TOC
* removed duplication
* minor brushing
* Caching to the next level in TOC
* brushing
* more on the perf counters ( for latency and dynamic cases)
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
* Updated common IE pipeline infer-request section (#10844)
* Updated common IE pipeline infer-reqest section
* Update ov_infer_request.md
* Apply suggestions from code review
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
* DOCS: Removed useless 4 spaces in snippets (#10870)
* Updated snippets
* Added link to encryption
* [DOCS] ARM CPU plugin docs (#10885)
* initial commit
ARM_CPU.md added
ARM CPU is added to the list of supported devices
* Update the list of supported properties
* Update Device_Plugins.md
* Update CODEOWNERS
* Removed quotes in limitations section
* NVIDIA and Android are added to the list of supported devices
* Added See Also section and reg sign to arm
* Added Preprocessing acceleration section
* Update the list of supported layers
* updated list of supported layers
* fix typos
* Added support disclaimer
* update trade and reg symbols
* fixed typos
* fix typos
* reg fix
* add reg symbol back
Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com>
* Try to fix visualization (#10896)
* Try to fix visualization
* New try
* Update Install&Deployment for migration guide to 22/1 (#10933)
* updates
* update
* Getting started improvements (#10948)
* Onnx updates (#10962)
* onnx changes
* onnx updates
* onnx updates
* fix broken anchors api reference (#10976)
* add ote repo (#10979)
* DOCS: Increase content width (#10995)
* fixes
* fix
* Fixed compilation
Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com>
Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Yuan Xu <yuan1.xu@intel.com>
Co-authored-by: Victoria Yashina <victoria.yashina@intel.com>
Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
* CPU device documentation refresh
* Bfloat16 inference page aligned with the new API
* Bfloat16 inference section moved to CPU main
* First review comments applied
* Second review step comments applied
* OneDNN reference changed to the GitHub page
* AvgPool added to the oneDNN ops list
* Add readvalue, assign to templte plugin test
* Fix clang error
* Fix clang error
* Remove unnecessary comment
* Fix type-casting error
* Fix ci issue regarding const value
* Change Function to Model
* Fix op scope
* Change way to get variable
* Fix type-casting error
* Set variable id to const
* Fix side-effect in ieFuncTests
* Implement Assign-3, ReadValue-3 in evaluates_map
* Correct setting attribute
* Correct setting attribute
* Remove unnecessarily added method
* Roll back v6
* Use member variable for variable_id in assign-3, read_value-3
* Get data pointer from host tensor
* Remove visitor API test for ReadValue-6, Assign-6
* Implement visitor api test for read_value-6, assign-6
* Fix clang error
* Split read_value and assign into each file for visitor test
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
This behavior is already used by default because ONNX is enabled by default and thirdparty/onnx/onnx/CMakeLists.txt forcing CMAKE_BUILD_TYPE to Release if it is not set
It fixes the following issues:
- When ONNX frontend is disabled - source is built for Debug, which is very unexpected comparing to Release with ONNX frontend enabled
- When ONNX frontend is disabled, even libopenvino.so could not be built due to some generated makefiles issues
It is set to 'Release' (not to 'Debug') to comply with default behavior when ONNX is enabled (it is default option working for most users)
* Performance improvement for constant creation
The issue is that 'are_all_data_elements_bitwise_identical()' is called every time in Constant constructor, and it potentially checks all buffer which is O(N) complexity.
While it is needed only if client uses 'get_all_data_elements_bitwise_identical'
Solution:
- Defer calculation until first call of 'get_all_data_elements_bitwise_identical'
- Store calculated value in mutable class member to reuse it on next calls of 'get_all_data_elements_bitwise_identical'
Test verifies both cases:
a) that constant creation with shared memory data (now O(1)) is significantly faster than creation+bitwiseCheck O(N)
b) Than once calculated, value is taken from cache, which is significantly faster than re-calculation
* fix clang-format
* Stash - Linux implementation
* Windows mmap implementation + unicode
* Clang for windows
* removed debug print
* Add handling of empty bin file
* fix windows includes
* Fix python test
* Unit tests
Fix for Constant with size > 4GB
* Fix review comments
* refactoring: get bias shape in bc and fbc algoritms
* use scipy to take most frequent shape
* pylint
* update reference
* pylint
* Update test_sanity.py
* update test_sanity.py
* Update test_sanity.py
* [GNA] Added SW_FP32 mode w/o SF for BasicLSTM
* deleted additional test
added sw_fp32 mode for exisiting test
changed reference output for new mode
* [GNA] Fixed according to review
* [GNA] Parametrized weights range
* fixed after review
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* Written header files for the nGraph operations RDFT and IRDFT.
* Written nGraph shell for the operation RDFT.
* Added missed include.
* Added RDFT to opset9 table.
* Code style fixes.
* Written the nGraph shell of the operation IRDFT.
* Added IRDFT to opset9 table.
* Started to write shape infer tests for RDFT.
* Refactoring: shape infer functions of RDFT and IRDFT moved into separate files.
* Written shape infer tests for RDFT.
* Written shape infer tests for IRDFT operation.
* Fixed code style.
* Fixes in the shape infer function of RDFT.
* Fixes in the shape infer function of RDFT.
* Fixes in the shape infer function of IRDFT.
* Deleted redundant includes in include/ngraph/op/irdft.hpp and include/ngraph/op/rdft.hpp
* Deleted redundant includes in include/openvino/op/rdft.hpp and include/openvino/op/irdft.hpp.
* Deleted redundant includes in cpp-files of nGraph shells of operations IRDFT and RDFT.
* Code style fixes.
* Shape inference functions of operations RDFT and IRDFT moved to the namespace ov::op::util.
* Deleted RDFT and IRDFT from docs/template_plugin/backend/opset_int_tbl.hpp.
* Deleted 'using namespace ngraph' from cpp-files of nGraph shells of operations RDFT and IRDFT.
* Fixed typos.
* Merged some loops in shape inference functions of RDFT and IRDFT.
* Written visitor tests for RDFT and IRDFT.
* Small change.
* Common part of RDFT and IRDFT shape validation moved into the separate file.
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* don't check dynamic shape when there is only one device
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* remove redundant if
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* mod docs/_static/images/dataset.png and docs/_static/images/inputs.png
* add new hint cumulative_throughput
* clang format properties.hpp
* add set properties and get properties test case for CUMULATIVE_THROUGHPUT
* reset docs/_static/images/dataset.png and docs/_static/images/inputs.png
* reset docs/_static/images/dataset.png and docs/_static/images/inputs.png
* reset dataset.png and inputs.png
* reset dataset.png and inputs.png
* remove test value cumulative_throughput from gpuplugin and cpuplugin testcase
* rollback dataset.png and inputs.png to 41818a377
* add fps log
add format '%lf' for log
add INFO_RUN and DEBUG_RUN, code only run when greater than special log level
add fps log for device
print device config info with DEBUG_RUN
add mock test for DEBUG_RUN and INFO_RUN
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* use n / end -start instead of (n-1) / ((nst start) -(1st start))
Signed-off-by: Hu, Yuan2 <yuan2.hu@intel.com>
* Mark `get_type_info_static()` as hidden
Each plugin linked with openvino library contains `type_info_static` symbols. In case when one of the libraries is unloaded and app tries to get opset, it leads to segfault. So mark `get_type_info_static()` as hidden to use only one implementation exactly from openvino lib
* Fix "'visibility' attribute ignored" issue by moving `TestPass` out of test scope
* Fix clang format
* Small update of `If` op
* Revert "fix 79520 (#10449)" to correctly compare DiscreteTypeInfo via `==`
This reverts commit 29883a152a.
The change fixes FQ fusions for subgraphs like 'Const weights'->FQ->Transpose->Multiply.
After PullTransposeThroughFQUp transformation, we end up with following:
'Const weights'->Transpose->FQ->Multiply. Because of the Transpose on first
FakeQuantize inputs, Multiply could not be fused since FakeQuantizeMulFusion
expected that weights is a Constant node.
Ticket: 77785
* Performance improvement for constant creation
The issue is that 'are_all_data_elements_bitwise_identical()' is called every time in Constant constructor, and it potentially checks all buffer which is O(N) complexity.
While it is needed only if client uses 'get_all_data_elements_bitwise_identical'
Solution:
- Defer calculation until first call of 'get_all_data_elements_bitwise_identical'
- Store calculated value in mutable class member to reuse it on next calls of 'get_all_data_elements_bitwise_identical'
Test verifies both cases:
a) that constant creation with shared memory data (now O(1)) is significantly faster than creation+bitwiseCheck O(N)
b) Than once calculated, value is taken from cache, which is significantly faster than re-calculation
* fix clang-format
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
* InputTensorInfo::from implementation
If user's application already has `ov::runtime::Tensor` object created,
it will be possible to reuse basic characteristics for input (shape, precision) from tensor using InputTensorInfo::from method
* Rename 'from' to 'set_from' as in Python 'from' keyword is used for import modules
Python bindings: from ov.Tensor and from numpy array
* Style fix (quotes)
* Apply suggestions from code review
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* Fix code style
* Use set_from in hello_classification CPP sample
Co-authored-by: Ilya Churaev <ilyachur@gmail.com>
* add placeholder for python version of first snippet
* fix problem with placeholder
* fix wrong file name
* fix fragment name
* update python snippets
* move imports to the top of the code fragments
* [GNA] Single lstm-cell test added
* Added additional config for test
* one more input and hidden shape
* Added cell with ReLU
Deleted deprecated test
* test added as lstm_cell_basic
* Enabled gna_compact_mode
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* enabled compact_mode in all tests
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
* fix .ncc_style target names
it was breaking configure on system with libclang-12-dev, clang-12,
ninja and cmake 3.17+(ninja complains about duplicate
target). with lower cmake version configure succeeds, but build exits
immediately with error. by replacing ninja with make error becomes
warning(it's still significant, make just skips duplicate rules, i.e.
doesn't check style of some source files, rule duplication is genuine
bug). without libclang-12-dev and clang-12 ENABLE_NCC_STYLE is OFF and
bug is not triggered
* silence uninitialized warning in core_integration
probably it was always initialized before use, but compiler wasn't made
aware of it
* fix function spelling to unbreak code style checks in benchmark_app
* include <thread> for std::this_thread
existing code was relying on namespace pollution by old libstdc++
* replace is_pod with is_standard_layout && is_trivial
is_pod is deprecated, it breaks build on current gcc
Co-authored-by: Serhii Pavlovskyi <spavlovskyi@lohika.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
When partial build is called for dryrun, do constant propagate too.
In normal case, partial build is not doing constant propate for saving build time of internal program.
However, if partial build is called with dryrun, it will fail at transfer_constants due to the generic nodes which does not have impl.
* Update IEEngine with the Dynamic models support
* Update with the batch
* Method naming fix
* Update image_loader & tests with dynamic models
* Update test_sanity.py
* Replace custom_mo_config from the model
* Modified the workflow diagram
* Moved supported topology lists to separate topics
* Additional changes
* Removed Supported Topologies list and Deprecated pages
* Created the Model Conversion Tutorials section for instructions for specific models
* Topic names alignment, removed Default_Model_Optimizer_Optimizations.md
* Additional structural changes
* Fixed links
* heading fixes
* [MO] Remove IR frontend from available frontend list in MO
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix issue - forget to pass FEM
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix issue for TF with new FE and default legacy
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix result saving when batch size is not 1
* Remove useless if statement
* improved processing scores for model with more than one outputs
* added checking on count of model outputs
* improve if statements
* divide fix for model with several outputs to other PR
Co-authored-by: Maxim Gordeev <maxim.gordeev@intel.com>
* [GPU] update the condition for minimize_local_reorders
* Update to check needs reorder condition in quantize.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Add coommont test of the key PERFORMANCE_HINT for AUTO plugin API 2.0.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Add common test case for config check.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Update.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Use the implemented property test case.
Signed-off-by: Wang, Yang <yang4.wang@intel.com>
* Written the draft of the specification of the operation RFFT.
* Started to write the specification of the operation IRFFT.
* Small fix.
* Renamed RFFT operation as RDFT.
* Fix in Operations_specifications.md.
* Written the specification of the operation IRDFT.
* Fixes in examples.
* Fixes in opset9.md and Operations_specifications.md.
* Small fix.
* Replaced opset8 by opset9 in opset9.md.
* Deleted redundant sentences.
* Small fix.
* Replaced input_shape by data_shape.
* Fixed mistypes.
* Fixes of mistypes.
* Fixed typo.
* Fixed RDFT specification, in order to perform signal_size input as in TF and PyTorch.
* Fixes in examples for RDFT.
* Fixes in the output shape calculation of IRDFT. Now this calculation is as in TF and PyTorch.
We suppose that you are an enthusiastic coder, want to contribute some code. For that purpose OpenVINO project now has a repository on the GitHub, to simplify everybody's life! All the bug fixes, new functionality, new tutorials etc. should be submitted via the GitHub's mechanism of pull requests.
We welcome community contributions to OpenVINO™. Please read the following guide to learn how to find ideas for contribution, practices for good pull requests, checking your changes with our tests and more.
If you are not familiar with the mechanism - do not worry, it's very simple. Keep reading.
## Before you start contributing you should
- Make sure you agree to contribute your code under [OpenVINO (Apache 2.0)](https://github.com/openvinotoolkit/openvino/blob/master/LICENSE) license.
-If you are submitting a new module, you should gointo [openvino_contrib](https://github.com/openvinotoolkit/openvino_contrib) repository by default.
- If you are going to fix a bug, check that it's still exists. This can be done by building the latest [releases/2020/3](https://github.com/openvinotoolkit/openvino/tree/releases/2020/3) branch (LTS release) or the latest master branch, and make sure that the error is still reproducible there. We do not fix bugs that only affect older non-LTS releases like 2020.2 for example (more details about [branching strategy](https://github.com/openvinotoolkit/openvino/wiki/Branches))
- Make sure that nobody beat you into fixing or reporting the issue by doing a search on the [Github OpenVINO issues](https://github.com/openvinotoolkit/openvino/issues) page, and making sure that there isn't someone working on it. In the latter case you might provide support or suggestion in the issue or in the linked pull request.
- If you have a question about the software, then this is **NOT** the right place. You should open up a question at the [OpenVINO forum](https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit). In order to post a decent question from the start, feel free to read the official forum guidelines.
- Make sure you agree to contribute your code under [OpenVINO™ (Apache 2.0)](https://github.com/openvinotoolkit/openvino/blob/master/LICENSE) license.
-Figure out what you’re going to contribute. If you don’t know what you are going to work on, navigate to the [Github "Issues" tab](https://github.com/openvinotoolkit/openvino/issues). Make sure that there isn't someone working on it. In the latter case you might provide support or suggestion in the issue or in the linked pull request.
- If you are going to fix a bug, check that it's still exists in the latest release. This can be done by building the latest master branch, and make sure that the error is still reproducible there. We do not fix bugs that only affect older non-LTS releases like 2020.2 for example (more details about [branching strategy](https://github.com/openvinotoolkit/openvino/wiki/Branches)).
Before you open up anything on the OpenVINO GitHub page, be sure that you are at the right place with your problem.
## "Fork & Pull Request model" for code contribution
### [](https://github.com/openvinotoolkit/openvino/wiki/Contribute#the-instruction-in-brief)The instruction in brief
### [](https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md#the-instruction-in-brief)The instruction in brief
- Register at GitHub. Create your fork of OpenVINO repository [https://github.com/openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino) (see [https://help.github.com/articles/fork-a-repo](https://help.github.com/articles/fork-a-repo) for details).
- Register at GitHub. Create your fork of OpenVINO™ repository [https://github.com/openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino) (see [https://help.github.com/articles/fork-a-repo](https://help.github.com/articles/fork-a-repo) for details).
- Install Git.
- Set your user name and email address in a Git configuration according to GitHub account (see [https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) for details).
- Choose a task for yourself. It could be a bugfix or some new code.
- Choose a base branch for your work. More details about branches and policies are here: [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches)
- Clone your fork to your computer.
- Create a new branch (with a meaningful name) from the base branch you chose.
- Modify / add the code following our [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines) and [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation).
- Modify / add the code following our [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines).
- If you want to add a new sample, please look at this [Guide for contributing to C++/C/Python IE samples](https://github.com/openvinotoolkit/openvino/wiki/SampleContribute)
- If you want to contribute to the documentation and want to add a new guide, follow that instruction [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation)
- Run testsuite locally:
- execute each test binary from the artifacts directory, e.g. `<source dir>/bin/intel64/Release/ieFuncTests`
- If you contribute to the documentation and want to add a new guide:
- Create a new markdown file in an appropriate folder.
-**REQUIRED:** The document title must contain a document label in a form: `{#openvino_docs_<name>}`. For example: `Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™ {#openvino_docs_MO_DG_IR_and_opsets}`.
- Add your file to the documentation structure. Open the documentation structure file [`docs/doxygen/ie_docs.xml`](https://github.com/openvinotoolkit/openvino/blob/master/docs/doxygen/ie_docs.xml) and add your file path to the appropriate section.
- When you are done, make sure that your branch is to date with latest state of the branch you want to contribute to (e.g. `git fetch upstream && git merge upstream/master`), push your branch to your GitHub fork; then create a pull request from your branch to the base branch (see [https://help.github.com/articles/using-pull-requests](https://help.github.com/articles/using-pull-requests) for details).
## Making a good pull request
Following these guidelines will increase the likelihood of your pull request being accepted:
-Before pushing your PR to the repository, make sure that it builds perfectly fine on your local system.
-Add enough information, like a meaningful title, the reason why you made the commit and a link to the issue page if you opened one for this PR.
-Scope your PR to one issue. Before submitting, make sure the diff contains no unrelated changes. If you want to cover more than one issue, submit your changes for each as separate pull requests.
-If you have added new functionality, you should update/create the relevant documentation, as well as add tests for it to the testsuite.
-Try not to include "oops" commits - ones that just fix an error in the previous commit. If you have those, then before submitting [squash](https://github.com/openvinotoolkit/openvino/wiki/Contribute#https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History#Squashing-Commits) those fixes directly into the commits where they belong.
-Make sure to choose the right base branch and to follow the [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines) for your code or [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation) you are changing documentation files.
-Make sure to add test for new functionality or test that reproduces fixed bug with related test data. Please do not add extra images or videos, if some of existing media files are suitable.
-One PR – one issue.
-Build perfectly on your local system.
-Choose the right base branch [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches).
-Follow the [Coding Style Guide](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLines) for your code.
-Update documentation using [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation) if needed.
-Cover your changes with test.
-Add license at the top of new files [C++ example](https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/classification_sample_async/main.cpp#L1-L2), [Python example](https://github.com/openvinotoolkit/openvino/blob/master/samples/python/hello_classification/hello_classification.py#L3-L4).
- Add enough information: a meaningful title, the reason why you made the commit and a link to the issue page if exists.
- Remove unrelated to PR changes.
- If it is still WIP and you want to check CI test results early then use _Draft_ PR.
- Submit your PR and become an OpenVINO™ contributor!
## Testing and merging pull requests
-Your pull request will be automatically tested by OpenVINO's precommit (testing status are automatically reported as "green" or "red" circles in precommit steps on PR's page). If any builders have failed, you should fix the issue. To rerun the automatic builds just push changes to your branch on GitHub. No need to close pull request and open a new one!
- Once all the builders are "green", one of OpenVINO developers will review your code. Reviewer could ask you to modify your pull request. Please provide timely response for reviewers (within weeks, not months), otherwise you submission could be postponed or even rejected.
Your pull request will be automatically tested by OpenVINO™'s precommit (testing status are automatically reported as "green" or "red" circles in precommit steps on PR's page). If any builders have failed, you need fix the issue. To rerun the automatic builds just push changes to your branch on GitHub. No need to close pull request and open a new one!
## PR review good practices
- Originator is responsible for driving the review of changes and should ping reviewers periodically.
- Originator should close comments from the Reviewer when it is resolved. The Reviewer may re-open the comment if he does not agree with the resolution.
- Originator should request re-review from the Reviewer when all comments are resolved by pushing the button in the “Reviewers” section.
- If it is still WIP and you want to check CI test results early then use _Draft_ PR.
- Do **NOT** rewrite history (push -f) once you converted draft PR into regular one, add new commits instead. Looking at diffs makes review easier.
- Write meaningful description of commits resulting from review. _"Addressing review comments"_ is **NOT** a good description! Having a quick look at good descriptions can tell you much what is going on in PR without a need to go through all of resolved comments.
## Merging PR
As soon as the reviewer is fine with the pull request and Precommit likes your code and shows "green" status, the "Approved" review status is put, which signals OpenVINO maintainers that they can merge your pull request.
As soon as the reviewer is fine with the pull request and precommit shows "green" status, the "Approved" review status is put, which signals OpenVINO™ maintainers that they can merge your pull request.
This toolkit allows developers to deploy pre-trained deep learning models
through a high-level OpenVINO™ Runtime C++ and Python APIs integrated with application logic.
## Contents:
This open source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
- [Products which use OpenVINO](#products-which-use-openvino)
- [System requirements](#system-requirements)
- [How to build](#how-to-build)
- [How to contribute](#how-to-contribute)
- [Get a support](#get-a-support)
- [See also](#see-also)
## What is OpenVINO toolkit?
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference.
- Boost deep learning performance in computer vision, automatic speech recognition, natural language processing and other common tasks
- Use models trained with popular frameworks like TensorFlow, PyTorch and more
- Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud
This open-source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from the [Open Model Zoo], along with 100+ open
source and public models in popular formats such as TensorFlow, ONNX, PaddlePaddle, MXNet, Caffe, Kaldi.
## Repository components
* [OpenVINO™ Runtime]
* [Model Optimizer]
* [Post-Training Optimization Tool]
* [Samples]
### Components
* [OpenVINO™ Runtime] - is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions on the platform of your choice.
* [core](https://github.com/openvinotoolkit/openvino/tree/master/src/core) - provides the base API for model representation and modification.
* [inference](https://github.com/openvinotoolkit/openvino/tree/master/src/inference) - provides an API to infer models on device.
* [transformations](https://github.com/openvinotoolkit/openvino/tree/master/src/common/transformations) - contains the set of common transformations which are used in OpenVINO plugins.
* [low precision transformations](https://github.com/openvinotoolkit/openvino/tree/master/src/common/low_precision_transformations) - contains the set of transformations which are used in low precision models
* [bindings](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings) - contains all awailable OpenVINO bindings which are maintained by OpenVINO team.
* [c](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings/c) - provides C API for OpenVINO™ Runtime
* [python](https://github.com/openvinotoolkit/openvino/tree/master/src/bindings/python) - Python API for OpenVINO™ Runtime
* [Plugins](https://github.com/openvinotoolkit/openvino/tree/master/src/plugins) - contains OpenVINO plugins which are maintained in open-source by OpenVINO team. For more information please taje a look to the [list of supported devices](#supported-hardware-matrix).
* [Frontends](https://github.com/openvinotoolkit/openvino/tree/master/src/frontends) - contains available OpenVINO frontends which allow to read model from native framework format.
* [Model Optimizer] - is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
* [Post-Training Optimization Tool] - is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning, for example, post-training 8-bit quantization.
* [Samples] - applications on C, C++ and Python languages which shows basic use cases of OpenVINO usages.
## Supported Hardware matrix
The OpenVINO™ Runtime can infer models on different hardware devices. This section provides the list of supported devices.
<td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td>
* [Intel® Distribution of OpenVINO™ toolkit Product Page](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html)
* [Intel® Distribution of OpenVINO™ toolkit Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
## Documentation
### User documentation
The latest documentation for OpenVINO™ Toolkit is availabe [here](https://docs.openvino.ai/). This documentation contains detailed information about all OpenVINO components and provides all important information which could be needed if you create an application which is based on binary OpenVINO distribution or own OpenVINO version without source code modification.
### Developer documentation
[Developer documentation](#todo-add) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
Please take a look to [OpenVINO Wiki](https://github.com/openvinotoolkit/openvino/wiki#how-to-build) to get more information about OpenVINO build process.
## How to contribute
See [CONTRIBUTING](./CONTRIBUTING.md) for details. Thank you!
## Get a support
## Support
Please report questions, issues and suggestions using:
* The [`openvino`](https://stackoverflow.com/questions/tagged/openvino) tag on StackOverflow\*
* [Intel® Distribution of OpenVINO™ toolkit Product Page](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html)
* [Intel® Distribution of OpenVINO™ toolkit Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
* [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) - a suite of advanced algorithms for model inference optimization including quantization, filter pruning, binarization and sparsity
* [OpenVINO™ Training Extensions (OTE)](https://github.com/openvinotoolkit/training_extensions) - convenient environment to train Deep Learning models and convert them using OpenVINO for optimized inference.
* [OpenVINO™ Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) - a scalable, high-performance solution for serving deep learning models optimized for Intel architectures
* [DL Workbench](https://docs.openvino.ai/nightly/workbench_docs_Workbench_DG_Introduction.html) - An alternative, web-based version of OpenVINO designed to make production of pretrained deep learning models significantly easier.
* [Computer Vision Annotation Tool (CVAT)](https://github.com/openvinotoolkit/cvat) - an online, interactive video and image annotation tool for computer vision purposes.
* [Dataset Management Framework (Datumaro)](https://github.com/openvinotoolkit/datumaro) - a framework and CLI tool to build, transform, and analyze datasets.
---
\* Other names and brands may be claimed as the property of others.
User Guide <workbench_docs_Workbench_DG_User_Guide>
workbench_docs_Workbench_DG_Troubleshooting
@endsphinxdirective
Deep Learning Workbench (DL Workbench) is an official OpenVINO™ graphical interface designed to make the production of pretrained deep learning Computer Vision and Natural Language Processing models significantly easier.
Minimize the inference-to-deployment workflow timing for neural models right in your browser: import a model, analyze its performance and accuracy, visualize the outputs, optimize and make the final model deployment-ready in a matter of minutes. DL Workbench takes you through the full OpenVINO™ workflow, providing the opportunity to learn about various toolkit components.
DL Workbench enables you to get a detailed performance assessment, explore inference configurations, and obtain an optimized model ready to be deployed on various Intel® configurations, such as client and server CPU, Intel® Processor Graphics (GPU), Intel® Movidius™ Neural Compute Stick 2 (NCS 2), and Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.
DL Workbench also provides the [JupyterLab environment](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Jupyter_Notebooks.html#doxid-workbench-docs-workbench-d-g-jupyter-notebooks) that helps you quick start with OpenVINO™ API and command-line interface (CLI). Follow the full OpenVINO workflow created for your model and learn about different toolkit components.
DL Workbench helps achieve your goals depending on the stage of your deep learning journey.
If you are a beginner in the deep learning field, the DL Workbench provides you with
learning opportunities:
* Learn what neural networks are, how they work, and how to examine their architectures.
* Learn the basics of neural network analysis and optimization before production.
* Get familiar with the OpenVINO™ ecosystem and its main components without installing it on your system.
If you have enough experience with neural networks, DL Workbench provides you with a
convenient web interface to optimize your model and prepare it for production:
* Measure and interpret model performance.
* Tune the model for enhanced performance.
* Analyze the quality of your model and visualize output.
## General Workflow
The diagram below illustrates the typical DL Workbench workflow. Click to see the full-size image:

Get a quick overview of the workflow in the DL Workbench User Interface:

## OpenVINO™ Toolkit Components
The intuitive web-based interface of the DL Workbench enables you to easily use various
OpenVINO™ toolkit components:
Component | Description
|------------------|------------------|
| [Open Model Zoo](https://docs.openvinotoolkit.org/latest/omz_tools_downloader.html)| Get access to the collection of high-quality pre-trained deep learning [public](https://docs.openvinotoolkit.org/latest/omz_models_group_public.html) and [Intel-trained](https://docs.openvinotoolkit.org/latest/omz_models_group_intel.html) models trained to resolve a variety of different tasks.
| [Model Optimizer](https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) |Optimize and transform models trained in supported frameworks to the IR format. <br>Supported frameworks include TensorFlow\*, Caffe\*, Kaldi\*, MXNet\*, and ONNX\* format.
| [Benchmark Tool](https://docs.openvinotoolkit.org/latest/openvino_inference_engine_tools_benchmark_tool_README.html)| Estimate deep learning model inference performance on supported devices.
| [Accuracy Checker](https://docs.openvinotoolkit.org/latest/omz_tools_accuracy_checker.html)| Evaluate the accuracy of a model by collecting one or several metric values.
| [Post-Training Optimization Tool](https://docs.openvinotoolkit.org/latest/pot_README.html)| Optimize pretrained models with lowering the precision of a model from floating-point precision(FP32 or FP16) to integer precision (INT8), without the need to retrain or fine-tune models. |
# Introduction to Model Processing {#openvino_docs_model_processing_introduction}
Every deep learning workflow begins with obtaining a model. You can choose to prepare a custom one, use a ready-made solution and adjust it to your needs, or even download and run a pre-trained network from an online database, such as OpenVINO's [Open Model Zoo](../model_zoo.md).
This section describes how to obtain and prepare your model for work with OpenVINO to get the best inference results:
* [Browse a database of models for use in your projects](../model_zoo.md).
* [Convert different model formats to the OpenVINO IR format](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
* [Automate model-related tasks with Model Downloader and additional OMZ Tools](https://docs.openvino.ai/latest/omz_tools_downloader.html).
OpenVINO™ is not just one tool. It is an expansive ecosystem of utilities, providing a comprehensive workflow for deep learning solution development. Learn more about each of them to reach the full potential of OpenVINO™ Toolkit.
### OpenVINO™ Model Server (OVMS)
OpenVINO Model Server is a scalable, high-performance solution for serving deep learning models optimized for Intel® architectures. The server uses Inference Engine libraries as a backend and exposes gRPC and HTTP/REST interfaces for inference that are fully compatible with TensorFlow Serving.
* [Red Hat Ecosystem Catalog](https://catalog.redhat.com/software/container-stacks/detail/60649e41ccfb383fe395a167)
### Neural Network Compression Framework (NNCF)
A suite of advanced algorithms for Neural Network inference optimization with minimal accuracy drop. NNCF applies quantization, filter pruning, binarization and sparsity algorithms to PyTorch and TensorFlow models during training.
A solution empowering TensorFlow developers with OpenVINO's optimization capabilities. With just two lines of code in your application, you can offload inference to OpenVINO, while keeping the TensorFlow API.
A streaming media analytics framework, based on the GStreamer multimedia framework, for creating complex media analytics pipelines.
More resources:
* [documentation on GitHub](https://openvinotoolkit.github.io/dlstreamer_gst/)
* [installation Guide on GitHub](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/Install-Guide)
### DL Workbench
A web-based tool for deploying deep learning models. Built on the core of OpenVINO and equipped with a graphics user interface, DL Workbench is a great way to explore the possibilities of the OpenVINO workflow, import, analyze, optimize, and build your pre-trained models. You can do all that by visiting [Intel® DevCloud for the Edge](https://software.intel.com/content/www/us/en/develop/tools/devcloud.html) and launching DL Workbench on-line.
# OpenVINO™ integration with TensorFlow {#ovtf_integration}
**OpenVINO™ integration with TensorFlow** is a solution for TensorFlow developers who want to get started with OpenVINO™ in their inferencing applications. By adding just two lines of code you can now take advantage of OpenVINO™ toolkit optimizations with TensorFlow inference applications across a range of Intel® computation devices.
This is all you need:
```bash
import openvino_tensorflow
openvino_tensorflow.set_backend('<backend_name>')
```
**OpenVINO™ integration with TensorFlow** accelerates inference across many AI models on a variety of Intel® technologies, such as:
- Intel® CPUs
- Intel® integrated GPUs
- Intel® Movidius™ Vision Processing Units - referred to as VPU
- Intel® Vision Accelerator Design with 8 Intel Movidius™ MyriadX VPUs - referred to as VAD-M or HDDL
> **NOTE**: For maximum performance, efficiency, tooling customization, and hardware control, we recommend developers to adopt native OpenVINO™ solutions.
To find out more about the product itself, as well as learn how to use it in your project, check its dedicated [GitHub repository](https://github.com/openvinotoolkit/openvino_tensorflow/tree/master/docs).
To see what you can do with **OpenVINO™ integration with TensorFlow**, explore the demos located in the [examples folder](https://github.com/openvinotoolkit/openvino_tensorflow/tree/master/examples) in our GitHub repository.
Sample tutorials are also hosted on [Intel® DevCloud](https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/build/ovtfoverview.html). The demo applications are implemented using Jupyter Notebooks. You can interactively execute them on Intel® DevCloud nodes, compare the results of **OpenVINO™ integration with TensorFlow**, native TensorFlow, and OpenVINO™.
## License
**OpenVINO™ integration with TensorFlow** is licensed under [Apache License Version 2.0](https://github.com/openvinotoolkit/openvino_tensorflow/blob/master/LICENSE).
By contributing to the project, you agree to the license and copyright terms therein
and release your contribution under these terms.
## Support
Submit your questions, feature requests and bug reports via [GitHub issues](https://github.com/openvinotoolkit/openvino_tensorflow/issues).
## How to Contribute
We welcome community contributions to **OpenVINO™ integration with TensorFlow**. If you have an idea for improvement:
* Share your proposal via [GitHub issues](https://github.com/openvinotoolkit/openvino_tensorflow/issues).
* Submit a [pull request](https://github.com/openvinotoolkit/openvino_tensorflow/pulls).
We will review your contribution as soon as possible. If any additional fixes or modifications are necessary, we will guide you and provide feedback. Before you make your contribution, make sure you can build **OpenVINO™ integration with TensorFlow** and run all the examples with your fix/patch. If you want to introduce a large feature, create test cases for your feature. Upon our verification of your pull request, we will merge it to the repository provided that the pull request has met the above mentioned requirements and proved acceptable.
---
\* Other names and brands may be claimed as the property of others.
@@ -52,7 +52,7 @@ Depending on model format used for import, mapping of custom operation is implem
2. If model is represented in TensorFlow, Caffe, Kaldi or MXNet formats, then [Model Optimizer Extensions](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) should be used. This approach is available for model conversion in Model Optimizer only.
Existing of two approaches simultaneously is explained by two different types of frontends used for model conversion in OpenVINO: new frontends (ONNX, PaddlePaddle) and legacy frontends (TensorFlow, Caffe, Kaldi and MXNet). Model Optimizer can use both front-ends in contrast to the direct import of model with `read_model` method which can use new frontends only. Follow one of the appropriate guides referenced above to implement mappings depending on framework frontend.
Existing of two approaches simultaneously is explained by two different types of frontends used for model conversion in OpenVINO: new frontends (ONNX, PaddlePaddle) and legacy frontends (TensorFlow, Caffe, Kaldi and Apache MXNet). Model Optimizer can use both front-ends in contrast to the direct import of model with `read_model` method which can use new frontends only. Follow one of the appropriate guides referenced above to implement mappings depending on framework frontend.
If you are implementing extensions for ONNX or PaddlePaddle new frontends and plan to use Model Optimizer `--extension` option for model conversion, then the extensions should be
# How to Implement Custom Layers for VPU (Intel® Neural Compute Stick 2) {#openvino_docs_IE_DG_Extensibility_DG_VPU_Kernel}
# How to Implement Custom Layers for VPU (Intel® Neural Compute Stick 2) {#openvino_docs_Extensibility_UG_VPU_Kernel}
To enable operations not supported by OpenVINO™ out of the box, you need a custom extension for Model Optimizer, a custom nGraph operation set, and a custom kernel for the device you will target. This page describes custom kernel support for one the VPU, the Intel® Neural Compute Stick 2 device, which uses the MYRIAD device plugin.
@@ -37,8 +37,8 @@ The implementation `CompileNetwork` is fully device-specific.
The function accepts a const shared pointer to `ngraph::Function` object and performs the following steps:
1. Applies ngraph passes using `TransformNetwork` function, which defines plugin-specific conversion pipeline. To support low precision inference, the pipeline can include Low Precision Transformations. These transformations are usually hardware specific. You can find how to use and configure Low Precisions Transformations in [Low Precision Transformations](@ref openvino_docs_OV_UG_lpt) guide.
2. Maps the transformed graph to a backend specific graph representation (for example, to MKLDNN graph for Intel CPU).
1. Applies nGraph passes using `TransformNetwork` function, which defines plugin-specific conversion pipeline. To support low precision inference, the pipeline can include Low Precision Transformations. These transformations are usually hardware specific. You can find how to use and configure Low Precisions Transformations in [Low Precision Transformations](@ref openvino_docs_OV_UG_lpt) guide.
2. Maps the transformed graph to a backend specific graph representation (for example, to CPU plugin internal graph representation).
3. Allocates and fills memory for graph weights, backend specific memory handles and so on.
Model Optimizer is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
Using Model Optimizer tool assumes you already have a deep learning model trained using one of the supported frameworks: TensorFlow, PyTorch, PaddlePaddle, MXNet, Caffe, Kaldi, or represented in ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the model, which can be inferred with [OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
To use it, you need a pre-trained deep learning model in one of the supported formats: TensorFlow, PyTorch, PaddlePaddle, MXNet, Caffe, Kaldi, or ONNX. Model Optimizer converts the model to the OpenVINO Intermediate Representation format (IR), which you can infer later with [OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md).
> **NOTE**: Model Optimizer does not infer models. Model Optimizer is an offline tool that converts a model into IR and optimizes before the inference takes place.
Note that Model Optimizer does not infer models.
The scheme below illustrates the typical workflow for deploying a trained deep learning model:
The figure below illustrates the typical workflow for deploying a trained deep learning model:

The IR is a pair of files describing the model:
where IR is a pair of files describing the model:
*<code>.xml</code> - Describes the network topology
*<code>.xml</code> - Describes the network topology.
*<code>.bin</code> - Contains the weights and biases binary data.
> **NOTE**: The generated IR can be additionally optimized for inference by [Post-training optimization](../../tools/pot/docs/Introduction.md)
The generated IR can be additionally optimized for inference by [Post-training optimization](../../tools/pot/docs/Introduction.md)
> that applies post-training quantization methods.
> **TIP**: You also can work with the Model Optimizer inside the OpenVINO™ [Deep Learning Workbench](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html) (DL Workbench).
> [DL Workbench](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html) is a web-based graphical environment that enables you to optimize, fine-tune, analyze, visualize, and compare performance of deep learning models.
> **TIP**: You can also work with Model Optimizer in OpenVINO™ [Deep Learning Workbench (DL Workbench)](https://docs.openvino.ai/latest/workbench_docs_Workbench_DG_Introduction.html), which is a web-based tool with GUI for optimizing, fine-tuning, analyzing, visualizing, and comparing performance of deep learning models.
## Run Model Optimizer
## How to Run Model Optimizer
To convert the model to IR, run Model Optimizer:
To convert a model to IR, you can run Model Optimizer by using the following command:
```sh
mo --input_model INPUT_MODEL
```
If out-of-the-box conversion (only the `--input_model` parameter is specified) is not succeed,
try to use parameters for overriding input shapes and cutting the model, mentioned below.
If the out-of-the-box conversion (only the `--input_model` parameter is specified) is not successful, use the parameters mentioned below to override input shapes and cut the model:
To override original input shapes for model conversion, Model Optimizer provides two parameters: `--input` and `--input_shape`.
For more information about these parameters, refer to [Setting Input Shapes](prepare_model/convert_model/Converting_Model.md).
- Model Optimizer provides two parameters to override original input shapes for model conversion: `--input` and `--input_shape`.
For more information about these parameters, refer to the [Setting Input Shapes](prepare_model/convert_model/Converting_Model.md) guide.
To cut off unwanted parts of a model, such as unsupported operations and training sub-graphs,
the `--input` and `--output` parameters can be used, defining new inputs and outputs of the converted model.
For a more detailed description, refer to [Cutting Off Parts of a Model](prepare_model/convert_model/Cutting_Model.md).
-To cut off unwanted parts of a model (such as unsupported operations and training sub-graphs),
use the `--input` and `--output` parameters to define new inputs and outputs of the converted model.
For a more detailed description, refer to the [Cutting Off Parts of a Model](prepare_model/convert_model/Cutting_Model.md) guide.
Also, you can insert additional input pre-processing sub-graphs into the converted model using
You can also insert additional input pre-processing sub-graphs into the converted model by using
the `--mean_values`, `scales_values`, `--layout`, and other parameters described
in [Embedding Preprocessing Computation](prepare_model/Additional_Optimizations.md).
in the [Embedding Preprocessing Computation](prepare_model/Additional_Optimizations.md) article.
Model Optimizer's compression parameter `--data_type` allows to generate IR of the `FP16` data type. For more details,
please refer to [Compression of a Model to FP16](prepare_model/FP16_Compression.md).
The `--data_type` compression parameter in Model Optimizer allows generating IR of the `FP16` data type. For more details, refer to the [Compression of a Model to FP16](prepare_model/FP16_Compression.md) guide.
To get the full list of conversion parameters available in Model Optimizer, run the following command:
@@ -81,54 +77,51 @@ mo --help
## Examples of CLI Commands
Below is a list of separate examples for different frameworks and Model Optimizer parameters.
Below is a list of separate examples for different frameworks and Model Optimizer parameters:
1. Launch Model Optimizer for a TensorFlow MobileNet model in the binary protobuf format.
1. Launch Model Optimizer for a TensorFlow MobileNet model in the binary protobuf format:
```sh
mo --input_model MobileNet.pb
```
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format, with three inputs. Explicitly specify input shapes
where the batch size and the sequence length equal 2 and 30 respectively.
Launch Model Optimizer for a TensorFlow BERT model in the SavedModel format with three inputs. Specify input shapes explicitly
where the batch size and the sequence length equal 2 and 30 respectively:
```sh
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
```
For more information on TensorFlow modelconversion,
refer to [Converting a TensorFlow Model](prepare_model/convert_model/Convert_Model_From_TensorFlow.md).
For more information, refer to the [Converting a TensorFlow Model](prepare_model/convert_model/Convert_Model_From_TensorFlow.md) guide.
2. Launch Model Optimizer for an ONNX OCR model and explicitly specify new output.
2. Launch Model Optimizer for an ONNX OCR model and specify new output explicitly:
```sh
mo --input_model ocr.onnx --output probabilities
```
For more information on ONNX model conversion,
please refer to [Converting an ONNX Model](prepare_model/convert_model/Convert_Model_From_ONNX.md).
Note that PyTorch models must be exported to the ONNX format before its conversion into IR.
More details can be found in [Converting a PyTorch Model](prepare_model/convert_model/Convert_Model_From_PyTorch.md).
For more information, refer to the [Converting an ONNX Model (prepare_model/convert_model/Convert_Model_From_ONNX.md) guide.
3. Launch Model Optimizer for a PaddlePaddle UNet model and apply mean-scale normalization to the input.
> **NOTE**: PyTorch models must be exported to the ONNX format before conversion into IR. More information can be found in [Converting a PyTorch Model](prepare_model/convert_model/Convert_Model_From_PyTorch.md).
3. Launch Model Optimizer for a PaddlePaddle UNet model and apply mean-scale normalization to the input:
```sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
```
For more information on PaddlePaddle modelconversion, please refer to
[Converting a PaddlePaddle Model](prepare_model/convert_model/Convert_Model_From_Paddle.md).
For more information, refer to the [Converting a PaddlePaddle Model](prepare_model/convert_model/Convert_Model_From_Paddle.md) guide.
4. Launch Model Optimizer for an MXNet SSD Inception V3 model and specify first-channel layout for the input.
4. Launch Model Optimizer for an Apache MXNet SSD Inception V3 model and specify first-channel layout for the input:
```sh
mo --input_model ssd_inception_v3-0000.params --layout NCHW
```
For more information on MXNet models conversion, please refer to [Converting an MXNet Model](prepare_model/convert_model/Convert_Model_From_MxNet.md).
For more information, refer to the [Converting an Apache MXNet Model](prepare_model/convert_model/Convert_Model_From_MxNet.md) guide.
5. Launch Model Optimizer for a Caffe AlexNet model with input channels in the RGB format, which needs to be reversed.
5. Launch Model Optimizer for a Caffe AlexNet model with input channels in the RGB format which needs to be reversed:
```sh
mo --input_model alexnet.caffemodel --reverse_input_channels
```
For more information on Caffe model conversion, please refer to [Converting a Caffe Model](prepare_model/convert_model/Convert_Model_From_Caffe.md).
For more information, refer to the [Converting a Caffe Model](prepare_model/convert_model/Convert_Model_From_Caffe.md) guide.
6. Launch Model Optimizer for a Kaldi LibriSpeech nnet2 model.
6. Launch Model Optimizer for a Kaldi LibriSpeech nnet2 model:
```sh
mo --input_model librispeech_nnet2.mdl --input_shape [1,140]
```
For more information on Kaldi modelconversion,
refer to [Converting a Kaldi Model](prepare_model/convert_model/Convert_Model_From_Kaldi.md).
For more information, refer to the [Converting a Kaldi Model](prepare_model/convert_model/Convert_Model_From_Kaldi.md) guide.
To get conversion recipes for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models,
refer to [Model Conversion Tutorials](prepare_model/convert_model/Convert_Model_Tutorials.md).
-To get conversion recipes for specific TensorFlow, ONNX, PyTorch, Apache MXNet, and Kaldi models,
refer to the [Model Conversion Tutorials](prepare_model/convert_model/Convert_Model_Tutorials.md).
- For more information about IR, see [Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™](IR_and_opsets.md).
# Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™ {#openvino_docs_MO_DG_IR_and_opsets}
This document provides essential information on the format used for representation of deep learning models in OpenVINO™ toolkit and supported operation sets.
This article provides essential information on the format used for representation of deep learning models in OpenVINO toolkit and supported operation sets.
## Overview of Artificial Neural Networks Representation
This paragraph provides an overview of how a deep learning network is represented in various deep learning frameworks.
A deep learning network is usually represented as a directed graph describing the flow of data from the network input data to the inference results.
Input data can be represented as a photograph, video, audio information or some preprocessed data that represent object from the target area of interest in a convenient way.
Input data can be in the form of images, video, audio, or preprocessed information representing objects from the target area of interest.
Here is an illustration of a small graph representing a model that consists of a single Convolutional layer and activation function:

Vertices in the graph represent layers or operation instances, like convolution, pooling or element-wise operations with tensors.
Layer and operation terms are used interchangeably along the OpenVINO™ documentation and define how input data is processed to produce output data for a node in a graph.
Vertices in the graph represent layers or operation instances such as convolution, pooling, and element-wise operations with tensors.
The terms of "layer" and "operation" are used interchangeably within OpenVINO documentation and define how input data is processed to produce output data for a node in a graph.
An operation node in a graph may consume data at one or multiple input ports.
For example, element-wise addition operation has two input ports which accepts tensors that are added together.
Some operations don't have any input ports, for example Const operation which knowns the data to be produced without any input.
An edge between operations represent data flow or data dependency implied from one operation node to another operation node.
For example, an element-wise addition operation has two input ports which accept tensors that are to be summed.
Some operations do not have any input ports, for example the `Const` operation, which knows the data to be produced without any input.
An edge between operations represents data flow or data dependency implied from one operation node to another.
Each operation produces data on one or multiple output ports. For example, convolution produces output tensor with activations at a single output port. Split operation usually has multiple output ports each producing part of an input tensor.
Each operation produces data on one or multiple output ports. For example, convolution produces output tensor with activations at a single output port. Split operation usually has multiple output ports, each producing part of an input tensor.
Depending on a deep learning framework, the graph can also contain extra nodes that explicitly represent tensors between operations.
In such representations, operation nodes are not connected directly to each other, rather using data nodes as intermediate stops for data flow.
In such representations, operation nodes are not connected to each other directly. They are rather using data nodes as intermediate stops for data flow.
If data nodes are not used, the produced data is associated with an output port of a corresponding operation node that produces the data.
A set of various operations used in a network is usually fixed for each deep learning framework.
It determines expressiveness and level of representation available in that framework.
It may happen that a network that can be represented in one framework is hard or impossible to be represented in another one or should use significantly different graph because operation sets used in those two frameworks do not match.
Sometimes, a network that can be represented in one framework is hard or impossible to be represented in another one or should use significantly different graph, because operation sets used in those two frameworks do not match.
## Intermediate Representation Used in OpenVINO™
## Intermediate Representation Used in OpenVINO
OpenVINO™ toolkit introduces its own format of graph representation and its own operation set.
OpenVINO toolkit introduces its own format of graph representation and its own operation set.
A graph is represented with two files: an XML file and a binary file.
This representation is commonly referred to as the *Intermediate Representation* or *IR*.
The XML file describes a network topology using a `<layer>` tag for an operation node and an `<edge>` tag for a data-flow connection.
Each operation has a fixed number of attributes that define operation flavor used for a node.
For example, `Convolution` operation has such attributes as `dilation`, `stride`, `pads_begin` and `pads_end`.
For example, the`Convolution` operation has such attributes as `dilation`, `stride`, `pads_begin`, and `pads_end`.
The XML file doesn't have big constant values, like convolution weights.
The XML file does not have big constant values like convolution weights.
Instead, it refers to a part of the accompanying binary file that stores such values in a binary format.
Here is an example of a small IR XML file that corresponds to a graph from the previous section:
@@ -151,48 +149,48 @@ Here is an example of a small IR XML file that corresponds to a graph from the p
</net>
```
The IR doesn't use explicit data nodes described in the previous section.
The IR does not use explicit data nodes described in the previous section.
In contrast, properties of data such as tensor dimensions and their data types are described as properties of input and output ports of operations.
## Operation Set
## Operation Sets
Operations in the OpenVINO™ Operation Set are selected based on capabilities of supported deep learning frameworks and hardware capabilities of the target inference device.
Operations in OpenVINO Operation Sets are selected based on capabilities of supported deep learning frameworks and hardware capabilities of the target inference device.
It consists of several groups of operations:
* Conventional deep learning layers like Convolution, MaxPool, MatMul (also known as FullyConnected).
* Conventional deep learning layers such as `Convolution`, `MaxPool`, and `MatMul` (also known as `FullyConnected`).
* Various activation functions, e.g. ReLU, Tanh, PReLU.
* Various activation functions such as `ReLU`, `Tanh`, and `PReLU`.
* Generic element-wise arithmetic tensor operations like Add, Subtract, Multiply.
* Generic element-wise arithmetic tensor operations such as `Add`, `Subtract`, and `Multiply`.
* Comparison operations that compare two numeric tensors and produce boolean tensors, for example Less, Equeal, Greater.
* Comparison operations that compare two numeric tensors and produce boolean tensors, for example, `Less`, `Equeal`, `Greater`.
* Logical operations that are dealing with boolean tensors, like And, Xor, Not.
* Logical operations that are dealing with boolean tensors, for example, `And`, `Xor`, `Not`.
* Data movement operations which are dealing with parts of tensors: Concat, Split, StridedSlice, Select.
* Data movement operations which are dealing with parts of tensors, for example, `Concat`, `Split`, `StridedSlice`, `Select`.
* Specialized operations that implement complex algorithms dedicated for models of specific type: DetectionOutput, RegionYolo, PriorBox.
* Specialized operations that implement complex algorithms dedicated for models of specific type, for example, `DetectionOutput`, `RegionYolo`, `PriorBox`.
Refer to the complete description of the supported operation sets in the [Available Operation Sets](../ops/opset.md) document.
For more information, refer to the complete description of the supported operation sets in the [Available Operation Sets](../ops/opset.md) article.
## IR Versions vs Operation Set Versions
The expressiveness of operations in OpenVINO™ is highly dependent on the supported frameworks and target hardware capabilities.
The expressiveness of operations in OpenVINO is highly dependent on the supported frameworks and target hardware capabilities.
As the frameworks and hardware capabilities grow over time, the operation set is constantly evolving to support new models.
To maintain backward compatibility and growing demands, both IR format and operation set have versioning.
Version of IR specifies the rules which are used to read the XML and binary files that represent a model. It defines an XML schema and compatible operation set that can be used to describe operations.
Historically, there are two major IR version epochs.
Historically, there are two major IR version epochs:
1. The older one includes IR versions from version 1 to version 7 without versioning of the operation set. During that epoch, the operation set has been growing evolutionally accumulating more layer types and extending existing layer semantics. Changing of the operation set for those versions meant increasing of IR version.
2. OpenVINO™ 2020.1 is the starting point of the next epoch. With IR version 10 introduced in OpenVINO™ 2020.1, the versioning of the operation set is tracked separately from the IR versioning. Also, the operation set was significantly reworked as the result of nGraph integration to the OpenVINO.
2. OpenVINO 2020.1 is the starting point of the next epoch. With IR version 10 introduced in OpenVINO 2020.1, the versioning of the operation set is tracked separately from the IR versioning. Also, the operation set was significantly reworked as the result of nGraph integration to the OpenVINO.
The first supported operation set in the new epoch is `opset1`.
The number after `opset` is going to be increased each time when new operations are added or old operations deleted at the release cadence.
The number after `opset` is going to be increased each time new operations are added or old operations deleted at the release cadence.
The operations from the new epoch cover more TensorFlow* and ONNX* operators in a form that is closer to the original operation semantics from the frameworks in comparison to the operation set used in former versions of IR (7 and lower).
The operations from the new epoch cover more TensorFlow and ONNX operators in a form that is closer to the original operation semantics from the frameworks in comparison to the operation set used in former versions of IR (7 and lower).
The name of the opset is specified for each operation in IR.
The IR version is specified once per whole IR.
@@ -215,31 +213,30 @@ Here is an example from the IR snippet:
...
```
The attributes `type="Parameter"` and `version="opset1"` in the example above mean "use that version of operation`Parameter` that is included into the operation set`opset1`".
The `type="Parameter"` and `version="opset1"` attributes in the example above mean "use that version of the`Parameter` operation that is included in the`opset1` operation set. "
When a new operation set is introduced, the significant part of the operations remains unchanged and it is just aliased from the previous operation set within a new one.
The goal of operation set versions evolution is adding new operations, and probably changing of small fraction of existing operations (fixing bugs and extending semantics).
However such changes affect only new versions of operations from a new operation set, while old operations are used by specifying an appropriate `version`.
When the old `version` is specified, the behavior is kept unchanged from that specified version to provide the backward compatibility with older IRs.
When a new operation set is introduced, most of the operations remain unchanged and are just aliased from the previous operation set within a new one.
The goal of operation set version evolution is to add new operations, and probably change small fractions of existing operations (fixing bugs and extending semantics).
However, such changes affect only new versions of operations from a new operation set, while old operations are used by specifying an appropriate `version`.
When an old `version` is specified, the behavior will be kept unchanged from that specified version to provide backward compatibility with older IRs.
A single `xml` file with IR may contain operations from different opsets.
An operation that is included into several opsets may be referred to with `version` which points to any opset that includes that operation.
For example, the same `Convolution` can be used with `version="opset1"` and `version="opset2"` because both opsets have the same operations `Convolution`.
An operation that is included in several opsets may be referred to with `version` which points to any opset that includes that operation.
For example, the same `Convolution` can be used with `version="opset1"` and `version="opset2"` because both opsets have the same `Convolution` operations.
## How to Read the Specification
## How to Read Opset Specification
In the [Available Operation Sets](../ops/opset.md) there are opsets and there are operations.
Each opset specification has a list of links to operations descriptions that are included into that specific opset.
Two or more opsets may refer to the same operation.
That means an operation is kept unchanged from one operation set to another.
Each operation description has a field `Versioned name`.
For example, `ReLU` entry point in [`opset1`](../ops/opset1.md) refers to [`ReLU-1`](../ops/activation/ReLU_1.md) as the versioned name.
And`ReLU` in `opset2` refers to the same `ReLU-1` and both `ReLU` operations are the same operation and it has a single [description](../ops/activation/ReLU_1.md).
So `opset1` and `opset2` share the same operation `ReLU`.
The description of each operation has a `Versioned name` field.
For example, the`ReLU` entry point in [`opset1`](../ops/opset1.md) refers to [`ReLU-1`](../ops/activation/ReLU_1.md) as the versioned name.
Meanwhile,`ReLU` in `opset2` refers to the same `ReLU-1` and both `ReLU` operations are the same operation and it has a single [description](../ops/activation/ReLU_1.md), which means that `opset1` and `opset2` share the same operation `ReLU`.
To differentiate versions of the same operation type, like`ReLU`, the suffix `-N` is used in a versioned name of the operation.
`N` usually refers to the first `opsetN` where this version of the operation is introduced.
It is not guaranteed that new operations will be named according to that rule, the naming convention might be changed, but not for old operations which are frozen completely.
To differentiate versions of the same operation type such as`ReLU`, the `-N` suffix is used in a versioned name of the operation.
The `N` suffix usually refers to the first occurrence of `opsetN` where this version of the operation is introduced.
There is no guarantee that new operations will be named according to that rule. The naming convention might be changed, but not for old operations which are frozen completely.
Input data for inference can be different from the training dataset and requires additional preprocessing before inference.
To accelerate the whole pipeline including preprocessing and inference, Model Optimizer provides special parameters such as `--mean_values`,
`--scale_values`, `--reverse_input_channels`, and `--layout`. Based on these parameters, Model Optimizer generates IR with additionally
inserted sub-graph that performs the defined preprocessing. This preprocessing block can perform mean-scale normalization of input data,
reverting data along channel dimension, and changing the data layout. For more details about these parameters, refer to the paragraphs below.
The same functionality is also available in runtime, please refer to [Overview of Preprocessing API](../../OV_Runtime_UG/preprocessing_overview.md)
for more information.
## When to Specify Layout
`--scale_values`, `--reverse_input_channels`, and `--layout`. Based on these parameters, Model Optimizer generates OpenVINO IR with additionally
inserted sub-graphs to perform the defined preprocessing. This preprocessing block can perform mean-scale normalization of input data,
reverting data along channel dimension, and changing the data layout.
See the following sections for details on the parameters, or the [Overview of Preprocessing API](../../OV_Runtime_UG/preprocessing_overview.md) for the same functionality in OpenVINO Runtime.
You may need to set input layouts, as it is required by some preprocessing, for example, setting a batch,
applying mean or scales, and reversing input channels (BGR<->RGB).
## Specifying Layout
You may need to set input layouts, as it is required by some preprocessing, for example, setting a batch, applying mean or scales, and reversing input channels (BGR<->RGB).
Layout defines the meaning of dimensions in shape and can be specified for both inputs and outputs. Some preprocessing requires to set input layouts, for example, setting a batch, applying mean or scales, and reversing input channels (BGR<->RGB).
Layout defines the meaning of dimensions in shape and can be specified for both inputs and outputs.
For the layout syntax, check the [Layout API overview](../../OV_Runtime_UG/layout_overview.md).
To specify the layout, you can use `--layout` option followed by the layout value.
To specify the layout, you can use the `--layout` option followed by the layout value.
For example, for Tensorflow\*`nasnet_large` model that was exported to ONNX format and thus has input with `NHWC` layout:
For example, the following command specifies the `NHWC` layout for a Tensorflow `nasnet_large` model that was exported to the ONNX format:
```
mo --input_model tf_nasnet_large.onnx --layout nhwc
```
Additionally, if a model has more than one input or needs both input and output layouts specified,
you need to provide the name of each input or output to which you apply the layout.
Additionally, if a model has more than one input or needs both input and output layouts specified, you need to provide the name of each input or output to apply the layout.
For example, for ONNX\*`Yolo v3 Tiny` model that has first input `input_1` in `NCHW` layout and second input `image_shape`
with 2 dimensions: batch and size of the image which can be expressed as `N?` layout:
For example, the following command specifies the layout for an ONNX `Yolo v3 Tiny` model with its first input `input_1` in `NCHW` layout and second input `image_shape` having two dimensions: batch and size of the image expressed as the `N?` layout:
```
mo --input_model yolov3-tiny.onnx --layout input_1(nchw),image_shape(n?)
```
## How to Change Layout of a Model Inputs and Outputs
## Changing Model Layout
Changing the model layout may be necessary if it differs from the one presented by input data.
To change the layout, you can use either `--layout` or `--source_layout` with `--target_layout`.
Use either `--layout` or `--source_layout` with `--target_layout` to change the layout.
For example, for the same `nasnet_large`that were mentioned previously we may want to provide data in `NCHW` layout:
For example, for the same `nasnet_large`model mentioned previously, you can use the following commands to provide data in the `NCHW` layout:
```
mo --input_model tf_nasnet_large.onnx --source_layout nhwc --target_layout nchw
mo --input_model tf_nasnet_large.onnx --layout "nhwc->nchw"
```
Again, if a model has more than one input or needs both input and output layouts specified, you need to provide the name of each input or output to which you apply the layout.
Again, if a model has more than one input or needs both input and output layouts specified, you need to provide the name of each input or output to apply the layout.
For example, to provide data in the `NHWC` layout for the `Yolo v3 Tiny` model mentioned earlier:
For example, to provide data in the `NHWC` layout for the `Yolo v3 Tiny` model mentioned earlier, use the following commands:
```
mo --input_model yolov3-tiny.onnx --source_layout "input_1(nchw),image_shape(n?)" --target_layout "input_1(nhwc)"
mo --input_model yolov3-tiny.onnx --layout "input_1(nchw->nhwc),image_shape(n?)"
```
## When to Specify Mean and Scale Values
Usually neural network models are trained with the normalized input data. This means that the input data values are converted to be in a specific range,
for example, `[0, 1]` or `[-1, 1]`. Sometimes the mean values (mean images) are subtracted from the input data values as part of the pre-processing.
There are two cases of how the input data pre-processing is implemented.
* The input pre-processing operations are a part of a model. In this case, the application does not pre-process the input data as a separate step: everything is embedded into the model itself.
* The input pre-processing operations are not a part of a model and the pre-processing is performed within the application which feeds the model with input data.
## Specifying Mean and Scale Values
Neural network models are usually trained with the normalized input data. This means that the input data values are converted to be in a specific range,
for example, `[0, 1]` or `[-1, 1]`. Sometimes, the mean values (mean images) are subtracted from the input data values as part of the preprocessing.
In the first case, the Model Optimizer generates the IR with required pre-processing operations and no `mean` and `scale` parameters are required.
There are two cases of how the input data preprocessing is implemented.
* The input preprocessing operations are a part of a model.
In this case, the application does not perform a separate preprocessing step: everything is embedded into the model itself. Model Optimizer will generate the OpenVINO IR format with required preprocessing operations, and no `mean` and `scale` parameters are required.
* The input preprocessing operations are not a part of a model and the preprocessing is performed within the application which feeds the model with input data.
In this case, information about mean/scale values should be provided to Model Optimizer to embed it to the generated OpenVINO IR format.
In the second case, information about mean/scale values should be provided to the Model Optimizer to embed it to the generated IR.
Model Optimizer provides command-line parameters to specify the values: `--mean_values`, `--scale_values`, `--scale`.
Using these parameters, Model Optimizer embeds the corresponding preprocessing block for mean-value normalization of the input data
and optimizes this block so that the preprocessing takes negligible time for inference.
For example, run the Model Optimizer for the PaddlePaddle* UNet model and apply mean-scale normalization to the input data.
For example, the following command runs Model Optimizer for the PaddlePaddle UNet model and applies mean-scale normalization to the input data:
```sh
mo --input_model unet.pdmodel --mean_values [123,117,104] --scale 255
```
## When to Reverse Input Channels <a name="when_to_reverse_input_channels"></a>
Sometimes input images for your application can be of the RGB (BGR) format and the model is trained on images of the BGR (RGB) format,
the opposite color channel order. In this case, it is important to preprocess the input images by reverting the color channels before inference.
To embed this preprocessing step into IR, Model Optimizer provides the `--reverse_input_channels` command-line parameter to shuffle the color channels.
Sometimes, input images for your application can be of the RGB (or BGR) format and the model is trained on images of the BGR (or RGB) format,
which is in the opposite order of color channels. In this case, it is important to preprocess the input images by reverting the color channels before inference.
The `--reverse_input_channels` parameter applies to an input of the model in two cases.
To embed this preprocessing step into OpenVINO IR, Model Optimizer provides the `--reverse_input_channels` command-line parameter to shuffle the color channels.
The `--reverse_input_channels` parameter can be used to preprocess the model input in the following cases:
* Only one dimension in the input shape has a size equal to 3.
* One dimension has an undefined size and is marked as `C` channel using `layout` parameters.
Using the `--reverse_input_channels` parameter, Model Optimizer embeds the corresponding preprocessing block for reverting
the input data along channel dimension and optimizes this block so that the preprocessing takes negligible time for inference.
the input data along channel dimension and optimizes this block so that the preprocessing takes only negligible time for inference.
For example, launch the Model Optimizer for the TensorFlow* AlexNet model and embed `reverse_input_channel` preprocessing block into IR.
For example, the following command launches Model Optimizer for the TensorFlow AlexNet model and embeds the`reverse_input_channel` preprocessing block into OpenVINO IR:
```sh
mo --input_model alexnet.pb --reverse_input_channels
```
> **NOTE**: If both mean and scale values are specified, the mean is subtracted first and then the scale is applied regardless of the order of options
in the commandline. Input values are *divided* by the scale value(s). If also`--reverse_input_channels` option is used, the`reverse_input_channels`
in the command-line. Input values are *divided* by the scale value(s). If the`--reverse_input_channels` option is also used, `reverse_input_channels`
will be applied first, then `mean` and after that `scale`. The data flow in the model looks as follows:
`Parameter -> ReverseInputChannels -> Mean apply-> Scale apply -> the original body of the model`.
## See Also
## Additional Resources
* [Overview of Preprocessing API](../../OV_Runtime_UG/preprocessing_overview.md)
This guide introduces things to notice and how to use the benchmark_app to get performance numbers. It also explains how the performance numbers are reflected through internal inference performance counters and execution graphs. In the last section, it includes information on using ITT and Intel® VTune™ Profiler to get performance insights.
## Tip 1. Measure the Proper Set of Operations
## Tip 1: Select Proper Set of Operations to Measure
When evaluating performance of your model with the OpenVINO Runtime, you must measure the proper set of operations. To do so, consider the following tips:
When evaluating the performance of a model with OpenVINO Runtime, it is required to measure proper set of operations. Remember the following tips:
- Avoid including one-time costs such as model loading.
-Avoid including one-time costs like model loading.
-Track operations that occur outside OpenVINO Runtime (such as video decoding) separately.
- Track separately the operations that happen outside the OpenVINO Runtime, like video decoding.
> **NOTE**: Some image pre-processing can be baked into OpenVINO IR and accelerated accordingly. For more information, refer to [Embedding the Pre-processing](Additional_Optimizations.md) and [General Runtime Optimizations](../../optimization_guide/dldt_deployment_optimization_common).
> **NOTE**: Some image pre-processing can be baked into the IR and accelerated accordingly. For more information, refer to [Embedding the Preprocessing](Additional_Optimizations.md). Also consider [Runtime Optimizations of the Preprocessing](../../optimization_guide/dldt_deployment_optimization_common).
## Tip 2: Try to Get Credible Data
## Tip 2. Getting Credible Performance Numbers
You need to build your performance conclusions on reproducible data. Do the performance measurements with a large number of invocations of the same routine. Since the first iteration is almost always significantly slower than the subsequent ones, you can use an aggregated value for the execution time for final projections:
Performance conclusions should be build upon reproducible data. As for the performance measurements, they should be done with a large number of invocations of the same routine. Since the first iteration is almost always significantly slower than the subsequent ones, an aggregated value can be used for the execution time for final projections:
- If the warm-up run does not help or execution time still varies, you can try running a large number of iterations and then average or find a mean of the results.
-For time values that range too much, consider geomean.
-Beware of the throttling and other power oddities. A device can exist in one of several different power states. When optimizing your model, for better performance data reproducibility consider fixing the device frequency. However the endtoend (application) benchmarking should be also performed under real operational conditions.
-If the time values range too much, consider geomean.
- Be aware of the throttling and other power oddities. A device can exist in one of several different power states. When optimizing your model, consider fixing the device frequency for better performance data reproducibility. However, the end-to-end (application) benchmarking should also be performed under real operational conditions.
## Tip 3. Measure Reference Performance Numbers with OpenVINO's benchmark_app
## Using benchmark_app to Measure Reference Performance Numbers
To get performance numbers, use the dedicated [Benchmark App](../../../samples/cpp/benchmark_app/README.md) sample which is the best way to produce the performance reference.
It has a lot of device-specific knobs, but the primary usage is as simple as:
To get performance numbers, use the dedicated [OpenVINO Benchmark app](../../../samples/cpp/benchmark_app/README.md) sample, which is the most-recommended solution to produce performance reference.
It includes a lot of device-specific knobs, but the primary usage is as simple as in the following command to measure the performance of the model on GPU:
Each of the [OpenVINO supported devices](../../OV_Runtime_UG/supported_plugins/Supported_Devices.md) offers performance settings that have command-line equivalents in the [Benchmark App](../../../samples/cpp/benchmark_app/README.md).
While these settings provide really low-level control and allow to leverage the optimal model performance on the _specific_ device, we suggest always starting the performance evaluation with the [OpenVINO High-Level Performance Hints](../../OV_Runtime_UG/performance_hints.md) first:
Each of the [OpenVINO supported devices](../../OV_Runtime_UG/supported_plugins/Supported_Devices.md) offers performance settings that contain command-line equivalents in the [Benchmark app](../../../samples/cpp/benchmark_app/README.md).
While these settings provide really low-level control and allow leveraging the optimal model performance on the _specific_ device, it is recommended to always start the performance evaluation with the [OpenVINO High-Level Performance Hints](../../OV_Runtime_UG/performance_hints.md) first:
- benchmark_app **-hint tput** -d 'device' -m 'path to your model'
- benchmark_app **-hint latency** -d 'device' -m 'path to your model'
## Comparing Performance with Native/Framework Code
## Notes for Comparing Performance with Native/Framework Code
When comparing the OpenVINO Runtime performance with the framework or another reference code, make sure that both versions are as similar as possible:
- Wrap exactly the inference execution (refer to the [Benchmark App](../../../samples/cpp/benchmark_app/README.md) for examples).
- Wrap the exact inference execution (refer to the [Benchmark app](../../../samples/cpp/benchmark_app/README.md) for examples).
- Do not include model loading time.
- Ensure the inputs are identical for the OpenVINO Runtime and the framework. For example, beware of random values that can be used to populate the inputs.
-Consider [Image Pre-processing and Conversion](../../OV_Runtime_UG/preprocessing_overview.md), while any user-side pre-processing should be tracked separately.
-When applicable, leverage the [Dynamic Shapes support](../../OV_Runtime_UG/ov_dynamic_shapes.md)
- Ensure that the inputs are identical for OpenVINO Runtime and the framework. For example, watch out for random values that can be used to populate the inputs.
-In situations when any user-side pre-processing should be tracked separately, consider [image pre-processing and conversion](../../OV_Runtime_UG/preprocessing_overview.md).
- When applicable, leverage the [Dynamic Shapes support](../../OV_Runtime_UG/ov_dynamic_shapes.md).
- If possible, demand the same accuracy. For example, TensorFlow allows `FP16` execution, so when comparing to that, make sure to test the OpenVINO Runtime with the `FP16` as well.
## Internal Inference Performance Counters and Execution Graphs <a name="performance-counters"></a>
Further, finer-grained insights into inference performance breakdown can be achieved with device-specific performance counters and/or execution graphs.
Both [C++](../../../samples/cpp/benchmark_app/README.md) and [Python](../../../tools/benchmark_tool/README.md) versions of the `benchmark_app` supports a `-pc` command-line parameter that outputs internal execution breakdown.
## Data from Internal Inference Performance Counters and Execution Graphs <a name="performance-counters"></a>
More detailed insights into inference performance breakdown can be achieved with device-specific performance counters and/or execution graphs.
Both [C++](../../../samples/cpp/benchmark_app/README.md) and [Python](../../../tools/benchmark_tool/README.md) versions of the `benchmark_app` support a `-pc` command-line parameter that outputs internal execution breakdown.
For example, below is the part of performance counters for quantized [TensorFlow* implementation of ResNet-50](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf) model inference on [CPU Plugin](../../OV_Runtime_UG/supported_plugins/CPU.md).
Notice that since the device is CPU, the layers wall clock `realTime` and the `cpu` time are the same. Information about layer precision is also stored in the performance counters.
For example, the table shown below is the part of performance counters for quantized [TensorFlow implementation of ResNet-50](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf) model inference on [CPU Plugin](../../OV_Runtime_UG/supported_plugins/CPU.md).
Keep in mind that since the device is CPU, the `realTime` wall clock and the `cpu` time layers are the same. Information about layer precision is also stored in the performance counters.
The `exeStatus` column of the table includes possible values:
-`EXECUTED` - layer was executed by standalone primitive,
-`NOT_RUN` - layer was not executed by standalone primitive or was fused with another operation and executed in another layer primitive.
The `exeStatus` column of the table includes the following possible values:
-`EXECUTED` - the layer was executed by standalone primitive.
-`NOT_RUN` - the layer was not executed by standalone primitive or was fused with another operation and executed in another layer primitive.
The `execType` column of the table includes inference primitives with specific suffixes. The layers have the following marks:
*Suffix`I8` for layers that had 8-bit data type input and were computed in 8-bit precision
*Suffix`FP32` for layers computed in 32-bit precision
The `execType` column of the table includes inference primitives with specific suffixes. The layers could have the following marks:
*The`I8` suffix is for layers that had 8-bit data type input and were computed in 8-bit precision.
*The`FP32` suffix is for layers computed in 32-bit precision.
All `Convolution` layers are executed in int8 precision. Rest layers are fused into Convolutions using postoperations optimization technique, which is described in [Internal CPU Plugin Optimizations](../../OV_Runtime_UG/supported_plugins/CPU.md).
This contains layers name (as seen in IR), layers type and execution statistics.
All `Convolution` layers are executed in `int8` precision. The rest of the layers are fused into Convolutions using post-operation optimization, as described in [CPU Device](../../OV_Runtime_UG/supported_plugins/CPU.md).
This contains layer names (as seen in OpenVINO IR), type of the layer, and execution statistics.
Both benchmark_app versions also support "exec_graph_path" command-line option governing the OpenVINO to output the same per-layer execution statistics, but in the form of the plugin-specific [Netron-viewable](https://netron.app/) graph to the specified file.
Both `benchmark_app` versions also support the `exec_graph_path` command-line option. It requires OpenVINO to output the same execution statistics per layer, but in the form of plugin-specific [Netron-viewable](https://netron.app/) graph to the specified file.
Notice that on some devices, the execution graphs/counters may be pretty intrusive overhead-wise.
Also, especially when performance-debugging the [latency case](../../optimization_guide/dldt_deployment_optimization_latency.md) notice that the counters do not reflect the time spent in the plugin/device/driver/etc queues. If the sum of the counters is too different from the latency of an inference request, consider testing with less inference requests. For example running single [OpenVINO stream](../../optimization_guide/dldt_deployment_optimization_tput.md) with multiple requests would produce nearly identical counters as running single inference request, yet the actual latency can be quite different.
Especially when performance-debugging the [latency](../../optimization_guide/dldt_deployment_optimization_latency.md), note that the counters do not reflect the time spent in the `plugin/device/driver/etc` queues. If the sum of the counters is too different from the latency of an inference request, consider testing with less inference requests. For example, running single [OpenVINO stream](../../optimization_guide/dldt_deployment_optimization_tput.md) with multiple requests would produce nearly identical counters as running a single inference request, while the actual latency can be quite different.
Finally, the performance statistics with both performance counters and execution graphs is averaged, so such a data for the [dynamically-shaped inputs](../../OV_Runtime_UG/ov_dynamic_shapes.md) should be measured carefully (ideally by isolating the specific shape and executing multiple times in a loop, to gather the reliable data).
Lastly, the performance statistics with both performance counters and execution graphs are averaged, so such data for the [inputs of dynamic shapes](../../OV_Runtime_UG/ov_dynamic_shapes.md) should be measured carefully, preferably by isolating the specific shape and executing multiple times in a loop, to gather the reliable data.
OpenVINO in general and individual plugins are heavily instrumented with Intel® instrumentation and tracing technology (ITT), so another option is to compile the OpenVINO from the source code with the ITT enabled and using tools like [Intel® VTune™ Profiler](https://software.intel.com/en-us/vtune) to get detailed inference performance breakdown and additional insights in the application-level performance on the timeline view.
## Using ITT to Get Performance Insights
In general, OpenVINO and its individual plugins are heavily instrumented with Intel® Instrumentation and Tracing Technology (ITT). Therefore, you can also compile OpenVINO from the source code with ITT enabled and use tools like [Intel® VTune™ Profiler](https://software.intel.com/en-us/vtune) to get detailed inference performance breakdown and additional insights in the application-level performance on the timeline view.
# Model Optimization Techniques {#openvino_docs_MO_DG_prepare_model_Model_Optimization_Techniques}
Optimization offers methods to accelerate inference with the convolution neural networks (CNN) that do not require model retraining.
* * *
## Linear Operations Fusing
Many convolution neural networks includes `BatchNormalization` and `ScaleShift` layers (for example, Resnet\*, Inception\*) that can be presented as a sequence of linear operations: additions and multiplications. For example ScaleShift layer can be presented as Mul → Add sequence. These layers can be fused into previous `Convolution` or `FullyConnected` layers, except when Convolution comes after an Add operation (due to Convolution paddings).
### Usage
In the Model Optimizer, this optimization is turned on by default. To disable it, you can pass `--disable_fusing` parameter to the Model Optimizer.
### Optimization Description
This optimization method consists of three stages:
1.`BatchNormalization` and `ScaleShift` decomposition: in this stage, `BatchNormalization` layer is decomposed to `Mul → Add → Mul → Add` sequence, and `ScaleShift` layer is decomposed to `Mul → Add` layers sequence.
2. Linear operations merge: in this stage, the `Mul` and `Add` operations are merged into a single `Mul → Add` instance.
For example, if there is a `BatchNormalization → ScaleShift` sequence in the topology, it is replaced with `Mul → Add` in the first stage. In the next stage, the latter is replaced with a `ScaleShift` layer if there is no available `Convolution` or `FullyConnected` layer to fuse into next.
3. Linear operations fusion: in this stage, the tool fuses `Mul` and `Add` operations to `Convolution` or `FullyConnected` layers. Note that it searches for `Convolution` and `FullyConnected` layers both backward and forward in the graph (except for `Add` operation that cannot be fused to `Convolution` layer in forward direction).
### Usage Examples
The picture below shows the depicted part of Caffe Resnet269 topology where `BatchNorm` and `ScaleShift` layers will be fused to `Convolution` layers.

* * *
## ResNet optimization (stride optimization)
ResNet optimization is a specific optimization that applies to Caffe ResNet topologies such as ResNet50, ResNet101, ResNet152 and to ResNet-based topologies. This optimization is turned on by default, and can be disabled with the `--disable_resnet_optimization` key.
### Optimization Description
In the picture below, you can see the original and optimized parts of a Caffe ResNet50 model. The main idea of this optimization is to move the stride that is greater than 1 from Convolution layers with the kernel size = 1 to upper Convolution layers. In addition, the Model Optimizer adds a Pooling layer to align the input shape for a Eltwise layer, if it was changed during the optimization.

In this example, the stride from the `res3a_branch1` and `res3a_branch2a` Convolution layers moves to the `res2c_branch2b` Convolution layer. In addition, to align the input shape for `res2c` Eltwise, the optimization inserts the Pooling layer with kernel size = 1 and stride = 2.
* * *
## Grouped Convolution Fusing
Grouped convolution fusing is a specific optimization that applies for TensorFlow topologies. The main idea of this optimization is to combine convolutions results for the `Split` outputs and then recombine them using `Concat` operation in the same order as they were out from `Split`.

* * *
## Disabling Fusing
Model Optimizer allows to disable optimizations for specified nodes via `--finegrain_fusing <node_name1>,<node_name2>,...` (regex is also supported). Using this key, you mark nodes that will noy be touched by any optimizations.
### Examples of usage
On the picture below you can see two visualized Intermediate Representations (IR) of TensorFlow InceptionV4 topology.
The first one is original IR that will be produced by the Model Optimizer.
The second one will be produced by the Model Optimizer with key `--finegrain_fusing InceptionV4/InceptionV4/Conv2d_1a_3x3/Conv2D`, where you can see that `Convolution` was not fused with `Mul1_3752` and `Mul1_4061/Fused_Mul_5096/FusedScaleShift_5987` operations.

@@ -4,25 +4,25 @@ If your question is not covered by the topics below, use the [OpenVINO™ Su
#### 1. What does the message "[ ERROR ]: Current caffe.proto does not contain field" mean? <a name="question-1"></a>
Internally, the Model Optimizer uses a protobuf library to parse and load Caffe\* models. This library requires a file grammar and a generated parser. For a Caffe fallback, the Model Optimizer uses a Caffe-generated parser for a Caffe-specific `.proto` file (which is usually located in the `src/caffe/proto` directory). So, if you have Caffe installed on your machine with Python* interface available, make sure that this is exactly the version of Caffe that was used to create the model.
Internally, Model Optimizer uses a protobuf library to parse and load Caffe models. This library requires a file grammar and a generated parser. For a Caffe fallback, Model Optimizer uses a Caffe-generated parser for a Caffe-specific `.proto` file (which is usually located in the `src/caffe/proto` directory). Make sure that you install exactly the same version of Caffe (with Python interface) as that was used to create the model.
If you just want to experiment with the Model Optimizer and test a Python extension for working with your custom
If you just want to experiment with Model Optimizer and test a Python extension for working with your custom
layers without building Caffe, add the layer description to the `caffe.proto` file and generate a parser for it.
For example, to add the description of the `CustomReshape` layer, which is an artificial layer not present in any `caffe.proto` files:
1. Add the following lines to of the `caffe.proto` file:
1. Add the following lines to the `caffe.proto` file:
```shell
package mo_caffe; // to avoid conflict with system Caffe* it is highly recommended to specify different package name
package mo_caffe; // To avoid conflict with Caffe system, it is highly recommended to specify different package name.
...
message LayerParameter {
// other layers parameters description
// Other layers parameters description.
...
optional CustomReshapeParameter custom_reshape_param= 546; // 546 - ID is any number not present in caffe.proto
optional CustomReshapeParameter custom_reshape_param= 546; // 546 - ID is any number not present in caffe.proto.
}
// these lines to end of the file - describing contents of this parameter
// The lines from here to the end of the file are describing contents of this parameter.
message CustomReshapeParameter {
optional BlobShape shape= 1; // we just use the same parameter type as some other Caffe layers
optional BlobShape shape= 1; // Just use the same parameter type as some other Caffe layers.
}
```
@@ -31,15 +31,15 @@ For example, to add the description of the `CustomReshape` layer, which is an ar
cd <SITE_PACKAGES_WITH_INSTALLED_OPENVINO>/openvino/tools/mo/front/caffe/proto
where `PATH_TO_CUSTOM_CAFFE` is the path to the root directory of custom Caffe\*.
where `PATH_TO_CUSTOM_CAFFE` is the path to the root directory of custom Caffe.
3. Now, the Model Optimizer is able to load the model into memory and start working with your extensions if there are any.
3. Now, Model Optimizer is able to load the model into memory and start working with your extensions if there are any.
However, because your model has custom layers, you must register your custom layers as custom. To learn more about it, refer to the section [Custom Layers in Model Optimizer](customize_model_optimizer/Customize_Model_Optimizer.md).
However, since your model has custom layers, you must register them as custom. To learn more about it, refer to [Custom Layers in Model Optimizer](customize_model_optimizer/Customize_Model_Optimizer.md).
#### 2. How do I create a bare caffemodel, if I have only prototxt? <a name="question-2"></a>
You need the Caffe\* Python\* interface. In this case, do the following:
You need the Caffe Python interface. In this case, do the following:
#### 3. What does the message "[ ERROR ]: Unable to create ports for node with id" mean? <a name="question-3"></a>
Most likely, the Model Optimizer does not know how to infer output shapes of some layers in the given topology.
To lessen the scope, compile the list of layers that are custom for the Model Optimizer: present in the topology,
absent in [list of supported layers](Supported_Frameworks_Layers.md) for the target framework. Then refer to available options in the corresponding section in [Custom Layers in Model Optimizer](customize_model_optimizer/Customize_Model_Optimizer.md).
Most likely, Model Optimizer does not know how to infer output shapes of some layers in the given topology.
To lessen the scope, compile the list of layers that are custom for Model Optimizer: present in the topology,
absent in the [list of supported layers](Supported_Frameworks_Layers.md) for the target framework. Then, refer to available options in the corresponding section in the [Custom Layers in Model Optimizer](customize_model_optimizer/Customize_Model_Optimizer.md) page.
#### 4. What does the message "Input image of shape is larger than mean image from file" mean? <a name="question-4"></a>
Your model input shapes must be smaller than or equal to the shapes of the mean image file you provide. The idea behind the mean file is to subtract its values from the input image in an element-wise manner. When the mean file is smaller than the input image, there are not enough values to perform element-wise subtraction. Also, make sure that you use the mean file that was used during the network training phase. Note that the mean file is dataset dependent.
Your model input shapes must be smaller than or equal to the shapes of the mean image file you provide. The idea behind the mean file is to subtract its values from the input image in an element-wise manner. When the mean file is smaller than the input image, there are not enough values to perform element-wise subtraction. Also, make sure you use the mean file that was used during the network training phase. Note that the mean file is dependent on dataset.
#### 5. What does the message "Mean file is empty" mean? <a name="question-5"></a>
Most likely, the mean file that you have is specified with `--mean_file` flag, while launching the Model Optimizer is empty. Make sure that this is exactly the required mean file and try to regenerate it from the given dataset if possible.
Most likely, the mean file specified with the `--mean_file` flag is empty while Model Optimizer is launched. Make sure that this is exactly the required mean file and try to regenerate it from the given dataset if possible.
#### 6. What does the message "Probably mean file has incorrect format" mean? <a name="question-6"></a>
The mean file that you provide for the Model Optimizer must be in a`.binaryproto` format. You can try to check the content using recommendations from the BVLC Caffe\* ([#290](https://github.com/BVLC/caffe/issues/290)).
The mean file that you provide for Model Optimizer must be in the`.binaryproto` format. You can try to check the content, using recommendations from the BVLC Caffe ([#290](https://github.com/BVLC/caffe/issues/290)).
#### 7. What does the message "Invalid proto file: there is neither 'layer' nor 'layers' top-level messages" mean? <a name="question-7"></a>
The structure of any Caffe\* topology is described in the `caffe.proto` file of any Caffe version. For example, in the Model Optimizer, you can find the following proto file, used by default: `mo/front/caffe/proto/my_caffe.proto`. There you can find the structure:
The structure of any Caffe topology is described in the `caffe.proto` file of any Caffe version. For example, the following `.proto` file in Model Optimizer is used by default: `mo/front/caffe/proto/my_caffe.proto`, with the structure:
```
message NetParameter {
// ... some other parameters
@@ -81,7 +81,7 @@ This means that any topology should contain layers as top-level structures in `p
#### 8. What does the message "Old-style inputs (via 'input_dims') are not supported. Please specify inputs via 'input_shape'" mean? <a name="question-8"></a>
The structure of any Caffe\* topology is described in the `caffe.proto` file for any Caffe version. For example, in the Model Optimizer you can find the following `.proto` file, used by default: `mo/front/caffe/proto/my_caffe.proto`. There you can find the structure:
The structure of any Caffe topology is described in the `caffe.proto` file for any Caffe version. For example, the following `.proto` file in Model Optimizer is used by default: `mo/front/caffe/proto/my_caffe.proto`, with the structure:
```sh
message NetParameter {
@@ -98,7 +98,7 @@ message NetParameter {
// ... other parameters
}
```
So, the input layer of the provided model must be specified in one of the following styles:
Therefore, the input layer of the provided model must be specified in one of the following styles:
*
```sh
@@ -154,43 +154,43 @@ input_dim: 3
input_dim: 500
```
However, if your model contains more than one input, the Model Optimizer is able to convert the model with inputs specified in a form of 1, 2, 3 of the list above. The last form is not supported for multi-input topologies.
However, if your model contains more than one input, Model Optimizer is able to convert the model with inputs specified in one of the first three forms in the above list. The 4th form is not supported for multi-input topologies.
#### 9. What does the message "Mean file for topologies with multiple inputs is not supported" mean? <a name="question-9"></a>
Model Optimizer does not support mean file processing for topologies with more than one input. In this case, you need to perform preprocessing of the inputs for a generated Intermediate Representation in the OpenVINO Runtime to perform subtraction for every input of your multi-input model, see [Overview of Preprocessing](../../OV_Runtime_UG/preprocessing_overview.md) for details.
Model Optimizer does not support mean file processing for topologies with more than one input. In this case, you need to perform preprocessing of the inputs for a generated Intermediate Representation in OpenVINO Runtime to perform subtraction for every input of your multi-input model. See the [Overview of Preprocessing](../../OV_Runtime_UG/preprocessing_overview.md) for details.
#### 10. What does the message "Cannot load or process mean file: value error" mean? <a name="question-10"></a>
There are multiple reasons why the Model Optimizer does not accept the mean file. See FAQs [#4](#question-4), [#5](#question-5), and [#6](#question-6).
There are multiple reasons why Model Optimizer does not accept the mean file. See FAQs [#4](#question-4), [#5](#question-5), and [#6](#question-6).
#### 11. What does the message "Invalid prototxt file: value error" mean? <a name="question-11"></a>
There are multiple reasons why the Model Optimizer does not accept a Caffe* topology. See FAQs [#7](#question-7) and [#20](#question-20).
There are multiple reasons why Model Optimizer does not accept a Caffe topology. See FAQs [#7](#question-7) and [#20](#question-20).
#### 12. What does the message "Error happened while constructing caffe.Net in the Caffe fallback function" mean? <a name="question-12"></a>
Model Optimizer tried to infer a specified layer via the Caffe\* framework, however it cannot construct a net using the Caffe Python* interface. Make sure that your `caffemodel` and `prototxt` files are correct. To prove that the problem is not in the `prototxt` file, see FAQ [#2](#question-2).
Model Optimizer tried to infer a specified layer via the Caffe framework. However, it cannot construct a net using the Caffe Python interface. Make sure that your `caffemodel` and `prototxt` files are correct. To ensure that the problem is not in the `prototxt` file, see FAQ [#2](#question-2).
#### 13. What does the message "Cannot infer shapes due to exception in Caffe" mean? <a name="question-13"></a>
Model Optimizer tried to infer a custom layer via the Caffe\* framework, but an error occurred, meaning that the model could not be inferred using Caffe. This might happen if you try to convert the model with some noise weights and biases that result in problems with layers that have dynamic shapes. You should write your own extension for every custom layer you topology might have. For more details, refer to [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md).
Model Optimizer tried to infer a custom layer via the Caffe framework, but the model could not be inferred using Caffe. This might happen if you try to convert the model with some noise weights and biases, which conflict with layers that have dynamic shapes. You should write your own extension for every custom layer your topology might have. For more details, refer to the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) page.
#### 14. What does the message "Cannot infer shape for node {} because there is no Caffe available. Please register python infer function for op or use Caffe for shape inference" mean? <a name="question-14"></a>
Your model contains a custom layer and you have correctly registered it with the `CustomLayersMapping.xml` file. These steps are required to offload shape inference of the custom layer with the help of the system Caffe\*. However, the Model Optimizer could not import a Caffe package. Make sure that you have built Caffe with a `pycaffe` target and added it into the `PYTHONPATH` environment variable. At the same time, it is highly recommend to avoid dependency on Caffe and write your own Model Optimizer extension for your custom layer. For more information, refer to the FAQ [#45](#question-45).
Your model contains a custom layer and you have correctly registered it with the `CustomLayersMapping.xml` file. These steps are required to offload shape inference of the custom layer with the help of the system Caffe. However, Model Optimizer could not import a Caffe package. Make sure that you have built Caffe with a `pycaffe` target and added it to the `PYTHONPATH` environment variable. At the same time, it is highly recommended to avoid dependency on Caffe and write your own Model Optimizer extension for your custom layer. For more information, refer to FAQ [#44](#question-44).
#### 15. What does the message "Framework name can not be deduced from the given options. Use --framework to choose one of Caffe, TensorFlow, MXNet" mean? <a name="question-15"></a>
You have run the Model Optimizer without a flag `--framework caffe|tf|mxnet`. Model Optimizer tries to deduce the framework by the input model file extension (`.pb` for TensorFlow\*, `.caffemodel` for Caffe\*, `.params` for MXNet\*). Your input model might have a different extension and you need to explicitly set the source framework. For example, use `--framework caffe`.
You have run Model Optimizer without a flag `--framework caffe|tf|mxnet`. Model Optimizer tries to deduce the framework by the extension of input model file (`.pb` for TensorFlow, `.caffemodel` for Caffe, `.params` for Apache MXNet). Your input model might have a different extension and you need to explicitly set the source framework. For example, use `--framework caffe`.
#### 16. What does the message "Input shape is required to convert MXNet model. Please provide it with --input_shape" mean? <a name="question-16"></a>
Input shape was not provided. That is mandatory for converting an MXNet\* model to the Intermediate Representation, because MXNet models do not contain information about input shapes. Please, use the `--input_shape` flag to specify it. For more information about using the `--input_shape`, refer to the FAQ [#57](#question-57).
Input shape was not provided. That is mandatory for converting an MXNet model to the OpenVINO Intermediate Representation, because MXNet models do not contain information about input shapes. Use the `--input_shape` flag to specify it. For more information about using the `--input_shape`, refer to FAQ [#56](#question-56).
#### 17. What does the message "Both --mean_file and mean_values are specified. Specify either mean file or mean values" mean? <a name="question-17"></a>
`--mean_file` and `--mean_values` are two ways of specifying preprocessing for the input. However, they cannot be used together, as it would mean double subtraction and lead to ambiguity. Choose one of these options and pass it using the corresponding CLI option.
The `--mean_file` and `--mean_values` options are two ways of specifying preprocessing for the input. However, they cannot be used together, as it would mean double subtraction and lead to ambiguity. Choose one of these options and pass it with the corresponding CLI option.
#### 18. What does the message "Negative value specified for --mean_file_offsets option. Please specify positive integer values in format '(x,y)'" mean? <a name="question-18"></a>
@@ -198,157 +198,157 @@ You might have specified negative values with `--mean_file_offsets`. Only positi
#### 19. What does the message "Both --scale and --scale_values are defined. Specify either scale factor or scale values per input channels" mean? <a name="question-19"></a>
`--scale` sets a scaling factor for all channels.`--scale_values` sets a scaling factor per each channel. Using both of them simultaneously produces ambiguity, so you must use only one of them. For more information, refer to the Using Framework-Agnostic Conversion Parameters: for <ahref="ConvertFromCaffe.html#using-framework-agnostic-conv-param">Converting a Caffe* Model</a>, <ahref="ConvertFromTensorFlow.html#using-framework-agnostic-conv-param">Converting a TensorFlow* Model</a>, <ahref="ConvertFromMXNet.html#using-framework-agnostic-conv-param">Converting an MXNet* Model</a>.
The `--scale` option sets a scaling factor for all channels, while`--scale_values` sets a scaling factor per each channel. Using both of them simultaneously produces ambiguity, so you must use only one of them. For more information, refer to the **Using Framework-Agnostic Conversion Parameters** section: for <ahref="ConvertFromCaffe.html#using-framework-agnostic-conv-param">Converting a Caffe Model</a>, <ahref="ConvertFromTensorFlow.html#using-framework-agnostic-conv-param">Converting a TensorFlow Model</a>, <ahref="ConvertFromMXNet.html#using-framework-agnostic-conv-param">Converting an MXNet Model</a>.
#### 20. What does the message "Cannot find prototxt file: for Caffe please specify --input_proto - a protobuf file that stores topology and --input_model that stores pre-trained weights" mean? <a name="question-20"></a>
Model Optimizer cannot find a `.prototxt` file for a specified model. By default, it must be located in the same directory as the input model with the same name (except extension). If any of these conditions is not satisfied, use `--input_proto` to specify the path to the `.prototxt` file.
#### 22. What does the message "Failed to create directory .. . Permission denied!" mean? <a name="question-22"></a>
#### 21. What does the message "Failed to create directory .. . Permission denied!" mean? <a name="question-21"></a>
Model Optimizer cannot create a directory specified via `--output_dir`. Make sure that you have enough permissions to create the specified directory.
#### 23. What does the message "Discovered data node without inputs and value" mean? <a name="question-23"></a>
#### 22. What does the message "Discovered data node without inputs and value" mean? <a name="question-22"></a>
One of the layers in the specified topology might not have inputs or values. Please make sure that the provided `caffemodel` and `protobuf` files are correct.
One of the layers in the specified topology might not have inputs or values. Make sure that the provided `caffemodel` and `protobuf` files are correct.
#### 24. What does the message "Part of the nodes was not translated to IE. Stopped" mean? <a name="question-24"></a>
#### 23. What does the message "Part of the nodes was not translated to IE. Stopped" mean? <a name="question-23"></a>
Some of the operations are not supported by the OpenVINO Runtime and cannot be translated to an Intermediate Representation. You can extend the Model Optimizer by allowing generation of new types of operations and implement these operations in the dedicated OpenVINO plugins. For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md)
Some of the operations are not supported by OpenVINO Runtime and cannot be translated to OpenVINO Intermediate Representation. You can extend Model Optimizer by allowing generation of new types of operations and implement these operations in the dedicated OpenVINO plugins. For more information, refer to the [OpenVINO Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 25. What does the message "While creating an edge from .. to .. : node name is undefined in the graph. Check correctness of the input model" mean? <a name="question-25"></a>
#### 24. What does the message "While creating an edge from .. to .. : node name is undefined in the graph. Check correctness of the input model" mean? <a name="question-24"></a>
Model Optimizer cannot build a graph based on a specified model. Most likely, it is incorrect.
#### 26. What does the message "Node does not exist in the graph" mean? <a name="question-26"></a>
#### 25. What does the message "Node does not exist in the graph" mean? <a name="question-25"></a>
You might have specified an output node via the `--output` flag that does not exist in a provided model. Make sure that the specified output is correct and this node exists in the current model.
#### 27. What does the message "--input parameter was provided. Other inputs are needed for output computation. Provide more inputs or choose another place to cut the net" mean? <a name="question-27"></a>
#### 26. What does the message "--input parameter was provided. Other inputs are needed for output computation. Provide more inputs or choose another place to cut the net" mean? <a name="question-26"></a>
Most likely, the Model Optimizer tried to cut the model by a specified input. However, other inputs are needed.
Most likely, Model Optimizer tried to cut the model by a specified input. However, other inputs are needed.
#### 28. What does the message "Placeholder node does not have an input port, but input port was provided" mean? <a name="question-28"></a>
#### 27. What does the message "Placeholder node does not have an input port, but input port was provided" mean? <a name="question-27"></a>
You might have specified a placeholder node with an input node, while the placeholder node does not have it the model.
You might have specified a placeholder node with an input node, while the placeholder node does not have it in the model.
#### 29. What does the message "Port index is out of number of available input ports for node" mean? <a name="question-29"></a>
#### 28. What does the message "Port index is out of number of available input ports for node" mean? <a name="question-28"></a>
This error occurs when an incorrect input port is specified with the `--input` command line argument. When using `--input`, you can optionally specify an input port in the form: `X:node_name`, where `X` is an integer index of the input port starting from 0 and `node_name` is the name of a node in the model. This error occurs when the specified input port `X` is not in the range 0..(n-1), where n is the number of input ports for the node. Please, specify a correct port index, or do not use it if it is not needed.
This error occurs when an incorrect input port is specified with the `--input` command line argument. When using `--input`, you may optionally specify an input port in the form: `X:node_name`, where `X` is an integer index of the input port starting from 0 and `node_name` is the name of a node in the model. This error occurs when the specified input port `X` is not in the range 0..(n-1), where n is the number of input ports for the node. Specify a correct port index, or do not use it if it is not needed.
#### 30. What does the message "Node has more than 1 input and input shapes were provided. Try not to provide input shapes or specify input port with PORT:NODE notation, where PORT is an integer" mean? <a name="question-30"></a>
#### 29. What does the message "Node has more than 1 input and input shapes were provided. Try not to provide input shapes or specify input port with PORT:NODE notation, where PORT is an integer" mean? <a name="question-29"></a>
This error occurs when an incorrect combination of the `--input` and `--input_shape` command line options is used. Using both `--input` and `--input_shape` is valid only if `--input` points to the `Placeholder` node, a node with one input port or `--input` has the form `PORT:NODE`, where `PORT` is an integer port index of input for node `NODE`. Otherwise, the combination of `--input` and `--input_shape` is incorrect.
#### 31. What does the message "Input port > 0 in --input is not supported if --input_shape is not provided. Node: NAME_OF_THE_NODE. Omit port index and all input ports will be replaced by placeholders. Or provide --input_shape" mean? <a name="question-31"></a>
#### 30. What does the message "Input port > 0 in --input is not supported if --input_shape is not provided. Node: NAME_OF_THE_NODE. Omit port index and all input ports will be replaced by placeholders. Or provide --input_shape" mean? <a name="question-30"></a>
When using the `PORT:NODE` notation for the `--input` command line argument and `PORT` > 0, you should specify `--input_shape` for this input. This is a limitation of the current Model Optimizer implementation.
**NOTE**: It is no longer relevant message since the limitation on input port index for model truncation has been resolved.
#### 32. What does the message "No or multiple placeholders in the model, but only one shape is provided, cannot set it" mean? <a name="question-32"></a>
#### 31. What does the message "No or multiple placeholders in the model, but only one shape is provided, cannot set it" mean? <a name="question-31"></a>
Looks like you have provided only one shape for the placeholder, however there are no or multiple inputs in the model. Please, make sure that you have provided correct data for placeholder nodes.
You might have provided only one shape for the placeholder, while there are none or multiple inputs in the model. Make sure that you have provided the correct data for placeholder nodes.
#### 33. What does the message "The amount of input nodes for port is not equal to 1" mean? <a name="question-33"></a>
#### 32. What does the message "The amount of input nodes for port is not equal to 1" mean? <a name="question-32"></a>
This error occurs when the `SubgraphMatch.single_input_node` function is used for an input port that supplies more than one node in a sub-graph. The `single_input_node` function can be used only for ports that has a single consumer inside the matching sub-graph. When multiple nodes are connected to the port, use the `input_nodes` function or `node_by_pattern` function instead of `single_input_node`. Please, refer to **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) documentation for more details.
This error occurs when the `SubgraphMatch.single_input_node` function is used for an input port that supplies more than one node in a sub-graph. The `single_input_node` function can be used only for ports that has a single consumer inside the matching sub-graph. When multiple nodes are connected to the port, use the `input_nodes` function or `node_by_pattern` function instead of `single_input_node`. For more details, refer to the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 34. What does the message "Output node for port has already been specified" mean? <a name="question-34"></a>
#### 33. What does the message "Output node for port has already been specified" mean? <a name="question-33"></a>
This error occurs when the `SubgraphMatch._add_output_node` function is called manually from user's extension code. This is an internal function, and you should not call it directly.
#### 35. What does the message "Unsupported match kind.... Match kinds "points" or "scope" are supported only" mean? <a name="question-35"></a>
#### 34. What does the message "Unsupported match kind.... Match kinds "points" or "scope" are supported only" mean? <a name="question-34"></a>
While using configuration file to implement a TensorFlow\* front replacement extension, an incorrect match kind was used. Only `points` or `scope` match kinds are supported. Please refer to [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) for more details.
While using configuration file to implement a TensorFlow front replacement extension, an incorrect match kind was used. Only `points` or `scope` match kinds are supported. For more details, refer to the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 36. What does the message "Cannot write an event file for the TensorBoard to directory" mean? <a name="question-36"></a>
#### 35. What does the message "Cannot write an event file for the TensorBoard to directory" mean? <a name="question-35"></a>
Model Optimizer tried to write an event file in the specified directory but failed to do that. That could happen because the specified directory does not exist or you do not have enough permissions to write in it.
Model Optimizer tried to write an event file in the specified directory but failed to do that. That could happen when the specified directory does not exist or you do not have permissions to write in it.
#### 37. What does the message "There is no registered 'infer' function for node with op = .. . Please implement this function in the extensions" mean? <a name="question-37"></a>
#### 36. What does the message "There is no registered 'infer' function for node with op = .. . Please implement this function in the extensions" mean? <a name="question-36"></a>
Most likely, you tried to extend Model Optimizer with a new primitive, but did not specify an infer function. For more information on extensions, see [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
Most likely, you tried to extend Model Optimizer with a new primitive, but you did not specify an infer function. For more information on extensions, see the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 38. What does the message "Stopped shape/value propagation at node" mean? <a name="question-38"></a>
#### 37. What does the message "Stopped shape/value propagation at node" mean? <a name="question-37"></a>
Model Optimizer cannot infer shapes or values for the specified node. It can happen because of a bug in the custom shape infer function, because the node inputs have incorrect values/shapes, or because the input shapes are incorrect.
Model Optimizer cannot infer shapes or values for the specified node. It can happen because of the following reasons: a bug exists in the custom shape infer function, the node inputs have incorrect values/shapes, or the input shapes are incorrect.
#### 39. What does the message "The input with shape .. does not have the batch dimension" mean? <a name="question-39"></a>
#### 38. What does the message "The input with shape .. does not have the batch dimension" mean? <a name="question-38"></a>
Batch dimension is the first dimension in the shape and it should be equal to 1 or undefined. In your case, it is not equal to either 1 or undefined, which is why the `-b` shortcut produces undefined and unspecified behavior. To resolve the issue, specify full shapes for each input with the `--input_shape` option. Run Model Optimizer with the `--help` option to learn more about the notation for input shapes.
Batch dimension is the first dimension in the shape and it should be equal to 1 or undefined. In your case, it is not either equal to 1 or undefined, which is why the `-b` shortcut produces undefined and unspecified behavior. To resolve the issue, specify full shapes for each input with the `--input_shape` option. Run Model Optimizer with the `--help` option to learn more about the notation for input shapes.
#### 40. What does the message "Not all output shapes were inferred or fully defined for node" mean? <a name="question-40"></a>
#### 39. What does the message "Not all output shapes were inferred or fully defined for node" mean? <a name="question-39"></a>
Most likely, the shape is not defined (partially or fully) for the specified node. You can use `--input_shape` with positive integers to override model input shapes.
#### 41. What does the message "Shape for tensor is not defined. Can not proceed" mean? <a name="question-41"></a>
#### 40. What does the message "Shape for tensor is not defined. Can not proceed" mean? <a name="question-40"></a>
This error occurs when the `--input` commandline option is used to cut a model and `--input_shape` is not used to override shapes for a node and a shape for the node cannot be inferred by Model Optimizer. You need to help Model Optimizer and specify shapes with `--input_shape` for each node that is specified with the `--input` commandline option.
This error occurs when the `--input` command-line option is used to cut a model and `--input_shape` is not used to override shapes for a node, so a shape for the node cannot be inferred by Model Optimizer. You need to help Model Optimizer by specifying shapes with `--input_shape` for each node specified with the `--input` command-line option.
#### 42. What does the message "Module TensorFlow was not found. Please install TensorFlow 1.2 or higher" mean? <a name="question-42"></a>
#### 41. What does the message "Module TensorFlow was not found. Please install TensorFlow 1.2 or higher" mean? <a name="question-41"></a>
To convert TensorFlow\* models with Model Optimizer, TensorFlow 1.2 or newer must be installed. For more information on prerequisites, see [Configuring the Model Optimizer](../Deep_Learning_Model_Optimizer_DevGuide.md).
To convert TensorFlow models with Model Optimizer, TensorFlow 1.2 or newer must be installed. For more information on prerequisites, see the [Configuring Model Optimizer](../Deep_Learning_Model_Optimizer_DevGuide.md) guide.
#### 43. What does the message "Cannot read the model file: it is incorrect TensorFlow model file or missing" mean? <a name="question-43"></a>
#### 42. What does the message "Cannot read the model file: it is incorrect TensorFlow model file or missing" mean? <a name="question-42"></a>
The model file should contain a frozen TensorFlow\* graph in the text or binary format. Make sure that `--input_model_is_text` is provided for a model in the text format. By default, a model is interpreted as binary file.
The model file should contain a frozen TensorFlow graph in the text or binary format. Make sure that `--input_model_is_text` is provided for a model in the text format. By default, a model is interpreted as binary file.
#### 44. What does the message "Cannot pre-process TensorFlow graph after reading from model file. File is corrupt or has unsupported format" mean? <a name="question-44"></a>
#### 43. What does the message "Cannot pre-process TensorFlow graph after reading from model file. File is corrupt or has unsupported format" mean? <a name="question-43"></a>
Most likely, there is a problem with the specified file for model. The file exists, but it has bad formatting or is corrupted.
Most likely, there is a problem with the specified file for the model. The file exists, but it has an invalid format or is corrupted.
#### 45. What does the message "Found custom layer. Model Optimizer does not support this layer. Please, register it in CustomLayersMapping.xml or implement extension" mean? <a name="question-45"></a>
#### 44. What does the message "Found custom layer. Model Optimizer does not support this layer. Please, register it in CustomLayersMapping.xml or implement extension" mean? <a name="question-44"></a>
This means that the layer `{layer_name}` is not supported in the Model Optimizer. You can find a list of all unsupported layers in the corresponding section. You should implement the extensions for this layer ([OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md)).
This means that the layer `{layer_name}` is not supported in Model Optimizer. You will find a list of all unsupported layers in the corresponding section. You should implement the extensions for this layer. See [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) for more information.
#### 46. What does the message "Custom replacement configuration file does not exist" mean? <a name="question-46"></a>
#### 45. What does the message "Custom replacement configuration file does not exist" mean? <a name="question-45"></a>
Path to the custom replacement configuration file was provided with the `--transformations_config` flag, but the file could not be found. Please, make sure that the specified path is correct and the file exists.
A path to the custom replacement configuration file was provided with the `--transformations_config` flag, but the file could not be found. Make sure the specified path is correct and the file exists.
#### 47. What does the message "Extractors collection have case insensitive duplicates" mean? <a name="question-47"></a>
#### 46. What does the message "Extractors collection have case insensitive duplicates" mean? <a name="question-46"></a>
When extending Model Optimizer with new primitives keep in mind that their names are caseinsensitive. Most likely, another operation with the same name is already defined. For more information, see [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
When extending Model Optimizer with new primitives, keep in mind that their names are case-insensitive. Most likely, another operation with the same name is already defined. For more information, see the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 48. What does the message "Input model name is not in an expected format, cannot extract iteration number" mean? <a name="question-48"></a>
#### 47. What does the message "Input model name is not in an expected format, cannot extract iteration number" mean? <a name="question-47"></a>
Model Optimizer cannot load an MXNet\* model in the specified file format. Please, use the `.json` or `.param` format.
Model Optimizer cannot load an MXNet model in the specified file format. Make sure you use the `.json` or `.param` format.
#### 49. What does the message "Cannot convert type of placeholder because not all of its outputs are 'Cast' to float operations" mean? <a name="question-49"></a>
#### 48. What does the message "Cannot convert type of placeholder because not all of its outputs are 'Cast' to float operations" mean? <a name="question-48"></a>
There are models where `Placeholder` has the UINT8 type and the first operation after it is 'Cast', which casts the input to FP32. Model Optimizer detected that the `Placeholder` has the UINT8 type, but the next operation is not 'Cast' to float. Model Optimizer does not support such a case. Please, change the model to have placeholder FP32 data type.
There are models where `Placeholder` has the UINT8 type and the first operation after it is 'Cast', which casts the input to FP32. Model Optimizer detected that the `Placeholder` has the UINT8 type, but the next operation is not 'Cast' to float. Model Optimizer does not support such a case. Make sure you change the model to have `Placeholder` for FP32.
#### 50. What does the message "Data type is unsupported" mean? <a name="question-50"></a>
#### 49. What does the message "Data type is unsupported" mean? <a name="question-49"></a>
Model Optimizer cannot convert the model to the specified data type. Currently, FP16 and FP32 are supported. Please, specify the data type with the `--data_type` flag. The available values are: FP16, FP32, half, float.
Model Optimizer cannot convert the model to the specified data type. Currently, FP16 and FP32 are supported. Make sure you specify the data type with the `--data_type` flag. The available values are: FP16, FP32, half, float.
#### 51. What does the message "No node with name ..." mean? <a name="question-51"></a>
#### 50. What does the message "No node with name ..." mean? <a name="question-50"></a>
Model Optimizer tried to access a node that does not exist. This could happen if you have incorrectly specified placeholder, input or output node name.
#### 52. What does the message "Module mxnet was not found. Please install MXNet 1.0.0" mean? <a name="question-52"></a>
#### 51. What does the message "Module MXNet was not found. Please install MXNet 1.0.0" mean? <a name="question-51"></a>
To convert MXNet\* models with Model Optimizer, MXNet 1.0.0 must be installed. For more information about prerequisites, see [Configuring the Model Optimizer](../Deep_Learning_Model_Optimizer_DevGuide.md).
To convert MXNet models with Model Optimizer, Apache MXNet 1.0.0 must be installed. For more information about prerequisites, see the[Configuring Model Optimizer](../Deep_Learning_Model_Optimizer_DevGuide.md) guide.
#### 53. What does the message "The following error happened while loading MXNet model .." mean? <a name="question-53"></a>
#### 52. What does the message "The following error happened while loading MXNet model .." mean? <a name="question-52"></a>
Most likely, there is a problem with loading of the MXNet\* model. Please, make sure that the specified path is correct, the model exists, it is not corrupted, and you have sufficient permissions to work with it.
Most likely, there is a problem with loading of the MXNet model. Make sure the specified path is correct, the model exists and is not corrupted, and you have sufficient permissions to work with it.
#### 54. What does the message "The following error happened while processing input shapes: .." mean? <a name="question-54"></a>
#### 53. What does the message "The following error happened while processing input shapes: .." mean? <a name="question-53"></a>
Please, make sure that inputs are defined and have correct shapes. You can use `--input_shape` with positive integers to override model input shapes.
Make sure inputs are defined and have correct shapes. You can use `--input_shape` with positive integers to override model input shapes.
#### 55. What does the message "Attempt to register of custom name for the second time as class. Note that custom names are case-insensitive" mean? <a name="question-55"></a>
#### 54. What does the message "Attempt to register of custom name for the second time as class. Note that custom names are case-insensitive" mean? <a name="question-54"></a>
When extending Model Optimizer with new primitives keep in mind that their names are caseinsensitive. Most likely, another operation with the same name is already defined. For more information, see [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
When extending Model Optimizer with new primitives, keep in mind that their names are case-insensitive. Most likely, another operation with the same name is already defined. For more information, see the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 56. What does the message "Both --input_shape and --batch were provided. Please, provide only one of them" mean? <a name="question-56"></a>
#### 55. What does the message "Both --input_shape and --batch were provided. Please, provide only one of them" mean? <a name="question-55"></a>
You cannot specify the batch and the input shape at the same time. You should specify a desired batch as the first value of the input shape.
Specifying the batch and the input shapes at the same time is not supported. You must specify a desired batch as the first value of the input shape.
#### 57. What does the message "Input shape .. cannot be parsed" mean? <a name="question-57"></a>
#### 56. What does the message "Input shape .. cannot be parsed" mean? <a name="question-56"></a>
The specified input shape cannot be parsed. Please, define it in one of the following ways:
The specified input shape cannot be parsed. Define it in one of the following ways:
*
```shell
@@ -365,141 +365,141 @@ The specified input shape cannot be parsed. Please, define it in one of the foll
Keep in mind that there is no space between and inside the brackets for input shapes.
#### 58. What does the message "Please provide input layer names for input layer shapes" mean? <a name="question-58"></a>
#### 57. What does the message "Please provide input layer names for input layer shapes" mean? <a name="question-57"></a>
When specifying input shapes for several layers, you must provide names for inputs, whose shapes will be overwritten. For usage examples, see [Converting a Caffe* Model](convert_model/Convert_Model_From_Caffe.md. Additional information for `--input_shape` is in FAQ [#57](#question-57).
When specifying input shapes for several layers, you must provide names for inputs, whose shapes will be overwritten. For usage examples, see the [Converting a Caffe Model](convert_model/Convert_Model_From_Caffe.md). Additional information for `--input_shape` is in FAQ [#56](#question-56).
#### 59. What does the message "Values cannot be parsed" mean? <a name="question-59"></a>
#### 58. What does the message "Values cannot be parsed" mean? <a name="question-58"></a>
Mean values for the given parameter cannot be parsed. It should be a string with a list of mean values. For example, in '(1,2,3)', 1 stands for the RED channel, 2 for the GREEN channel, 3 for the BLUE channel.
#### 60. What does the message ".. channels are expected for given values" mean? <a name="question-60"></a>
#### 59. What does the message ".. channels are expected for given values" mean? <a name="question-59"></a>
The number of channels and the number of given values for mean values do not match. The shape should be defined as '(R,G,B)' or '[R,G,B]'. The shape should not contain undefined dimensions (? or -1). The order of values is as follows: (value for a RED channel, value for a GREEN channel, value for a BLUE channel).
#### 61. What does the message "You should specify input for each mean value" mean? <a name="question-61"></a>
#### 60. What does the message "You should specify input for each mean value" mean? <a name="question-60"></a>
Most likely, you have not specified inputs using `--mean_values`. Please, specify inputs with the `--input` flag. For usage examples, please, refer to FAQ [#63](#question-63).
Most likely, you didn't specify inputs using `--mean_values`. Specify inputs with the `--input` flag. For usage examples, refer to the FAQ [#62](#question-62).
#### 62. What does the message "You should specify input for each scale value" mean? <a name="question-62"></a>
#### 61. What does the message "You should specify input for each scale value" mean? <a name="question-61"></a>
Most likely, you have not specified inputs using `--scale_values`. Please, specify inputs with the `--input` flag. For usage examples, please, refer to FAQ [#64](#question-64).
Most likely, you didn't specify inputs using `--scale_values`. Specify inputs with the `--input` flag. For usage examples, refer to the FAQ [#63](#question-63).
#### 63. What does the message "Number of inputs and mean values does not match" mean? <a name="question-63"></a>
#### 62. What does the message "Number of inputs and mean values does not match" mean? <a name="question-62"></a>
The number of specified mean values and the number of inputs must be equal. Please, refer to [Converting a Caffe* Model](convert_model/Convert_Model_From_Caffe.md) for a usage example.
The number of specified mean values and the number of inputs must be equal. For a usage example, refer to the [Converting a Caffe Model](convert_model/Convert_Model_From_Caffe.md) guide.
#### 64. What does the message "Number of inputs and scale values does not match" mean? <a name="question-64"></a>
#### 63. What does the message "Number of inputs and scale values does not match" mean? <a name="question-63"></a>
The number of specified scale values and the number of inputs must be equal. Please, refer to [Converting a Caffe* Model](convert_model/Convert_Model_From_Caffe.md) for a usage example.
The number of specified scale values and the number of inputs must be equal. For a usage example, refer to the [Converting a Caffe Model](convert_model/Convert_Model_From_Caffe.md) guide.
#### 65. What does the message "No class registered for match kind ... Supported match kinds are .. " mean? <a name="question-65"></a>
#### 64. What does the message "No class registered for match kind ... Supported match kinds are .. " mean? <a name="question-64"></a>
A replacement defined in the configuration file for sub-graph replacement using node names patterns or start/end nodes has the `match_kind` attribute. The attribute may have only one of the values: `scope` or `points`. If a different value is provided, this error is displayed.
A replacement defined in the configuration file for sub-graph replacement, using node names patterns or start/end nodes, has the `match_kind` attribute. The attribute may have only one of the values: `scope` or `points`. If a different value is provided, this error is displayed.
#### 66. What does the message "No instance(s) is(are) defined for the custom replacement" mean? <a name="question-66"></a>
#### 65. What does the message "No instance(s) is(are) defined for the custom replacement" mean? <a name="question-65"></a>
A replacement defined in the configuration file for sub-graph replacement using node names patterns or start/end nodes has the `instances` attribute. This attribute is mandatory, and it causes this error if it is missing. Refer to documentation with a description of the sub-graph replacement feature.
A replacement defined in the configuration file for sub-graph replacement, using node names patterns or start/end nodes, has the `instances` attribute. This attribute is mandatory. This error will occur if the attribute is missing. For more details, refer to the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 67. What does the message "The instance must be a single dictionary for the custom replacement with id .." mean? <a name="question-67"></a>
#### 66. What does the message "The instance must be a single dictionary for the custom replacement with id .." mean? <a name="question-66"></a>
A replacement defined in the configuration file for sub-graph replacement using start/end nodes has the `instances` attribute. For this type of replacement, the instance must be defined with a dictionary with two keys `start_points` and `end_points`. Values for these keys are lists with the start and end node names, respectively. Refer to documentation with a description of the sub-graph replacement feature.
A replacement defined in the configuration file for sub-graph replacement, using start/end nodes, has the `instances` attribute. For this type of replacement, the instance must be defined with a dictionary with two keys `start_points` and `end_points`. Values for these keys are lists with the start and end node names, respectively. For more details, refer to the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 68. What does the message "No instances are defined for replacement with id .. " mean? <a name="question-68"></a>
#### 67. What does the message "No instances are defined for replacement with id .. " mean? <a name="question-67"></a>
A replacement for the specified id is not defined in the configuration file. Please, refer to FAQ [#66](#question-66) for more information.
A replacement for the specified id is not defined in the configuration file. For more information, refer to the FAQ [#65](#question-65).
#### 69. What does the message "Custom replacements configuration file .. does not exist" mean? <a name="question-69"></a>
#### 68. What does the message "Custom replacements configuration file .. does not exist" mean? <a name="question-68"></a>
Path to a custom replacement configuration file was provided with the `--transformations_config` flag, but it cannot be found. Please, make sure that the specified path is correct and the file exists.
The path to a custom replacement configuration file was provided with the `--transformations_config` flag, but it cannot be found. Make sure the specified path is correct and the file exists.
#### 70. What does the message "Failed to parse custom replacements configuration file .." mean? <a name="question-70"></a>
#### 69. What does the message "Failed to parse custom replacements configuration file .." mean? <a name="question-69"></a>
The file for custom replacement configuration provided with the `--transformations_config` flag cannot be parsed. In particular, it should have a valid JSON structure. For more details, refer to [JSON Schema Reference](https://spacetelescope.github.io/understanding-json-schema/reference/index.html).
The file for custom replacement configuration provided with the `--transformations_config` flag cannot be parsed. In particular, it should have a valid JSON structure. For more details, refer to the [JSON Schema Reference](https://spacetelescope.github.io/understanding-json-schema/reference/index.html) page.
#### 71. What does the message "One of the custom replacements in the configuration file .. does not contain attribute 'id'" mean? <a name="question-71"></a>
#### 70. What does the message "One of the custom replacements in the configuration file .. does not contain attribute 'id'" mean? <a name="question-70"></a>
Every custom replacement should declare a set of mandatory attributes and their values. For more details, refer to FAQ [#72](#question-72).
Every custom replacement should declare a set of mandatory attributes and their values. For more details, refer to FAQ [#71](#question-71).
#### 72. What does the message "File .. validation failed" mean? <a name="question-72"></a>
#### 71. What does the message "File .. validation failed" mean? <a name="question-71"></a>
The file for custom replacement configuration provided with the `--transformations_config` flag cannot pass validation. Make sure that you have specified `id`, `instances` and `match_kind` for all the patterns.
The file for custom replacement configuration provided with the `--transformations_config` flag cannot pass validation. Make sure you have specified `id`, `instances`, and `match_kind` for all the patterns.
#### 73. What does the message "Cannot update the file .. because it is broken" mean? <a name="question-73"></a>
#### 72. What does the message "Cannot update the file .. because it is broken" mean? <a name="question-72"></a>
The custom replacement configuration file provided with the `--tensorflow_custom_operations_config_update` cannot be parsed. Please, make sure that the file is correct and refer to FAQs [#69](#question-69), [#70](#question-70), [#71](#question-71), and [#72](#question-72).
The custom replacement configuration file provided with the `--tensorflow_custom_operations_config_update` cannot be parsed. Make sure that the file is correct and refer to FAQ [#68](#question-68), [#69](#question-69), [#70](#question-70), and [#71](#question-71).
#### 74. What does the message "End node .. is not reachable from start nodes: .." mean? <a name="question-74"></a>
#### 73. What does the message "End node .. is not reachable from start nodes: .." mean? <a name="question-73"></a>
This error occurs when you try to make a sub-graph match. It is detected that between the start and end nodes that were specified as inputs/outputs of the subgraph to find, there are nodes that are marked as outputs but there is no path from them to the input nodes. Make sure that the subgraph you want to match does actually contain all the specified output nodes.
This error occurs when you try to make a sub-graph match. It is detected that between the start and end nodes that were specified as inputs/outputs for the subgraph to find, there are nodes marked as outputs but there is no path from them to the input nodes. Make sure the subgraph you want to match does actually contain all the specified output nodes.
#### 75. What does the message "Sub-graph contains network input node .." mean? <a name="question-75"></a>
#### 74. What does the message "Sub-graph contains network input node .." mean? <a name="question-74"></a>
Start or end node for the sub-graph replacement using start/end nodes is specified incorrectly. Model Optimizer finds internal nodes of the sub-graph strictly "between" the start and end nodes. Then it adds all input nodes to the sub-graph (and inputs of their inputs and so on) for these "internal" nodes. The error reports, that the Model Optimizer reached input node during this phase. This means that the start/end points are specified incorrectly in the configuration file. Refer to documentation with a description of the sub-graph replacement feature.
The start or end node for the sub-graph replacement using start/end nodes is specified incorrectly. Model Optimizer finds internal nodes of the sub-graph strictly "between" the start and end nodes, and then adds all input nodes to the sub-graph (and the inputs of their inputs, etc.) for these "internal" nodes. This error reports that Model Optimizer reached input node during this phase. This means that the start/end points are specified incorrectly in the configuration file. For more details, refer to the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 76. What does the message "... elements of ... were clipped to infinity while converting a blob for node [...] to ..." mean? <a name="question-76"></a>
#### 75. What does the message "... elements of ... were clipped to infinity while converting a blob for node [...] to ..." mean? <a name="question-75"></a>
This message may appear when the `--data_type=FP16` commandline option is used. This option implies conversion of all the blobs in the node to FP16. If a value in a blob is out of the range of valid FP16 values, the value is converted to positive or negative infinity. It may lead to incorrect results of inference or may not be a problem, depending on the model. The number of such elements and the total number of elements in the blob is printed out together with the name of the node, where this blob is used.
This message may appear when the `--data_type=FP16` command-line option is used. This option implies conversion of all the blobs in the node to FP16. If a value in a blob is out of the range of valid FP16 values, the value is converted to positive or negative infinity. It may lead to incorrect results of inference or may not be a problem, depending on the model. The number of such elements and the total number of elements in the blob is printed out together with the name of the node, where this blob is used.
#### 77. What does the message "... elements of ... were clipped to zero while converting a blob for node [...] to ..." mean? <a name="question-77"></a>
#### 76. What does the message "... elements of ... were clipped to zero while converting a blob for node [...] to ..." mean? <a name="question-76"></a>
This message may appear when the `--data_type=FP16` commandline option is used. This option implies conversion of all blobs in the mode to FP16. If a value in the blob is so close to zero that it cannot be represented as a valid FP16 value, it is converted to a true zero FP16 value. Depending on the model, it may lead to incorrect results of inference or may not be a problem. The number of such elements and the total number of elements in the blob are printed out together with a name of the node, where this blob is used.
This message may appear when the `--data_type=FP16` command-line option is used. This option implies conversion of all blobs in the mode to FP16. If a value in the blob is so close to zero that it cannot be represented as a valid FP16 value, it is converted to a true zero FP16 value. Depending on the model, it may lead to incorrect results of inference or may not be a problem. The number of such elements and the total number of elements in the blob are printed out together with a name of the node, where this blob is used.
#### 78. What does the message "The amount of nodes matched pattern ... is not equal to 1" mean? <a name="question-78"></a>
#### 77. What does the message "The amount of nodes matched pattern ... is not equal to 1" mean? <a name="question-77"></a>
This error occurs when the `SubgraphMatch.node_by_pattern` function is used with a pattern that does not uniquely identify a single node in a sub-graph. Try to extend the pattern string to make unambiguous match to a single sub-graph node. For more details, refer to **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) documentation.
This error occurs when the `SubgraphMatch.node_by_pattern` function is used with a pattern that does not uniquely identify a single node in a sub-graph. Try to extend the pattern string to make unambiguous match to a single sub-graph node. For more details, refer to the **Graph Transformation Extensions** section in the [Model Optimizer Extensibility](customize_model_optimizer/Customize_Model_Optimizer.md) guide.
#### 79. What does the message "The topology contains no "input" layers" mean? <a name="question-79"></a>
#### 78. What does the message "The topology contains no "input" layers" mean? <a name="question-78"></a>
Your Caffe\* topology `.prototxt` file is intended for training. Model Optimizer expects a deployment-ready `.prototxt` file. To fix the problem, prepare a deployment-ready `.prototxt` file. Usually, preparation of a deploy-ready topology results in removing `data` layer(s), adding `input` layer(s), and removing loss layer(s).
Your Caffe topology `.prototxt` file is intended for training. Model Optimizer expects a deployment-ready `.prototxt` file. To fix the problem, prepare a deployment-ready `.prototxt` file. Preparation of a deploy-ready topology usually results in removing `data` layer(s), adding `input` layer(s), and removing loss layer(s).
#### 80. What does the message "Warning: please expect that Model Optimizer conversion might be slow" mean? <a name="question-80"></a>
#### 79. What does the message "Warning: please expect that Model Optimizer conversion might be slow" mean? <a name="question-79"></a>
You are using an unsupported Python\* version. Use only versions 3.4 - 3.6 for the C++ `protobuf` implementation that is supplied with the OpenVINO Toolkit. You can still boost conversion speed by building protobuf library from sources. For complete instructions about building `protobuf` from sources, see the appropriate section in [Converting a Model to Intermediate Representation](../Deep_Learning_Model_Optimizer_DevGuide.md).
You are using an unsupported Python version. Use only versions 3.4 - 3.6 for the C++ `protobuf` implementation that is supplied with OpenVINO toolkit. You can still boost the conversion speed by building the protobuf library from sources. For complete instructions about building `protobuf` from sources, see the appropriate section in the[Converting a Model to Intermediate Representation](../Deep_Learning_Model_Optimizer_DevGuide.md) guide.
#### 81. What does the message "Arguments --nd_prefix_name, --pretrained_model_name and --input_symbol should be provided. Please provide all or do not use any." mean? <a name="question-81"></a>
#### 80. What does the message "Arguments --nd_prefix_name, --pretrained_model_name and --input_symbol should be provided. Please provide all or do not use any." mean? <a name="question-80"></a>
This error occurs if you do not provide `--nd_prefix_name`, `--pretrained_model_name` and `--input_symbol` parameters.
Model Optimizer requires both `.params` and `.nd` model files to merge into the result file (`.params`). Topology
description (`.json` file) should be prepared (merged) in advance and provided with `--input_symbol` parameter.
This error occurs if you did not provide the `--nd_prefix_name`, `--pretrained_model_name`, and `--input_symbol` parameters.
Model Optimizer requires both `.params` and `.nd` model files to merge into the result file (`.params`).
Topology description (`.json` file) should be prepared (merged) in advance and provided with the `--input_symbol` parameter.
If you add to your model additional layers and weights that are in `.nd` files, the Model Optimizer can build a model
If you add additional layers and weights that are in `.nd` files to your model, Model Optimizer can build a model
from one `.params` file and two additional `.nd` files (`*_args.nd`, `*_auxs.nd`).
To do that, provide both CLI options or do not pass them if you want to convert an MXNet model without additional weights.
For more information, refer to [Converting a MXNet* Model](convert_model/Convert_Model_From_MxNet.md).
For more information, refer to the [Converting an MXNet Model](convert_model/Convert_Model_From_MxNet.md) guide.
#### 82. What does the message "You should specify input for mean/scale values" mean? <a name="question-82"></a>
#### 81. What does the message "You should specify input for mean/scale values" mean? <a name="question-81"></a>
In case when the model has multiple inputs and you want to provide mean/scale values, you need to pass those values for each input. More specifically, a number of passed values should be the same as the number of inputs of the model.
For more information, refer to [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md).
When the model has multiple inputs and you want to provide mean/scale values, you need to pass those values for each input. More specifically, the number of passed values should be the same as the number of inputs of the model.
For more information, refer to the [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md) guide.
#### 83. What does the message "Input with name ... not found!" mean? <a name="question-83"></a>
#### 82. What does the message "Input with name ... not found!" mean? <a name="question-82"></a>
When you passed the mean/scale values and specify names of input layers of the model, you might have used the name that does not correspond to any input layer. Make sure that by passing values with `--input` option, you list only names of the input layers of your model.
For more information, refer to the [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md).
When you passed the mean/scale values and specify names of input layers of the model, you might have used the name that does not correspond to any input layer. Make sure that you list only names of the input layers of your model when passing values with the `--input` option.
For more information, refer to the [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md) guide.
#### 84. What does the message "Specified input json ... does not exist" mean? <a name="question-84"></a>
#### 83. What does the message "Specified input json ... does not exist" mean? <a name="question-83"></a>
Most likely, `.json` file does not exist or has a name that does not match the notation of MXNet. Make sure that the file exists and it has a correct name.
For more information, refer to [Converting a MXNet\* Model](convert_model/Convert_Model_From_MxNet.md).
Most likely, `.json` file does not exist or has a name that does not match the notation of Apache MXNet. Make sure the file exists and has a correct name.
For more information, refer to the [Converting an MXNet Model](convert_model/Convert_Model_From_MxNet.md) guide.
#### 85. What does the message "Unsupported Input model file type ... Model Optimizer support only .params and .nd files format" mean? <a name="question-85"></a>
#### 84. What does the message "Unsupported Input model file type ... Model Optimizer support only .params and .nd files format" mean? <a name="question-84"></a>
Model Optimizer for MXNet supports only `.params` and `.nd` files formats. Most likely, you specified some unsupported file format in `--input_model`.
For more information, refer to [Converting a MXNet* Model](convert_model/Convert_Model_From_MxNet.md).
Model Optimizer for Apache MXNet supports only `.params` and `.nd` files formats. Most likely, you specified an unsupported file format in `--input_model`.
For more information, refer to [Converting an MXNet Model](convert_model/Convert_Model_From_MxNet.md).
#### 86. What does the message "Operation ... not supported. Please register it as custom op" mean? <a name="question-86"></a>
#### 85. What does the message "Operation ... not supported. Please register it as custom op" mean? <a name="question-85"></a>
Model Optimizer tried to load the model that contains some unsupported operations.
If you want to convert model that contains unsupported operations you need to prepare extension for all such operations.
For more information, refer to [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
If you want to convert model that contains unsupported operations, you need to prepare extension for all such operations.
For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 87. What does the message "Can not register Op ... Please, call function 'register_caffe_python_extractor' with parameter 'name'" mean? <a name="question-87"></a>
#### 86. What does the message "Can not register Op ... Please, call function 'register_caffe_python_extractor' with parameter 'name'" mean? <a name="question-86"></a>
This error appears if the class of implementation of op for Python Caffe layer could not be used by Model Optimizer. Python layers should be handled differently compared to ordinary Caffe layers.
This error appears if the class of implementation of `Op` for Python Caffe layer could not be used by Model Optimizer. Python layers should be handled differently comparing to ordinary Caffe layers.
In particular, you need to call the function `register_caffe_python_extractor` and pass `name` as the second argument of the function.
The name should be the compilation of the layer name and the module name separated by a dot.
The name should be the compilation of the layer name with the module name separated by a dot.
For example, your topology contains this layer with type `Python`:
@@ -516,7 +516,7 @@ layer {
}
```
What you do first is implementing an extension for this layer in the Model Optimizer as an ancestor of `Op` class.
The first step is to implement an extension for this layer in Model Optimizer as an ancestor of `Op` class:
Note that the first call <code>register_caffe_python_extractor(ProposalPythonExampleOp, 'rpn.proposal_layer.ProposalLayer')</code> registers extension of the layer in the Model Optimizer that will be found by the specific name (mandatory to join module name and layer name): <code>rpn.proposal_layer.ProposalLayer</code>.
Note that the first call <code>register_caffe_python_extractor(ProposalPythonExampleOp, 'rpn.proposal_layer.ProposalLayer')</code> registers an extension of the layer in Model Optimizer, which will be found by the specific name (mandatory to join module name and layer name): <code>rpn.proposal_layer.ProposalLayer</code>.
The second call prevents Model Optimizer from using this extension as if it is an extension for
a layer with type `Proposal`. Otherwise, this layer can be chosen as an implementation of extension that can lead to potential issues.
For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 88. What does the message "Model Optimizer is unable to calculate output shape of Memory node .." mean? <a name="question-88"></a>
#### 87. What does the message "Model Optimizer is unable to calculate output shape of Memory node .." mean? <a name="question-87"></a>
Model Optimizer supports only `Memory` layers, in which `input_memory` goes before `ScaleShift` or `FullyConnected` layer.
This error message means that in your model the layer after input memory is not of type `ScaleShift` or `FullyConnected`.
Model Optimizer supports only `Memory` layers, in which `input_memory` goes before `ScaleShift` or the `FullyConnected` layer.
This error message means that in your model the layer after input memory is not of the `ScaleShift` or `FullyConnected` type.
This is a known limitation.
#### 89. What do the messages "File ... does not appear to be a Kaldi file (magic number does not match)", "Kaldi model should start with <Nnet> tag" mean? <a name="question-89"></a>
#### 88. What do the messages "File ... does not appear to be a Kaldi file (magic number does not match)", "Kaldi model should start with <Nnet> tag" mean? <a name="question-88"></a>
These error messages mean that the Model Optimizer does not support your Kaldi\* model, because checksum of the model is not
16896 (the model should start with this number) or model file does not contain tag`<Net>` as a starting one.
Double check that you provide a path to a true Kaldi model and try again.
These error messages mean that Model Optimizer does not support your Kaldi model, because the `checksum` of the model is not
16896 (the model should start with this number), or the model file does not contain the`<Net>` tag as a starting one.
Make sure that you provide a path to a true Kaldi model and try again.
#### 90. What do the messages "Expect counts file to be one-line file." or "Expect counts file to contain list of integers" mean? <a name="question-90"></a>
#### 89. What do the messages "Expect counts file to be one-line file." or "Expect counts file to contain list of integers" mean? <a name="question-89"></a>
These messages mean that you passed the file counts containing not one line. The count file should start with
`[` and end with `]`, and integer values should be separated by space between those signs.
These messages mean that the file counts you passed contain not one line. The count file should start with
`[` and end with `]`, and integer values should be separated by spaces between those brackets.
#### 91. What does the message "Model Optimizer is not able to read Kaldi model .." mean? <a name="question-91"></a>
#### 90. What does the message "Model Optimizer is not able to read Kaldi model .." mean? <a name="question-90"></a>
There are multiple reasons why the Model Optimizer does not accept a Kaldi topology:
file is not available or does not exist. Refer to FAQ [#89](#question-89).
There are multiple reasons why Model Optimizer does not accept a Kaldi topology, including:
the file is not available or does not exist. Refer to FAQ [#88](#question-88).
#### 92. What does the message "Model Optimizer is not able to read counts file .." mean? <a name="question-92"></a>
#### 91. What does the message "Model Optimizer is not able to read counts file .." mean? <a name="question-91"></a>
There are multiple reasons why the Model Optimizer does not accept a counts file:
file is not available or does not exist. Also refer to FAQ [#90](#question-90).
There are multiple reasons why Model Optimizer does not accept a counts file, including:
the file is not available or does not exist. Refer to FAQ [#89](#question-89).
#### 93. What does the message "For legacy MXNet models Model Optimizer does not support conversion of old MXNet models (trained with 1.0.0 version of MXNet and lower) with custom layers." mean? <a name="question-93"></a>
#### 92. What does the message "For legacy MXNet models Model Optimizer does not support conversion of old MXNet models (trained with 1.0.0 version of MXNet and lower) with custom layers." mean? <a name="question-92"></a>
This message means that if you have model with custom layers and its json file has been generated with MXNet version
lower than 1.0.0, Model Optimizer does not support such topologies. If you want to convert it you have to rebuild
MXNet with unsupported layers or generate new json with MXNet version 1.0.0 and higher. Also you need to implement
OpenVINO extension for used custom layers.
For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md).
This message means that if you have a model with custom layers and its JSON file has been generated with Apache MXNet version
lower than 1.0.0, Model Optimizer does not support such topologies. If you want to convert it, you have to rebuild
MXNet with unsupported layers or generate a new JSON file with Apache MXNet version 1.0.0 or higher. You also need to implement
OpenVINO extension to use custom layers.
For more information, refer to the [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md) guide.
#### 97. What does the message "Graph contains a cycle. Can not proceed .." mean? <a name="question-97"></a>
#### 93. What does the message "Graph contains a cycle. Can not proceed .." mean? <a name="question-93"></a>
Model Optimizer supports only straightforward models without cycles.
@@ -586,56 +586,56 @@ For Tensorflow:
For all frameworks:
1. [Replace cycle containing Sub-graph in Model Optimizer](customize_model_optimizer/Customize_Model_Optimizer.md)
2. See [OpenVINO™ Extensibility Mechanism](../../Extensibility_UG/Intro.md)
or
* Edit model in original framework to exclude cycle.
* Edit the model in its original framework to exclude cycle.
#### 98. What does the message "Can not transpose attribute '..' with value .. for node '..' .." mean? <a name="question-98"></a>
#### 94. What does the message "Can not transpose attribute '..' with value .. for node '..' .." mean? <a name="question-94"></a>
This message means that model is not supported. It may be caused by using shapes larger than 4-D.
This message means that the model is not supported. It may be caused by using shapes larger than 4-D.
There are two ways to avoid such message:
1. [Cutting Off Parts of a Model](convert_model/Cutting_Model.md)
2. Edit network in original framework to exclude such layers.
* [Cut off parts of the model](convert_model/Cutting_Model.md).
* Edit the network in its original framework to exclude such layers.
#### 99. What does the message "Expected token `</ParallelComponent>`, has `...`" mean? <a name="question-99"></a>
#### 95. What does the message "Expected token `</ParallelComponent>`, has `...`" mean? <a name="question-95"></a>
This error messages mean that Model Optimizer does not support your Kaldi model, because the Net contains `ParallelComponent` that does not end by tag`</ParallelComponent>`.
Double check that you provide a path to a true Kaldi model and try again.
This error messages mean that Model Optimizer does not support your Kaldi model, because the Net contains `ParallelComponent` that does not end with the`</ParallelComponent>` tag.
Make sure that you provide a path to a true Kaldi model and try again.
#### 100. What does the message "Interp layer shape inference function may be wrong, please, try to update layer shape inference function in the file (extensions/ops/interp.op at the line ...)." mean? <a name="question-100"></a>
#### 96. What does the message "Interp layer shape inference function may be wrong, please, try to update layer shape inference function in the file (extensions/ops/interp.op at the line ...)." mean? <a name="question-96"></a>
There are many flavors of Caffe framework, and most layers in them are implemented identically.
But there are exceptions. For example, output value of layer Interp is calculated differently in Deeplab-Caffe and classic Caffe. So if your model contain layer Interp and converting of your model has failed, please modify the 'interp_infer' function in the file extensions/ops/interp.op according to the comments of the file.
However, there are exceptions. For example, the output value of layer Interp is calculated differently in Deeplab-Caffe and classic Caffe. Therefore, if your model contains layer Interp and the conversion of your model has failed, modify the `interp_infer` function in the `extensions/ops/interp.op` file according to the comments in the file.
#### 101. What does the message "Mean/scale values should ..." mean? <a name="question-101"></a>
#### 97. What does the message "Mean/scale values should ..." mean? <a name="question-97"></a>
It means that your mean/scale values have wrong format. Specify mean/scale values using the form `layer_name(val1,val2,val3)`.
You need to specify values for each input of the model. For more information, refer to [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md).
It means that your mean/scale values have a wrong format. Specify mean/scale values in the form of `layer_name(val1,val2,val3)`.
You need to specify values for each input of the model. For more information, refer to the [Converting a Model to Intermediate Representation](convert_model/Converting_Model.md) guide.
#### 102. What does the message "Operation _contrib_box_nms is not supported ..." mean? <a name="question-102"></a>
#### 98. What does the message "Operation _contrib_box_nms is not supported ..." mean? <a name="question-98"></a>
It means that you trying to convert the topology which contains '_contrib_box_nms' operation which is not supported directly. However the sub-graph of operations including the '_contrib_box_nms' could be replaced with DetectionOutput layer if your topology is one of the gluoncv topologies. Specify '--enable_ssd_gluoncv' commandline parameter for the Model Optimizer to enable this transformation.
It means that you are trying to convert a topology contains the `_contrib_box_nms` operation which is not supported directly. However, the sub-graph of operations including `_contrib_box_nms` could be replaced with the DetectionOutput layer if your topology is one of the `gluoncv` topologies. Specify the `--enable_ssd_gluoncv` command-line parameter for Model Optimizer to enable this transformation.
#### 103. What does the message "ModelOptimizer is not able to parse *.caffemodel" mean? <a name="question-103"></a>
#### 99. What does the message "ModelOptimizer is not able to parse *.caffemodel" mean? <a name="question-99"></a>
If a '*.caffemodel' file exists and it is correct, the error possibly occured due to the use of Python protobuf implementation. In some cases, it shows error message during model parsing, for example: "'utf-8' codec can't decode byte 0xe0 in position 4: invalid continuation byte in field: mo_caffe.SpatialTransformerParameter.transform_type". You can either use Python 3.6/3.7 or build 'cpp' implementation of protobuf yourself for your version of Python. For the complete instructions about building `protobuf` from sources, see the appropriate section in [Converting a Model to Intermediate Representation](../Deep_Learning_Model_Optimizer_DevGuide.md).
If a `*.caffemodel` file exists and is correct, the error occurred possibly because of the use of Python protobuf implementation. In some cases, error messages may appear during model parsing, for example: "`utf-8` codec can't decode byte 0xe0 in position 4: invalid continuation byte in field: mo_caffe.SpatialTransformerParameter.transform_type". You can either use Python 3.6/3.7 or build the `cpp` implementation of `protobuf` yourself for your version of Python. For the complete instructions about building `protobuf` from sources, see the appropriate section in the [Converting Models with Model Optimizer](../Deep_Learning_Model_Optimizer_DevGuide.md) guide.
#### 104. What does the message "SyntaxError: 'yield' inside list comprehension" during MxNet\* model conversion mean? <a name="question-104"></a>
#### 100. What does the message "SyntaxError: 'yield' inside list comprehension" during MxNet model conversion mean? <a name="question-100"></a>
The issue "SyntaxError: 'yield' inside list comprehension" might occur during converting MXNet\* models (mobilefacedet-v1-mxnet, brain-tumor-segmentation-0001) on Windows* platform with Python* 3.8 environment. This issue is caused by API changes for `yield expression` in Python 3.8.
The issue "SyntaxError: `yield` inside list comprehension" might occur during converting MXNet models (`mobilefacedet-v1-mxnet`, `brain-tumor-segmentation-0001`) on Windows platform with Python 3.8 environment. This issue is caused by the API changes for `yield expression` in Python 3.8.
The following workarounds are suggested to resolve this issue:
1. Use Python 3.6/3.7 to convert MXNet\* models on Windows
2. Update MXNet: pip install mxnet=1.7.0.post2
Note that you might have conflicts between previously installed PyPI dependencies.
1. Use Python 3.6/3.7 to convert MXNet models on Windows
2. Update Apache MXNet by using `pip install mxnet==1.7.0.post2`
Note that it might have conflicts with previously installed PyPI dependencies.
#### 105. What does the message "The IR preparation was executed by the legacy MO path. ..." mean? <a name="question-105"></a>
#### 101. What does the message "The IR preparation was executed by the legacy MO path. ..." mean? <a name="question-101"></a>
For the models in ONNX* format, there are two available paths of IR conversion.
The old one is handled by the old Python* implementation, while the new one uses new C++ frontends.
For the models in ONNX format, there are two available paths of IR conversion.
The old one is handled by the old Python implementation, while the new one uses new C++ frontends.
Starting from the 2022.1 version, the default IR conversion path for ONNX models is processed using the new ONNX frontend.
Certain features, such as `--extensions` and `--transformations_config`, are not yet fully supported on the new frontends.
For `--extensions`, the new frontends support only paths to shared libraries (.dll and .so). For `--transformations_config`, they support JSON configurations with defined library fields.
Inputs freezing (enabled by `--freeze_placeholder_with_value` or `--input` arguments) is not supported on the new frontends.
The IR conversion falls back to the old path if a user does not select any expected path of conversion explicitly (by`--use_new_frontend` or `--use_legacy_frontend` MO arguments) and unsupported pre-defined scenario is detected on the new frontend path.
The new frontends support only paths to shared libraries (.dll and .so) for `--extensions`. They support JSON configurations with defined library fields for `--transformations_config`.
Inputs freezing (enabled by `--freeze_placeholder_with_value` or `--input` arguments) is not supported by the new frontends.
The IR conversion falls back to the old path if a user does not select any expected path of conversion explicitly (with`--use_new_frontend` or `--use_legacy_frontend` MO arguments) and unsupported pre-defined scenario is detected on the new frontend path.
Some TensorFlow operations do not match to any OpenVINO operation, but are still supported by the Model Optimizer and can be used on constant propagation path. These layers are labeled 'Constant propagation' in the table.
Some of TensorFlow operations do not match any OpenVINO operations. Yet, they are still supported by Model Optimizer and can be used on constant propagation path. These layers are labeled with `Constant propagation` in the table below:
| Operation Name in TensorFlow | Limitations|
@@ -165,19 +168,19 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| ArgMax | |
| ArgMin | |
| Asinh | |
| Assert | Not needed for inference |
| Assign | Not needed for inference |
| AssignSub | Not needed for inference |
| Assert | Not needed for inference. |
| Assign | Not needed for inference. |
| AssignSub | Not needed for inference. |
| Atanh | |
| AvgPool | |
| AvgPoolV2 | Supported only for constant-foldable kernel_size and strides inputs |
| AvgPoolV2 | Supported only for constant-foldable `kernel_size` and strides inputs. |
| AvgPool3D | |
| BatchMatMul | |
| BatchMatMulV2 | |
| BatchToSpaceND | |
| BiasAdd | |
| BlockLSTM | |
| Bucketize | CPU only |
| Bucketize | CPU only. |
| BroadcastTo | |
| Cast | |
| Ceil | |
@@ -191,30 +194,30 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Conv3DBackpropInputV2 | |
| Cos | |
| Cosh | |
| CropAndResize | "method" = "bilinear" only |
| CTCGreedyDecoder | Supported only with decoded indices output in a dense format |
| CTCLoss | Supported only with decoded indices input in a dense format |
| CropAndResize | `method` = `bilinear` only. |
| CTCGreedyDecoder | Supported only with decoded indices output in a dense format. |
| CTCLoss | Supported only with decoded indices input in a dense format. |
| CumSum | |
| DepthToSpace| |
| DepthwiseConv2dNative| |
| Einsum | Supported only with equation that does not contain repeated labels within a subscript |
| Einsum | Supported only with equation that does not contain repeated labels within a subscript. |
| Elu | |
| EmptyTensorList | Supported only when it is part of a sub-graph of the special form |
| Enter | Supported only when it is fused to the TensorIterator layer |
| EmptyTensorList | Supported only when it is part of a sub-graph of the special form. |
| Enter | Supported only when it is fused to the TensorIterator layer. |
| Equal | |
| Erf | |
| Exit | Supported only when it is fused to the TensorIterator layer |
| Exit | Supported only when it is fused to the TensorIterator layer. |
| Exp | |
| ExpandDims | |
| ExperimentalSparseWeightedSum | CPU only |
| ExperimentalSparseWeightedSum | CPU only. |
| ExtractImagePatches | |
| EuclideanNorm | |
| FakeQuantWithMinMaxVars | |
| FakeQuantWithMinMaxVarsPerChannel | |
| FFT | Supported only when it is part of a sub-graph of the special form |
| FFT2D | Supported only when it is part of a sub-graph of the special form |
| FFT3D | Supported only when it is part of a sub-graph of the special form |
| FIFOQueueV2 | Supported only when it is part of a sub-graph of the special form |
| FFT | Supported only when it is part of a sub-graph of the special form. |
| FFT2D | Supported only when it is part of a sub-graph of the special form. |
| FFT3D | Supported only when it is part of a sub-graph of the special form. |
| FIFOQueueV2 | Supported only when it is part of a sub-graph of the special form. |
| Fill | |
| Floor | |
| FloorDiv | |
@@ -228,12 +231,12 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| GatherV2 | |
| Greater | |
| GreaterEqual | |
| Identity | Not needed for shape inference |
| Identity | Not needed for shape inference. |
| IdentityN | |
| IFFT | Supported only when it is part of a sub-graph of the special form |
| IFFT2D | Supported only when it is part of a sub-graph of the special form |
| IFFT3D | Supported only when it is part of a sub-graph of the special form |
| IteratorGetNext | Supported only when it is part of a sub-graph of the special form |
| IFFT | Supported only when it is part of a sub-graph of the special form. |
| IFFT2D | Supported only when it is part of a sub-graph of the special form. |
| IFFT3D | Supported only when it is part of a sub-graph of the special form. |
| IteratorGetNext | Supported only when it is part of a sub-graph of the special form. |
| LRN | |
| LeakyRelu | |
| Less | |
@@ -244,23 +247,23 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| LogicalOr | |
| LogicalNot | |
| LogSoftmax | |
| LookupTableInsertV2 | Supported only when it is part of a sub-graph of the special form |
| LoopCond | Supported only when it is fused to the TensorIterator layer |
| LookupTableInsertV2 | Supported only when it is part of a sub-graph of the special form. |
| LoopCond | Supported only when it is fused to the TensorIterator layer. |
| MatMul | |
| Max | |
| MaxPool | |
| MaxPoolV2 | Supported only for constant-foldable kernel_size and strides inputs |
| MaxPoolV2 | Supported only for constant-foldable `kernel_size` and strides inputs. |
| MaxPool3D | |
| Maximum | |
| Mean | |
| Merge | Supported only when it is fused to the TensorIterator layer |
| Merge | Supported only when it is fused to the TensorIterator layer. |
| Min | |
| Minimum | |
| MirrorPad | |
| Mod | |
| Mul | |
| Neg | |
| NextIteration | Supported only when it is fused to the TensorIterator layer |
| NextIteration | Supported only when it is fused to the TensorIterator layer. |
| NonMaxSuppressionV2 | |
| NonMaxSuppressionV3 | |
| NonMaxSuppressionV4 | |
@@ -274,9 +277,9 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Placeholder | |
| PlaceholderWithDefault | |
| Prod | |
| QueueDequeue | Supported only when it is part of a sub-graph of the special form |
| QueueDequeueUpToV2 | Supported only when it is part of a sub-graph of the special form |
| QueueDequeueV2 | Supported only when it is part of a sub-graph of the special form |
| QueueDequeue | Supported only when it is part of a sub-graph of the special form. |
| QueueDequeueUpToV2 | Supported only when it is part of a sub-graph of the special form. |
| QueueDequeueV2 | Supported only when it is part of a sub-graph of the special form. |
| RandomUniform | |
| RandomUniformInt | |
| Range | |
@@ -290,7 +293,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| ResizeNearestNeighbor | |
| ResourceGather| |
| ReverseSequence | |
| ReverseV2 | Supported only when it can be converted to the ReverseSequence operation |
| ReverseV2 | Supported only when it can be converted to the ReverseSequence operation. |
| Roll | |
| Round | |
| Pow | |
@@ -309,10 +312,10 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Softsign | |
| SpaceToBatchND | |
| SpaceToDepth | |
| SparseFillEmptyRows | Supported only when it is part of a sub-graph of the special form |
| SparseReshape | Supported only when it is part of a sub-graph of the special form |
| SparseSegmentSum | Supported only when it is part of a sub-graph of the special form |
| SparseSegmentMean | Supported only when it is part of a sub-graph of the special form |
| SparseFillEmptyRows | Supported only when it is part of a sub-graph of the special form. |
| SparseReshape | Supported only when it is part of a sub-graph of the special form. |
| SparseSegmentSum | Supported only when it is part of a sub-graph of the special form. |
| SparseSegmentMean | Supported only when it is part of a sub-graph of the special form. |
| SparseToDense | CPU only |
| Split | |
| SplitV | |
@@ -320,31 +323,31 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Square | |
| SquaredDifference | |
| Square| |
| Squeeze | The case when squeeze axis is not specified is not supported |
| Squeeze | Cases in which squeeze axis is not specified are not supported. |
| StatelessWhile | |
| StopGradient | Not needed for shape inference |
| StridedSlice | Supported only for constant-foldable begin, end, and strides inputs |
| StopGradient | Not needed for shape inference. |
| StridedSlice | Supported only for constant-foldable `begin`, `end`, and `strides` inputs. |
| Sub | |
| Sum | |
| Swish | |
| swish_f32 | |
| Switch | Control flow propagation |
| Switch | Control flow propagation. |
| Tan | |
| Tanh | |
| TensorArrayGatherV3 | Supported only when it is fused to the TensorIterator layer |
| TensorArrayReadV3 | Supported only when it is fused to the TensorIterator layer |
| TensorArrayScatterV3 | Supported only when it is fused to the TensorIterator layer |
| TensorArraySizeV3 | Supported only when it is fused to the TensorIterator layer |
| TensorArrayV3 | Supported only when it is fused to the TensorIterator layer |
| TensorArrayWriteV3 | Supported only when it is fused to the TensorIterator layer |
| TensorListPushBack | Supported only when it is part of a sub-graph of the special form |
| TensorArrayGatherV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorArrayReadV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorArrayScatterV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorArraySizeV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorArrayV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorArrayWriteV3 | Supported only when it is fused to the TensorIterator layer. |
| TensorListPushBack | Supported only when it is part of a sub-graph of the special form. |
| Tile | |
| TopkV2 | |
| Transpose | |
| Unpack | |
| Variable | |
| VariableV2 | |
| Where | Supported only when it is part of a sub-graph of the special form |
| Where | Supported only when it is part of a sub-graph of the special form. |
| ZerosLike | |
@@ -366,7 +369,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Bidirectional | |
| Concatenate | |
| Conv1D | |
| Conv1DTranspose | Not supported if dilation is not equal to 1 |
| Conv1DTranspose | Not supported if `dilation` is not equal to 1. |
| Conv2D | |
| Conv2DTranspose | |
| Conv3D | |
@@ -375,7 +378,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Cropping2D | |
| Cropping3D | |
| Dense | |
| DenseFeatures | Not supported for categorical and crossed features |
| DenseFeatures | Not supported for categorical and crossed features. |
| DepthwiseConv2D | |
| Dot | |
| Dropout | |
@@ -407,7 +410,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| Multiply | |
| PReLU | |
| Permute | |
| RNN | Not supported for some custom cells |
| RNN | Not supported for some custom cells. |
| ReLU | |
| RepeatVector | |
| Reshape | |
@@ -442,7 +445,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| affinetransform | |
| backproptruncationcomponent | |
| batchnormcomponent | |
| clipgradientcomponent | Not needed for inference |
| clipgradientcomponent | Not needed for inference. |
| concat | |
| convolutional1dcomponent | |
| convolutionalcomponent | |
@@ -452,7 +455,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| fixedaffinecomponent | |
| fixedbiascomponent | |
| fixedscalecomponent | |
| generaldropoutcomponent| Not needed for inference |
| generaldropoutcomponent| Not needed for inference. |
| linearcomponent | |
| logsoftmaxcomponent | |
| lstmnonlinearitycomponent | |
@@ -461,7 +464,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| maxpoolingcomponent | |
| naturalgradientaffinecomponent | |
| naturalgradientperelementscalecomponent | |
| noopcomponent | Not needed for inference |
| noopcomponent | Not needed for inference. |
| normalizecomponent | |
| parallelcomponent | |
| pnormcomponent | |
@@ -471,7 +474,7 @@ Some TensorFlow operations do not match to any OpenVINO operation, but are still
| sigmoidcomponent | |
| softmax | |
| softmaxComponent | |
| specaugmenttimemaskcomponent | Not needed for inference |
| specaugmenttimemaskcomponent | Not needed for inference. |
| splicecomponent | |
| tanhcomponent | |
| tdnncomponent | |
@@ -667,68 +670,95 @@ paddlepaddle>=2.1
| Operator Name in PaddlePaddle| Limitations|
| :----------| :----------|
| adpative_pool2d | 'NHWC' data_layout is not supported |
| arg_max | 'int32' output data_type is not supported |
| adpative_pool2d | The `NHWC` data_layout is not supported. |
| arg_max | The `int32` output data_type is not supported. |
| assign | |
| assign_value | |
| batch_norm | |
| bilinear_interp | 'NCW' 'NWC' 'NHWC' 'NCDHW' 'NDHWC' data_layout are not supported |
| bilinear_interp_v2 | 'NCW' 'NWC' 'NHWC' 'NCDHW' 'NDHWC' data_layout are not supported |
| bilinear_interp | `NCW`, `NWC`, `NHWC`, `NCDHW`, `NDHWC` data_layout are not supported. |
| bilinear_interp_v2 | `NCW`, `NWC`, `NHWC`, `NCDHW`, `NDHWC` data_layout are not supported. |
| bmm | |
| cast | |
| clip | |
| concat | |
| conv2d | 'NHWC' data_layout is not supported |
| depthwise_conv2d | 'NHWC' data_layout is not supported |
| conv2d | `NHWC` data_layout is not supported. |
| deformable_conv | |
| depthwise_conv2d | `NHWC` data_layout is not supported. |
| elementwise_add | |
| elementwise_div | |
| elementwise_max | |
| elementwise_min | |
| elementwise_mul | |
| elementwise_not_equal | |
| elementwise_pow | |
| elementwise_sub | |
| equal | |
| expand_v2 | |
| exp | |
| expand | |
| expand_v2 | |
| fill_any_like | |
| fill_constant_batch_size_like | |
| fill_constant | |
| fill_constant_batch_size_like | |
| fill_zeros_like | |
| flatten_contiguous_range | |
| floor | |
| gather | |
| gather_tree | |
| gelu | |
| generate_proposals_v2 | |
| greater_equal | |
| greater_than | |
| hard_sigmoid | |
| hard_swish | |
| layer_norm | |
| leaky_relu | |
| less_than | |
| log | |
| logical_and | |
| logical_not | |
| logical_or | |
| logical_xor | |
| lookup_table_v2 | |
| matmul | |
| matmul_v2 | |
| matrix_nms | Only supports IE CPU plugin with 'number of selected boxes' static shape(e.g.: min(min(num_boxes, nms_top_k) * num_classes_output, keep_top_k)) |
| matrix_nms | Only supports IE CPU plugin with *"number of selected boxes"* static shape(e.g.: `min(min(num_boxes, nms_top_k) * num_classes_output, keep_top_k)`). |
| max_pool2d_with_index | |
| meshgrid | |
| mul | |
| multiclass_nms3 | Only supports IE CPU plugin with 'number of selected boxes' static shape(e.g.: min(min(num_boxes, nms_top_k) * num_classes_output, keep_top_k)) |
| nearest_interp | 'NCW' 'NWC' 'NHWC' 'NCDHW' 'NDHWC' data_layout are not supported |
| nearest_interp_v2 | 'NCW' 'NWC' 'NHWC' 'NCDHW' 'NDHWC' data_layout are not supported |
| pad3d | 'Circular' mode is not supported |
| multiclass_nms3 | Only supports IE CPU plugin with *"number of selected boxes"* static shape(e.g.: `min(min(num_boxes, nms_top_k) * num_classes_output, keep_top_k)`). |
| nearest_interp | `NCW`, `NWC`, `NHWC`, `NCDHW`, `NDHWC` data_layout are not supported. |
| nearest_interp_v2 | `NCW`, `NWC`, `NHWC`, `NCDHW`, `NDHWC` data_layout are not supported. |
| pad3d | `Circular` mode is not supported. |
| pool2d | `NHWC` data_layout is not supported. |
| pow | |
| pool2d | 'NHWC' data_layout is not supported |
| prior_box | |
| range | |
| reduce_max | |
| reduce_mean | |
| reduce_min | |
| reduce_prod | |
| reduce_sum | |
| relu | |
| relu6 | |
| reshape2 | |
| rnn | 'SimpleRNN' and 'GRU' modes are not supported |
| rnn | `SimpleRNN` and `GRU` modes are not supported. |
# Converting a Caffe* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe}
# Converting a Caffe Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe}
## Convert a Caffe* Model <a name="Convert_From_Caffe"></a>
To convert a Caffe\* model, run Model Optimizer with the path to the input model `.caffemodel` file:
<aname="Convert_From_Caffe"></a>To convert a Caffe model, run Model Optimizer with the path to the input model `.caffemodel` file:
```sh
mo --input_model <INPUT_MODEL>.caffemodel
```
The following list provides the Caffe\*-specific parameters.
The following list provides the Caffe-specific parameters.
```
Caffe*-specific parameters:
Caffe-specific parameters:
--input_proto INPUT_PROTO, -d INPUT_PROTO
Deploy-ready prototxt file that contains a topology
structure and layer attributes
@@ -45,14 +44,16 @@ Caffe*-specific parameters:
attributes without flattening nested parameters.
```
### Command-Line Interface (CLI) Examples Using Caffe\*-Specific Parameters
### CLI Examples Using Caffe-Specific Parameters
* Launching the Model Optimizer for the [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `prototxt` file. This is needed when the name of the Caffe\* model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file.
* Launching Model Optimizer for [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `prototxt` file.
This is needed when the name of the Caffe model and the `.prototxt` file are different or are placed in different directories. Otherwise, it is enough to provide only the path to the input `model.caffemodel` file.
```sh
mo --input_model bvlc_alexnet.caffemodel --input_proto bvlc_alexnet.prototxt
```
* Launching the Model Optimizer for the [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `CustomLayersMapping` file. This is the legacy method of quickly enabling model conversion if your model has custom layers. This requires the Caffe\* system on the computer.
Optional parameters without default values and not specified by the user in the `.prototxt` file are removed from the Intermediate Representation, and nested parameters are flattened:
* Launching Model Optimizer for [bvlc_alexnet.caffemodel](https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet) with a specified `CustomLayersMapping` file.
This is the legacy method of quickly enabling model conversion if your model has custom layers. This requires the Caffe system on the computer.
The optional parameters without default values and not specified by the user in the `.prototxt` file are removed from the Intermediate Representation, and nested parameters are flattened:
```sh
mo --input_model bvlc_alexnet.caffemodel -k CustomLayersMapping.xml --disable_omitting_optional --enable_flattening_nested_params
```
@@ -82,22 +83,22 @@ Optional parameters without default values and not specified by the user in the
```
## Custom Layer Definition
Internally, when you run the Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in this list of known layers, the Model Optimizer classifies them as custom.
Internally, when you run Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list. If your topology contains such kind of layers, Model Optimizer classifies them as custom.
## Supported Caffe\* Layers
Refer to [Supported Framework Layers](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Supported Caffe Layers
For the list of supported standard layers, refer to the [Supported Framework Layers](../Supported_Frameworks_Layers.md) page.
## Frequently Asked Questions (FAQ)
The Model Optimizer provides explanatory messages if it is unable to run to completion due to issues like typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md). The FAQ has instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
Model Optimizer provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to [Model Optimizer FAQ](../Model_Optimizer_FAQ.md) which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections to help you understand what went wrong.
## Summary
In this document, you learned:
* Basic information about how the Model Optimizer works with Caffe\* models
* Which Caffe\* models are supported
* How to convert a trained Caffe\* model using the Model Optimizer with both framework-agnostic and Caffe-specific command-line options
* Basic information about how the Model Optimizer works with Caffe models.
* Which Caffe models are supported.
* How to convert a trained Caffe model by using Model Optimizer with both framework-agnostic and Caffe-specific command-line options.
# Converting a Kaldi* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi}
# Converting a Kaldi Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Kaldi}
> **NOTE**: The Model Optimizer supports the [nnet1](http://kaldi-asr.org/doc/dnn1.html) and [nnet2](http://kaldi-asr.org/doc/dnn2.html) formats of Kaldi models. Support of the [nnet3](http://kaldi-asr.org/doc/dnn3.html) format is limited.
## Convert a Kaldi* Model <a name="Convert_From_Kaldi"></a>
To convert a Kaldi\* model, run Model Optimizer with the path to the input model `.nnet` or `.mdl` file:
> **NOTE**: Model Optimizer supports the [nnet1](http://kaldi-asr.org/doc/dnn1.html) and [nnet2](http://kaldi-asr.org/doc/dnn2.html) formats of Kaldi models. The support of the [nnet3](http://kaldi-asr.org/doc/dnn3.html) format is limited.
<aname="Convert_From_Kaldi"></a>To convert a Kaldi model, run Model Optimizer with the path to the input model `.nnet` or `.mdl` file:
```sh
mo --input_model <INPUT_MODEL>.nnet
```
### Using Kaldi\*-Specific Conversion Parameters <a name="kaldi_specific_conversion_params"></a>
## Using Kaldi-Specific Conversion Parameters <a name="kaldi_specific_conversion_params"></a>
The following list provides the Kaldi\*-specific parameters.
The following list provides the Kaldi-specific parameters.
```sh
Kaldi-specific parameters:
@@ -21,14 +20,14 @@ Kaldi-specific parameters:
--remove_memory Remove the Memory layer and add new inputs and outputs instead
```
### Examples of CLI Commands
## Examples of CLI Commands
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with the specified `.nnet` file:
* To launch Model Optimizer for the `wsj_dnn5b_smbr` model with the specified `.nnet` file:
```sh
mo --input_model wsj_dnn5b_smbr.nnet
```
* To launch the Model Optimizer for the wsj_dnn5b_smbr model with existing file that contains counts for the last layer with biases:
* To launch Model Optimizer for the `wsj_dnn5b_smbr` model with the existing file that contains counts for the last layer with biases:
```sh
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts
```
@@ -44,7 +43,7 @@ Kaldi-specific parameters:
\f$|C|\f$ - number of elements in the counts array;
* The normalized counts are subtracted from biases of the last or next to last layer (if last layer is SoftMax).
> **NOTE**: Model Optimizer will show warning if model contains counts values inside model and `--counts` option is not used.
> **NOTE**: Model Optimizer will show a warning if a model contains values of counts and the `--counts` option is not used.
* If you want to remove the last SoftMax layer in the topology, launch the Model Optimizer with the
`--remove_output_softmax` flag:
@@ -52,28 +51,30 @@ Kaldi-specific parameters:
mo --input_model wsj_dnn5b_smbr.nnet --counts wsj_dnn5b_smbr.counts --remove_output_softmax
```
The Model Optimizer finds the last layer of the topology and removes this layer only if it is a SoftMax layer.
The Model Optimizer finds the last layer of the topology and removes this layer only if it is a SoftMax layer.
> **NOTE**: Model Optimizer can remove SoftMax layer only if the topology has one output.
> **NOTE**: Model Optimizer can remove SoftMax layer only if the topology has one output.
> **NOTE**: For sample inference of Kaldi models, you can use the OpenVINO Speech Recognition sample application. The sample supports models with one output. If your model has several outputs, specify the desired one with the `--output` option.
* You can use the *OpenVINO Speech Recognition* sample application for the sample inference of Kaldi models. This sample supports models with only one output. If your model has several outputs, specify the desired one with the `--output` option.
If you want to convert a model for inference on Intel® Movidius™ Myriad™, use the `--remove_memory` option.
It removes Memory layers from the IR. Instead of it, additional inputs and outputs appear in the IR.
The Model Optimizer outputs the mapping between inputs and outputs. For example:
## Converting a Model for Intel® Movidius™ Myriad™ VPU
If you want to convert a model for inference on Intel® Movidius™ Myriad™ VPU, use the `--remove_memory` option.
It removes the Memory layers from the OpenVINO IR files. Additional inputs and outputs will appear in the IR files instead.
Model Optimizer will output the mapping between inputs and outputs. For example:
Based on this mapping, link inputs and outputs in your application manually as follows:
Based on this mapping, link inputs and outputs in your application manually as follows:
1. Initialize inputs from the mapping as zeros in the first frame of an utterance.
2. Copy output blobs from the mapping to the corresponding inputs. For example, data from `Result_for_Offset_fastlstm2.r_trunc__2Offset_fastlstm2.r_trunc__2_out`
must be copied to `Parameter_0_for_Offset_fastlstm2.r_trunc__2Offset_fastlstm2.r_trunc__2_out`.
## Supported Kaldi\* Layers
Refer to [Supported Framework Layers ](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Supported Kaldi Layers
For the list of supported standard layers, refer to the [Supported Framework Layers ](../Supported_Frameworks_Layers.md) page.
# Converting an MXNet* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet}
# Converting an MXNet Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_MxNet}
## Convert an MXNet* Model <a name="ConvertMxNet"></a>
To convert an MXNet\* model, run Model Optimizer with a path to the input model `.params` file:
<aname="ConvertMxNet"></a>To convert an MXNet model, run Model Optimizer with the path to the *`.params`* file of the input model:
```sh
mo --input_model model-file-0000.params
```
### Using MXNet\*-Specific Conversion Parameters <a name="mxnet_specific_conversion_params"></a>
The following list provides the MXNet\*-specific parameters.
## Using MXNet-Specific Conversion Parameters <a name="mxnet_specific_conversion_params"></a>
The following list provides the MXNet-specific parameters.
```
MXNet-specific parameters:
@@ -23,36 +22,36 @@ MXNet-specific parameters:
--save_params_from_nd
Enable saving built parameters file from .nd files
--legacy_mxnet_model
Enable MXNet loader to make a model compatible with the latest MXNet version.
Use only if your model was trained with MXNet version lower than 1.0.0
Enable Apache MXNet loader to make a model compatible with the latest Apache MXNet version.
Use only if your model was trained with Apache MXNet version lower than 1.0.0
--enable_ssd_gluoncv
Enable transformation for converting the gluoncv ssd topologies.
Use only if your topology is one of ssd gluoncv topologies
```
> **NOTE**: By default, the Model Optimizer does not use the MXNet loader, as it transforms the topology to another format, which is compatible with the latest
> version of MXNet, but it is required for models trained with lower version of MXNet. If your model was trained with MXNet version lower than 1.0.0, specify the
> `--legacy_mxnet_model` key to enable the MXNet loader. However, the loader does not support models with custom layers. In this case, you must manually
> recompile MXNet with custom layers and install it to your environment.
> **NOTE**: By default, Model Optimizer does not use the Apache MXNet loader. It transforms the topology to another format which is compatible with the latest
> version of Apache MXNet. However, the Apache MXNet loader is required for models trained with lower version of Apache MXNet. If your model was trained with an Apache MXNet version lower than 1.0.0, specify the
> `--legacy_mxnet_model` key to enable the Apache MXNet loader. Note that the loader does not support models with custom layers. In this case, you must manually
> recompile Apache MXNet with custom layers and install it in your environment.
## Custom Layer Definition
Internally, when you run the Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list of known layers. If your topology contains any layers that are not in this list of known layers, the Model Optimizer classifies them as custom.
Internally, when you run Model Optimizer, it loads the model, goes through the topology, and tries to find each layer type in a list of known layers. Custom layers are layers that are not included in the list. If your topology contains such kind of layers, Model Optimizer classifies them as custom.
## Supported MXNet\* Layers
Refer to [Supported Framework Layers](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Supported MXNet Layers
For the list of supported standard layers, refer to the [Supported Framework Layers](../Supported_Frameworks_Layers.md) page.
## Frequently Asked Questions (FAQ)
The Model Optimizer provides explanatory messages if it is unable to run to completion due to issues like typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md). The FAQ has instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
Model Optimizer provides explanatory messages when it is unable to complete conversions due to typographical errors, incorrectly used options, or other issues. A message describes the potential cause of the problem and gives a link to [Model Optimizer FAQ](../Model_Optimizer_FAQ.md) which provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections to help you understand what went wrong.
## Summary
In this document, you learned:
* Basic information about how the Model Optimizer works with MXNet\* models
* Which MXNet\* models are supported
* How to convert a trained MXNet\* model using the Model Optimizer with both framework-agnostic and MXNet-specific command-line options
* Basic information about how Model Optimizer works with MXNet models.
* Which MXNet models are supported.
* How to convert a trained MXNet model by using the Model Optimizer with both framework-agnostic and MXNet-specific command-line options.
# Converting an ONNX Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_ONNX}
## Introduction to ONNX
[ONNX*](https://github.com/onnx/onnx) is a representation format for deep learning models. ONNX allows AI developers easily transfer models between different frameworks that helps to choose the best combination for them. Today, PyTorch\*, Caffe2\*, Apache MXNet\*, Microsoft Cognitive Toolkit\* and other tools are developing ONNX support.
[ONNX](https://github.com/onnx/onnx) is a representation format for deep learning models that allows AI developers to easily transfer models between different frameworks. It is hugely popular among deep learning tools, like PyTorch, Caffe2, Apache MXNet, Microsoft Cognitive Toolkit, and many others.
This page gives instructions on how to convert a model from ONNX format to OpenVINO IR format using Model Optimizer. To use Model Optimizer, install OpenVINO Development Tools by following the [installation instructions here](https://docs.openvino.ai/latest/openvino_docs_install_guides_install_dev_tools.html).
## Converting an ONNX Model <a name="Convert_From_ONNX"></a>
ONNX models are directly compatible with OpenVINO Runtime and can be loaded in their native `.onnx` format using `net = ie.read_model("model.onnx")`. The benefit of converting ONNX models to the OpenVINO IR format is that it allows them to be easily optimized for target hardware with advanced OpenVINO tools such as [NNCF](../../../optimization_guide/nncf_introduction.md).
This page provides instructions on how to convert a model from the ONNX format to the OpenVINO IR format using Model Optimizer. To use Model Optimizer, install OpenVINO Development Tools by following the [installation instructions](https://docs.openvino.ai/latest/openvino_docs_install_guides_install_dev_tools.html).
## Convert an ONNX* Model <a name="Convert_From_ONNX"></a>
The Model Optimizer process assumes you have an ONNX model that was directly downloaded from a public repository or converted from any framework that supports exporting to the ONNX format.
To convert an ONNX\* model, run Model Optimizer with the path to the input model `.onnx` file:
To convert an ONNX model, run Model Optimizer with the path to the input model `.onnx` file:
```sh
mo --input_model <INPUT_MODEL>.onnx
```
There are no ONNX\* specific parameters, so only framework-agnostic parameters are available to convert your model. For details, see the General Conversion Parameters section on the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) page.
There are no ONNX specific parameters, so only framework-agnostic parameters are available to convert your model. For details, see the *General Conversion Parameters* section in the [Converting a Model to Intermediate Representation (IR)](Converting_Model.md) guide.
## Supported ONNX\* Layers
Refer to [Supported Framework Layers](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Supported ONNX Layers
For the list of supported standard layers, refer to the [Supported Framework Layers](../Supported_Frameworks_Layers.md) page.
## See Also
This page provided general instructions for converting ONNX models. See the [Model Conversion Tutorials](Convert_Model_Tutorials.md) page for a set of tutorials that give step-by-step instructions for converting specific ONNX models. Here are some example tutorials:
## Additional Resources
See the [Model Conversion Tutorials](Convert_Model_Tutorials.md) page for a set of tutorials providing step-by-step instructions for converting specific ONNX models. Here are some examples:
# Converting a PyTorch* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch}
# Converting a PyTorch Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_PyTorch}
## Typical Steps to Convert PyTorch Model <a name="typical-pytorch"></a>
PyTorch* framework is supported through export to ONNX\* format. A summary of the steps for optimizing and deploying a model that was trained with the PyTorch\* framework:
The PyTorch framework is supported through export to the ONNX format. In order to optimize and deploy a model that was trained with it:
1. [Export PyTorch model to ONNX\*](#export-to-onnx).
2. [Convert an ONNX\* model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
1. [Export a PyTorch model to ONNX](#export-to-onnx).
2. [Convert the ONNX model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
## Export PyTorch\* Model to ONNX\* Format <a name="export-to-onnx"></a>
PyTorch models are defined in a Python\* code, to export such models use `torch.onnx.export()` method. Usually code to
evaluate or test the model is provided with the model code and can be used to initialize and export model.
Only the basics will be covered here, the step to export to ONNX\* is crucial but it is covered by PyTorch\* framework.
For more information, please refer to [Exporting PyTorch models to ONNX format](https://pytorch.org/docs/stable/onnx.html).
## Exporting a PyTorch Model to ONNX Format <a name="export-to-onnx"></a>
PyTorch models are defined in Python. To export them, use the `torch.onnx.export()` method. The code to
evaluate or test the model is usually provided with its code and can be used for its initialization and export.
The export to ONNX is crucial for this process, but it is covered by PyTorch framework, therefore, It will not be covered here in detail.
For more information, refer to the [Exporting PyTorch models to ONNX format](https://pytorch.org/docs/stable/onnx.html) guide.
To export a PyTorch\* model you need to obtain the model as an instance of `torch.nn.Module` class and call the `export` function.
To export a PyTorch model, you need to obtain the model as an instance of `torch.nn.Module` class and call the `export` function.
*Not all PyTorch\* operations can be exported to ONNX\* opset 9 which is used by default, as of version 1.8.1.
*As of version 1.8.1, not all PyTorch operations can be exported to ONNX opset 9 which is used by default.
It is recommended to export models to opset 11 or higher when export to default opset 9 is not working. In that case, use `opset_version`
option of the `torch.onnx.export`. For more information about ONNX* opset, refer to the [Operator Schemas](https://github.com/onnx/onnx/blob/master/docs/Operators.md).
option of the `torch.onnx.export`. For more information about ONNX opset, refer to the [Operator Schemas](https://github.com/onnx/onnx/blob/master/docs/Operators.md) page.
# Converting a TensorFlow* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow}
This page gives instructions on how to convert a model from TensorFlow format to OpenVINO IR format using Model Optimizer. The instructions are different depending on if your model was created with TensorFlow v1.X or TensorFlow v2.X.
# Converting a TensorFlow Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow}
To use Model Optimizer, install OpenVINO Development Tools by following the [installation instructions here](https://docs.openvino.ai/latest/openvino_docs_install_guides_install_dev_tools.html).
This page provides general instructions on how to convert a model from a TensorFlow format to the OpenVINO IR format using Model Optimizer. The instructions are different depending on whether your model was created with TensorFlow v1.X or TensorFlow v2.X.
To use Model Optimizer, install OpenVINO Development Tools by following the [installation instructions](../../../install_guides/installing-model-dev-tools.md).
### Convert Frozen Model Format <a name="Convert_From_TF"></a>
To convert a TensorFlow model, use the `mo` script to simply convert a model with the path to the input model `.pb` file:
### Converting Frozen Model Format <a name="Convert_From_TF"></a>
To convert a TensorFlow model, use the *`mo`* script to simply convert a model with a path to the input model *`.pb`* file:
```sh
mo --input_model <INPUT_MODEL>.pb
```
### Convert Non-Frozen Model Formats <a name="loading-nonfrozen-models"></a>
### Converting Non-Frozen Model Formats <a name="loading-nonfrozen-models"></a>
There are three ways to store non-frozen TensorFlow models and convert them by Model Optimizer:
1.**Checkpoint**. In this case, a model consists of two files: `inference_graph.pb` (or `inference_graph.pbtxt`) and `checkpoint_file.ckpt`.
If you do not have an inference graph file, refer to [Freezing Custom Models in Python](#freeze-the-tensorflow-model).
To convert the model with the inference graph in `.pb` format, run the `mo` script with the path to the checkpoint file to convert a model:
If you do not have an inference graph file, refer to the [Freezing Custom Models in Python](#freeze-the-tensorflow-model) section.
To convert the model with the inference graph in `.pb` format, run the `mo` script with a path to the checkpoint file:
```sh
mo --input_model <INFERENCE_GRAPH>.pb --input_checkpoint <INPUT_CHECKPOINT>
```
To convert the model with the inference graph in `.pbtxt` format, run the `mo` script with the path to the checkpoint file to convert a model:
To convert the model with the inference graph in `.pbtxt` format, run the `mo` script with a path to the checkpoint file:
```sh
mo --input_model <INFERENCE_GRAPH>.pbtxt --input_checkpoint <INPUT_CHECKPOINT> --input_model_is_text
```
2.**MetaGraph**. In this case, a model consists of three or four files stored in the same directory: `model_name.meta`, `model_name.index`,
`model_name.data-00000-of-00001` (digit part may vary), and `checkpoint` (optional).
`model_name.data-00000-of-00001` (the numbers may vary), and `checkpoint` (optional).
To convert such TensorFlow model, run the `mo` script with a path to the MetaGraph `.meta` file:
```sh
mo --input_meta_graph <INPUT_META_GRAPH>.meta
@@ -45,11 +46,10 @@ If a model contains operations currently unsupported by OpenVINO, prune these op
To determine custom input nodes, display a graph of the model in TensorBoard. To generate TensorBoard logs of the graph, use the `--tensorboard_logs` option.
TensorFlow 2.x SavedModel format has a specific graph due to eager execution. In case of pruning, find custom input nodes in the `StatefulPartitionedCall/*` subgraph of TensorFlow 2.x SavedModel format.
### Freezing Custom Models in Python\* <a name="freeze-the-tensorflow-model"></a>
When a network is defined in Python\* code, you have to create an inference graph file. Usually graphs are built in a form
that allows model training. That means that all trainable parameters are represented as variables in the graph.
To be able to use such graph with Model Optimizer such graph should be frozen.
The graph is frozen and dumped to a file with the following code:
### Freezing Custom Models in Python <a name="freeze-the-tensorflow-model"></a>
When a network is defined in Python code, you have to create an inference graph file. Graphs are usually built in a form
that allows model training. That means all trainable parameters are represented as variables in the graph.
To be able to use such graph with Model Optimizer, it should be frozen and dumped to a file with the following code:
Then follow the above instructions for the SavedModel format.
> **NOTE**: Do not use other hacks to resave TensorFlow* 2 models into TensorFlow* 1 formats.
> **NOTE**: Do not use other hacks to resave TensorFlow 2 models into TensorFlow 1 formats.
## Command-Line Interface (CLI) Examples Using TensorFlow\*-Specific Parameters
## Command-Line Interface (CLI) Examples Using TensorFlow-Specific Parameters
* Launching the Model Optimizer for Inception V1 frozen model when model file is a plain text protobuf:
```sh
@@ -132,29 +132,29 @@ Then follow the above instructions for the SavedModel format.
mo --input_model inception_v1.pb -b 1 --tensorboard_logdir /tmp/log_dir
```
* Launching the Model Optimizer for BERT model in the SavedModel format, with three inputs. Explicitly specify input shapes
* Launching the Model Optimizer for BERT model in the SavedModel format, with three inputs. Specify explicitly the input shapes
where the batch size and the sequence length equal 2 and 30 respectively.
```sh
mo --saved_model_dir BERT --input mask,word_ids,type_ids --input_shape [2,30],[2,30],[2,30]
```
## Supported TensorFlow\* and TensorFlow 2 Keras\* Layers
Refer to [Supported Framework Layers ](../Supported_Frameworks_Layers.md) for the list of supported standard layers.
## Supported TensorFlow and TensorFlow 2 Keras Layers
For the list of supported standard layers, refer to the [Supported Framework Layers ](../Supported_Frameworks_Layers.md) page.
## Frequently Asked Questions (FAQ)
The Model Optimizer provides explanatory messages if it is unable to run to completion due to issues like typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md). The FAQ has instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
The Model Optimizer provides explanatory messages if it is unable to run to completion due to typographical errors, incorrectly used options, or other issues. The message describes the potential cause of the problem and gives a link to the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md). The FAQ provides instructions on how to resolve most issues. The FAQ also includes links to relevant sections in the Model Optimizer Developer Guide to help you understand what went wrong.
## Summary
In this document, you learned:
* Basic information about how the Model Optimizer works with TensorFlow models
* Which TensorFlow models formats are supported
* How to freeze a TensorFlow 1 model
* How to convert a trained TensorFlow model using the Model Optimizer with both framework-agnostic and TensorFlow-specific command-line options
* Basic information about how the Model Optimizer works with TensorFlow models.
* Which TensorFlow models are supported.
* How to freeze a TensorFlow model.
* How to convert a trained TensorFlow model using the Model Optimizer with both framework-agnostic and TensorFlow-specific command-line options.
## See Also
This page provided general instructions for converting TensorFlow models. For step-by-step instructions showing how to convert specific TensorFlow models, see the [Model Conversion Tutorials](Convert_Model_Tutorials.md) page. Here are some example tutorials:
## Additional Resources
For step-by-step instructions on how to convert specific TensorFlow models, see the [Model Conversion Tutorials](Convert_Model_Tutorials.md) page. Here are some examples:
This section provides you with a set of tutorials that demonstrate conversion steps for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models.
It contains conversion recipes for concrete models, that unnecessarily cover your case.
Try to convert the model out-of-the-box, meaning only the `--input_model` parameter is specified in the command line, before studying the tutorials.
This section provides a set of tutorials that demonstrate conversion methods for specific TensorFlow, ONNX, PyTorch, MXNet, and Kaldi models, that unnecessarily cover your case.
Before studying the tutorials, try to convert the model out-of-the-box by specifying only the `--input_model` parameter in the command line.
You can also find a collection of [Python tutorials](../../../tutorials.md) written for running on Jupyter* notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference.
You will find a collection of [Python tutorials](../../../tutorials.md) written for running on Jupyter notebooks that provide an introduction to the OpenVINO™ toolkit and explain how to use the Python API and tools for optimized deep learning inference.
Model Optimizer provides the option of making models more efficient by providing additional shape definition.
It is achieved with two parameters: `--input_shape` and `--static_shape`, used under certain conditions.
With Model Optimizer you can increase your model's efficiency by providing an additional shape definition, with these two parameters: `--input_shape` and `--static_shape`.
@anchor when_to_specify_input_shapes
## When to Specify --input_shape Command-line Parameter
Model Optimizer supports conversion of models with dynamic input shapes that contain undefined dimensions.
However, if the shape of data is not going to change from one inference to another,
However, if the shape of data is not going to change from one inference request to another,
it is recommended to set up static shapes (when all dimensions are fully defined) for the inputs.
It can be beneficial from a performance perspective and memory consumption.
Doing it at this stage, instead of during inference in runtime, can be beneficial in terms of performance and memory consumption.
To set up static shapes, Model Optimizer provides the `--input_shape` parameter.
The same functionality is also available in runtime via `reshape` method, please refer to [Changing input shapes](../../../OV_Runtime_UG/ShapeInference.md).
For more information about dynamic shapes in runtime, refer to [Dynamic Shapes](../../../OV_Runtime_UG/ov_dynamic_shapes.md)
For more information on input shapes under runtime, refer to the [Changing input shapes](../../../OV_Runtime_UG/ShapeInference.md) guide.
To learn more about dynamic shapes in runtime, refer to the [Dynamic Shapes](../../../OV_Runtime_UG/ov_dynamic_shapes.md) guide.
OpenVINO Runtime API can have limitations to infer models with undefined dimensions on some hardware (see [Features support matrix](../../../OV_Runtime_UG/supported_plugins/Device_Plugins.md) for reference).
The OpenVINO Runtime API may present certain limitations in inferring models with undefined dimensions on some hardware. See the [Features support matrix](../../../OV_Runtime_UG/supported_plugins/Device_Plugins.md) for reference.
In this case, the `--input_shape` parameter and the [reshape method](../../../OV_Runtime_UG/ShapeInference.md) can help to resolve undefined dimensions.
Sometimes Model Optimizer is unable to convert models out-of-the-box (only the `--input_model` parameter is specified).
Sometimes, Model Optimizer is unable to convert models out-of-the-box (only the `--input_model` parameter is specified).
Such problem can relate to models with inputs of undefined ranks and a case of cutting off parts of a model.
In this case, user has to specify input shapes explicitly using`--input_shape` parameter.
In this case, input shapes must be specified explicitly with the`--input_shape` parameter.
For example, run the Model Optimizer for the TensorFlow* MobileNet model with the single input
and specify input shape `[2,300,300,3]`.
For example, run Model Optimizer for the TensorFlow MobileNet model with the single input
and specify the input shape of `[2,300,300,3]`:
```sh
mo --input_model MobileNet.pb --input_shape [2,300,300,3]
```
If a model has multiple inputs, `--input_shape` must be used in conjunction with `--input` parameter.
The parameter `--input` contains a list of input names for which shapes in the same order are defined via `--input_shape`.
For example, launch the Model Optimizer for the ONNX* OCR model with a pair of inputs `data` and `seq_len`
and specify shapes `[3,150,200,1]` and `[3]` for them.
The `--input` parameter contains a list of input names, for which shapes in the same order are defined via `--input_shape`.
For example, launch Model Optimizer for the ONNX OCR model with a pair of inputs `data` and `seq_len`
and specify shapes `[3,150,200,1]` and `[3]` for them:
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [3,150,200,1],[3]
```
The alternative way to specify input shapes is to use the `--input` parameter as follows:
Alternatively, specify input shapes, using the `--input` parameter as follows:
```sh
mo --input_model ocr.onnx --input data[3150200 1],seq_len[3]
```
The parameter `--input_shape` allows overriding original input shapes to the shapes compatible with a given model.
Dynamic shapes, i.e. with dynamic dimensions, in the original model can be replaced with static shapes for the converted model, and vice versa.
The dynamic dimension can be marked in Model Optimizer command-line as `-1` or `?`.
For example, launch the Model Optimizer for the ONNX* OCR model and specify dynamic batch dimension for inputs.
The `--input_shape` parameter allows overriding original input shapes to ones compatible with a given model.
Dynamic shapes, i.e. with dynamic dimensions, can be replaced in the original model with static shapes for the converted model, and vice versa.
The dynamic dimension can be marked in Model Optimizer command-line as `-1`* or *`?`.
For example, launch Model Optimizer for the ONNX OCR model and specify dynamic batch dimension for inputs:
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [-1,150,200,1],[-1]
To optimize memory consumption for models with undefined dimensions in run-time, Model Optimizer provides the capability to define boundaries of dimensions.
The boundaries of undefined dimension can be specified with ellipsis.
For example, launch the Model Optimizer for the ONNX* OCR model and specify a boundary for the batch dimension.
For example, launch Model Optimizer for the ONNX OCR model and specify a boundary for the batch dimension:
```sh
mo --input_model ocr.onnx --input data,seq_len --input_shape [1..3,150,200,1],[1..3]
Practically, some models are not ready for input shapes change.
In this case, a new input shape cannot be set via Model Optimizer.
Learn more about shape [inference troubleshooting](@ref troubleshooting_reshape_errors) and [ways to relax shape inference flow](@ref how-to-fix-non-reshape-able-model).
For more information about shape follow the [inference troubleshooting](@ref troubleshooting_reshape_errors) and [ways to relax shape inference flow](@ref how-to-fix-non-reshape-able-model) guides.
## When to Specify --static_shape Command-line Parameter
Model Optimizer provides the `--static_shape` parameter that allows evaluating shapes of all operations in the model for fixed input shapes
and to fold shape computing sub-graphs into constants. The resulting IR can be more compact in size and the loading time for such IR can be decreased.
and folding shape computing sub-graphs into constants. The resulting IR may be more compact in size and the loading time for such IR may decrease.
However, the resulting IR will not be reshape-able with the help of the [reshape method](../../../OV_Runtime_UG/ShapeInference.md) from OpenVINO Runtime API.
It is worth noting that the `--input_shape` parameter does not affect reshape-ability of the model.
It is worth noting that the `--input_shape` parameter does not affect reshapeability of the model.
For example, launch the Model Optimizer for the ONNX* OCR model using `--static_shape`.
For example, launch Model Optimizer for the ONNX OCR model using `--static_shape`:
```sh
mo --input_model ocr.onnx --input data[3150200 1],seq_len[3] --static_shape
# Cutting Off Parts of a Model {#openvino_docs_MO_DG_prepare_model_convert_model_Cutting_Model}
Sometimes some parts of a model must be removed while the Model Optimizer is converting models to the Intermediate Representation. This chapter describes methods of doing cutting off parts of a model using Model Optimizer command-line options. Model cutting applies mostly to TensorFlow\* models, but is also useful for other frameworks. In this chapter, TensorFlow examples are used for illustration.
Sometimes, it is necessary to remove parts of a model when converting it to OpenVINO IR. This chapter describes how to do it, using Model Optimizer command-line options. Model cutting applies mostly to TensorFlow models, which is why TensorFlow will be used in this chapter's examples, but it may be also useful for other frameworks.
## Purpose of Model Cutting
The following examples are the situations when model cutting is useful or even required:
* model has pre- or post-processing parts that cannot be translated to existing OpenVINO operations.
* model has a training part that is convenient to be kept in the model, but not used during inference.
* model is too complex (contains lots of unsupported operations that cannot be easily implemented as custom layers), so the complete model cannot be converted in one shot.
* problem with model conversion in the Model Optimizer or inference in the OpenVINO Runtime occurred. To localize the issue, limit the scope for conversion by iteratively searching for problematic places in the model.
* single custom layer or a combination of custom layers is isolated for debugging purposes.
*A model has pre- or post-processing parts that cannot be translated to existing OpenVINO operations.
*A model has a training part that is convenient to be kept in the model but not used during inference.
*A model is too complex be converted at once, because it contains a lot of unsupported operations that cannot be easily implemented as custom layers.
*A problem occurs with model conversion in Model Optimizer or inference in OpenVINO™ Runtime. To identify the issue, limit the conversion scope by iterative search for problematic areas in the model.
*A single custom layer or a combination of custom layers is isolated for debugging purposes.
## Command-Line Options
@@ -19,21 +19,21 @@ Model Optimizer provides command line options `--input` and `--output` to specif
*`--input` option accepts a comma-separated list of layer names of the input model that should be treated as new entry points to the model.
*`--output` option accepts a comma-separated list of layer names of the input model that should be treated as new exit points from the model.
The `--input` option is required for cases unrelated to model cutting. For example, when the model contains several inputs and `--input_shape` or `--mean_values` options are used, you should use the `--input` option to specify the order of input nodes for correct mapping between multiple items provided in `--input_shape` and `--mean_values` and the inputs in the model. Details on these options are out of scope for this document, which focuses on model cutting.
The `--input` option is required for cases unrelated to model cutting. For example, when the model contains several inputs and `--input_shape` or `--mean_values` options are used, the `--input` option specifies the order of input nodes for correct mapping between multiple items provided in `--input_shape` and `--mean_values` and the inputs in the model.
Model cutting is illustrated with Inception V1. This model is in`models/research/slim` repository. [This section](Converting_Model.md) describes pre-work to prepare the model for the Model Optimizer to be ready to proceed with this chapter.
Model cutting is illustrated with the Inception V1 model, found in the`models/research/slim` repository. To proceed with this chapter, make sure you do the necessary steps to [prepare the model for Model Optimizer](Converting_Model.md).
## Default Behavior without --input and --output
The input model is converted as a whole if neither `--input` nor `--output` command line options are used. All `Placeholder` operations in a TensorFlow\* graph are automatically identified as entry points. The `Input` layer type is generated for each of them. All nodes that have no consumers are automatically identified as exit points.
The input model is converted as a whole if neither `--input` nor `--output` command line options are used. All `Placeholder` operations in a TensorFlow graph are automatically identified as entry points. The `Input` layer type is generated for each of them. All nodes that have no consumers are automatically identified as exit points.
For Inception_V1, there is one `Placeholder`: input. If the model is viewed in the TensorBoard\*, the input operation is easy to find:
For Inception_V1, there is one `Placeholder`: input. If the model is viewed in TensorBoard, the input operation is easy to find:

There is only one output operation, which enclosed in a nested name scope `InceptionV1/Logits/Predictions`, the `Reshape` operation has a full name `InceptionV1/Logits/Predictions/Reshape_1`.
`Reshape` is the only output operation, which is enclosed in a nested name scope of `InceptionV1/Logits/Predictions`, under the full name of `InceptionV1/Logits/Predictions/Reshape_1`.
In the TensorBoard, it looks the following way together with some predecessors:
In TensorBoard, along with some of its predecessors, it looks as follows:

@@ -41,7 +41,7 @@ Convert this model and put the results in a writable output directory:
```sh
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
```
(The other examples on this page assume that you first cd to the `model_optimizer` directory and add the `--output_dir` argument with a directory where you have write permissions.)
(The other examples on this page assume that you first go to the `model_optimizer` directory and add the `--output_dir` argument with a directory where you have read/write permissions.)
The output `.xml` file with an Intermediate Representation contains the `Input` layer among other layers in the model:
```xml
@@ -78,7 +78,7 @@ The last layer in the model is `InceptionV1/Logits/Predictions/Reshape_1`, which
</output>
</layer>
```
Due to automatic identification of inputs and outputs, you do not need to provide the `--input` and `--output` options to convert the whole model. The following commands are equivalent for the Inception V1 model:
Due to automatic identification of inputs and outputs, providing the `--input` and `--output` options to convert the whole model is not required. The following commands are equivalent for the Inception V1 model:
```sh
mo --input_model inception_v1.pb -b 1 --output_dir <OUTPUT_MODEL_DIR>
@@ -88,7 +88,7 @@ The Intermediate Representations are identical for both conversions. The same is
## Model Cutting
Now consider how to cut some parts of the model off. This chapter uses the first convolution block `InceptionV1/InceptionV1/Conv2d_1a_7x7` of the Inception V1 model to illustrate cutting:
Now, consider how to cut some parts of the model off. This chapter describes the first convolution block `InceptionV1/InceptionV1/Conv2d_1a_7x7` of the Inception V1 model to illustrate cutting:

@@ -138,7 +138,7 @@ If you want to cut your model at the end, you have the following options:
</edges>
</net>
```
As you can see in the TensorBoard picture, the original model has more nodes than Intermediate Representation. Model Optimizer has fused batch normalization `InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm` to the convolution `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`, and it is not present in the final Intermediate Representation. This is not an effect of the `--output` option, it is usual behavior of the Model Optimizer for batch normalizations and convolutions. The effect of the `--output` is that the `ReLU` layer becomes the last one in the converted model.
As shown in the TensorBoard picture, the original model has more nodes than its Intermediate Representation. Model Optimizer has fused batch normalization `InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm` with convolution `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`, which is why it is not present in the final model. This is not an effect of the `--output` option, it is the typical behavior of Model Optimizer for batch normalizations and convolutions. The effect of the `--output` is that the `ReLU` layer becomes the last one in the converted model.
2. The following command cuts the edge that comes from 0 output port of the `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` and the rest of the model, making this node the last one in the model:
```sh
@@ -182,7 +182,7 @@ If you want to cut your model at the end, you have the following options:
</edges>
</net>
```
This type of cutting is useful to cut edges in case of multiple output edges.
This type of cutting is useful for cutting multiple output edges.
3. The following command cuts the edge that comes to 0 input port of the `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` and the rest of the model including `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu`, deleting this node and making the previous node `InceptionV1/InceptionV1/Conv2d_1a_7x7/Conv2D` the last in the model:
```sh
@@ -222,7 +222,7 @@ If you want to cut your model at the end, you have the following options:
If you want to go further and cut the beginning of the model, leaving only the `ReLU` layer, you have the following options:
1. You can use the following command line, where `--input` and `--output` specify the same node in the graph:
1. Use the following command line, where `--input` and `--output` specify the same node in the graph:
```sh
mo --input_model=inception_v1.pb -b 1 --output InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --input InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu --output_dir <OUTPUT_MODEL_DIR>
```
@@ -250,11 +250,11 @@ If you want to go further and cut the beginning of the model, leaving only the `
</edges>
</net>
```
`Input` layer is automatically created to feed the layer that is converted from the node specified in `--input`, which is `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` in this case. Model Optimizer does not replace the `ReLU` node by the `Input` layer, it produces such Intermediate Representation to make the node be the first executable node in the final Intermediate Representation. So the Model Optimizer creates enough `Inputs` to feed all input ports of the node that is passed in `--input`.<br>
Even though `--input_shape` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow* model to the point at which the new input is defined. It has the same shape [1,64,112,112] as the model converted as a whole or without cutting off the beginning.
`Input` layer is automatically created to feed the layer that is converted from the node specified in `--input`, which is `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` in this case. Model Optimizer does not replace the `ReLU` node by the `Input` layer. It produces such Intermediate Representation to make the node the first executable node in the final Intermediate Representation. Therefore, Model Optimizer creates enough `Inputs` to feed all input ports of the node that is passed in `--input`.<br>
Even though `--input_shape` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape [1,64,112,112] as the model converted as a whole or without cutting off the beginning.
2. You can cut edge incoming to layer by port number. To specify incoming port use notation `--input=port:input_node`.
So, to cut everything before `ReLU` layer, cut edge incoming in port 0 of `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` node:
2. Cut the edge incoming to layer by port number. To specify the incoming port, use the following notation `--input=port:input_node`.
To cut everything before `ReLU` layer, cut the edge incoming to port 0 of `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` node:
@@ -282,11 +282,11 @@ So, to cut everything before `ReLU` layer, cut edge incoming in port 0 of `Incep
</edges>
</net>
```
`Input` layer is automatically created to feed the layer that is converted from the node specified in `--input`, which is `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` in this case. Model Optimizer does not replace the `ReLU` node by the `Input` layer, it produces such Intermediate Representation to make the node be the first executable node in the final Intermediate Representation. So the Model Optimizer creates enough `Inputs` to feed all input ports of the node that is passed in `--input`.<br>
Even though `--input_shape` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow* model to the point at which the new input is defined. It has the same shape [1,64,112,112] as the model converted as a whole or without cutting off the beginning.
`Input` layer is automatically created to feed the layer that is converted from the node specified in `--input`, which is `InceptionV1/InceptionV1/Conv2d_1a_7x7/Relu` in this case. Model Optimizer does not replace the `ReLU` node by the `Input` layer, it produces such Intermediate Representation to make the node be the first executable node in the final Intermediate Representation. Therefore, Model Optimizer creates enough `Inputs` to feed all input ports of the node that is passed in `--input`.<br>
Even though `--input_shape` is not specified in the command line, the shapes for layers are inferred from the beginning of the original TensorFlow model to the point, at which the new input is defined. It has the same shape [1,64,112,112] as the model converted as a whole or without cutting off the beginning.
3. You can cut edge outcoming from layer by port number. To specify outcoming port use notation `--input=input_node:port`.
So, to cut everything before `ReLU` layer, cut edge from `InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1` node to `ReLU`:
3. Cut edge outcoming from layer by port number. To specify the outcoming port, use the following notation `--input=input_node:port`.
To cut everything before `ReLU` layer, cut edge from `InceptionV1/InceptionV1/Conv2d_1a_7x7/BatchNorm/batchnorm/add_1` node to `ReLU`:
@@ -354,18 +354,18 @@ gives the following shapes in the `Input` and `ReLU` layers:
</output>
</layer>
```
An input shape [1,20,5,10] in the final Intermediate Representation differs from the shape [1,5,10,20] specified in the command line, because the original TensorFlow\* model uses NHWC layout, but the Intermediate Representation uses NCHW layout. So usual NHWC to NCHW layout conversion occurred.
An input shape [1,20,5,10] in the final Intermediate Representation differs from the shape [1,5,10,20] specified in the command line, because the original TensorFlow model uses NHWC layout, but the Intermediate Representation uses NCHW layout. Thus, usual NHWC to NCHW layout conversion occurred.
When `--input_shape` is specified, shape inference inside the Model Optimizer is not performed for the nodes in the beginning of the model that are not included in the translated region. It differs from the case when `--input_shape` is not specified as noted in the previous section where the shape inference is still performed for such nodes to deduce shape for the layers that should fall into the final Intermediate Representation. So `--input_shape` should be used for a model with a complex graph with loops, which are not supported by the Model Optimizer, to exclude such parts from the Model Optimizer shape inference process completely.
When `--input_shape` is specified, shape inference inside Model Optimizer is not performed for the nodes in the beginning of the model that are not included in the translated region. It differs from the case when `--input_shape` is not specified as noted in the previous section, where the shape inference is still performed for such nodes to deduce shape for the layers that should fall into the final Intermediate Representation. Therefore, `--input_shape` should be used for a model with a complex graph with loops, which are not supported by Model Optimizer, to exclude such parts from the Model Optimizer shape inference process completely.
## Inputs with Multiple Input Ports
There are operations that contain more than one input ports. In the example considered here, the convolution `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution` is such operation. When `--input_shape` is not provided, a new `Input` layer is created for each dynamic input port for the node. If a port is evaluated to a constant blob, this constant remains in the model and a corresponding input layer is not created. TensorFlow convolution used in this model contains two ports:
There are operations that contain more than one input port. In the example considered here, the convolution `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution` is such operation. When `--input_shape` is not provided, a new `Input` layer is created for each dynamic input port for the node. If a port is evaluated to a constant blob, this constant remains in the model and a corresponding input layer is not created. TensorFlow convolution used in this model contains two ports:
* port 0: input tensor for convolution (dynamic)
* port 1: convolution weights (constant)
Following this behavior, the Model Optimizer creates an `Input` layer for port 0 only, leaving port 1 as a constant. So the result of:
Following this behavior, Model Optimizer creates an `Input` layer for port 0 only, leaving port 1 as a constant. Thus, the result of:
```sh
mo --input_model inception_v1.pb -b 1 --input InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --output_dir <OUTPUT_MODEL_DIR>
@@ -377,13 +377,13 @@ Different behavior occurs when `--input_shape` is also used as an attempt to ove
```sh
mo --input_model inception_v1.pb--input=InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution --input_shape [1,224,224,3] --output_dir <OUTPUT_MODEL_DIR>
```
An error occurs (for more information, see <a href="MO_FAQ.html#FAQ30">FAQ #30</a>):
An error occurs (for more information, see the [Model Optimizer FAQ](../Model_Optimizer_FAQ.md#FAQ30)):
```sh
[ ERROR ] Node InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution has more than 1 input and input shapes were provided.
Try not to provide input shapes or specify input port with PORT:NODE notation, where PORT is an integer.
For more information, see FAQ #30
```
In this case, when `--input_shape` is specified and the node contains multiple input ports, you need to specify an input port index together with an input node name. The input port index is specified in front of the node name with ':' as a separator (`PORT:NODE`). In the considered case, the port index 0 of the node `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution` should be specified as `0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`.
When `--input_shape` is specified and the node contains multiple input ports, you need to provide an input port index together with an input node name. The input port index is specified in front of the node name with ':' as a separator (`PORT:NODE`). In this case, the port index 0 of the node `InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution` should be specified as `0:InceptionV1/InceptionV1/Conv2d_1a_7x7/convolution`.
OpenVINO Runtime CPU and GPU devices can infer models in the low precision.
For details, refer to [Model Optimization Guide](@ref openvino_docs_model_optimization_guide).
OpenVINO Runtime CPU and GPU devices can infer models in low precision.
For more details, refer to the [Model Optimization Guide](@ref openvino_docs_model_optimization_guide).
Intermediate Representation (IR) should be specifically formed to be suitable for low precision inference.
Such an IR is called a Low Precision IR and you can generate it in two ways:
- [Quantize regular IR with the Post-Training Optimization tool](@ref pot_introduction)
- Use the Model Optimizer for a model pretrained for Low Precision inference: TensorFlow\* pre-TFLite models (`.pb` model file with `FakeQuantize*` operations) and ONNX\* quantized models.
Both TensorFlow and ONNX quantized models could be prepared by [Neural Network Compression Framework](https://github.com/openvinotoolkit/nncf/blob/develop/README.md).
Intermediate Representation should be specifically formed to be suitable for low precision inference.
Such a model is called a Low Precision IR and can be generated in two ways:
- By [quantizing regular IR with the Post-Training Optimization tool](@ref pot_introduction)
- Using Model Optimizer for a model pre-trained for Low Precision inference: TensorFlow pre-TFLite models (`.pb` model file with `FakeQuantize*` operations) and ONNX quantized models.
Both TensorFlow and ONNX quantized models can be prepared by [Neural Network Compression Framework](https://github.com/openvinotoolkit/nncf/blob/develop/README.md).
For an operation to be executed in INT8, it must have `FakeQuantize` operations as inputs.
See the [specification of `FakeQuantize` operation](../../../ops/quantization/FakeQuantize_1.md) for details.
For more details, see the [specification of `FakeQuantize` operation](../../../ops/quantization/FakeQuantize_1.md).
To execute the `Convolution` operation in INT8 on CPU, both data and weight inputs should have `FakeQuantize` as an input operation:
Low precision IR is also suitable for FP32 and FP16 inference if a chosen plugin supports all operations of the IR, because the only difference between a Low Precision IR and FP16 or FP32 IR is the existence of `FakeQuantize` in the Low Precision IR.
Plugins with Low Precision Inference support recognize these sub-graphs and quantize them during the inference time.
Plugins without Low Precision support execute all operations, including `FakeQuantize`, as is in the FP32 or FP16 precision.
Low precision IR is also suitable for FP32 and FP16 inference if a chosen plugin supports all operations of the IR. The only difference between a Low Precision IR and FP16 or FP32 IR is the existence of `FakeQuantize` in the Low Precision IR.
Plugins that support Low Precision Inference recognize these sub-graphs and quantize them during inference.
The ones that do not, execute all operations, including `FakeQuantize`, as is in the FP32 or FP16 precision.
Accordingly, the presence of FakeQuantize operations in the IR is a recommendation for a plugin on how to quantize particular operations in the model.
If capable, a plugin accepts the recommendation and performs Low Precision Inference, otherwise, the plugin ignores the recommendation and executes a model in the floating-point precision.
Consequently, when `FakeQuantize` operations are present in an OpenVINO IR, it suggests to the inference device how to quantize particular operations in the model.
If the device is capable, it accepts the suggestion and performs Low Precision Inference. If not, it executes the model in the floating-point precision.
## Compressed Low Precision Weights
Weighted operations, like`Convolution`,`MatMul`, and others, store weights as floating-point `Constant` in the graph followed by the `FakeQuantize` operation.
`Constant` followed by the `FakeQuantize` operation could be optimized memory-wise due to the `FakeQuantize` operation semantics.
Weighted operations, such as`Convolution` and`MatMul`, store weights as the floating-point `Constant` in the graph followed by the `FakeQuantize` operation.
The `Constant` followed by the `FakeQuantize` operation could be optimized memory-wise due to the `FakeQuantize` operation semantics.
The resulting weights sub-graph stores weights in Low Precision `Constant`, which gets unpacked back to floating point with the `Convert` operation.
Weights compression replaces `FakeQuantize` with optional `Subtract` and `Multiply` operation leaving output arithmetically the same and weights storing takes four times less memory.
# Convert Kaldi* ASpIRE Chain Time Delay Neural Network (TDNN) Model {#openvino_docs_MO_DG_prepare_model_convert_model_kaldi_specific_Aspire_Tdnn_Model}
# Converting a Kaldi ASpIRE Chain Time Delay Neural Network (TDNN) Model {#openvino_docs_MO_DG_prepare_model_convert_model_kaldi_specific_Aspire_Tdnn_Model}
You can [download a pre-trained model](https://kaldi-asr.org/models/1/0001_aspire_chain_model.tar.gz)
for the ASpIRE Chain Time Delay Neural Network (TDNN) from the Kaldi* project official website.
At the beginning, you should [download a pre-trained model](https://kaldi-asr.org/models/1/0001_aspire_chain_model.tar.gz)
for the ASpIRE Chain Time Delay Neural Network (TDNN) from the Kaldi project official website.
## Convert ASpIRE Chain TDNN Model to IR
## Converting an ASpIRE Chain TDNN Model to IR
To generate the Intermediate Representation (IR) of the model, run the Model Optimizer with the following parameters:
Generate the Intermediate Representation of the model by running Model Optimizer with the following parameters:
```sh
mo --input_model exp/chain/tdnn_7b/final.mdl --output output
```
The IR will have two inputs: `input` for data and `ivector` for ivectors.
The IR will have two inputs: `input` for data, and `ivector` for ivectors.
## Example: Run ASpIRE Chain TDNN Model with the Speech Recognition Sample
## Example: Running ASpIRE Chain TDNN Model with the Speech Recognition Sample
These instructions show how to run the converted model with the [Speech Recognition sample](../../../../../samples/cpp/speech_sample/README.md).
In this example, the input data contains one utterance from one speaker.
> **NOTE**: Before you continue with this part of the article, get familiar with the [Speech Recognition sample](../../../../../samples/cpp/speech_sample/README.md).
To follow the steps described below, you must first do the following:
In this example, the input data contains one utterance from one speaker.
To run the ASpIRE Chain TDNN Model with Speech Recognition sample, You need to prepare environment. Do it by following the steps below :
1. Download a [Kaldi repository](https://github.com/kaldi-asr/kaldi).
2. Build it using instructions in `README.md`in the repository.
2. Build it by following instructions in `README.md`from the repository.
3. Download the [model archive](https://kaldi-asr.org/models/1/0001_aspire_chain_model.tar.gz) from Kaldi website.
4. Extract the downloaded model archive to the `egs/aspire/s5` folder of the Kaldi repository.
To run the ASpIRE Chain TDNN Model with Speech Recognition sample:
Once everything has been prepared, you can start a proper run:
1. Prepare the model for decoding. Refer to the `README.txt` file from the downloaded model archive for instructions.
2. Convert data and ivectors to `.ark` format. Refer to the corresponding sections below for instructions.
### Prepare Data
### Preparing Data
If you have a `.wav` data file, you can convert it to `.ark` format using the following command:
If you have a `.wav` data file, convert it to the `.ark` format using the following command:
To simplify the preparation of ivectors for the Speech Recognition sample,
specify the maximum number of frames in utterances as a parameter for `--ivector_period`
You can simplify the preparation of ivectors for the Speech Recognition sample. To do it, specify the maximum number of frames in utterances as a parameter for `--ivector_period`
to get only one ivector per utterance.
To get the maximum number of frames in utterances, you can use the following command line:
To get the maximum number of frames in utterances, use the following command line:
This document provides the instructions and examples on how to use Model Optimizer to convert [GluonCV SSD and YOLO-v3 models](https://gluon-cv.mxnet.io/model_zoo/detection.html) to IR.
This article provides the instructions and examples on how to use Model Optimizer to convert [GluonCV SSD and YOLO-v3 models](https://gluon-cv.mxnet.io/model_zoo/detection.html) to IR.
1. Choose the topology available from the [GluonCV Model Zoo](https://gluon-cv.mxnet.io/model_zoo/detection.html) and export to the MXNet format using the GluonCV API. For example, for the `ssd_512_mobilenet1.0` topology:
```python
@@ -10,7 +10,7 @@ net = model_zoo.get_model('ssd_512_mobilenet1.0_voc', pretrained=True)
As a result, you will get an MXNet model representation in `ssd_512_mobilenet1.0.params` and `ssd_512_mobilenet1.0.json` files generated in the current directory.
2. Run the Model Optimizer tool specifying the `--enable_ssd_gluoncv` option. Make sure the `--input_shape` parameter is set to the input shape layout of your model (NHWC or NCHW). The examples below illustrates running the Model Optimizer for the SSD and YOLO-v3 models trained with the NHWC layout and located in the `<model_directory>`:
2. Run the Model Optimizer tool, specifying the `--enable_ssd_gluoncv` option. Make sure the `--input_shape` parameter is set to the input shape layout of your model (NHWC or NCHW). The examples below illustrate running the Model Optimizer for the SSD and YOLO-v3 models trained with the NHWC layout and located in the `<model_directory>`:
* **For GluonCV SSD topologies:**
```sh
mo --input_model <model_directory>/ssd_512_mobilenet1.0.params --enable_ssd_gluoncv --input_shape [1,512,512,3] --input data --output_dir <OUTPUT_MODEL_DIR>
# Convert MXNet Style Transfer Model {#openvino_docs_MO_DG_prepare_model_convert_model_mxnet_specific_Convert_Style_Transfer_From_MXNet}
# Converting an MXNet Style Transfer Model {#openvino_docs_MO_DG_prepare_model_convert_model_mxnet_specific_Convert_Style_Transfer_From_MXNet}
The tutorial explains how to generate a model for style transfer using the public MXNet\* neural style transfer sample.
To use the style transfer sample from OpenVINO™, follow the steps below as no public pre-trained style transfer model is provided with the OpenVINO toolkit.
This article provides instructions on how to generate a model for style transfer, using the public MXNet neural style transfer sample.
#### 1. Download or clone the repository with an MXNet neural style transfer sample: [Zhaw's Neural Style Transfer repository](https://github.com/zhaw/neural_style).
**Step 1**: Download or clone the repository [Zhaw's Neural Style Transfer repository](https://github.com/zhaw/neural_style) with an MXNet neural style transfer sample.
#### 2. Prepare the environment required to work with the cloned repository:
1. Install packages dependency:<br>
**Step 2**: Prepare the environment required to work with the cloned repository:
> **NOTE**: Python-tk installation is needed only for Linux. Python for Windows includes it by default.
1. Install packages dependency.<br>
```sh
sudo apt-get install python-tk
```
Installing python-tk step is needed only for Linux, as it is included by default in Python\* for Windows\*.
2. Install Python\* requirements:
2.Install Python requirements:
```sh
pip3 install --user mxnet
pip3 install --user matplotlib
pip3 install --user scikit-image
```
#### 3. Download the pre-trained [VGG19 model](https://github.com/dmlc/web-data/raw/master/mxnet/neural-style/model/vgg19.params) and save it to the root directory of the cloned repository because the sample expects the model `vgg19.params` file to be in that directory.<br>
**Step 3**: Download the pretrained [VGG19 model](https://github.com/dmlc/web-data/raw/master/mxnet/neural-style/model/vgg19.params) and save it to the root directory of the cloned repository. The sample expects the model `vgg19.params` file to be in that directory.<br>
#### 4. Modify source code files of style transfer sample from cloned repository.<br>
**Step 4**: Modify source code files of style transfer sample from the cloned repository:<br>
1. Go to the `fast_mrf_cnn` subdirectory.
```sh
cd ./fast_mrf_cnn
```
2. Open the `symbol.py` file and modify the `decoder_symbol()` function. Replace.
2. Open the `symbol.py` file and modify the `decoder_symbol()` function. You should see the following code there:
#### 5. Run the sample with a decoder model according to the instructions from the `README.md` file in the `fast_mrf_cnn` directory of the cloned repository.
For example, to run the sample with the pre-trained decoder weights from the `models` folder and output shape, use the following code:<br>
**Step 5**: Follow the instructions from the `README.md` file in the `fast_mrf_cnn` directory of the cloned repository and run the sample with a decoder model.
For example, use the following code to run the sample with the pretrained decoder weights from the `models` folder and output shape:<br>
Where the `models/13` string is composed of the following substrings:
*`models/`: path to the folder that contains .nd files with pre-trained styles weights
*`13`: prefix pointing to 13_decoder, which is the default decoder for the repository.
The `models/13` string in the code above is composed of the following substrings:
* `models/` -- path to the folder that contains `.nd` files with pretrained styles weights.
* `13` -- prefix pointing to the default decoder for the repository, `13_decoder`.
> **NOTE**: If you get an error saying "No module named 'cPickle'", try running the script from this step in Python 2. Then return to Python 3 for the remaining steps.
> **NOTE**: If an error prompts with "No module named `cPickle`", try running the script from Step 5 in Python 2. After that return to Python 3 for the remaining steps.
You can choose any style from [collection of pre-trained weights](https://pan.baidu.com/s/1skMHqYp). (On the Chinese-language page, click the down arrow next to a size in megabytes. Then wait for an overlay box to appear, and click the blue button in it to download.) The `generate()` function generates `nst_vgg19-symbol.json` and `vgg19-symbol.json` files for the specified shape. In the code, it is [1024 x 768] for a 4:3 ratio, and you can specify another, for example, [224,224] for a square ratio.
Any style can be selected from [collection of pretrained weights](https://pan.baidu.com/s/1skMHqYp). On the Chinese-language page, click the down arrow next to a size in megabytes. Then wait for an overlay box to appear, and click the blue button in it to download. The `generate()` function generates `nst_vgg19-symbol.json` and `vgg19-symbol.json` files for the specified shape. In the code, it is [1024 x 768] for a 4:3 ratio. You can specify another, for example, [224,224] for a square ratio.
#### 6. Run the Model Optimizer to generate an Intermediate Representation (IR):
**Step 6**: Run the Model Optimizer to generate an Intermediate Representation (IR):
1. Create a new directory. For example:<br>
```sh
mkdir nst_model
```
2. Copy the initial and generated model files to the created directory. For example, to copy the pre-trained decoder weights from the `models` folder to the `nst_model` directory, run the following commands:<br>
2. Copy the initial and generated model files to the created directory. For example, to copy the pretrained decoder weights from the `models` folder to the `nst_model` directory, run the following commands:<br>
> **NOTE**: Make sure that all the `.params` and `.json` files are in the same directory as the `.nd` files. Otherwise, the conversion process fails.
3. Run the Model Optimizer for MXNet. Use the `--nd_prefix_name` option to specify the decoder prefix and `--input_shape` to specify input shapes in [N,C,W,H] order. For example:<br>
3. Run the Model Optimizer for Apache MXNet. Use the `--nd_prefix_name` option to specify the decoder prefix and `--input_shape` to specify input shapes in [N,C,W,H] order. For example:<br>
# Convert ONNX* Faster R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Faster_RCNN}
# Converting an ONNX Faster R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Faster_RCNN}
These instructions are applicable only to the Faster R-CNN model converted to the ONNX* file format from the [facebookresearch/maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark).
The instructions below are applicable **only** to the Faster R-CNN model converted to the ONNX file format from the [maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark):
**Step 1**. Download the pre-trained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn) (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117).
1. Download the pretrained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn):
**Step 2**. To generate the Intermediate Representation (IR) of the model, change your current working directory to the Model Optimizer installation directory and run the Model Optimizer with the following parameters:
2. Generate the Intermediate Representation of the model, by changing your current working directory to the Model Optimizer installation directory, and running Model Optimizer with the following parameters:
```sh
mo \
--input_model FasterRCNN-10.onnx \
@@ -14,6 +15,9 @@ These instructions are applicable only to the Faster R-CNN model converted to th
Note that the height and width specified with the `input_shape` command line parameter could be different. Refer to the [documentation](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn) for more information about supported input image dimensions and required pre- and post-processing steps.
Be aware that the height and width specified with the `input_shape` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the [Faster R-CNN article](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/faster-rcnn).
**Step 3**. Interpret the outputs. The generated IR file has several outputs: class indices, probabilities and box coordinates. These are outputs from the "DetectionOutput" layer.
3. Interpret the outputs of the generated IR: class indices, probabilities and box coordinates. Below are the outputs from the "DetectionOutput" layer:
# Convert ONNX* GPT-2 Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_GPT2}
# Converting an ONNX GPT-2 Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_GPT2}
[Public pre-trained GPT-2 model](https://github.com/onnx/models/tree/master/text/machine_comprehension/gpt-2) is a large
[Public pretrained GPT-2 model](https://github.com/onnx/models/tree/master/text/machine_comprehension/gpt-2) is a large
transformer-based language model with a simple objective: predict the next word, given all of the previous words within some text.
## Download the Pre-Trained Base GPT-2 Model
## Downloading the Pre-Trained Base GPT-2 Model
To download the model, click **Download** on [https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.onnx](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.onnx).
To download the model, go to [this model](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.onnx), and press **Download**.
To download the model and sample test data, click **Download** on [https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.tar.gz](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.tar.gz).
To download the model and sample test data, go to [this model](https://github.com/onnx/models/blob/master/text/machine_comprehension/gpt-2/model/gpt2-10.tar.gz), and press **Download**.
## Convert ONNX* GPT-2 Model to IR
## Converting an ONNX GPT-2 Model to IR
To generate the Intermediate Representation (IR) of the model GPT-2, run the Model Optimizer with the following parameters:
Generate the Intermediate Representation of the model GPT-2 by running Model Optimizer with the following parameters:
```sh
mo --input_model gpt2-10.onnx --input_shape [X,Y,Z] --output_dir <OUTPUT_MODEL_DIR>
# Convert ONNX* Mask R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Mask_RCNN}
# Converting an ONNX Mask R-CNN Model {#openvino_docs_MO_DG_prepare_model_convert_model_onnx_specific_Convert_Mask_RCNN}
These instructions are applicable only to the Mask R-CNN model converted to the ONNX* file format from the [facebookresearch/maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark).
The instructions below are applicable **only** to the Mask R-CNN model converted to the ONNX file format from the [maskrcnn-benchmark model](https://github.com/facebookresearch/maskrcnn-benchmark).
**Step 1**. Download the pre-trained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn) (commit-SHA: 8883e49e68de7b43e263d56b9ed156dfa1e03117).
1. Download the pretrained model file from [onnx/models](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn):
**Step 2**. To generate the Intermediate Representation (IR) of the model, change your current working directory to the Model Optimizer installation directory and run the Model Optimizer with the following parameters:
2. Generate the Intermediate Representation of the model by changing your current working directory to the Model Optimizer installation directory and running Model Optimizer with the following parameters:
```sh
mo \
--input_model mask_rcnn_R_50_FPN_1x.onnx \
@@ -14,6 +15,12 @@ These instructions are applicable only to the Mask R-CNN model converted to the
Note that the height and width specified with the `input_shape` command line parameter could be different. Refer to the [documentation](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn) for more information about supported input image dimensions and required pre- and post-processing steps.
Be aware that the height and width specified with the `input_shape` command line parameter could be different. For more information about supported input image dimensions and required pre- and post-processing steps, refer to the [documentation](https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/mask-rcnn).
**Step 3**. Interpret the outputs. The generated IR file has several outputs: masks, class indices, probabilities and box coordinates. The first one is a layer with the name "6849/sink_port_0". The rest three are outputs from the "DetectionOutput" layer.
3. Interpret the outputs of the generated IR file: masks, class indices, probabilities and box coordinates.
* masks.
* class indices.
* probabilities.
* box coordinates.
The first one is a layer with the name `6849/sink_port_0`, and rest are outputs from the `DetectionOutput` layer.
# Convert PyTorch* BERT-NER Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_Bert_ner}
# Converting a PyTorch BERT-NER Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_Bert_ner}
## Download and Convert the Model to ONNX*
The goal of this article is to present a step-by-step guide on how to convert PyTorch BERT-NER model to OpenVINO IR. First, you need to download the model and convert it to ONNX.
To download a pre-trained model or train the model yourself, refer
to the [instruction](https://github.com/kamalkraj/BERT-NER/blob/dev/README.md) in the
BERT-NER model repository. The model with config files is stored in the `out_base` directory.
To convert the model to ONNX* format, create and run the script with the following content in the root
directory of the model repository. If you download the pre-trained model, you need
## Downloading and Converting the Model to ONNX
To download a pretrained model or train the model yourself, refer
to the [instructions](https://github.com/kamalkraj/BERT-NER/blob/dev/README.md) in the
BERT-NER model repository. The model with configuration files is stored in the `out_base` directory.
To convert the model to ONNX format, create and run the following script in the root
directory of the model repository. If you download the pretrained model, you need
to download [`bert.py`](https://github.com/kamalkraj/BERT-NER/blob/dev/bert.py) to run the script.
The instruction was tested with the repository hash commit `e5be564156f194f1becb0d82aeaf6e762d9eb9ed`.
The instructions were tested with the commit-SHA:`e5be564156f194f1becb0d82aeaf6e762d9eb9ed`.
```python
importtorch
@@ -44,12 +47,12 @@ torch.onnx.export(ner_model,
)
```
The script generates ONNX* model file `bert-ner.onnx`.
The script generates ONNX model file `bert-ner.onnx`.
## Convert ONNX* BERT-NER model to IR
## Converting an ONNX BERT-NER model to IR
```bash
mo --input_model bert-ner.onnx --input "input_mask[1 128],segment_ids[1 128],input_ids[1 128]"
```
where `1` is `batch_size` and `128` is `sequence_length`.
where `1` is `batch_size` and `128` is `sequence_length`.
# Convert PyTorch Cascade RCNN R-101 Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_Cascade_RCNN_res101}
# Converting a PyTorch Cascade RCNN R-101 Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_Cascade_RCNN_res101}
## Download and Convert Model to ONNX
The goal of this article is to present a step-by-step guide on how to convert a PyTorch Cascade RCNN R-101 model to OpenVINO IR. First, you need to download the model and convert it to ONNX.
## Downloading and Converting Model to ONNX
* Clone the [repository](https://github.com/open-mmlab/mmdetection):
> **NOTE**: To set up an environment, refer to this [instruction](https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md#installation).
> **NOTE**: To set up an environment, refer to the [instructions](https://github.com/open-mmlab/mmdetection/blob/master/docs/en/get_started.md#installation).
* Download the pre-trained [model](https://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_r101_fpn_1x_coco/cascade_rcnn_r101_fpn_1x_coco_20200317-0b6a2fbf.pth). You can also find the link to the model [here](https://github.com/open-mmlab/mmdetection/blob/master/configs/cascade_rcnn/README.md).
* Download the pretrained [model](https://download.openmmlab.com/mmdetection/v2.0/cascade_rcnn/cascade_rcnn_r101_fpn_1x_coco/cascade_rcnn_r101_fpn_1x_coco_20200317-0b6a2fbf.pth). The model is also available [here](https://github.com/open-mmlab/mmdetection/blob/master/configs/cascade_rcnn/README.md).
* To convert the model to ONNX format, use this [script](https://github.com/open-mmlab/mmdetection/blob/master/tools/deployment/pytorch2onnx.py).
The script generates ONNX model file `cascade_rcnn_r101_fpn_1x_coco.onnx` in the directory `tools/deployment/`. If required, you can specify the model name or output directory using `--output-file <path-to-dir>/<model-name>.onnx`
The script generates ONNX model file `cascade_rcnn_r101_fpn_1x_coco.onnx` in the directory `tools/deployment/`. If required, specify the model name or output directory, using `--output-file <path-to-dir>/<model-name>.onnx`.
## Convert ONNX Cascade RCNN R-101 Model to IR
## Converting an ONNX Cascade RCNN R-101 Model to OpenVINO IR
```bash
mo --input_model cascade_rcnn_r101_fpn_1x_coco.onnx --mean_values [123.675,116.28,103.53] --scale_values [58.395,57.12,57.375]
# Convert PyTorch* F3Net Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_F3Net}
# Converting a PyTorch F3Net Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_F3Net}
[F3Net](https://github.com/weijun88/F3Net): Fusion, Feedback and Focus for Salient Object Detection
## Clone the F3Net Repository
## Cloning the F3Net Repository
To clone the repository, run the following command:
@@ -10,10 +10,10 @@ To clone the repository, run the following command:
git clone http://github.com/weijun88/F3Net.git
```
## Download and Convert the Model to ONNX*
## Downloading and Converting the Model to ONNX
To download the pre-trained model or train the model yourself, refer to the
[instruction](https://github.com/weijun88/F3Net/blob/master/README.md) in the F3Net model repository. First, convert the model to ONNX\* format. Create and run the following Python script in the `src` directory of the model repository:
To download the pretrained model or train the model yourself, refer to the
[instructions](https://github.com/weijun88/F3Net/blob/master/README.md) in the F3Net model repository. First, convert the model to ONNX format. Create and run the following Python script in the `src` directory of the model repository:
The script generates the ONNX\* model file f3net.onnx. This model conversion was tested with the repository hash commit `eecace3adf1e8946b571a4f4397681252f9dc1b8`.
The script generates the ONNX model file `f3net.onnx`. The model conversion was tested with the commit-SHA:`eecace3adf1e8946b571a4f4397681252f9dc1b8`.
# Convert PyTorch* RCAN Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_RCAN}
# Converting a PyTorch RCAN Model {#openvino_docs_MO_DG_prepare_model_convert_model_pytorch_specific_Convert_RCAN}
[RCAN](https://github.com/yulunzhang/RCAN): Image Super-Resolution Using Very Deep Residual Channel Attention Networks
## Download and Convert the Model to ONNX*
## Downloading and Converting the Model to ONNX
To download the pre-trained model or train the model yourself, refer to the
[instruction](https://github.com/yulunzhang/RCAN/blob/master/README.md) in the RCAN model repository. Firstly,
convert the model to ONNX\* format. Create and run the script with the following content in the root
To download the pretrained model or train the model yourself, refer to the [instruction](https://github.com/yulunzhang/RCAN/blob/master/README.md) in the RCAN model repository. First, convert the model to ONNX format. Create and run the script with the following content in the root
directory of the model repository:
```python
fromargparseimportNamespace
@@ -22,9 +21,9 @@ net.eval()
dummy_input=torch.randn(1,3,360,640)
torch.onnx.export(net,dummy_input,'RCAN.onnx')
```
The script generates the ONNX\* model file RCAN.onnx. You can find more information about model parameters (`n_resblocks`, `n_resgroups`, and others) in the model repository and use different values of them. The model conversion was tested with the repository hash commit `3339ebc59519c3bb2b5719b87dd36515ec7f3ba7`.
The script generates the ONNX model file `RCAN.onnx`. More information about model parameters (`n_resblocks`, `n_resgroups`, and others) and their different values can be found in the model repository. The model conversion was tested with the commit-SHA:`3339ebc59519c3bb2b5719b87dd36515ec7f3ba7`.
## Convert ONNX* RCAN Model to IR
## Converting an ONNX RCAN Model to IR
```sh
mo --input_model RCAN.onnx
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.