openvino/docs/MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md

Model Optimizer Developer Guide

Model Optimizer is a cross-platform command-line tool that facilitates the transition between the training and deployment environment, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.

Model Optimizer process assumes you have a network model trained using a supported deep learning framework. The scheme below illustrates the typical workflow for deploying a trained deep learning model:

Model Optimizer produces an Intermediate Representation (IR) of the network, which can be read, loaded, and inferred with the Inference Engine. The Inference Engine API offers a unified API across a number of supported Intel® platforms. The Intermediate Representation is a pair of files describing the model:

• .xml - Describes the network topology

• .bin - Contains the weights and biases binary data.

What's New in the Model Optimizer in this Release?

Deprecation Notice

Deprecation Begins	June 1, 2020
Removal Date	December 1, 2020

Starting with the OpenVINO™ toolkit 2020.2 release, all of the features previously available through nGraph have been merged into the OpenVINO™ toolkit. As a result, all the features previously available through ONNX RT Execution Provider for nGraph have been merged with ONNX RT Execution Provider for OpenVINO™ toolkit.

Therefore, ONNX RT Execution Provider for nGraph will be deprecated starting June 1, 2020 and will be completely removed on December 1, 2020. Users are recommended to migrate to the ONNX RT Execution Provider for OpenVINO™ toolkit as the unified solution for all AI inferencing on Intel® hardware.

• Common changes:

  • Implemented generation of a compressed OpenVINO IR suitable for INT8 inference, which takes up to 4 times less disk space than an expanded one. Use the --disable_weights_compression Model Optimizer command-line parameter to get an expanded version.
  • Implemented an optimization transformation to replace a sub-graph with the Erf operation into the GeLU operation.
  • Implemented an optimization transformation to replace an upsamping pattern that is represented as a sequence of Split and Concat operations to a single Interpolate operation.
  • Fixed a number of Model Optimizer bugs to generate reshape-able IRs of many models with the command line parameter --keep_shape_ops.
  • Fixed a number of Model Optimizer transformations to set operations name in an IR equal to the original framework model operation name.
  • The following operations are no longer generated with version="opset1": MVN, ROIPooling, ReorgYolo. They became a part of new opset2 operation set and generated with version="opset2". Before this fix, the operations were generated with version="opset1" by mistake, they were not a part of opset1 nGraph namespace; opset1 specification was fixed accordingly.

• ONNX*:

  • Added support for the following operations: MeanVarianceNormalization if normalization is performed over spatial dimensions.

• TensorFlow*:

  • Added support for the TensorFlow Object Detection models version 1.15.X.
  • Added support for the following operations: BatchToSpaceND, SpaceToBatchND, Floor.

• MXNet*:

  • Added support for the following operations:
    • Reshape with input shape values equal to -2, -3, and -4.

NOTE: Intel® System Studio is an all-in-one, cross-platform tool suite, purpose-built to simplify system bring-up and improve system and IoT device application performance on Intel® platforms. If you are using the Intel® Distribution of OpenVINO™ with Intel® System Studio, go to Get Started with Intel® System Studio.

Table of Content

• Introduction to OpenVINO™ Deep Learning Deployment Toolkit

• Preparing and Optimizing your Trained Model with Model Optimizer

  • Configuring Model Optimizer

  • Converting a Model to Intermediate Representation (IR)

    • Converting a Model Using General Conversion Parameters
    • Converting Your Caffe* Model
    • Converting Your TensorFlow* Model

      • Converting BERT from TensorFlow

      • Converting GNMT from TensorFlow

      • Converting YOLO from DarkNet to TensorFlow and then to IR

      • Converting Wide and Deep Models from TensorFlow

      • Converting FaceNet from TensorFlow

      • Converting DeepSpeech from TensorFlow

      • Converting Language Model on One Billion Word Benchmark from TensorFlow

      • Converting Neural Collaborative Filtering Model from TensorFlow*

      • Converting TensorFlow* Object Detection API Models

      • Converting TensorFlow*-Slim Image Classification Model Library Models

      • Converting CRNN Model from TensorFlow*

    • Converting Your MXNet* Model
      • Converting a Style Transfer Model from MXNet
    • Converting Your Kaldi* Model
    • Converting Your ONNX* Model
      • Converting Mask-RCNN ONNX* Model
      • Converting DLRM ONNX* Model
    • Model Optimizations Techniques
    • Cutting parts of the model
    • Sub-graph Replacement in Model Optimizer
      • (Deprecated) Case-Study: Converting SSD models created with the TensorFlow* Object Detection API
      • (Deprecated) Case-Study: Converting Faster R-CNN models created with the TensorFlow* Object Detection API
    • Supported Framework Layers
    • Intermediate Representation and Operation Sets
    • Operations Specification
    • Intermediate Representation suitable for INT8 inference

  • Custom Layers in Model Optimizer

    • Extending Model Optimizer with New Primitives
    • Legacy Mode for Caffe* Custom Layers

  • Model Optimizer Frequently Asked Questions

• Known Issues

Typical Next Step: Introduction to Intel® Deep Learning Deployment Toolkit