openvino/docs/OV_Runtime_UG/preprocessing_usecase_save.md

Use Case - Integrate and Save Preprocessing Steps Into IR

Introduction

In previous sections we've covered how to add preprocessing steps and got the overview of Layout API.

For many applications it is also important to minimize model's read/load time, so performing integration of preprocessing steps every time on application startup after ov::runtime::Core::read_model may look not convenient. In such cases, after adding of Pre- and Post-processing steps it can be useful to store new execution model to Intermediate Representation (IR, .xml format).

Most part of existing preprocessing steps can also be performed via command line options using Model Optimizer tool. Refer to Model Optimizer - Optimize Preprocessing Computation for details os such command line options.

Code example - saving model with preprocessing to IR

In case if you have some preprocessing steps which can't be integrated into execution graph using Model Optimizer command line options (e.g. YUV->RGB color space conversion, Resize, etc.) it is possible to write simple code which:

• Reads original model (IR, ONNX, Paddle)
• Adds preprocessing/postprocessing steps
• Saves resulting model as IR (.xml/.bin)

Let's consider the example, there is an original ONNX model which takes one float32 input with shape {1, 3, 224, 224} with RGB channels order, with mean/scale values applied. User's application can provide BGR image buffer with not fixed size. Additionally, we'll also imagine that our application provides input images as batches, each batch contains 2 images. Here is how model conversion code may look like in your model preparation script

• Includes / Imports @sphinxdirective

.. tab:: C++

  .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
     :language: cpp
     :fragment: [ov:preprocess:save_headers]

.. tab:: Python

  .. doxygensnippet:: docs/snippets/ov_preprocessing.py
     :language: python
     :fragment: [ov:preprocess:save_headers]

@endsphinxdirective

• Preprocessing & Saving to IR code @sphinxdirective

.. tab:: C++

  .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
     :language: cpp
     :fragment: [ov:preprocess:save]

.. tab:: Python

  .. doxygensnippet:: docs/snippets/ov_preprocessing.py
     :language: python
     :fragment: [ov:preprocess:save]

@endsphinxdirective

Application code - load model to target device

After this, your application's code can load saved file and don't perform preprocessing anymore. In this example we'll also enable model caching to minimize load time when cached model is available

@sphinxdirective

.. tab:: C++

  .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
     :language: cpp
     :fragment: [ov:preprocess:save_load]

.. tab:: Python

  .. doxygensnippet:: docs/snippets/ov_preprocessing.py
     :language: python
     :fragment: [ov:preprocess:save_load]

@endsphinxdirective

See Also

• Preprocessing Details
• Layout API overview
• Model Optimizer - Optimize Preprocessing Computation
• Model Caching Overview
• ov::preprocess::PrePostProcessor C++ class documentation
• ov::pass::Serialize - pass to serialize model to XML/BIN
• ov::set_batch - update batch dimension for a given model