Docs: preprocessing use case with saving model to IR (#10698)

* Docs: added preprocessing use case with saving resulting model to IR

* Enable Model Caching to 'application code' section

* Fix review comments

docs/OV_Runtime_UG/preprocessing_overview.md

@@ -8,6 +8,7 @@
 
    openvino_docs_OV_Runtime_UG_Preprocessing_Details
    openvino_docs_OV_Runtime_UG_Layout_Overview
+   openvino_docs_OV_Runtime_UG_Preprocess_Usecase_save
 
 @endsphinxdirective
 

docs/OV_Runtime_UG/preprocessing_usecase_save.md

@@ -0,0 +1,83 @@
+# Use Case - Integrate and Save Preprocessing Steps Into IR {#openvino_docs_OV_Runtime_UG_Preprocess_Usecase_save}
+
+## Introduction
+
+In previous sections we've covered how to add [preprocessing steps](./preprocessing_details.md) and got the overview of [Layout](./layout_overview.md) API.
+
+For many applications it is also important to minimize model's read/load time, so performing integration of preprocessing steps every time on application startup after `ov::runtime::Core::read_model` may look not convenient. In such cases, after adding of Pre- and Post-processing steps it can be useful to store new execution model to Intermediate Representation (IR, .xml format).
+
+Most part of existing preprocessing steps can also be performed via command line options using Model Optimizer tool. Refer to [Model Optimizer - Optimize Preprocessing Computation](../MO_DG/prepare_model/Additional_Optimizations.md) for details os such command line options.
+
+## Code example - saving model with preprocessing to IR
+
+In case if you have some preprocessing steps which can't be integrated into execution graph using Model Optimizer command line options (e.g. `YUV->RGB` color space conversion, Resize, etc.) it is possible to write simple code which:
+ - Reads original model (IR, ONNX, Paddle)
+ - Adds preprocessing/postprocessing steps
+ - Saves resulting model as IR (.xml/.bin)
+
+Let's consider the example, there is an original `ONNX` model which takes one `float32` input with shape `{1, 3, 224, 224}` with `RGB` channels order, with mean/scale values applied. User's application can provide `BGR` image buffer with not fixed size. Additionally, we'll also imagine that our application provides input images as batches, each batch contains 2 images. Here is how model conversion code may look like in your model preparation script
+
+- Includes / Imports
+@sphinxdirective
+
+.. tab:: C++
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
+         :language: cpp
+         :fragment: [ov:preprocess:save_headers]
+
+.. tab:: Python
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.py
+         :language: python
+         :fragment: [ov:preprocess:save_headers]
+
+@endsphinxdirective
+
+- Preprocessing & Saving to IR code
+@sphinxdirective
+
+.. tab:: C++
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
+         :language: cpp
+         :fragment: [ov:preprocess:save]
+
+.. tab:: Python
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.py
+         :language: python
+         :fragment: [ov:preprocess:save]
+
+@endsphinxdirective
+
+
+## Application code - load model to target device
+
+After this, your application's code can load saved file and don't perform preprocessing anymore. In this example we'll also enable [model caching](./Model_caching_overview.md) to minimize load time when cached model is available
+
+@sphinxdirective
+
+.. tab:: C++
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.cpp
+         :language: cpp
+         :fragment: [ov:preprocess:save_load]
+
+.. tab:: Python
+
+      .. doxygensnippet:: docs/snippets/ov_preprocessing.py
+         :language: python
+         :fragment: [ov:preprocess:save_load]
+
+@endsphinxdirective
+
+
+## See Also
+* [Preprocessing Details](./preprocessing_details.md)
+* [Layout API overview](./layout_overview.md)
+* [Model Optimizer - Optimize Preprocessing Computation](../MO_DG/prepare_model/Additional_Optimizations.md)
+* [Model Caching Overview](./Model_caching_overview.md)
+* <code>ov::preprocess::PrePostProcessor</code> C++ class documentation
+* <code>ov::pass::Serialize</code> - pass to serialize model to XML/BIN
+* <code>ov::set_batch</code> - update batch dimension for a given model

docs/snippets/ov_preprocessing.cpp

@@ -150,3 +150,60 @@ int main() {
  OPENVINO_ASSERT(model, "Model is invalid");
  return 0;
 }
+
+ //! [ov:preprocess:save_headers]
+ #include <openvino/runtime/core.hpp>
+ #include <openvino/core/preprocess/pre_post_process.hpp>
+ #include <openvino/pass/serialize.hpp>
+ //! [ov:preprocess:save_headers]
+
+void save_example() {
+ //! [ov:preprocess:save]
+ // ========  Step 0: read original model =========
+ ov::Core core;
+ std::shared_ptr<ov::Model> model = core.read_model("/path/to/some_model.onnx");
+
+ // ======== Step 1: Preprocessing ================
+ ov::preprocess::PrePostProcessor prep(model);
+ // Declare section of desired application's input format
+ prep.input().tensor()
+        .set_element_type(ov::element::u8)
+        .set_layout("NHWC")
+        .set_color_format(ov::preprocess::ColorFormat::BGR)
+        .set_spatial_dynamic_shape();
+ // Specify actual model layout
+ prep.input().model()
+        .set_layout("NCHW");
+ // Explicit preprocessing steps. Layout conversion will be done automatically as last step
+ prep.input().preprocess()
+        .convert_element_type()
+        .convert_color(ov::preprocess::ColorFormat::RGB)
+        .resize(ov::preprocess::ResizeAlgorithm::RESIZE_LINEAR)
+        .mean({123.675, 116.28, 103.53}) // Subtract mean after color conversion
+        .scale({58.624, 57.12, 57.375});
+ // Dump preprocessor
+ std::cout << "Preprocessor: " << prep << std::endl;
+ model = prep.build();
+
+ // ======== Step 2: Change batch size ================
+ // In this example we also want to change batch size to increase throughput
+ ov::set_batch(model, 2);
+
+ // ======== Step 3: Save the model ================
+ std::string xml = "/path/to/some_model_saved.xml";
+ std::string bin = "/path/to/some_model_saved.bin";
+ ov::pass::Serialize(xml, bin).run_on_model(model);
+ //! [ov:preprocess:save]
+
+}
+
+void load_aftersave_example() {
+ //! [ov:preprocess:save_load]
+ ov::Core core;
+ core.set_property(ov::cache_dir("/path/to/cache/dir"));
+
+ // In case that no preprocessing is needed anymore, we can load model on target device directly
+ // With cached model available, it will also save some time on reading original model
+ ov::CompiledModel compiled_model = core.compile_model("/path/to/some_model_saved.xml", "CPU");
+ //! [ov:preprocess:save_load]
+}

docs/snippets/ov_preprocessing.py

@@ -169,3 +169,59 @@ def custom_abs(output: Output):
 ppp.output("result_image").postprocess()\
     .custom(custom_abs)
 # ! [ov:preprocess:postprocess]
+
+# ! [ov:preprocess:save_headers]
+from openvino.preprocess import PrePostProcessor, ColorFormat, ResizeAlgorithm
+from openvino.runtime import Core, Layout, Type, set_batch
+from openvino.runtime.passes import Manager
+# ! [ov:preprocess:save_headers]
+
+# ! [ov:preprocess:save]
+# ========  Step 0: read original model =========
+core = Core()
+model = core.read_model(model='/path/to/some_model.onnx')
+
+# ======== Step 1: Preprocessing ================
+ppp = PrePostProcessor(model)
+# Declare section of desired application's input format
+ppp.input().tensor() \
+    .set_element_type(Type.u8) \
+    .set_spatial_dynamic_shape() \
+    .set_layout(Layout('NHWC')) \
+    .set_color_format(ColorFormat.BGR)
+
+# Specify actual model layout
+ppp.input().model().set_layout(Layout('NCHW'))
+
+# Explicit preprocessing steps. Layout conversion will be done automatically as last step
+ppp.input().preprocess() \
+    .convert_element_type() \
+    .convert_color(ColorFormat.RGB) \
+    .resize(ResizeAlgorithm.RESIZE_LINEAR) \
+    .mean([123.675, 116.28, 103.53]) \
+    .scale([58.624, 57.12, 57.375])
+
+# Dump preprocessor
+print(f'Dump preprocessor: {ppp}')
+model = ppp.build()
+
+# ======== Step 2: Change batch size ================
+# In this example we also want to change batch size to increase throughput
+set_batch(model, 2)
+
+# ======== Step 3: Save the model ================
+pass_manager = Manager()
+pass_manager.register_pass(pass_name="Serialize",
+                           xml_path='/path/to/some_model_saved.xml',
+                           bin_path='/path/to/some_model_saved.bin')
+pass_manager.run_passes(model)
+# ! [ov:preprocess:save]
+
+# ! [ov:preprocess:save_load]
+core = Core()
+core.set_property({'CACHE_DIR': '/path/to/cache/dir'})
+
+# In case that no preprocessing is needed anymore, we can load model on target device directly
+# With cached model available, it will also save some time on reading original model
+compiled_model = core.compile_model('/path/to/some_model_saved.xml', 'CPU')
+# ! [ov:preprocess:save_load]