Anastasiia Pnevskaia
59872ee072
* Fixed output_model logic. * Removed InputCutInfo, disabled input cut in ovc. * Disabled output cut, added tests for setting shapes or types for not all inputs. * Returned support of numpy type. * Separated MO and OVC python API tests. * Small corrections. * Added output dir test, exceptions test. * Tests fixed. * Corrected extension param description. * Corrected input description, minor code corrections.
Post-Training Optimization Tool
Introduction
Post-training Optimization Tool (POT) is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning, for example, post-training 8-bit quantization. Therefore, the tool does not require a training dataset or a pipeline. To apply post-training algorithms from the POT, you need:
- A floating-point precision model, FP32 or FP16, converted into the OpenVINO™ Intermediate Representation (IR) format and run on CPU with the OpenVINO™.
- A representative calibration dataset representing a use case scenario, for example, 300 samples.
Figure below shows the optimization workflow:
To get started with POT tool refer to the corresponding OpenVINO™ documentation.
Installation
From PyPI
POT is distributed as a part of OpenVINO™ Development Tools package. For installation instruction please refer to this document.
From GitHub
As prerequisites, you should install OpenVINO™ Runtime and other dependencies such as Model Optimizer and Accuracy Checker.
To install POT from source:
- Clone OpenVINO repository
git clone --recusive https://github.com/openvinotoolkit/openvino.git - Navigate to
openvino/tools/pot/folder - Install POT package:
python3 setup.py install
After installation POT is available as a Python library under openvino.tools.pot.* and in the command line by the pot alias. To verify it, run pot -h.
Examples
OpenVINO provides several examples to demonstrate the POT optimization workflow:
- Command-line example:
- API tutorials:
- API examples: