DOCS shift to rst - Quantizing Models with Accuracy Control, Documentation, Get Started & Learn OpenVINO (#16997)
* Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * Update AccuracyAwareQuantizationUsage.md * rst * fixes
This commit is contained in:
Maciej Smyk authored and 
GitHub committed
1 parent
49b5d039db
commit
72533a7da1
4 files changed
+201
-149
No files matched your search
@@ -13,13 +13,9 @@
|
||||
Media Processing and CV Libraries <media_processing_cv_libraries>
|
||||
OpenVINO™ Security <openvino_docs_security_guide_introduction>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
|
||||
This section provides reference documents that guide you through the OpenVINO toolkit workflow, from preparing models, optimizing them, to deploying them in your own deep learning applications.
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
| :doc:`API Reference doc path <api/api_reference>`
|
||||
| A collection of reference articles for OpenVINO C++, C, and Python APIs.
|
||||
|
||||
|
||||
+87
-49
@@ -10,10 +10,8 @@
|
||||
Additional Configurations <openvino_docs_install_guides_configurations_header>
|
||||
Uninstalling <openvino_docs_install_guides_uninstalling_openvino>
|
||||
Troubleshooting <openvino_docs_get_started_guide_troubleshooting>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
|
||||
.. raw:: html
|
||||
|
||||
<link rel="stylesheet" type="text/css" href="_static/css/getstarted_style.css">
|
||||
@@ -24,69 +22,109 @@
|
||||
<a href="#learn-openvino" >Learn OpenVINO</a>
|
||||
</p>
|
||||
<div style="clear:both;"> </div>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
## <a name="quick-start-example"></a>1. Quick Start Example (No Installation Required)
|
||||
.. _quick-start-example:
|
||||
|
||||
<img src="https://user-images.githubusercontent.com/15709723/127752390-f6aa371f-31b5-4846-84b9-18dd4f662406.gif">
|
||||
1. Quick Start Example (No Installation Required)
|
||||
#################################################
|
||||
|
||||
Try out OpenVINO's capabilities with this quick start example that estimates depth in a scene using an OpenVINO monodepth model. <a href="https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F201-vision-monodepth%2F201-vision-monodepth.ipynb">Run the example in a Jupyter Notebook inside your web browser</a> to quickly see how to load a model, prepare an image, inference the image, and display the result.
|
||||
.. image:: https://user-images.githubusercontent.com/15709723/127752390-f6aa371f-31b5-4846-84b9-18dd4f662406.gif
|
||||
:width: 400
|
||||
|
||||
|
||||
## <a name="install-openvino-gsg"></a>2. Install OpenVINO
|
||||
|
||||
See the [installation overview page](./install_guides/installing-openvino-overview.md) for options to install OpenVINO and set up a development environment on your device.
|
||||
|
||||
## <a name="get-started-tutorials"></a>3. Learn OpenVINO
|
||||
|
||||
OpenVINO provides a wide array of examples and documentation showing how to work with models, run inference, and deploy applications. Step through the sections below to learn the basics of OpenVINO and explore its advanced optimization features. For further details, visit [OpenVINO documentation](./documentation.md).
|
||||
|
||||
OpenVINO users of all experience levels can try [Intel® DevCloud](https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/overview.html), a free web-based service for developing, testing, and running OpenVINO applications on an online cluster of the latest Intel® hardware.
|
||||
Try out OpenVINO's capabilities with this quick start example that estimates depth in a scene using an OpenVINO monodepth model. `Run the example in a Jupyter Notebook inside your web browser <https://mybinder.org/v2/gh/openvinotoolkit/openvino_notebooks/HEAD?filepath=notebooks%2F201-vision-monodepth%2F201-vision-monodepth.ipynb>`__ to quickly see how to load a model, prepare an image, inference the image, and display the result.
|
||||
|
||||
.. _install-openvino-gsg:
|
||||
|
||||
2. Install OpenVINO
|
||||
###################
|
||||
|
||||
See the :doc:`installation overview page <openvino_docs_install_guides_overview>` for options to install OpenVINO and set up a development environment on your device.
|
||||
|
||||
.. _get-started-tutorials:
|
||||
|
||||
3. Learn OpenVINO
|
||||
#################
|
||||
|
||||
OpenVINO provides a wide array of examples and documentation showing how to work with models, run inference, and deploy applications. Step through the sections below to learn the basics of OpenVINO and explore its advanced optimization features. For further details, visit :doc:`OpenVINO documentation <documentation>`.
|
||||
|
||||
OpenVINO users of all experience levels can try `Intel® DevCloud <https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/overview.html>`__ , a free web-based service for developing, testing, and running OpenVINO applications on an online cluster of the latest Intel® hardware.
|
||||
|
||||
.. _openvino-basics:
|
||||
|
||||
OpenVINO Basics
|
||||
+++++++++++++++
|
||||
|
||||
### <a name="openvino-basics"></a>OpenVINO Basics
|
||||
Learn the basics of working with models and inference in OpenVINO. Begin with “Hello World” Interactive Tutorials that show how to prepare models, run inference, and retrieve results using the OpenVINO API. Then, explore other examples from the Open Model Zoo and OpenVINO Code Samples that can be adapted for your own application.
|
||||
|
||||
.. _interactive-tutorials:
|
||||
|
||||
#### <a name="interactive-tutorials"></a>Interactive Tutorials - Jupyter Notebooks
|
||||
Start with <a href="tutorials.html">interactive Python tutorials</a> that show the basics of model inferencing, the OpenVINO API, how to convert models to OpenVINO format, and more.
|
||||
* <a href="notebooks/001-hello-world-with-output.html">Hello Image Classification</a> - Load an image classification model in OpenVINO and use it to apply a label to an image
|
||||
* <a href="notebooks/002-openvino-api-with-output.html">OpenVINO Runtime API Tutorial</a> - Learn the basic Python API for working with models in OpenVINO
|
||||
* <a href="notebooks/101-tensorflow-to-openvino-with-output.html">Convert TensorFlow Models to OpenVINO</a>
|
||||
* <a href="notebooks/102-pytorch-onnx-to-openvino-with-output.html">Convert PyTorch Models to OpenVINO</a>
|
||||
Interactive Tutorials - Jupyter Notebooks
|
||||
-----------------------------------------
|
||||
|
||||
#### <a name="code-samples"></a>OpenVINO Code Samples
|
||||
View <a href="openvino_docs_OV_UG_Samples_Overview.html">sample code</a> for various C++ and Python applications that can be used as a starting point for your own application. For C++ developers, step through the [Get Started with C++ Samples](./get_started/get_started_demos.md) to learn how to build and run an image classification program that uses OpenVINO’s C++ API.
|
||||
Start with :doc:`interactive Python tutorials <tutorials>` that show the basics of model inferencing, the OpenVINO API, how to convert models to OpenVINO format, and more.
|
||||
|
||||
* `Hello Image Classification <https://docs.openvino.ai/nightly/notebooks/001-hello-world-with-output.html>`__ - Load an image classification model in OpenVINO and use it to apply a label to an image
|
||||
* `OpenVINO Runtime API Tutorial <https://docs.openvino.ai/nightly/notebooks/002-openvino-api-with-output.html>`__ - Learn the basic Python API for working with models in OpenVINO
|
||||
* `Convert TensorFlow Models to OpenVINO <https://docs.openvino.ai/nightly/notebooks/101-tensorflow-to-openvino-with-output.html>`__
|
||||
* `Convert PyTorch Models to OpenVINO <https://docs.openvino.ai/nightly/notebooks/102-pytorch-onnx-to-openvino-with-output.html>`__
|
||||
|
||||
.. _code-samples:
|
||||
|
||||
OpenVINO Code Samples
|
||||
---------------------
|
||||
|
||||
View :doc:`sample code <openvino_docs_OV_UG_Samples_Overview>` for various C++ and Python applications that can be used as a starting point for your own application. For C++ developers, step through the :doc:`Get Started with C++ Samples <openvino_docs_get_started_get_started_demos>` to learn how to build and run an image classification program that uses OpenVINO’s C++ API.
|
||||
|
||||
#### <a name="integrate-openvino"></a>Integrate OpenVINO With Your Application
|
||||
Learn how to <a href="openvino_docs_OV_UG_Integrate_OV_with_your_application.html">use the OpenVINO API to implement an inference pipeline</a> in your application.
|
||||
.. _integrate-openvino:
|
||||
|
||||
Integrate OpenVINO With Your Application
|
||||
----------------------------------------
|
||||
|
||||
Learn how to :doc:`use the OpenVINO API to implement an inference pipeline <openvino_docs_OV_UG_Integrate_OV_with_your_application>` in your application.
|
||||
|
||||
.. _openvino-advanced-features:
|
||||
|
||||
OpenVINO Advanced Features
|
||||
++++++++++++++++++++++++++
|
||||
|
||||
### <a name="openvino-advanced-features"></a>OpenVINO Advanced Features
|
||||
OpenVINO provides features to improve your model’s performance, optimize your runtime, maximize your application’s throughput on target hardware, and much more. Visit the links below to learn more about these features and how to use them.
|
||||
|
||||
#### Model Compression and Quantization
|
||||
Model Compression and Quantization
|
||||
----------------------------------
|
||||
|
||||
Use OpenVINO’s model compression tools to reduce your model’s latency and memory footprint while maintaining good accuracy.
|
||||
* Tutorial - <a href="notebooks/111-detection-quantization-with-output.html">OpenVINO Post-Training Model Quantization</a>
|
||||
* Tutorial - <a href="notebooks/305-tensorflow-quantization-aware-training-with-output.html">Quantization-Aware Training in TensorFlow with OpenVINO NNCF</a>
|
||||
* Tutorial - <a href="notebooks/302-pytorch-quantization-aware-training-with-output.html">Quantization-Aware Training in PyTorch with NNCF</a>
|
||||
* <a href="notebooks/openvino_docs_model_optimization_guide.html">Model Optimization Guide</a>
|
||||
|
||||
#### Automated Device Configuration
|
||||
* Tutorial - `OpenVINO Post-Training Model Quantization <https://docs.openvino.ai/nightly/notebooks/111-detection-quantization-with-output.html>`__
|
||||
* Tutorial - `Quantization-Aware Training in TensorFlow with OpenVINO NNCF <https://docs.openvino.ai/nightly/notebooks/305-tensorflow-quantization-aware-training-with-output.html>`__
|
||||
* Tutorial - `Quantization-Aware Training in PyTorch with NNCF <https://docs.openvino.ai/nightly/notebooks/302-pytorch-quantization-aware-training-with-output.html>`__
|
||||
* `Model Optimization Guide <https://docs.openvino.ai/nightly/notebooks/openvino_docs_model_optimization_guide.html>`__
|
||||
|
||||
Automated Device Configuration
|
||||
------------------------------
|
||||
|
||||
OpenVINO’s hardware device configuration options enable you to write an application once and deploy it anywhere with optimal performance.
|
||||
* Increase application portability with [Automatic Device Selection (AUTO)](./OV_Runtime_UG/auto_device_selection.md)
|
||||
* Perform parallel inference across processors with [Multi-Device Execution (MULTI)](./OV_Runtime_UG/multi_device.md)
|
||||
* Efficiently split inference between hardware cores with [Heterogeneous Execution (HETERO)](./OV_Runtime_UG/hetero_execution.md)
|
||||
|
||||
#### Flexible Model and Pipeline Configuration
|
||||
* Increase application portability with :doc:`Automatic Device Selection (AUTO) <openvino_docs_OV_UG_supported_plugins_AUTO>`
|
||||
* Perform parallel inference across processors with :doc:`Multi-Device Execution (MULTI) <openvino_docs_OV_UG_Running_on_multiple_devices>`
|
||||
* Efficiently split inference between hardware cores with :doc:`Heterogeneous Execution (HETERO) <openvino_docs_OV_UG_Hetero_execution>`
|
||||
|
||||
Flexible Model and Pipeline Configuration
|
||||
-----------------------------------------
|
||||
|
||||
Pipeline and model configuration features in OpenVINO Runtime allow you to easily optimize your application’s performance on any target hardware.
|
||||
* [Automatic Batching](./OV_Runtime_UG/automatic_batching.md) performs on-the-fly grouping of inference requests to maximize utilization of the target hardware’s memory and processing cores.
|
||||
* [Performance Hints](./OV_Runtime_UG/performance_hints.md) automatically adjust runtime parameters to prioritize for low latency or high throughput
|
||||
* [Dynamic Shapes](./OV_Runtime_UG/ov_dynamic_shapes.md) reshapes models to accept arbitrarily-sized inputs, increasing flexibility for applications that encounter different data shapes
|
||||
* [Benchmark Tool](../tools/benchmark_tool/README.md) characterizes model performance in various hardware and pipeline configurations
|
||||
|
||||
* :doc:`Automatic Batching <openvino_docs_OV_UG_Automatic_Batching>` performs on-the-fly grouping of inference requests to maximize utilization of the target hardware’s memory and processing cores.
|
||||
* :doc:`Performance Hints <openvino_docs_OV_UG_Performance_Hints>` automatically adjust runtime parameters to prioritize for low latency or high throughput
|
||||
* :doc:`Dynamic Shapes <openvino_docs_OV_UG_DynamicShapes>` reshapes models to accept arbitrarily-sized inputs, increasing flexibility for applications that encounter different data shapes
|
||||
* :doc:`Benchmark Tool <openvino_inference_engine_tools_benchmark_tool_README>` characterizes model performance in various hardware and pipeline configurations
|
||||
|
||||
### <a name="additional-resources"></a>Additional Resources
|
||||
* [OpenVINO Success Stories](https://www.intel.com/content/www/us/en/internet-of-things/ai-in-production/success-stories.html) - See how Intel partners have successfully used OpenVINO in production applications to solve real-world problems.
|
||||
* [OpenVINO Supported Models](./resources/supported_models.md) - Check which models OpenVINO supports on your hardware.
|
||||
* [Performance Benchmarks](./benchmarks/performance_benchmarks.md) - View results from benchmarking models with OpenVINO on Intel hardware.
|
||||
.. _additional-resources:
|
||||
|
||||
Additional Resources
|
||||
====================
|
||||
|
||||
* `OpenVINO Success Stories <https://www.intel.com/content/www/us/en/internet-of-things/ai-in-production/success-stories.html>`__ - See how Intel partners have successfully used OpenVINO in production applications to solve real-world problems.
|
||||
* :doc:`OpenVINO Supported Models <openvino_supported_models>` - Check which models OpenVINO supports on your hardware.
|
||||
* :doc:`Performance Benchmarks <openvino_docs_performance_benchmarks>` - View results from benchmarking models with OpenVINO on Intel hardware.
|
||||
|
||||
@endsphinxdirective
|
||||
@@ -9,16 +9,12 @@
|
||||
Interactive Tutorials (Python) <tutorials>
|
||||
Sample Applications (Python & C++) <openvino_docs_OV_UG_Samples_Overview>
|
||||
OpenVINO API 2.0 Transition <openvino_2_0_transition_guide>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
This section will help you get a hands-on experience with OpenVINO even if you are just starting
|
||||
to learn what OpenVINO is and how it works. It includes various types of learning materials
|
||||
accommodating different learning needs, which means you should find it useful if you are a beginning,
|
||||
as well as an experienced user.
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
| :doc:`Tutorials <tutorials>`
|
||||
| A collection of interactive Python tutorials. It introduces you to the OpenVINO™ toolkit explaining how to use the Python API and tools for optimized deep learning inference. The tutorials are available in Jupyter notebooks and can be run in your browser. No installation required.
|
||||
|
||||
|
||||
@@ -8,42 +8,54 @@
|
||||
|
||||
AccuracyAwareQuantization Method <accuracy_aware_README>
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
The Accuracy-aware Quantization algorithm allows to perform quantization while maintaining accuracy within a pre-defined range. Note that it should be used only if the [Default Quantization](@ref pot_default_quantization_usage) introduces a significant accuracy degradation. The reason for it not being the primary choice is its potential for performance degradation, due to some layers getting reverted to the original precision.
|
||||
The Accuracy-aware Quantization algorithm allows to perform quantization while maintaining accuracy within a pre-defined range. Note that it should be used only if the :doc:`Default Quantization <pot_default_quantization_usage>` introduces a significant accuracy degradation. The reason for it not being the primary choice is its potential for performance degradation, due to some layers getting reverted to the original precision.
|
||||
|
||||
To proceed with this article, make sure you have read how to use [Default Quantization](@ref pot_default_quantization_usage).
|
||||
To proceed with this article, make sure you have read how to use :doc:`Default Quantization <pot_default_quantization_usage>`.
|
||||
|
||||
> **NOTE**: The Accuracy-aware Quantization algorithm's behavior is different for the GNA `target_device`. In this case it searches for the best configuration and selects between INT8 and INT16 precisions for weights of each layer. The algorithm works for the `performance` preset only. It is not useful for the `accuracy` preset, since the whole model is already in INT16 precision.
|
||||
.. note::
|
||||
|
||||
The Accuracy-aware Quantization algorithm's behavior is different for the GNA ``target_device``. In this case it searches for the best configuration and selects between INT8 and INT16 precisions for weights of each layer. The algorithm works for the ``performance`` preset only. It is not useful for the ``accuracy`` preset, since the whole model is already in INT16 precision.
|
||||
|
||||
A script for Accuracy-aware Quantization includes four steps:
|
||||
|
||||
1. Prepare data and dataset interface.
|
||||
2. Define accuracy metric.
|
||||
3. Select quantization parameters.
|
||||
4. Define and run quantization process.
|
||||
|
||||
## Prepare data and dataset interface
|
||||
This step is the same as in [Default Quantization](@ref pot_default_quantization_usage). The only difference is that `__getitem__()` should return `(data, annotation)` or `(data, annotation, metadata)`. The `annotation` is required and its format should correspond to the expectations of the `Metric` class. The `metadata` is an optional field that can be used to store additional information required for post-processing.
|
||||
Prepare data and dataset interface
|
||||
##################################
|
||||
|
||||
## Define accuracy metric
|
||||
To control accuracy during optimization, the `openvino.tools.pot.Metric` interface should be implemented. Each implementation should override the following properties and methods:
|
||||
This step is the same as in :doc:`Default Quantization <pot_default_quantization_usage>`. The only difference is that ``__getitem__()`` should return ``(data, annotation)`` or ``(data, annotation, metadata)``. The ``annotation`` is required and its format should correspond to the expectations of the ``Metric`` class. The ``metadata`` is an optional field that can be used to store additional information required for post-processing.
|
||||
|
||||
Define accuracy metric
|
||||
######################
|
||||
|
||||
To control accuracy during optimization, the ``openvino.tools.pot.Metric`` interface should be implemented. Each implementation should override the following properties and methods:
|
||||
|
||||
**Properties**
|
||||
- `value` - returns the accuracy metric value for the last model output in a format of `Dict[str, numpy.array]`.
|
||||
- `avg_value` - returns the average accuracy metric over collected model results in a format of `Dict[str, numpy.array]`.
|
||||
- `higher_better` if a higher value of the metric corresponds to better performance, returns `True` , otherwise, `False`. The default implementation returns `True`.
|
||||
|
||||
- ``value`` - returns the accuracy metric value for the last model output in a format of ``Dict[str, numpy.array]``.
|
||||
- ``avg_value`` - returns the average accuracy metric over collected model results in a format of ``Dict[str, numpy.array]``.
|
||||
- ``higher_better`` if a higher value of the metric corresponds to better performance, returns ``True`` , otherwise, ``False``. The default implementation returns ``True``.
|
||||
|
||||
**Methods**
|
||||
- `update(output, annotation)` - calculates and updates the accuracy metric value, using the last model output and annotation. The model output and annotation should be passed in this method. It should also contain the model-specific post-processing in case the model returns the raw output.
|
||||
- `reset()` - resets collected accuracy metric.
|
||||
- `get_attributes()` - returns a dictionary of metric attributes:
|
||||
```
|
||||
{metric_name: {attribute_name: value}}
|
||||
```
|
||||
Required attributes:
|
||||
- `direction` - (`higher-better` or `higher-worse`) a string parameter defining whether metric value
|
||||
should be increased in accuracy-aware algorithms.
|
||||
- `type` - a string representation of a metric type. For example, "accuracy" or "mean_iou".
|
||||
|
||||
- ``update(output, annotation)`` - calculates and updates the accuracy metric value, using the last model output and annotation. The model output and annotation should be passed in this method. It should also contain the model-specific post-processing in case the model returns the raw output.
|
||||
- ``reset()`` - resets collected accuracy metric.
|
||||
- ``get_attributes()`` - returns a dictionary of metric attributes:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
{metric_name: {attribute_name: value}}
|
||||
|
||||
Required attributes:
|
||||
|
||||
- ``direction`` - (``higher-better`` or ``higher-worse``) a string parameter defining whether metric value should be increased in accuracy-aware algorithms.
|
||||
- ``type`` - a string representation of a metric type. For example, "accuracy" or "mean_iou".
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
Below is an example of the accuracy top-1 metric implementation with POT API:
|
||||
```python
|
||||
@@ -97,87 +109,97 @@ class Accuracy(Metric):
|
||||
'type': 'accuracy'}}
|
||||
```
|
||||
|
||||
An instance of the `Metric` implementation should be passed to `IEEngine` object responsible for model inference.
|
||||
@sphinxdirective
|
||||
|
||||
```python
|
||||
metric = Accuracy()
|
||||
engine = IEEngine(config=engine_config, data_loader=data_loader, metric=metric)
|
||||
```
|
||||
An instance of the ``Metric`` implementation should be passed to ``IEEngine`` object responsible for model inference.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
metric = Accuracy()
|
||||
engine = IEEngine(config=engine_config, data_loader=data_loader, metric=metric)
|
||||
|
||||
Select quantization parameters
|
||||
##############################
|
||||
|
||||
## Select quantization parameters
|
||||
Accuracy-aware Quantization uses the Default Quantization algorithm at the initialization step in such an order that all its parameters are also valid and can be specified. The only parameter required exclusively by Accuracy-aware Quantization is:
|
||||
- `"maximal_drop"` - the maximum accuracy drop which has to be achieved after the quantization. The default value is `0.01` (1%).
|
||||
|
||||
## Run quantization
|
||||
- ``"maximal_drop"`` - the maximum accuracy drop which has to be achieved after the quantization. The default value is ``0.01`` (1%).
|
||||
|
||||
The example code below shows a basic quantization workflow with accuracy control. `UserDataLoader()` is a placeholder for the implementation of `DataLoader`.
|
||||
Run quantization
|
||||
################
|
||||
|
||||
```python
|
||||
from openvino.tools.pot import IEEngine
|
||||
from openvino.tools.pot load_model, save_model
|
||||
from openvino.tools.pot import compress_model_weights
|
||||
from openvino.tools.pot import create_pipeline
|
||||
The example code below shows a basic quantization workflow with accuracy control. ``UserDataLoader()`` is a placeholder for the implementation of ``DataLoader``.
|
||||
|
||||
# Model config specifies the model name and paths to model .xml and .bin file
|
||||
model_config = Dict(
|
||||
{
|
||||
"model_name": "model",
|
||||
"model": path_to_xml,
|
||||
"weights": path_to_bin,
|
||||
}
|
||||
)
|
||||
.. code-block:: python
|
||||
|
||||
from openvino.tools.pot import IEEngine
|
||||
from openvino.tools.pot load_model, save_model
|
||||
from openvino.tools.pot import compress_model_weights
|
||||
from openvino.tools.pot import create_pipeline
|
||||
|
||||
# Model config specifies the model name and paths to model .xml and .bin file
|
||||
model_config = Dict(
|
||||
{
|
||||
"model_name": "model",
|
||||
"model": path_to_xml,
|
||||
"weights": path_to_bin,
|
||||
}
|
||||
)
|
||||
|
||||
# Engine config
|
||||
engine_config = Dict({"device": "CPU"})
|
||||
|
||||
algorithms = [
|
||||
{
|
||||
"name": "AccuracyAwareQuantization",
|
||||
"params": {
|
||||
"target_device": "ANY",
|
||||
"stat_subset_size": 300,
|
||||
'maximal_drop': 0.02
|
||||
},
|
||||
}
|
||||
]
|
||||
|
||||
# Step 1: Implement and create user's data loader.
|
||||
data_loader = UserDataLoader()
|
||||
|
||||
# Step 2: Implement and create user's data loader.
|
||||
metric = Accuracy()
|
||||
|
||||
# Step 3: Load the model.
|
||||
model = load_model(model_config=model_config)
|
||||
|
||||
# Step 4: Initialize the engine for metric calculation and statistics collection.
|
||||
engine = IEEngine(config=engine_config, data_loader=data_loader, metric=metric)
|
||||
|
||||
# Step 5: Create a pipeline of compression algorithms and run it.
|
||||
pipeline = create_pipeline(algorithms, engine)
|
||||
compressed_model = pipeline.run(model=model)
|
||||
|
||||
# Step 6 (Optional): Compress model weights to quantized precision
|
||||
# in order to reduce the size of the final .bin file.
|
||||
compress_model_weights(compressed_model)
|
||||
|
||||
# Step 7: Save the compressed model to the desired path.
|
||||
# Set save_path to the directory where the model should be saved.
|
||||
compressed_model_paths = save_model(
|
||||
model=compressed_model,
|
||||
save_path="optimized_model",
|
||||
model_name="optimized_model",
|
||||
)
|
||||
|
||||
# Step 8 (Optional): Evaluate the compressed model. Print the results.
|
||||
metric_results = pipeline.evaluate(compressed_model)
|
||||
|
||||
|
||||
# Engine config
|
||||
engine_config = Dict({"device": "CPU"})
|
||||
It is worth noting that now the ``evaluate`` method that can compute accuracy on demand is also available in the ``Pipeline`` object.
|
||||
|
||||
algorithms = [
|
||||
{
|
||||
"name": "AccuracyAwareQuantization",
|
||||
"params": {
|
||||
"target_device": "ANY",
|
||||
"stat_subset_size": 300,
|
||||
'maximal_drop': 0.02
|
||||
},
|
||||
}
|
||||
]
|
||||
In case when Accuracy-aware Quantization does not allow achieving the desired accuracy-performance trade-off, it is recommended to try Quantization-aware Training from :doc:`NNCF <tmo_introduction>`.
|
||||
|
||||
# Step 1: Implement and create user's data loader.
|
||||
data_loader = UserDataLoader()
|
||||
Examples
|
||||
########
|
||||
|
||||
# Step 2: Implement and create user's data loader.
|
||||
metric = Accuracy()
|
||||
* `Quantization of Object Detection model with control of accuracy <https://github.com/openvinotoolkit/openvino/tree/master/tools/pot/openvino/tools/pot/api/samples/object_detection>`__
|
||||
|
||||
# Step 3: Load the model.
|
||||
model = load_model(model_config=model_config)
|
||||
|
||||
# Step 4: Initialize the engine for metric calculation and statistics collection.
|
||||
engine = IEEngine(config=engine_config, data_loader=data_loader, metric=metric)
|
||||
|
||||
# Step 5: Create a pipeline of compression algorithms and run it.
|
||||
pipeline = create_pipeline(algorithms, engine)
|
||||
compressed_model = pipeline.run(model=model)
|
||||
|
||||
# Step 6 (Optional): Compress model weights to quantized precision
|
||||
# in order to reduce the size of the final .bin file.
|
||||
compress_model_weights(compressed_model)
|
||||
|
||||
# Step 7: Save the compressed model to the desired path.
|
||||
# Set save_path to the directory where the model should be saved.
|
||||
compressed_model_paths = save_model(
|
||||
model=compressed_model,
|
||||
save_path="optimized_model",
|
||||
model_name="optimized_model",
|
||||
)
|
||||
|
||||
# Step 8 (Optional): Evaluate the compressed model. Print the results.
|
||||
metric_results = pipeline.evaluate(compressed_model)
|
||||
```
|
||||
|
||||
It is worth noting that now the `evaluate` method that can compute accuracy on demand is also available in the `Pipeline` object.
|
||||
|
||||
In case when Accuracy-aware Quantization does not allow achieving the desired accuracy-performance trade-off, it is recommended to try Quantization-aware Training from [NNCF](@ref tmo_introduction).
|
||||
|
||||
## Examples
|
||||
|
||||
* [Quantization of Object Detection model with control of accuracy](https://github.com/openvinotoolkit/openvino/tree/master/tools/pot/openvino/tools/pot/api/samples/object_detection)
|
||||
@endsphinxdirective
|
||||
|
||||
Reference in new issue
Block a user