[Documentation]: Added description for NNCF PTQ (#14437)

docs/optimization_guide/nncf/ptq/basic_quantization_flow.md

@@ -0,0 +1,135 @@
+# Basic Quantization Flow {#basic_qauntization_flow}
+
+## Introduction
+
+The basic quantization flow is the simplest way to apply 8-bit quantization to the model. It is available for models in the following frameworks: PyTorch, TensorFlow 2.x, ONNX, and OpenVINO. The basic quantization flow is based on the following steps:
+* Set up an environment and install dependencies.
+* Prepare the **calibration dataset** that is used to estimate quantization parameters of the activations within the model.
+* Call the quantization API to apply 8-bit quantization to the model.
+
+## Set up an Environment
+
+It is recommended to set up a separate Python environment for quantization with NNCF. To do this, run the following command:
+```bash
+python3 -m venv nncf_ptq_env
+```
+Install all the packages required to instantiate the model object, for example, DL framework. After that, install NNCF on top of the environment:
+```bash
+pip install nncf
+```
+
+## Prepare a Calibration Dataset
+
+At this step, create an instance of the `nncf.Dataset` class that represents the calibration dataset. The `nncf.Dataset` class can be a wrapper over the framework dataset object that is used for model training or validation. The class constructor receives the dataset object and the transformation function. For example, if you use PyTorch, you can pass an instance of the `torch.utils.data.DataLoader` object. 
+
+The transformation function is a function that takes a sample from the dataset and returns data that can be passed to the model for inference. For example, this function can take a tuple of a data tensor and labels tensor, and return the former while ignoring the latter. The transformation function is used to avoid modifying the dataset code to make it compatible with the quantization API. The function is applied to each sample from the dataset before passing it to the model for inference. The following code snippet shows how to create an instance of the `nncf.Dataset` class:
+
+@sphinxtabset
+
+@sphinxtab{PyTorch}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_torch.py dataset
+
+@endsphinxtab
+
+@sphinxtab{ONNX}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_onnx.py dataset
+
+@endsphinxtab
+
+@sphinxtab{OpenVINO}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_openvino.py dataset
+
+@endsphinxtab
+
+@sphinxtab{TensorFlow}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py dataset
+
+@endsphinxtab
+
+@endsphinxtabset
+
+If there is no framework dataset object, you can create your own entity that implements the `Iterable` interface in Python and returns data samples feasible for inference. In this case, a transformation function is not required.
+
+
+## Run a Quantized Model
+
+Once the dataset is ready and the model object is instantiated, you can apply 8-bit quantization to it:
+@sphinxtabset
+
+@sphinxtab{PyTorch}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_torch.py quantization
+
+@endsphinxtab
+
+@sphinxtab{ONNX}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_torch.py quantization
+
+@endsphinxtab
+
+@sphinxtab{OpenVINO}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_torch.py quantization
+
+@endsphinxtab
+
+@sphinxtab{TensorFlow}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py quantization
+
+@endsphinxtab
+
+@endsphinxtabset
+
+> **NOTE**: The `model` is an instance of the `torch.nn.Module` class for PyTorch, `onnx.ModelProto` for ONNX, and `openvino.runtime.Model` for OpenVINO.
+
+After that the model can be exported into th OpenVINO Intermediate Representation if needed and run faster with OpenVINO.
+
+## Tune quantization parameters
+
+`nncf.quantize()` function has several parameters that allow to tune quantization process to get more accurate model. Below is the list of parameters and their description:
+* `model_type` - used to specify quantization scheme required for specific type of the model. For example, **Transformer** models (BERT, distillBERT, etc.) require a special quantization scheme to preserve accuracy after quantization.
+  ```python
+  nncf.quantize(model, dataset, model_type=nncf.ModelType.Transformer)
+  ```
+* `preset` - defines quantization scheme for the model. Two types of presets are available:
+  * `PERFORMANCE` (default) - defines symmetric quantization of weigths and activations
+  * `MIXED` - weights are quantized with symmetric quantization and the activations are quantized with asymmetric quantization. This preset is recommended for models with non-ReLU and asymmetric activation funstions, e.g. ELU, PReLU, GELU, etc.
+  ```python
+    nncf.quantize(model, dataset, preset=nncf.Preset.MIXED)
+  ```
+* `fast_bias_correction` - enables more accurate bias (error) correction algorithm that can be used to improve accuracy of the model. This parameter is available only for OpenVINO representation. `True` is used by default.
+  ```python
+    nncf.quantize(model, dataset, fast_bias_correction=False)
+  ```
+* `subset_size` - defines the number of samples from the calibration dataset that will be used to estimate quantization parameters of activations. The default value is 300.
+  ```python
+    nncf.quantize(model, dataset, subset_size=1000)
+  ```
+* `ignored_scope` - this parameter can be used to exclude some layers from quantization process. For example, if you want to exclude the last layer of the model from quantization. Below are some examples of how to use this parameter:
+  * Exclude by layer name:
+    ```python
+    names = ['layer_1', 'layer_2', 'layer_3']
+    nncf.quantize(model, dataset, ignored_scope=nncf.IgnoredScope(names=names))
+    ```
+  * Exclude by layer type:
+    ```python
+    types = ['Conv2d', 'Linear']
+    nncf.quantize(model, dataset, ignored_scope=nncf.IgnoredScope(types=types))
+    ```
+  * Exclude by regular expression:
+    ```python
+    regex = '.*layer_.*'
+    nncf.quantize(model, dataset, ignored_scope=nncf.IgnoredScope(patterns=regex))
+    ```
+
+If the accuracy of the quantized model is not satisfactory, you can try to use the [Quantization with accuracy control](@ref quantization_w_accuracy_control) flow.
+
+## See also
+
+* [Example of basic quantization flow in PyTorch](https://github.com/openvinotoolkit/nncf/tree/develop/examples/post_training_quantization/torch/mobilenet_v2)

docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py

@@ -0,0 +1,49 @@
+# Copyright (C) 2018-2022 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+#! [dataset]
+import nncf
+import torch
+
+calibration_loader = torch.utils.data.DataLoader(...)
+
+def transform_fn(data_item):
+    images, _ = data_item
+    return images
+
+calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
+validation_dataset = nncf.Dataset(calibration_loader, transform_fn)
+#! [dataset]
+
+#! [validation]
+import numpy as np
+import torch
+import openvino
+from sklearn.metrics import accuracy_score
+
+def validate(model: openvino.runtime.CompiledModel, 
+             validation_loader: torch.utils.data.DataLoader) -> float:
+    predictions = []
+    references = []
+
+    output = model.outputs[0]
+
+    for images, target in validation_loader:
+        pred = model(images)[output]
+        predictions.append(np.argmax(pred, axis=1))
+        references.append(target)
+
+    predictions = np.concatenate(predictions, axis=0)
+    references = np.concatenate(references, axis=0)  
+    return accuracy_score(predictions, references)
+#! [validation]
+
+#! [quantization]
+model = ... # openvino.runtime.Model object
+
+quantized_model = nncf.quantize_with_accuracy_control(model,
+                        calibration_dataset=calibration_dataset,
+                        validation_dataset=validation_dataset,
+                        validation_fn=validate,
+                        max_drop=0.01)
+#! [quantization]

docs/optimization_guide/nncf/ptq/code/ptq_onnx.py

@@ -0,0 +1,22 @@
+# Copyright (C) 2018-2022 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+#! [dataset]
+import nncf
+import torch
+
+calibration_loader = torch.utils.data.DataLoader(...)
+
+def transform_fn(data_item):
+    images, _ = data_item
+    return {input_name: images.numpy()} # input_name should be taken from the model, 
+                                        # e.g. model.graph.input[0].name
+
+calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
+#! [dataset]
+
+#! [quantization]
+model = ... # onnx.ModelProto object
+
+quantized_model = nncf.quantize(model, calibration_dataset)
+#! [quantization]

docs/optimization_guide/nncf/ptq/code/ptq_openvino.py

@@ -0,0 +1,21 @@
+# Copyright (C) 2018-2022 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+#! [dataset]
+import nncf
+import torch
+
+calibration_loader = torch.utils.data.DataLoader(...)
+
+def transform_fn(data_item):
+    images, _ = data_item
+    return images.numpy()
+
+calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
+#! [dataset]
+
+#! [quantization]
+model = ... # openvino.runtime.Model object
+
+quantized_model = nncf.quantize(model, calibration_dataset)
+#! [quantization]

docs/optimization_guide/nncf/ptq/code/ptq_tensorflow.py

@@ -0,0 +1,21 @@
+# Copyright (C) 2018-2022 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+#! [dataset]
+import nncf
+import tensorflow_datasets as tfds
+
+calibration_loader = tfds.load(...)
+
+def transform_fn(data_item):
+    images, _ = data_item
+    return images
+
+calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
+#! [dataset]
+
+#! [quantization]
+model = ... # tensorflow.Module object
+
+quantized_model = nncf.quantize(model, calibration_dataset)
+#! [quantization]

docs/optimization_guide/nncf/ptq/code/ptq_torch.py

@@ -0,0 +1,21 @@
+# Copyright (C) 2018-2022 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+#! [dataset]
+import nncf
+import torch
+
+calibration_loader = torch.utils.data.DataLoader(...)
+
+def transform_fn(data_item):
+    images, _ = data_item
+    return images
+
+calibration_dataset = nncf.Dataset(calibration_loader, transform_fn)
+#! [dataset]
+
+#! [quantization]
+model = ... # torch.nn.Module object
+
+quantized_model = nncf.quantize(model, calibration_dataset)
+#! [quantization]

docs/optimization_guide/nncf/ptq/ptq_introduction.md

@@ -0,0 +1,22 @@
+# Post-training Quantization w/ NNCF (new) {#nncf_ptq_introduction}
+
+@sphinxdirective
+
+.. toctree::
+   :maxdepth: 1
+   :hidden:
+
+   basic_qauntization_flow
+   quantization_w_accuracy_control
+
+@endsphinxdirective
+
+Neural Network Compression Framework (NNCF) provides a new post-training quantization API available in Python that is aimed at reusing the code for model training or validation that is usually available with the model in the source framework, for example, PyTorch* or TensroFlow*. The API is cross-framework and currently supports models representing in the following frameworks: PyTorch, TensorFlow 2.x, ONNX, and OpenVINO. 
+This API has two main capabilities to apply 8-bit post-training quantization:
+* [Basic quantization](@ref basic_qauntization_flow) - the simplest quantization flow that allows to apply 8-bit integer quantization to the model.
+* [Quantization with accuracy control](@ref quantization_w_accuracy_control) - the most advanced quantization flow that allows to apply 8-bit quantization to the model with accuracy control.
+
+## See also
+
+* [NNCF GitHub](https://github.com/openvinotoolkit/nncf)
+* [Optimizing Models at Training Time](@ref tmo_introduction)

docs/optimization_guide/nncf/ptq/quantization_w_accuracy_control.md

@@ -0,0 +1,66 @@
+# Quantizing with accuracy control {#quantization_w_accuracy_control}
+
+## Introduction
+
+This is the advanced quantization flow that allows to apply 8-bit quantization to the model with control of accuracy metric. This is achieved by keeping the most impactful operations within the model in the original precision. The flow is based on the [Basic 8-bit quantization](@ref basic_qauntization_flow) and has the following differences:
+* Besided the calibration dataset, a **validation dataset** is required to compute accuracy metric. They can refer to the same data in the simplest case.
+* **Validation function**, used to compute accuracy metric is required. It can be a function that is already available in the source framework or a custom function.
+* Since accuracy validation is run several times during the quantization process, quantization with accuracy control can take more time than the [Basic 8-bit quantization](@ref basic_qauntization_flow) flow.
+* The resulted model can provide smaller performance improvement than the [Basic 8-bit quantization](@ref basic_qauntization_flow) flow because some of the operations are kept in the original precision.
+
+> **NOTE**: Currently, this flow is available only for models in OpenVINO representation.
+
+The steps for the quantizatation with accuracy control are described below.
+
+## Prepare datasets
+
+This step is similar to the [Basic 8-bit quantization](@ref basic_qauntization_flow) flow. The only difference is that two datasets, calibration and validation, are required.
+
+@sphinxtabset
+
+@sphinxtab{OpenVINO}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py dataset
+
+@endsphinxtab
+
+@endsphinxtabset
+
+## Prepare validation function
+
+Validation funtion receives `openvino.runtime.CompiledModel` object and 
+validation dataset and returns accuracy metric value. The following code snippet shows an example of validation function for OpenVINO model:
+
+@sphinxtabset
+
+@sphinxtab{OpenVINO}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py validation
+
+@endsphinxtab
+
+@endsphinxtabset
+
+## Run quantization with accuracy control
+
+Now, you can run quantization with accuracy control. The following code snippet shows an example of quantization with accuracy control for OpenVINO model:  
+
+@sphinxtabset
+
+@sphinxtab{OpenVINO}
+
+@snippet docs/optimization_guide/nncf/ptq/code/ptq_aa_openvino.py quantization
+
+@endsphinxtab
+
+@endsphinxtabset
+
+`max_drop` defines the accuracy drop threshold. The quantization process stops when the degradation of accuracy metric on the validation dataset is less than the `max_drop`. 
+
+`nncf.quantize_with_accuracy_control()` API supports all the parameters of `nncf.quantize()` API. For example, you can use `nncf.quantize_with_accuracy_control()` to quantize a model with a custom configuration.
+
+## See also
+
+* [Optimizing Models at Training Time](@ref tmo_introduction)
+
+