IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Documentation fixes (#11044)

Browse Source 
* Benchmark app usage

* Fixed link to the devices

* More fixes

* Update docs/OV_Runtime_UG/multi_device.md

Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com>

* Removed several hardcoded links

Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com>
This commit is contained in:
Ilya Lavrenov
2022-03-18 18:24:17 +03:00
committed by GitHub
parent 56e626d4b1
commit c3b05978e2
19 changed files with 36 additions and 405 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/MO_DG/prepare_model/convert_model/Converting_Model.md
View File

@@ -12,7 +12,7 @@ To set up static shapes, Model Optimizer provides the `--input_shape` parameter.
The same functionality is also available in runtime via `reshape` method, please refer to [Changing input shapes](../../../OV_Runtime_UG/ShapeInference.md).
For more information about dynamic shapes in runtime, refer to [Dynamic Shapes](../../../OV_Runtime_UG/ov_dynamic_shapes.md)
OpenVINO Runtime API can have limitations to infer models with undefined dimensions on some hardware.
OpenVINO Runtime API can have limitations to infer models with undefined dimensions on some hardware (see [Features support matrix](../../../OV_Runtime_UG/supported_plugins/Device_Plugins.md) for reference).
In this case, the `--input_shape` parameter and the `reshape` method can help to resolve undefined dimensions.
Sometimes Model Optimizer is unable to convert models out-of-the-box (only the `--input_model` parameter is specified).

2
docs/MO_DG/prepare_model/convert_model/tf_specific/Convert_GNMT_From_Tensorflow.md
View File

@@ -240,7 +240,7 @@ Outputs of the model:
1. With benchmark app:
```sh
python3 benchmark_app.py -m <path to the generated GNMT IR> -d CPU
benchmark_app -m <path to the generated GNMT IR> -d CPU
```

6
docs/OV_Runtime_UG/Int8Inference.md
View File

@@ -30,14 +30,12 @@ At runtime, the quantized model is loaded to the plugin. The plugin uses the `Lo
Let's explore quantized [TensorFlow* implementation of the ResNet-50](https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/resnet-50-tf) model. Use [Model Downloader](@ref omz_tools_downloader) to download the `FP16` model from [OpenVINO™ Toolkit - Open Model Zoo repository](https://github.com/openvinotoolkit/open_model_zoo):
> **NOTE**: If you installed OpenVINO with pip, use `omz_downloader` and `omz_quantizer` instead of `download.py` and `quantize.py`. See [Open Model Zoo documentation](https://github.com/openvinotoolkit/open_model_zoo/tree/master/tools/model_tools#model-downloader-usage). Replace `./benchmark_app` with `benchmark_app`.
```sh
<omz_dir>/tools/downloader/downloader.py --name resnet-50-tf --precisions FP16-INT8
omz_downloader --name resnet-50-tf --precisions FP16-INT8
```
After that you should quantize the model with the [Model Quantizer](@ref omz_tools_downloader) tool.
```sh
<omz_dir>/tools/downloader/quantizer.py --model_dir public/resnet-50-tf --dataset_dir <DATASET_DIR> --precisions=FP16-INT8
omz_quantizer --model_dir public/resnet-50-tf --dataset_dir <DATASET_DIR> --precisions=FP16-INT8
```
The simplest way to infer the model and collect performance counters is the [Benchmark Application](../../samples/cpp/benchmark_app/README.md):

2
docs/OV_Runtime_UG/ShapeInference.md
View File

@@ -166,7 +166,7 @@ To feed input data of a shape that is different from the model input shape, resh
Once the input shape of IENetwork is set, call the `IECore.load_network` method to get an ExecutableNetwork object for inference with updated shapes.
There are other approaches to reshape the model during the stage of IR generation or [OpenVINO model](https://docs.openvino.ai/latest/openvino_docs_nGraph_DG_PythonAPI.html#create_an_ngraph_function_from_a_graph) creation.
There are other approaches to reshape the model during the stage of IR generation or [OpenVINO model](model_representation.md) creation.
Practically, some models are not ready to be reshaped. In this case, a new input shape cannot be set with the Model Optimizer or the `IENetwork.reshape` method.

70
docs/OV_Runtime_UG/auto_device_selection.md
View File

@@ -108,38 +108,16 @@ Alternatively, you can combine all the individual device settings into single co
Note that every OpenVINO sample or application that supports the "-d" (which stands for "device") command-line option transparently accepts the Auto-device. The Benchmark Application is the best example of the optimal usage of the Auto-device. You do not need to set the number of requests and CPU threads, as the application provides optimal out-of-the-box performance. Below is the example command-line to evaluate AUTO performance with that:
@sphinxdirective
.. tab:: Package, Docker, open-source installation
.. code-block:: sh
./benchmark_app.py –d AUTO –m <model>
.. tab:: pip installation
.. code-block:: sh
benchmark_app –d AUTO –m <model>
@endsphinxdirective
```sh
benchmark_app –d AUTO –m <model>
```
You can also use the auto-device with limit device choice:
@sphinxdirective
.. tab:: Package, Docker, open-source installation
.. code-block:: sh
./benchmark_app.py –d AUTO:CPU,GPU –m <model>
.. tab:: pip installation
.. code-block:: sh
benchmark_app –d AUTO:CPU,GPU –m <model>
@endsphinxdirective
```sh
benchmark_app –d AUTO:CPU,GPU –m <model>
```
**NOTES:**
* The default CPU stream is 1 if using `-d AUTO`.
@@ -297,36 +275,12 @@ exec_net = ie.load_network(network=net, device_name="AUTO", config=full_config)
Note that every OpenVINO sample or application that supports the "-d" (which stands for "device") command-line option transparently accepts the Auto-device. The Benchmark Application is the best example of the optimal usage of the Auto-device. You do not need to set the number of requests and CPU threads, as the application provides optimal out-of-the-box performance. Below is the example command-line to evaluate AUTO performance with that:
@sphinxdirective
.. tab:: Package, Docker, open-source installation
.. code-block:: sh
./benchmark_app.py –d AUTO –m <model>
.. tab:: pip installation
.. code-block:: sh
benchmark_app –d AUTO –m <model>
@endsphinxdirective
```sh
benchmark_app –d AUTO –m <model>
```
You can also use the auto-device with limit device choice:
@sphinxdirective
.. tab:: Package, Docker, open-source installation
.. code-block:: sh
./benchmark_app.py –d AUTO:CPU,GPU –m <model>
.. tab:: pip installation
.. code-block:: sh
benchmark_app –d AUTO:CPU,GPU –m <model>
@endsphinxdirective
> **NOTE**: If you installed OpenVINO with pip, use `benchmark_app -d AUTO:CPU,GPU -m <model>`
```sh
benchmark_app –d AUTO:CPU,GPU –m <model>
```

4
docs/OV_Runtime_UG/multi_device.md
View File

@@ -286,11 +286,9 @@ Note that while the performance of accelerators works well with Multi-Device, th
Every OpenVINO sample that supports the `-d` (which stands for "device") command-line option transparently accepts Multi-Device. The [Benchmark application](../../tools/benchmark_tool/README.md) is the best reference for the optimal usage of Multi-Device. As discussed earlier, you do not need to set up the number of requests, CPU streams or threads because the application provides optimal performance out of the box. Below is an example command to evaluate CPU+GPU performance with the Benchmark application:
```sh
./benchmark_app.py –d MULTI:CPU,GPU –m <model>
benchmark_app –d MULTI:CPU,GPU –m <model>
```
> **NOTE**: If you installed OpenVINO with pip, use `benchmark_app -d MULTI:CPU,GPU -m <model>`
The Multi-Device plugin supports FP16 IR files. The CPU plugin automatically upconverts it to FP32 and the other devices support it natively. Note that no demos are (yet) fully optimized for Multi-Device, by means of supporting the OPTIMAL_NUMBER_OF_INFER_REQUESTS metric, using the GPU streams/throttling, and so on.
### Video: MULTI Plugin

1
docs/OV_Runtime_UG/openvino_intro.md
View File

@@ -20,7 +20,6 @@
openvino_docs_OV_UG_Automatic_Batching
openvino_docs_IE_DG_network_state_intro
openvino_docs_OV_Runtime_UG_Python_API_exclusives
openvino_2_0_transition_guide
@endsphinxdirective

1
docs/documentation.md
View File

@@ -17,6 +17,7 @@
:hidden:
openvino_docs_OV_Runtime_User_Guide
openvino_2_0_transition_guide
openvino_docs_install_guides_deployment_manager_tool
openvino_inference_engine_tools_compile_tool_README

10
docs/get_started/get_started_demos.md
View File

@@ -11,8 +11,6 @@ You will perform the following steps:
4. <a href="run-image-classification">Run inference on the sample and see the results:</a>
- <a href="run-image-classification">Image Classification Code Sample</a>
If you installed OpenVINO™ via `pip` you can quickly getting started with the product by using these [tutorials](https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks).
This guide assumes you completed all installation and configuration steps. If you have not yet installed and configured the toolkit:
@sphinxdirective
@@ -94,13 +92,13 @@ This guide uses the OpenVINO™ Model Downloader to get pre-trained models. You
* List the models available in the downloader
``` sh
info_dumper --print_all
omz_info_dumper --print_all
```
* Use `grep` to list models that have a specific name pattern
``` sh
info_dumper --print_all | grep <model_name>
omz_info_dumper --print_all | grep <model_name>
```
* Use Model Downloader to download models.
@@ -244,7 +242,7 @@ Create an `<ir_dir>` directory to contain the model's Intermediate Representatio
@endsphinxdirective
The OpenVINO Runtime can perform inference on different precision formats, such as FP32, FP16, or INT8. To generate an IR with a specific precision, run the Model Optimizer with the appropriate `--data_type` option.
The OpenVINO Runtime can infer models where floating-point weights are [compressed to FP16](../MO_DG/prepare_model/FP16_Compression.md). To generate an IR with a specific precision, run the Model Optimizer with the appropriate `--data_type` option.
Generic Model Optimizer script:
@@ -370,7 +368,7 @@ To run the **Image Classification** code sample with an input image using the IR
@endsphinxdirective
The following commands run the Image Classification Code Sample using the [`dog.bmp`](https://storage.openvinotoolkit.org/data/test_data/images/224x224/dog.bmp) file as an input image, the model in IR format from the `ir` directory, and on different hardware devices:
The following commands run the Image Classification Code Sample using the [dog.bmp](https://storage.openvinotoolkit.org/data/test_data/images/224x224/dog.bmp) file as an input image, the model in IR format from the `ir` directory, and on different hardware devices:
**CPU:**
@sphinxdirective

70
docs/how_tos/MonoDepth_how_to.md
View File

@@ -1,70 +0,0 @@
# OpenVINO™ MonoDepth Python Demo
This tutorial describes the example from the following YouTube* video:
///
To learn more about how to run the MonoDepth Python* demo application, refer to the [documentation](https://docs.openvino.ai/latest/omz_demos_monodepth_demo_python.html).
Tested on OpenVINO™ 2021, Ubuntu 18.04.
## 1. Set Environment
Define the OpenVINO™ install directory:
```
export OV=/opt/intel/openvino_2022/
```
Define the working directory. Make sure the directory exist:
```
export WD=~/MonoDepth_Python/
```
## 2. Install Prerequisits
Initialize OpenVINO™:
```
source $OV/setupvars.sh
```
Install the Model Optimizer prerequisites:
```
cd $OV/tools/model_optimizer/install_prerequisites/
sudo ./install_prerequisites.sh
```
Install the Model Downloader prerequisites:
```
cd $OV/extras/open_model_zoo/tools/downloader/
python3 -mpip install --user -r ./requirements.in
sudo python3 -mpip install --user -r ./requirements-pytorch.in
sudo python3 -mpip install --user -r ./requirements-caffe2.in
```
## 3. Download Models
Download all models from the Demo Models list:
```
python3 $OV/extras/open_model_zoo/tools/downloader/downloader.py --list $OV/deployment_tools/inference_engine/demos/python_demos/monodepth_demo/models.lst -o $WD
```
## 4. Convert Models to Intermediate Representation (IR)
Use the convert script to convert the models to ONNX*, and then to IR format:
```
cd $WD
python3 $OV/extras/open_model_zoo/tools/downloader/converter.py --list $OV/deployment_tools/inference_engine/demos/python_demos/monodepth_demo/models.lst
```
## 5. Run Demo
Install required Python modules, for example, kiwisolver or cycler, if you get missing module indication.
Use your input image:
```
python3 $OV/inference_engine/demos/python_demos/monodepth_demo/monodepth_demo.py -m $WD/public/midasnet/FP32/midasnet.xml -i input-image.jpg
```
Check the result depth image:
```
eog disp.png &
```
You can also try to use another model. Note that the algorithm is the same, but the depth map will be different.

163
docs/how_tos/POT_how_to_example.md
View File

@@ -1,163 +0,0 @@
# Post-Training Optimization Tool - A real example
This tutorial describes the example from the following YouTube* video:
https://www.youtube.com/watch?v=cGQesbWuRhk&t=49s
Watch this video to learn the basics of Post-training Optimization Tool (POT):
https://www.youtube.com/watch?v=SvkI25Ca_SQ
The example has been tested on OpenVINO™ 2021 on Ubuntu 18.04 Operating System.
## 1. Installation
Install OpenVINO™ toolkit and Model Optimizer, Accuracy Checker, and Post-training Optimization Tool components.
1. Define the OpenVINO™ install directory:
```
export OV=/opt/intel/openvino_2022/
```
2. Install the Model Optimizer prerequisites:
```
cd $OV/tools/model_optimizer/install_prerequisites
sudo ./install_prerequisites.sh
```
3. Install the Accuracy Checker requirements:
```
cd $OV/tools/accuracy_checker
sudo python3 setup.py install
```
4. Install the Post-training Optimization Tool:
```
cd $OV/tools/post_training_optimization_toolkit
sudo python3 setup.py install
```
## 2. Download Model
This tutorial describes MobileNet v2 model from PyTorch* framework. You can choose any other model.
Download the MobileNet v2 PyTorch* model using the commands below:
```
mkdir ~/POT
```
```
cd ~/POT
```
```
python3 $OV/extras/open_model_zoo/tools/downloader/downloader.py --name mobilenet-v2-pytorch -o .
```
## 3. Prepare Model for Inference
Install requirements for PyTorch using the commands below:
```
cd $OV/extras/open_model_zoo/tools/downloader
```
```
python3 -mpip install --user -r ./requirements-pytorch.in
```
You can find the parameters for Mobilnet v2 conversion here:
```
vi /opt/intel/openvino_2022/extras/open_model_zoo/models/public/mobilenet-v2-pytorch/model.yml
```
Convert the model from PyTorch to ONNX*:
```
cd ~/POT/public/mobilenet-v2-pytorch
python3 /opt/intel/openvino_2022/extras/open_model_zoo/tools/downloader/pytorch_to_onnx.py \
--model-name=MobileNetV2 \
--model-path=. \
--weights=mobilenet-v2.pth \
--import-module=MobileNetV2 \
--input-shape=1,3,224,224 /
--output-file=mobilenet-v2.onnx \
--input-names=data \
--output-names=prob
```
Convert the model from ONNX to the OpenVINO™ Intermediate Representation (IR):
```
mo \
-m mobilenet-v2.onnx \
--input=data \
--mean_values=data[123.675,116.28,103.53] \
--scale_values=data[58.624,57.12,57.375] \
--reverse_input_channels \
--output=prob
```
Move the IR files to my directory:
```
mv mobilenet-v2.xml ~/POT/model.xml
mv mobilenet-v2.bin ~/POT/model.bin
```
## 4. Edit Configurations
Edit the configuration files:
```
sudo vi $OV/tools/accuracy_checker/dataset_definitions.yml
(edit imagenet_1000_classes)
```
```
export DEFINITIONS_FILE=/opt/intel/openvino_2022/tools/accuracy_checker/dataset_definitions.yml
```
Copy the JSON file to my directory and edit:
```
cp $OV/tools/post_training_optimization_toolkit/configs/examples/quantization/classification/mobilenetV2_pytorch_int8.json ~/POT
```
```
vi mobilenetV2_pytorch_int8.json
```
Copy the YML file to my directory and edit:
```
cp /opt/intel/openvino_2022/tools/accuracy_checker/configs/mobilenet-v2.yml ~/POT
```
```
vi mobilenet-v2.yml
```
## 5. Run Baseline
Run Accuracy Checker on the original model:
```
accuracy_check -c mobilenet-v2.yml
```
Install the Benchmark Tool first. To learn more about Benchmark Tool refer to [Benchmark C++ Tool](https://docs.openvino.ai/latest/openvino_inference_engine_samples_benchmark_app_README.html)
or [Benchmark Python* Tool](https://docs.openvino.ai/latest/openvino_inference_engine_tools_benchmark_tool_README.html).
Run performance benchmark:
```
~/inference_engine_cpp_samples_build/intel64/Release/benchmark_app -m ~/POT/model.xml
```
## 6. Run Integer Calibration
You can edit the JSON file to switch between two modes of calibration:
- AccuracyAwareQuantization
- DefaultQuantization
```
pot --config /home/~/POT/mobilenetV2_pytorch_int8.json \
--output-dir /home/~/POT/ \
--evaluate \
--log-level INFO
```
Run the Benchmark Tool for the calibrated model. Make sure the name contains `DafultQuantization/.../optimized/...`
```
~/inference_engine_cpp_samples_build/intel64/Release/benchmark_app -m mobilenetv2_DefaultQuantization/2021-03-07/optimized/mobilenetv2.xml
```

86
docs/how_tos/how-to-links.md
View File

@@ -1,86 +0,0 @@
# "Hot Topic" How-To Links
## Blogs & Articles
* [Maximize CPU Inference Performance with Improved Threads and Memory Management in Intel® Distribution of OpenVINO™ toolkit](https://www.edge-ai-vision.com/2020/03/maximize-cpu-inference-performance-with-improved-threads-and-memory-management-in-intel-distribution-of-openvino-toolkit/)
* [Simplifying Cloud to Edge AI Deployments with the Intel® Distribution of OpenVINO™ Toolkit, Microsoft Azure, and ONNX Runtime](https://www.intel.ai/microsoft-azure-openvino-toolkit/#gs.11oa13)
* [Streamline your Intel® Distribution of OpenVINO™ Toolkit development with Deep Learning Workbench](https://www.intel.ai/openvino-dlworkbench/#gs.wwj3bq)
* [Enhanced Low-Precision Pipeline to Accelerate Inference with OpenVINO Toolkit](https://www.intel.ai/open-vino-low-precision-pipeline/)
* [Improving DL Performance Using Binary Convolution Support in OpenVINO Toolkit](https://www.intel.ai/binary-convolution-openvino)
* [Automatic Multi-Device Inference with the Intel® Distribution of OpenVINO™ toolkit](https://www.intel.ai/automatic-multi-device-inference-with-intel-distribution-of-openvino-toolkit/)
* [CPU Inference Performance Boost with “Throughput” Mode in the Intel® Distribution of OpenVINO™ Toolkit](https://www.intel.ai/cpu-inference-performance-boost-openvino/)
* [Introducing int8 quantization for fast CPU inference using OpenVINO](https://www.intel.ai/introducing-int8-quantization-for-fast-cpu-inference-using-openvino/)
* [Accelerate Vision-based AI with Intel® Distribution of OpenVINO™ Toolkit](https://www.intel.ai/accelerate-vision-based-ai-with-intel-distribution-of-openvino-toolkit/)
## Custom Operations Guide
To learn about what is *custom operation* and how to work with them in the Deep Learning Deployment Toolkit, see the [Custom Operations Guide](../Extensibility_UG/Intro.md).
## Introducing OpenVINO™ and Computer Vision | IoT Developer Show Season 2 | Intel Software
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/M6Nyh2JDLQs" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## OpenVINO™ Toolkit and Two Hardware Development Kits | IoT Developer Show Season 2 | Intel Software
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/GtJPBYjuyVU" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## Intel Demonstration of High Performance Vision Deployment - The OpenVINO Toolkit in Action
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/1_iI_4Zgufw" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## Computer Vision at the Edge with OpenVINO by Krishnakumar Shetti at ODSC_India
@sphinxdirective
..raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/RfRCrq35LXg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## Model optimizer concept
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/Kl1ptVb7aI8" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## Computer Vision with Intel
@sphinxdirective
.. raw:: html
<iframe allowfullscreen mozallowfullscreen msallowfullscreen oallowfullscreen webkitallowfullscreen width="560" height="315" src="https://www.youtube.com/embed/FZZD4FCvO9c" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
@endsphinxdirective
## Case Studies
|| Link to tutorial |
|:---:|:---:|
|![dl_healthcare]"" | [Deep Learning for Healthcare Imaging](https://ai.intel.com/wp-content/uploads/sites/53/2018/03/IntelSWDevTools_OptimizeDLforHealthcare.pdf) |
|![performance-boost-dl]"" | [Performance Boost for a Deep Learning Algorithm](https://software.intel.com/en-us/download/geovision-case-study) |
|![digital-security-surveillance]"" | [Digital Security & Surveillance Solutions](https://software.intel.com/en-us/download/agent-vi-case-study) |
|![robotics-with-AI]"" | [Robotics with AI for Industry 4.0](https://software.intel.com/en-us/download/intel-vision-accelerator-design-products-intel-nexcom-solution-brief) |
|![people-counter-syestem]"" | [People Counter Reference Implementation](https://software.intel.com/en-us/articles/iot-reference-implementation-people-counter) |
[dl_healthcare]: ../img/DL-for-Healthcare-Imaging.jpg
[performance-boost-dl]: ../img/performance-boost-DL-algorithm.jpg
[digital-security-surveillance]: ../img/digital-security-surveillance.jpg
[robotics-with-AI]: ../img/robotics-with-AI.jpg
[people-counter-syestem]: ../img/people-counter-syestem.jpg

2
docs/index.rst
View File

@@ -29,7 +29,7 @@ OpenVINO™ Documentation
alt="OpenVINO allows to process models built with Caffe, Keras, mxnet, TensorFlow, ONNX, and PyTorch. They can be easily optimized and deployed on devices running Windows, Linux, or MacOS." />
<div style="clear:both;"> </div>
<p>Check the full range of supported hardware in the
<a href="openvino_docs_IE_DG_supported_plugins_Supported_Devices.html"> Supported Devices page</a> and see how it stacks up in our
<a href="openvino_docs_OV_UG_Working_with_devices.html"> Supported Devices page</a> and see how it stacks up in our
<a href="openvino_docs_performance_benchmarks.html"> Performance Benchmarks page.</a> <br />
Supports deployment on Windows, Linux, and macOS.
</p>

8
docs/install_guides/deployment-manager-tool.md
View File

@@ -170,11 +170,13 @@ To deploy the OpenVINO Runtime components from the development machine to the ta
.. code-block:: sh
tar xf openvino_deployment_package.tar.gz -C <destination_dir>
.. tab:: Windows
Use the archiver of your choice to unzip the file.
.. code-block:: bat
Use the archiver of your choice to unzip the file.
.. tab:: macOS
.. code-block:: sh
@@ -209,7 +211,7 @@ The package is unpacked to the destination directory and the following files and
.. tab:: Windows
.. code-block:: bat
.. code-block:: bat
cd <destination_dir>\openvino\
.\setupvars.bat

6
docs/install_guides/pypi-openvino-dev.md
View File

@@ -8,10 +8,10 @@ OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applicatio
| Component | Console Script | Description |
|------------------|---------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [Model Optimizer](https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html) | `mo` |**Model Optimizer** imports, converts, and optimizes models that were trained in popular frameworks to a format usable by OpenVINO components. <br>Supported frameworks include Caffe\*, TensorFlow\*, MXNet\*, PaddlePaddle\*, and ONNX\*. |
| [Benchmark Tool](https://docs.openvino.ai/latest/openvino_inference_engine_tools_benchmark_tool_README.html)| `benchmark_app` | **Benchmark Application** allows you to estimate deep learning inference performance on supported devices for synchronous and asynchronous modes. |
| [Model Optimizer](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) | `mo` |**Model Optimizer** imports, converts, and optimizes models that were trained in popular frameworks to a format usable by OpenVINO components. <br>Supported frameworks include Caffe\*, TensorFlow\*, MXNet\*, PaddlePaddle\*, and ONNX\*. |
| [Benchmark Tool](../../tools/benchmark_tool/README.md)| `benchmark_app` | **Benchmark Application** allows you to estimate deep learning inference performance on supported devices for synchronous and asynchronous modes. |
| [Accuracy Checker](https://docs.openvino.ai/latest/omz_tools_accuracy_checker.html) and <br> [Annotation Converter](https://docs.openvino.ai/latest/omz_tools_accuracy_checker_annotation_converters.html) | `accuracy_check` <br> `convert_annotation` |**Accuracy Checker** is a deep learning accuracy validation tool that allows you to collect accuracy metrics against popular datasets. The main advantages of the tool are the flexibility of configuration and a set of supported datasets, preprocessing, postprocessing, and metrics. <br> **Annotation Converter** is a utility that prepares datasets for evaluation with Accuracy Checker. |
| [Post-Training Optimization Tool](https://docs.openvino.ai/latest/pot_README.html)| `pot` |**Post-Training Optimization Tool** allows you to optimize trained models with advanced capabilities, such as quantization and low-precision optimizations, without the need to retrain or fine-tune models. Optimizations are also available through the [API](https://docs.openvino.ai/latest/pot_compression_api_README.html). |
| [Post-Training Optimization Tool](../../tools/pot/README.md)| `pot` |**Post-Training Optimization Tool** allows you to optimize trained models with advanced capabilities, such as quantization and low-precision optimizations, without the need to retrain or fine-tune models. Optimizations are also available through the [API](../../tools/pot/openvino/tools/pot/api/README.md). |
| [Model Downloader and other Open Model Zoo tools](https://docs.openvino.ai/latest/omz_tools_downloader.html)| `omz_downloader` <br> `omz_converter` <br> `omz_quantizer` <br> `omz_info_dumper`| **Model Downloader** is a tool for getting access to the collection of high-quality and extremely fast pre-trained deep learning [public](https://docs.openvino.ai/latest/omz_models_group_public.html) and [Intel](https://docs.openvino.ai/latest/omz_models_group_intel.html)-trained models. These free pre-trained models can be used to speed up the development and production deployment process without training your own models. The tool downloads model files from online sources and, if necessary, patches them to make them more usable with Model Optimizer. A number of additional tools are also provided to automate the process of working with downloaded models:<br> **Model Converter** is a tool for converting Open Model Zoo models that are stored in an original deep learning framework format into the OpenVINO Intermediate Representation (IR) using Model Optimizer. <br> **Model Quantizer** is a tool for automatic quantization of full-precision models in the IR format into low-precision versions using the Post-Training Optimization Tool. <br> **Model Information Dumper** is a helper utility for dumping information about the models to a stable, machine-readable format.
The developer package also installs the OpenVINO™ Runtime package as a dependency.

2
docs/install_guides/pypi-openvino-rt.md
View File

@@ -4,7 +4,7 @@
OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applications and solutions that solve a variety of tasks including emulation of human vision, automatic speech recognition, natural language processing, recommendation systems, and many others. Based on latest generations of artificial neural networks, including Convolutional Neural Networks (CNNs), recurrent and attention-based networks, the toolkit extends computer vision and non-vision workloads across Intel® hardware, maximizing performance. It accelerates applications with high-performance, AI and deep learning inference deployed from edge to cloud.
[OpenVINO™ Runtime](https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_User_Guide.html) package for Python includes a set of libraries for an easy inference integration into your Python applications and supports of heterogeneous execution across Intel® CPU and Intel® GPU hardware.
[OpenVINO™ Runtime](../OV_Runtime_UG/openvino_intro.md) package for Python includes a set of libraries for an easy inference integration into your Python applications and supports of heterogeneous execution across Intel® CPU and Intel® GPU hardware.
## System Requirements
The complete list of supported hardware is available in the [Release Notes](https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-relnotes.html).

2
docs/optimization_guide/dldt_deployment_optimization_common.md
View File

@@ -13,7 +13,7 @@ The API of the inference requests offers Sync and Async execution. While the `ov
A typical use-case for the `ov::InferRequest::infer()` is running a dedicated application thread per source of inputs (e.g. a camera), so that every step (frame capture, processing, results parsing and associated logic) is kept serial within the thread.
In contrast, the `ov::InferRequest::start_async()` and `ov::InferRequest::wait()` allow the application to continue its activities and poll or wait for the inference completion when really needed. So one reason for using asynchronous code is _efficiency_.
**NOTE**: Although the Synchronous API can be somewhat easier to start with, in the production code always prefer to use the Asynchronous (callbacks-based, below) API, as it is the most general and scalable way to implement the flow control for any possible number of requests (and hence both latency and throughput scenarios).
> **NOTE**: Although the Synchronous API can be somewhat easier to start with, in the production code always prefer to use the Asynchronous (callbacks-based, below) API, as it is the most general and scalable way to implement the flow control for any possible number of requests (and hence both latency and throughput scenarios).
Let's see how the OpenVINO Async API can improve overall throughput rate of the application. The key advantage of the Async approach is as follows: while a device is busy with the inference, the application can do other things in parallel (e.g. populating inputs or scheduling other requests) rather than wait for the inference to complete.

2
docs/optimization_guide/dldt_deployment_optimization_latency.md
View File

@@ -21,7 +21,7 @@ As expected, the lowest latency is achieved with only one concurrent inference a
However, for example, specific configurations, like multi-socket CPUs can deliver as high number of requests (at the same minimal latency) as there are NUMA nodes in the machine.
Thus, human expertise is required to get the most out of the device even in the latency case. Consider using [OpenVINO high-level performance hints](../OV_Runtime_UG/performance_hints.md) instead.
**NOTE**: [OpenVINO performance hints](./dldt_deployment_optimization_hints.md) is a recommended way for performance configuration, which is both device-agnostic and future-proof.
> **NOTE**: [OpenVINO performance hints](./dldt_deployment_optimization_hints.md) is a recommended way for performance configuration, which is both device-agnostic and future-proof.
In the case when there are multiple models to be used simultaneously, consider using different devices for inferencing the different models. Finally, when multiple models are executed in parallel on the device, using additional `ov::hint::model_priority` may help to define relative priorities of the models (please refer to the documentation on the [matrix features support for OpenVINO devices](../OV_Runtime_UG/supported_plugins/Device_Plugins.md) to check for the support of the feature by the specific device).

2
docs/optimization_guide/dldt_deployment_optimization_tput.md
View File

@@ -19,7 +19,7 @@ Overall, the latency-throughput is not linearly dependent and very _device_ spec
As for the possible inference devices the scenery had already become pretty diverse, the OpenVINO has introduced the dedicated notion of the high-level performance configuration "hints" to describe the target application scenarios.
The hints are described [here](./dldt_deployment_optimization_hints.md).
**NOTE**: [OpenVINO performance hints](./dldt_deployment_optimization_hints.md) is a recommended way for performance configuration, which is both device-agnostic and future-proof.
> **NOTE**: [OpenVINO performance hints](./dldt_deployment_optimization_hints.md) is a recommended way for performance configuration, which is both device-agnostic and future-proof.
The rest of the document provides low-level details on the OpenVINO's low-level ways to optimize the throughput.