* Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
5.2 KiB
Documentation
@sphinxdirective
.. toctree:: :maxdepth: 1 :caption: Converting and Preparing Models :hidden:
openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide omz_tools_downloader
.. toctree:: :maxdepth: 1 :caption: Deploying Inference :hidden:
openvino_docs_OV_Runtime_User_Guide openvino_docs_install_guides_deployment_manager_tool openvino_inference_engine_tools_compile_tool_README
.. toctree:: :maxdepth: 1 :caption: Tuning for Performance :hidden:
openvino_docs_optimization_guide_dldt_optimization_guide openvino_docs_MO_DG_Getting_Performance_Numbers openvino_docs_model_optimization_guide openvino_docs_deployment_optimization_guide_dldt_optimization_guide openvino_docs_tuning_utilities openvino_docs_performance_benchmarks
.. toctree::
:maxdepth: 1
:caption: Graphical Web Interface for OpenVINO™ toolkit
:hidden:
workbench_docs_Workbench_DG_Introduction workbench_docs_Workbench_DG_Install workbench_docs_Workbench_DG_Work_with_Models_and_Sample_Datasets workbench_docs_Workbench_DG_User_Guide workbench_docs_security_Workbench workbench_docs_Workbench_DG_Troubleshooting
.. toctree:: :maxdepth: 1 :hidden: :caption: Media Processing and Computer Vision Libraries
Intel® Deep Learning Streamer <openvino_docs_dlstreamer> openvino_docs_gapi_gapi_intro OpenCV* Developer Guide https://docs.opencv.org/master/ OpenCL™ Developer Guide https://software.intel.com/en-us/openclsdk-devguide
.. toctree:: :maxdepth: 1 :caption: Add-Ons :hidden:
ovms_what_is_openvino_model_server ote_documentation ovsa_get_started
.. toctree:: :maxdepth: 1 :caption: OpenVINO Extensibility :hidden:
openvino_docs_Extensibility_UG_Intro openvino_docs_transformations Inference Engine Plugin Developer Guide <openvino_docs_ie_plugin_dg_overview> groupie_dev_api Plugin Transformation Pipeline <openvino_docs_IE_DG_plugin_transformation_pipeline>
.. toctree:: :maxdepth: 1 :hidden: :caption: Use OpenVINO™ Toolkit Securely
openvino_docs_security_guide_introduction openvino_docs_security_guide_workbench openvino_docs_IE_DG_protecting_model_guide ovsa_get_started
@endsphinxdirective
This section provides reference documents that guide you through developing your own deep learning applications with the OpenVINO™ toolkit. These documents will most helpful if you have first gone through the Get Started guide.
Converting and Preparing Models
With the [Model Downloader](@ref omz_tools_downloader) and Model Optimizer guides, you will learn to download pre-trained models and convert them for use with the OpenVINO™ toolkit. You can provide your own model or choose a public or Intel model from a broad selection provided in the Open Model Zoo.
Deploying Inference
The OpenVINO™ Runtime User Guide explains the process of creating your own application that runs inference with the OpenVINO™ toolkit. The API Reference defines the OpenVINO Runtime API for Python, C++, and C. The OpenVINO Runtime API is what you'll use to create an OpenVINO™ inference application, use enhanced operations sets and other features. After writing your application, you can use the Deployment Manager for deploying to target devices.
Tuning for Performance
The toolkit provides a Performance Optimization Guide and utilities for squeezing the best performance out of your application, including [Accuracy Checker](@ref omz_tools_accuracy_checker), [Post-Training Optimization Tool](@ref pot_README), and other tools for measuring accuracy, benchmarking performance, and tuning your application.
Graphical Web Interface for OpenVINO™ Toolkit
You can choose to use the [OpenVINO™ Deep Learning Workbench](@ref workbench_docs_Workbench_DG_Introduction), a web-based tool that guides you through the process of converting, measuring, optimizing, and deploying models. This tool also serves as a low-effort introduction to the toolkit and provides a variety of useful interactive charts for understanding performance.
Media Processing and Computer Vision Libraries
The OpenVINO™ toolkit also works with the following media processing frameworks and libraries:
- [Intel® Deep Learning Streamer (Intel® DL Streamer)](@ref openvino_docs_dlstreamer) — A streaming media analytics framework based on GStreamer, for creating complex media analytics pipelines optimized for Intel hardware platforms. Go to the Intel® DL Streamer documentation website to learn more.
- Intel® oneAPI Video Processing Library (oneVPL) — A programming interface for video decoding, encoding, and processing to build portable media pipelines on CPUs, GPUs, and other accelerators.
You can also add computer vision capabilities to your application using optimized versions of OpenCV.