Files
openvino/docs/optimization_guide/dldt_deployment_optimization_latency.md

48 lines
4.4 KiB
Markdown
Raw Normal View History

## Optimizing for Latency {#openvino_docs_deployment_optimization_guide_latency}
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
@sphinxdirective
.. toctree::
:maxdepth: 1
:hidden:
DOCS: ported changes from 2022.1 release branch (#11206) * Extensibility guide with FE extensions and remove OV_FRAMEWORK_MAP from docs * Rework of Extensibility Intro, adopted examples to missing OPENVINO_FRAMEWORK_MAP * Removed OPENVINO_FRAMEWORK_MAP reference * Frontend extension detailed documentation * Fixed distributed snippets * Fixed snippet inclusion in FE extension document and chapter headers * Fixed wrong name in a snippet reference * Fixed test for template extension due to changed number of loaded extensions * Update docs/Extensibility_UG/frontend_extensions.md Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * Minor fixes in extension snippets * Small grammar fix Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * DOCS: transition banner (#10973) * transition banner * minor fix * update transition banner * updates * update custom.js * updates * updates * Documentation fixes (#11044) * Benchmark app usage * Fixed link to the devices * More fixes * Update docs/OV_Runtime_UG/multi_device.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Removed several hardcoded links Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Updated documentation for compile_tool (#11049) * Added deployment guide (#11060) * Added deployment guide * Added local distribution * Updates * Fixed more indentations * Removed obsolete code snippets (#11061) * Removed obsolete code snippets * NCC style * Fixed NCC for BA * Add a troubleshooting issue for PRC installation (#11074) * updates * adding gna to linux * add missing reference * update * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * update * minor updates * add gna item to yum and apt * add gna to get started page * update reference formatting * merge commit * add a troubleshooting issue * update * update * fix CVS-71846 Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * DOCS: fixed hardcoded links (#11100) * Fixes * Use links * applying reviewers comments to the Opt Guide (#11093) * applying reviewrs comments * fixed refs, more structuring (bold, bullets, etc) * refactoring tput/latency sections * next iteration (mostly latency), also brushed the auto-batching and other sections * updates sync/async images * common opts brushed * WIP tput redesigned * minor brushing of common and auto-batching * Tput fully refactored * fixed doc name in the link * moved int8 perf counters to the right section * fixed links * fixed broken quotes * fixed more links * add ref to the internals to the TOC * Added a note on the batch size Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> * [80085] New images for docs (#11114) * change doc structure * fix manager tools * fix manager tools 3 step * fix manager tools 3 step * new img * new img for OV Runtime * fix steps * steps * fix intendents * change list * fix space * fix space * code snippets fix * change display * Benchmarks 2022 1 (#11130) * Minor fixes * Updates for 2022.1 * Edits according to the review * Edits according to review comments * Edits according to review comments * Edits according to review comments * Fixed table * Edits according to review comments * Removed config for Intel® Core™ i7-11850HE * Removed forward-tacotron-duration-prediction-241 graph * Added resnet-18-pytorch * Add info about Docker images in Deployment guide (#11136) * Renamed user guides (#11137) * fix screenshot (#11140) * More conservative recommendations on dynamic shapes usage in docs (#11161) * More conservative recommendations about using dynamic shapes * Duplicated statement from C++ part to Python part of reshape doc (no semantical changes) * Update ShapeInference.md (#11168) * Benchmarks 2022 1 updates (#11180) * Updated graphs * Quick fix for TODO in Dynamic Shapes article * Anchor link fixes * Fixed DM config (#11199) * DOCS: doxy sphinxtabs (#11027) * initial implementation of doxy sphinxtabs * fixes * fixes * fixes * fixes * fixes * WA for ignored visibility attribute * Fixes Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com> Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Ilya Naumov <ilya.naumov@intel.com> Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
2022-03-24 22:27:29 +03:00
openvino_docs_OV_UG_Model_caching_overview
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
A significant portion of deep learning use cases involve applications loading a single model and using a single input at a time, which is the of typical "consumer" scenario.
While an application can create more than one request if needed, for example to support :ref:`asynchronous inputs population <async_api>`, its **inference performance depends on how many requests are being inferenced in parallel** on a device.
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
Similarly, when multiple models are served on the same device, it is important whether the models are executed simultaneously or in a chain, for example, in the inference pipeline.
As expected, the easiest way to achieve **low latency is by running only one inference at a time** on one device. Accordingly, any additional concurrency usually results in latency rising fast.
DOCS: ported changes from 2022.1 release branch (#11206) * Extensibility guide with FE extensions and remove OV_FRAMEWORK_MAP from docs * Rework of Extensibility Intro, adopted examples to missing OPENVINO_FRAMEWORK_MAP * Removed OPENVINO_FRAMEWORK_MAP reference * Frontend extension detailed documentation * Fixed distributed snippets * Fixed snippet inclusion in FE extension document and chapter headers * Fixed wrong name in a snippet reference * Fixed test for template extension due to changed number of loaded extensions * Update docs/Extensibility_UG/frontend_extensions.md Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * Minor fixes in extension snippets * Small grammar fix Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> * DOCS: transition banner (#10973) * transition banner * minor fix * update transition banner * updates * update custom.js * updates * updates * Documentation fixes (#11044) * Benchmark app usage * Fixed link to the devices * More fixes * Update docs/OV_Runtime_UG/multi_device.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Removed several hardcoded links Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Updated documentation for compile_tool (#11049) * Added deployment guide (#11060) * Added deployment guide * Added local distribution * Updates * Fixed more indentations * Removed obsolete code snippets (#11061) * Removed obsolete code snippets * NCC style * Fixed NCC for BA * Add a troubleshooting issue for PRC installation (#11074) * updates * adding gna to linux * add missing reference * update * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * Update docs/install_guides/installing-model-dev-tools.md Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * update * minor updates * add gna item to yum and apt * add gna to get started page * update reference formatting * merge commit * add a troubleshooting issue * update * update * fix CVS-71846 Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> * DOCS: fixed hardcoded links (#11100) * Fixes * Use links * applying reviewers comments to the Opt Guide (#11093) * applying reviewrs comments * fixed refs, more structuring (bold, bullets, etc) * refactoring tput/latency sections * next iteration (mostly latency), also brushed the auto-batching and other sections * updates sync/async images * common opts brushed * WIP tput redesigned * minor brushing of common and auto-batching * Tput fully refactored * fixed doc name in the link * moved int8 perf counters to the right section * fixed links * fixed broken quotes * fixed more links * add ref to the internals to the TOC * Added a note on the batch size Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> * [80085] New images for docs (#11114) * change doc structure * fix manager tools * fix manager tools 3 step * fix manager tools 3 step * new img * new img for OV Runtime * fix steps * steps * fix intendents * change list * fix space * fix space * code snippets fix * change display * Benchmarks 2022 1 (#11130) * Minor fixes * Updates for 2022.1 * Edits according to the review * Edits according to review comments * Edits according to review comments * Edits according to review comments * Fixed table * Edits according to review comments * Removed config for Intel® Core™ i7-11850HE * Removed forward-tacotron-duration-prediction-241 graph * Added resnet-18-pytorch * Add info about Docker images in Deployment guide (#11136) * Renamed user guides (#11137) * fix screenshot (#11140) * More conservative recommendations on dynamic shapes usage in docs (#11161) * More conservative recommendations about using dynamic shapes * Duplicated statement from C++ part to Python part of reshape doc (no semantical changes) * Update ShapeInference.md (#11168) * Benchmarks 2022 1 updates (#11180) * Updated graphs * Quick fix for TODO in Dynamic Shapes article * Anchor link fixes * Fixed DM config (#11199) * DOCS: doxy sphinxtabs (#11027) * initial implementation of doxy sphinxtabs * fixes * fixes * fixes * fixes * fixes * WA for ignored visibility attribute * Fixes Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> Co-authored-by: Ivan Tikhonov <ivan.tikhonov@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com> Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Ilya Naumov <ilya.naumov@intel.com> Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
2022-03-24 22:27:29 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
However, some conventional "root" devices (i.e., CPU or GPU) can be in fact internally composed of several "sub-devices". In many cases, letting OpenVINO leverage the "sub-devices" transparently helps to improve application's throughput (e.g., serve multiple clients simultaneously) without degrading latency. For example, multi-socket CPUs can deliver as many requests at the same minimal latency as there are NUMA nodes in the system. Similarly, a multi-tile GPU, which is essentially multiple GPUs in a single package, can deliver a multi-tile scalability with the number of inference requests, while preserving the single-tile latency.
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Typically, human expertise is required to get more "throughput" out of the device, even in the inherently latency-oriented cases. OpenVINO can take this configuration burden via :doc:`high-level performance hints <openvino_docs_OV_UG_Performance_Hints>`, the `ov::hint::PerformanceMode::LATENCY <enumov_1_1hint_1_1PerformanceMode.html#doxid-group-ov-runtime-cpp-prop-api-1gga032aa530efa40760b79af14913d48d73a501069dd75f76384ba18f133fdce99c2>`__ specified for the ``ov::hint::performance_mode`` property for the ``compile_model``.
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
.. note::
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
:doc:`OpenVINO performance hints <openvino_docs_OV_UG_Performance_Hints>` is a recommended way for performance configuration, which is both device-agnostic and future-proof.
* feature support by device
When multiple models are to be used simultaneously, consider running inference on separate devices for each of them. Finally, when multiple models are executed in parallel on a device, using additional ``ov::hint::model_priority`` may help to define relative priorities of the models. Refer to the documentation on the :ref:`OpenVINO feature support for devices <devicesupport-feature-support-matrix>` to check if your device supports the feature.
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
**First-Inference Latency and Model Load/Compile Time**
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
In some cases, model loading and compilation contribute to the "end-to-end" latency more than usual.
For example, when the model is used exactly once, or when it is unloaded and reloaded in a cycle, to free the memory for another inference due to on-device memory limitations.
DOCS: port changes from releases/2022/1 (#11040) * Added migration for deployment (#10800) * Added migration for deployment * Addressed comments * more info after the What's new Sessions' questions (#10803) * more info after the What's new Sessions' questions * generalizing the optimal_batch_size vs explicit value message * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Perf Hints docs and General Opt Guide refactoring (#10815) * Brushed the general optimization page * Opt GUIDE, WIP * perf hints doc placeholder * WIP * WIP2 * WIP 3 * added streams and few other details * fixed titles, misprints etc * Perf hints * movin the runtime optimizations intro * fixed link * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * some details on the FIL and other means when pure inference time is not the only factor * shuffled according to general->use-case->device-specifics flow, minor brushing * next iter * section on optimizing for tput and latency * couple of links to the features support matrix * Links, brushing, dedicated subsections for Latency/FIL/Tput * had to make the link less specific (otherwise docs compilations fails) * removing the Temp/Should be moved to the Opt Guide * shuffled the tput/latency/etc info into separated documents. also the following docs moved from the temp into specific feature, general product desc or corresponding plugins - openvino_docs_IE_DG_Model_caching_overview - openvino_docs_IE_DG_Int8Inference - openvino_docs_IE_DG_Bfloat16Inference - openvino_docs_OV_UG_NoDynamicShapes * fixed toc for ov_dynamic_shapes.md * referring the openvino_docs_IE_DG_Bfloat16Inference to avoid docs compilation errors * fixed main product TOC, removed ref from the second-level items * reviewers remarks * reverted the openvino_docs_OV_UG_NoDynamicShapes * reverting openvino_docs_IE_DG_Bfloat16Inference and openvino_docs_IE_DG_Int8Inference * "No dynamic shapes" to the "Dynamic shapes" as TOC * removed duplication * minor brushing * Caching to the next level in TOC * brushing * more on the perf counters ( for latency and dynamic cases) Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Updated common IE pipeline infer-request section (#10844) * Updated common IE pipeline infer-reqest section * Update ov_infer_request.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * DOCS: Removed useless 4 spaces in snippets (#10870) * Updated snippets * Added link to encryption * [DOCS] ARM CPU plugin docs (#10885) * initial commit ARM_CPU.md added ARM CPU is added to the list of supported devices * Update the list of supported properties * Update Device_Plugins.md * Update CODEOWNERS * Removed quotes in limitations section * NVIDIA and Android are added to the list of supported devices * Added See Also section and reg sign to arm * Added Preprocessing acceleration section * Update the list of supported layers * updated list of supported layers * fix typos * Added support disclaimer * update trade and reg symbols * fixed typos * fix typos * reg fix * add reg symbol back Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> * Try to fix visualization (#10896) * Try to fix visualization * New try * Update Install&Deployment for migration guide to 22/1 (#10933) * updates * update * Getting started improvements (#10948) * Onnx updates (#10962) * onnx changes * onnx updates * onnx updates * fix broken anchors api reference (#10976) * add ote repo (#10979) * DOCS: Increase content width (#10995) * fixes * fix * Fixed compilation Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com> Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Aleksandr Voron <aleksandr.voron@intel.com> Co-authored-by: Vitaly Tuzov <vitaly.tuzov@intel.com> Co-authored-by: Ilya Churaev <ilya.churaev@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Victoria Yashina <victoria.yashina@intel.com> Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
2022-03-18 17:48:45 +03:00
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
Such a "first-inference latency" scenario may pose an additional limitation on the model load\compilation time, as inference accelerators (other than the CPU) usually require a certain level of model compilation upon loading.
The :doc:`model caching <openvino_docs_OV_UG_Model_caching_overview>` option is a way to lessen the impact over multiple application runs. If model caching is not possible, for example, it may require write permissions for the application, the CPU offers the fastest model load time almost every time.
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
To improve common "first-inference latency" scenario, model reading was replaced with model mapping (using `mmap`) into a memory. But in some use cases (first of all, if model is located on removable or network drive) mapping may lead to latency increase. To switch mapping to reading, specify ``ov::enable_mmap(false)`` property for the ``ov::Core``.
Another way of dealing with first-inference latency is using the :doc:`AUTO device selection inference mode <openvino_docs_OV_UG_supported_plugins_AUTO>`. It starts inference on the CPU, while waiting for the actual accelerator to load the model. At that point, it shifts to the new device seamlessly.
Proofreading-OV-Runtime (#11658) * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/protecting_model_guide.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/optimization_guide/dldt_deployment_optimization_common.md Co-authored-by: Sebastian Golebiewski <sebastianx.golebiewski@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/Device_Plugins.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GPU_RemoteTensor_API.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/HDDL.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/MYRIAD.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/ov_dynamic_shapes.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/preprocessing_details.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/performance_hints.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Apply suggestions from code review * Update ref links * Update Getting_performance_numbers.md * Update deployment_intro.md * Update preprocessing_details.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/automatic_batching.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/default/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update automatic_batching.md * Update docs/OV_Runtime_UG/automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md * Update deployment-manager-tool.md * Update deployment-manager-tool.md * Update docs/OV_Runtime_UG/deployment/deployment-manager-tool.md * Update automatic_batching.md * Update automatic_batching.md * Update docs/OV_Runtime_UG/ShapeInference.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/integrate_with_your_application.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update integrate_with_your_application.md * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md Removing redundant information. * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Apply suggestions from code review Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update Additional_Optimizations.md * Update docs/OV_Runtime_UG/model_representation.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/OV_Runtime_UG/layout_overview.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update model_representation.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/openvino/tools/pot/algorithms/quantization/accuracy_aware/README.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/GNA.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update docs/OV_Runtime_UG/supported_plugins/CPU.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/SaturationIssue.md * Update tools/pot/docs/SaturationIssue.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update README.md * Update README.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Removing one-liners Removing introductory sentences from 'Supported Features' sections. * Update docs/OV_Runtime_UG/openvino_intro.md Co-authored-by: Yuan Xu <yuan1.xu@intel.com> * Update docs/benchmarks/performance_benchmarks_ovms.md Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/Introduction.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> * Update tools/pot/docs/DefaultQuantizationUsage.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/BestPractices.md * Update tools/pot/docs/AccuracyAwareQuantizationUsage.md * Update docs/optimization_guide/model_optimization_guide.md * Update docs/optimization_guide/dldt_deployment_optimization_guide.md * Update docs/OV_Runtime_UG/supported_plugins/config_properties.md * Update docs/OV_Runtime_UG/supported_plugins/GNA.md * Update docs/OV_Runtime_UG/supported_plugins/CPU.md * Update docs/OV_Runtime_UG/preprocessing_usecase_save.md * Apply suggestions from code review Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: Maciej Smyk <maciejx.smyk@intel.com> Co-authored-by: Yuan Xu <yuan1.xu@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com> Co-authored-by: msmykx <101244365+msmykx-intel@users.noreply.github.com> Co-authored-by: Piotr Milewski <piotr.milewski@intel.com>
2022-07-08 13:34:45 +02:00
Finally, note that any :doc:`throughput-oriented options <openvino_docs_deployment_optimization_guide_tput>` may significantly increase the model uptime.
@endsphinxdirective