[DOCS] Adding metadata to articles (#18331)
* adding-metadata * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> * Apply suggestions from code review Co-authored-by: Tatiana Savina <tatiana.savina@intel.com> --------- Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
This commit is contained in:
1 parent
cb8d34ddc1
commit
152c9b63e2
428 files changed
+2077
-158
No files matched your search
@@ -2,6 +2,12 @@
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. meta::
|
||||
:description: General optimizations include application-level optimization
|
||||
methods that improve data pipelining, pre-processing
|
||||
acceleration and both latency and throughput.
|
||||
|
||||
|
||||
This article covers application-level optimization techniques, such as asynchronous execution, to improve data pipelining, pre-processing acceleration and so on.
|
||||
While the techniques (e.g. pre-processing) can be specific to end-user applications, the associated performance improvements are general and shall improve any target scenario -- both latency and throughput.
|
||||
|
||||
|
||||
@@ -16,6 +16,11 @@
|
||||
openvino_docs_deployment_optimization_guide_internals
|
||||
openvino_docs_memory_optimization_guide
|
||||
|
||||
.. meta::
|
||||
:description: Improving inference performance involves model and runtime
|
||||
optimizations that can be done independently. Inference
|
||||
speed depends on latency and throughput.
|
||||
|
||||
|
||||
Runtime optimization, or deployment optimization, focuses on tuning inference parameters and execution means (e.g., the optimum number of requests executed simultaneously). Unlike model-level optimizations, they are highly specific to the hardware and case they are used for, and often come at a cost.
|
||||
`ov::hint::inference_precision <groupov_runtime_cpp_prop_api.html#doxid-group-ov-runtime-cpp-prop-api-1gad605a888f3c9b7598ab55023fbf44240>`__ is a "typical runtime configuration" which trades accuracy for performance, allowing ``fp16/bf16`` execution for the layers that remain in ``fp32`` after quantization of the original ``fp32`` model.
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. meta::
|
||||
:description: Automatic Batching moves asynchronicity from individual
|
||||
requests to groups of requests, and the CPU streams are
|
||||
inference threads grouped by CPU cores.
|
||||
|
||||
|
||||
Throughput on the CPU: Internals
|
||||
################################
|
||||
|
||||
|
||||
@@ -8,6 +8,10 @@
|
||||
|
||||
openvino_docs_OV_UG_Model_caching_overview
|
||||
|
||||
.. meta::
|
||||
:description: OpenVINO provides methods that help to preserve minimal
|
||||
latency despite the number of inference requests and
|
||||
improve throughput without degrading latency.
|
||||
|
||||
|
||||
A significant portion of deep learning use cases involve applications loading a single model and using a single input at a time, which is the of typical "consumer" scenario.
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. meta::
|
||||
:description: Throughput-oriented approaches in OpenVINO involve
|
||||
execution of a large number of inference requests
|
||||
simultaneously which improves the device utilization.
|
||||
|
||||
|
||||
As described in the section on the :doc:`latency-specific considerations <openvino_docs_deployment_optimization_guide_latency>`, one of the possible use cases is *delivering every single request at the minimal delay*.
|
||||
Throughput, on the other hand, is about inference scenarios in which potentially **large number of inference requests are served simultaneously to improve the device utilization**.
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
.. meta::
|
||||
:description: With OpenVINO streams a device may handle processing multiple
|
||||
inference requests and the batching helps to saturate the
|
||||
device and leads to higher throughput.
|
||||
|
||||
|
||||
OpenVINO Streams
|
||||
####################
|
||||
|
||||
|
||||
Reference in new issue
Block a user