0 Fork 0

Files

T

Karol BlaszczakandIlya Churaev 2e9454c77b Docs benchmarks page update (#13162 )

* update page and benchmark config data

benchmarks articles
update data tables
delete image

* hide / remove ovms benchmarks page

Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>

2022-09-22 17:04:12 +04:00

7.1 KiB

Raw Blame History

Intel® Distribution of OpenVINO™ toolkit Benchmark Results

@sphinxdirective .. toctree:: :maxdepth: 1 :hidden:

openvino_docs_performance_benchmarks_faq openvino_docs_performance_int8_vs_fp32 Performance Data Spreadsheet (download xlsx) https://docs.openvino.ai/2022.2/_static/benchmarks_files/OV-2022.2-Performance-Data.xlsx

@endsphinxdirective

Benchmark Setup Information

This benchmark setup includes a single machine on which both the benchmark application and the OpenVINO™ installation reside. The presented performance benchmark numbers are based on release 2022.2 of the Intel® Distribution of OpenVINO™ toolkit.

The benchmark application loads the OpenVINO™ Runtime and executes inference on the specified hardware (CPU, GPU or VPU). It measures the time spent on actual inferencing (excluding any pre or post processing) and then reports on the inferences per second (or Frames Per Second - FPS). For additional information on the benchmark application, refer to the entry 5 in the FAQ section.

Measuring inference performance involves many variables and is extremely use-case and application dependent. Below are four parameters used for measurements, which are key elements to consider for a successful deep learning inference application:

Throughput - Measures the number of inferences delivered within a latency threshold (for example, number of FPS). When deploying a system with deep learning inference, select the throughput that delivers the best trade-off between latency and power for the price and performance that meets your requirements.
Value - While throughput is important, what is more critical in edge AI deployments is the performance efficiency or performance-per-cost. Application performance in throughput per dollar of system cost is the best measure of value.
Efficiency - System power is a key consideration from the edge to the data center. When selecting deep learning solutions, power efficiency (throughput/watt) is a critical factor to consider. Intel designs provide excellent power efficiency for running deep learning workloads.
Latency - This parameter measures the synchronous execution of inference requests and is reported in milliseconds. Each inference request (i.e., preprocess, infer, postprocess) is allowed to complete before the next one is started. This performance metric is relevant in usage scenarios where a single image input needs to be acted upon as soon as possible. An example of that kind of a scenario would be real-time or near real-time applications, i.e., the response of an industrial robot to its environment or obstacle avoidance for autonomous vehicles.

For a listing of all platforms and configurations used for testing, refer to the following: @sphinxdirective

:download:HW platforms (pdf) <_static/benchmarks_files/platform_list_22.2.pdf>
:download:Configuration Details (xlsx) <_static/benchmarks_files/OV-2022.2-system-info-detailed.xlsx>

@endsphinxdirective

Benchmark Performance Results

Benchmark performance results below are based on testing as of September 20, 2022. They may not reflect all publicly available updates at the time of testing.

Performance varies by use, configuration and other factors, which are elaborated further in here. Used Intel optimizations (for Intel® compilers or other products) may not optimize to the same degree for non-Intel products.