0 Fork 0

Files

T

Maxim Shevtsov 3bec32449f OV Performance Hints (CPU and GPU logic for selecting the actual configs), while AUTO/MULTI are passing them thru) (#6993 )

* rebasing the perf-modes-2021.3 to the 2021.4

Caveats:
the (explicit) setting #streams is not disabled (as it was before for experiments with DLBenchmark), and the logic slighlty differ (streamsSet)

(cherry picked from commit 1ae1edc0ed)

* overriding streams (to force the TPUT mode to the DLBenchnark)

(cherry picked from commit 7f506cda31)

* disabling reducing #streams to fully mimic baseline c4df94d42d of the 2021.3 (before experiments)

(cherry picked from commit 85073dd1dd)

* clang/identation

(cherry picked from commit 050a4155a9)

* splitting the Transformation to general and CPU specific.

Now hopefully,this fully mimics the  baseline c4df94d42d of the 2021.3 (before experiments), as the streams reduce num (as well as early exit on GRU/LSTM/TensorIterator) is deisabled

(cherry picked from commit e98b2c1a67)

* disabling GRU/LSTM/TI + reducing of streams + 5D considered compute-limited only for int8

(cherry picked from commit 32b8d80dee)

* refactored to avoid compute_limited_ratio, reverted the reducing #streams, removed LSTM from limitations

(cherry picked from commit f2b972171b)

* isa-based threshold logic

(cherry picked from commit b218457e1a)

* mode->hint

(cherry picked from commit ec20aa8eca)

* optional PERFORMANCE_HINT_NUM_REQUESTS

(cherry picked from commit 5a3883e3f3)

* moving the perfHints to the common OV config class + initial tests (CPU only, as the actual AUTO/MULTI should be accommodated on the master)

(cherry picked from commit (then fixed)45bafe7d527f466507dea0693aeed51be4ebf776)

* AUTO support for PerfHints

* MULTI support for PerfHints

* Enabling Perf hints for the GPU plugin

* brushing settings output a bit

* disabling "throughput" perf hint being default (until OV 2.0)

* uncommenting the logic which was disabled to force the DLBenchmark to use the throughput mode by default

* removing dead and experimental code, and debug printfs

* clang/code-style

* code-review remarks

* Moved the output of the actual params that the hint produced to the right place

* aligning MULTI's GetConfig beh to HETERO's as captured in the preso (CVS-59960) ratified with the ArchForum

* clang

* benchmark_app brushing

* Update inference-engine/samples/benchmark_app/README.md

* propagating the perf hints thru one more scenario in the merged AUTO-MULTI

* fixed mispint

* Python benchmark_app update for perf hints

* addresssing reviewers comments on the python benchmark_app

* simplifying/brushing logic a bit

* refactor the heuristic to the separate file (to be shared with iGPU soon)

* refactor conversion of modes to the specific GPU config per feedback from Vladimir

2021-09-13 15:40:36 +03:00

benchmark_tool

OV Performance Hints (CPU and GPU logic for selecting the actual configs), while AUTO/MULTI are passing them thru) (#6993 )

2021-09-13 15:40:36 +03:00

compile_tool

Removed information about FPGA plugin (#7474 )

2021-09-13 14:01:49 +03:00

cross_check_tool

Merge tools and inference_engine/tools folders (#7359 )

2021-09-07 12:43:33 +03:00

deployment_manager

Merge tools and inference_engine/tools folders (#7359 )

2021-09-07 12:43:33 +03:00

.gitignore

Align folder structure for Benchmark Tool (#6625 )

2021-07-19 11:57:51 +03:00

CMakeLists.txt

Merge tools and inference_engine/tools folders (#7359 )

2021-09-07 12:43:33 +03:00

README.md

Align folder structure for Benchmark Tool (#6625 )

2021-07-19 11:57:51 +03:00

README.md

OpenVINO™ Python* development tools

General

OpenVINO includes following tools:

openvino.tools.benchmark

Please, refer to https://docs.openvinotoolkit.org for details.

Installation

Prerequisites

Install prerequisites first:

1. Python

Install Python prerequisites:

Run following command to install these prerequisites on Ubuntu*:

sudo apt-get install python3 python3-dev python3-setuptools python3-pip

Python setuptools and python package manager (pip) install packages into system directory by default. There are several options:

work inside [virtual environment][virtualenv] (best solution).
use --user option for all pip commands.
install all dependencies with sudo permissions.

In order to use virtual environment you should install it:

python3 -m pip install virtualenv
python3 -m virtualenv -p `which python3` <directory_for_environment>

Before starting to work inside virtual environment, it should be activated:

source <directory_for_environment>/bin/activate

Virtual environment can be deactivated using command

deactivate

2. Install packages

You can install tools by specifying path to tool with setup.py in pip install command:

python3 -m pip install <tools_folder>/

For example, to install Benchmark Tool, use the following command:

python3 -m pip install benchmark_tool/

Configuration

Each subpackage has specific configuration. Please, refer to specific subpackage documentation for details.