Maxim Shevtsov
3bec32449f
* rebasing the perf-modes-2021.3 to the 2021.4 Caveats: the (explicit) setting #streams is not disabled (as it was before for experiments with DLBenchmark), and the logic slighlty differ (streamsSet) (cherry picked from commit1ae1edc0ed) * overriding streams (to force the TPUT mode to the DLBenchnark) (cherry picked from commit7f506cda31) * disabling reducing #streams to fully mimic baselinec4df94d42dof the 2021.3 (before experiments) (cherry picked from commit85073dd1dd) * clang/identation (cherry picked from commit050a4155a9) * splitting the Transformation to general and CPU specific. Now hopefully,this fully mimics the baselinec4df94d42dof the 2021.3 (before experiments), as the streams reduce num (as well as early exit on GRU/LSTM/TensorIterator) is deisabled (cherry picked from commite98b2c1a67) * disabling GRU/LSTM/TI + reducing of streams + 5D considered compute-limited only for int8 (cherry picked from commit32b8d80dee) * refactored to avoid compute_limited_ratio, reverted the reducing #streams, removed LSTM from limitations (cherry picked from commitf2b972171b) * isa-based threshold logic (cherry picked from commitb218457e1a) * mode->hint (cherry picked from commitec20aa8eca) * optional PERFORMANCE_HINT_NUM_REQUESTS (cherry picked from commit5a3883e3f3) * moving the perfHints to the common OV config class + initial tests (CPU only, as the actual AUTO/MULTI should be accommodated on the master) (cherry picked from commit (then fixed)45bafe7d527f466507dea0693aeed51be4ebf776) * AUTO support for PerfHints * MULTI support for PerfHints * Enabling Perf hints for the GPU plugin * brushing settings output a bit * disabling "throughput" perf hint being default (until OV 2.0) * uncommenting the logic which was disabled to force the DLBenchmark to use the throughput mode by default * removing dead and experimental code, and debug printfs * clang/code-style * code-review remarks * Moved the output of the actual params that the hint produced to the right place * aligning MULTI's GetConfig beh to HETERO's as captured in the preso (CVS-59960) ratified with the ArchForum * clang * benchmark_app brushing * Update inference-engine/samples/benchmark_app/README.md * propagating the perf hints thru one more scenario in the merged AUTO-MULTI * fixed mispint * Python benchmark_app update for perf hints * addresssing reviewers comments on the python benchmark_app * simplifying/brushing logic a bit * refactor the heuristic to the separate file (to be shared with iGPU soon) * refactor conversion of modes to the specific GPU config per feedback from Vladimir
51 lines
1.2 KiB
C++
51 lines
1.2 KiB
C++
// Copyright (C) 2018-2021 Intel Corporation
|
|
// SPDX-License-Identifier: Apache-2.0
|
|
//
|
|
|
|
#pragma once
|
|
|
|
#include <threading/ie_istreams_executor.hpp>
|
|
#include <ie_performance_hints.hpp>
|
|
#include "utils/debug_capabilities.h"
|
|
|
|
#include <string>
|
|
#include <map>
|
|
|
|
namespace MKLDNNPlugin {
|
|
|
|
struct Config {
|
|
Config();
|
|
|
|
enum LPTransformsMode {
|
|
Off,
|
|
On,
|
|
};
|
|
|
|
bool collectPerfCounters = false;
|
|
bool exclusiveAsyncRequests = false;
|
|
bool enableDynamicBatch = false;
|
|
std::string dumpToDot = "";
|
|
int batchLimit = 0;
|
|
InferenceEngine::IStreamsExecutor::Config streamExecutorConfig;
|
|
InferenceEngine::PerfHintsConfig perfHintsConfig;
|
|
#if defined(__arm__) || defined(__aarch64__)
|
|
// Currently INT8 mode is not optimized on ARM, fallback to FP32 mode.
|
|
LPTransformsMode lpTransformsMode = LPTransformsMode::Off;
|
|
bool enforceBF16 = false;
|
|
#else
|
|
LPTransformsMode lpTransformsMode = LPTransformsMode::On;
|
|
bool enforceBF16 = true;
|
|
bool manualEnforceBF16 = false;
|
|
#endif
|
|
|
|
#ifdef CPU_DEBUG_CAPS
|
|
DebugCaps::Config debugCaps;
|
|
#endif
|
|
|
|
void readProperties(const std::map<std::string, std::string> &config);
|
|
void updateProperties();
|
|
std::map<std::string, std::string> _config;
|
|
};
|
|
|
|
} // namespace MKLDNNPlugin
|