IntenseWebs/openvino
3
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Extend throughput benchmark with device CLI parameter (#20816)

Browse Source 
* Extend throughput benchmark CLI parameters

* Added device name as the second CLI parameter with default CPU value

* Update samples/cpp/benchmark/throughput_benchmark/main.cpp

Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>

* Fix comments to review

* Modified python version
* Modified documentation

* Fix comments to review

* Fixed the comment
* Modified python doc
* Fixed device name handling in python version

* Update main.cpp

* Update throughput_benchmark.py

---------

Co-authored-by: Zlobin Vladimir <vladimir.zlobin@intel.com>
This commit is contained in:
Anatoliy TalamanovandZlobin Vladimir authored and GitHub committed 2023-11-03 09:57:04 +01:00
1 parent c20d52dc4f
commit f890bf7930
4 files changed
+17 -10
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

samples/cpp/benchmark/throughput_benchmark/main.cpp
+8 -4
View File
@@ -22,8 +22,12 @@ int main(int argc, char* argv[]) {
try {
slog::info << "OpenVINO:" << slog::endl;
slog::info << ov::get_openvino_version();
if (argc != 2) {
slog::info << "Usage : " << argv[0] << " <path_to_model>" << slog::endl;
std::string device_name = "CPU";
if (argc == 3) {
device_name = argv[2];
} else if (argc != 2) {
slog::info << "Usage : " << argv[0] << " <path_to_model> <device_name>(default: CPU)" << slog::endl;
return EXIT_FAILURE;
}
// Optimize for throughput. Best throughput can be reached by
@@ -31,10 +35,10 @@ int main(int argc, char* argv[]) {
ov::AnyMap tput{{ov::hint::performance_mode.name(), ov::hint::PerformanceMode::THROUGHPUT}};
// Create ov::Core and use it to compile a model.
// Pick a device by replacing CPU, for example MULTI:CPU(4),GPU(8).
// Select the device by providing the name as the second parameter to CLI.
// It is possible to set CUMULATIVE_THROUGHPUT as ov::hint::PerformanceMode for AUTO device
ov::Core core;
ov::CompiledModel compiled_model = core.compile_model(argv[1], "CPU", tput);
ov::CompiledModel compiled_model = core.compile_model(argv[1], device_name, tput);
// Create optimal number of ov::InferRequest instances
uint32_t nireq = compiled_model.get_property(ov::optimal_number_of_infer_requests);
std::vector<ov::InferRequest> ireqs(nireq);