2bf8d910f6
* [Docs][PyOV] update python snippets * first snippet * Fix samples debug * Fix linter * part1 * Fix speech sample * update model state snippet * add serialize * add temp dir * CPU snippets update (#134) * snippets CPU 1/6 * snippets CPU 2/6 * snippets CPU 3/6 * snippets CPU 4/6 * snippets CPU 5/6 * snippets CPU 6/6 * make module TODO: REMEMBER ABOUT EXPORTING PYTONPATH ON CIs ETC * Add static model creation in snippets for CPU * export_comp_model done * leftovers * apply comments * apply comments -- properties * small fixes * rempve debug info * return IENetwork instead of Function * apply comments * revert precision change in common snippets * update opset * [PyOV] Edit docs for the rest of plugins (#136) * modify main.py * GNA snippets * GPU snippets * AUTO snippets * MULTI snippets * HETERO snippets * Added properties * update gna * more samples * Update docs/OV_Runtime_UG/model_state_intro.md * Update docs/OV_Runtime_UG/model_state_intro.md * attempt1 fix ci * new approach to test * temporary remove some files from run * revert cmake changes * fix ci * fix snippet * fix py_exclusive snippet * fix preprocessing snippet * clean-up main * remove numpy installation in gha * check for GPU * add logger * iexclude main * main update * temp * Temp2 * Temp2 * temp * Revert temp * add property execution devices * hide output from samples --------- Co-authored-by: p-wysocki <przemyslaw.wysocki@intel.com> Co-authored-by: Jan Iwaszkiewicz <jan.iwaszkiewicz@intel.com> Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
56 lines
2.1 KiB
Python
56 lines
2.1 KiB
Python
# Copyright (C) 2018-2023 Intel Corporation
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
#
|
|
|
|
import openvino as ov
|
|
from utils import get_model
|
|
|
|
|
|
def main():
|
|
|
|
core = ov.Core()
|
|
model = get_model()
|
|
|
|
if "GPU" not in core.available_devices:
|
|
return 0
|
|
|
|
# [compile_model]
|
|
import openvino.runtime.properties as props
|
|
import openvino.runtime.properties.hint as hints
|
|
|
|
config = {hints.performance_mode(): hints.PerformanceMode.THROUGHPUT}
|
|
compiled_model = core.compile_model(model, "GPU", config)
|
|
# [compile_model]
|
|
|
|
# [compile_model_no_auto_batching]
|
|
# disabling the automatic batching
|
|
# leaving intact other configurations options that the device selects for the 'throughput' hint
|
|
config = {hints.performance_mode(): hints.PerformanceMode.THROUGHPUT,
|
|
hints.allow_auto_batching(): False}
|
|
compiled_model = core.compile_model(model, "GPU", config)
|
|
# [compile_model_no_auto_batching]
|
|
|
|
# [query_optimal_num_requests]
|
|
# when the batch size is automatically selected by the implementation
|
|
# it is important to query/create and run the sufficient requests
|
|
config = {hints.performance_mode(): hints.PerformanceMode.THROUGHPUT}
|
|
compiled_model = core.compile_model(model, "GPU", config)
|
|
num_requests = compiled_model.get_property(props.optimal_number_of_infer_requests())
|
|
# [query_optimal_num_requests]
|
|
|
|
# [hint_num_requests]
|
|
config = {hints.performance_mode(): hints.PerformanceMode.THROUGHPUT,
|
|
hints.num_requests(): "4"}
|
|
# limiting the available parallel slack for the 'throughput'
|
|
# so that certain parameters (like selected batch size) are automatically accommodated accordingly
|
|
compiled_model = core.compile_model(model, "GPU", config)
|
|
# [hint_num_requests]
|
|
|
|
# [hint_plus_low_level]
|
|
config = {hints.performance_mode(): hints.PerformanceMode.THROUGHPUT,
|
|
props.inference_num_threads(): "4"}
|
|
# limiting the available parallel slack for the 'throughput'
|
|
# so that certain parameters (like selected batch size) are automatically accommodated accordingly
|
|
compiled_model = core.compile_model(model, "CPU", config)
|
|
# [hint_plus_low_level]]
|