song, bell 2b9e726040 enable batching plugin through auto (#9520)
* initial commit

Signed-off-by: fishbell <bell.song@intel.com>

* remove debug code

Signed-off-by: fishbell <bell.song@intel.com>

* fix bug  with CPU,GPU

Signed-off-by: fishbell <bell.song@intel.com>

* refine code and test case for auto batching

Signed-off-by: fishbell <bell.song@intel.com>

* optimize the batching query

Signed-off-by: fishbell <bell.song@intel.com>

* refine test case

Signed-off-by: fishbell <bell.song@intel.com>

* optimize if/else

Signed-off-by: fishbell <bell.song@intel.com>

* fix sporadic crash

Signed-off-by: fishbell <bell.song@intel.com>

* refine logic

Signed-off-by: fishbell <bell.song@intel.com>

* refine case

Signed-off-by: fishbell <bell.song@intel.com>

* temp solution to disable batching in auto

Signed-off-by: fishbell <bell.song@intel.com>

* add debug logs

Signed-off-by: fishbell <bell.song@intel.com>

* merge with #9724

Signed-off-by: fishbell <bell.song@intel.com>

* refine the logic

Signed-off-by: fishbell <bell.song@intel.com>

* accurate debug message

Signed-off-by: fishbell <bell.song@intel.com>
2022-01-25 16:28:49 +08:00
2022-01-24 14:23:16 +03:00
2022-01-24 14:23:16 +03:00
2021-05-31 15:24:56 +03:00
2018-10-16 13:45:03 +03:00
2020-11-17 16:44:44 +03:00

OpenVINO™ Toolkit

Stable release Apache License Version 2.0 GitHub branch checks state Azure DevOps builds (branch) PyPI Downloads

This toolkit allows developers to deploy pre-trained deep learning models through a high-level C++ Inference Engine API integrated with application logic.

This open source version includes several components: namely Model Optimizer, nGraph and Inference Engine, as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics. It supports pre-trained models from the Open Model Zoo, along with 100+ open source and public models in popular formats such as Caffe*, TensorFlow*, MXNet* and ONNX*.

Repository components:

License

Deep Learning Deployment Toolkit is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

Resources:

Support

Please report questions, issues and suggestions using:


* Other names and brands may be claimed as the property of others.

Languages
C++ 80.5%
Python 15.5%
C 2.8%
CMake 0.9%
Cython 0.1%