OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
aicomputer-visiondeep-learningdeploy-aidiffusion-modelsgenerative-aigood-first-issueinferencellm-inferencenatural-language-processingnlpopenvinooptimize-aiperformance-boostrecommendation-systemspeech-recognitionstable-diffusiontransformersyolo
d0eef043fd
* zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these * Optimized Infer Request Scheduling * remoteblob checks in the conventional SetBlob * correctly (with status) reporting NOT_IMPLEMENTED * SetBlob to accomodate for the RemoteBobs * Tests for remote blobs support via MULTI: creating the shared_test in case the other (closed source) plugins would want to use that (in the private shared_tests instantiations). Also instantiating the remote blobs tests for the some basic combinations to test the MULTI supports them * macos compilation (and general plugin platform support) fix * shuffled files, so that the MULTI tests are now part of the ieFuncTests (and need no separate target). Also brushed the macro that handales the NOT_IMPLEMENTED as bit * further shuffled files, so that the initial MULTI tests are now part of the IE tests, yet specific instances do need separate targets * Fixed misprint * Brushing the code and comments a bit * further brushing of the ScheduleToWorkerRequest: moving the task execution directly into the loop over devices (avoids pointers and 'else' clause) * 1) zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these 2) Initial MULTI section in the opt guide (primarily to document a tip on helping the MULTI to keep the zero-copy path) * [MULTI] remote context support and associated scheduling (respecting the remote data affinity) * fix CentOS (old) gcc issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81880 since the intriduced therad_local string is template the bug manifests itself (and the string is not allocated/initialized). the QA is to wrap the std::string into the function * further fix for the old gcc versions issue, now with non-trivial thread_local destruction sefault: switching from the std::string to the plain const char* * additional tests for the MULTI and remote blobs (no remote context and multi GPUs cases) * fix for the tests (that now can check for more specific NotImplemented exeption). Alos couple of line endings |
||
|---|---|---|
| .ci | ||
| .github | ||
| cmake | ||
| docs | ||
| inference-engine | ||
| licensing | ||
| model-optimizer | ||
| ngraph | ||
| openvino | ||
| scripts | ||
| tests | ||
| tools | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| CMakeLists.txt | ||
| CODEOWNERS | ||
| install_build_dependencies.sh | ||
| Jenkinsfile | ||
| LICENSE | ||
| README.md | ||
| SECURITY.md | ||
OpenVINO™ Toolkit - Deep Learning Deployment Toolkit repository
This toolkit allows developers to deploy pre-trained deep learning models through a high-level C++ Inference Engine API integrated with application logic.
This open source version includes several components: namely Model Optimizer, ngraph and Inference Engine, as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics. It supports pre-trained models from the Open Model Zoo, along with 100+ open source and public models in popular formats such as Caffe*, TensorFlow*, MXNet* and ONNX*.
Repository components:
License
Deep Learning Deployment Toolkit is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
Resources:
- Docs: https://docs.openvinotoolkit.org/
- Wiki: https://github.com/openvinotoolkit/openvino/wiki
- Issue tracking: https://github.com/openvinotoolkit/openvino/issues
- Additional OpenVINO modules: https://github.com/openvinotoolkit/openvino_contrib
- HomePage
- OpenVINO™ Release Notes
Support
Please report questions, issues and suggestions using:
- The
openvinotag on StackOverflow* - GitHub* Issues
- Forum
* Other names and brands may be claimed as the property of others.