OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference
aicomputer-visiondeep-learningdeploy-aidiffusion-modelsgenerative-aigood-first-issueinferencellm-inferencenatural-language-processingnlpopenvinooptimize-aiperformance-boostrecommendation-systemspeech-recognitionstable-diffusiontransformersyolo
e61a594199
* [IE]: Enables Abstract class -> Parameter conversion support Parameter has templated constructor allowing to write code ``` Parameter p = i; // i of type int for example ``` This constructor uses SFINAE to resolve ambiguity with move-constructor, so checks that argument is not of the same type. In case it's not the same type it calls std::tuple constructors that constructs an instance of argument type. In the following case: ``` Parameter p = static_cast<Parameter>(abstractRef); // abstractRef is a reference to abstract class ``` We have a reference to some abstract class that defines explicit cast operator to Parameter. In contrast with expectations, instead of cast operator, Parameter constructor is instantiated, since template type deduction for Parameter constructor didn't fail (abstract class has not the same type as Parameter). Instantiation of tuple constructor used inside failed: it's impossible to create an instance of abstract class what lead to compile-time error. To resolve the issue additional condition introduced to check if argument type is abstract. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE]: Enables PrintTo method for Parameter and tests on it Inference Engine API for configuration options uses Parameter type as a return type of GetConfig method. Parameter is intended to store object associated with configuration option. To support objects of different types its constructor is templated. Parameter overloads cast operators which are templated as well. Both constructor and cast operators are implicit, which makes it possible to implicitly convert any type to Parameter and vice versa. Since Parameter is a part of Inference Engine configuration API it's essential google tests on API contain Parameter as tests parameter. For each test parameter Google Test framework tries to print it to an output stream. For that purpose, Google Test checks if test parameter has output stream operator or PrintTo method. If not, it checks if it could be implicitly converted to integral type and, in this case, prints it as a long integer. InferenceEngine::Parameter does not define output stream operator, but could be implicitly converted to an integer, according cast operators mentioned above, so Google Test tries to convert to integer. Since Parameter not necessarily contains integer, this conversion throws an exception of type mismatch, which makes it impossible to use Parameter in Google Test framework as is. In order to resolve that issue Parameter should define either output stream operator or PrintTo method. If Parameter will define output stream operator it will make it possible to compile streaming almost any object to an output stream. The reason for it is C++ checks if object could be implicitly converted to other type which defines output stream operator, if objects itself doesn't do it (e.g. `stream << "text";` calls std::string::operator<<, since char const* is implicitly convertible to std::string). Taking this into consideration the only way to support Parameter in Google Test without breaking backward compatibility is define PrintTo method. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE]: Fixes ill-formed extending std names According to the standard: The behavior of a C++ program is undefined if it adds declarations or definitions to namespace std or to a namespace within namespace std unless otherwise specified. A program may add a template specialization for any standard library template to namespace std only if the declaration depends on a user-defined type and the specialization meets the standard library requirements for the original template and is not explicitly prohibited. As as an unexpected result, InferenceEngine::Parameter that contains std::vector<std::string> can be printed via PrintTo. In that case operator<< version from Inference Engine is picked up. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Moves CompilationConfig out of GT header Keeping config in a separate header simplifies migration to new interface. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Removes Platform enum Since there is enum from MVNC for the same purpose there is no need in Platform anyway Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Introduces containers utility header Contains some helpers to work with C++ maps Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Introduces new configuration API The main ideas are separate option-specific logic from common container, automate logic processing public vs private, deprecated, compile-time vs runtime-time options and remove code duplication. Since IE defines configuration API using std::string and Parameter, options have to provide ways to be represented as Parameter (ex.: GetConfig is called) and be defined using std::string (ex.: SetConfig is called). Keeping information about actual key value is useful for error reporting. New API fallbacks to previous version in case of unsupported options are requested. This way migration becomes iterative and looks simpler. Options containers are related to corresponding components: CompilationConfig (name to be changed) - GraphTransformer, PluginConfiguration - base class for plugins configurations, MyriadConfiguration - Myriad plugin configuration, HDDLConfiguration - HDDL plugin configuration (to be introduced in a separate request) Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Replaces CompilationConfig with PluginConfiguration Some of options to be refactored are stored inside CompilationConfig. CompilationConfig is passed to graph transformer as a compiler to be processed. Since it's separate data structure and migration process is iterative we need a mechanism to provide some of compilation options from new interface and some from old. It cannot be done via plugin specific class (MyriadConfiguration), since there are others plugins as graph transformer users. Plugin specific class (MyriadConfiguration) already inherits from old version (MyriadConfig), which in turn inherits from ParsedConfig containing CompilationConfig. To resolve the issue MyriadConfig inheritance from ParsedConfig is made virtual to make it possible for PluginConfiguration to virtually inherit from ParsedConfig as well an so make PluginConfiguration data structure for configuration options for graph transformer. Since PluginConfiguration is base class of MyriadConfiguration as well as MyriadConfig and inheritance is virtual plugin just casts its specific configuration to base one passing to graph transformer. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Enables new tests on configuration API * Enables following new shared tests on configuration API * Can load network with empty configuration * Check default value for configuration option * Can load network with correct configuration * Check custom value for configuration option (set and compare) * Check public configuration options are visible through API * Check private configuration options are invisible through API * Check GetConfig throws an exception on incorrect key * Refactors myriad plugin instantiations for shared tests Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Extracts LogLevel enum to a separate header Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Refactors LOG_LEVEL configuration option Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Refactors COPY_OPTIMIZATION configuration option Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Fixes behavior tests build Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Updates tests on new exception class Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Removes unused variable from mvnc test Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> * [IE][VPU]: Removes SizeVector streaming call New assertion macro IE_ASSERT implementation uses output streaming operator with r-value reference argument as a stream. This prevents the compiler from picking up overload from InferenceEngine::details, since our version takes stream by non-const l-value reference. Since there is no simple solution to provide output streaming operator overload for r-value references as well and this call is just a message for assert in test utilities, it was decided just to remove call for now. Signed-off-by: Gladilov, Gleb <gleb.gladilov@intel.com> |
||
|---|---|---|
| .ci | ||
| .github | ||
| cmake | ||
| docs | ||
| inference-engine | ||
| licensing | ||
| model-optimizer | ||
| ngraph | ||
| openvino | ||
| scripts | ||
| tests | ||
| thirdparty | ||
| tools | ||
| .gitattributes | ||
| .gitignore | ||
| .gitmodules | ||
| CMakeLists.txt | ||
| CODEOWNERS | ||
| install_build_dependencies.sh | ||
| Jenkinsfile | ||
| LICENSE | ||
| README.md | ||
| SECURITY.md | ||
OpenVINO™ Toolkit
This toolkit allows developers to deploy pre-trained deep learning models through a high-level C++ Inference Engine API integrated with application logic.
This open source version includes several components: namely Model Optimizer, nGraph and Inference Engine, as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics. It supports pre-trained models from the Open Model Zoo, along with 100+ open source and public models in popular formats such as Caffe*, TensorFlow*, MXNet* and ONNX*.
Repository components:
License
Deep Learning Deployment Toolkit is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
Resources:
- Docs: https://docs.openvinotoolkit.org/
- Wiki: https://github.com/openvinotoolkit/openvino/wiki
- Issue tracking: https://github.com/openvinotoolkit/openvino/issues
- Storage: https://storage.openvinotoolkit.org/
- Additional OpenVINO™ modules: https://github.com/openvinotoolkit/openvino_contrib
- Intel® Distribution of OpenVINO™ toolkit Product Page
- Intel® Distribution of OpenVINO™ toolkit Release Notes
Support
Please report questions, issues and suggestions using:
- The
openvinotag on StackOverflow* - GitHub* Issues
- Forum
* Other names and brands may be claimed as the property of others.