Removed obsolete documentation (#10504)
* Removed obsolete documentation * Fixed documentation * Additional fix
This commit is contained in:
Ilya Churaev
GitHub
@@ -43,7 +43,7 @@ Please report questions, issues and suggestions using:
|
||||
\* Other names and brands may be claimed as the property of others.
|
||||
|
||||
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
|
||||
[OpenVINO™ Runtime]:https://docs.openvino.ai/latest/openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html
|
||||
[OpenVINO™ Runtime]:https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_User_Guide.html
|
||||
[Model Optimizer]:https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html
|
||||
[Post-Training Optimization Tool]:https://docs.openvino.ai/latest/pot_README.html
|
||||
[tag on StackOverflow]:https://stackoverflow.com/search?q=%23openvino
|
||||
|
||||
@@ -32,7 +32,7 @@ There are three steps to support inference of a model with custom operation(s):
|
||||
the Model Optimizer can generate the IR with the operation.
|
||||
2. Create an operation set and implement a custom nGraph operation in it as described in the
|
||||
[Custom nGraph Operation](../OV_Runtime_UG/Extensibility_DG/AddingNGraphOps.md).
|
||||
3. Implement a customer operation in one of the [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md)
|
||||
3. Implement a customer operation in one of the [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md)
|
||||
plugins to support inference of this operation using a particular target hardware (CPU, GPU or VPU).
|
||||
|
||||
To see the operations that are supported by each device plugin for the Inference Engine, refer to the
|
||||
|
||||
@@ -22,7 +22,7 @@
|
||||
|
||||
Model Optimizer is a cross-platform command-line tool that facilitates the transition between the training and deployment environment, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
|
||||
|
||||
Model Optimizer process assumes you have a network model trained using supported deep learning frameworks: Caffe*, TensorFlow*, Kaldi*, MXNet* or converted to the ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the network, which can be inferred with the [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md).
|
||||
Model Optimizer process assumes you have a network model trained using supported deep learning frameworks: Caffe*, TensorFlow*, Kaldi*, MXNet* or converted to the ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the network, which can be inferred with the [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md).
|
||||
|
||||
> **NOTE**: Model Optimizer does not infer models. Model Optimizer is an offline tool that runs before the inference takes place.
|
||||
|
||||
|
||||
@@ -10,8 +10,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
|
||||
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Caffe\*.
|
||||
2. [Convert a Caffe\* Model](#Convert_From_Caffe) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
|
||||
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
|
||||
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in your application to deploy the model in the target environment
|
||||
|
||||
## Supported Topologies
|
||||
|
||||
|
||||
@@ -16,8 +16,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
|
||||
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Kaldi\*.
|
||||
2. [Convert a Kaldi\* Model](#Convert_From_Kaldi) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
|
||||
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
|
||||
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in your application to deploy the model in the target environment.
|
||||
|
||||
> **NOTE**: The Model Optimizer supports the [nnet1](http://kaldi-asr.org/doc/dnn1.html) and [nnet2](http://kaldi-asr.org/doc/dnn2.html) formats of Kaldi models. Support of the [nnet3](http://kaldi-asr.org/doc/dnn3.html) format is limited.
|
||||
|
||||
|
||||
@@ -17,8 +17,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
|
||||
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for MXNet* (MXNet was used to train your model)
|
||||
2. [Convert a MXNet model](#ConvertMxNet) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
|
||||
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
|
||||
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in your application to deploy the model in the target environment
|
||||
|
||||
## Supported Topologies
|
||||
|
||||
|
||||
@@ -4,8 +4,8 @@ A summary of the steps for optimizing and deploying a model trained with Paddle\
|
||||
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Paddle\*.
|
||||
2. [Convert a Paddle\* Model](#Convert_From_Paddle) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases.
|
||||
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
|
||||
3. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in your application to deploy the model in the target environment.
|
||||
|
||||
## Supported Topologies
|
||||
|
||||
|
||||
@@ -48,7 +48,7 @@ PyTorch* framework is supported through export to ONNX\* format. A summary of th
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for ONNX\*.
|
||||
2. [Export PyTorch model to ONNX\*](#export-to-onnx).
|
||||
3. [Convert an ONNX\* model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
|
||||
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
5. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
|
||||
|
||||
## Export PyTorch\* Model to ONNX\* Format <a name="export-to-onnx"></a>
|
||||
|
||||
@@ -31,7 +31,7 @@ A summary of the steps for optimizing and deploying a model that was trained wit
|
||||
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for TensorFlow\* (TensorFlow was used to train your model).
|
||||
2. [Freeze the TensorFlow model](#freeze-the-tensorflow-model) if your model is not already frozen or skip this step and use the [instruction](#loading-nonfrozen-models) to a convert a non-frozen model.
|
||||
3. [Convert a TensorFlow\* model](#Convert_From_TF) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
|
||||
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
4. Test the model in the Intermediate Representation format using the [OpenVINO™ Runtime](../../../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
|
||||
5. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
|
||||
|
||||
## Supported Topologies
|
||||
|
||||
@@ -1,756 +1,10 @@
|
||||
# Inference Engine API Changes History {#openvino_docs_IE_DG_API_Changes}
|
||||
# OpenVINO™ Runtime API Changes History {#openvino_docs_OV_Runtime_API_Changes}
|
||||
|
||||
The sections below contain detailed list of changes made to the Inference Engine API in recent releases.
|
||||
The sections below contain detailed list of changes made to the OpenVINO™ Runtime API in recent releases.
|
||||
|
||||
## 2021.4
|
||||
## 2022.1
|
||||
|
||||
### New API
|
||||
|
||||
* InferenceEngine::Core::LoadNetwork(modelPath, deviceName, config) simplified API to read and load network in one call
|
||||
* The OpenVINO™ 2.0 API was introduced.
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**InferenceEngine::Parameter**
|
||||
|
||||
* InferenceEngine::Parameter(const std::shared_ptr<ngraph::Variant>&)
|
||||
* InferenceEngine::Parameter(std::shared_ptr<ngraph::Variant>& var)
|
||||
* std::shared_ptr<ngraph::Variant> InferenceEngine::Parameter::asVariant() const
|
||||
* InferenceEngine::Parameter::operator std::shared_ptr<ngraph::Variant>() const
|
||||
|
||||
**GPU plugin configuration keys**
|
||||
* KEY_CLDNN_NV12_TWO_INPUTS GPU plugin option. Use KEY_GPU_NV12_TWO_INPUTS instead
|
||||
* KEY_CLDNN_PLUGIN_PRIORITY GPU plugin option. Use KEY_GPU_PLUGIN_PRIORITY instead
|
||||
* KEY_CLDNN_PLUGIN_THROTTLE GPU plugin option. Use KEY_GPU_PLUGIN_THROTTLE instead
|
||||
* KEY_CLDNN_MEM_POOL GPU plugin option
|
||||
* KEY_CLDNN_GRAPH_DUMPS_DIR GPU plugin option
|
||||
* KEY_CLDNN_SOURCES_DUMPS_DIR GPU plugin option
|
||||
* KEY_DUMP_KERNELS GPU plugin option
|
||||
* KEY_TUNING_MODE GPU plugin option
|
||||
* KEY_TUNING_FILE GPU plugin option
|
||||
|
||||
**InferenceEngine::IInferRequest**
|
||||
* IInferRequest interface is deprecated, use InferRequest wrapper:
|
||||
* Constructor for InferRequest from IInferRequest:: Ptr is deprecated
|
||||
* Cast operator for InferRequest to IInferRequest shared pointer is deprecated
|
||||
|
||||
**InferenceEngine::ICNNNetwork**
|
||||
* ICNNNetwork interface is deprecated by means of deprecation of all its methods, use CNNNetwork wrapper
|
||||
* CNNNetwork methods working with ICNNNetwork are deprecated:
|
||||
* Cast to ICNNNetwork shared pointer
|
||||
* Cast to reference to ICNNNetwork interface
|
||||
* Constructor from ICNNNetwork shared pointer
|
||||
|
||||
**InferenceEngine::IExecutableNetwork**
|
||||
* IExecutableNetwork is deprecated, use ExecutableNetwork wrappers:
|
||||
* Constructor of ExecutableNetwork from IExecutableNetwork shared pointer is deprecated
|
||||
* The following ExecutableNetwork methods are deprecated:
|
||||
* ExecutableNetwork::reset
|
||||
* Cast operator to IExecutableNetwork shared pointer
|
||||
* ExecutableNetwork::CreateInferRequestPtr - use ExecutableNetwork::CreateInferRequest instead
|
||||
|
||||
**Extensions API**
|
||||
* InferenceEngine::make_so_pointer which is used to create Extensions library is replaced by std::make_shared<Extension>(..)
|
||||
* InferenceEngine::IExtension::Release is deprecated with no replacement
|
||||
* Use IE_DEFINE_EXTENSION_CREATE_FUNCTION helper macro instead of explicit declaration of CreateExtension function, which create extension.
|
||||
|
||||
**Other changes**
|
||||
* Version::ApiVersion structure is deprecated, Inference Engine does not have API version anymore
|
||||
* LowLatency - use lowLatency2 instead
|
||||
* CONFIG_KEY(DUMP_EXEC_GRAPH_AS_DOT) - use InferenceEngine::ExecutableNetwork::GetExecGraphInfo::serialize() instead
|
||||
* Core::ImportNetwork with no device - pass device name explicitly.
|
||||
* details::InferenceEngineException - use InferenceEngine::Exception and its derivatives instead.
|
||||
|
||||
## 2021.3
|
||||
|
||||
### New API
|
||||
|
||||
* InferenceEngine::InferRequest::Cancel to cancel inference request execution
|
||||
* InferenceEngine::Layout::HWC to support HWC layout for input or output blobs
|
||||
* InferenceEngine::Precision::F64 data precision for f64 data type
|
||||
* InferenceEngine::CNNNetwork::getOVNameForTensor to map frameworks tensor names to OpenVINO internal tensor names
|
||||
|
||||
### Deprecated API
|
||||
|
||||
* InferenceEngine::IVariableState interface is deprecated, use InferenceEngine::VariableState wrapper
|
||||
|
||||
## 2021.2
|
||||
|
||||
### New API
|
||||
|
||||
**State API**
|
||||
|
||||
* InferenceEngine::InferRequest::QueryState query state value of network on current infer request
|
||||
* InferenceEngine::IVariableState class instead of IMemoryState (rename)
|
||||
* InferenceEngine::IVariableState::GetState instead of IMemoryState::GetLastState (rename)
|
||||
|
||||
**BatchedBlob** - represents a InferenceEngine::BatchedBlob containing other blobs - one per batch.
|
||||
|
||||
**Transformations API** - added a new header `ie_transformations.hpp` which contains transformations for InferenceEngine::CNNNetwork object. Such transformations can be called prior to loading network for compilation for particular device:
|
||||
|
||||
* InferenceEngine::LowLatency
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**State API**
|
||||
|
||||
* InferenceEngine::ExecutableNetwork::QueryState - use InferenceEngine::InferRequest::QueryState
|
||||
* InferenceEngine::IVariableState::GetLastState - use InferenceEngine::IVariableState::GetState
|
||||
|
||||
## 2021.1
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**Utility functions to convert Unicode paths**
|
||||
|
||||
* InferenceEngine::stringToFileName - use OS-specific native conversion functions
|
||||
* InferenceEngine::fileNameToString - use OS-specific native conversion functions
|
||||
|
||||
### Removed API
|
||||
|
||||
**Plugin API:**
|
||||
|
||||
* InferenceEngine::InferencePlugin C++ plugin wrapper class
|
||||
* InferenceEngine::IInferencePlugin plugin interface
|
||||
* InferenceEngine::PluginDispatcher class
|
||||
* InferenceEngine::InferenceEnginePluginPtr typedef
|
||||
* InferenceEngine::ICNNNetReader reader interface
|
||||
* InferenceEngine::CNNNetReader class
|
||||
|
||||
**Extensibility API:**
|
||||
|
||||
* InferenceEngine::ILayerImplFactory class
|
||||
* InferenceEngine::IShapeInferImpl class
|
||||
* InferenceEngine::IShapeInferExtension class
|
||||
* InferenceEngine::IExtension::getFactoryFor(ILayerImplFactory\*& factory, const CNNLayer\* cnnLayer, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::IExtension::getPrimitiveTypes(char\*\*& types, unsigned int& size, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::ShapeInferImpl class
|
||||
* InferenceEngine::Extension::getFactoryFor(ILayerImplFactory\*& factory, const CNNLayer\* cnnLayer, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::Extension::getPrimitiveTypes(char\*\*& types, unsigned int& size, ResponseDesc\* resp) noexcept method
|
||||
|
||||
**Network API:**
|
||||
|
||||
* InferenceEngine::details::CNNNetworkIterator class
|
||||
* InferenceEngine::CNNNetwork::getPrecision() const method
|
||||
* InferenceEngine::CNNNetwork::getLayerByName(const char\* layerName) const method
|
||||
* InferenceEngine::CNNNetwork::size() const method
|
||||
* InferenceEngine::CNNNetwork::begin() const method
|
||||
* InferenceEngine::CNNNetwork::end() const method
|
||||
* InferenceEngine::CNNNetwork::AddExtension(const IShapeInferExtensionPtr& extension) method
|
||||
* InferenceEngine::ICNNNetwork::getPrecision() const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getName(char\* pName, size_t len) const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getData(const char\* dname) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::addLayer(const CNNLayerPtr& layer) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getLayerByName(const char\* layerName, CNNLayerPtr& out, ResponseDesc\* resp) const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::AddExtension(const IShapeInferExtensionPtr& extension, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getStats(ICNNNetworkStats\*\* stats, ResponseDesc\* resp) const noexcept method
|
||||
* InferenceEngine::ICNNNetworkStats class
|
||||
* InferenceEngine::NetworkNodeStats class
|
||||
* InferenceEngine::Data::getCreatorLayer() method
|
||||
* InferenceEngine::Data::getInputTo() method
|
||||
* InferenceEngine::LayerParams class
|
||||
|
||||
**Layer API:**
|
||||
|
||||
* InferenceEngine::CNNLayer class
|
||||
* InferenceEngine::WeightableLayer class
|
||||
* InferenceEngine::BatchNormalizationLayer class
|
||||
* InferenceEngine::BatchToSpaceLayer class
|
||||
* InferenceEngine::BinaryConvolutionLayer class
|
||||
* InferenceEngine::BroadcastLayer class
|
||||
* InferenceEngine::BucketizeLayer class
|
||||
* InferenceEngine::ClampLayer class
|
||||
* InferenceEngine::ConcatLayer class
|
||||
* InferenceEngine::ConvolutionLayer class
|
||||
* InferenceEngine::CropLayer class
|
||||
* InferenceEngine::DeconvolutionLayer class
|
||||
* InferenceEngine::DeformableConvolutionLayer class
|
||||
* InferenceEngine::DepthToSpaceLayer class
|
||||
* InferenceEngine::EltwiseLayer class
|
||||
* InferenceEngine::ExperimentalDetectronPriorGridGenerator class
|
||||
* InferenceEngine::ExperimentalDetectronPriorGridGeneratorLayer class
|
||||
* InferenceEngine::ExperimentalSparseWeightedReduceLayer class
|
||||
* InferenceEngine::FillLayer class
|
||||
* InferenceEngine::FullyConnectedLayer class
|
||||
* InferenceEngine::GRNLayer class
|
||||
* InferenceEngine::GRUCell class
|
||||
* InferenceEngine::GatherLayer class
|
||||
* InferenceEngine::GemmLayer class
|
||||
* InferenceEngine::LSTMCell class
|
||||
* InferenceEngine::MVNLayer class
|
||||
* InferenceEngine::MathLayer class
|
||||
* InferenceEngine::NonMaxSuppression class
|
||||
* InferenceEngine::NormLayer class
|
||||
* InferenceEngine::OneHotLayer class
|
||||
* InferenceEngine::PReLULayer class
|
||||
* InferenceEngine::PadLayer class
|
||||
* InferenceEngine::PoolingLayer class
|
||||
* InferenceEngine::PowerLayer class
|
||||
* InferenceEngine::QuantizeLayer class
|
||||
* InferenceEngine::RNNCell class
|
||||
* InferenceEngine::RNNCellBase class
|
||||
* InferenceEngine::RNNSequenceLayer class
|
||||
* InferenceEngine::RangeLayer class
|
||||
* InferenceEngine::ReLU6Layer class
|
||||
* InferenceEngine::ReLULayer class
|
||||
* InferenceEngine::ReduceLayer class
|
||||
* InferenceEngine::ReshapeLayer class
|
||||
* InferenceEngine::ReverseSequenceLayer class
|
||||
* InferenceEngine::ScaleShiftLayer class
|
||||
* InferenceEngine::ScatterLayer class
|
||||
* InferenceEngine::SelectLayer class
|
||||
* InferenceEngine::ShuffleChannelsLayer class
|
||||
* InferenceEngine::SoftMaxLayer class
|
||||
* InferenceEngine::SpaceToBatchLayer class
|
||||
* InferenceEngine::SpaceToDepthLayer class
|
||||
* InferenceEngine::SparseFillEmptyRowsLayer class
|
||||
* InferenceEngine::SparseSegmentReduceLayer class
|
||||
* InferenceEngine::SparseToDenseLayer class
|
||||
* InferenceEngine::SplitLayer class
|
||||
* InferenceEngine::StridedSliceLayer class
|
||||
* InferenceEngine::TensorIterator class
|
||||
* InferenceEngine::TileLayer class
|
||||
* InferenceEngine::TopKLayer class
|
||||
* InferenceEngine::UniqueLayer class
|
||||
|
||||
## 2020.4
|
||||
|
||||
### New API
|
||||
|
||||
**CPU Plugin API:**
|
||||
|
||||
* InferenceEngine::PluginConfigParams::KEY_ENFORCE_BF16 config key
|
||||
|
||||
**Metrics and values for Query API:**
|
||||
|
||||
* METRIC_KEY(OPTIMIZATION_CAPABILITIES)
|
||||
* METRIC_VALUE(BF16)
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**MYRIAD Plugin API:**
|
||||
|
||||
* VPU_CONFIG_KEY(IGNORE_IR_STATISTIC)
|
||||
|
||||
### Removed API
|
||||
|
||||
**Inference Engine NN Builder API:**
|
||||
|
||||
* InferenceEngine::Builder::EltwiseLayer
|
||||
* InferenceEngine::Builder::MemoryLayer
|
||||
* InferenceEngine::Builder::ROIPoolingLayer
|
||||
* InferenceEngine::Builder::DeconvolutionLayer
|
||||
* InferenceEngine::Builder::ReLULayer
|
||||
* InferenceEngine::Builder::TanHLayer
|
||||
* InferenceEngine::Builder::InputLayer
|
||||
* InferenceEngine::Builder::PoolingLayer
|
||||
* InferenceEngine::Builder::CropLayer
|
||||
* InferenceEngine::Builder::GRUSequenceLayer
|
||||
* InferenceEngine::Builder::NormLayer
|
||||
* InferenceEngine::Builder::LSTMSequenceLayer
|
||||
* InferenceEngine::Builder::ClampLayer
|
||||
* InferenceEngine::Builder::PSROIPoolingLayer
|
||||
* InferenceEngine::Builder::Layer
|
||||
* InferenceEngine::Builder::RNNSequenceLayer
|
||||
* InferenceEngine::Builder::ReorgYoloLayer
|
||||
* InferenceEngine::Builder::NormalizeLayer
|
||||
* InferenceEngine::Builder::PriorBoxClusteredLayer
|
||||
* InferenceEngine::Builder::MVNLayer
|
||||
* InferenceEngine::Builder::PermuteLayer
|
||||
* InferenceEngine::Builder::SimplerNMSLayer
|
||||
* InferenceEngine::Builder::ConstLayer
|
||||
* InferenceEngine::Builder::DeformableConvolutionLayer
|
||||
* InferenceEngine::Builder::FullyConnectedLayer
|
||||
* InferenceEngine::Builder::PriorBoxLayer
|
||||
* InferenceEngine::Builder::SoftMaxLayer
|
||||
* InferenceEngine::Builder::OutputLayer
|
||||
* InferenceEngine::Builder::TileLayer
|
||||
* InferenceEngine::Builder::SplitLayer
|
||||
* InferenceEngine::Builder::PReLULayer
|
||||
* InferenceEngine::Builder::RegionYoloLayer
|
||||
* InferenceEngine::Builder::ReshapeLayer
|
||||
* InferenceEngine::Builder::ConvolutionLayer
|
||||
* InferenceEngine::Builder::DetectionOutputLayer
|
||||
* InferenceEngine::Builder::ConcatLayer
|
||||
* InferenceEngine::Builder::ELULayer
|
||||
* InferenceEngine::Builder::GRNLayer
|
||||
* InferenceEngine::Builder::LRNLayer
|
||||
* InferenceEngine::Builder::ArgMaxLayer
|
||||
* InferenceEngine::Builder::ReLU6Layer
|
||||
* InferenceEngine::Builder::ScaleShiftLayer
|
||||
* InferenceEngine::Builder::ProposalLayer
|
||||
* InferenceEngine::Builder::SigmoidLayer
|
||||
* InferenceEngine::Builder::ResampleLayer
|
||||
* InferenceEngine::Builder::CTCGreedyDecoderLayer
|
||||
* InferenceEngine::Builder::BatchNormalizationLayer
|
||||
* InferenceEngine::Builder::LayerDecorator
|
||||
* InferenceEngine::Builder::PowerLayer
|
||||
* InferenceEngine::Builder::Network
|
||||
* InferenceEngine::Builder::PortInfo
|
||||
* InferenceEngine::Builder::Connection
|
||||
* InferenceEngine::Builder::PortData
|
||||
* InferenceEngine::Builder::Port
|
||||
* InferenceEngine::Builder::ILayer
|
||||
* InferenceEngine::Builder::INetworkIterator
|
||||
* InferenceEngine::Builder::INetwork
|
||||
* InferenceEngine::Builder::ILayer
|
||||
|
||||
## 2020.2
|
||||
|
||||
### New API
|
||||
|
||||
**Extensibility API:**
|
||||
|
||||
* InferenceEngine::IExtension::getImplTypes(const std::shared_ptr<ngraph::Node>& node) method
|
||||
* InferenceEngine::IExtension::getImplementation(const std::shared_ptr<ngraph::Node>& node, const std::string& implType) method
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**Extensibility API:**
|
||||
|
||||
* InferenceEngine::ILayerImplFactory class
|
||||
* InferenceEngine::IShapeInferImpl class
|
||||
* InferenceEngine::IShapeInferImpl class
|
||||
* InferenceEngine::IShapeInferExtension class
|
||||
* InferenceEngine::IExtension::getFactoryFor(ILayerImplFactory\*& factory, const CNNLayer\* cnnLayer, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::IExtension::getPrimitiveTypes(char\*\*& types, unsigned int& size, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::ShapeInferImpl class
|
||||
* InferenceEngine::Extension::getFactoryFor(ILayerImplFactory\*& factory, const CNNLayer\* cnnLayer, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::Extension::getPrimitiveTypes(char\*\*& types, unsigned int& size, ResponseDesc\* resp) noexcept method
|
||||
|
||||
**Network API:**
|
||||
|
||||
* InferenceEngine::details::CNNNetworkIterator class
|
||||
* InferenceEngine::CNNNetwork::getPrecision() const method
|
||||
* InferenceEngine::CNNNetwork::getLayerByName(const char\* layerName) const method
|
||||
* InferenceEngine::CNNNetwork::size() const method
|
||||
* InferenceEngine::CNNNetwork::begin() const method
|
||||
* InferenceEngine::CNNNetwork::end() const method
|
||||
* InferenceEngine::CNNNetwork::AddExtension(const IShapeInferExtensionPtr& extension) method
|
||||
* InferenceEngine::ICNNNetwork::getPrecision() const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getName(char\* pName, size_t len) const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getData(const char\* dname) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::addLayer(const CNNLayerPtr& layer) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getLayerByName(const char\* layerName, CNNLayerPtr& out, ResponseDesc\* resp) const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::AddExtension(const IShapeInferExtensionPtr& extension, ResponseDesc\* resp) noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getStats(ICNNNetworkStats\*\* stats, ResponseDesc\* resp) const noexcept method
|
||||
* InferenceEngine::ICNNNetworkStats class
|
||||
* InferenceEngine::NetworkNodeStats class
|
||||
* InferenceEngine::Data::getCreatorLayer() method
|
||||
* InferenceEngine::Data::getInputTo() method
|
||||
* InferenceEngine::LayerParams class
|
||||
|
||||
**Layer API:**
|
||||
|
||||
* InferenceEngine::CNNLayer class
|
||||
* InferenceEngine::WeightableLayer class
|
||||
* InferenceEngine::BatchNormalizationLayer class
|
||||
* InferenceEngine::BatchToSpaceLayer class
|
||||
* InferenceEngine::BinaryConvolutionLayer class
|
||||
* InferenceEngine::BroadcastLayer class
|
||||
* InferenceEngine::BucketizeLayer class
|
||||
* InferenceEngine::ClampLayer class
|
||||
* InferenceEngine::ConcatLayer class
|
||||
* InferenceEngine::ConvolutionLayer class
|
||||
* InferenceEngine::CropLayer class
|
||||
* InferenceEngine::DeconvolutionLayer class
|
||||
* InferenceEngine::DeformableConvolutionLayer class
|
||||
* InferenceEngine::DepthToSpaceLayer class
|
||||
* InferenceEngine::EltwiseLayer class
|
||||
* InferenceEngine::ExperimentalDetectronPriorGridGenerator class
|
||||
* InferenceEngine::ExperimentalDetectronPriorGridGeneratorLayer class
|
||||
* InferenceEngine::ExperimentalSparseWeightedReduceLayer class
|
||||
* InferenceEngine::FillLayer class
|
||||
* InferenceEngine::FullyConnectedLayer class
|
||||
* InferenceEngine::GRNLayer class
|
||||
* InferenceEngine::GRUCell class
|
||||
* InferenceEngine::GatherLayer class
|
||||
* InferenceEngine::GemmLayer class
|
||||
* InferenceEngine::LSTMCell class
|
||||
* InferenceEngine::MVNLayer class
|
||||
* InferenceEngine::MathLayer class
|
||||
* InferenceEngine::NonMaxSuppression class
|
||||
* InferenceEngine::NormLayer class
|
||||
* InferenceEngine::OneHotLayer class
|
||||
* InferenceEngine::PReLULayer class
|
||||
* InferenceEngine::PadLayer class
|
||||
* InferenceEngine::PoolingLayer class
|
||||
* InferenceEngine::PowerLayer class
|
||||
* InferenceEngine::QuantizeLayer class
|
||||
* InferenceEngine::RNNCell class
|
||||
* InferenceEngine::RNNCellBase class
|
||||
* InferenceEngine::RNNSequenceLayer class
|
||||
* InferenceEngine::RangeLayer class
|
||||
* InferenceEngine::ReLU6Layer class
|
||||
* InferenceEngine::ReLULayer class
|
||||
* InferenceEngine::ReduceLayer class
|
||||
* InferenceEngine::ReshapeLayer class
|
||||
* InferenceEngine::ReverseSequenceLayer class
|
||||
* InferenceEngine::ScaleShiftLayer class
|
||||
* InferenceEngine::ScatterLayer class
|
||||
* InferenceEngine::SelectLayer class
|
||||
* InferenceEngine::ShuffleChannelsLayer class
|
||||
* InferenceEngine::SoftMaxLayer class
|
||||
* InferenceEngine::SpaceToBatchLayer class
|
||||
* InferenceEngine::SpaceToDepthLayer class
|
||||
* InferenceEngine::SparseFillEmptyRowsLayer class
|
||||
* InferenceEngine::SparseSegmentReduceLayer class
|
||||
* InferenceEngine::SparseToDenseLayer class
|
||||
* InferenceEngine::SplitLayer class
|
||||
* InferenceEngine::StridedSliceLayer class
|
||||
* InferenceEngine::TensorIterator class
|
||||
* InferenceEngine::TileLayer class
|
||||
* InferenceEngine::TopKLayer class
|
||||
* InferenceEngine::UniqueLayer class
|
||||
|
||||
## 2020.1
|
||||
|
||||
### New API
|
||||
|
||||
**Integration with ngraph API:**
|
||||
|
||||
* InferenceEngine::CNNNetwork(const std::shared_ptr<ngraph::Function>& network) ctor from ngraph::Function
|
||||
* InferenceEngine::CNNNetwork::getFunction() const noexcept method
|
||||
* InferenceEngine::ICNNNetwork::getFunction() const noexcept method
|
||||
* InferenceEngine::Parameter(const std::shared_ptr<ngraph::Variant>& var) ctor
|
||||
* InferenceEngine::Parameter::asVariant() const method
|
||||
* InferenceEngine::Parameter::operator std::shared_ptr<ngraph::Variant>() const operator
|
||||
* InferenceEngine::Core::ReadNetwork(const std::wstring& modelPath, const std::wstring& binPath) method
|
||||
* InferenceEngine::Core::ReadNetwork(const std::string& modelPath, const std::string& binPath = "") method
|
||||
* InferenceEngine::Core::ReadNetwork(const std::string& model, const Blob::CPtr& weights) method
|
||||
* InferenceEngine::Code::AddExtension(const IExtensionPtr& extension) method
|
||||
* InferenceEngine::IExtension::getOpSets() method
|
||||
|
||||
|
||||
**Offline compilation: import / export to std::stream:**
|
||||
|
||||
* InferenceEngine::ExecutableNetwork::Export(std::ostream& networkModel) method
|
||||
* InferenceEngine::Core::ImportNetwork(std::istream& networkModel, const std::string& deviceName = {}, const std::map<std::string, std::string>& config = {}) method
|
||||
* InferenceEngine::IExecutableNetwork::Export(std::ostream& networkModel, ResponseDesc \*resp) noexcept method
|
||||
|
||||
|
||||
**RemoteBlob accelerator memory sharing API:**
|
||||
|
||||
* InferenceEngine::RemoteContext class
|
||||
* InferenceEngine::RemoteBlob class
|
||||
* InferenceEngine::Core::CreateContext(const std::string& deviceName, const ParamMap& params) method
|
||||
* InferenceEngine::Core::GetDefaultContext(const std::string& deviceName) method
|
||||
* InferenceEngine::Core::LoadNetwork(CNNNetwork network, RemoteContext::Ptr context, const std::map<std::string, std::string>& config = std::map<std::string, std::string>()) method
|
||||
|
||||
|
||||
**GNA firmware model image generation:**
|
||||
|
||||
* GNA_CONFIG_KEY(FIRMWARE_MODEL_IMAGE_GENERATION) config key
|
||||
* GNA_CONFIG_VALUE(GEN) value
|
||||
* GNA_CONFIG_VALUE(GEN_EXACT) value
|
||||
* GNA_CONFIG_VALUE(SSE) value
|
||||
* GNA_CONFIG_VALUE(SSE_EXACT) value
|
||||
* GNA_CONFIG_VALUE(AVX1) value
|
||||
* GNA_CONFIG_VALUE(AVX1_EXACT) value
|
||||
* GNA_CONFIG_VALUE(AVX2) value
|
||||
* GNA_CONFIG_VALUE(AVX2_EXACT) value
|
||||
|
||||
**MemoryBlob mapping of memory to the user space:**
|
||||
|
||||
* InferenceEngine::MemoryBlob::rwmap() noexcept method
|
||||
* InferenceEngine::MemoryBlob::rmap() noexcept method
|
||||
* InferenceEngine::MemoryBlob::wmap() noexcept method
|
||||
|
||||
**Memory interoperability on acceleration devices. General classes and GPU helper functions**
|
||||
* InferenceEngine::RemoteBlob class
|
||||
* InferenceEngine::RemoteContext class
|
||||
* InferenceEngine::Core::CreateContext(const std::string& deviceName, const ParamMap& params) method
|
||||
* InferenceEngine::Core::GetDefaultContext(const std::string& deviceName) method
|
||||
* InferenceEngine::make_shared_blob(const TensorDesc& desc, RemoteContext::Ptr ctx) function
|
||||
* InferenceEngine::gpu::make_shared_blob_nv12(size_t height, size_t width, RemoteContext::Ptr ctx, VASurfaceID nv12_surf) function
|
||||
* InferenceEngine::gpu::make_shared_context(Core& core, std::string deviceName, VADisplay device) function
|
||||
* InferenceEngine::gpu::make_shared_blob(const TensorDesc& desc, RemoteContext::Ptr ctx, VASurfaceID surface, uint32_t plane = 0) function
|
||||
* InferenceEngine::gpu::make_shared_blob_nv12(RemoteContext::Ptr ctx, cl::Image2D& nv12_image_plane_y, cl::Image2D& nv12_image_plane_uv) function
|
||||
* InferenceEngine::gpu::make_shared_context(Core& core, std::string deviceName, cl_context ctx) function
|
||||
* InferenceEngine::gpu::make_shared_blob(const TensorDesc& desc, ClContext::Ptr ctx) function
|
||||
* InferenceEngine::gpu::make_shared_blob(const TensorDesc& desc, RemoteContext::Ptr ctx, cl::Buffer& buffer) function
|
||||
* InferenceEngine::gpu::make_shared_blob(const TensorDesc& desc, RemoteContext::Ptr ctx, cl_mem buffer) function
|
||||
* InferenceEngine::gpu::make_shared_blob(const TensorDesc& desc, RemoteContext::Ptr ctx, cl::Image2D& image) function
|
||||
|
||||
### Deprecated API
|
||||
|
||||
**Inference Engine NN Builder API:**
|
||||
|
||||
* InferenceEngine::Builder::EltwiseLayer
|
||||
* InferenceEngine::Builder::MemoryLayer
|
||||
* InferenceEngine::Builder::ROIPoolingLayer
|
||||
* InferenceEngine::Builder::DeconvolutionLayer
|
||||
* InferenceEngine::Builder::ReLULayer
|
||||
* InferenceEngine::Builder::TanHLayer
|
||||
* InferenceEngine::Builder::InputLayer
|
||||
* InferenceEngine::Builder::PoolingLayer
|
||||
* InferenceEngine::Builder::CropLayer
|
||||
* InferenceEngine::Builder::GRUSequenceLayer
|
||||
* InferenceEngine::Builder::NormLayer
|
||||
* InferenceEngine::Builder::LSTMSequenceLayer
|
||||
* InferenceEngine::Builder::ClampLayer
|
||||
* InferenceEngine::Builder::PSROIPoolingLayer
|
||||
* InferenceEngine::Builder::Layer
|
||||
* InferenceEngine::Builder::RNNSequenceLayer
|
||||
* InferenceEngine::Builder::ReorgYoloLayer
|
||||
* InferenceEngine::Builder::NormalizeLayer
|
||||
* InferenceEngine::Builder::PriorBoxClusteredLayer
|
||||
* InferenceEngine::Builder::MVNLayer
|
||||
* InferenceEngine::Builder::PermuteLayer
|
||||
* InferenceEngine::Builder::SimplerNMSLayer
|
||||
* InferenceEngine::Builder::ConstLayer
|
||||
* InferenceEngine::Builder::DeformableConvolutionLayer
|
||||
* InferenceEngine::Builder::FullyConnectedLayer
|
||||
* InferenceEngine::Builder::PriorBoxLayer
|
||||
* InferenceEngine::Builder::SoftMaxLayer
|
||||
* InferenceEngine::Builder::OutputLayer
|
||||
* InferenceEngine::Builder::TileLayer
|
||||
* InferenceEngine::Builder::SplitLayer
|
||||
* InferenceEngine::Builder::PReLULayer
|
||||
* InferenceEngine::Builder::RegionYoloLayer
|
||||
* InferenceEngine::Builder::ReshapeLayer
|
||||
* InferenceEngine::Builder::ConvolutionLayer
|
||||
* InferenceEngine::Builder::DetectionOutputLayer
|
||||
* InferenceEngine::Builder::ConcatLayer
|
||||
* InferenceEngine::Builder::ELULayer
|
||||
* InferenceEngine::Builder::GRNLayer
|
||||
* InferenceEngine::Builder::LRNLayer
|
||||
* InferenceEngine::Builder::ArgMaxLayer
|
||||
* InferenceEngine::Builder::ReLU6Layer
|
||||
* InferenceEngine::Builder::ScaleShiftLayer
|
||||
* InferenceEngine::Builder::ProposalLayer
|
||||
* InferenceEngine::Builder::SigmoidLayer
|
||||
* InferenceEngine::Builder::ResampleLayer
|
||||
* InferenceEngine::Builder::CTCGreedyDecoderLayer
|
||||
* InferenceEngine::Builder::BatchNormalizationLayer
|
||||
* InferenceEngine::Builder::LayerDecorator
|
||||
* InferenceEngine::Builder::PowerLayer
|
||||
* InferenceEngine::Builder::Network
|
||||
* InferenceEngine::Builder::PortInfo
|
||||
* InferenceEngine::Builder::Connection
|
||||
* InferenceEngine::Builder::PortData
|
||||
* InferenceEngine::Builder::Port
|
||||
* InferenceEngine::Builder::ILayer
|
||||
* InferenceEngine::Builder::INetworkIterator
|
||||
* InferenceEngine::Builder::INetwork
|
||||
* InferenceEngine::Builder::ILayer
|
||||
|
||||
**Plugin API:**
|
||||
|
||||
* InferenceEngine::InferencePlugin C++ plugin wrapper class
|
||||
* InferenceEngine::IInferencePlugin plugin interface
|
||||
* InferenceEngine::PluginDispatcher class
|
||||
* InferenceEngine::InferenceEnginePluginPtr typedef
|
||||
* InferenceEngine::ICNNNetReader reader interface
|
||||
* InferenceEngine::CNNNetReader class
|
||||
|
||||
**Blob API:**
|
||||
|
||||
* Blob::element_size() const noexcept method
|
||||
* Blob::buffer() noexcept method
|
||||
* Blob::cbuffer() noexcept method
|
||||
* MemoryBlob::buffer() noexcept method
|
||||
* MemoryBlob::cbuffer() noexcept method
|
||||
|
||||
|
||||
### Removed API
|
||||
|
||||
Removed all [Inference Engine API which deprecated in 2019'R2](https://docs.openvino.ai/2019_R3/_docs_IE_DG_API_Changes.html#deprecated_api)
|
||||
|
||||
## 2019 R3
|
||||
|
||||
### New API
|
||||
|
||||
**New supported layers:**
|
||||
|
||||
* InferenceEngine::SparseFillEmptyRowsLayer new class
|
||||
* InferenceEngine::UniqueLayer new class
|
||||
* InferenceEngine::NonMaxSuppressionLayer new class
|
||||
* InferenceEngine::ScatterLayer new class
|
||||
|
||||
**FPGA plugin streaming support:**
|
||||
|
||||
* DLIA_METRIC_VALUE(INPUT_STREAMING) value to METRIC_KEY(OPTIMIZATION_CAPABILITIES)
|
||||
* DLIA_CONFIG_KEY(ENABLE_STREAMING) config key
|
||||
|
||||
### Removed API
|
||||
|
||||
* InferenceEngine::EltwiseLayer::Select from InferenceEngine::EltwiseLayer::eOperation enumeration
|
||||
|
||||
## 2019 R2
|
||||
|
||||
### New API
|
||||
|
||||
**Inference Engine Core API:**
|
||||
|
||||
* Introduced InferenceEngine::Core high level class to manage devices
|
||||
|
||||
**Query API extensions to InferenceEngine::ExecutableNetwork and InferenceEngine::IExecutableNetwork:**
|
||||
|
||||
* InferenceEngine::ExecutableNetwork::SetConfig method
|
||||
* InferenceEngine::ExecutableNetwork::GetConfig method
|
||||
* InferenceEngine::ExecutableNetwork::GetMetric method
|
||||
* InferenceEngine::IExecutableNetwork::SetConfig method
|
||||
* InferenceEngine::IExecutableNetwork::GetConfig method
|
||||
* InferenceEngine::IExecutableNetwork::GetMetric method
|
||||
|
||||
**Metrics and values for Query API:**
|
||||
|
||||
* METRIC_KEY(AVAILABLE_DEVICES)
|
||||
* METRIC_KEY(SUPPORTED_METRICS)
|
||||
* METRIC_KEY(SUPPORTED_CONFIG_KEYS)
|
||||
* METRIC_KEY(FULL_DEVICE_NAME)
|
||||
* METRIC_KEY(OPTIMIZATION_CAPABILITIES)
|
||||
* METRIC_VALUE(FP32)
|
||||
* METRIC_VALUE(FP16)
|
||||
* METRIC_VALUE(INT8)
|
||||
* METRIC_VALUE(BIN)
|
||||
* METRIC_VALUE(WINOGRAD)
|
||||
* DLIA_METRIC_VALUE(FP11)
|
||||
* METRIC_KEY(RANGE_FOR_STREAMS)
|
||||
* METRIC_KEY(NUMBER_OF_WAITING_INFER_REQUESTS)
|
||||
* METRIC_KEY(NUMBER_OF_EXEC_INFER_REQUESTS)
|
||||
* METRIC_KEY(DEVICE_THERMAL)
|
||||
* METRIC_KEY(RANGE_FOR_ASYNC_INFER_REQUESTS)
|
||||
* EXEC_NETWORK_METRIC_KEY(NETWORK_NAME)
|
||||
* EXEC_NETWORK_METRIC_KEY(OPTIMAL_NUMBER_OF_INFER_REQUESTS)
|
||||
|
||||
**Common API:**
|
||||
|
||||
* CLDNN_CONFIG_KEY(INT8_ENABLED) config key
|
||||
* CONFIG_KEY(GPU_THROUGHPUT_AUTO)
|
||||
* CONFIG_KEY(GPU_THROUGHPUT_STREAMS)
|
||||
* DLIA_CONFIG_KEY(IO_TRANSFORMATIONS_NATIVE) config key
|
||||
* DLIA_CONFIG_KEY(DUMP_SUPPORTED_LAYERS_INFORMATION) config key
|
||||
* GNA_CONFIG_VALUE(SW_FP32) config value for GNA_CONFIG_KEY(DEVICE_MODE) key
|
||||
* MULTI_CONFIG_KEY(DEVICE_PRIORITIES) config key for `MULTI` device
|
||||
* InferenceEngine::CNNNetReader::ReadNetwork(const std::wstring &filepath) new method
|
||||
* InferenceEngine::CNNNetReader::ReadWeights(const std::wstring &filepath) new method
|
||||
* InferenceEngine::ExecutableNetwork::ExecutableNetwork(IExecutableNetwork::Ptr actual, InferenceEnginePluginPtr plg) constructor with additional `plg` parameter
|
||||
* InferenceEngine::InferRequest::InferRequest(IInferRequest::Ptr request, InferenceEnginePluginPtr plg) constructor with additional `plg` parameter
|
||||
* InferenceEngine::Data::setName method
|
||||
* InferenceEngine::QueryNetworkResult::supportedLayersMap
|
||||
* InferenceEngine::Precision::I64 extension to InferenceEngine::Precision::ePrecision enumeration
|
||||
|
||||
**New supported primitives:**
|
||||
|
||||
* InferenceEngine::Builder::DeformableConvolutionLayer new class
|
||||
* InferenceEngine::DeformableConvolutionLayer new class
|
||||
* InferenceEngine::EltwiseLayer::Logical_NOT, InferenceEngine::EltwiseLayer::Mean, InferenceEngine::EltwiseLayer::Select extensions to InferenceEngine::EltwiseLayer::eOperation enumeration
|
||||
* InferenceEngine::OneHotLayer new class
|
||||
* InferenceEngine::SelectLayer new class
|
||||
* InferenceEngine::BroadcastLayer new class
|
||||
* InferenceEngine::MathLayer new class
|
||||
* InferenceEngine::ReduceLayer new class
|
||||
* InferenceEngine::TopKLayer new class
|
||||
|
||||
**Extensions to Blob creation API:**
|
||||
|
||||
* InferenceEngine::Blob::is method
|
||||
* InferenceEngine::Blob::is const method
|
||||
* InferenceEngine::Blob::as method
|
||||
* InferenceEngine::Blob::as const method
|
||||
* InferenceEngine::Blob::getAllocator abstract method
|
||||
* InferenceEngine::Blob::getHandle abstract method
|
||||
* InferenceEngine::MemoryBlob class
|
||||
* InferenceEngine::ColorFormat enumeration
|
||||
* InferenceEngine::PreProcessInfo::setColorFormat method
|
||||
* InferenceEngine::PreProcessInfo::getColorFormat method
|
||||
* InferenceEngine::CompoundBlob class to work with blobs consisting of several planes
|
||||
* InferenceEngine::NV12Blob class representing NV12 blob with two planes
|
||||
|
||||
### Deprecated API
|
||||
|
||||
The methods listed below are deprecated and will be removed in 2019 R4 release:
|
||||
|
||||
**Common API:**
|
||||
|
||||
* InferenceEngine::InputInfo::getInputPrecision method
|
||||
* InferenceEngine::InputInfo::setInputPrecision method
|
||||
* InferenceEngine::InputInfo::getDims method
|
||||
* InferenceEngine::CNNLayer::GetParamsAsBool method
|
||||
* InferenceEngine::CNNNetwork::CNNNetwork(ICNNNetwork* actual) constructor
|
||||
* InferenceEngine::CNNNetwork::setTargetDevice method
|
||||
* HETERO_CONFIG_KEY(DUMP_DLA_MESSAGES) config key
|
||||
* InferenceEngine::ILayerImplFactory::getShapes method
|
||||
* InferenceEngine::IShapeInferImpl::inferShapes(const std::vector<SizeVector>&, const std::map<std::string, std::string>& , const std::map<std::string, Blob::Ptr>&, std::vector<SizeVector>&, ResponseDesc\*) method
|
||||
* InferenceEngine::Data::setBatchSize method
|
||||
* InferenceEngine::QueryNetworkResult::supportedLayers field
|
||||
* InferenceEngine::ICNNNetwork::setBatchSize(const size_t size) method
|
||||
* InferenceEngine::Blob::Resize method
|
||||
* InferenceEngine::Blob::Reshape method
|
||||
* InferenceEngine::TBlob::set method
|
||||
|
||||
**InferenceEngine::IInferencePlugin and InferenceEngine:InferencePlugin obsolete methods:**
|
||||
|
||||
* InferenceEngine::InferencePlugin::LoadNetwork(ICNNNetwork &network) method
|
||||
* InferenceEngine::InferencePlugin::Infer method
|
||||
* InferenceEngine::InferencePlugin::GetPerformanceCounts method
|
||||
* InferenceEngine::InferencePlugin::QueryNetwork(const ICNNNetwork &network, QueryNetworkResult &res) const method
|
||||
* InferenceEngine::IInferencePlugin::LoadNetwork(ICNNNetwork &network, ResponseDesc \*resp) method
|
||||
* InferenceEngine::IInferencePlugin::Infer(const Blob &input, Blob &result, ResponseDesc \*resp) method
|
||||
* InferenceEngine::IInferencePlugin::Infer(const BlobMap &input, BlobMap &result, ResponseDesc \*resp) method
|
||||
* InferenceEngine::IInferencePlugin::GetPerformanceCounts method
|
||||
* InferenceEngine::IInferencePlugin::QueryNetwork(const ICNNNetwork& network, QueryNetworkResult& res) const method
|
||||
|
||||
|
||||
**Fields in InferenceEngine::Data class are replaced with appropriate methods:**
|
||||
|
||||
* InferenceEngine::Data::precision field
|
||||
* InferenceEngine::Data::layout field
|
||||
* InferenceEngine::Data::dims field
|
||||
* InferenceEngine::Data::creatorLayer field
|
||||
* InferenceEngine::Data::name field
|
||||
* InferenceEngine::Data::inputTo field
|
||||
* InferenceEngine::Data::userObject field
|
||||
|
||||
**Heterogeneous plugin:**
|
||||
|
||||
* InferenceEngine::IHeteroDeviceLoader class
|
||||
* InferenceEngine::IHeteroInferencePlugin class
|
||||
* InferenceEngine::HeteroPluginPtr class
|
||||
* operator InferenceEngine::InferencePlugin::HeteroPluginPtr operator
|
||||
|
||||
**Blob creation API with dimensions in reverse order:**
|
||||
|
||||
* InferenceEngine::Blob::Blob(Precision p) constructor
|
||||
* InferenceEngine::Blob::Blob(Precision p, Layout l) constructor
|
||||
* InferenceEngine::Blob::Blob(Precision p, const SizeVector &dims) constructor
|
||||
* InferenceEngine::Blob::Blob(Precision p, Layout l, const SizeVector &dims) constructor
|
||||
* InferenceEngine::TBlob::TBlob(Precision p, Layout l) constructor
|
||||
* InferenceEngine::TBlob::TBlob(Precision p, Layout l, const SizeVector& dims) constructor
|
||||
* InferenceEngine::TBlob::TBlob(Precision p, Layout l, const SizeVector& dims, T* ptr, size_t data_size) constructor
|
||||
* InferenceEngine::TBlob::TBlob(Precision p, Layout l, const SizeVector &dims, std::shared_ptr<IAllocator> alloc) constructor
|
||||
* InferenceEngine::Blob::type() method
|
||||
* InferenceEngine::Blob::precision() method
|
||||
* InferenceEngine::Blob::layout() method
|
||||
* InferenceEngine::Blob::dims() method
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l, const SizeVector &dims) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, const SizeVector &dims) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l, const TArg &arg) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, const TArg &arg) function
|
||||
* InferenceEngine::make_shared_blob(TBlob<TypeTo> &&arg) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l, SizeVector dims, const std::vector<TypeTo> &arg) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l, const std::vector<TypeTo> &arg) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, const std::vector<TypeTo> &arg) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, Layout l, const SizeVector &dims, TypeTo * ptr, size_t size) function
|
||||
* InferenceEngine::make_shared_blob(Precision p, const SizeVector &dims, TypeTo * ptr, size_t size) function
|
||||
* InferenceEngine::I_N variable
|
||||
* InferenceEngine::I_C variable
|
||||
* InferenceEngine::I_H variable
|
||||
* InferenceEngine::I_W variable
|
||||
* InferenceEngine::LayoutOffsetCounter class
|
||||
* InferenceEngine::ConvertLayout function
|
||||
|
||||
**API working with device enumeration:**
|
||||
|
||||
* InferenceEngine::TargetDevice enumeration
|
||||
* InferenceEngine::TargetDeviceInfo class
|
||||
* InferenceEngine::getDeviceName function
|
||||
* InferenceEngine::FindPluginRequest class
|
||||
* InferenceEngine::FindPluginResponse class
|
||||
* InferenceEngine::findPlugin(const FindPluginRequest &req, FindPluginResponse &result, ResponseDesc *resp) function
|
||||
* InferenceEngine::ICNNNetwork::setTargetDevice method
|
||||
* InferenceEngine::ICNNNetwork::getTargetDevice method
|
||||
* InferenceEngine::PluginDispatcher::getPluginByDevice method
|
||||
* InferenceEngine::PluginDispatcher::getSuitablePlugin method
|
||||
|
||||
@@ -28,7 +28,7 @@ An implementation constructor checks parameters of an nGraph operation, stores r
|
||||
|
||||
### `getSupportedConfigurations`
|
||||
|
||||
The InferenceEngine::ILayerExecImpl::getSupportedConfigurations method returns all supported configuration formats (input/output tensor layouts) for your implementation. To specify formats of data, use InferenceEngine::TensorDesc. Refer to the [Memory Primitives](../Memory_primitives.md) section for instructions.
|
||||
The InferenceEngine::ILayerExecImpl::getSupportedConfigurations method returns all supported configuration formats (input/output tensor layouts) for your implementation. To specify formats of data, use InferenceEngine::TensorDesc.
|
||||
|
||||
@snippet template_extension/old/cpu_kernel.cpp cpu_implementation:getSupportedConfigurations
|
||||
|
||||
|
||||
@@ -80,8 +80,6 @@ Optionally, configure input and output of the model using the steps below:
|
||||
|
||||
auto network = core.ReadNetwork("model.onnx");
|
||||
|
||||
You can find more information about the ONNX format support in the document `ONNX format support in the OpenVINO™ <https://docs.openvino.ai/latest/openvino_docs_IE_DG_ONNX_Support.html>`_
|
||||
|
||||
.. tab:: nGraph
|
||||
|
||||
.. code-block:: c
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
# Known Issues and Limitations {#openvino_docs_IE_DG_Known_Issues_Limitations}
|
||||
|
||||
## Multiple OpenMP Loadings
|
||||
|
||||
If the application uses the Inference Engine with third-party components that depend on Intel OpenMP, multiple loadings of the libiomp library may occur and cause OpenMP runtime initialization conflicts. This may happen, for example, if the application uses Intel® Math Kernel Library (Intel® MKL) through the “Single Dynamic Library” (<code>libmkl_rt.so</code>) mechanism and calls Intel MKL after loading the Inference Engine plugin.
|
||||
The error log looks like this:
|
||||
|
||||
```sh
|
||||
OMP: Error #15: Initializing libiomp5.so, but found libiomp5.so already initialized.
|
||||
OMP: Hint: This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
|
||||
```
|
||||
|
||||
Possible workarounds:
|
||||
|
||||
* Preload the OpenMP runtime using the <code>LD_PRELOAD</code> variable:
|
||||
```sh
|
||||
LD_PRELOAD=<path_to_libiomp5.so> <path_to your_executable>
|
||||
```
|
||||
This eliminates multiple loadings of libiomp, and makes all the components use this specific version of OpenMP.
|
||||
|
||||
* Alternatively, you can set <code>KMP_DUPLICATE_LIB_OK=TRUE</code>. However, performance degradation or incorrect results may occur in this case.
|
||||
|
||||
|
||||
## Old proto compiler breaks protobuf library
|
||||
|
||||
With python protobuf library version 3.5.1, the following incompatibility can happen.
|
||||
The known case is for Cent OS 7.4.
|
||||
|
||||
The error log looks like this:
|
||||
|
||||
```sh
|
||||
File "../lib64/python3.5/site-packages/google/protobuf/descriptor.py", line 829, in _new_
|
||||
return _message.default_pool.AddSerializedFile(serialized_pb)
|
||||
TypeError: expected bytes, str found
|
||||
```
|
||||
|
||||
A possible workaround is to upgrade default protobuf compiler (libprotoc 2.5.0) to newer version, for example libprotoc 2.6.1.
|
||||
|
||||
[protobuf_issue]: https://github.com/google/protobuf/issues/4272
|
||||
|
||||
## Dynamic batching
|
||||
Refer to the **Limitations** section of the [Dynamic batching page](DynamicBatching.md).
|
||||
|
||||
## Static Shape Infer
|
||||
Refer to the **Limitations** section of the [Static Shape Infer page](ShapeInference.md).
|
||||
|
||||
|
||||
## Image Pre-Processing Performance Optimization Issue
|
||||
|
||||
As described in [documentation for the new API](Integrate_with_customer_application_new_API.md), you can set an image blob of any size to an
|
||||
infer request using resizable input. Resize is executed during inference using the configured resize algorithm.
|
||||
|
||||
But currently, resize algorithms are not completely optimized. So expect performance degradation if resizable input is
|
||||
specified and an input blob (to be resized) is set using `SetBlob()`. The best performance is for the
|
||||
[CPU](supported_plugins/CPU.md) plugin only (because enabled openMP* provides parallelism).
|
||||
|
||||
Another limitation is that currently, resize algorithms support NCHW layout only. So if you set NHWC layout for an input
|
||||
blob, NHWC is converted to NCHW before resize and back to NHWC after resize.
|
||||
@@ -1,60 +0,0 @@
|
||||
# Inference Engine Memory Primitives {#openvino_docs_IE_DG_Memory_primitives}
|
||||
|
||||
## Inference Memory Primitives (C++)
|
||||
|
||||
@sphinxdirective
|
||||
.. raw:: html
|
||||
|
||||
<div id="switcher-cpp" class="switcher-anchor">C++</div>
|
||||
@endsphinxdirective
|
||||
|
||||
## Blobs
|
||||
|
||||
<code>InferenceEngine::Blob</code> is the main class intended for working with memory.
|
||||
Using this class you can read and write memory, get information about the memory structure etc.
|
||||
|
||||
The right way to create <code>Blob</code> objects with a specific layout is to use constructors with <code>InferenceEngine::TensorDesc</code>.
|
||||
<pre class="brush:cpp">
|
||||
InferenceEngine::TensorDesc tdesc(FP32, {1, 3, 227, 227}, InferenceEngine::Layout::NCHW);
|
||||
InferenceEngine::Blob::Ptr blob = InferenceEngine::make_shared_blob<float>(tdesc);
|
||||
</pre>
|
||||
|
||||
## Layouts
|
||||
|
||||
<code>InferenceEngine::TensorDesc</code> is a special class that provides layout format description.
|
||||
|
||||
This class allows to create planar layouts using the standard formats (like <code>InferenceEngine::Layout::NCDHW</code>, <code>InferenceEngine::Layout::NCHW</code>, <code>InferenceEngine::Layout::NC</code>, <code>InferenceEngine::Layout::C</code> and etc) and also non-planar layouts using <code>InferenceEngine::BlockingDesc</code>.
|
||||
|
||||
In order to create a complex layout you should use <code>InferenceEngine::BlockingDesc</code>, which allows you to define the blocked memory with offsets and strides.
|
||||
|
||||
## Examples
|
||||
|
||||
1. You can define a blob with dimensions {N: 1, C: 25, H: 20, W: 20} and format NHWC with using next parameters:<br/>
|
||||
<pre class="brush:cpp">
|
||||
InferenceEngine::BlockingDesc({1, 20, 20, 25}, {0, 2, 3, 1}); // or
|
||||
InferenceEngine::BlockingDesc({1, 20, 20, 25}, InferenceEngine::Layout::NHWC);
|
||||
</pre>
|
||||
2. If you have a memory with real dimensions {N: 1, C: 25, H: 20, W: 20} but with channels that are blocked by 8, you can define it using next parameters:<br/>
|
||||
<pre class="brush:cpp">
|
||||
InferenceEngine::BlockingDesc({1, 4, 20, 20, 8}, {0, 1, 2, 3, 1})
|
||||
</pre>
|
||||
3. Also you can set strides and offsets if layout contains it.
|
||||
4. If you have a complex blob layout and you don't want to calculate the real offset to data you can use the <code>InferenceEngine::TensorDesc::offset(size_t l)</code> or <code>InferenceEngine::TensorDesc::offset(SizeVector v)</code> methods.<br/>
|
||||
For example:
|
||||
<pre class="brush:cpp">
|
||||
InferenceEngine::BlockingDesc blk({1, 4, 20, 20, 8}, {0, 1, 2, 3, 1});
|
||||
InferenceEngine::TensorDesc tdesc(FP32, {1, 25, 20, 20}, blk);
|
||||
tdesc.offset(0); // = 0
|
||||
tdesc.offset(1); // = 8
|
||||
tdesc.offset({0, 0, 0, 2}); // = 16
|
||||
tdesc.offset({0, 1, 0, 2}); // = 17
|
||||
</pre>
|
||||
5. If you would like to create a TensorDesc with a planar format and for N dimensions (N can be different 1, 2, 4 and etc), you can use the <code>InferenceEngine::TensorDesc::getLayoutByDims</code> method.
|
||||
<pre class="brush:cpp">
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1}); // InferenceEngine::Layout::C
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1, 2}); // InferenceEngine::Layout::NC
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1, 2, 3, 4}); // InferenceEngine::Layout::NCHW
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1, 2, 3}); // InferenceEngine::Layout::BLOCKED
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1, 2, 3, 4, 5}); // InferenceEngine::Layout::NCDHW
|
||||
InferenceEngine::TensorDesc::getLayoutByDims({1, 2, 3, 4, 5, ...}); // InferenceEngine::Layout::BLOCKED
|
||||
</pre>
|
||||
@@ -8,7 +8,7 @@
|
||||
<div id="switcher-cpp" class="switcher-anchor">C++</div>
|
||||
@endsphinxdirective
|
||||
|
||||
As described in the [Inference Engine Developer Guide](Deep_Learning_Inference_Engine_DevGuide.md), a common application flow consists of the following steps:
|
||||
As described in the [OpenVINO™ Runtime User Guide](OpenVINO_Runtime_User_Guide.md), a common application flow consists of the following steps:
|
||||
|
||||
1. **Create an Inference Engine Core object**: First step to manage available devices and read network objects
|
||||
|
||||
|
||||
@@ -1,91 +0,0 @@
|
||||
# ONNX Format Support {#openvino_docs_IE_DG_ONNX_Support}
|
||||
|
||||
## Introduction (C++)
|
||||
|
||||
@sphinxdirective
|
||||
.. raw:: html
|
||||
|
||||
<div id="switcher-cpp" class="switcher-anchor">C++</div>
|
||||
@endsphinxdirective
|
||||
|
||||
Starting with the 2020.4 release, OpenVINO™ supports reading native ONNX models. The `Core::ReadNetwork()` method provides a uniform way to read models from IR or ONNX format, it is a recommended approach to reading models. Example:
|
||||
|
||||
```cpp
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("model.onnx");
|
||||
```
|
||||
|
||||
### Reshape Feature
|
||||
OpenVINO™ does not provide a mechanism to specify pre-processing (like mean values subtraction, reverse input channels) for the ONNX format. If an ONNX model contains dynamic shapes for input, please use the `CNNNetwork::reshape` method to reshape the model.
|
||||
|
||||
### Weights Saved in External Files
|
||||
|
||||
OpenVINO™ supports ONNX models that store weights in external files. It is especially useful for models larger than 2GB because of protobuf limitations. To read such models, use the `ReadNetwork` overload which takes `modelPath` as input parameter (both `std::string` and `std::wstring`). Note that the `binPath` argument of `ReadNetwork` should be empty in this case, because paths to external weights are saved directly in an ONNX model.
|
||||
Otherwise, a runtime exception is thrown. Reading models with external weights is not supported by the `ReadNetwork(const std::string& model, const Blob::CPtr& weights)` overload.
|
||||
|
||||
Paths to external weight files are saved in an ONNX model; these paths are relative to the model's directory path.
|
||||
It means that if a model is located at `home/user/workspace/models/model.onnx` and a file that contains external weights is in `home/user/workspace/models/data/weights.bin`, then the path saved in the model should be:
|
||||
`data/weights.bin`
|
||||
|
||||
> **NOTE**: A single model can use many external weights files.
|
||||
|
||||
> **NOTE**: Data of many tensors can be stored in a single external weights file (it is processed using offset and length values, which can be also saved in a model).
|
||||
|
||||
The described mechanism is the only way to read weights from external files. The following input parameters of the `ReadNetwork` function overloads are NOT supported for ONNX models and should be passed as empty:
|
||||
* `const std::wstring& binPath`
|
||||
* `const std::string& binPath`
|
||||
* `const Blob::CPtr& weights`
|
||||
|
||||
You can find more details about the external data mechanism in [ONNX documentation](https://github.com/onnx/onnx/blob/master/docs/ExternalData.md).
|
||||
To convert a model to use the external data feature, you can use [ONNX helper functions](https://github.com/onnx/onnx/blob/master/onnx/external_data_helper.py).
|
||||
|
||||
Unsupported types of tensors:
|
||||
* string
|
||||
* complex64
|
||||
* complex128
|
||||
|
||||
## Introduction (Python)
|
||||
|
||||
@sphinxdirective
|
||||
.. raw:: html
|
||||
|
||||
<div id="switcher-python" class="switcher-anchor">Python</div>
|
||||
@endsphinxdirective
|
||||
|
||||
Starting with the 2020.4 release, OpenVINO™ supports reading native ONNX models. The `IECore.read_network()` method provides a uniform way to read models from IR or ONNX format, it is a recommended approach to reading models. Example:
|
||||
|
||||
```python
|
||||
from openvino.inference_engine import IECore
|
||||
|
||||
ie = IECore()
|
||||
net = ie.read_network(model=path_to_onnx_file)
|
||||
```
|
||||
|
||||
### Reshape Feature
|
||||
OpenVINO™ does not provide a mechanism to specify pre-processing (like mean values subtraction, reverse input channels) for the ONNX format. If an ONNX model contains dynamic shapes for input, please use the [IENetwork.reshape](api/ie_python_api/_autosummary/openvino.inference_engine.IENetwork.html#openvino.inference_engine.IENetwork.reshape) method to reshape the model.
|
||||
|
||||
```python
|
||||
from openvino.inference_engine import IECore
|
||||
|
||||
ie = IECore()
|
||||
net = ie.read_network(model=path_to_onnx_file)
|
||||
input_layer = next(iter(net.input_info))
|
||||
net.reshape({input_layer: new_shape})
|
||||
```
|
||||
|
||||
### Weights Saved in External Files
|
||||
|
||||
OpenVINO™ supports ONNX models that store weights in external files. It is especially useful for models larger than 2GB because of protobuf limitations. To read such models, use the `model` parameter in the `IECore.read_network(model=path_to_onnx_file)` method. Note that the parameter for the path to the binary weight file, `weights=` should be empty in this case, because paths to external weights are saved directly in an ONNX model. Otherwise, a runtime exception is thrown. Reading models with external weights is **NOT** supported by the `read_network(weights=path_to_bin_file)` parameter.
|
||||
|
||||
Paths to external weight files are saved in an ONNX model; these paths are relative to the model’s directory path. It means that if a model is located at: `$HOME/workspace/models/model.onnx` and a file that contains external weights: `$HOME/workspace/models/data/weights.bin`, the path saved in model should be: data/weights.bin.
|
||||
|
||||
**NOTE**:
|
||||
* A single model can use many external weights files.
|
||||
* Data of many tensors can be stored in a single external weights file (it is processed using offset and length values, which can be also saved in a model).
|
||||
|
||||
The described mechanism is the only possibility to read weights from external files. The `weights` input parameter of the [IECore.read_network](api/ie_python_api/_autosummary/openvino.inference_engine.IECore.html#openvino.inference_engine.IECore.read_network) function is NOT supported for ONNX models and should not be passed, or set as None.
|
||||
|
||||
Unsupported types of tensors:
|
||||
* string
|
||||
* complex64
|
||||
* complex128
|
||||
@@ -1,4 +1,4 @@
|
||||
# OpenVINO™ Runtime User Guide {#openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide}
|
||||
# OpenVINO™ Runtime User Guide {#openvino_docs_OV_Runtime_User_Guide}
|
||||
|
||||
@sphinxdirective
|
||||
|
||||
@@ -7,27 +7,22 @@
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:hidden:
|
||||
|
||||
|
||||
openvino_2_0_transition_guide
|
||||
openvino_docs_IE_DG_Integrate_with_customer_application_new_API
|
||||
openvino_docs_OV_Runtime_UG_Model_Representation
|
||||
ngraph_transformation
|
||||
openvino_docs_deployment_optimization_guide_dldt_optimization_guide
|
||||
openvino_docs_IE_DG_Device_Plugins
|
||||
Direct ONNX Format Support <openvino_docs_IE_DG_ONNX_Support>
|
||||
openvino_docs_IE_DG_Paddle_Support
|
||||
openvino_docs_IE_DG_Int8Inference
|
||||
openvino_docs_IE_DG_Bfloat16Inference
|
||||
openvino_docs_IE_DG_DynamicBatching
|
||||
openvino_docs_IE_DG_ShapeInference
|
||||
openvino_docs_IE_DG_Model_caching_overview
|
||||
openvino_docs_IE_DG_Extensibility_DG_Intro
|
||||
openvino_docs_IE_DG_Memory_primitives
|
||||
openvino_docs_IE_DG_network_state_intro
|
||||
openvino_docs_IE_DG_API_Changes
|
||||
openvino_docs_IE_DG_Known_Issues_Limitations
|
||||
openvino_docs_IE_DG_Glossary
|
||||
|
||||
openvino_docs_OV_Runtime_API_Changes
|
||||
|
||||
@endsphinxdirective
|
||||
|
||||
## Introduction
|
||||
@@ -1,52 +0,0 @@
|
||||
# Paddle Support in OpenVINO™ {#openvino_docs_IE_DG_Paddle_Support}
|
||||
|
||||
Starting from the 2022.1 release, OpenVINO™ supports reading native Paddle models.
|
||||
The `Core::ReadNetwork()` method provides a uniform way to read models from either the Paddle format or IR, which is the recommended approach.
|
||||
|
||||
## Read Paddle Models from IR
|
||||
|
||||
The Paddle Model can be read after it is [converted](../MO_DG/prepare_model/convert_model/Convert_Model_From_Paddle.md) to [Intermediate Representation (IR)](../MO_DG/IR_and_opsets.md).
|
||||
|
||||
**C++ Example:**
|
||||
|
||||
```cpp
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("model.xml");
|
||||
```
|
||||
|
||||
**Python Example:**
|
||||
|
||||
```sh
|
||||
from openvino.inference_engine import IECore
|
||||
ie = IECore()
|
||||
net = ie.read_network("model.xml")
|
||||
```
|
||||
|
||||
## Read Paddle Models from The Paddle Format (Paddle `inference model` model type)
|
||||
|
||||
**C++ Example:**
|
||||
|
||||
```cpp
|
||||
InferenceEngine::Core core;
|
||||
auto network = core.ReadNetwork("model.pdmodel");
|
||||
```
|
||||
|
||||
**Python Example:**
|
||||
|
||||
```sh
|
||||
from openvino.inference_engine import IECore
|
||||
ie = IECore()
|
||||
net = ie.read_network("model.pdmodel")
|
||||
```
|
||||
|
||||
**The Reshape feature:**
|
||||
|
||||
OpenVINO™ does not provide a mechanism to specify pre-processing, such as mean values subtraction or reverse input channels, for the Paddle format.
|
||||
If a Paddle model contains dynamic shapes for input, use the `CNNNetwork::reshape` method for shape specialization.
|
||||
|
||||
## NOTES
|
||||
|
||||
* The Paddle [`inference model`](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.1/doc/doc_en/inference_en.md) mainly contains two kinds of files `model.pdmodel`(model file) and `model.pdiparams`(params file), which are used for inference.
|
||||
* The list of supported Paddle models and a description of how to export them can be found in [Convert a Paddle Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Paddle.md). The following Paddle models are supported by intel CPU only: `Fast-SCNN`, `Yolo v3`, `ppyolo`, `MobileNetv3-SSD`, `BERT`.
|
||||
* For `Normalize` Paddle Models, the input data should be in FP32 format.
|
||||
* When reading Paddle models from The Paddle format, make sure that `model.pdmodel` and `model.pdiparams` are in the same folder directory.
|
||||
@@ -1,14 +0,0 @@
|
||||
# OpenVINO™ Python* Package
|
||||
|
||||
OpenVINO™ Python\* package includes types to measure model and calibrate to low precision.
|
||||
|
||||
The OpenVINO™ Python\* package available in the `<INSTALL_DIR>/python/python3.X` directory.
|
||||
|
||||
The OpenVINO™ Python\* package includes the following sub-packages:
|
||||
|
||||
- [openvino.inference_engine](../../src/bindings/python/docs/api_overview.md) - Python\* wrapper on OpenVINO™ Inference Engine.
|
||||
- `openvino.tools.accuracy_checker` - Measure accuracy.
|
||||
- `openvino.tools.benchmark` - Measure latency and throughput.
|
||||
|
||||
## See Also
|
||||
* [Integrate with Customer Application New API](Integrate_with_customer_application_new_API.md)
|
||||
@@ -270,4 +270,4 @@ sample, read the sample documentation by clicking the sample name in the samples
|
||||
list above.
|
||||
|
||||
## See Also
|
||||
* [Inference Engine Developer Guide](Deep_Learning_Inference_Engine_DevGuide.md)
|
||||
* [OpenVINO™ Runtime User Guide](OpenVINO_Runtime_User_Guide.md)
|
||||
|
||||
@@ -43,8 +43,7 @@ If a model has a hard-coded batch dimension, use `InferenceEngine::CNNNetwork::s
|
||||
|
||||
Inference Engine takes three kinds of a model description as an input, which are converted into an `InferenceEngine::CNNNetwork` object:
|
||||
1. [Intermediate Representation (IR)](../MO_DG/IR_and_opsets.md) through `InferenceEngine::Core::ReadNetwork`
|
||||
2. [ONNX model](../OV_Runtime_UG/ONNX_Support.md) through `InferenceEngine::Core::ReadNetwork`
|
||||
3. [OpenVINO Model](../OV_Runtime_UG/model_representation.md) through the constructor of `InferenceEngine::CNNNetwork`
|
||||
2. [OpenVINO Model](../OV_Runtime_UG/model_representation.md) through the constructor of `InferenceEngine::CNNNetwork`
|
||||
|
||||
`InferenceEngine::CNNNetwork` keeps an `ngraph::Function` object with the model description internally.
|
||||
The object should have fully-defined input shapes to be successfully loaded to Inference Engine plugins.
|
||||
|
||||
@@ -54,7 +54,7 @@ should be called with `weights` passed as an empty `Blob`.
|
||||
- Intel® Distribution of OpenVINO™ toolkit home page: [https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit)
|
||||
- OpenVINO™ toolkit online documentation: [https://docs.openvino.ai](https://docs.openvino.ai)
|
||||
- Model Optimizer Developer Guide: [Model Optimizer Developer Guide](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
|
||||
- Inference Engine Developer Guide: [Inference Engine Developer Guide](Deep_Learning_Inference_Engine_DevGuide.md)
|
||||
- [OpenVINO™ runTime User Guide](OpenVINO_Runtime_User_Guide.md)
|
||||
- For more information on Sample Applications, see the [Inference Engine Samples Overview](Samples_Overview.md)
|
||||
- For information on a set of pre-trained models, see the [Overview of OpenVINO™ Toolkit Pre-Trained Models](@ref omz_models_group_intel)
|
||||
- For IoT Libraries and Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
:caption: Deploying Inference
|
||||
:hidden:
|
||||
|
||||
openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide
|
||||
openvino_docs_OV_Runtime_User_Guide
|
||||
openvino_docs_install_guides_deployment_manager_tool
|
||||
openvino_inference_engine_tools_compile_tool_README
|
||||
|
||||
@@ -93,7 +93,7 @@ This section provides reference documents that guide you through developing your
|
||||
With the [Model Downloader](@ref omz_tools_downloader) and [Model Optimizer](MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) guides, you will learn to download pre-trained models and convert them for use with the OpenVINO™ toolkit. You can provide your own model or choose a public or Intel model from a broad selection provided in the [Open Model Zoo](model_zoo.md).
|
||||
|
||||
## Deploying Inference
|
||||
The [OpenVINO™ Runtime User Guide](OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) explains the process of creating your own application that runs inference with the OpenVINO™ toolkit. The [API Reference](./api_references.html) defines the Inference Engine API for Python, C++, and C and the nGraph API for Python and C++. The Inference Engine API is what you'll use to create an OpenVINO™ application, while the nGraph API is available for using enhanced operations sets and other features. After writing your application, you can use the [Deployment Manager](install_guides/deployment-manager-tool.md) for deploying to target devices.
|
||||
The [OpenVINO™ Runtime User Guide](OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) explains the process of creating your own application that runs inference with the OpenVINO™ toolkit. The [API Reference](./api_references.html) defines the Inference Engine API for Python, C++, and C and the nGraph API for Python and C++. The Inference Engine API is what you'll use to create an OpenVINO™ application, while the nGraph API is available for using enhanced operations sets and other features. After writing your application, you can use the [Deployment Manager](install_guides/deployment-manager-tool.md) for deploying to target devices.
|
||||
|
||||
## Tuning for Performance
|
||||
The toolkit provides a [Performance Optimization Guide](optimization_guide/dldt_optimization_guide.md) and utilities for squeezing the best performance out of your application, including [Accuracy Checker](@ref omz_tools_accuracy_checker), [Post-Training Optimization Tool](@ref pot_README), and other tools for measuring accuracy, benchmarking performance, and tuning your application.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Glossary {#openvino_docs_IE_DG_Glossary}
|
||||
# Glossary {#openvino_docs_OV_Glossary}
|
||||
|
||||
## Acronyms and Abbreviations
|
||||
|
||||
@@ -12,7 +12,6 @@
|
||||
| CPU | Central Processing Unit |
|
||||
| CV | Computer Vision |
|
||||
| DL | Deep Learning |
|
||||
| DLDT | Intel(R) Deep Learning Deployment Toolkit |
|
||||
| DLL | Dynamic Link Library |
|
||||
| DNN | Deep Neural Networks |
|
||||
| ELU | Exponential Linear rectification Unit |
|
||||
@@ -21,7 +20,6 @@
|
||||
| GCC | GNU Compiler Collection |
|
||||
| GPU | Graphics Processing Unit |
|
||||
| HD | High Definition |
|
||||
| IE | Inference Engine |
|
||||
| IR | Intermediate Representation |
|
||||
| JIT | Just In Time |
|
||||
| JTAG | Joint Test Action Group |
|
||||
@@ -55,31 +53,26 @@
|
||||
|
||||
## Terms
|
||||
|
||||
Glossary of terms used in the Inference Engine
|
||||
Glossary of terms used in the OpenVINO™
|
||||
|
||||
|
||||
| Term | Description |
|
||||
| :--- | :--- |
|
||||
| Batch | Number of images to analyze during one call of infer. Maximum batch size is a property of the network and it is set before loading of the network to the plugin. In NHWC, NCHW and NCDHW image data layout representation, the N refers to the number of images in the batch |
|
||||
| Blob | Memory container used for storing inputs, outputs of the network, weights and biases of the layers |
|
||||
| Tensor | Memory container used for storing inputs, outputs of the network, weights and biases of the layers |
|
||||
| Device (Affinitity) | A preferred Intel(R) hardware device to run the inference (CPU, GPU, etc.) |
|
||||
| Extensibility mechanism, Custom layers | The mechanism that provides you with capabilities to extend the Inference Engine and Model Optimizer so that they can work with topologies containing layers that are not yet supported |
|
||||
| <code>CNNNetwork</code> | A class of the Convolutional Neural Network that Inference Engine reads from IR. Consists of topology, weights and biases |
|
||||
| <code>ExecutableNetwork</code> | An instance of the loaded network which allows the Inference Engine to request (several) infer requests and perform inference synchronously or asynchronously |
|
||||
| Extensibility mechanism, Custom layers | The mechanism that provides you with capabilities to extend the OpenVINO™ Runtime and Model Optimizer so that they can work with topologies containing layers that are not yet supported |
|
||||
| <code>ov::Model</code> | A class of the Model that OpenVINO™ Runtime reads from IR. Consists of topology, weights and biases |
|
||||
| <code>ov::CompiledModel</code> | An instance of the loaded network which allows the OpenVINO™ Runtime to request (several) infer requests and perform inference synchronously or asynchronously |
|
||||
| <code>InferRequest</code> | A class that represents the end point of inference on the model loaded to the plugin and represented by executable network. Inputs are set here, outputs should be requested from this interface as well |
|
||||
| <code>InferenceEngineProfileInfo</code> | Represents basic inference profiling information per layer |
|
||||
| Inference Engine | A C++ library with a set of classes that you can use in your application to infer input data (images) and get the result |
|
||||
| Inference Engine API | The basic default API for all supported devices, which allows you to load a model from Intermediate Representation, set input and output formats and execute the model on various devices |
|
||||
| Inference Engine <code>Core</code> | Inference Engine Core is a software component that manages inference on certain Intel(R) hardware devices: CPU, GPU, MYRIAD, GNA, etc. |
|
||||
| Layer catalog or Operations specification | A list of supported layers or operations and its parameters. Sets of supported layers are different for different plugins, please check the documentation on plugins to verify if the Inference Engine supports certain layer on the dedicated hardware |
|
||||
| <code>Layout</code> | Image data layout refers to the representation of images batch. Layout shows a sequence of 4D or 5D tensor data in memory. A typical NCHW format represents pixel in horizontal direction, rows by vertical dimension, planes by channel and images into batch |
|
||||
| <code>OutputsDataMap</code> | Structure which contains information about output precisions and layouts |
|
||||
| Precision | Represents data precision. For example, FP32 is 32-bit floating point, FP16 is 16-bit floating point. Precision can be changed before loading the network to the plugin |
|
||||
| <code>PreProcessInfo</code> | Class that represents input data for the network. It contains information about input precision, its layout, and pre-processing |
|
||||
| <code>ResponseDesc</code> | Represents debug information for an error |
|
||||
| <code>ov::ProfileInfo</code> | Represents basic inference profiling information per layer |
|
||||
| OpenVINO™ Runtime | A C++ library with a set of classes that you can use in your application to infer input data (images) and get the result |
|
||||
| OpenVINO™ API | The basic default API for all supported devices, which allows you to load a model from Intermediate Representation, set input and output formats and execute the model on various devices |
|
||||
| OpenVINO™ <code>Core</code> | OpenVINO™ Core is a software component that manages inference on certain Intel(R) hardware devices: CPU, GPU, MYRIAD, GNA, etc. |
|
||||
| <code>ov::Layout</code> | Image data layout refers to the representation of images batch. Layout shows a sequence of 4D or 5D tensor data in memory. A typical NCHW format represents pixel in horizontal direction, rows by vertical dimension, planes by channel and images into batch |
|
||||
| <code>ov::element::Type</code> | Represents data element type. For example, f32 is 32-bit floating point, f16 is 16-bit floating point. Element type can be changed before loading the network to the plugin |
|
||||
|
||||
|
||||
## See Also
|
||||
* [Deep Learning Model Optimizer IR Operations Catalog](../ops/opset.md)
|
||||
* [Inference Engine Memory primitives](Memory_primitives.md)
|
||||
* [Terminology](supported_plugins/Supported_Devices.md)
|
||||
* [Available Operations Sets](ops/opset.md)
|
||||
* [Terminology](OV_Runtime_UG/supported_plugins/Supported_Devices.md)
|
||||
@@ -72,7 +72,7 @@ OpenVINO™ Documentation
|
||||
<h3>DL Workbench </h3>
|
||||
<p> Learn about the alternative, web-based version of OpenVINO. DL Workbench container installation Required. </p>
|
||||
</a>
|
||||
<a href="openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html" >
|
||||
<a href="openvino_docs_OV_Runtime_User_Guide.html" >
|
||||
<h3>Inference Engine </h3>
|
||||
<p> Learn about OpenVINO's inference mechanism which executes the IR and ONNX models on target devices. </p>
|
||||
</a>
|
||||
|
||||
@@ -21,7 +21,7 @@ The following components are installed with the OpenVINO runtime package:
|
||||
|
||||
| Component | Description|
|
||||
|-----------|------------|
|
||||
| [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md)| The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
| [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md)| The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
| [OpenCV*](https://docs.opencv.org/master/) | OpenCV* community version compiled for Intel® hardware. |
|
||||
| Deep Learning Streamer (DL Streamer) | Streaming analytics framework, based on GStreamer, for constructing graphs of media analytics components. For the DL Streamer documentation, see [DL Streamer Samples](@ref gst_samples_README), [API Reference](https://openvinotoolkit.github.io/dlstreamer_gst/), [Elements](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/Elements), [Tutorial](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/DL-Streamer-Tutorial). |
|
||||
|
||||
@@ -32,7 +32,7 @@ The following components are installed with the OpenVINO developer package:
|
||||
| Component | Description|
|
||||
|-----------|------------|
|
||||
| [Model Optimizer](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md) | This tool imports, converts, and optimizes models that were trained in popular frameworks to a format usable by Intel tools, especially the Inference Engine. <br>Popular frameworks include Caffe\*, TensorFlow\*, MXNet\*, and ONNX\*. |
|
||||
| [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) | The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications.|
|
||||
| [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) | The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications.|
|
||||
| [OpenCV*](https://docs.opencv.org/master/) | OpenCV\* community version compiled for Intel® hardware |
|
||||
| [Sample Applications](../OV_Runtime_UG/Samples_Overview.md) | A set of simple console applications demonstrating how to use the Inference Engine in your applications. |
|
||||
| [Demo Applications](@ref omz_demos) | A set of console applications that demonstrate how you can use the Inference Engine in your applications to solve specific use cases. |
|
||||
@@ -158,7 +158,7 @@ sudo apt autoremove intel-openvino-<PACKAGE_TYPE>-ubuntu<OS_VERSION>-<VERSION>.<
|
||||
- Intel® Distribution of OpenVINO™ toolkit home page: [https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit).
|
||||
- OpenVINO™ toolkit online documentation: [https://docs.openvino.ai](https://docs.openvino.ai).
|
||||
- [Model Optimizer Developer Guide](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
|
||||
- [Inference Engine Developer Guide](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md).
|
||||
- [OpenVINO™ Runtime User Guide](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md).
|
||||
- For more information on Sample Applications, see the [Inference Engine Samples Overview](../OV_Runtime_UG/Samples_Overview.md).
|
||||
- For IoT Libraries & Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ The **runtime package** includes the following components installed by default:
|
||||
|
||||
| Component | Description |
|
||||
|-----------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) | This is the engine that runs the deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
| [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md) | This is the engine that runs the deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
|
||||
## System Requirements
|
||||
|
||||
@@ -81,7 +81,7 @@ Now you can start developing your application.
|
||||
- Intel® Distribution of OpenVINO™ toolkit home page: [https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit).
|
||||
- OpenVINO™ toolkit online documentation: [https://docs.openvino.ai](https://docs.openvino.ai).
|
||||
- [Model Optimizer Developer Guide](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
|
||||
- [Inference Engine Developer Guide](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md).
|
||||
- [OpenVINO™ Runtime User Guide](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md).
|
||||
- For more information on Sample Applications, see the [Inference Engine Samples Overview](../OV_Runtime_UG/Samples_Overview.md).
|
||||
- Intel® Distribution of OpenVINO™ toolkit Anaconda* home page: [https://anaconda.org/intel/openvino-ie4py](https://anaconda.org/intel/openvino-ie4py)
|
||||
|
||||
|
||||
@@ -37,5 +37,5 @@ For system requirements and more detailed steps, see <https://pypi.org/project/o
|
||||
|
||||
- [Intel® Distribution of OpenVINO™ toolkit](https://software.intel.com/en-us/openvino-toolkit)
|
||||
- [Model Optimizer Developer Guide](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md)
|
||||
- [Inference Engine Developer Guide](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md)
|
||||
- [OpenVINO™ Runtime User Guide](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md)
|
||||
- [Inference Engine Samples Overview](../OV_Runtime_UG/Samples_Overview.md)
|
||||
|
||||
@@ -22,7 +22,7 @@ The following components are installed with the OpenVINO runtime package:
|
||||
|
||||
| Component | Description|
|
||||
|-----------|------------|
|
||||
| [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md)| The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
| [OpenVINO™ Runtime](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md)| The engine that runs a deep learning model. It includes a set of libraries for an easy inference integration into your applications. |
|
||||
| [OpenCV*](https://docs.opencv.org/master/) | OpenCV* community version compiled for Intel® hardware. |
|
||||
| Deep Learning Stream (DL Streamer) | Streaming analytics framework, based on GStreamer, for constructing graphs of media analytics components. For the DL Streamer documentation, see [DL Streamer Samples](@ref gst_samples_README), [API Reference](https://openvinotoolkit.github.io/dlstreamer_gst/), [Elements](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/Elements), [Tutorial](https://github.com/openvinotoolkit/dlstreamer_gst/wiki/DL-Streamer-Tutorial). |
|
||||
|
||||
@@ -138,7 +138,7 @@ sudo yum autoremove intel-openvino-runtime-centos<OS_VERSION>-<VERSION>.<UPDATE>
|
||||
- Intel® Distribution of OpenVINO™ toolkit home page: [https://software.intel.com/en-us/openvino-toolkit](https://software.intel.com/en-us/openvino-toolkit)
|
||||
- OpenVINO™ toolkit online documentation: [https://docs.openvino.ai](https://docs.openvino.ai)
|
||||
- [Model Optimizer Developer Guide](../MO_DG/Deep_Learning_Model_Optimizer_DevGuide.md).
|
||||
- [Inference Engine Developer Guide](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md).
|
||||
- [OpenVINO™ Runtime User Guide](../OV_Runtime_UG/OpenVINO_Runtime_User_Guide.md).
|
||||
- For more information on Sample Applications, see the [Inference Engine Samples Overview](../OV_Runtime_UG/Samples_Overview.md).
|
||||
- For IoT Libraries & Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
OpenVINO™ toolkit is a comprehensive toolkit for quickly developing applications and solutions that solve a variety of tasks including emulation of human vision, automatic speech recognition, natural language processing, recommendation systems, and many others. Based on latest generations of artificial neural networks, including Convolutional Neural Networks (CNNs), recurrent and attention-based networks, the toolkit extends computer vision and non-vision workloads across Intel® hardware, maximizing performance. It accelerates applications with high-performance, AI and deep learning inference deployed from edge to cloud.
|
||||
|
||||
[OpenVINO™ Runtime](https://docs.openvino.ai/latest/openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide.html) package for Python includes a set of libraries for an easy inference integration into your Python applications and supports of heterogeneous execution across Intel® CPU and Intel® GPU hardware.
|
||||
[OpenVINO™ Runtime](https://docs.openvino.ai/latest/openvino_docs_OV_Runtime_User_Guide.html) package for Python includes a set of libraries for an easy inference integration into your Python applications and supports of heterogeneous execution across Intel® CPU and Intel® GPU hardware.
|
||||
|
||||
## System Requirements
|
||||
The complete list of supported hardware is available in the [Release Notes](https://www.intel.com/content/www/us/en/developer/articles/release-notes/openvino-relnotes.html).
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
Release Notes <https://software.intel.com/content/www/us/en/develop/articles/openvino-relnotes.html>
|
||||
openvino_docs_IE_DG_supported_plugins_Supported_Devices
|
||||
openvino_docs_IE_DG_Glossary
|
||||
openvino_docs_OV_Glossary
|
||||
openvino_docs_Legal_Information
|
||||
|
||||
|
||||
@@ -48,7 +48,5 @@ This section includes a variety of reference information in three broad categori
|
||||
|
||||
[Broadcast Rules for Elementwise Operations](ops/broadcast_rules.md) explains the rules used for to support an arbitrary number of dimensions in neural nets.
|
||||
|
||||
[Operation Specifications](OV_Runtime_UG/Operations_specifications.md) is a detailed reference of supported operations.
|
||||
|
||||
### Case Studies
|
||||
Links to [articles](https://www.intel.com/openvino-success-stories) about real-world examples of OpenVINO™ usage.
|
||||
|
||||
@@ -39,8 +39,7 @@ The diagram below shows the optimization flow for the new model with OpenVINO an
|
||||
|
||||
|
||||
|
||||
- **Step 0: Model enabling**. In this step we should ensure that the model trained on the target dataset can be successfully inferred with OpenVINO
|
||||
[Inference Engine](@ref openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide) in floating-point precision.
|
||||
- **Step 0: Model enabling**. In this step we should ensure that the model trained on the target dataset can be successfully inferred with [OpenVINO™ Runtime](@ref openvino_docs_OV_Runtime_User_Guide) in floating-point precision.
|
||||
This process involves use of [Model Optimizer](@ref openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide) tool to convert the model from the source framework
|
||||
to the OpenVINO Intermediate Representation (IR) and run it on CPU with Inference Engine.
|
||||
> **NOTE**: This step presumes that the model has the same accuracy as in the original training framework and enabled in the [Accuracy Checker](@ref omz_tools_accuracy_checker) tool or through the custom validation sample.
|
||||
@@ -49,4 +48,4 @@ Post-training quantization is also really fast and usually takes several minutes
|
||||
For more information on best practices of post-training optimization please refer to the [Post-training Optimization Best practices](BestPractices.md).
|
||||
- **Step2: Quantization-Aware Training**: If the accuracy of the quantized model does not satisfy accuracy criteria, there is step two which implies QAT using [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) for [PyTorch*](https://pytorch.org/) and [TensorFlow*](https://www.tensorflow.org/) models.
|
||||
At this step, we assume the user has an original training pipeline of the model written on TensorFlow or PyTorch and NNCF is integrated into it.
|
||||
After this step, you can get an accurate optimized model that can be converted to OpenVINO Intermediate Representation (IR) using Model Optimizer component and inferred with OpenVINO Inference Engine.
|
||||
After this step, you can get an accurate optimized model that can be converted to OpenVINO Intermediate Representation (IR) using Model Optimizer component and inferred with OpenVINO Inference Engine.
|
||||
|
||||
@@ -52,7 +52,7 @@ The POT Python* API for model optimization can be used in the following cases:
|
||||
- [Accuracy Checker](@ref omz_tools_accuracy_checker) tool does not support the model or dataset.
|
||||
- POT does not support the model in the [Simplified Mode](@ref pot_docs_BestPractices) or produces the optimized model with low
|
||||
accuracy in this mode.
|
||||
- You already have the Python* script to validate the accuracy of the model using the [OpenVINO™ Inference Engine](@ref openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide).
|
||||
- You already have the Python* script to validate the accuracy of the model using the [OpenVINO™ Runtime](@ref openvino_docs_OV_Runtime_User_Guide).
|
||||
|
||||
## API Description
|
||||
|
||||
@@ -362,7 +362,7 @@ Before running the optimization tool it's highly recommended to make sure that
|
||||
|
||||
As was described above, `DataLoader`, `Metric` and `Engine` interfaces should be implemented in order to create
|
||||
the custom optimization pipeline for your model. There might be a case you have the Python* validation script for your
|
||||
model using the [OpenVINO™ Inference Engine](@ref openvino_docs_IE_DG_Deep_Learning_Inference_Engine_DevGuide),
|
||||
model using the [OpenVINO™ Runtime](@ref openvino_docs_OV_Runtime_User_Guide),
|
||||
which in practice includes loading a dataset, model inference, and calculating the accuracy metric.
|
||||
So you just need to wrap the existing functions of your validation script in `DataLoader`, `Metric` and `Engine` interfaces.
|
||||
In another case, you need to implement interfaces from scratch.
|
||||
|
||||
Reference in New Issue
Block a user