* Initial commit. Need to remove debug code
* Remove extra flags. Fix comparation in the matchers
* Fix small issue with the default args
* Update eltwise.hpp
* Update ov_subgraph.cpp
* Optimized any compilation time
* Fixed Any compilation time
* any::addressof
* reverted
* Fixed read write
* format fix
* Fixed build
* format fix
* Moved any tests back
* removed inline
* fix format
* used static inline
* format fix
* removed inline static
* fixed merge confkicts
After enabling deconv b32 onednn, colorization-siggraph f16 b32 has regresison,
Fix it. Add to check sum post ops in case deconv onednn.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Added info on DockerHub CI Framework
* Feature/azaytsev/change layout (#3295)
* Changes according to feedback comments
* Replaced @ref's with html links
* Fixed links, added a title page for installing from repos and images, fixed formatting issues
* Added links
* minor fix
* Added DL Streamer to the list of components installed by default
* Link fixes
* Link fixes
* ovms doc fix (#2988)
* added OpenVINO Model Server
* ovms doc fixes
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* Updated openvino_docs.xml
* Updated the link to software license agreements
* Revert "Updated the link to software license agreements"
This reverts commit 706dac500e.
* Removed the Intel logo
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
- Replace find with compare func to avoid dumping all layers that contain layer name
Signed-off-by: Andrew Kwangwoong Park <andrew.kwangwoong.park@intel.com>
When post-op has pattern like below, binary_mul was ignored previously.
1. binary_add
2. eltwise_linear
3. binary_mul
4. binary_add
It happens when prev_post_op_idx == 2, cur_post_op_idx == 4.
prev_post_op_idx was supposed to proceed to idx 3, but it did not.
* Use tensor names instead of friendly names, handle one output tensor to several Result ops case
* fix python tests
* fix python test
* fix incorrect merge
* remove redundant files
* fix variable names generation, fix python test
* Apply review comments
* fix python test
Even though it is not possible to hit into this situation using existing plugins - there is theoretical possibility that some plugin may return 'nullptr' as it is allowed.
So this check shall remain in generic part which should not rely on plugin-specific behavior
* handle 'and' marker in requirements
* Revert "handle 'and' marker in requirements"
This reverts commit 952bb949ca.
* moved importlib-metadata from requirements.txt into setup.py
* Remove fp16 of Convert layer test from skip_tests.config.cpp as it works now
* update repo
* add initial op reference code of TensorIterator with LSTM body function
* add GRU/RNN case in setup
* add all other test cases
* add visitor api test
* remove unnecessary header files
* fix clang-format issue
* fix copyright year and remove ngraph_helper namespace
* rename ti.cpp to tensor_iterator.cpp in core unit test
* apply suggestions
* [CPU] Add transformation that aligns elementwise input ranks
* fix tests - check also aBcd16b format
* add support for fq
* add test for sqr diff
* move to moc transformations
* fix tests
* align only for numpy autobroadcast type
* fix fetching autob from fq
* [CPU] Eltwise tests corrected & callback for CPU removed
* remove transformation callback call
* revert changes to getMKLDNNOutputMemoryFormats
* remove comment
* use single wrap_type
Co-authored-by: Vladislav Golubev <vladislav.golubev@intel.com>
* [DOCS] add minor changes to install guides
[DOCS] add minor changes to install guides
[DOCS] add minor changes to install guides
[DOCS] add minor changes to install guides
[DOCS] add minor changes to install guides
[DOCS] add minor changes to install guides
* [DOCS] add minor changes to install guides
* Fix ONNX's PriorBoxClustered accuracy
If step_heights == 0 and step_heights == 0, but 'step' is 16, then we should treat this as both = 16
* Removed workaround for ONNX frontend
* save work
* save work
* save work
* basic changes with api 2.0
* Support input file mapping and bin files
* Some impovements
* remove mapping support
* Add -ref_layers parameter
* Fix error handler
* Update Readme and remove old parameters
* Fix readme
* remove info about precision
* rename layer to op
* rename blob to tensor
* remove info about shape
* remove unused imports
* Added info on DockerHub CI Framework
* Feature/azaytsev/change layout (#3295)
* Changes according to feedback comments
* Replaced @ref's with html links
* Fixed links, added a title page for installing from repos and images, fixed formatting issues
* Added links
* minor fix
* Added DL Streamer to the list of components installed by default
* Link fixes
* Link fixes
* ovms doc fix (#2988)
* added OpenVINO Model Server
* ovms doc fixes
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* Updated openvino_docs.xml
* Updated the link to software license agreements
* Revert "Updated the link to software license agreements"
This reverts commit 706dac500e.
* Revised dlstreamer documentation
* Minor edits
* Fixed link
* Fix
* Edits after review
* Shorten DL Streamer name in the TOC
* Update documentation.md
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* [GPU] Support batch32 deconv onednn
onednn rls-v2.6-pc2 support deconv batch32,
so remove the batch size limitation.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Update to merge duplicated checking onednn condidton in deconv.
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Update to use is_node_for_onednn func in get_preferred_impl_type
Signed-off-by: hyunback <hyunback.kim@intel.com>
* Delete _extension suffix in file names; add extension.hpp header to include all extensions
* add extension.hpp file to include all extensions
* codestyle
* fixed perf-counters
* explicit auto-batching params that should guarantee the auto-batching is triggered ( to avoid fallback to no-batching when the selected batch1 size is just 1)
* makeConvPoolReluNoReshapes and using that whenever applicable to gaurantee the auto-batching is required (not important for things like plugin/executable-network config tests, but important for the inference-requests)
* getDefaultNGraphFunctionForTheDevice moved to the ov_behavior_test_utils.hpp
* fixed version comparison: for comparsion extracted hashes are used
* shortened 7 -> 11 to match the current version fromat from nightly
* corrected regex, added comparing by minimal hash len
* remove formatTimeMilli from time_utils.cpp
* add traceCallStacks test case
* add traceCallStacks test case in format_test.cpp
* add param:"test" to function TraceCallStacks()
* catch the exception of checkFormat
* add space for try catch
* rollback time_utils.cpp time_utils.hpp and log_utils_format_test.cpp
* modify testcase for log.hpp
* modify testcase from format_s to format_s_d_ld_u_lu2
fix canConvolutionBeTransformed arguments
fix isAsymmetricOnWeights in GPU plugin
added defaultPrecisions in TestTransformationParams
set new default attribute precisions
try to set const default precisions in network_helper.cpp
apply precision_set
[LPT] Default precisions
rebase
remove extra const
used defaultPrecision in tests
fixed SimpleLowPrecisionTransformer default argument
fixed AttributeParameters default argument
added defaultPrecisions in functions
fix assign_and_read_value_transformation tests
fixed wrong defaultPrecisions definition
fixed ConcatWithNeighborsWithConvolutionTransformation tests
remove getDefaultPrecisions
rebase
remove getDefaultPrecisions from gpu plugin
remove getDefaultPrecisions from lpt_mkldnn_plugin.cpp
use predefined member
update mkldnn_plugin.cpp & lpt_mkldnn_plugin.cpp
resolved conversations
make all lambda captures by ref
* Used new config for streams and threads
* Fixed review coments in ba
* format fix
* fixed hello_query_device
* Added STL string io
* fixed tests
* Fixed test
* Fixed build
* fixed format
* Fixed build
* try fix win
* other any io specialization
* Fixed after merge
* renamed streams
* build fixed
* fixed build
* fixed format
* fix for old mac build
* Fixed type of exception
* test fix
* Added info on DockerHub CI Framework
* Feature/azaytsev/change layout (#3295)
* Changes according to feedback comments
* Replaced @ref's with html links
* Fixed links, added a title page for installing from repos and images, fixed formatting issues
* Added links
* minor fix
* Added DL Streamer to the list of components installed by default
* Link fixes
* Link fixes
* ovms doc fix (#2988)
* added OpenVINO Model Server
* ovms doc fixes
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* Updated openvino_docs.xml
* Updated the link to software license agreements
* Revert "Updated the link to software license agreements"
This reverts commit 706dac500e.
* Added POT, replaced IE with OV Runtime
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
- Change the constant value to the maximum work group size
- Add CLK_GLOBAL_MEM_FENCE barrier to synchronize storing result in intermediate buffer
- Add condition to prevent access local array out of range
Signed-off-by: Andrew Kwangwoong Park <andrew.kwangwoong.park@intel.com>
+ Reverted WA for fsv32 format first conv
+ Applied blocked input format bsv8fsv4 & bsv8fsv2 for onednn first conv
+ Implemented onednn usage for first conv of feature size 1
+ Added new weight format ABcd16a4b
+ Bugfix in fetch_weight
+ Updated thirdparty onednn_gpu
+ Known issue : AcdB16a4b is not supported
Signed-off-by: Min, Byungil <byungil.min@intel.com>
Current SmartReshape finds matched to Param->Reshape->Proposal patterns
For FP16 models, there is additional 'Convert' is inserted after 'Parameter'.
It causes transformation is not applied and 'ov::set_batch' or CNNNetwork::set_batch will throw
Proposal1Scales and Proposal4Scales transformations were updated to handle these conditions
* forced split argument dtype to int
* added unit-test
* fixed typo in split_test.py
* set explicitly np.int64 instead of np.int
* use split_length's dtype
* Update LSTMSequence backend_attrs
* Add missed attribute clip
* Update backend_attrs for all *sequence operations
* Add extender for GRUSequence
* Add GRUSequence to custom ops list
* use has_and_set instead if direct acces to attributes
* Automation for preserving rt info in output ports; Update FunctionComparator to compare rt info correctly
* Update LPT tests to use real rt_info attributes, so they can be compared
* Fix tests
* Reshape op pruning support
* Minor reshape fix
* GroupConv reshape extended support
* Comment ir test
* Fix: reshape can only work with constants
* Apply comments
* Fix output shape computing for reshape op
* Fix comment
* fix bug in Serialize (#74447)
add simple serialization test to check pads changes
clang fix
add check and change pads in conv
refactor ov::clone_model
fix
check in test
* fix FrameworkNode and add test
* fix assert in identiry.cpp
* fix clone_nodes
* remove for node and constructor for node_input.cpp
add spaces
add space
* Add sqrt extender
* Update check to not use default infer in infer was set before
* Update comment
* Fix comment
* Remove Sqrt extender
* Remove unnecessary changes
* Add MO implementation of SQRT operation
* Added info on DockerHub CI Framework
* Feature/azaytsev/change layout (#3295)
* Changes according to feedback comments
* Replaced @ref's with html links
* Fixed links, added a title page for installing from repos and images, fixed formatting issues
* Added links
* minor fix
* Added DL Streamer to the list of components installed by default
* Link fixes
* Link fixes
* ovms doc fix (#2988)
* added OpenVINO Model Server
* ovms doc fixes
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* Updated openvino_docs.xml
* Updated the link to software license agreements
* Revert "Updated the link to software license agreements"
This reverts commit 706dac500e.
* Added a link to the omz repo
Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
* [GNA] Add support for non-functional subgraphs
Details:
* Insert copy before the last layer to allow nonfunc subgraphs
Tickets:
57363
* Traverse graph in upstream order
* Add param-reshape-result tests
* Fix insert condition
* review comments
* Added compatibility check of layout with partial shape
E.g. layout "NC" in not compatible with PartialShape{1,3,224,224}
Check is added:
- For parameter set_layout
- For parameter set_partial_shape
- For result set_layout
- Checked also compatibility for all results after 'validate_and_infer_types'
* Fix incorrect tests
* Fix of more incorrect tests
* Removed couple of obsoleted error-handling tests - these are catched now on earlier stages
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
* Any value can fromm inner string
* Fixed review coment
* strict str to value conversion
* fix format
* [VPU] update config header (#9857)
* [VPU] update config header
* Review fixes
* Performance hint config update
* Removal deprecated vpu config stuff
* Review changes
* Rename myriad properties from camelCase to snake_case
* Review changes
* Review fixes
* Removal intel_myriad::common namespace
* OV throughput stream option
* Test fix
* Reverted disable_convert & disable_reorder
* Bugfixes
* Change default value for PerformanceHintNumRequestsOption
* fixed excessive outputs copying (in case when the fallback happened) and updated the test for that (#10110)
* fixed excessive outputs copying (in case when the fallback happened) and updated the test for that
* enum eExecutionFlavor to cover initial state
* Transformations: eltwise and FQ fusings fixes (#10078)
* FQ fusings fixes
* FQ Fusings: added negative test-cases for non-broadcasted constant
* Disable single-image-super-resolution-1032 from MemCheck precommit (#10133)
* add performance hint to time infer
* disable model from memcheck
* Fixed input cut for case when port is not specified. (#10134)
* Fix for android type info comparison
Co-authored-by: Aleksandr Korolev <aleksandr.korolev@intel.com>
Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com>
Co-authored-by: Vladislav Golubev <vladislav.golubev@intel.com>
Co-authored-by: Victor Kuznetsov <victor.kuznetsov@intel.com>
Co-authored-by: Anastasia Popova <anastasia.popova@intel.com>
* Adjust preferred format of resample operation
* Applied review comment
* Not to fix resample layout when there is permute user unless the permute order is rotating
* Check the selected frontend to correspond use_new/legacy_frontend options
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Fix a default case when no frontend is found
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Added op names to tensor names for MaskRCNN replacement transformation. Fixed output layout for MaskRCNN.
* Applied commentes left from PR with tensor names fix.
* Added tests for remove_tensor_names().
* Added checks in emitter.
* Removed debug output.
* Small fix.
* Small fix.
* Depricated Any implicit cast
* Fixed test
* fixed gna build
* Fixed warnings in benchmark_app
* Fixed test build
* ncc exception for PrintTo
* Error mesage in test
* Error mesage in test
* fixed build
* Fixed tensor names setting in InputCut, fixed tensor names losing in AutomlEfficientDet.
* Changed op name adding to tensor names in InputCut for output port case only.
* Remove fp16 of Convert layer test from skip_tests.config.cpp as it works now
* update repo
* fix demension dynamic support bug in mish op reference test
* Remove fp16 of Convert layer test from skip_tests.config.cpp as it works now
* update repo
* add initial op reference test of rnn_sequence
* add op reference test of GRUSequence
* replace input and refOut data to hard coded value
* update copyright year and namespace of Tensor
* rename S_t to sequence_lengths
* Added new correct gna frequency result for Alder Lake
* Update samples/cpp/speech_sample/utils.hpp
Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
* Refactored statistics output with JSON support
* Detailed/average reports are added
* stylefix
* Update samples/cpp/benchmark_app/statistics_report.hpp
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* Linux Fixes
* stylefixes
* data_shape field format is changed
* stylefix
Co-authored-by: Ivan Vikhrev <ivan.vikhrev@intel.com>
* [Python API] Rename configuration API + update tests/tools
* keep old api for compatibility
* add deprecation warnings
* apply comments to query sample
* remove convert to pyobject
* use Any instead of string
* update tests
* update set_property
* fix sample
* update test + try-except for pot
* add docstrings
* fix codestyle for pot
* change order of transformations to work correctly with Convolutions in Kaldi LSTM networks
* removed unneeded changes and add unit tests
* remove comment
* remove changes from memory_offset_adjustment, move all fixes inside add_reshape_transpose_around_conv_pool to avoid new bugs
* removed test for deleted changes
* replace -1 by None
* Calculate value bounds in ReshapeSequenceFusion
* Reashape fusion upper bounds check
* Revert last return to false
* Add transformation unit tests
* Use output node as check param
* Use evaluate helper and remove deprecation macro
* Header update
* Checks refactor and comments
* Update unit tests
* Get element type from node_out
* Moved memory tests to OV API 2.0
* Added configs for OV api 2, updated configs for api 1
* Commented several models in configs (no such models on omz)
* Updated fillTensors
* Fix to get network inputs
* Updated fillTensors and configs
i32 or i64 is used for index_element_type. So it is more convenient to get rid of the condition and stay only with the i32 option.
Tickets:
75748
75747
75029
* Correct Loaders for TensorFlow StridedSlice and Pack operations
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Supress INFO and WARNING messages from TensorFlow
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
* Install libGNAconfig.cmake
* Refactor gnaConfig to correctly find from OV package
* remove ENABLE_INTEL_GNA option from CI
* Apply comments and fix CI
* re-trigger CI (demos issue)
* Enable GNA/samples smoke tests
* rename GNA to GNA_EXT_DIR
* re-trigger CI (mxnet cpu test issue)
* Pick azhogov changes to check CI
* try win wa
* fix win build
* re-trigger onnx
* tests
* disable win samples tests
Co-authored-by: Alexander Zhogov <alexander.zhogov@intel.com>
This toolkit allows developers to deploy pre-trained deep learning models
through a high-level C++ Inference Engine API integrated with application logic.
through a high-level OpenVINO™ Runtime C++ and Python APIs integrated with application logic.
This open source version includes several components: namely [Model Optimizer], [nGraph] and
[Inference Engine], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
This open source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from the [Open Model Zoo], along with 100+ open
source and public models in popular formats such as Caffe\*, TensorFlow\*,
MXNet\* and ONNX\*.
source and public models in popular formats such as TensorFlow, ONNX, PaddlePaddle, MXNet, Caffe, Kaldi.
## Repository components:
* [Inference Engine]
* [nGraph]
## Repository components
* [OpenVINO™ Runtime]
* [Model Optimizer]
* [Post-Training Optimization Tool]
## License
Deep Learning Deployment Toolkit is licensed under [Apache License Version 2.0](LICENSE).
By contributing to the project, you agree to the license and copyright terms therein
# Mute -fsanitize=function Indirect call of a function through a function pointer of the wrong type.
# Sample cases:
# call to function GetAPIVersion through pointer to incorrect function type 'void *(*)()'
# Mute -fsanitize=alignment Use of a misaligned pointer or creation of a misaligned reference. Also sanitizes assume_aligned-like attributes.
# Sample cases:
# VPU_FixedMaxHeapTest.DefaultConstructor test case load of misaligned address 0x62000000187f for type 'const DataType', which requires 4 byte alignment
# Mute -fsanitize=bool Load of a bool value which is neither true nor false.
# Samples cases:
# ie_c_api_version.apiVersion test case load of value 32, which is not a valid value for type 'bool'
# Mute -fsanitize=enum Load of a value of an enumerated type which is not in the range of representable values for that enumerated type.
# Samples cases:
# load of value 4294967295, which is not a valid value for type 'const (anonymous namespace)::onnx::Field'
> **NOTE**: If a device doesn't support a particular operation, an alternative to creating a new operation is to target
> an additional device using the HETERO plugin. The [Heterogeneous Plugin](../IE_DG/supported_plugins/HETERO.md) may be
> an additional device using the HETERO plugin. The [Heterogeneous Plugin](../OV_Runtime_UG/supported_plugins/HETERO.md) may be
> used to run an inference model on multiple devices allowing the unsupported operations on one device to "fallback" to
> run on another device (e.g., CPU) that does support those operations.
@@ -61,20 +61,20 @@ operation. Refer to the "Operation Extractor" section of
## Custom Operations Extensions for the Inference Engine
Inference Engine provides an extension mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
Inference Engine provides an extension mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../OV_Runtime_UG/Extensibility_DG/Intro.md).
Each device plugin includes a library of optimized implementations to execute known operations which must be extended to execute a custom operation. The custom operation extension is implemented according to the target device:
- Custom Operation CPU Extension
- A compiled shared library (`.so` or `.dll`) needed by the CPU Plugin for executing the custom operation
on a CPU. Refer to the [How to Implement Custom CPU Operations](../IE_DG/Extensibility_DG/CPU_Kernel.md) for more
on a CPU. Refer to the [How to Implement Custom CPU Operations](../OV_Runtime_UG/Extensibility_DG/CPU_Kernel.md) for more
details.
- Custom Operation GPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with an operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the [How to Implement Custom GPU Operations](../IE_DG/Extensibility_DG/GPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with an operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the [How to Implement Custom GPU Operations](../OV_Runtime_UG/Extensibility_DG/GPU_Kernel.md) for more details.
- Custom Operation VPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with an operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to [How to Implement Custom Operations for VPU](../IE_DG/Extensibility_DG/VPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with an operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to [How to Implement Custom Operations for VPU](../OV_Runtime_UG/Extensibility_DG/VPU_Kernel.md) for more details.
Also, it is necessary to implement nGraph custom operation according to [Custom nGraph Operation](../IE_DG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
Also, it is necessary to implement nGraph custom operation according to [Custom nGraph Operation](../OV_Runtime_UG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
operation and correctly infer output tensor shape and type.
## Enabling Magnetic Resonance Image Reconstruction Model
@@ -125,7 +125,7 @@ Firstly, open the model in TensorBoard or other TensorFlow* model visualization
batch dimension because the value for the batch dimension is not hardcoded in the model. Model Optimizer need to set all
dynamic dimensions to some specific value to create the IR, therefore specify the command line parameter `-b 1` to set
the batch dimension equal to 1. The actual batch size dimension can be changed at runtime using the Inference Engine API
described in the [Using Shape Inference](../IE_DG/ShapeInference.md). Also refer to the General Conversion Parameters section in [Converting a Model to Intermediate Representation (IR)](../MO_DG/prepare_model/convert_model/Converting_Model.md) and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
described in the [Using Shape Inference](../OV_Runtime_UG/ShapeInference.md). Also refer to the General Conversion Parameters section in [Converting a Model to Intermediate Representation (IR)](../MO_DG/prepare_model/convert_model/Converting_Model.md) and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
for more details and command line parameters used for the model conversion.
```sh
@@ -263,7 +263,7 @@ The sub-graph corresponding to the originally non-supported one is depicted in t
### Inference Engine Extension Implementation
Now it is necessary to implement the extension for the CPU plugin with operation "FFT" introduced previously. The code
below is based on the template extension described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
below is based on the template extension described in [Inference Engine Extensibility Mechanism](../OV_Runtime_UG/Extensibility_DG/Intro.md).
#### CMake Build File
The first step is to create a CMake configuration file which builds the extension. The content of the "CMakeLists.txt"
@@ -284,7 +284,7 @@ in the `fft_op.cpp` file with the following content:
Model Optimizer is a cross-platform command-line tool that facilitates the transition between the training and deployment environment, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
Model Optimizer process assumes you have a network model trained using supported deep learning frameworks: Caffe*, TensorFlow*, Kaldi*, MXNet* or converted to the ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the network, which can be inferred with the [Inference Engine](../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md).
Model Optimizer process assumes you have a network model trained using supported deep learning frameworks: Caffe*, TensorFlow*, Kaldi*, MXNet* or converted to the ONNX* format. Model Optimizer produces an Intermediate Representation (IR) of the network, which can be inferred with the [Inference Engine](../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md).
> **NOTE**: Model Optimizer does not infer models. Model Optimizer is an offline tool that runs before the inference takes place.
@@ -9,7 +9,7 @@ Model Optimizer performs preprocessing to a model. It is possible to optimize th
If, for example, your network assumes the RGB inputs, the Model Optimizer can swap the channels in the first convolution using the `--reverse_input_channels` command line option, so you do not need to convert your inputs to RGB every time you get the BGR image, for example, from OpenCV*.
- **Larger batch size**<br>
Notice that the devices like GPU are doing better with larger batch size. While it is possible to set the batch size in the runtime using the Inference Engine [ShapeInference feature](../../IE_DG/ShapeInference.md).
Notice that the devices like GPU are doing better with larger batch size. While it is possible to set the batch size in the runtime using the Inference Engine [ShapeInference feature](../../OV_Runtime_UG/ShapeInference.md).
- **Resulting IR precision**<br>
The resulting IR precision, for instance, `FP16` or `FP32`, directly affects performance. As CPU now supports `FP16` (while internally upscaling to `FP32` anyway) and because this is the best precision for a GPU target, you may want to always convert models to `FP16`. Notice that this is the only precision that Intel® Movidius™ Myriad™ 2 and Intel® Myriad™ X VPUs support.
@@ -18,7 +18,7 @@ You need to build your performance conclusions on reproducible data. Do the perf
- If the warm-up run does not help or execution time still varies, you can try running a large number of iterations and then average or find a mean of the results.
- For time values that range too much, use geomean.
Refer to the [Inference Engine Samples](../../IE_DG/Samples_Overview.md) for code examples for the performance measurements. Almost every sample, except interactive demos, has a `-ni` option to specify the number of iterations.
Refer to the [Inference Engine Samples](../../OV_Runtime_UG/Samples_Overview.md) for code examples for the performance measurements. Almost every sample, except interactive demos, has a `-ni` option to specify the number of iterations.
## Getting performance numbers using OpenVINO tool
@@ -39,7 +39,7 @@ to execute on the CPU instead.
For example, for the CPU throughput mode from the previous section, you can play with number of streams (`-nstreams` command-line param).
Try different values of the `-nstreams` argument from `1` to a number of CPU cores and find one that provides the best performance. For example, on a 8-core CPU, compare the `-nstreams 1` (which is a latency-oriented scenario) to the `2`, `4` and `8` streams. Notice that `benchmark_app` automatically queries/creates/runs number of requests required to saturate the given number of streams.
Finally, notice that when you don't specify number of streams with `-nstreams`, "AUTO" value for the streams is used, e.g. for the CPU this is [CPU_THROUGHPUT_AUTO](../../IE_DG/supported_plugins/CPU.md). You can spot the actual value behind "AUTO" for your machine in the application output.
Finally, notice that when you don't specify number of streams with `-nstreams`, "AUTO" value for the streams is used, e.g. for the CPU this is [CPU_THROUGHPUT_AUTO](../../OV_Runtime_UG/supported_plugins/CPU.md). You can spot the actual value behind "AUTO" for your machine in the application output.
Notice that the "AUTO" number is not necessarily most optimal, so it is generally recommended to play either with the benchmark_app's "-nstreams" as described above, or via [new Workbench tool](@ref workbench_docs_Workbench_DG_Introduction).This allows you to simplify the app-logic, as you don't need to combine multiple inputs into a batch to achieve good CPU performance.
Instead, it is possible to keep a separate infer request per camera or another source of input and process the requests in parallel using Async API.
@@ -47,7 +47,7 @@ Instead, it is possible to keep a separate infer request per camera or another s
When comparing the Inference Engine performance with the framework or another reference code, make sure that both versions are as similar as possible:
- Wrap exactly the inference execution (refer to the [Inference Engine Samples](../../IE_DG/Samples_Overview.md) for examples).
- Wrap exactly the inference execution (refer to the [Inference Engine Samples](../../OV_Runtime_UG/Samples_Overview.md) for examples).
- Do not include model loading time.
- Ensure the inputs are identical for the Inference Engine and the framework. For example, Caffe\* allows to auto-populate the input with random values. Notice that it might give different performance than on real images.
- Similarly, for correct performance comparison, make sure the access pattern, for example, input layouts, is optimal for Inference Engine (currently, it is NCHW).
@@ -64,7 +64,7 @@ Alternatively, you can gather the raw profiling data that samples report, the se
Almost every sample (inspect command-line options for a specific sample with `-h`) supports a `-pc` command that outputs internal execution breakdown. Refer to the [samples code](../../IE_DG/Samples_Overview.md) for the actual Inference Engine API behind that.
Almost every sample (inspect command-line options for a specific sample with `-h`) supports a `-pc` command that outputs internal execution breakdown. Refer to the [samples code](../../OV_Runtime_UG/Samples_Overview.md) for the actual Inference Engine API behind that.
Below is example of CPU plugin output for a network (since the device is CPU, the layers wall clock `realTime` and the `cpu` time are the same):
@@ -214,7 +214,7 @@ One of the layers in the specified topology might not have inputs or values. Ple
#### 24. What does the message "Part of the nodes was not translated to IE. Stopped" mean? <a name="question-24"></a>
Some of the layers are not supported by the Inference Engine and cannot be translated to an Intermediate Representation. You can extend the Model Optimizer by allowing generation of new types of layers and implement these layers in the dedicated Inference Engine plugins. For more information, refer to the [Custom Layers Guide](../../HOWTO/Custom_Layers_Guide.md) and [Inference Engine Extensibility Mechanism](../../IE_DG/Extensibility_DG/Intro.md)
Some of the layers are not supported by the Inference Engine and cannot be translated to an Intermediate Representation. You can extend the Model Optimizer by allowing generation of new types of layers and implement these layers in the dedicated Inference Engine plugins. For more information, refer to the [Custom Layers Guide](../../HOWTO/Custom_Layers_Guide.md) and [Inference Engine Extensibility Mechanism](../../OV_Runtime_UG/Extensibility_DG/Intro.md)
#### 25. What does the message "While creating an edge from .. to .. : node name is undefined in the graph. Check correctness of the input model" mean? <a name="question-25"></a>
@@ -638,4 +638,4 @@ Starting from the 2022.1 version, the default IR conversion path for ONNX models
Certain features, such as `--extensions` and `--transformations_config`, are not yet fully supported on the new frontends.
For `--extensions`, the new frontends support only paths to shared libraries (.dll and .so). For `--transformations_config`, they support JSON configurations with defined library fields.
Inputs freezing (enabled by `--freeze_placeholder_with_value` or `--input` arguments) is not supported on the new frontends.
The IR conversion falls back to the old path if a user does not select any expected path of conversion explicitly (by `--use_new_frontend` or `--use_legacy_frontend` MO arguments) and unsupported pre-defined scenario is detected on the new frontend path.
The IR conversion falls back to the old path if a user does not select any expected path of conversion explicitly (by `--use_new_frontend` or `--use_legacy_frontend` MO arguments) and unsupported pre-defined scenario is detected on the new frontend path.
# Converting a Caffe* Model {#openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caffe}
@sphinxdirective
.. _convert model caffe:
@endsphinxdirective
A summary of the steps for optimizing and deploying a model that was trained with Caffe\*:
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Caffe\*.
2. [Convert a Caffe\* Model](#Convert_From_Caffe) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../IE_DG/Samples_Overview.md)
4. [Integrate](../../../IE_DG/Samples_Overview.md) the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
@@ -14,8 +16,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Kaldi\*.
2. [Convert a Kaldi\* Model](#Convert_From_Kaldi) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../IE_DG/Samples_Overview.md).
4. [Integrate](../../../IE_DG/Samples_Overview.md) the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
> **NOTE**: The Model Optimizer supports the [nnet1](http://kaldi-asr.org/doc/dnn1.html) and [nnet2](http://kaldi-asr.org/doc/dnn2.html) formats of Kaldi models. Support of the [nnet3](http://kaldi-asr.org/doc/dnn3.html) format is limited.
@@ -15,8 +17,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for MXNet* (MXNet was used to train your model)
2. [Convert a MXNet model](#ConvertMxNet) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../IE_DG/Samples_Overview.md)
4. [Integrate](../../../IE_DG/Samples_Overview.md) the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md)
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment
@@ -4,8 +4,8 @@ A summary of the steps for optimizing and deploying a model trained with Paddle\
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for Paddle\*.
2. [Convert a Paddle\* Model](#Convert_From_Paddle) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases.
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../IE_DG/Samples_Overview.md).
4. [Integrate](../../../IE_DG/Samples_Overview.md) the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
3. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided Inference Engine [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
4. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in your application to deploy the model in the target environment.
@@ -48,8 +48,8 @@ PyTorch* framework is supported through export to ONNX\* format. A summary of th
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for ONNX\*.
2. [Export PyTorch model to ONNX\*](#export-to-onnx).
3. [Convert an ONNX\* model](Convert_Model_From_ONNX.md) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../IE_DG/Samples_Overview.md).
5. [Integrate](../../../IE_DG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
5. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
## Export PyTorch\* Model to ONNX\* Format <a name="export-to-onnx"></a>
@@ -29,8 +31,8 @@ A summary of the steps for optimizing and deploying a model that was trained wit
1. [Configure the Model Optimizer](../../Deep_Learning_Model_Optimizer_DevGuide.md) for TensorFlow\* (TensorFlow was used to train your model).
2. [Freeze the TensorFlow model](#freeze-the-tensorflow-model) if your model is not already frozen or skip this step and use the [instruction](#loading-nonfrozen-models) to a convert a non-frozen model.
3. [Convert a TensorFlow\* model](#Convert_From_TF) to produce an optimized [Intermediate Representation (IR)](../../IR_and_opsets.md) of the model based on the trained network topology, weights, and biases values.
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../IE_DG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../IE_DG/Samples_Overview.md).
5. [Integrate](../../../IE_DG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
4. Test the model in the Intermediate Representation format using the [Inference Engine](../../../OV_Runtime_UG/Deep_Learning_Inference_Engine_DevGuide.md) in the target environment via provided [sample applications](../../../OV_Runtime_UG/Samples_Overview.md).
5. [Integrate](../../../OV_Runtime_UG/Samples_Overview.md) the Inference Engine in your application to deploy the model in the target environment.
mo --input_model rnnt_joint.onnx --input "0[1 1 1024],1[1 1 320]"
```
Please note that hardcoded value for sequence length = 157 was taken from the MLCommons but conversion to IR preserves
network [reshapeability](../../../../IE_DG/ShapeInference.md), this means you can change input shapes manually to any value either during conversion or
network [reshapeability](../../../../OV_Runtime_UG/ShapeInference.md), this means you can change input shapes manually to any value either during conversion or
> * Starting with the 2022.1 release, the Model Optimizer can convert the TensorFlow\* Object Detection API Faster and Mask RCNNs topologies differently. By default, the Model Optimizer adds operation "Proposal" to the generated IR. This operation needs an additional input to the model with name "image_info" which should be fed with several values describing the pre-processing applied to the input image (refer to the [Proposal](../../../../ops/detection/Proposal_4.md) operation specification for more information). However, this input is redundant for the models trained and inferred with equal size images. Model Optimizer can generate IR for such models and insert operation [DetectionOutput](../../../../ops/detection/DetectionOutput_1.md) instead of `Proposal`. The `DetectionOutput` operation does not require additional model input "image_info" and moreover, for some models the produced inference results are closer to the original TensorFlow\* model. In order to trigger new behaviour the attribute "operation_to_add" in the corresponding JSON transformation configuration file should be set to value "DetectionOutput" instead of default one "Proposal".
> * Starting with the 2021.1 release, the Model Optimizer converts the TensorFlow\* Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the Inference Engine using dedicated reshape API. Refer to [Using Shape Inference](../../../../IE_DG/ShapeInference.md) for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
> * Starting with the 2021.1 release, the Model Optimizer converts the TensorFlow\* Object Detection API SSDs, Faster and Mask RCNNs topologies keeping shape-calculating sub-graphs by default, so topologies can be re-shaped in the Inference Engine using dedicated reshape API. Refer to [Using Shape Inference](../../../../OV_Runtime_UG/ShapeInference.md) for more information on how to use this feature. It is possible to change the both spatial dimensions of the input image and batch size.
> * To generate IRs for TF 1 SSD topologies, the Model Optimizer creates a number of `PriorBoxClustered` operations instead of a constant node with prior boxes calculated for the particular input image size. This change allows you to reshape the topology in the Inference Engine using dedicated Inference Engine API. The reshaping is supported for all SSD topologies except FPNs which contain hardcoded shapes for some operations preventing from changing topology input shape.
## How to Convert a Model
@@ -63,7 +63,7 @@ based on deep learning in various tasks, including Image Classifiacton, Visual O
Speech Recognition, Natural Language Processing and others. Refer to the links below for more details.
3. Download [coco.names](https://raw.githubusercontent.com/pjreddie/darknet/master/data/coco.names) file from the DarkNet website **OR** use labels that fit your task.
3. Download [coco.names](https://github.com/AlexeyAB/darknet/blob/master/data/coco.names) file from the DarkNet website **OR** use labels that fit your task.
4. Download the [yolov3.weights](https://pjreddie.com/media/files/yolov3.weights) (for the YOLOv3 model) or [yolov3-tiny.weights](https://pjreddie.com/media/files/yolov3-tiny.weights) (for the YOLOv3-tiny model) file **OR** use your pre-trained weights with the same structure.
5. Install PIL, which is used by the conversion script in the repo:
```sh
pip install PIL
pip install pillow
```
6. Run a converter:
> **NOTE**: This converter works with TensorFlow 1.x and numpy 1.19 or lower.
-`custom_attributes` is a parameter that stores all the YOLOv3 specific attributes:
-`classes`, `coords`, `num`, and `masks` are attributes that you should copy from the configuration
file that was used for model training. If you used DarkNet officially shared weights,
you can use `yolov3.cfg` or `yolov3-tiny.cfg` configuration file from https://github.com/pjreddie/darknet/tree/master/cfg. Replace the default values in `custom_attributes` with the parameters that
you can use `yolov3.cfg` or `yolov3-tiny.cfg` configuration file from https://github.com/david8862/keras-YOLOv3-model-set/tree/master/cfg. Replace the default values in `custom_attributes` with the parameters that
follow the `[yolo]` titles in the configuration file.
-`anchors` is an optional parameter that is not used while inference of the model, but it used in a demo to parse `Region` layer output
-`entry_points` is a node name list to cut off the model and append the Region layer with custom attributes specified above.
Direct ONNX Format Support <openvino_docs_IE_DG_ONNX_Support>
@@ -35,8 +39,6 @@ The scheme below illustrates the typical workflow for deploying a trained deep l
\\* _nGraph_ is the internal graph representation in the OpenVINO™ toolkit. Use it to [build a model from source code](https://docs.openvino.ai/latest/openvino_docs_nGraph_DG_build_function.html).
@@ -70,7 +70,7 @@ The example below demonstrates how to unregister an operator from the destructor
## Requirements for Building with CMake
A program that uses the `register_operator` functionality requires `openvino::core` and `openvino::frontend::onnx` libraries in addition to the OpenVINO Inference Runtime.
The `ov_onnx_frontend` is a component of the `OpenVINO` package , so `find_package(OpenVINO REQUIRED COMPONENTS ONNX)` can find both.
The `openvino::frontend::onnx` is a component of the `OpenVINO` package , so `find_package(OpenVINO REQUIRED COMPONENTS ONNX)` can find both.
Those libraries need to be passed to the `target_link_libraries` command in the CMakeLists.txt file.
@@ -34,15 +34,13 @@ Read the sections below to learn about each item.
```
2. **Include Inference Engine, nGraph and OpenCV libraries** in `project/CMakeLists.txt`
[OpenCV](https://docs.opencv.org/master/db/df5/tutorial_linux_gcc_cmake.html) integration is needed mostly for pre-processing input data and nGraph for more complex applications using [nGraph API](../nGraph_DG/nGraph_dg.md).
[OpenCV](https://docs.opencv.org/master/db/df5/tutorial_linux_gcc_cmake.html) integration is needed mostly for pre-processing input data and model representation in OpenVINO™ Runtime for more complex applications using [OpenVINO Model API](../OV_Runtime_UG/model_representation.md).
### Use Inference Engine API to Implement Inference Pipeline
@@ -457,7 +455,7 @@ Load the model to the device using `load_network()`:
@endsphinxdirective
This example is designed for CPU device, refer to the [Supported Devices](../IE_DG/supported_plugins/Supported_Devices.md) page to read about more devices.
This example is designed for CPU device, refer to the [Supported Devices](../OV_Runtime_UG/supported_plugins/Supported_Devices.md) page to read about more devices.
#### Step 4. Prepare input
```py
@@ -491,4 +489,4 @@ Congratulations, you have made your first Python application with OpenVINO™ to
@@ -43,8 +43,8 @@ If a model has a hard-coded batch dimension, use `InferenceEngine::CNNNetwork::s
Inference Engine takes three kinds of a model description as an input, which are converted into an `InferenceEngine::CNNNetwork` object:
1. [Intermediate Representation (IR)](../MO_DG/IR_and_opsets.md) through `InferenceEngine::Core::ReadNetwork`
2. [ONNX model](../IE_DG/ONNX_Support.md) through `InferenceEngine::Core::ReadNetwork`
3. [nGraph function](../nGraph_DG/nGraph_dg.md) through the constructor of `InferenceEngine::CNNNetwork`
2. [ONNX model](../OV_Runtime_UG/ONNX_Support.md) through `InferenceEngine::Core::ReadNetwork`
3. [OpenVINO Model](../OV_Runtime_UG/model_representation.md) through the constructor of `InferenceEngine::CNNNetwork`
`InferenceEngine::CNNNetwork` keeps an `ngraph::Function` object with the model description internally.
The object should have fully-defined input shapes to be successfully loaded to Inference Engine plugins.
@@ -66,7 +66,7 @@ To feed input data of a shape that is different from the model input shape, resh
Once the input shape of `InferenceEngine::CNNNetwork` is set, call the `InferenceEngine::Core::LoadNetwork` method to get an `InferenceEngine::ExecutableNetwork` object for inference with updated shapes.
There are other approaches to reshape the model during the stage of <ahref="_docs_MO_DG_prepare_model_convert_model_Converting_Model.html#when_to_specify_input_shapes">IR generation</a> or [nGraph::Function creation](../nGraph_DG/build_function.md).
There are other approaches to reshape the model during the stage of <ahref="_docs_MO_DG_prepare_model_convert_model_Converting_Model.html#when_to_specify_input_shapes">IR generation</a> or [ov::Model creation](../OV_Runtime_UG/model_representation.md).
Practically, some models are not ready to be reshaped. In this case, a new input shape cannot be set with the Model Optimizer or the `InferenceEngine::CNNNetwork::reshape` method.
@@ -223,4 +223,4 @@ The Inference Engine provides a special mechanism that allows adding support of
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
