Commit Graph

48 Commits

Author SHA1 Message Date
Mikhail Nosov
6ef59ce3e4 [OV2.0] Model Optimizer: mean/scale/reverse_input_channels/layout for new frontends (#8751)
* Preprocessing API - base classes

Includes API definition for trivial mean/scale operations (which don't require layout)

Mean/scale with 'layout' support will be done under separate task together
 with Layout

Current test code coverage: 100%

* Python bindings for base preprocessing API

* remove pre_post_process directory from ngraph/core

* remove files from ngraph/python dir

* move pyngraph pre_post_process files from ngraph/python to runtime

* remove pre_post_process test from CMakeList

* move include to the header

* update include path for pre_post_process

* style fix

* bind InputTensorInfo::set_layout

* cleaned test_preprocess

* fix test expected output

* remove duplicate test

* update description of set_element_type

* fix style

* move preprocess from pyngraph to pyopenvino/graph

* update test_preprocess imports and remove unnecessary test

* remove duplicate import

* update custom method

* update test

* update test

* create decorator that changes Node into Output<Node>

* create function that cast Node to Output<Node>

* update test_preprocess to use decorator for custom function

* change _cast_to_output -> _from_node

* move frontend folder to pyopenvino

* rename includes and add compile options

* include frontend to pyopenvino

* move __init__.py

* move tests

* remove mock from tests_compatibility

* rename import module

* Fix code style cpp

* refactor a few lines

* style fix

* update few lines in mo

* add tests fro scale and mean with vector input

* style fix

* add docstring for custom_preprocess_function

* bind InputInfo network method

* style fix

* Add pyopenvino to dependencies

* bind OutputInfo

* fix description of preprocess submodule

* fix style

* update copyright year

* Fix mock

* update docstring

* bind OutputTensorInfo

* bind OutputNetworkInfo and InputNetworkInfo

* bind ColorFormat and ResizeAlgorithm

* clean imports

* fix typo

* add PostProcessSteps to init

* bind PreProcessSteps

* create additional tests

* Fix mo test

* remove module local

* fix code style

* update comment

* fix return type

* update docs

* fix code style

* change ngraph.Type to ov.Type

* fix typo

* move _from_node to node_output.hpp

* add read_model from buffer

* update imports

* add new line

* remove bad quotes

* update imports

* style fix

* add new line

* rename functin args

* remove Type import

* update tests

* style fix

* test clean

* remove blank line

* update PrePostProcessor init and build methods

* create test with model update tests with new PrePostProcessor init and build

* # Conflicts:
#	inference-engine/ie_bridges/python/src/openvino/offline_transformations/offline_transformations_api.pyx
#	inference-engine/ie_bridges/python/src/openvino/offline_transformations/offline_transformations_api_impl.cpp
#	inference-engine/ie_bridges/python/src/openvino/offline_transformations/offline_transformations_api_impl.hpp
#	inference-engine/ie_bridges/python/src/openvino/offline_transformations/offline_transformations_api_impl_defs.pxd
#	inference-engine/tests/ie_test_utils/common_test_utils/ngraph_test_utils.cpp
#	inference-engine/tests/ie_test_utils/common_test_utils/ngraph_test_utils.hpp
#	model-optimizer/mo/moc_frontend/serialize.py
#	thirdparty/gflags/gflags
#	thirdparty/gtest/gtest

* Stash

* move preprocess module from openvino.impl to openvino

* fix building

* fix code style

* try to move MO to use new api

* Intermediate commit

* try to move MO to use new api

* Test pybind11 custom holder for Preprocessing types (InputInfo and PreProcessingSteps)

* Initial code for source_target layout handling for preprocessing
Initial implementation of reverse input channels

* Use input's tensor names instead of friendly names

* Skeleton for guessing layouts and clearing it after preprocessing

* updated package_BOM.txt

* Use reference_wrapper for preprocess bindings

* Update tests

* Layout::find_permutation - support of dynamic layouts
Covered case for 'trivial convert' where no permutation is needed
It is needed for Model Optimizer for logic which will guess model's layout, like "?c??"

* Stash

* add bindings to I420_SINGLE_PLANE and I420_THREE_PLANES

* remove init from all classes except PrePostProcessor and add RGBX and BGRX to ColorFormat enum

* Guess layout so that existing mean/scale tests passed

* update test name

* Draft to guess layout for 'reverse_input_channels'

* More unit tests (error cases)

* pylint & flake8

* pylint - ignore import error

* Stash

* Moved preprocessing to 'back' folder

* More tests

* Update package_BOM

* Support layout_values with no names
Support layout set for 'outputs'
Tests

* Export more enum names from nrgaph

* Basic --layout parsing

* removed debug prints

* Further updates after rebase

* Update imports

* Removed part from 8829

* Fix imports in test code

* Minor cosmetics

* Don't guess 'C' if layout is already set by model
Expose 'Layout::empty' method

* Style fix

* Apply review comments
Restricted 'heuristics'

C++: Added 'fp16', 'fp64' support to mean/scale

* Applied review comments

* Added some dynamic test cases

* Move call of 'apply_preprocessing' to 'serialize.py'

* Unnecessary change

* Added more comments to code

Co-authored-by: pszmel <piotr.szmelczynski@intel.com>
Co-authored-by: Alexey Lebedev <alexey.lebedev@intel.com>
Co-authored-by: bszmelcz <bartosz.szmelczynski@intel.com>
Co-authored-by: Anastasia Kuporosova <anastasia.kuporosova@intel.com>
Co-authored-by: y <ilya.lavrenov@intel.com>
Co-authored-by: Vafin, Maxim <maxim.vafin@intel.com>
2021-12-07 14:31:55 +03:00
Roman Kazantsev
42892767f5 [MO, nGraph, transforms., ref., python] Support DetectionOutput-8 (#8853)
* Implement MO part for DetectionOutput-8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement nGraph part for DetectionOutput-8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement deduction of a number of classes method

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix build

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement nGraph tests to check shape inference for DetectionOutput-8 with deduction of num_classes

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement attributes tests for DetectionOutput-8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Remove DetectionOutput-1 from opset8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Add python bindings for DetectionOutput-8 and tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement downgrading and upgrading transformations for DetectionOutput

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Implement tests for downgrading and upgrading transformations for DetectionOutput

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Adopt existing reference for DetectionOutput-8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix DetectionOutput-8 reference

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Add tests for DetectionOutput-8 reference

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Finish resolving merge conflict

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix python bindings after re-structuring

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix BOM package list

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix multibox_detection_test

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix python bindings for DetectionOutput-8

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Insert blank line between summary and description for python binding

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply review comments: identation, for loop

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix formatting in python bindings

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix supported_attrs for DetectionOutput from opset1

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Add missed imports for python if_op

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Apply feedback and fix Win build

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Use the same test suite name for test mixture

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix transformation tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Recover BWDCMP_RTTI for legacy use and remove unused variables in tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Remove unused variables in type_prop tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Recover legacy referenceDetectionOutput constructor

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix python binding for DetectionOutput

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-12-07 01:17:49 +03:00
Alexey Lebedev
4e6eeea6ff [PYTHON API] move frontend bindings to pyopenvino + move MO to use new Python API (#8301)
* move frontend folder to pyopenvino

* rename includes and add compile options

* include frontend to pyopenvino

* move __init__.py

* move tests

* remove mock from tests_compatibility

* rename import module

* Fix code style cpp

* refactor a few lines

* update few lines in mo

* Add pyopenvino to dependencies

* Fix mock

* update docstring

* Fix mo test

* remove module local

* fix code style

* update comment

* fix return type

* update docs

* fix code style

* fix building

* fix code style

* try to move MO to use new api

* Export more enum names from nrgaph

* [Python API] quick fix of packaging

* update tests

* fix setup.py

* small fix

* small fixes according to comments

* skip mo frontend tests

* update mo to new imports

* try to fix win wheel

* fix win wheel

* fix code style

Co-authored-by: Anastasia Kuporosova <anastasia.kuporosova@intel.com>
Co-authored-by: y <ilya.lavrenov@intel.com>
2021-12-01 21:31:16 +03:00
undef-nnov
c1515d92e8 Reverse infer (#8767) 2021-12-01 10:49:04 +03:00
Anastasia Kuporosova
f6df0a9c13 [Python API] quick fix of packaging (#8870)
* [Python API] quick fix of packaging

* update tests

* fix setup.py

* small fix

* small fixes according to comments

* skip mo frontend tests
2021-11-30 12:28:34 +03:00
Mateusz Tabaka
7720d82366 [MO] Fix fetching output Place in moc_frontend (#8431) 2021-11-23 14:40:57 +01:00
Anastasia Popova
5e15288c8a Separated old_api_map into two attributes. (#8632)
* Separated old_api_map into old_api_map_order and old_api_element_type.

* Clang format, small fix.

* Corrected comment.

* Removed OldApiMapOrderAttr class.
2021-11-18 22:04:50 +03:00
Mikhail Nosov
2245ea8be2 [FrontEnd] Rename all frontends from "*_ngraph_frontend*" to "_ov_frontend*" (#8645)
* Renaming all frontends from "*_ngraph_frontend*" to "_ov_frontend*"

Also Debug builds on Windows release frontends without "d" suffix will not be loaded

* Fix review comments and add wheels test debug prints

* More debug prints

* Load by absolute path and remove debug prints
2021-11-18 09:26:13 +03:00
Victor Kuznetsov
925e24525a Move unit_tests from test_pkg/tools to test_pkg/tests (#8548)
* add performance hint to time infer

* move unit_tests to tests pkg

* change unit_tests path in azure lin

* change path to mo in bom test

* fix tests_path

* change mo dir path
2021-11-15 14:18:16 +03:00
Yegor Kruglov
90a140ae98 Extend MO/nGraph for operation MaxPool-8 (#6776)
* MO update for MaxPool-8

* using attrs in op constructor

* remove pads_value attr and pad fusing

* added axis and index_element_type parameters

* updated mo maxpool-8

* added maxpool-8->maxpool-1 transformation, disabled Pad to MaxPool fusing

* added remove_values_output to pooling extractors

* moved remove_values_output attribute to pooling infer function

* fixed ir_comparator tests

* disabled pad to maxpool fusing test

* added downgrade transformation test

* downgrade transformation update

* updated ir reader and tf pooling layer tests

* updated onnx pooling layer tests and MO infer unit test

* updated ir reader extender

* uncommented layer tests code

* disabled MaxPool-8 python binding test

* comment resolving, removed PadMaxPool fusing

* removed test

* downgrade transformation fix, MO codestyle changes

* removed axis check from downgrade transformation

* mark max_pool_test as xfail

* updated downgrade transformation test

* using OPENVINO_RTTI
2021-10-29 14:47:00 +03:00
Anastasia Popova
bf8f9164ed Unification of layout and data types of Parameter and Result in MO (#7630)
* Added transposes insertion for Parameter and Result.

* Separated into several transformations.

* Corrected runtime_info format.

* Fixed runtime info serialization.

* Code refactoring.

* Corrected checks.

* Added debug output.

* Added check.

* Fixed unit tests.

* Changed old api map format, removed debug output.

* Moved serialize to rt_info property, code corrections.

* Refactored RTInfo class.

* Small corrections.

* Small corrections.

* Removed redurant import.

* Added tests, added undefined default type.

* Code reformat.

* Fixed serialization unit tests.

* Added comment.

* Added comment.

* Small test correction.

* Changed default values of old_api_map to values from old API IR.

* np.array -> int64_array

* Update MO to use FE to read IR; Swith MO IR version to 11

* Preserve output node name when inserting Transpose

* Codestyle

* Fix layer tests

* Pylint fix

* Disable ref_graphs comparision in layer tests

* codestyle

* Updated MO IR reader.

* Moved version initialization to constructor of OldApiMap.

* Added shape infer after transpose insertion.

* Fixed Pylint

* Removed wrong attribute removal.

* Added transposes insertion for Parameter and Result.

* Separated into several transformations.

* Corrected runtime_info format.

* Fixed runtime info serialization.

* Code refactoring.

* Corrected checks.

* Added debug output.

* Added check.

* Fixed unit tests.

* Changed old api map format, removed debug output.

* Moved serialize to rt_info property, code corrections.

* Refactored RTInfo class.

* Small corrections.

* Small corrections.

* Removed redurant import.

* Added tests, added undefined default type.

* Code reformat.

* Fixed serialization unit tests.

* Added comment.

* Added comment.

* Small test correction.

* Changed default values of old_api_map to values from old API IR.

* np.array -> int64_array

* Update MO to use FE to read IR; Swith MO IR version to 11

* Preserve output node name when inserting Transpose

* Codestyle

* Fix layer tests

* Pylint fix

* Disable ref_graphs comparision in layer tests

* codestyle

* Updated MO IR reader.

* Moved version initialization to constructor of OldApiMap.

* Added shape infer after transpose insertion.

* Fixed Pylint

* Removed wrong attribute removal.

* Serialize fix.

Co-authored-by: Gleb Kazantaev <gleb.kazantaev@intel.com>
2021-10-27 03:24:54 +03:00
Pavel Esir
9f64f77a3c [MO] allow zero size dimensions in Slice output (#7791)
* allow zero size dimensions in Slice output

* updated comments regarding 0-size dimensions
2021-10-20 13:16:06 +03:00
Anton Chetverikov
a56d81345d [MO] IR Reader meta_info using update (#7133)
* Initial change to allow saving graphto IR without meta_info

* Update saving and restoring functions, add more comments

* Add unit tests for define_data_type() function

* Fix wrong name

* Update condition

* Update meta_data checks

* Update data_type restoretion and missed outpurt ports handling

* Update and add new test

* Update comments

* Remove commented code

* Rename function

* Update temporary Result operations processing

* Remove define_data_type function

* Move node_normalize_output function to Op class methods

* Update comments

* Update comments
2021-10-05 15:07:44 +03:00
Michał Karzyński
bd5b1bf99f Change the way Model Optimizer loads frontends (#7330)
* Adds two switches `use_new_frontend` and `use_legacy_frontend` to override defaults.
* Rename ONNX frontend from `onnx_experimental` to `onnx`
2021-09-30 12:17:36 +02:00
Roman Donchenko
8598ce89a3 Add missing spaces between words in string literals (#7540)
And in one case, a missing period.
2021-09-20 15:38:49 +03:00
Anton Chetverikov
9cfdad9afc Change sort functions to fix incorrect order in port dictionaries (#7283)
* Change sort functions to fix incorrect order

* Add separate sorts for different cases

* Update sort function to correct control flow edges handling

* Refactor all sorts for dictionary ordering

* Update build_graph function for correct control_flow edges creation

* Add tests for nodes and ports sort equality

* Fix wrong port value

* Add direct conversion from bool to int

* Use .replace instead of .strip

* Add test with both control_flow and ordinary edges, refactored edges lists filling

* Refactored nodes dict filling

* Delete unused test code
2021-09-20 10:17:44 +03:00
Evgeny Lazarev
4547818fb1 Move TF OD API docs to code + several fixes for TF OD API models conversion (#7377)
* Refactored code, updated comments and documentation related to TF OD API models pre-processing.

* Improved MO messages related to pre-processor block removal during conversion of the TD OD API models. Remove mean/scale if padding is used and mean/scale is applied before resize

* Updated TF OD API transformation and documentation for SSD models

* Updated comments and documentation for the ObjectDetectionAPIMaskRCNNSigmoidReplacement transformation

* Updated comments and documentation for the ObjectDetectionAPIMaskRCNNROIPoolingSecondReplacement transformation

* Updated comments and documentation for the ObjectDetectionAPIPSROIPoolingReplacement transformation

* Updated comments and documentation for the ObjectDetectionAPIProposalReplacement transformation

* Updated comments and documentation for the ObjectDetectionAPIDetectionOutputReplacement transformation

* Minor code style fixes

* Fixed unit tests for ObjectDetectionAPIPreprocessor2Replacement transformation

* Improved unit test for pipeline.config parser. Fixed very long bug with incorrect test data for the PipelineConfig parser class

* Code style fixes

* Get rid of "coordinates_swap_method" parameter in the JSON configuration file for TF OD API models

* Code style fixes and minor refactoring

* Simplied code related to swapping Proposal coordinates

* Removed incorrectly removed code

* Fixed code review comments about the code comments
2021-09-08 10:03:01 +03:00
Evgeny Lazarev
3775dad345 MO dynamic shapes support (#5918)
* Allow MO to generate IR with -1 in dimensions

* Some fixes to support -1 for StridedSlice operation

* Updated TensorArrayGatherV3 shape infer to support dynamic output shape

* Several fixes to support undefined dimensions in the Broadcast,Reshape,Slice and Tile

* Fixed bug in the normalization transformation of TF NMS to opset NMS

* Updated shape infer functions related to StridedSlice and NMS

* Updated Select shape inference function to use common shape broadcasting function supporting dynamism

* Fixed operation TFResize shape infer function to work correctly for case when model is converted with --disable_nhwc_to_nchw

* Dynamic Range and update asserts in NMS

* Changed the way how dynamic dimensions are specified. Refactored shape inference functions and common places to use new approach

* More fixes to support dynamic shapes

* More fixes for support of dynamic shapes

* Fixed generation of IR with dynamic dimensions

* Allow reading IRs with undefined dimensions

* More changes in the IE to support dynamic dimensions

* Fixes for Switch, Merge, Concat shape and value infer related to dynamism

* Fixed TensorArray related ops to properly handle dynamic dimensions. Fixed StridedSlice infer for case with new_axis

* Fixed shape_for_layout function to generate masked array

* Fixed shape inference for Convolution and Poolings to support dynamic spatial dimensions

* Updated shape infer functions for CTCGreedyDecotder, CTCLoss and Enter

* Fixed shape inference with dynamic dimensions for MatMul, Split, Upsample, SpaceToBatch, some fixes for the TI

* Fixes for undefined dimensions support for Proposal and DetectionOutput

* Fixed ExtractImagePatches, DepthToSpace and RegionYolo shape infer functions to work with partially dynamic dimensions

* Changes in tf_window_op_pad_infer to better work with dynamic dimensions

* Fixed output shape calculation for StridedSlice operation

* More StridedSlice fixes

* Fixed resolve_convolution_with_group

* Fixed unit tests

* Fixed unit tests

* Fixed Switch op unit tests

* Fixed shape inference for Upsample operation

* Updated unit tests for the Concat operation

* Fixed eltwise shape infer unit tests

* Fixed shape infer tests for Convolution and DetectionOutput ops

* Fixed Crop shape infer function tests

* Fixed Slice op unit test and minor fix in the shape inference. Fixed emitter

* Updated unit test for telemetry and match_shape function for dynamism

* Fixed unit test for the DetectionOutput

* Added support for the TF ClipByValue operation

* Fixed GatherND shape inference for dynamic shapes support

* Dynamic shapes support for the MO IR Reader

* Fixed BlockLSTM operation to not work as an extractor

* Allow to serialize IRs with partially defined shapes

* Updated SelectBroadcast transformation to not check shape values

* Fixed MO IR comparator

* Fixed SS value propagation when slices are dynamic

* Do not re-run graph clean-up for ProposalMutation

* Fixed InterpolateSequenceToInterpolate transformation to support dynamic dimensions

* Fixed Loop iteration count calculation and reading IteratorGetNext shapes

* Fixed unit test for serialization

* Fixed serialization test

* Fixed RandomUniform shape infer

* Fixed several transformations related to RNN to respect dynamic output shapes

* Fixed Deconvolutin shape calculation for dynamic batch. Eltwise shape infer improvements

* Fixed shape infer functions for ExperimentalDetectron ops, reverted changes for NonZero and removed debug prints

* Fixed check for dynamism of a list, fixed value propagation for Concat op and remove redundant shape infer for reshape

* Update Eltwise value propagation to use np.ma

* Fixed ExpandDims shape infer function

* Shape infer functions fixes and improvements

* Remove Accum op from the MO

* Updated activation functions shape infer

* Removed unsupported operation Correlation

* Fixed shape infers for several functions

* Removed unsupported DataAugmentation operation

* Fixed shape infer functions for several ops in extensions directory

* Removed not-support operation PowerFile

* Removed unsupported SpatialTransformer,SimplerNMS and PredictionHeatmap operations

* More shape infer functions updates

* Merge shape infer fix

* Fixed typo

* Fixed TensorArraySize shape infer function

* Fixed VariadicSplit and Squeeze shape infer

* Fixed ONNX models Parameter extractor

* Updated Select value propagation for the dynamic case

* Fixed ReorgYolo shape infer and test

* Removed unnecessary tests

* Fixed Tile shape infer

* Fixed SparseFillEmptryRows unit tests

* Fixed package bom

* Added extractor for the TF operation Mod

* Fixed value propagation for MatMul operation

* Updated Parameter extender to generate shape_array when shape is partially defined only

* Fixed BOM file

* Fixed issue with the TF OD API models and DetectionOutput op. Now the shape infer function for the DO do not re-infer "num_classes" attribute value if it is already known

* Fixed unit test for the DO infer

* Fixed num classes calculation for the DO generation for Faster/Mask-RCNN models

* Changed NMS op to produce static output shape

* Restore dynamic output shape calculation for the NMS for NMS-5

* Fixed CellNormalizer transformation. It should work for static shapes only

* RNNCell Op class fixes

* Revert some changes

* Updated documentation with a list of supported operations

* Revert changes

* Fixes for the ConstantFill op

* Removed redundant SequenceLengthToMask transformation

* TensorArray* ops shape infer code style and refactoring

* Reverse some unnecessary changes in the ConvolutionNormalizer

* Fixes and unit tests for shape_array, compare_shapes, is_fully_defined functions

* Implemented shape_insert, shape_delete functions and tests for them

* Modified code to use shape_delete function

* Added usage of shape_insert function where necessary

* Use shape_insert function in many places

* Some fixes in shape inference for various ops

* Updated shape_delete function to support negative indices

* Changes and unit tests for the MatMul infer function

* Removed strange code from the TF Merge infer function

* Merge op shape infer fixes

* Fixed value propagation in the transformation EltwiseInputReshape.py for the dynamic dimension case

* Code cleanup

* Updated GatherND to support dynamic dimensions

* Minor fixes

* Fixed shape_insert and shape_delete to support np.int64 and np.int32 types

* Updated Upsample operation unit tests with dynamic input shapes

* Minor change in the extensions/back/ConvolutionNormalizer.py to make sure that input dimensions are static

* Fixed ConvertGroupedStridedSlice transformation and added unit tests

* Revert debug changes

* Fixed value propagation for Unsqueeze to work with partially defined input values

* Typo fix

* Added unit tests for the Unsqueeze op shape infer

* broadcasting functions changes and unit tests

* Fixed Tile value inference for partially defined input tensor

* Unit tests for Split and VariadicSplit ops

* Fixes for the Concat infer + unit tests

* Removed redundant tf_pack shape infer

* Fixed Concat value infer and added unit tests

* Fixed StridedSlice shape inference for case with dynamic slices

* Fixes related to StridedSlice shape infer, changes in tests

* Unit tests for the eltwise shape and value infer

* Fixed Pad op value propagation to allow dynamic input values to be propagated

* Unit test for Pooling dynamic input shape infer

* Squeeze op unit tests for dynamic input shape

* Added assert to the Squeeze op shape infer for case when squeeze dimension is dynamic value

* Added message to the MO when input shapes are dynamic

* Convolution dynamic unit test

* Removed redundant transformation GroupedConvWeightsNormalize

* Removed non-ascii character from the message

* Fixed typo in the BOM file

* Code style and comment fixes

* Fixed copy-paste issue in the DO shape infer function

* Fixed setting dynamic shape in the MO command line

* Added function to compare tensor with dynamic values. Fixes in the unit tests and shape infer functions

* Improved Reshape shape infer + added unit tests

* Fixed value propagation for Select op

* Renamed several internal functions, minor code fixes.

* Code style fixes

* Modified condition in the _set_shape method of the Port class to not check shape if the "override_output_shape" attribute is specified

* Fixed constant value propagation for ReduceOps when inputs have dynamic values. Added unit test

* Fixed shape infer for the Loop for dynamic dimensions case

* Fix in the NMS shape infer to avoid ragged numpy array generation. Fixed Scatter shape infer validation

* Improved shapes infer for eltwise ops with respect to dynamic dimensions

* Changed code comments

* Renamed tensor names in the ClipByValueTFTransformation

* Changed np.ma.allequal to strict_compare_tensors in the Merge op infer

* Chanded np.ma.allequal with strict_compare_tensor.

* Fixed Merge op value infer

* Fixed debug code

* Removed commented line

* Updated condition to check for dynamic shapes in the Partial infer to not fail for MxNet models

* Improvements to the get_shape_from_slice and is_dynamic_slice functions

* Reverted change in the `normalize_slices_attr` for ellipsis mask case

* Updated shape conditions in the ScatterNDBase op to support dynamic dimensions

* Crop op file refactoring

* Set "type" attribute to None for SparseFillEmptyRows op which is not from any opset

* Removed unnecessary extractor test

* Restored Crop operation type

* Removed "type" attribute from the Crop operation and updated the MO code to find Crop by "op" attribute

* Fixed If shape infer function to produce dynamic dimensions

* Updated If shape and value infer to properly work when condition is static

* Fixed fusing transformation check to work with dynamic dimensions. Change comparison in the shape_inference function to not use strict shapes comparison

* Optimize imports in the LayerNorm

* ConvertGroupedStridedSlice minor fixes related to dynamism support

* Fixed ConvertGroupedStridedSlice to properly check if the dimension is sliced
2021-09-01 14:35:06 +03:00
Mikhail Nosov
c4bd0a45d3 MO: Support of discovering of suitable MOC frontend (#6888)
* MO: Support of discovering of suitable MOC frontend if --framework is not specified

* Ready for review

* Fix: don't use FrontEndManager if framework is not in list of available frontends

* Apply review comments
2021-08-10 09:23:30 +03:00
Mikhail Nosov
868fad33ab [MO] Add support to moc_frontend of ":" as delimiter for --input and --output (#6543)
* [MO] Add support to moc_frontend of ":" as delimiter for --input

Additions:
Changed default logic for 'Place::get_in(out)put_port' to return nullptr
Changed default logic for 'InputModel::get_place_by_tensor(operation)_name' to return nullptr

* Corrected comments in code

* Missing empty line

* Clang format fixes

* Fix review comments

* Updated test to verify review comments fixes

* Update unit tests after rebase

* Apply review comments
2021-07-29 13:09:51 +03:00
Mikhail Nosov
25fc647b2f Azure CI: run MO unit tests after OpenVINO build (#6772) 2021-07-23 17:42:49 +03:00
iliya mironov
e1bffcc36c Add json validate (#6449)
* Add json validate

* Fix json schema

* Fix schema loader

* Add unit test

* Update bom file

* Update all requarments

* Update dev requarments

* Update requrments

* Update path to schema

* Update schema

* Add some unit tests

* Move schema to root dir

* Update schema path in bom file

* Fix unit test

* Fix bom

* Change path to schema

* update setup

* Fix setup

* Fix mo args test

* Refactoring some code

* Refactoring according to review

* Update sort imports

* Remove id attribute from schema

* Refactoring validator

* Fix according to review

* Move schema from json to dict. Update unit tests.

* Fix BOM file

* Update bom file
2021-07-22 21:12:44 +03:00
Yegor Kruglov
b56164e804 [MO] Updating condition in TF Const value extracting (#6322)
* update value extracting condition

* added a comment

* updated unittest

* added a scalar test, little fix for scalar values extracting
2021-07-08 15:55:29 +05:00
Evgeny Lazarev
21d060ac2b Updated conversion of TF OD API 2.4 SSD models (#6473)
* Updated conversion of TF OD API 2.4 SSD models

* Fixed issue when more Conv2D nodes were selected for weights permutation when converting TF OD API models

* Code style fixes

* Fixed code comments
2021-07-02 17:35:59 +03:00
Mikhail Nosov
ae624ea18a [MO] Fix showing graceful error message when no ngraph case (#6379)
* [MO] Fix showing graceful error message when no ngraph case

Move frontend-related imports inside appropriate functions
And use try/except if import fails to show graceful error message

* Revert "[MO] Fix showing graceful error message when no ngraph case"

This reverts commit 72d52bd946.

* Print error before running of subprocess if IE or nGraph Python API is not found

* Added pytest to requirements_dev.txt

* Fixed review comment
2021-07-01 09:49:18 +03:00
Mikhail Nosov
d49405a0e8 [MO] Initial support of nGraph Frontend API (#6001)
* [NG Frontend] Introduce classes and macros for Frontend-specific exceptions

* [nGraph Frontend] Python bindings

Exposing Python API for nGraph FrontEndManager introduced in PR #5470

* Exposing 'Dimension' class to Python nGraph API

* Model optimizer code from pdpd_frontend

Removed 'use_legacy_frontend' option as not needed for PDPD integration

* Added XML/BIN prints for 'ngraph-FE' serialization path

* Update python bindings to latest code from PR #5995

* Add OV_FRONTEND_PATH to LD_LIBRARY_PATH by default for Model Optimizer

* Test with 'mock' frontend

* Added meta info to generated IR

Added more tests to cover currently supported scenarios
Clang config for MO mock frontend

* some flake8 and pylint fixes

* Try to 'import' ngraph-related packages on top of python files

This will help to specify type of function arguments, but can break unit tests if they're using functions directly without ngraph

* CI fixes

* Run tests from 'main_test.py' inside ngraph environment

* Fixed review comments

* Fixed comments
Added more tests for checking if inputs and outputs are same

* Renamed front_ng to moc_frontend

* Update package_BOM.txt
2021-06-24 12:21:15 +03:00
Gleb Kazantaev
ea3ed8af21 Turn on IE and NG python APIs by default inside Model Optimizer (#5721)
* Turn on IE and NG python APIs by default inside Model Optimizer

* Remove fallback

* Fix mo_ut

* Remove MO wheel tests

* Add model_optimizer custom target to gather all MO deps inside single traget

* Fix PR comments
2021-06-16 03:12:24 +03:00
Anastasia Popova
327c3149f6 Fixed feature dimension calculation in grouped_convolutions_fusing(). (#6054)
* Fixed feature dim calculation in concat_convolutions().

* Fixed feature dim calculation in concat_convolutions().

* Added comments, added unit test.
2021-06-08 09:53:39 +03:00
Ivan Tikhonov
c1608628d4 LowLatency v2 ngraph transformation (#5160)
* LowLatency 2.0: transformation and unit tests

* low latency 2.0: unit tests

* documentation and ngraph codestyle

* update CNN Interface of LowLatency transformation

* fix build on Windows

* fix build on Windows

* investigation of a failed build on Win OS

* ngraph codestyle

* fix build (werrors)

* New unit tests, refactoring

* update functional tests for Memory

* update LowLatency functional tests

* extend Memory tests to cover LowLatency v2 transformation

* clean up, code style

* fix unit tests

* update and fix unit tests, add feature to apply LLTv2 after LLTv1

* update docs, refactoring

* add several gna tests to skip config

* fix python api tests

* update python api, rename LowLatency_v2 to LowLatency2

* deprecate LowLatency v1

* Deprecate LowLatency v1 in IE

* fix wrong merge, codestyle

* resolve review comments

* fix python test

* update skip config

* apply online review notes, fix unit tests

* clean up, code style

* fix docs

* Use debug_messages instead of exceptions in llt v2

* fix unit tests

* Resolve review remarks
2021-06-07 15:13:41 +03:00
Eugeny Volosenkov
859a3b8a30 Deprecated warnings fix in MO (#6016)
* Add code style fixes

* Revert "Add code style fixes"

This reverts commit 490934f243.

* Fix Invalid escape sequence

* Fix Invalid escape sequence
2021-06-04 16:53:30 +03:00
Anastasia Popova
b1e7a5cdb1 Fixed duplicate names appearance in graph after fuse mul transformation. (#5842)
* Added manual set of unique name to the data node.

* Added attributes checks in unit test.

* Removed redundant line break.
2021-05-28 16:37:43 +03:00
Anton Chetverikov
9d3780648f Add missed attributes for Parameter layer (#5676)
* Add shape and element_type to ir_v10_attrs structure

* Add comment

* Update add_input_op_input_port_with_data() and add_input_op_output_port_with_data() functions with new GraphAPI

* Remove incorrectly added test

* Apply comments

* Update missed tests

* Add more checks to tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-05-28 10:22:06 +03:00
Anastasia Popova
a9230a916b LSTMCell tensor names fix (#5696)
* Added handling of debug information in create_node().

* Code refactoring.

* Checks fixed.

* Added comments, added unit test.

* Renamed unit test class.

* Fixed port number in unit test.
2021-05-27 15:13:44 +03:00
Evgenya Stepyreva
ffc9c95a2b Revert "Fuse mul transformation fix (#5518)" (#5831)
This reverts commit 84b94c9834.
2021-05-26 16:07:46 +03:00
Maxim Vafin
d8d9819b5f Fix security issue with XML parsing (#5686)
* Fix security issue with XML parsing

* Apply review feedback

* Rework defusing stdlib solution

* Apply review feedback
2021-05-26 14:17:55 +03:00
Anastasia Popova
04de4e34bc Additional telemetry events in MO. (#5662)
* Added additional telemetry events.

* Separated sending tf1 and tf2.

* Small correction.

* Unit test fix.

* Added op_names_statistic field in graph. Added op names saving in loop ext, while ext.

* Optimize imports.

* Added debug print.

* Added comments, removed debug print.

* Added comment.

* Renamed dynamic shapes event label to partially defined, added unit tests.

* Added attribute checks, moved telemetry methods to separate file.

* Small corrections.

* Updated BOM file.
2021-05-24 15:21:29 +03:00
Svetlana Dolinina
8e956654bf fix stride for 1d maxpooling (#5710)
* fix stride for 1d maxpooling

* fix test
2021-05-21 11:57:40 +03:00
Roman Kazantsev
d98e954f5d Make order of port names determined in IR (#5701)
* Make order of port names determined in IR

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Make port names in determined order and adopted tests

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-05-20 13:44:06 +03:00
Evgeny Lazarev
717499cf2f Common telemetry (#5032)
* Moved telemetry to repo root directory from MO

* New telemetry package in the "openvino" sub-directory

* Removed telemetry from the MO BOM

* Updated MO BOM and added stub file for telemetry

* Fixed license header

* Fixed license headers and cleaned up the telemetry setup.py

* Fixed import

* Added temporary dependency for openvino-telemetry

* Added ignore for pylint issues

* Fixed import statements

* Updated imports in the telemetry library

* Removed telemetry library. Added link to another private repo

* Removed redundant start_session event for the MO

* Changed approach to import the telemetry library

* Minor code refactoring

* Updated MO telemetry events sending messages

* Refactor sending events for the IE runtime check

* Disable forcing warnings for deprecated methods

* Removed changes from the requirements.txt to install telemetry library to avoid merge conflicts

* Update copyright in the model-optimizer/mo/utils/telemetry_stub.py

Co-authored-by: Gleb Kazantaev <gleb.nnstu@gmail.com>

Co-authored-by: Gleb Kazantaev <gleb.nnstu@gmail.com>
2021-05-14 21:56:03 +03:00
Gleb Kazantaev
b4565b7b4f Add Model Optimizer --transform option (#5504)
* Execute MO stages inside subprocess to have single IE check

* Add --transform key

* Updated ofline transformations to execute user specified passes; updated logic to raise when --transform is used

* Parametrize LowLatency transformation with num_iterations key

* Fixed MO and IE versions comparision

* Use subprocess for offline transformations execution to catch errors

* remove ie_is_available from IR; fixed typo

* Fix for old IE versions

* Update parse_transform key unit tests

* Show available transformations

* Fixed typo

* Fix review comments

* Fix python2 compatibility

* Fixed review comments

* Fixed __main__ import
2021-05-12 18:42:56 +03:00
Anastasia Popova
84b94c9834 Fuse mul transformation fix (#5518)
* Changed fuse_mul behaviour for proper data node connection.

* Corrected the comment.

* Corrected the comment.

* Added permutation attribute saving.

* Added comment.

* Added unit tests, comments corrections.
2021-05-07 17:01:31 +03:00
Anton Chetverikov
b3c1dd9de3 Add sys_platform environment marker to version checker (#5437)
* Add sys_platform environment marker

* Update sys_platform check

* Add unit tests for sys_platform marker

* apply review comments

* Fix typo

* Update checker and tests, apply comments

* Update comments parsing and tests

* Fix commrnt

* Resolve comments and update check logic

* Update tests and fix bug with negative tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-05-06 16:20:37 +03:00
Svetlana Dolinina
7b52e3155a Tdnnf (#5255)
* initial changes (IR not generated)

* extractor fix

* convert tdnnf (with correct infer)

* refactoring + comments in code

* added unit tests + couple fixes based on tests

* change order for old convolutions

* fix pylint

* small refactoring

* added if to remove changes in old irs

* doc updated

* fix layout and kernel shapes for old convolutions

* fixed test

* moved test

* fix import in test

* fixed backward compatibility

* review fixes
2021-04-29 14:38:08 +03:00
Anastasia Popova
07214d0a47 Changed "out_port_id" attribute setting in mapping file to store tensor names. (#5344)
* Removed port id from fw_tensor_debug_info attribute.

* Added port number to tensor names in kaldi, mxnet. Fixed Const naming.

* Sort imports.
2021-04-29 14:05:35 +03:00
Anton Chetverikov
8395ce1b0f Update modules version checker in MO to support ~= (#5342)
* Add check for ~= sign for modules versions check

* Add unit test

* Update check for environment markers

* Update check for ~= sign

* Update unit tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-28 11:23:08 +03:00
Svetlana Dolinina
3028c78594 fix typo + test (#5392) 2021-04-27 13:37:02 +03:00
Anton Chetverikov
576e692b1d Update errors raising cases in modules versions checker in MO (#5141)
* Change error raising to message printing

* Update modules versions check

* Update check and add unit test

* Fix wrong function call

* Update unit tests

* Fix test

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-23 13:23:06 +03:00
Evgeny Lazarev
f3d1aa6490 Moved MO unit test files to a separate directory (#5312)
* Removed test-generator from all MO requirement files except the dev one

* Moved all MO unit tests files to a separate directory

* Added __init__.py files to the tests directory. Fixed importing paths for some unit tests

* Fixed imports in all unit tests. Moved all unit test related files from the MO code to the dedicated directory

* Renamed directory with unit test utils

* Updated imports in unit tests
2021-04-20 14:47:18 +03:00