Commit Graph
89 Commits
Author SHA1 Message Date
Vladimir Gavrilov 0c288d506c MO support for operations DFT and IDFT (#5197)
* Written MO classes for DFT and IDFT operations.

* Added class to read TF (I)FFT operations.

* Written extractors for TF operations FFT, FFT2D, FFT3D, IFFT, IFFT2D, IFFT3D.

* Written MO Roll operation and TF Roll operation extractor.

* Started to write needed transformations.

* Written transformation StridedSlices + Complex + Roll + (i)FFTxD + Roll + (Imag, Real) + Pack -> Roll + (I)DFT + Roll.

* Written transformation for Complex + ComplexAbs.

* Written correction of axes of Roll.

* Small fix.

* Small fix.

* Some fixes.

* Some changes.

* Now TF Roll is read as TFRoll. Written inserting Transposes before and after (I)DFT.

* Small fix.

* Written tests for the transformation TFRollToRoll.

* Added comments to some transformations.

* Deleted redundant import.

* Written tests for the transformation TransposeDFT.

* Fixes in MO IR Reader to read/write (I)DFT.

* Fixes in the list of supported TF layers.

* Started to write tests for SSliceComplexRolledFFTPackBlockReplacement transformation.

* Written tests for the MO transformation SSliceComplexRolledFFTPackBlockReplacement.

* Written tests for the MO transformation ComplexAbs.

* Tests for transformations were moved into unit_tests directory.

* All extractors for (I)FFTxD are in one file now.

* Deleted redundant transformations.

* Fixed extractor for TF Roll: now this operation is read as MO Roll.

* Added comments to TFFFT operation.

* The method insert_transpose of classes TransposeDFT and LayoutChangeForGatherND was moved into the separate function in the file model-optimizer/extensions/middle/InsertLayoutPropagationTransposes.py.

* Fixed comment for the transformation TransposeDFT.

* Small fix.

* Some fixes.

* Deleted shape infer function for the operation TFFFT. Sorted imports in complex_abs.py.

* Small fixes.

* Deleted redundant import.

* Fixes in some asserts.

* Small fix.

* Added names for created nodes in the transformation ComplexAbs.

* Added comments to the method canonicalize_axes.

* The transformation SSliceComplexRolledFFTPackBlockReplacement was split into the sequence of transformations SSliceComplexRollReplacement -> RollRealImagPackReplacement -> TFFFTToDFT.

* Written tests for the transformation SSliceComplexRollReplacement.

* Written tests for the transformation RollRealImagPackReplacement.

* Written tests for the transformation TFFFTToDFT.

* Deleted commented code.

* Fixed types of constants in the transformation ComplexAbs.

* Written tests for canonicalization of signal_size value.

* Deleted 'Replacement' from names of files and classes.

* Used comarison of ids, not names.

* replace_sub_graph was replaced with find_and_replace_pattern.

* Now the transformation RollRealImagPack is executed before running transformation model-optimizer/extensions/front/Pack.py.

* The body of the function create_dft_from_tffft is a part of the transformation TFFFTToDFT body now.

* Now method correct_roll_axes of classes RollRealImagPack and SSliceComplexRoll is moved to the function in mo/front/tf/graph_utils.py.

* Small changes.

* Added comment before mark_input_as_in_correct_layout(roll, 2).

* Now the functions correct_roll_axes generates sub-graph in the input port 2 of Roll.

* Corrected tests for the transformation SSliceComplexRoll.

* Corrected tests for the transformation RollRealImagPack.

* Deleted commented code.

* Some renaming.

* Added decomposition of the separate operation ComplexAbs (without Complex before it).

* Added comment to the transformation ComplexAbsAfterComplex.

* Optimized imports for the transformation TFFFTToDFT.

* The transformation SSliceComplexRoll was split into the sequence SSliceComplex -> CorrectRollAxes and disabled.

* Written tests for the transformation ComplexAbs.

* Written tests for the transformation SSliceComplex.

* Written tests for the transformation CorrectRollAxes.

* Deleted the transformation SSliceComplexRoll.

* Deleted renaming nodes.

* Fixed comment.

* Small fixes.

* Small fix.

* The attribute need_correction was renamed as input_rank_changed.

* Small fixes.

* Deleted commented code.

* Now we iterate over all complex_node.out_port(0).get_connection().get_destinations() input ports and mark the corresponding nodes with the marker attribute.

* Added the attribute 'in_ports_count' into the class FFTBase.

* Tests for the transformation TransposeDFT were rewritten using helper functions.

* Now the transformation RollRealImagPack uses existing Roll node instead of creating new one.

* Small fixes.

* Fix in the documentation.

* Written class to read MxNet (I)FFT operations. Written corresponding extractors.

* Corrected shape infer function for MXFFT operation. Written transformation to convert MXFFT to (I)DFT.

* Fixed shape infer function.

* Fixed the conversion MXFFT to (I)DFT.

* Written tests for the transformation MXFFTToDFT.

* The function correct_roll_axes was replaced with more generic function add_constant_to_negative_values.

* Fixes in classes TFFFT, FFTBase, DFT, IDFT, MXFFT.

* Added asserts in constructors of operations TFFFT and MXFFT.

* Refactored transformation MXFFTToDFT: conversion of DFT and IDFT were moved into separated functions.

* Moved some commented code.

* Fixed BOM file.

* Written function convert_ifft_to_dft.

* Started to rewrite tests for MXFFTToDFT transformations, in the case is_inverse=False.

* Small fixes.

* Fixes in the transformation RollRealImagPack.

* Renaming tests class for the transformation SSliceComplex.

* Fixes in the function compare_graphs. Now we get all output nodes of op node, and these output nodes are sorted by names.

* Fixed tests for the transformation MXFFTToDFT.

* Fix in the transformation ThresholdedReluDecomposition: added disconnect for trelu input port.

* Fixes in test for the transformation TFSliceToSlice.

* Small fix in the transformation ObjectDetectionAPIPreprocessor2Replacement.

* Small fix in comment.

* Optimized imports.

* Used remove_node in the transformation ThresholdedReluDecomposition and remove_nodes_from in the transformation RollRealImagPack, instead of ports disconnection.

* Deleted commented code.

* Deleted test case test_slice_replacer_begin_with_2_inputs.
2021-05-07 09:44:24 +03:00
Anton Chetverikovandachetver b3c1dd9de3 Add sys_platform environment marker to version checker (#5437)
* Add sys_platform environment marker

* Update sys_platform check

* Add unit tests for sys_platform marker

* apply review comments

* Fix typo

* Update checker and tests, apply comments

* Update comments parsing and tests

* Fix commrnt

* Resolve comments and update check logic

* Update tests and fix bug with negative tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-05-06 16:20:37 +03:00
Gleb Kazantaev b291ca8cfa Use Serialization as a default engine in MO (#5347)
* Use Serialization as a default engine in MO

* Added cmd option to use old serialization

* Added mapping file generation

* Test mapping file generation

* Fix setBatchsize parameters order; fix mapping file generation

* Added FrameworkNode; added method to read models with custom ops but without extensions

* Added python API for read_network_without_extensions function; updated mo not to use IECore

* Added read_model_without_extensions to IReader and IParser

* Fix V7 IR reader

* Fix pword value

* Fix dllexport macro usage

* Add metainfo to IR

* Fix nGraph code style

* Fix license header

* Restore prepare_emit_ir behaviour

* Fix compare_function to resolve situation when Result input port has multiple names

* Update Compare Functions

* Fix FrameworkNode validation

* Self-review

* CodeStyle check

* --use_fallback -> --use_legacy_ir_generation

* Sort imports in main.py

* --path_to_model -> --input_model

* Use logging instead of print

* Code simplifucation&cleanup

* Fix offline_Transformations key

* Fix GeneraeMappingFile comments

* Use Extension approach to work with custom ops

* Fix versions check

* Code clean-up

* Moved FrameworkNode to inference_engine_transformations library

* Fix FrameworkNode includes

* Code clean-up
2021-05-04 16:40:20 +03:00
Anastasia Popova 07214d0a47 Changed "out_port_id" attribute setting in mapping file to store tensor names. (#5344)
* Removed port id from fw_tensor_debug_info attribute.

* Added port number to tensor names in kaldi, mxnet. Fixed Const naming.

* Sort imports.
2021-04-29 14:05:35 +03:00
Anton Chetverikovandachetver 8395ce1b0f Update modules version checker in MO to support ~= (#5342)
* Add check for ~= sign for modules versions check

* Add unit test

* Update check for environment markers

* Update check for ~= sign

* Update unit tests

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-28 11:23:08 +03:00
Anton Chetverikovandachetver 576e692b1d Update errors raising cases in modules versions checker in MO (#5141)
* Change error raising to message printing

* Update modules versions check

* Update check and add unit test

* Fix wrong function call

* Update unit tests

* Fix test

Co-authored-by: achetver <anton.chetverikov@.intel.com>
2021-04-23 13:23:06 +03:00
Evgeny Lazarev f3d1aa6490 Moved MO unit test files to a separate directory (#5312)
* Removed test-generator from all MO requirement files except the dev one

* Moved all MO unit tests files to a separate directory

* Added __init__.py files to the tests directory. Fixed importing paths for some unit tests

* Fixed imports in all unit tests. Moved all unit test related files from the MO code to the dedicated directory

* Renamed directory with unit test utils

* Updated imports in unit tests
2021-04-20 14:47:18 +03:00
Sergey Lyubimtsev 8013d2f7ba MO cannot convert Caffe model from PyPI "openvino-dev" distribution (#5239)
* MO cannot convert Caffe model from PyPI "openvino-dev" distribution

* use os.pardir instead of '..'

* remove mo subdir
2021-04-16 13:35:25 +03:00
iliya mironov efeff1ee3e Add keep split output ports without consumers (#5136)
* Add keep split output ports without consumers

* Fix ir reader for split outputs

* Update unit tests

* Refactoring code according to review

* Fix unit test

* Fix
2021-04-12 17:49:53 +03:00
Yegor Kruglov c966aea879 Update StridedSlice extender to handle StridedSlice op with empty attributes (#4980)
* update ss extender

* change the condition

* add an assert

* fix for IR comparator tests failures
2021-04-02 09:40:46 +03:00
Dmitry Kurtaev 5272bd4ba9 Test MO wheel content (#5054) 2021-04-01 18:03:28 +03:00
Pavel Esir 6477e8ec01 [MO] Fix fp16 in shapeof subgraphs (#4524)
* Initial working solution

* moved bfs_search_apply_on_shapeof_subgraph_nodes from utils/graph.py to MarkShapeOfSubgraphDataType.py

* Reused bfs from MarkSubgraphsWithCorrectLayout.py

* fixed e2e precomit issues: specified correct const data_types, fixed BFS search staring point to avoid nodeless shapeof subgraphs

* fixed mxnet_rnnt: added converting all Const nodes in ShapeOf subgraph in MarkAndChangeDataTypeInShapeOfSubgraphs.py, revised Const values in transformations that affect ShapeOf subgraph nodes

* reverter ReverseV2ToReverseSequence.py and DecomposeBidirectionalRNNSequence.py

* in MarkSubgraphsWithCorrectLayout BFS search beauty applied

* apply review comments, returned back 'in_shape_subgraph' attribute

* graph condition added

* MO IR reader fix for mixed FP16 models, added replacer order placement comment

* moved to back phase

* new solution with marking nodes from bottom to top (WIP)

* successfully tested on back phase

* corrected unittest

* removed check for start nodes size in bfs

* fix transformations that insert f64 to f32 in shape subgraph

* corrected log.warning -> log.debug

* revised list if shape input operations added unittest for Const shape inputs

* applied @lazarevevgeny's comments

* licence head corrections
2021-03-30 23:16:32 +03:00
Alexey Suhov 6478f1742a Align copyright notice in python scripts (CVS-51320) (#4974)
* Align copyright notice in python scripts (CVS-51320)
2021-03-26 17:54:28 +03:00
Evgenya Stepyreva 2e75aafbe2 INT4 compression (#4912)
* INT4 compression

* packing tests

* (U)INT4 IR Reader enabling

* Cast test header added

* Fix graph.clean_up

* Review discussions addressed: comments clarification, variable naming, fuller test coverage
2021-03-26 01:06:44 +03:00
Evgeny Lazarev 522ad39a48 Tf 2.0 od api models with loop (#4772)
* Do not run transformations for TF 2.X OD API models recursively (needed for models with Loop operation)

* Added anchor front transformation to group all TF OD API transformations. Added new necessary dependencies from KerasRNN transformations related to While support

* Added JSON configuration files for TF 2.4 OD API SSD and EfficientDet models

* Updated documentation with table of supported TF 2.x OD API models

* Improved visualization of the dependency graph

* Updated version of the pre-processing transformation for TF 2.4 OD API models

* Fixes in the TF 2.x OD API models conversion

* Fixed order of applying mean/scale values for TF 2.X OD API pre-processing

* Updates to the documentation

* Fixes for the preprocessor block transformation for the TF OD API models

* Added code comments

* Fixed bom file

* Unit tests for the TF 2.4 OD API ObjectDetectionAPIPreprocessor2Replacement transformation

* Code cleanup

* Updates to the documentation on how to convert TF OD API models and graph dumper

* Added assert to make sure that operations in the `get_specific_ops_with_const_inputs` has exactly 2 inputs
2021-03-25 17:56:35 +03:00
Yegor Kruglov 244af264e6 Add remove_value_output attribute to onnx ArgMax extractor (#4831)
* add remove_value_output to onnx argmax extractor

* update TopK extender

* update copyright

* change condition in topk extender based on connection logic
2021-03-22 14:19:27 +03:00
Anastasia Popova 7111aff4a9 Fix of tensor names propagation in loops and in add_opoutput(). (#4565)
* Added tensor names propagation in loops and add_opoutput.

* Removed wrong changes.

* Fixed result rename test.

* Stage names fixed.

* Added tensor names propagation to outputs for caffe, tensorflow.

* Added tensor names propagation to outputs for kaldi.

* Fixed bugs.

* Bug fixed.

* Added fake outputs removal.

* Fixed bug.

* Added removal attr to fake outputs in kaldi, caffe.

* Fixed bug.

* Port indices fixed.

* Fixed fake outputs removal.

* Added get_attribute_between_nodes, set_attribute_between_nodes methods, fixed fake output removal.

* More comments added.

* Removed unnecessary stage sets in tests.

* Removed unnecessary output check.

* Revert wrong change.

* New line at the end of file.

* Output cut test.

* Output cut test added.

* Output cut test.

* Added unit tests, comments. Code refactoring.

* Removed unnecessary soft_get.

* Fixed bug in output_cut, code refactoring.

* Code refactoring.

* Added comments.

* Fixed bug in while loop.

* Corrected comments, code refactoring.

* Comments edited, code refactoring.

* Code refactoring.
2021-03-16 16:00:56 +03:00
Evgeny Lazarev 56a179c6a3 Added support for attribute 'pad_to_max_dimension' for TF OD API models (#4720)
* Added support for attribute 'pad_to_max_dimension' for TF OD API models

* Restored line removed by mistake
2021-03-11 13:21:20 +03:00
Evgeny Lazarev 699c63bdaf Updated copyright year in all MO files (#4630) 2021-03-05 11:57:42 +03:00
Anton Chetverikov 7770701fe8 Update pooling extender for 3D cases (#4504)
* Update pooling extender for 3d cases

* Update error message
2021-03-01 18:54:47 +03:00
Gleb Kazantaev 1bdccb41df Telemetry for IE dependency warnings (#4431)
* Initial telemetry for IE dependency warnings

* Unified MO/IE version extraction; updated telemetry messages to use simplified versions

* Update telemetry

* Renamings; improvements; comments

* Turn On telemetry

* Added tests

* Added versions_mismatch flag

* plarform -> system
2021-03-01 13:30:40 +03:00
Maxim Vafin bf6b061709 Fix MVN in MO (#4311)
* Fix MVN in MO

* Apply feedback and add test

* Remove unneeded import

* Apply review feedback

* Apply review feedback

* Fix issue with class name
2021-02-20 11:11:51 +03:00
Anton Chetverikov 8c79d4134f Update Convert extender (#4409)
* Update Convert extender

* Update copyright year
2021-02-19 20:50:46 +03:00
Gleb Kazantaev 5d647e6fab Add IE Python Dependency To MO (#4151)
* Added script that check IE existance

* Updated install_prerequisites.sh to check and insall IE python dependency

* Execute MOC transformation pipeline in subprocess

* Updated path to offline_transformations.py

* Updated IE dependency search; added libs search support

* Updated install_prerequisites.bat to check IE dependency

* Print warning if MO and IE versions do not match

* Set env before subprocess run

* Add mean and meta info to serialized IR

* Fix pylin import issue

* Update BOM file; added intel headers

* use default serialization; code style fix

* Codestyle

* Added subprocess timeout

* Check e2e status

* Update install_prerequisites.sh logic

* Update MO version to be generated even without cmake execution

* Update install_prereq to extract release version from python

* Simplify pyhton code

* Reduce memory consumtion to its original value

* Added OV find and search logic for windows batch script

* Remove model reading to avoid excess memory consumtption

* IMproved logging messages; simplified code

* Improved warnign message incase when MO and IE versions mismatch

* Fix test import

* Check IE bindings in the beginnig of MO pipeline

* Changed warning message; added RelWithDebInfo path

* Recomendation to build IE from source

* Resolve PR comments

* Apply review comments; Added CheckAPI pass

* Added path to IE local libs

* ModelOptimizer -> Model Optimizer (+IE)

* OpenVINO -> OpenVINO (TM) Toolkit

* Updated install_prerequisites.bat with new logic

* Move offline_transformations.py to back folder; improve warning mesages experience

* Update install_prerequisites.bat

* Move back some unnecessary changes

* Fix BOM

* Disable import errors for pylint

* Fixed grammar in messages

* pylint disable import error

* Final grammar fixes in bat and sh files
2021-02-19 14:13:20 +03:00
Anton ChetverikovandAnastasia Popova 2c4afd8cd4 Tensor names support in MO IR Reader (#4194)
* Added attributes save modes

* Added tensor names to IR

* Reformat code

* Add support for tensor names in MO IR Reader

* Unit tests and code refactoring

* Fixed error

* Code refactoring

* Code refactoring

* Code refactoring

* Error fixed

* Error fixed

* Bug fixed

* Bug fixed

* Additional unit tests and comments

* Small update

* Update fake infer function

* Update names restoring

* optimize imports

* Add support for old-style constants and for commas in reader

* Added dest mode in Fuse Mul

* Update default values

* Fix missed debug info in some specific cases

* Fix a lot of issues with missedand wrong names provoding

* Resolve review comments

* Update test IR's

* Refactor and simplify code

* More simplification

* Remove unneccessary changes

* model-optimizer/mo/utils/ir_reader/layer_to_class_test.py

* Add separate tests for names restoring

* Update copyright year

* Apply review comments

Co-authored-by: Anastasia Popova <anastasia.popova@intel.com>
2021-02-18 10:38:04 +03:00
iliya mironov f670b7cb3a Add CTCGreedyDecoder mo support (#4009)
* Add CTCGreedyDecoder mo support

* Update copiright

* Update bom file

* Add transformation

* Fix code style

* Fix according to review

* Add CTCGreedyDecoder v6 to ConvertPrecision

* Hot fix

* Add replasment for ctc_greedy_decoder

* Fix test

* Fix

* Update ie transform

* Draft ctc lost replaser

* Add ctcloss replaser

* Update

* Refactoring code

* Update transformation

* Update decoder

* Remove comments

* Convert seq mask from int to float

* Fix unit test

* Add dynamic tests

* Refactoring code

* Fix py code style

* update style

* Disable ctcgreedydecoder transform for mkldnn plugin

* Add some comments

* Add transfor code comments

* Enable transform from differend plagins

* Fix mo

* fix tests

* Fix comment

* Fix convert precition

* Update comment

* Fix prcition

* Refactoring according to reviw

* Add ir reder extender

* Rename transformation

* Update bom file

* Fix mo replacer

* Fix tests

* Move transform to decomp

* Add check blank_index

* Rafactoring ctcloss

* Change dinemic rank check

* Fix ctclos extractor

* Remove comment

* Fix code style

* Refactoring pattern matcher for transformation CTCGreedyDecoder

* Disavle transform for vpu

* Refactoring according to review

* Refactoring code

* Disable transformation for cldnn

* Remove unused code

* Reverse transfomation

* Fix code style

* Hot fix transform

* Fix unit tests

* Update transform

* Enable transform in common pipline

* Fix names replasments for mo transformations

* Hot fix

* Fix
2021-02-17 14:38:51 +03:00
Pavel Esir 22169a05b9 [MO] StridedSlice improvements (#4139)
* fix ss

* successfully converted

* successfully run moved infer and normalizer unit-tests

* successfully rewritten StridedSlice infer unittests

* int64 array

* Successfully converter crash-when-loading, xj_feauture and toy nets (cherry-picked maxpoolV4 and tf_broadcast_ext)

* successfully moved PermuteAttrs to general mechanism

* successfully converted xj_feauture and crash when loading with the new rewritten SS infer

* fixed get_shape_from_slice and moved to common utils

* fixed extending masks and some other

* some refactoring

* fixed extending masks in extractor, fixed licence year and some other code clearing

* corrected a couple of unittests

* fox permute for 5 rank slice and 4 rank inputs/

* WIP

* Added comments

* fixed StridedSlice in ProposalMutation.py

* rechecked shape_infer unittests added some new cases

* added shape_infer unit-tests after StridedSliceNormalizer pass and Permute unit-tests

* corrected unittests

* Applied review comments

* general permutations for inputs implemented, corrected ellipsis unrolling when shrink_axis is at the beginning, some other corrections

* removed code duplication in infer and normalizer, moved 'slices' attr normalizing to StridedSliceNormalizer.py

* removed some code duplication and other minor improvements

* Added tests

* minor corrections

* wider range of unittests added (froze the number)

* review comments applied

* enabled skipped unit-test

* comment corrections

* applied review comments: changed op -> type, added some asserts, corrected comments and other minor corrections

* sorted inputs, updated Supported_Frameworks_Layers.md, some minor
2021-02-16 11:48:49 +03:00
Evgeny LazarevandEvgeny Lazarev 66f4c69b90 Telemetry sender and MO instrumentation (#3804)
* Draft implementation of the telemetry sender utility

* Examples of sending telemetry from the MO

* More statistic about the model.

* Intentional broken file to fail Mask-RCNN ONNX model conversion

* Added joined list of ops used

* Added requests to the requrements file and update BOM to include necessary files related to telemetry

* Send telemetry alwasys

* Refactored usage of GUID usage in the telemetry

* Enabled sending telemetry always

* Simplified function "TelemetryBackend.send"

* Use other approach to send information about session to GA

* Added automatic registration of the telemetry backends and allow to choose it during the telemetry class instantiation

* Added "requests" as a requirement. Wrapped usage of requests module to not crash the app

* Added timeout for sending data to GA. Increased the queue size to 1000

* Finalize Telemetry class implementation

* Do not fail MO if non-critical component is not installed and updated Telemetry GA with the default property

* Added sending version to a separate event

* Use default TID to send the data

* Set lower bound for the requests module which does not contain vulnerabilities

Co-authored-by: Evgeny Lazarev <elazarev.nnov@gmail.com>
2021-02-10 10:51:31 +03:00
Edward ShogulinandEvgenya Stepyreva f57f1fafc7 [LPT] Q/DQ support integration (#3575)
* [ MO ] New INT weights form

	Ticket #34096

	FQ = Quantize -> Dequantize =
	= FQ` -> Convert(int8) -> Convert(src_dtype) -> FQ =
	= Constant(int8) -> Convert(src_dtype) -> FQ =
	= Constant(int8) -> Convert(src_dtype) -> Subtract(zero_point) -> Multiply(scale)

* [LPT] Q/DQ representation support

Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
2021-02-03 17:05:34 +03:00
Anton Chetverikov a4f28ffe3e Fix restoring constant ops with old numbering (#3951)
* Add check to correct restoring constant ops with old numbering

* Resolve comments

* Update comment
2021-01-25 14:41:30 +03:00
Anastasia Kazantaeva afb32262c1 Update OV telemetry message for 2021.3 (#3926)
* Update OV telemetry message for 2021.3

* Fixed missprint
2021-01-21 12:48:33 +03:00
Vladimir Gavrilov 318db6eccc Implement ExperimentalDetectronROIFeatureExtractor operation as nGraph op (#3739)
* Commit.

* Written the header file of the nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Started to write cpp-file for the nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Deleted in_ports_count attribute for the MO operation ExperimentalDetectronROIFeatureExtractor.

* Written validate_and_infer_type() method of ngraph::op::v6::ExperimentalDetectronROIFeatureExtractor.

* Code style fixes.

* Code style fixes.

* Small fixes.

* Code style fixes.

* Now operation ExperimentalDetectronROIFeatureExtractor is read as nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Implemented op::v6::ExperimentalDetectronROIFeatureExtractor::clone_with_new_inputs().

* Added macro NGRAPH_OP_SCOPE to the cpp-file of the nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Fixes in some tests.

* Code style fix.

* Fixed yet another test for ExperimentalDetectronROIFeatureExtractor.

* Added tests for reading ExperimentalDetectronROIFeatureExtractor as operation from opset6.

* Added more test for reading ExperimentalDetectronROIFeatureExtractor as operation from opset6.

* Started to write shape infer tests for the nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Corrected ngraph/test/CMakeLists.txt.

* Small changes.

* Code style fix.

* Small fixes.

* Added ctor of ExperimentalDetectronROIFeatureExtractor with NodeVector as argument.

* Added setting the attribute in_ports_count when the MO operation ExperimentalDetectronROIFeatureExtractor is creating in the MO transformation ONNXMaskRCNNTransformation.

* Added shape infer for the second output of the nGraph operation ExperimentalDetectronROIFeatureExtractor.

* Written shape infer tests for nGraph operation ExperimentalDetectronROIFeatureExtractor (case when input shapes are partially dynamic).

* Small fixes.

* Code style fix.

* Deleted redundant &expected_channels.

* Code style fix.

* Small refactoring.

* Small fixes.

* Small changes.

* Small fixes.

* Deleted attribute distribute_rois_between_levels of ExperimentalDetectronROIFeatureExtractoe.

* Deleted attribute preserve_rois_order of ExperimentalDetectronROIFeatureExtractoe.

* Deleted attribute image_id of ExperimentalDetectronROIFeatureExtractoe.

* Now MO generates attribute 'aligned' of ExperimentalDetectronROIFeatureExtractor only with values 'true' or 'false'.

* Small fix.

* Fix in the conversion of the attribute 'aligned' of MO operation ExperimentalDetectronROIFeatureExtractor to string.

* Tabs were replaced by spaces in some XMLs.

* Tabs were replaced by spaces in ngraph_reshape_tests.cpp.

* Fixed copyrights.

* Applied small patch to IREngine from PR https://github.com/openvinotoolkit/openvino/pull/3814.

* Tabs were replaced by spaces in cnn_ngraph_impl_tests.cpp.

* op::v6::ExperimentalDetectronROIFeatureExtractor::validate_and_infer_types() was rewritten using operator & for channels.

* Added tests for input shapes of op::v6::ExperimentalDetectronROIFeatureExtractor in the case when inputs shapes consist of intervals.

* Fixes in test type_prop.detectron_roi_feature_extractor_intervals.
2021-01-20 16:20:54 +03:00
Anton Chetverikov d462626826 Update MO extensions enabling/disabling mechanism (#3873)
* Add id for NormalizeToNarmalizeL2 transformation

* Update copyright year

* Update extensions enabling/disabling mechanism

* Remove copyright change

* Copyright year

* Update documentation

* Revert missed year in copyright
2021-01-18 10:32:02 +03:00
Evgeny Lazarev c6bfac6e05 Enable TF 2.0 Object Detection API models (#3556)
* Config for TF 2.0 Faster R-CNN models, refactored subgraph_between_nodes to use graph API

* Added support for new type of Preprocessing block in the TF 2.0 OD API models. Various fixes to enable the Faster R-CNN ResNet 50

* Updated text comments

* Fixed sub_graph_between_nodes for TensorIteratorMerge. Added support for the TF 2.X EfficientDet models (not yet reshape-able)

* Fixed unit tests

* Fixed regression for TF 1.X OD API SSD model, enabled TF 2.0 OD API SSD models

* Code clean up

* Switched TF 2.0 OD API Faster R-CNN to preprocessor replacement type 2

* Refactored ObjectDetectionAPIPreprocessorReplacement and ObjectDetectionAPIPreprocessor2Replacement

* Fixed bug in the Div transformation to Mul when input is integer.

* Added support for the TF 2.0 OD API Mask R-CNN

* Added unit tests for Div operation. Updated incorrectly modified mask_rcnn_support_api_v1.14.json

* Updated document with list of supported configuration files for TF OD API models

* Review comments

* Added tests for control flow edges for the sub_graph_between_nodes function

* Two more tests
2020-12-21 14:21:39 +03:00
Anton Chetverikov ab996da912 Update port renumbering logic for constants (#3578)
* Update port renumbering logic for constants

* Resolve comments
2020-12-15 21:39:56 +03:00
Eugeny Volosenkov b3124a5c77 Fix issue with np.expand_dims for numpy<18.x (#3436)
* Fix trouble with np.expand_dims for numpy<18.x

* Delete function expand_dims

* Added additional line
2020-12-03 09:48:17 +03:00
Pavel Esir 1076d32467 [MO] cli_parser fix when input contains substring with matching scale/mean values (#3146)
* fix MO cli_parser when input contains substring with matching scale/mean values

* some additions to cli_parser unit-tests

* fixed numpy array comparisons -- added assert_ prefix

* more general solution for mean/scale cli_parser, names with only digit values are processed correctly

* minor corrections
2020-11-20 11:42:42 +03:00
Vladimir Gavrilov 3aed19a46a XLNET models bugs (#1199)
* Started to write equality comparator for StridedSlice layers.

* Now considered only unique StridedSlice consumers of non-constant data nodes.

* Fixed test for the transformation ConvertGroupedStridedSlice.

* Deleted commented code.

* Small fixes.

* Moved functions unique_by and group_by_with_binary_predicate to mo/utils/utils.py.

* Deleted function collect_sequences.

* Added asserts into the constructor of StridedSlice.

* Written test for the case when 1) there are 4 StridedSlice operations; 2) 2 of StridedSlice have the same data; 3) 2 others StridedSlice have the same data; 4) all StridedSlice operations outputs are consumed by different operations.

* Added some comments.
2020-11-19 10:07:30 +03:00
Roman Kazantsev 10b18a00c6 Improve reshapeability of models with eltwise nodes influencing shapes (#2767)
* Fix ElementwiseInputReshape transformation

Reshape node always needs to be inserted
in order to preserve ShapeOf nodes (reshapability of a model) that can potentially be above
elementwise node.

Refactor EltwiseInputReshape_test and EltwiseInputNormalization_test since the logic of maintaining reshape for eltwise has been changed.

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Merge EltwiseInputNormalization and EltwiseInputReshape transformations

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Remove Unsqueeze from Fused_op

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix code after code review #1

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix code after review #2

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix code review #4

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Perform full normalization based on shapes of all inputs to eltwise

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Refactor much to avoid old API and edges with unsqueeze_dims attribute

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Fix code after review

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2020-11-16 09:50:41 +03:00
Evgeny Lazarev 67e039340e Added support for the Loop operation in the MO IR Reader (#2971)
* Added support for the Loop operation in the MO IR Reader
2020-11-05 20:53:44 +03:00
Sergey Lyubimtsev 93342f0860 [MO] Add CMake install for Model Optimizer (28291) (#2831)
* [MO] Add CMake install for Model Optimizer

* [MO] Update test for version.py

* [MO] Add CMake install for Model Optimizer

* [MO] Update test for version.py

* [MO] Add CMake install for Model Optimizer

* [MO] Update test for version.py

* [MO] fix file permissions for install location
2020-10-30 14:37:06 +03:00
Evgeny LazarevandEvgeny Lazarev cabf8d8534 ONNX Loop operation support (#2756)
* Generate TensorIterator without back edges from TensorFlow models

* Added a check in the MarkSubgraphsWithCorrectLayout to not fail when port is not connected

* Updated the 'protobuf2nx' to consume the graph protobuf message

* Cleanup TI from the IRv7 specific code

* Do not run some front transformations recursively

* Draft support for the ONNX Loop operation when 'cond' = True

* LoopToTI transformation changes

* Added draft of Loop operation and parser for ONNX Loop operation body

* Updated Loop body parser + added shape and type infer for the Loop operation

* Fixes for ONNX Loop operation parser

* Moved Loop parsing to Loop op extractor. Added generation of external edges for the Loop body ops

* Added support for ThresholdedRelu using decomposition

* Added support for Min ONNX operation

* Draft fixes for port_map generation for the Loop

* Rename transformation file and fix BOM

* Fixed shape inference for Loop scan outputs (axis is not None)

* Fixed shape inference for ONNX Loop operation

* Refactor checks in the TensorIteratorMerge transformation

* Code refactoring. Enabled commented transformations

* Documentation update for ONNX Loop, ThresholdedRelu and Min

* Fixed typo in the Loop front transformation where execution condition input is connected. Other refactorings

* Fixed in the Loop extractor

* Added printing 'internal_layer_id' attribute in the graph dumper

* Updated calculation of iterations number for the Loop

* Added missing code

* Fixed output port shapes and types generation for Loop operation

* Update function names and variable names in the Loop operation

* Fixed type inference for iteration count input

* Added removal of input/output ports of the Loop if they are not used

* Fixed renumbering Loop operations input/output ports to keep mandatory

* Fixed ThresholdedReluDecomposition transformation

* Updated MO IR Reader to know about Loop operation. But it is still not supported by the MO IR Reader

* Added unit test for Slice op shape infer (reverse the sequence of elements)

* Reverted changes in the ONNX loader function call to protobuf2nx

* Enable Reshape0DToSqueeze transformation recursively

* Refactored Loop operation support implementation

* Changed ThresholdedReluDecomposition to generate Const with shape [1] instead of scalar

* Code style and wording fixes

* Restored accidentally removed 'return' statement in the TI shape infer function

* Fixed comments

* Fixed comment

Co-authored-by: Evgeny Lazarev <elazarev.nnov@gmail.com>
2020-10-27 23:04:43 +03:00
44406691e5 Add Round-5 operation (#2328)
* Add Round-5 operation

* Add ONNX Round to supported operation list

* Add ngraph implementation for Round operation

* Update MO part

* Create UnaryElementwise class, update Round Operation

* Fix mode attr in mxnet extractor

* Add tests for Round shape infer

* Update 'enable' attr

* Update MO IR Reader to support UnaryElementwise operations

* Minor test refactor

* Update ngraph Round operation

* Add reference implementation

* Add test for reference implementation

* Add test for shape infer

* Add test for IE IR Reader

* AddRound operation to python api

* Fix missed mode attr

* Update Round operation version

* Fix codestyle

* Add MxNet Round to supported layers list

* Fix error in reference

* Fix comments style

* Update CMake file

* Update Ngraph reference test

* Update IE IR Reader tests

* Return v0::Round operation

* Update shape infer tests

* Fix v0::Round reference

* Fix codestyle

* Enum instead of string

* Fix codestyle

* Add Mode attribute adapter

* Update Mode attr

* Fix reference for v0::Round

* Fix codestyle

* Fix mode attr

* Fix get() method

* Fix codestyle in python api

* Update test info

* Fix ngraph api part

* Ad round v5 to interpreter tests

* Fix codestyle is ie reader test

* Update ngraph python api __init__.py file

* Adde opser5 to dafault opsets in ie_ir reader

* Add parser for Round layer

* Remove redundant spaces

* Add round creator to appropriate list

* Remove redundant import

* Commit to bump infrastructure version

I'm sorry for this, but this commit will be squashed on merge to master anyway and it is needed for your PR to correctly pass the pipeline

* Fix import

* fix codestyle

* Fix ngraph api part

* Add shape infer tests in python api

* Add .upper() for mode attr

* Refactor MO shape infer test for Round op

* Update tests and add comments

* Revert "Commit to bump infrastructure version"

This reverts commit 56e6ae1e4c.

* remove parser for Round layer

* Update Ronund-5 evaluate test

* Resolve review comments

Co-authored-by: User <user@nnlvdp-achetver.inn.intel.com>
Co-authored-by: Andrey Babushkin <andrey.babushkin@intel.com>
Co-authored-by: Anton Chetverikov <anton.chetverikov@.intel.com>
2020-10-20 18:36:19 +03:00
Evgeny Lazarev d86019d104 Leaky relu transformation refactor (#2640)
* Refactored LeakyRelu transformation

* Added unit test for LeakyRelu transformation + removed duplicate test function valued_const
2020-10-14 16:43:29 +03:00
Roman Donchenko 0e502ffb35 [MO] Fix many spelling and grammar errors (#2543) 2020-10-09 12:16:12 +03:00
Pavel Esir 9149d899b3 [MO] Fix: add PermuteInputs to shape broadcasting (#2419)
* fix: add PermuteInputs to shape broadcasting

* fix type declaration typo
2020-10-05 12:48:57 +03:00
Zoe Cayetano 62a33f7f4b Update get_ov_update_message.py (#2499) 2020-10-02 23:19:54 +03:00
Pavel EsirandEvgenya Stepyreva 1c97b13e9a [MO] Add explicit broadcasting mode (#2077)
* [MO] Add explicit broadcasting mode

* corrected value broadcast, added nontrivial test cases

* added negative unit-tests

* added axes normalization and permute inputs

* corrected axes normalization

* Update model-optimizer/mo/utils/broadcasting.py

Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
2020-09-25 13:26:47 +03:00
Evgenya StepyrevaandAndrey Zaytsev 1007b05104 [DOC] Reshape feature (#2023)
* [DOC] Reshape feature

* Comments adressed.

* Clarifications

* Converting_Model_General.md

* Update ShapeInference.md

* Update ShapeInference.md

* Comments

* Update Convert_Object_Detection_API_Models.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2020-09-11 21:41:42 +03:00
Roman Kazantsev 82e15a5a64 Support python 3.8 by the Model Optimizer tool in default configuration (#2078)
* Support python 3.8 by the Model Optimizer tool in default configuration

* Fix after review #1

* Fix after the second round review
2020-09-09 08:34:43 +03:00