Files
openvino/model-optimizer
20bf6ca6c4 Implement TelemetryExtension and enable op statistic collection in ONNX/Paddle/TF FrontEnds (#8666)
* Moved so loader to utils

* Fixed extension tests

* Fixed tests and style

* Fixed style and tests

* Fixed ARM build

* Fix windows

* Fix ieFuncTests

* Wrap runtime exception

* Fixed tests

* Added separate new extension

* Fixed unicode extension loading

* Try to fix windows

* Fixed windows

* Fixed macro

* Fixed doc

* Fixed build

* Fixed comments

* Try to fix build

* Fixed build

* Fixed build

* Fixed shared_from_this

* Temp commit

* Changed extension

* Fixed merge conflicts

* Removed ngraph namespace from new extensions

* Fixed code style

* Added core add_extension methods and tests

* Added new tests

* Implement tile operation

* Enabled new extensions support

* Fixed build

* Fixed code style

* Try to fix windows

* Changed base extension class

* Removed redundant Ptr

* Fixed comments

* Fixed friend decl

* Fixed Windows export

* Fixed centos

* Added template add_extension method

* WIP: experiments with FE extensions, python callback in C++ code, draft of transformation, op and telemetry extensions

* Move destructor to public

* WIP: merge with core extension WIP branch

* Temporary WA for defualt ctor in VariantTemp. TODO: revert back before merging to master

* Removed BaseExtension class

* Added variadic add_extension methods

* Fixed doc and typo

* Added BaseOpDestructor

* Allow to create new extension only for new operations

* Revert tests

* Fixed comments

* Fixed comments

* WIP with various types of extensions, finalizing merge

* Fixed merge artefacts and finalized Json extensions

* TelemetryExtension initial commit

* Telemetry tests

* Add telemetry to frontends

* delete debug prints

* missprint

* Resolve review comments

* update tests

* fix tests

* Collect op statistics in InputModel for PDPD, TF; Update tests

* delete debug prints, codestyle

* fix paddle frontend

* Add event_category to TelemetryExtension ctor, update event_send calls, update tests

* Add convertion_method Temetry event to MO

* pass TelemetryExtension to ONNX frontend

* Fix default argument

* Clean up

* delete unnecessary m_telemetry members

* fix windows build

* fix win build again

* try to link frontend manager statically to ONNXFrontEnd

* use static frontend_manager in unit tests

* fix win build: split declaration and definition of TelemetryException

* Resolve review remarks

* send Telemetry events for subgraph ops in ONXX, fix unit tests

* fix build

* add missign files

* resolve review comments, fix centos

* Add blank lines

* Set correct TID for Telemetry

* resolve review commetns

Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Lyalin, Sergey <sergey.lyalin@intel.com>
2021-12-01 09:51:00 +03:00
..
2021-03-22 19:35:32 +03:00
2020-04-15 21:46:27 +03:00
2021-09-15 16:49:11 +03:00
2021-07-22 21:12:44 +03:00

Prerequisites

Model Optimizer requires:

  1. Python 3 or newer

  2. [Optional] Please read about use cases that require Caffe* to be available on the machine in the documentation.

Installation instructions

  1. Go to the Model Optimizer folder:
    cd PATH_TO_INSTALL_DIR/tools/model_optimizer
  1. Create virtual environment and activate it. This option is strongly recommended as it creates a Python sandbox and dependencies for the Model Optimizer do not influence global Python configuration, installed libraries etc. At the same time, special flag ensures that system-wide Python libraries are also available in this sandbox. Skip this step only if you do want to install all Model Optimizer dependencies globally:

    • Create environment:
          virtualenv -p /usr/bin/python3.6 .env3 --system-site-packages
        
    • Activate it:
        . .env3/bin/activate
      
  2. Install dependencies. If you want to convert models only from particular framework, you should use one of available requirements_*.txt files corresponding to the framework of choice. For example, for Caffe use requirements_caffe.txt and so on. When you decide to switch later to other frameworks, please install dependencies for them using the same mechanism:

    pip3 install -r requirements.txt
    

    Or you can use the installation scripts from the "install_prerequisites" directory.

  3. [OPTIONAL] If you use Windows OS, most probably you get python version of protobuf library. It is known to be rather slow, and you can use a boosted version of library by building the .egg file (Python package format) yourself, using instructions below (section 'How to boost Caffe model loading') for the target OS and Python, or install it with the pre-built .egg (it is built for Python 3.4, 3.5, 3.6, 3.7):

         python3 -m easy_install protobuf-3.6.1-py3.6-win-amd64.egg
    

    It overrides the protobuf python package installed by the previous command.

    Set environment variable to enable boost in protobuf performance:

         set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp
    

Setup development environment

How to run unit-tests

  1. Run tests with:
    python -m unittest discover -p "*_test.py" [-s PATH_TO_DIR]

How to capture unit-tests coverage

  1. Run tests with:
    coverage run -m unittest discover -p "*_test.py" [-s PATH_TO_DIR]
  1. Build html report:
    coverage html

How to run code linting

  1. Run the following command:
    pylint mo/ extensions/ mo.py