Files
openvino/docs/Inference_Engine_Development_Procedure/IE_Dev_Procedure.md
Andrey Zaytsev 940eb43095 Feature/azaytsev/merge to master (#2786)
* [IE CLDNN] Memory allocation optimizations (#2178)

* [GNA] Safety fixes (#2193)

* LSTMCell test [GNA] LSTMCell fix for GNA (#2216)

* [GNA] fix scale factor calculation for unfused bias after fc (2021.1) (#2195)

* [GNA] fix scale factor calculation for unfused bias after fc

* change check

* add test

* apply requested changes

* cpplint fix

* apply test changes

* modify model for test to match ::op::

* [LPT] Copy constant with several outputs before blob update (#2197)

* [LPT] Copy constant implementation

* [LPT] the same Constant ops as FQ interval boundaries

* [Scripts] Fixing issue with exporting path-like env when it undef  (#2164)

* setupvars.sh: Added logic for exporting path env in case if it not defined

* setupvars: Removed duplicated colon

* Kept quotes where they were

* setupvars: updated copyrights

* FakeQuantize + Mul fusion (#2133)

* FQ+Mul fusion transform skeleton

* FQ+Mul fusion transform tests prep

* Basic UT for the transform

* Basic implementation of the transform

* Parametrized UTs for FQMul transform

* Parametrization of FQ+Mul UTs

* Make sure that the shapes of constants match

* Check if the mul constant matches FQ data

* CentOs compilation error fix

* PR feedback and adjusted tests

* NHWC layout of the mul constant

* UT: FQ output limits 4D

* Redundant CF pass removed

* Rewrite the graph in a different way

* Shape checking infrastructure skeleton

* Handle some negative cases

* Check the rt info in the fusion test

* Fuse all Mul nodes detected after FQ node

* Dont cast the original FQ node

* Dont throw if CF fails in new output range calculation

* More UTs

* Accept any type of input to FQ in the transformation

* Test the fusion when all FQ inputs are non-const

* Fusion test when only one output limit is const

* Extend error message (#2174)

* some nGraph KW fixes (#2176)

* Removed redundant methods

* Fixed KW for linux

* Fix QueryNetwork for networks with KSO (#2202)

* Added a test to reproduce QueryNetwork with KSO

* Fixed QueryNetwork for networks with KSO

* Added additional test

* Fixed output names for case with redundant ops before result (#2209)

* [IE][VPU]: Workaround to support parameter Beta for layer Swish (#2207)

* Workaround to full support Swish layer. It is faster than native Swish for now.

* [IE][VPU]: Remove the second call of ngraph::CommonOptimizations (#2221)

* Remove the second call of ngraph::CommonOptimizations in myriad plugin
* Reuse code with vpu ngraph transformations

* Duplicate PR 2167 for release branch: GatherTree description was extended and outdated link fixed (#2235)

* add more alrifications to description

* move clarification to comment

* pseudo code become more accurate

* review changes

* Add exposing function signatures via Cython (#2244)

* [DOC] Reshape feature (#2194)

* [IE][VPU][OpenCL] 2021.1 release compiler (#2189)

* Statically analyzed issues. (#2261)

* [IE][VPU]: Fix K propagation through Reshape (2021.1) (#2180)

* Fix K propagation through Reshape
* Add test cases

* Revert "[IE TESTS] dynavic batch for mvn layer (#1010)" (#2256)

This reverts commit 2e3378c50f.

* Fixed KW warning and review issues (#2262)

* [IE][VPU]: update firmware 1381 (#2236)

* Reverting devicePriorities to be vector and respect the order, as opposed to the incorrect (recent?) refactoring that introduced the unordered_map that effectively ignores the priorities (#2251)

* update OpenCV version to 4.5.0 (#2260)

* Add VPUX configuration to compile_tool (#2248)

* [IE][TESTS] Fix compareRawBuffers and compareBlobData methods (#2246)

Use `<=` comparison instead of `<` with thresholds.
This allows to use `0` threshold for bit-exact comparison.

* [IE][VPU]: KW fixes (#2186)

* Some KW fixes
* Fix printTo in vpu ngraph transformations

* Fix for static PartialShape detection algorithm (#2177)

* Fixes for Interpolate-4. (#2281)

* Update get_ov_update_message.py (#2286)

* Clone a specific tag for pybind11 (#2296)

* [Scripts] Fix setting PYTHONPATH logic (#2305)

* setupvars.sh: Added logic for exporting path env in case if it not defined

* setupvars: Removed duplicated colon

* install_openvino_dependencies: Updated copyrights

setupvars.bat: Updated notification about incorrect Python version. Removed checking ICC2019
setupvars.sh: Removed logic with choosing higher version of installed Python. Added dynamic detecting python3 major and minor version for setting path. Add checking minimum required Python version(now 3.6)

* Added python3-gi package and fixed libglib2.0-0 package location. (#2294)

* [IE TESTS] CoreThreading_LoadNetwork tests were disabled for GPU plugin (#2245) (#2283)

* setupvars: Updated notifications, fixed calling python in Windows case (#2318)

* Updated operations specification documents (2021.1) (#2268)

* Updated documentation structure and remove incorrect added files for Acosh-1, Asinh-1 and Atanh-1

* Fixed broken links

* Fixed c samples build (#2278) (#2304)

* Fixed c samples build

fixed CVS-38816 - Failure to build samples in C

* Fixed issue with gflags

* Revert "[IE][VPU]: Fix K propagation through Reshape (2021.1) (#2180)" (#2322)

This reverts commit d604a03ac0.

* Added ONNX Resize-11 and ONNX Resize-13 to supported frameworks layers list. (#2325)

* Implement `run_executable.py` to run `TimeTests` several times (#2125) (#2188)

CI passed

* install_NEO_OCL_driver: Updated exit codes, messages. Updated way to remove old driver on Ubuntu (#2333)

* Bump cmake version to 3.13 (#2339)

* install_NEO_OCL_driver: Added checking of installed packages before trying to remove them. Added quotes for echo. (#2350)

* convert to doxygen comments

* add doxygen doc build configurations (#2191)

Co-authored-by: Nikolay Tyukaev <ntyukaev_lo@jenkins.inn.intel.com>

* [DOCS] Added an evaluate method for custom operation (#2272)

* Added an evaluate method for custom operation

* Fixed comments

* Downgrade cmake for samples (#2372)

* Downgrade cmake for samples

Downgraded cmake version to default version for Ubuntu 18.04

* Updated supported python version

The minimal python version in 2021.1 is 3.5

* Added notes about cmake requirements for samples and demo

* Install dependency refactoring. (#2381)

* Updated Transformation development doc (#2370)

* Delete xfail for resolved known issue (#2385)

* Fix layout links for dl streamer and c api (#2375)

* fix layouts

* change the dl-streamer link

Co-authored-by: Nikolay Tyukaev <ntyukaev_lo@jenkins.inn.intel.com>

* Added link options for cross-compilation (#2397)

* Added new GSG for macOS, made minor changes in Windows GSG (#2070) (#2405)

* Added new GSG for macOS, made minor changes in Windows GSG

* Update get_started_macos.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Fixed docs build on Windows (#2383)

* layouts and code comments

* Replace absolute links to docs.openvinotoolkit.org by relative ones (#2439)

* Replaced direct links to docs.openvinotoolkit.org with relative links

* Replaced direct links to docs.openvinotoolkit.org with relative links. Added GSGs for Win and macOS

* Minor fixes in GSGs

* Replaced direct links to docs.openvinotoolkit.org with relative links

* Removed links to OpenVINO markdown files that contain anchor - they don't work in the current implementation of the doc process

* Fixed Notes

* Removed links to OpenVINO markdown files that contain anchor - they don't work in the current implementation of the doc process

* fixed link to installing-openvino-linux.md

* Update the menu to align with POT doc headers (#2433)

* Update the menu to align with POT doc headers

It changes the menu to align with Post-training Optimization Toolkit documentation titles.

* Corrected one title

Run Examples => How to Run Examples

* Added closing braсket (#2466)

Fixed syntax error (b4b03b1)

* Remove the deprecation notice (#2314)

* Removed deprecation notice

* Removed the note from other files

* [DOCS] Update Installation Guide - GPU steps (#2308)

* Initial commit

* fixing lists

* Update installing-openvino-linux.md

* Get rid of the note

* Added the scrrenshot

* Update installing-openvino-linux.md

* fixes

* separate layout

* [Docs] Update MO What's new description (#2481)

* Azure CI: Add separated pipelines for Windows, Linux, Mac

* Feature/azaytsev/benchmarks 2021 1 (#2501)

* Initial changes for 2021.1

* Inserted Graphtool scripts, updated configurations info

* Updated FAQ and minor changes to performance_benchmarks.md

* Updated for 2021.1

* Updated

* incorporated review comments

* incorporated review comments for FAQ

* fixed link

* Update build-instruction.md for MacOsX (#2457)

* Update build-instruction.md for MacOsX

* Removed call of install_dependencies.sh from the steps

* Changed layouts

* Feature/azaytsev/cvs-38240 (#2469)

* Updated for 2020 version, replaced Ubuntu 16.04 with Ubuntu 20.04

* Updated the release package numbers

* Removed FPGA from the documentation

* Updated according to the comments in the ticket CVS-37827 (#2448)

* Updated according to CVS-38225

* some changes

* Update docs for speech libs and demos (#2518)

* Made changes to benchmarks according to review comments

* Remove `--collect_results_only` (#2523)

* Remove `--collect_results_only` from MemCheckTests

* Remove CLI keys from README

* Added logo info to the Legal_Information, updated Ubuntu, CentOS supported versions

* Updated supported Intel® Core™ processors list

* Fixed table formatting

* [Jenkinsfile] Bump infra (#2546)

* [GNA] Documentation updates for 2021.1 (#2460)

* [GNA] Documentation updates for 2021.1

* Take Mike's comments into account

* More fixes according to review

* Fix processor generation names

* update api layouts

* Added new index page with overview

* Changed CMake and Python versions

* Fixed links

* some layout changes

* some layout changes

* nGraph Python API tutorial (#2500)

* nGraph Python API tutorial

* Tweaks

* Code review comments

* Code review comments

* some layout changes

* COnverted svg images to png

* layouts

* update layout

* Added a label for nGraph_Python_API.md

* fixed links

* Fixed image

* First draft of nGraph documentation (#2271)

* First draft of nGraph documentation

* updated according to review comments

* Updated

* Reviewed the nGraph Transformation section, added missing images

* Update nGraph_dg.md

* Delete python_api.md

Removed since there is already the nGraph_Python_API.md document with a comprehensive overview.

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
Co-authored-by: CCR\avladimi <anastasiya.ageeva@intel.com>

* Feature/azaytsev/docs 2021 1 (#2560)

* Removed FPGA from the documentation

* Updated according to CVS-38225

* Added logo info to the Legal_Information, updated Ubuntu, CentOS supported versions

* Updated supported Intel® Core™ processors list

* Added new index page with overview

* Changed CMake and Python versions

* Fixed links

* COnverted svg images to png

* Added a label for nGraph_Python_API.md

* fixed links

* Fixed image

* Update SW requirements in build instructions and change latest release to 2021.1 (#2565)

* removed links to ../IE_DG/Introduction.md

* Removed links to tools overview page as removed

* some changes

* Remove link to Integrate_your_kernels_into_IE.md

* remove openvino_docs_IE_DG_Graph_debug_capabilities from layout as it was removed

* Fixed links to images (#2569)

* update layouts

* Added deprecation note for PassConfig class (#2593)

* Post-release fixes and installation path changes

* Added pip install documentation (#2465)

* Added pip install documentation

* Change references

* tiny fixes of links

* Update installing-openvino-pip.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update OpenVino ONNX CI check (#2599)

* Update OpenVino ONNX CI

* Change parallel execution to single

* Enlarge timeout

* Remove timeout

* Add timeout to test execution

* Added PIP installation and Build from Source to the layout

* Fixed formatting issue, removed broken link

* Renamed section EXAMPLES to RESOURCES according to review comments

* add mo faq navigation by url param

* Skip hanging test case of OpenVino ONNX CI (#2608)

* Update OpenVino ONNX CI

* Change parallel execution to single

* Enlarge timeout

* Remove timeout

* Add timeout to test execution

* Skip hanging test

* Add description to skip issue

* Removed DLDT description

* Replaced wrong links

* MInor fix for path to the cpp samples

* fixes

* Update ops.py

* Fix style

* Improve pip installation guide (#2644)

* Improve pip installation guide

* Updated after comments

* Feature/ntyukaev/separate layout (#2629)

* convert to doxygen comments

* layouts and code comments

* separate layout

* Changed layouts

* Removed FPGA from the documentation

* Updated according to CVS-38225

* some changes

* Made changes to benchmarks according to review comments

* Added logo info to the Legal_Information, updated Ubuntu, CentOS supported versions

* Updated supported Intel® Core™ processors list

* Fixed table formatting

* update api layouts

* Added new index page with overview

* Changed CMake and Python versions

* Fixed links

* some layout changes

* some layout changes

* some layout changes

* COnverted svg images to png

* layouts

* update layout

* Added a label for nGraph_Python_API.md

* fixed links

* Fixed image

* removed links to ../IE_DG/Introduction.md

* Removed links to tools overview page as removed

* some changes

* Remove link to Integrate_your_kernels_into_IE.md

* remove openvino_docs_IE_DG_Graph_debug_capabilities from layout as it was removed

* update layouts

* Post-release fixes and installation path changes

* Added PIP installation and Build from Source to the layout

* Fixed formatting issue, removed broken link

* Renamed section EXAMPLES to RESOURCES according to review comments

* add mo faq navigation by url param

* Removed DLDT description

* Replaced wrong links

* MInor fix for path to the cpp samples

* fixes

* Update ops.py

* Fix style

Co-authored-by: Nikolay Tyukaev <ntyukaev_lo@jenkins.inn.intel.com>
Co-authored-by: Tyukaev <nikolay.tyukaev@intel.com>
Co-authored-by: aalborov <alina.alborova@intel.com>
Co-authored-by: Rafal Blaczkowski <rafal.blaczkowski@intel.com>
Co-authored-by: Alexander Zhogov <alexander.zhogov@intel.com>

* Fixed CVS-35316 (#2072) (#2670)

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* [install_dependencies.sh] install latest cmake if current version is lower 3.13 (#2695) (#2701)

* [install_dependencies.sh] install latest cmake if current version is lower 3.13

* add shellcheck for Ubuntu

* install python 2.7 for Ubuntu

* Removed redundant file

* Exclude files that we didn't changed from merging

Co-authored-by: Sergey Shlyapnikov <sergey.shlyapnikov@intel.com>
Co-authored-by: Denis Orlov <denis.orlov@intel.com>
Co-authored-by: Kamil Magierski <kamil.magierski@intel.com>
Co-authored-by: Anna Alberska <anna.alberska@intel.com>
Co-authored-by: Edward Shogulin <edward.shogulin@intel.com>
Co-authored-by: Artyom Anokhov <artyom.anokhov@intel.com>
Co-authored-by: Tomasz Dołbniak <tomasz.dolbniak@intel.com>
Co-authored-by: Ilya Churaev <ilya.churaev@intel.com>
Co-authored-by: Roman Vyunov (Intel) <roman.vyunov@intel.com>
Co-authored-by: Maksim Doronin <maksim.doronin@intel.com>
Co-authored-by: Svetlana Dolinina <svetlana.a.dolinina@intel.com>
Co-authored-by: Evgeny Talanin <evgeny.talanin@intel.com>
Co-authored-by: Evgenya Stepyreva <evgenya.stepyreva@intel.com>
Co-authored-by: Maxim Kurin <maxim.kurin@intel.com>
Co-authored-by: Nikolay Shchegolev <nikolay.shchegolev@intel.com>
Co-authored-by: Andrew Bakalin <andrew.bakalin@intel.com>
Co-authored-by: Gorokhov Dmitriy <dmitry.gorokhov@intel.com>
Co-authored-by: Evgeny Latkin <evgeny.latkin@intel.com>
Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com>
Co-authored-by: Alexey Suhov <alexey.suhov@intel.com>
Co-authored-by: Alexander Novak <sasha-novak@yandex.ru>
Co-authored-by: Vladislav Vinogradov <vlad.vinogradov@intel.com>
Co-authored-by: Vladislav Volkov <vladislav.volkov@intel.com>
Co-authored-by: Vladimir Gavrilov <vladimir.gavrilov@intel.com>
Co-authored-by: Zoe Cayetano <zoe.cayetano@intel.com>
Co-authored-by: Dmitrii Denisov <dmitrii.denisov@intel.com>
Co-authored-by: Irina Efode <irina.efode@intel.com>
Co-authored-by: Evgeny Lazarev <evgeny.lazarev@intel.com>
Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
Co-authored-by: Vitaliy Urusovskij <vitaliy.urusovskij@intel.com>
Co-authored-by: Nikolay Tyukaev <ntyukaev_lo@jenkins.inn.intel.com>
Co-authored-by: Nikolay Tyukaev <nikolay.tyukaev@intel.com>
Co-authored-by: Gleb Kazantaev <gleb.kazantaev@intel.com>
Co-authored-by: Rafal Blaczkowski <rafal.blaczkowski@intel.com>
Co-authored-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
Co-authored-by: Maksim Proshin <mvproshin@gmail.com>
Co-authored-by: Alina Alborova <alina.alborova@intel.com>
Co-authored-by: Maxim Vafin <maxim.vafin@intel.com>
Co-authored-by: azhogov <alexander.zhogov@intel.com>
Co-authored-by: Alina Kladieva <alina.kladieva@intel.com>
Co-authored-by: Michał Karzyński <4430709+postrational@users.noreply.github.com>
Co-authored-by: Anton Romanov <anton.romanov@intel.com>
2020-10-27 00:41:46 +03:00

7.9 KiB

Development Flow for Adding New Changes to the Inference Engine

Develop Your Feature

  1. Create a branch based on the latest version of a target branch (master or release). Use the following branch naming:
    • feature// - stands for temporary work area for creating a feature or performing bug fixes
    • scout/ - is used for shared development if several developers work on one feature IMPORTANT: Do not use long branch name, because it may lead to failed CI jobs on Windows. Name length must be less 40 characters.
  2. Commit changes on your branch and push it to remote.

Create a Merge Request

  1. Go to the GitLab\* merge request page

  2. Create a merge request by pressing on New Merge Request or Create merge request. Choose an inference-engine project from the drop-down list.
    mr1

    a. Fill the Title and Description fields with meaningful information. This information should be enough to understand the changes you made:

    Use this template for the Title field. If you did not finish your work on current bug/feature, add [WIP] to the beginning of the title:

    [Domain] Small description for merge request (can use first string from the commit message)
    

    Use domain from the following list:

    • [IE COMMON] - if a solution impacts common Inference Engine functionality
    • [IE EXTENSION]
    • [IE PYTHON]
    • [IE SAMPLES]
    • [IE TESTS]
    • [IE DOCS]
    • [IE MKLDNN]
    • [IE GNA]
    • [IE CLDNN]
    • [IE MYRIAD]
    • [IE HDDL]
    • [IE HETERO]

    You can use several domains in one commit message. For example: [COMMON][MKLDNN]

    Use this template to fill Description field:

    <Multi-line description of the commit>
    
    JIRA: <url_to_jira_ticket>
    CI: <url_to_validation_result_in_CI>
        <url_to_passed_rebuilt_changes_in_CI>
    

    b. Add Milestone and Labels to the MR if it is possible.

    c. If your work is finished, assign the MR to a reviewer. If it is in progress, assing the MR to yourself ([WIP] case).

    Example of an MR:
    mr_example1

    Note

    : When creating an MR, please remember that even a person who does not know what this feature/merge request stands for >should be able to understand the reasons why this feature/change was created.

  3. Change the history of your branch:

    a. Squash commits on your branch into one (you can have some number of logically separated commits if it is needed). Use the following template for commit message:

    ```
    [Domain] Small description of commit
    
    Multiline commit additional comments:
    * additional comment 1
    * additional comment 2
    
    JIRA: CVS-xxxx
    ```
    

    Example of the commit message:
    commit_message

    Note

    : if there is no JIRA ticket for your task, you can leave this line empty.

    b. Rebase your branch to the latest version of the target branch. The difference between the last validated version of the branch must be no more than one day.

    c. Push your updates to the remote branch. You can use force push if needed, for example, when you rebase your branch to latest version of the target branch.

  4. Add required people as reviewers of the MR. Please list appropriate GitLab accounts in the discussion section of the MR.

  5. Run validation build (refer to the "Run tests under CI")

  6. When review is complited and you have one or several Thumb up in your MR: a. Make sure that all comments are fixed and all discussions are resolved b. Run validation build again, make sure that all tests are passed, and update link to CI in the MR description c. Create cherry-picked MR if needed (refer to the "Create a Cherry-Picked MR (Optional)") and validate it by CI.

    Example of an MR:
    mr_example

  7. Reassign the MR to the someone from integration managers.

  8. An integration manager will close or merge your MR.
    mr2

Create a Cherry-Picked MR (Optional)

  1. If you need to merge your changes in both target branch (release and master), create new merge request to another target branch containing cherry-pick with approved commit. Follow the rules above to create the MR (sections 1 and 2 in (Create a Merge Request)[#create-a-merge-request]), but add line "**cherry-picked from MR: !xxxx**" to the MR description and assign the MR to someine from the integration managers.

  2. Run validation build (refer to "Run tests under CI" section).

  3. Assign the MR to an integration manager.

  4. The integration manager will merge or close your MR.

Run Tests under CI

  1. Go to the CI page: TeamCity

  2. Click the Run Engineering Validation (if you want to merge your changes into master branch) or Run Engineering Validation for XXXX RX Release branch (if you want merge changes into release branch):
    run_engineering

  3. Select your branch in the top left corner:
    select_branch

  4. If you have not committed anything to your branch for the past several days, you might not see your branch in the list. In this case, you can choose it in the properties of Run Engineering Validation task (if you want merge changes into master branch) or Run Release Validation task (if you want merge changes into release branch). Click on Run button. On the third tab, choose your branch and click the Run Build:
    run_tests

  5. Click on an arrow right after 1 queued. On a new dialog window, click on a build number. In this case, it is 1825:
    view_results

    You will see the current status of tests:
    analyze_results

  6. Make sure that all tests are passed. If some test failed, see build log in TeamCity: choose failed build in dependencies, click on the result and go to Build log tab.

    • If it looks like an infrastructure issue (for example, an absence of a software or a network issue), restart the build.
    • If you think that your changes could not break the test, rebase your branch on latest version of master restart build. If the build failed again, explore build history: an incorrect code might have merged into the target branch before branching that is not fixed till current moment.
    • If you have an issue with code style, run code style checks <IE_repo>/scripts/run_code_check.sh locally to analyze the reason of failed CI jobs (see the picture below) and restart them if required:
      code_style_artifacts

    Commit your changes.

    Please add link to restarted build in MR description

Merge Changes to a Target Branch (master or release branches)

  1. The master and release branches are protected. Only integration managers can merge to these branches

  2. Assigned integration manager checks if all the requirements are met. If so, they can merge MR with the Merge button or manually.

  3. An integration manager removes a branch if an MR author has set an appropriate flag for this MR in GitLab GUI.