Compare commits

..
Author SHA1 Message Date
Andrey Zaytsev ba35364a53 Feature/benchmarks 2021 3 ehl (#5191)
* Added EHL config

* Updated graphs

* improve table formatting

* Wrap <iframe> tag with \htmlonly \endhtmlonly to avoid build errors

* Updated graphs

* Fixed links to TDP and Price for 8380
2021-04-12 15:38:07 +03:00
Anton Chetverikov 1c84064e06 Add specification for ExperimentalDetectron* oprations (#5128) 2021-04-07 14:01:08 +03:00
Nikolay Tyukaev c370284bc4 fix doc iframe issue - 2021.3 (#5090)
* wrap with htmlonly

* wrap with htmlonly
2021-04-06 11:34:10 +03:00
Dmitry Kurtaev 81b0aec201 Enable mo.front.common.extractors module (#5038)
* Enable mo.front.common.extractors module (#5018)

* Enable mo.front.common.extractors module

* Update package_BOM.txt

* Test MO wheel content
2021-04-01 13:54:09 +03:00
Alexey Suhov f37e14c614 update OpenCV version to 4.5.2 (#5069)
* update OpenCV version to 4.5.2
2021-04-01 13:30:03 +03:00
Sergey Lyubimtsev 7fe8264703 compression.configs.hardware config to package_data (#5066) 2021-04-01 11:34:59 +03:00
Andrey Zaytsev a5cfe0ecb2 fixes for graphs (#5057) 2021-03-31 16:21:09 +03:00
Andrey Zaytsev 22cf9efcdc Feature/doc fixes 2021 3 (#4971)
* Made changes for CVS-50424

* Changes for CVS-49349

* Minor change for CVS-49349

* Changes for CVS-49343

* Cherry-pick #PR4254

* Replaced /opt/intel/openvino/ with /opt/intel/openvino_2021/ as the default target directory

* (CVS-50786) Added a new section Reference IMplementations to keep Speech Library and Speech Recognition Demos

* Doc fixes

* Replaced links to inference_engine_intro.md with Deep_Learning_Inference_Engine_DevGuide.md, fixed links

* Fixed link

* Fixes

* Fixes

* Reemoved Intel® Xeon® processor E family
2021-03-25 21:31:29 +03:00
Alina Alborova 1fdc9e372f [49342] Update recommended CMake version on install guide in documentation (#4763)
* Inserted a disclaimer

* Another disclaimer

* Update installing-openvino-windows.md

* Update installing-openvino-windows.md

* Update installing-openvino-windows.md
2021-03-24 15:03:41 +03:00
Alexey Suhov 18e83a2177 [README.md] change latest release to 2021.3 2021-03-23 22:02:53 +03:00
Alina Alborova ef892a01df DevCloud call outs (#4904) 2021-03-23 18:09:30 +03:00
7965797103 Clarification of Low Latency Transformation and State API documentation (#4877)
* Assign/ReadValue, LowLatency and StateAPI clarifications

* Apply suggestions from code review: spelling mistakes

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* fixed wording

* cherry-pick missing commit to release branch: low latency documentation

* Resolve review remarks

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
Co-authored-by: Svetlana Dolinina <svetlana.a.dolinina@intel.com>
2021-03-23 16:23:16 +03:00
Andrey Zaytsev 28be0a0452 Feature/ovsa docs 2021 3 (#4914)
* Updated to 2021-3, fixed formatting issues

* Fixed formatting issues

* Fixed formatting issues

* Fixed formatting issues

* Update ovsa_get_started.md
2021-03-23 16:12:32 +03:00
Alina Alborova 8db14b987c [cherry-pick] Extensibility docs review (#4915) 2021-03-23 15:25:30 +03:00
Andrey Zaytsev 9910725a7b Feature/omz link changes (#4911)
* Changed labels for demos and model downloader

* Changed links to models and tools

* Changed links to models and tools

* Changed links to demos
2021-03-23 01:00:17 +03:00
Andrey Zaytsev c72f2ef88c Feature/benchmarks 2021 3 (#4910)
* added new topics, changed the intro text

* updated

* Updates

* Updates

* Updates

* Updates

* Updates

* Added yolo-v4-tf and unet-camvid-onnx graphs

* Date for pricing is updated to March 15th
2021-03-22 23:27:24 +03:00
Anastasiya Ageeva a135ab64af Avladimi/cherry pick from master (#4892)
* Fixed CVS-48061

* Reviewed and edited the Customization instructions

* Fixed broken links in the TOC

* Fixed links

* Fixed formatting in the IG for Raspberry
2021-03-22 21:07:15 +03:00
9a7e939a6f [DOC] Low-Precision 8-bit Integer Inference (#4834)
* [DOC] Low-Precision 8-bit Integer Inference

* [DOC] Low-Precision 8-bit Integer Inference: comment fixes

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* [DOC] LPT comments fix

* [DOC] LPT comments fix: absolute links are updated to relative

* Update Int8Inference.md

* Update Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-03-22 14:24:24 +03:00
Anastasiya AgeevaandAndrey Zaytsev f9075c808a avladimi/cvs-31369: Documented packages content to YUM/APT IGs (#4839)
* Documented runtime/dev packages content

* Minor formatting fixes

* Implemented review comments

* Update installing-openvino-apt.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-03-22 14:07:06 +03:00
Evgeny Lazarev 60059f2c75 Fixed dead lock in telemetry (#4873)
* Fixed dead lock in telemetry

* Refactored TelemetrySender.send function

* Refactored send function implementation to avoid deadlocks

* Unit tests for telemetry sender function

* Added legal header
2021-03-19 14:59:45 +03:00
Anton Chetverikov 58012dcd71 Add documentation on how to convert F3Net model (#4863)
* Add instruction for F3Net model pytorch->onnx conversion

* Fix style
2021-03-18 15:24:56 +03:00
Maxim VafinandTatiana Savina 71f27da6a8 Document YOLACT support (#4749)
* Document YOLACT support

* Add preprocessing section

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2021-03-18 12:09:39 +03:00
Helena Kloosterman 35a898ab85 Updated PIP install instructions (#4821) 2021-03-17 11:42:55 +03:00
Vladislav Volkov a5d9f96efd MKLDNN weights cache key calculation algorithm changed (#4790) 2021-03-16 21:42:26 +03:00
Alina Alborova 1ef890329c Fixed wrong link (#4817) 2021-03-16 18:00:26 +03:00
Andrew Bakalin a86ae42aed [VPU] Limit dlclose() WA to be used for Ubuntu only (#4806) 2021-03-16 14:54:40 +03:00
Andrey Dmitriev cef0696ef7 [DOCS] added iname/oname (#4735) 2021-03-16 12:41:54 +03:00
Andrey Zaytsev e57a96474d Feature/vpu doc fixes 2021 3 (#4635)
* Documentation fixes and updates for VPU

* minor correction

* minor correction

* Fixed links

* updated supported layers list for vpu
2021-03-15 16:40:17 +03:00
Ilya Lavrenov ed052022d3 Fixed some klockwork issues in C API samples (#4767) 2021-03-15 11:00:07 +03:00
Alina Alborova 6eda5c39c6 Inserted a disclaimer (#4760) 2021-03-12 18:38:58 +03:00
Mikhail Ryzhov 0100810dd6 Added mo.py to wheel packages (#4731) 2021-03-12 15:57:12 +03:00
Elizaveta Lobanova 882e377ef9 [GNA] Update documentation regarding splits and concatenations support (#4740) 2021-03-12 14:00:15 +03:00
Ilya Lavrenov c9d5d95e2c Blocked dims hwc 2021/3 (#4729)
* Fix for BlockedDims

* Added test for HWC layout
2021-03-11 16:02:25 +03:00
Maxim ShevtsovandAlina Alborova d77bc36dcd Docs update (#4626)
* Updated latency case desc to cover multi-socket machines

* updated opt guide a bit

* avoiding '#' which is interpreted as ref

* Update CPU.md

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

Co-authored-by: Alina Alborova <alina.alborova@intel.com>
2021-03-11 15:33:08 +03:00
Andrey Somsikov 19e1b6002e Catch std::except in fuzz tests (#4695)
Fuzz tests must catch all expected exceptions from IE. IE is using C++ std
library which may raise standard exceptions which IE pass through.
2021-03-11 14:09:36 +03:00
Ivan Tikhonov 6bcd0f6072 Fix python API for Loop/TensorIterator/Assign/ReadValue operations 2021-03-11 13:22:50 +03:00
Anton Chetverikov 0e8534a4a9 Add STN to list of supported models (#4728) 2021-03-11 11:55:45 +03:00
Ilya Churaev 08d7c3e75f Restored folded Operations for QueryNetwork (#4685)
* Restored folded Operations for QueryNetwork

* Fixed comment

* Add unfolded constant operations to supported layers map
2021-03-11 10:41:57 +03:00
Evgeny Lazarev 821d513150 Updated documentation about the supported YOLOv3 model from ONNX (#4722) (#4726) 2021-03-11 10:32:13 +03:00
Mikhail Letavin 38a48b9cbf Fix NormalizeL2 creation in QueryNetwork (cherry pick from master PR 4310) (#4651) 2021-03-10 19:41:35 +03:00
Mikhail Letavin c6d8905a88 [IE CLDNN] Fix missing variable initializations and types (#4669) 2021-03-10 16:23:10 +03:00
Ilya Churaev 3a80652d70 Updated nGraph custom op documentation (#4604)
* Updated nGraph custom op documentation

* Fixed comments
2021-03-10 16:16:57 +03:00
Roman Kazantsev 120d3a596d Document TensorFlow 2* Update: Layers Support and Remove Beta Status (#4474) (#4711)
* Document TensorFlow 2* Update: Layers Support and Remove Beta Status

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update documentation based on latest test results and feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Remove ConvLSTM2D from supported layers list

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Document Dot layer without limitation

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Address feedback upon DenseFeatures and RNN operations

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Do a grammar correction

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Do a grammar correction based on feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-03-10 13:35:46 +03:00
Andrew Bakalin 25af83db81 [VPU] WA for Segmentation fault on dlclose() issue (#4645) 2021-03-10 12:09:55 +03:00
Maksim Doronin 03c6f4e3fe [IE][VPU]: Fix empty output of CTCGreedyDecoderSeqLen (#4653)
* Allow the second output of CTCGreedyDecoderSeqLen to be nullptr in cases when it is not used but calculated in the Myriad plugin. In this case, parse the second output as FakeData
* It is a cherry-pick of #4652
* Update the firmware to release version
2021-03-10 11:48:13 +03:00
Mikhail RyzhovandAndrey Zaytsev 5d3d323bed [doc] Updated PyPI support OSes (#4643) (#4662)
* [doc] Updated PyPI support OSes (#4643)

* Updated PyPI support OSes

* Added python versions for win and mac

* Update pypi-openvino-dev.md

* Update pypi-openvino-dev.md

* Update pypi-openvino-rt.md

* Update pypi-openvino-dev.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-03-09 17:54:29 +03:00
Anastasia Popova a53524a554 Fixed bug in assign elimination transformation. (#4644) 2021-03-09 14:58:12 +03:00
Maxim VafinandAnastasiya Ageeva 02d2dbd0fa Add documentation on how to convert QuartzNet model (#4664)
* Add documentation on how to convert QuartzNet model (#4422)

* Add documentation on how to convert QuartzNet model

* Apply review feedback

* Small fix

* Apply review feedback

* Apply suggestions from code review

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Add reference to file

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
2021-03-09 13:55:46 +03:00
Ilya Lavrenov bfe0748b4c Docs api (#4657)
* Updated API changes document

* Comment for CVS-49440
2021-03-09 12:50:37 +03:00
Nikolay Shchegolev d78577aecb [CPU] Statically analyzed issues. (#4637) 2021-03-09 12:26:32 +03:00
Mikhail Ryzhov e09f0e4808 Corrected PyYAML dependency (#4598) (#4620)
5.4.2 is absent on PyPI
2021-03-05 17:24:36 +03:00
Roman Kazantsev ff73955354 Add workaround for control edges to support TF 2.4 RNN (#4634)
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-03-05 16:41:36 +03:00
Vladislav Volkov 18cb230af4 Fix for MKLDNN constant layers execution (#4642)
* Fix for MKLDNN constant layers execution

* Single mkldnn::engine for all MKLDNN graphs
2021-03-05 16:28:10 +03:00
Alina Kladieva 9067a25616 [.ci/azure] Add windows_conditional_compilation.yml (#4648) (#4655) 2021-03-05 15:59:59 +03:00
Gorokhov Dmitriy c4ff0ffa9d [CPU] Supported ANY layout for inputs in inferRequest (#4621) 2021-03-05 12:30:26 +03:00
Ilya Churaev 4675a12c8f Fixed KW hits (#4638) 2021-03-05 11:13:11 +03:00
Evgeny Lazarev 3cd5da0797 Fixed transformation to pull constants into Loop body (cherry-pick of PR 4591) (#4607)
* Cherry-pick of PR 4591

* Fixed typo

* Moved a check into the parameter_unchanged_after_iteration function
2021-03-04 17:56:54 +03:00
Andrey Zaytsev 9b402f226f Formula fix (#4624) 2021-03-04 16:24:56 +03:00
Alina Kladieva 784adca70a [.ci/azure] Enable CC build (#4619) 2021-03-04 14:49:50 +03:00
Ilya Churaev 8e1603f7fd Fixed clone rt info (#4597) 2021-03-04 13:02:11 +03:00
azhogov 66ede40e4e Merge branch 'releases/2021/3' of https://github.com/openvinotoolkit/openvino into releases/2021/3 2021-03-04 12:22:59 +03:00
azhogov 40a29a7aa3 Azure CI: Add "ref: releases/2021/3" 2021-03-04 12:22:31 +03:00
Vladislav Volkov a7e00dae54 Fix for broken CC in CPU plugin (#4595) 2021-03-04 12:22:15 +03:00
Andrey ZaytsevandTrawinski, Dariusz <dariusz.trawinski@intel.com> 4c40494605 Feature/azaytsev/gna model link fixes (#4599)
* Added info on DockerHub CI Framework

* Feature/azaytsev/change layout (#3295)

* Changes according to feedback comments

* Replaced @ref's with html links

* Fixed links, added a title page for installing from repos and images, fixed formatting issues

* Added links

* minor fix

* Added DL Streamer to the list of components installed by default

* Link fixes

* Link fixes

* ovms doc fix (#2988)

* added OpenVINO Model Server

* ovms doc fixes

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>

* Updated openvino_docs.xml

* Link Fixes

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
2021-03-04 12:00:07 +03:00
Andrey ZaytsevandTrawinski, Dariusz <dariusz.trawinski@intel.com> 3e2a4a5df1 Feature/azaytsev/cldnn doc fixes (#4600)
* Legal fixes, removed the Generating docs section

* Removed info regarding generating docs

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
2021-03-04 11:50:02 +03:00
10631 changed files with 485719 additions and 253225 deletions
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -1,6 +0,0 @@
TransposeOpTest.NHWC2NCHW
TransposeOpTest.NCHW2NHWC
TransposeOpTest.TwoDim_int16
GatherOpTest.Gather_axis1_indices2d_int16
SoftmaxOperator.ThreeDimsAxis1
SoftmaxOperator.ThreeDimsAxis0
-1
View File
@@ -1 +0,0 @@
rel-1.7.1
+19 -43
View File
@@ -1,25 +1,16 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/4
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/4
ref: releases/2021/3
jobs:
- job: Lin
@@ -40,7 +31,6 @@ jobs:
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
@@ -54,7 +44,6 @@ jobs:
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -64,16 +53,13 @@ jobs:
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
- script: |
set -e
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'Make dir'
- checkout: self
@@ -94,17 +80,12 @@ jobs:
path: testdata
- script: |
set -e
sudo apt --assume-yes update && sudo apt --assume-yes install libusb-1.0-0-dev
# For opencv-python: setuptools and upgrade
sudo apt-get install python3-setuptools patchelf
python3 -m pip install --upgrade pip
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/wheel/requirements-dev.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
@@ -120,15 +101,10 @@ jobs:
-DVERBOSE_BUILD=ON
-DENABLE_TEMPLATE_PLUGIN=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DBUILD_cuda_plugin=OFF
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/bin/python3.6
-DENABLE_WHEEL=ON
-DENABLE_TESTS=ON
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
@@ -138,28 +114,16 @@ jobs:
displayName: 'Build Lin'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/cpp/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/c/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
- script: $(BIN_DIR)/unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(BIN_DIR)/InferenceEngineUnitTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(BIN_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
@@ -196,6 +160,18 @@ jobs:
displayName: 'CPU FuncTests'
continueOnError: false
- script: $(BIN_DIR)/MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 $(WORK_DIR)/gtest-parallel/gtest-parallel $(BIN_DIR)/MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
workingDirectory: $(WORK_DIR)
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
+6 -17
View File
@@ -1,12 +1,3 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
jobs:
- job: LinCC
# About 150% of total time
@@ -19,12 +10,14 @@ jobs:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 16
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
@@ -37,7 +30,6 @@ jobs:
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -61,11 +53,10 @@ jobs:
path: openvino
- script: |
set -e
sudo apt --assume-yes update && sudo apt --assume-yes install libusb-1.0-0-dev
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
workingDirectory: $(WORK_DIR)
@@ -85,14 +76,12 @@ jobs:
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build LinCC'
displayName: 'Build'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
+29 -42
View File
@@ -1,42 +1,22 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
jobs:
- job: OpenVINO_ONNX_CI
strategy:
matrix:
Release:
BUILD_TYPE: 'Release'
PROTOBUF_LITE: 'ON'
TOX_COMMAND: 'tox && tox -e zoo_models'
Debug:
BUILD_TYPE: 'Debug'
PROTOBUF_LITE: 'ON'
TOX_COMMAND: 'tox'
maxParallel: 2
- job: nGraph_ONNX_Lin
# About 300% of total time
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_U20_WU2
name: LIN_VMSS_VENV_ONNX_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 8
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
ONNX_MODEL_ZOO_SHA: "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
steps:
- script: |
@@ -47,7 +27,6 @@ jobs:
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -61,10 +40,10 @@ jobs:
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(MODELS_DIR)/models_data
displayName: 'Make dirs'
- checkout: self
@@ -73,23 +52,31 @@ jobs:
submodules: recursive
path: openvino
- script: docker build --tag=openvino-onnx-ci-image --file=.ci/openvino-onnx/Dockerfile .
displayName: 'Docker build'
- script: ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d $(TMP_DIR) -o
displayName: 'Get models'
- script: |
set -e
sudo apt --assume-yes install git-lfs uidmap
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
##wget -O "$(TMP_DIR)/msft.zip" https://onnxruntimetestdata.blob.core.windows.net/models/20191107.zip
##unzip "$(TMP_DIR)/msft.zip" -d "$(MODELS_DIR)/msft"
#unzip "/mnt/onnxtestdata/models/20191107.zip" -d "$(MODELS_DIR)/msft"
#mv $(MODELS_DIR)/msft/opset9/LSTM_Seq_lens_unpacked/seq_lens_sorted $(MODELS_DIR)/msft/opset9/LSTM_Seq_lens_unpacked/test_data_set_0
#mv $(MODELS_DIR)/msft/opset9/LSTM_Seq_lens_unpacked/seq_lens_unsorted $(MODELS_DIR)/msft/opset9/LSTM_Seq_lens_unpacked/test_data_set_1
displayName: 'Get MSFT models'
enabled: false
- script: ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
displayName: 'Update models'
condition: ne(variables['BUILD_TYPE'], 'Debug')
- script: |
ls -alR $(MODELS_DIR)
ls -alR $(TMP_DIR)
displayName: 'List models'
enabled: false
- script: sudo docker build --tag=openvino-onnx-ci-image --file=.ci/openvino-onnx/Dockerfile --build-arg BUILD_TYPE=$(BUILD_TYPE) --build-arg PROTOBUF_LITE=$(PROTOBUF_LITE) .
displayName: 'Docker build $(BUILD_TYPE) protobuf-lite: $(PROTOBUF_LITE)'
- script: sudo fallocate -l 64G /swapfile ; sudo mkswap /swapfile ; sudo swapon /swapfile ; df ; free -h
- script: sudo fallocate -l 48G /swapfile ; sudo mkswap /swapfile ; sudo swapon /swapfile ; df ; free -h
displayName: 'Create swap'
- script: sudo docker run --name openvino-onnx-ci-container --volume $(MODELS_DIR)/models_data/model_zoo/onnx_model_zoo_$(ONNX_MODEL_ZOO_SHA):/root/.onnx/model_zoo/onnx_model_zoo --volume $(MODELS_DIR)/msft:/root/.onnx/model_zoo/MSFT openvino-onnx-ci-image /bin/bash -c "$(TOX_COMMAND)"
displayName: 'Docker run $(BUILD_TYPE) protobuf-lite: $(PROTOBUF_LITE)'
- script: |
docker run --name openvino-onnx-ci-container --volume $(TMP_DIR)/model_zoo:/root/.onnx/model_zoo --volume $(MODELS_DIR)/msft:/root/.onnx/model_zoo/MSFT openvino-onnx-ci-image
displayName: 'Docker run'
-167
View File
@@ -1,167 +0,0 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
jobs:
- job: onnxruntime
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
ONNXRUNTIME_REPO_DIR: $(REPO_DIR)/../onnxruntime
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
INSTALL_DIR: $(WORK_DIR)/install_pkg
BUILD_DIR: $(WORK_DIR)/build
ONNXRUNTIME_UTILS: $(REPO_DIR)/.ci/azure/ci_utils/onnxruntime
ONNXRUNTIME_BUILD_DIR: $(ONNXRUNTIME_REPO_DIR)/build
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
displayName: 'Make dirs'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
branch=`tr -s '\n ' < $(ONNXRUNTIME_UTILS)/version`
git clone --branch $branch --single-branch --recursive https://github.com/microsoft/onnxruntime.git $(ONNXRUNTIME_REPO_DIR)
displayName: 'Clone onnxruntime'
- script: |
set -e
$(REPO_DIR)/install_build_dependencies.sh
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/bin/python3.8
-DENABLE_VPU=OFF
-DENABLE_GNA=OFF
-DENABLE_OPENCV=OFF
-DENABLE_CPPLINT=OFF
-DENABLE_TESTS=OFF
-DENABLE_MKL_DNN=ON
-DENABLE_CLDNN=OFF
-DENABLE_PROFILING_ITT=OFF
-DENABLE_SAMPLES=OFF
-DENABLE_SPEECH_DEMO=OFF
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DNGRAPH_INTERPRETER_ENABLE=ON
-DNGRAPH_DEBUG_ENABLE=OFF
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin ONNX'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
echo "2021.2" > $(INSTALL_DIR)/deployment_tools/inference_engine/version.txt
CXXFLAGS="-Wno-error=deprecated-declarations" ./build.sh --config RelWithDebInfo --use_openvino CPU_FP32 --build_shared_lib --parallel --skip_tests --build_dir $(ONNXRUNTIME_BUILD_DIR)
workingDirectory: $(ONNXRUNTIME_REPO_DIR)
displayName: 'Build Lin ONNX Runtime'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
skip_tests=`tr -s '\n ' ':' < $(ONNXRUNTIME_UTILS)/skip_tests`
./onnxruntime_test_all --gtest_filter=-$skip_tests
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_test_all'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_shared_lib_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_shared_lib_test'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_global_thread_pools_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_global_thread_pools_test'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_api_tests_without_env
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_api_tests_without_env'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-converted"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-converted tests'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-operator"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-operator tests'
+23 -26
View File
@@ -1,25 +1,16 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/4
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/4
ref: releases/2021/3
jobs:
- job: Mac
@@ -33,6 +24,7 @@ jobs:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 3
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
@@ -47,11 +39,11 @@ jobs:
- script: |
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
which python3
python3 --version
which java
java -version
gcc --version
xcrun --sdk macosx --show-sdk-version
env
sysctl -a
@@ -100,27 +92,21 @@ jobs:
# Disable errors with Ninja
export CXXFLAGS="-Wno-error=unused-command-line-argument"
export CFLAGS="-Wno-error=unused-command-line-argument"
cmake -GNinja -DVERBOSE_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DBUILD_cuda_plugin=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules $(REPO_DIR)
cmake -GNinja -DVERBOSE_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: ls -alR $(REPO_DIR)/inference-engine/temp/
displayName: 'List temp SDKs'
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Mac'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: $(BIN_DIR)/unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU*:IE_CPU.onnx_model_sigmoid:IE_CPU/GRUSequenceOp.onnx_model_gru* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
@@ -149,10 +135,21 @@ jobs:
displayName: 'IE FuncTests'
continueOnError: false
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke*:-smoke_LPT/ReduceMinTransformation.CompareWithRefImpl/f32_Shape* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
continueOnError: false
enabled: false
- script: $(BIN_DIR)/MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 $(WORK_DIR)/gtest-parallel/gtest-parallel $(BIN_DIR)/MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke*:-smoke_MobileNet/ModelTransformationsTest.LPT/mobilenet_v2_tf_depthwise_batch1_inPluginDisabled_inTestDisabled_asymmetric* -- --gtest_print_time=1
workingDirectory: $(WORK_DIR)
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
+45 -42
View File
@@ -1,25 +1,16 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/4
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/4
ref: releases/2021/3
jobs:
- job: Win
@@ -27,31 +18,32 @@ jobs:
timeoutInMinutes: 120
pool:
name: WIN_VMSS_VENV_F16S_WU2
name: WIN_VMSS_VENV_F8S_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 8
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)\..\openvino_contrib
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: D:\build
BUILD_SAMPLES_DIR: D:\build_samples
BIN_DIR: $(REPO_DIR)\bin\intel64
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;%PATH%
IB_DIR: C:\Program Files (x86)\IncrediBuild
IB_TESTCONSOLE: $(IB_DIR)\IBTestConsole.exe
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;$(IB_DIR);%PATH%
steps:
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
where python3
python3 --version
where python
python --version
where java
@@ -66,9 +58,14 @@ jobs:
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(BUILD_SAMPLES_DIR) & mkdir $(BUILD_SAMPLES_DIR)
displayName: 'Make dir'
- script: |
certutil -urlcache -split -f https://incredibuilddiag1wu2.blob.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
call install_ib_console.bat
workingDirectory: $(WORK_DIR)
displayName: 'Install IncrediBuild'
- checkout: self
clean: true
lfs: false
@@ -87,8 +84,7 @@ jobs:
path: testdata
- script: |
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
certutil -urlcache -split -f https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-win.zip ninja-win.zip
powershell -command "Expand-Archive -Force ninja-win.zip"
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
@@ -96,38 +92,23 @@ jobs:
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DENABLE_TEMPLATE_PLUGIN=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DBUILD_cuda_plugin=OFF -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DENABLE_TEMPLATE_PLUGIN=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_TESTS=ON -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: dir $(REPO_DIR)\inference-engine\temp\ /s
displayName: 'List temp SDKs'
- script: call "$(MSVS_VARS_PATH)" && $(WORK_DIR)\ninja-win\ninja
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && "C:\Program Files (x86)\IncrediBuild\BuildConsole.exe" /COMMAND="ninja"
workingDirectory: $(BUILD_DIR)
displayName: 'Build Win'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\cpp\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\c\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
- script: rd /Q /S $(BUILD_DIR)
displayName: 'Clean build dir'
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
@@ -136,8 +117,8 @@ jobs:
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\InferenceEngineUnitTests.exe --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
"$(IB_TESTCONSOLE)" $(BIN_DIR)\InferenceEngineUnitTests.exe --gtest_output=xml:TEST-InferenceEngineUnitTests-IB.xml
displayName: 'IE UT old - IB'
- script: |
set PATH=$(TEST_ENV_PATH)
@@ -183,8 +164,25 @@ jobs:
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\cpuFuncTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
"$(IB_TESTCONSOLE)" $(BIN_DIR)\cpuFuncTests.exe --gtest_filter=*smoke*:-*CompareWithRefs/base_size=16_pre_nms_topn=100_post_nms_topn=100_nms_thresh=0.7_feat_stride=1_min_size=1_ratio* --gtest_output=xml:TEST-cpuFuncTests-IB.xml /testlevel=24
displayName: 'CPU FuncTests - IB'
continueOnError: false
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
# Add for gtest-parallel, it hangs now (CVS-33386)
#python $(WORK_DIR)\gtest-parallel\gtest-parallel $(BIN_DIR)\MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: |
set PATH=$(TEST_ENV_PATH)
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
rem "$(IB_TESTCONSOLE)" $(BIN_DIR)\MklDnnFunctionalTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-MklDnnFunctionalTests-IB.xml
$(BIN_DIR)\MklDnnFunctionalTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-MklDnnFunctionalTests.xml
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
@@ -207,3 +205,8 @@ jobs:
buildPlatform: 'x64' # Optional
buildConfiguration: 'Windows' # Optional
#publishRunAttachments: true # Optional
- script: echo Stop IncrediBuild_Agent && net stop IncrediBuild_Agent
displayName: Stop IncrediBuild
continueOnError: true
enabled: false
+22 -21
View File
@@ -1,16 +1,7 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/*
jobs:
- job: WinCC
# About 150% of total time
timeoutInMinutes: 60
timeoutInMinutes: 120
pool:
name: WIN_VMSS_VENV_F8S_WU2
@@ -19,22 +10,26 @@ jobs:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 8
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)\..\openvino_contrib
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: D:\build
BIN_DIR: $(REPO_DIR)\bin\intel64
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
IB_DIR: C:\Program Files (x86)\IncrediBuild
IB_TESTCONSOLE: $(IB_DIR)\IBTestConsole.exe
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;$(IB_DIR);%PATH%
steps:
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
where python3
python3 --version
where python
python --version
where java
@@ -51,6 +46,12 @@ jobs:
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
displayName: 'Make dir'
- script: |
certutil -urlcache -split -f https://incredibuilddiag1wu2.blob.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
call install_ib_console.bat
workingDirectory: $(WORK_DIR)
displayName: 'Install IncrediBuild'
- checkout: self
clean: true
lfs: false
@@ -58,8 +59,7 @@ jobs:
path: openvino
- script: |
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
certutil -urlcache -split -f https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-win.zip ninja-win.zip
powershell -command "Expand-Archive -Force ninja-win.zip"
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
@@ -70,19 +70,20 @@ jobs:
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: dir $(REPO_DIR)\inference-engine\temp\ /s
displayName: 'List temp SDKs'
- script: call "$(MSVS_VARS_PATH)" && $(WORK_DIR)\ninja-win\ninja
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && "C:\Program Files (x86)\IncrediBuild\BuildConsole.exe" /COMMAND="ninja"
workingDirectory: $(BUILD_DIR)
displayName: 'Build Win CC'
displayName: 'Build Win'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: echo Stop IncrediBuild_Agent && net stop IncrediBuild_Agent
displayName: Stop IncrediBuild
continueOnError: true
enabled: false
+7 -11
View File
@@ -1,10 +1,6 @@
FROM ubuntu:20.04
LABEL version=2021.03.30.1
# Build configuration arguments
ARG BUILD_TYPE=Release
ARG PROTOBUF_LITE=OFF
LABEL version=2020.07.09.1
ARG http_proxy
ARG https_proxy
@@ -55,7 +51,7 @@ RUN apt-get update && apt-get -y --no-install-recommends install \
COPY . /openvino/
WORKDIR /openvino/build
RUN cmake .. \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_VPU=OFF \
-DENABLE_GNA=OFF \
-DENABLE_OPENCV=OFF \
@@ -69,17 +65,17 @@ RUN cmake .. \
-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=/usr/bin/python3 \
-DNGRAPH_ONNX_IMPORT_ENABLE=ON \
-DNGRAPH_ONNX_EDITOR_ENABLE=ON \
-DNGRAPH_INTERPRETER_ENABLE=ON \
-DNGRAPH_DEBUG_ENABLE=OFF \
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON \
-DCMAKE_INSTALL_PREFIX=/openvino/dist \
-DNGRAPH_USE_PROTOBUF_LITE=${PROTOBUF_LITE}
-DCMAKE_INSTALL_PREFIX=/openvino/dist
RUN make -j $(nproc) install
# Run tests via tox
WORKDIR /openvino/ngraph/python
ENV ngraph_DIR=/openvino/dist/deployment_tools/ngraph
ENV NGRAPH_CPP_BUILD_PATH=/openvino/dist/deployment_tools/ngraph
ENV LD_LIBRARY_PATH=/openvino/dist/deployment_tools/ngraph/lib
ENV PYTHONPATH=/openvino/bin/intel64/${BUILD_TYPE}/lib/python_api/python3.8:${PYTHONPATH}
ENV NGRAPH_ONNX_IMPORT_ENABLE=TRUE
ENV PYTHONPATH=/openvino/bin/intel64/Release/lib/python_api/python3.8:${PYTHONPATH}
RUN git clone --recursive https://github.com/pybind/pybind11.git -b v2.5.0 --depth 1
CMD tox
+59 -116
View File
@@ -3,19 +3,12 @@
DOCKER_CONTAINER_NAME= "openvino-onnx-ci-container"
DOCKER_IMAGE_TAG = "openvino-onnx-ci-image"
ONNX_MODEL_ZOO_SHA = "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
BACKEND_CONFIGURATIONS = [
[ name: "Release", build_type: "Release", protobuf_lite : "OFF" ],
[ name: "Debug", build_type: "Debug", protobuf_lite : "OFF" ],
[ name: "Rel_Lite", build_type: "Release", protobuf_lite : "ON" ],
]
// workaround for aborting previous builds on PR update
@NonCPS
def stopPreviousRunningBuilds() {
def jobname = env.JOB_NAME
if (jobname.startsWith("onnx-ci/openvino onnx ci/openvino/PR")){
if (jobname.startsWith("onnx/openvino_ci/PR")){
def buildnum = env.BUILD_NUMBER.toInteger()
def job = Jenkins.instance.getItemByFullName(jobname)
def job_newest = job.builds.first()
@@ -30,7 +23,7 @@ BACKEND_CONFIGURATIONS = [
}
def getGitPrInfo(String project, String workdir) {
def getGitPrInfo(String project) {
def gitPrInfo = [
prAuthorEmail : "",
commitAuthorEmail : "",
@@ -38,7 +31,7 @@ def getGitPrInfo(String project, String workdir) {
commitSubject : ""
]
try {
dir ("${workdir}/${project}") {
dir ("${WORKDIR}/${project}") {
gitPrInfo.prAuthorEmail = sh (script: 'git log -1 --pretty="format:%ae" ', returnStdout: true).trim()
gitPrInfo.commitAuthorEmail = sh (script: 'git log -1 --pretty="format:%ce" ', returnStdout: true).trim()
gitPrInfo.commitSubject = sh (script: 'git log -1 --pretty="format:%H" ', returnStdout: true).trim()
@@ -71,146 +64,96 @@ def notifyByEmail(def gitPrInfo) {
}
}
def gitSubmoduleUpdate(String repository_name, String workdir) {
dir ("${workdir}/${repository_name}") {
def gitSubmoduleUpdate(String repository_name) {
dir ("${WORKDIR}/${repository_name}") {
sh label: "Init ${repository_name} submodules",
script:
"""
git submodule init && git submodule update \
--init \
--no-fetch \
--recursive
--recursive
"""
}
}
def prepare_repository(String workdir) {
dir("${workdir}") {
println "Preparing repository in directory: ${workdir}"
checkout scm
gitSubmoduleUpdate(PROJECT_NAME, workdir)
}
}
def updateModels() {
sh """
./ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d ${HOME}/ONNX_CI/models_data -o -s ${ONNX_MODEL_ZOO_SHA}
./ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d ${HOME}/ONNX_CI/data -o
"""
}
def get_docker_container_name(Map configuration){
println "RUN get_docker_container_name for ${configuration.name}"
String docker_container_name = "${DOCKER_CONTAINER_NAME}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
return docker_container_name
}
def buildDockerImage(Map configuration, String workdir) {
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
println "docker_image_tag: ${docker_image_tag}"
def buildDockerImage() {
updateModels()
sh """
docker build --tag=${docker_image_tag} \
--build-arg BUILD_TYPE=${configuration.build_type} \
--build-arg PROTOBUF_LITE=${configuration.protobuf_lite} \
--file=.ci/openvino-onnx/Dockerfile \
--build-arg http_proxy=http://proxy-ir.intel.com:911/ \
--build-arg https_proxy=http://proxy-ir.intel.com:911/ .
docker build --tag=${DOCKER_IMAGE_TAG} --file=.ci/openvino-onnx/Dockerfile \
--build-arg http_proxy=http://proxy-chain.intel.com:911/ \
--build-arg https_proxy=http://proxy-chain.intel.com:912/ .
"""
}
def runTests(Map configuration, String workdir) {
println "Run tests for ${configuration.name}"
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
String docker_container_name = get_docker_container_name(configuration)
// Run only basic unit tests in Debug configuration
if (configuration.build_type == "Debug") {
sh """
docker run --name ${docker_container_name} ${docker_image_tag}
"""
}
// Run unit-tests AND large model tests by default
else {
sh """
docker run --name ${docker_container_name} \
--volume ${HOME}/ONNX_CI/models_data/model_zoo/onnx_model_zoo_${ONNX_MODEL_ZOO_SHA}:/root/.onnx/model_zoo/onnx_model_zoo \
--volume ${HOME}/ONNX_CI/data/model_zoo/MSFT:/root/.onnx/model_zoo/MSFT \
${docker_image_tag} /bin/bash -c "tox && tox -e zoo_models"
"""
}
def runTests() {
sh """
docker run --name ${DOCKER_CONTAINER_NAME} \
--volume ${HOME}/ONNX_CI/data/model_zoo:/root/.onnx/model_zoo \
${DOCKER_IMAGE_TAG}
"""
}
def getConfigurationsMap() {
def configurationsMap = [:]
for (backend in BACKEND_CONFIGURATIONS) {
def configuration = backend.clone()
configurationsMap[configuration.name] = {
stage(configuration.name) { CONFIGURATION_WORKFLOW(configuration) }
}
pipeline {
agent {
label "OpenVino"
}
return configurationsMap
}
CONFIGURATION_WORKFLOW = { configuration ->
node("OpenVINO") {
try {
PROJECT_NAME = "openvino"
String workdir = "${HOME}/workspace/${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
stage("Clone repository") {
prepare_repository(workdir)
}
stage("Prepare Docker environment") {
dir("${workdir}") {
buildDockerImage(configuration, workdir)
environment {
PROJECT_NAME = "openvino"
WORKDIR = "${WORKSPACE}/${BUILD_NUMBER}"
}
options {
skipDefaultCheckout true
timeout(activity: true, time: 60, unit: 'MINUTES')
}
stages {
stage("Clone repository") {
steps{
stopPreviousRunningBuilds()
dir("${WORKDIR}") {
checkout scm
}
gitSubmoduleUpdate(PROJECT_NAME)
}
stage("Run tests") {
timeout(time: 60, unit: 'MINUTES') {
runTests(configuration, workdir)
}
stage("Prepare Docker environment") {
steps{
dir("${WORKDIR}") {
buildDockerImage()
}
}
}
catch(e) {
// Set result to ABORTED if exception contains exit code of a process interrupted by SIGTERM
if ("$e".contains("143")) {
currentBuild.result = "ABORTED"
} else {
currentBuild.result = "FAILURE"
}
def gitPrInfo = getGitPrInfo(PROJECT_NAME, workdir)
stage("Run tests") {
options {
timeout(time: 60, unit: 'MINUTES')
}
steps{
runTests()
}
}
}
post {
failure {
script {
gitPrInfo = getGitPrInfo(PROJECT_NAME)
notifyByEmail(gitPrInfo)
}
}
finally {
stage("Cleanup") {
cleanup {
dir("${WORKDIR}") {
deleteDir()
String docker_container_name = get_docker_container_name(configuration)
sh """
docker rm -f ${docker_container_name}
docker image prune -f
docker rm -f ${DOCKER_CONTAINER_NAME}
"""
}
}
}
}
pipeline {
agent none
options {
skipDefaultCheckout true
timeout(activity: true, time: 120, unit: 'MINUTES')
}
stages {
stage('Parallel CI') {
steps {
stopPreviousRunningBuilds()
script {
parallelStagesMap = getConfigurationsMap()
parallel parallelStagesMap
}
}
}
}
}
@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import datetime
+17 -60
View File
@@ -1,17 +1,16 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Check GitHub organization and invite members
"""
# pylint: disable=fixme,no-member,too-many-locals
# pylint: disable=fixme,no-member
from argparse import ArgumentParser
import github_api
from configs import Config
from github_api import GithubOrgApi, get_dev_emails
from ldap_api import LdapApi, print_user_info, InfoLevel
def main():
@@ -20,74 +19,32 @@ def main():
arg_parser.add_argument("--cfg-file", metavar="PATH", default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}")
arg_parser.add_argument("--teams", action="store_true", help="Check GitHub teams")
arg_parser.add_argument("--no-ldap", action="store_true", help="Don't use LDAP info")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
gh_api = GithubOrgApi()
gh_api = github_api.GithubOrgApi()
if args.teams:
gh_api.get_org_teams()
return
else:
dev_emails = github_api.get_dev_emails()
print(f'\nDeveloper emails {len(dev_emails)}:', '; '.join(dev_emails))
cfg_emails = get_dev_emails()
print(f'\nCfg developer emails {len(cfg_emails)}:', '; '.join(sorted(cfg_emails)))
org_emails = gh_api.get_org_emails()
print(f'\nOrg emails {len(org_emails)}:', '; '.join(org_emails))
dev_emails = set()
dev_emails.update(cfg_emails)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
if not args.no_ldap:
ldap_api = LdapApi()
ldap_emails = ldap_api.get_user_emails()
dev_emails.update(ldap_emails)
print(f'\nLDAP developer emails {len(ldap_emails)}:', '; '.join(sorted(ldap_emails)))
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f'\nInvite emails {len(invite_emails)}:', '; '.join(invite_emails))
cfg_emails_no_in_ldap = ldap_api.get_absent_emails(cfg_emails)
print(f'\nCfg developer emails - absent in LDAP at all {len(cfg_emails_no_in_ldap)}:',
'; '.join(sorted(cfg_emails_no_in_ldap)))
no_in_dev_emails = org_emails.difference(dev_emails)
print(f'\nOrg members - no in developers list {len(no_in_dev_emails)}:',
'; '.join(no_in_dev_emails))
cfg_ldap_inters = cfg_emails.intersection(ldap_emails)
print(f'\nCfg developer emails - present in LDAP developers {len(cfg_ldap_inters)}:',
'; '.join(sorted(cfg_ldap_inters)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
org_emails, org_logins_no_intel_email = gh_api.get_org_emails()
print(f'\nOrg emails {len(org_emails)}:', '; '.join(sorted(org_emails)))
org_emails_no_in_ldap = set()
if not args.no_ldap:
org_ldap_diff = org_emails.difference(ldap_emails)
print(f'\nOrg member emails - absent in LDAP developers {len(org_ldap_diff)}:',
'; '.join(sorted(org_ldap_diff)))
for email in org_ldap_diff:
user_info = ldap_api.get_user_info_by_email(email)
if user_info:
print_user_info(user_info, InfoLevel.PDL)
else:
org_emails_no_in_ldap.add(email)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f'\nInvite emails {len(invite_emails)}:', '; '.join(sorted(invite_emails)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
gh_api.invite_users(valid_github_users)
print('\nCheck accounts below and remove from the GitHub organization and cfg list')
cfg_emails_no_in_org = sorted(cfg_emails.difference(org_emails))
print(f'\nCfg developer emails - absent in GitHub organization {len(cfg_emails_no_in_org)}:',
'; '.join(cfg_emails_no_in_org))
org_emails_no_in_dev = sorted(org_emails.difference(dev_emails))
print(f'\nOrg member emails - absent in cfg and LDAP developers {len(org_emails_no_in_dev)}:',
'; '.join(org_emails_no_in_dev))
print(f'\nOrg member emails - absent in LDAP at all {len(org_emails_no_in_ldap)}:',
'; '.join(sorted(org_emails_no_in_ldap)))
print(f'\nOrg member logins - absent Intel email {len(org_logins_no_intel_email)}:',
'; '.join(sorted(org_logins_no_intel_email)))
gh_api.invite_users(valid_github_users)
if __name__ == '__main__':
+54 -164
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -8,7 +8,6 @@ Check GitHub PRs and set labels by type and categories, e.g. 'ExternalPR', 'cate
# pylint: disable=fixme,no-member
import re
import sys
import datetime
from argparse import ArgumentParser
from enum import Enum
@@ -19,11 +18,10 @@ from configs import Config
class PrType(Enum):
"""Constants for type of GitHub pull request by author membership"""
EXTERNAL = "ExternalPR"
INTEL = "ExternalIntelPR"
ORG = "OpenvinoPR"
BAD = "BadPR"
EXTERNAL = 'ExternalPR'
INTEL = 'ExternalIntelPR'
ORG = 'OpenvinoPR'
BAD = 'BadPR'
def get_pr_labels(pull):
@@ -35,23 +33,13 @@ def get_pr_labels(pull):
def set_pr_labels(pull, labels):
"""Sets new PR labels (all previously set labels are removed)"""
"""Sets PR labels"""
if not labels or Config().DRY_RUN:
return
print("Set PR labels:", labels)
# set_labels() should accept list but fails with empty "AssertionError:"
print(f'Set PR labels:', labels)
pull.set_labels(labels)
def add_pr_labels(pull, labels):
"""Adds PR labels"""
if not labels or Config().DRY_RUN:
return
print("Add PR labels:", labels)
for label in labels:
pull.add_to_labels(label)
def get_pr_type_by_labels(pull):
"""Gets PR type using labels"""
pr_lables = get_pr_labels(pull)
@@ -60,19 +48,19 @@ def get_pr_type_by_labels(pull):
if not pr_types_labels:
return None
if len(pr_types_labels) > 1:
print(f"Duplicated labels: {pr_types_labels}")
print(f'Duplicated labels: {pr_types_labels}')
return PrType.BAD
return PrType(PrType(pr_types_labels.pop()))
def get_label_by_team_name_re(team_name):
"""Generates label by PR reviwer team name using regular expressions"""
if "admins" in team_name:
return "category: ci"
re_compile_label = re.compile(rf"{Config().GITHUB_REPO}-(.+)-maintainers")
if 'admins' in team_name:
return 'category: ci'
re_compile_label = re.compile(rf'{Config().GITHUB_REPO}-(.+)-maintainers')
re_label = re_compile_label.match(team_name)
if re_label:
return f"category: {re_label.group(1).strip()}"
return f'category: {re_label.group(1).strip()}'
return None
@@ -92,113 +80,17 @@ def get_category_labels(pull):
return labels
def get_pr_info_str(pull):
"""Gets info about PR using a few workarounds"""
pr_title = pull.title.encode("ASCII", "ignore").decode()
# Workaround for PyGithub issue: https://github.com/PyGithub/PyGithub/issues/512
pr_created_at = pull.created_at.replace(tzinfo=datetime.timezone.utc).astimezone()
return (
f"PR: {pull.number} - {pr_title} - Created: {pr_created_at} - "
f"Labels: {get_pr_labels(pull)} - Type: {get_pr_type_by_labels(pull)}"
)
def update_labels(gh_api, pull, non_org_intel_pr_users, non_org_pr_users):
"""Checks and updates labels"""
print("Check and update labels:")
pr_type_by_labels = get_pr_type_by_labels(pull)
add_labels = []
# Checks PR source type
if gh_api.is_org_user(pull.user):
print(" - Org user")
elif github_api.is_intel_email(pull.user.email) or github_api.is_intel_company(
pull.user.company
):
print(" - Non org user with Intel email or company")
non_org_intel_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.INTEL:
print(f'NO "{PrType.INTEL.value}" label: ', end="")
github_api.print_users(pull.user)
add_labels.append(PrType.INTEL.value)
elif github_api.is_user_ignored(pull.user):
print(" - IGNORED non org user with NO Intel email or company")
else:
print(" - Non org user with NO Intel email or company")
non_org_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.EXTERNAL:
print(f'NO "{PrType.EXTERNAL.value}" label: ', end="")
github_api.print_users(pull.user)
add_labels.append(PrType.EXTERNAL.value)
add_labels += get_category_labels(pull)
add_pr_labels(pull, add_labels)
def get_wrong_commits(pull):
"""Returns commits with incorrect user and email"""
pr_author_email = (pull.user.email or "").lower()
print("GitHub PR author email:", pr_author_email)
print("Check commits:")
wrong_commits = set()
for commit in pull.get_commits():
# import pprint; pprint.pprint(commit.raw_data)
print("Commit SHA:", commit.sha)
# Use raw data because commit author can be non GitHub user
commit_author_email = (commit.raw_data["commit"]["author"]["email"] or "").lower()
commit_committer_email = (commit.raw_data["commit"]["committer"]["email"] or "").lower()
print(" Commit author email:", commit_author_email)
print(" Commit committer email:", commit_committer_email)
if not github_api.is_valid_user(commit.author):
print(
" ERROR: User with the commit author email is absent in GitHub:",
commit.raw_data["commit"]["author"]["name"],
)
wrong_commits.add(commit.sha)
if not github_api.is_valid_user(commit.committer):
print(
" ERROR: User with the commit committer email is absent in GitHub:",
commit.raw_data["commit"]["committer"]["name"],
)
wrong_commits.add(commit.sha)
if not commit.raw_data["commit"]["verification"]["verified"]:
print(
" WARNING: The commit is not verified. Reason:",
commit.raw_data["commit"]["verification"]["reason"],
)
if pr_author_email != commit_author_email or pr_author_email != commit_committer_email:
print(" WARNING: Commit emails and GitHub PR author public email are differnt")
return wrong_commits
def main():
"""The main entry point function"""
arg_parser = ArgumentParser()
arg_parser.add_argument(
"--cfg-file",
metavar="PATH",
default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}",
)
arg_parser.add_argument(
"--pr", metavar="NUMBER", help="Get GitHub pull request with the number"
)
arg_parser.add_argument(
"--pr-state",
default="open",
choices=["open", "closed"],
help="Set GitHub pull request state",
)
arg_parser.add_argument(
"--newer", metavar="MINUTES", help="Get newly created GitHub pull request only"
)
arg_parser.add_argument(
"--check-commits",
action="store_true",
help="Check and compare git commit email with GitHub account email",
)
arg_parser.add_argument("--cfg-file", metavar="PATH", default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}")
arg_parser.add_argument("--pr", metavar="NUMBER",
help="Get GitHub pull request with the number")
arg_parser.add_argument("--pr-state", default="open", choices=["open", "closed"],
help="Set GitHub pull request state")
arg_parser.add_argument("--newer", metavar="MINUTES",
help="Get newly created GitHub pull request only")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
@@ -208,52 +100,50 @@ def main():
pulls = [gh_api.repo.get_pull(int(args.pr))]
else:
pulls = gh_api.repo.get_pulls(state=args.pr_state)
print(f"\nPRs count ({args.pr_state}):", pulls.totalCount)
print(f'\nPRs count ({args.pr_state}):', pulls.totalCount)
if args.newer:
pr_created_after = (
datetime.datetime.now() - datetime.timedelta(minutes=int(args.newer))
).astimezone()
print("Checking PRs created after:", pr_created_after)
pr_created_after = datetime.datetime.now() - datetime.timedelta(minutes=int(args.newer))
print('PRs created after:', pr_created_after)
non_org_intel_pr_users = set()
non_org_pr_users = set()
wrong_pulls = {}
for pull in pulls:
pr_created_at = pull.created_at.replace(tzinfo=datetime.timezone.utc).astimezone()
if args.newer and pr_created_at <= pr_created_after:
print(f"\nIGNORE: {get_pr_info_str(pull)}")
if args.newer and pull.created_at <= pr_created_after:
print(f'\nIGNORE: {pull} - Created: {pull.created_at}')
continue
pr_lables = get_pr_labels(pull)
pr_type_by_labels = get_pr_type_by_labels(pull)
set_labels = []
print(f'\n{pull} - Created: {pull.created_at} - Labels: {pr_lables} -',
f'Type: {pr_type_by_labels}', end='')
print(f"\n{get_pr_info_str(pull)}")
if args.check_commits:
wrong_commits = get_wrong_commits(pull)
if wrong_commits:
wrong_pulls[pull.number] = wrong_commits
# Checks PR source type
if gh_api.is_org_user(pull.user):
print(' - Org user')
elif github_api.is_intel_email(pull.user.email) or \
github_api.is_intel_company(pull.user.company):
print(' - Non org user with Intel email or company')
non_org_intel_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.INTEL:
print(f'NO "{PrType.INTEL.value}" label: ', end='')
github_api.print_users(pull.user)
set_labels.append(PrType.INTEL.value)
else:
update_labels(gh_api, pull, non_org_intel_pr_users, non_org_pr_users)
print(f' - Non org user with NO Intel email or company')
non_org_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.EXTERNAL:
print(f'NO "{PrType.EXTERNAL.value}" label: ', end='')
github_api.print_users(pull.user)
set_labels.append(PrType.EXTERNAL.value)
if wrong_pulls:
for pull_number, wrong_commits in wrong_pulls.items():
print(
f"\nERROR: Remove or replace wrong commits in the PR {pull_number}:\n ",
"\n ".join(wrong_commits),
)
print(
"\nAbout commit signature verification:\n ",
"https://docs.github.com/en/github/authenticating-to-github/"
"managing-commit-signature-verification/about-commit-signature-verification",
)
sys.exit(1)
set_labels += get_category_labels(pull)
set_pr_labels(pull, set_labels)
if non_org_intel_pr_users:
print("\nNon org user with Intel email or company:")
github_api.print_users(non_org_intel_pr_users)
if non_org_pr_users:
print("\nNon org user with NO Intel email or company:")
github_api.print_users(non_org_pr_users)
print(f'\nNon org user with Intel email or company:')
github_api.print_users(non_org_intel_pr_users)
print(f'\nNon org user with NO Intel email or company:')
github_api.print_users(non_org_pr_users)
if __name__ == "__main__":
if __name__ == '__main__':
main()
+1 -3
View File
@@ -6,9 +6,7 @@
"openvino-ci",
"openvino-pushbot",
"lab-nerval",
"lab-nerval-onnx-ci",
"onnx-watchdog-agent",
"dependabot"
"lab-nerval-onnx-ci"
],
"EMAILS_FILE_PATH": "dev_emails-test.txt",
"PROXIES": {
+10 -10
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -57,19 +57,19 @@ class Config:
for name, value in self._json_cfg.items():
if hasattr(self, name):
raise ConfigException(f'Duplicating prosperity: {name}')
property_value = self._args.get(name) or os.getenv(name)
if property_value:
prosperity_value = self._args.get(name) or os.getenv(name)
if prosperity_value:
# Try to set prosperity_value as Python literal structures, e.g. DRY_RUN=False
try:
property_value = ast.literal_eval(property_value)
prosperity_value = ast.literal_eval(prosperity_value)
except Exception:
pass
if not isinstance(property_value, type(value)):
if not isinstance(prosperity_value, type(value)):
raise ConfigException(f'Python type of {name} parameter must be {type(value)}')
else:
property_value = value
setattr(self, name, property_value)
Config.properties[name] = property_value
prosperity_value = value
setattr(self, name, prosperity_value)
Config.properties[name] = prosperity_value
self.set_proxy()
@@ -78,7 +78,7 @@ class Config:
try:
with open(self._file_path) as conf:
self._json_cfg = json.load(conf)
except Exception:
except:
print('Failed to load configuration from:', self._file_path)
raise
@@ -105,7 +105,7 @@ class Config:
def _test():
"""Test and debug"""
print('Config.default_cfg_path:', Config.default_cfg_path)
cfg = Config(cli_args=['DRY_RUN', 'PROXIES={"NO_PROXY": "localhost"}'])
cfg = Config(cli_args=['DRY_RUN=True'])
print('Config.properties:', cfg.get_properties())
+11 -6
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -11,6 +11,7 @@ import re
import time
from github import Github, GithubException, RateLimitExceededException, IncompletableObject
from github import UnknownObjectException
from github.PaginatedList import PaginatedList
from configs import Config
@@ -109,13 +110,17 @@ class GithubOrgApi:
def is_org_user(self, user):
"""Checks that user is a member of GitHub organization"""
if is_valid_user(user):
# user.get_organization_membership(self.github_org) doesn't work with org members
# permissions, GITHUB_TOKEN must be org owner now
return self.github_org.has_in_members(user)
try:
membership = user.get_organization_membership(self.github_org)
# membership.role can be 'member' or 'admin'
if membership.state == 'active' and membership.role:
return True
except UnknownObjectException:
pass
return False
def get_org_emails(self):
"""Gets and prints emails of all GitHub organization members"""
"""Gets and prints all emails of GitHub organization members"""
org_members = self.github_org.get_members()
org_emails = set()
org_members_fix = set()
@@ -141,7 +146,7 @@ class GithubOrgApi:
'; '.join(org_logins_fix_intel_email))
print(f'\nOrg members - no real name {len(org_emails_fix_name)}:',
'; '.join(org_emails_fix_name))
return (org_emails, org_logins_fix_intel_email)
return org_emails
def get_org_invitation_emails(self):
"""Gets GitHub organization teams prints info"""
-236
View File
@@ -1,236 +0,0 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Gets info about users and groups via LDAP
"""
# pylint: disable=fixme,no-member
from enum import Enum
from ldap3 import Server, Connection, ALL, SUBTREE
from configs import Config
class LdapApiException(Exception):
"""Base LDAP API exception"""
class InfoLevel(Enum):
"""Constants for printing user info from LDAP"""
PDL = 'PDL' # Public Distribution List (group of e-mail addresses)
FULL = 'Full'
def print_user_info(info, info_level=None):
"""Pretty-print of a user info data structure (dict). info_level is the InfoLevel Enum"""
if not info or not info.get('mail'):
raise LdapApiException('ERROR: No info or absent mail')
def get_membership():
if info_level == InfoLevel.PDL:
membership_info = ' PDLs:'
elif info_level == InfoLevel.FULL:
membership_info = ' memberOf :'
else:
return ''
# Grouping groups by purpose
if info_level == InfoLevel.PDL:
sort_key = lambda i: i.split(',', 1)[0].lower()
else:
sort_key = lambda i: i.split(',', 1)[1] + i.split(',', 1)[0].lower()
for item in sorted(info['memberOf'], key=sort_key):
if info_level == InfoLevel.PDL and 'OU=Delegated' not in item:
continue
membership_info += f'\n {item}'
return membership_info
try:
text_info = \
f'\n{info["cn"]} <{info["mail"]}>; {info["sAMAccountName"]}; {info["employeeID"]}' \
f'\n Org group: {info["intelSuperGroupDescr"]} ({info["intelSuperGroupShortName"]}) /'\
f' {info["intelGroupDescr"]} ({info["intelGroupShortName"]}) /' \
f' {info["intelDivisionDescr"]} ({info["intelDivisionShortName"]}) /' \
f' {info["intelOrgUnitDescr"]}' \
f'\n Manager: {info["manager"]}' \
f'\n Location: {info["intelRegionCode"]} / {info["co"]} / {info["intelSiteCode"]} /' \
f' {info["intelBldgCode"]} ({info["intelSiteName"]}) /' \
f' {info["physicalDeliveryOfficeName"]}' \
f'\n Other: {info["employeeType"]} | {info["intelExportCountryGroup"]} |' \
f' {info["whenCreated"]} | {info["intelCostCenterDescr"]} | {info["jobDescription"]}'
except Exception as exc:
raise LdapApiException(f'ERROR: Failed to get info about "{info["mail"]}". ' \
f'Exception occurred:\n{repr(exc)}') from exc
print(text_info)
membership = get_membership()
if info_level == InfoLevel.PDL and membership:
print(membership)
elif info_level == InfoLevel.FULL:
for key in sorted(info):
if isinstance(info[key], list):
if key == 'memberOf':
print(membership)
else:
print(f' {key} :')
for item in info[key]:
print(' ', item)
else:
print(f' {key} : {info[key]}')
class LdapApi:
"""LDAP API for getting user info and emails"""
_binary_blobs = ['thumbnailPhoto', 'msExchUMSpokenName', 'msExchBlockedSendersHash']
_check_existing = [
'intelExportCountryGroup',
'physicalDeliveryOfficeName',
'intelSuperGroupShortName',
'intelGroupShortName',
'intelDivisionShortName',
]
null = '<null>'
def __init__(self):
self._cfg = Config()
self.server = Server(self._cfg.LDAP_SERVER, get_info=ALL)
self.connection = Connection(self.server,
user=self._cfg.LDAP_USER,
password=self._cfg.LDAP_PASSWORD,
auto_bind=True)
self.connection.bind()
def get_user_emails(self, groups=None):
"""Gets emails of LDAP groups and sub-groups"""
print('\nGet emails from LDAP groups:')
processed_ldap_members = {}
def process_group_members(member, parent_group):
if member in processed_ldap_members:
processed_ldap_members[member]['parent_groups'].append(parent_group)
print('\nWARNING: Ignore LDAP member to avoid duplication and recursive cycling '
f'of PDLs: {member}\n '
f'email: {processed_ldap_members[member].get("email")}\n parent_groups:')
for group in processed_ldap_members[member].get('parent_groups', []):
print(7 * ' ', group)
return
processed_ldap_members[member] = {'email': None, 'parent_groups': [parent_group]}
# AD moves terminated users to the boneyard OU in case the user returns,
# so it can be reactivated with little effort.
# After 30 days it is removed and the unix personality becomes unlinked.
if 'OU=Boneyard' in member:
return
self.connection.search(member, r'(objectClass=*)', SUBTREE,
attributes=['cn', 'member', 'mail'])
#print(self.connection.entries)
if not self.connection.response:
raise LdapApiException(f'ERROR: empty response. LDAP member: {member}')
# Check that the member is worker.
# The response can contain several items, but the first item is valid only
if 'OU=Workers' in member:
if self.connection.response[0]['attributes']['mail']:
processed_ldap_members[member]['email'] = \
self.connection.response[0]['attributes']['mail'].lower()
return
raise LdapApiException(f'ERROR: no mail. LDAP worker: {member}\n'
f'{self.connection.entries}')
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {member}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
if self.connection.response[0]['attributes']['member']:
for group_member in self.connection.response[0]['attributes']['member']:
process_group_members(group_member, member)
else:
print(f'\nERROR: no members in LDAP group: {member}\n{self.connection.entries}')
for group in groups or self._cfg.LDAP_PDLs:
print('\nProcess ROOT LDAP group:', group)
process_group_members(group, 'ROOT')
return {
member.get('email') for member in processed_ldap_members.values() if member.get('email')
}
def _get_user_info(self, query):
"""Gets user info from LDAP as dict matching key and values pairs from query"""
query_filter = ''.join(f'({key}={value})' for key, value in query.items())
for domain in self._cfg.LDAP_DOMAINS:
search_base = f'OU=Workers,DC={domain},DC=corp,DC=intel,DC=com'
self.connection.search(
search_base,
f'(&(objectcategory=person)(objectclass=user)(intelflags=1){query_filter})',
SUBTREE,
attributes=['*'])
if self.connection.response:
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {query_filter}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
info = self.connection.response[0]['attributes']
# remove long binary blobs
for blob in LdapApi._binary_blobs:
info[blob] = b''
for key in LdapApi._check_existing:
if not info.get(key):
info[key] = LdapApi.null
return info
return {}
def get_user_info_by_idsid(self, idsid):
"""Gets user info from LDAP as dict using account name for searching"""
return self._get_user_info({'sAMAccountName': idsid})
def get_user_info_by_name(self, name):
"""Gets user info from LDAP as dict using common name for searching"""
return self._get_user_info({'cn': name})
def get_user_info_by_email(self, email):
"""Gets user info from LDAP as dict using emails for searching"""
return self._get_user_info({'mail': email})
def get_absent_emails(self, emails):
"""Checks users by email in LDAP and returns absent emails"""
absent_emails = set()
for email in emails:
if not self.get_user_info_by_email(email):
absent_emails.add(email)
return absent_emails
def _test():
"""Test and debug"""
ldap = LdapApi()
emails = ldap.get_user_emails()
print(f'\nLDAP emails count: {len(emails)}\n{"; ".join(emails)}')
emails = ['foo@intel.com']
for email in emails:
info = ldap.get_user_info_by_email(email)
if info:
print_user_info(info, InfoLevel.PDL)
else:
print(f'\n{email} - not found')
if __name__ == '__main__':
_test()
-1
View File
@@ -1 +0,0 @@
pylint==2.5.3
-1
View File
@@ -1,2 +1 @@
PyGithub==1.51
ldap3==2.7
+1
View File
@@ -0,0 +1 @@
pylint==2.3.0
+7 -42
View File
@@ -3,7 +3,6 @@ on: [push, pull_request]
jobs:
Build_Doc:
if: github.repository == 'openvinotoolkit/openvino'
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
@@ -14,25 +13,14 @@ jobs:
- name: Install dependencies
run: |
set -e
# install doc dependencies
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev graphviz texlive
cd docs
python -m pip install -r requirements.txt --user
cd openvino_sphinx_theme
python setup.py install --user
cd ../..
# install doxyrest
wget https://github.com/vovkos/doxyrest/releases/download/doxyrest-2.1.3/doxyrest-2.1.3-linux-amd64.tar.xz
tar -xf doxyrest-2.1.3-linux-amd64.tar.xz
echo "$(pwd)/doxyrest-2.1.3-linux-amd64/bin/" >> $GITHUB_PATH
python3 -m pip install lxml
# install doxygen
mkdir doxygen
cd doxygen
git clone https://github.com/doxygen/doxygen.git
cd doxygen
git checkout Release_1_9_2
git checkout Release_1_9_1
mkdir build
cd build
cmake ..
@@ -43,37 +31,14 @@ jobs:
run: |
mkdir build
cd build
cmake -DENABLE_DOCS=ON -DENABLE_PYTHON=ON -DNGRAPH_PYTHON_BUILD_ENABLE=ON -DCMAKE_BUILD_TYPE=Release ..
cmake -DENABLE_DOCS=ON ..
- name: Build doc
run: |
cmake --build . --target sphinx_docs
run: cmake --build . --target openvino_docs
working-directory: build
- name: Archive HTML
run: |
zip -r openvino_html.zip _build
working-directory: build/docs
- name: Run Pytest
run: |
pytest --doxygen="./build/docs/doxygen.log" \
--confcutdir="./docs/scripts/tests/" \
--html="./build/docs/_artifacts/doc-generation.html" \
--doxygen-strip="$(pwd)" \
--doxygen-xfail="./docs/doxygen-xfail.txt" \
--self-contained-html ./docs/scripts/tests/test_docs.py
- name: 'Upload test results'
if: always()
- name: 'Upload doc'
uses: actions/upload-artifact@v2
with:
name: openvino_doc_pytest
path: build/docs/_artifacts/
- name: 'Upload html'
if: github.event_name == 'push'
uses: actions/upload-artifact@v2
with:
name: openvino_html
path: build/docs/openvino_html.zip
name: openvino_doc
path: build/docs/html/
-17
View File
@@ -1,17 +0,0 @@
name: PR Commits
on: [pull_request]
jobs:
Checks:
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@v2
- name: Install dependencies
run: python3 -m pip install -r ./.github/org_control/requirements.txt
- name: PR commits
run: python3 ./.github/org_control/check_pr.py --pr=${{ github.event.number }} --check-commits DRY_RUN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+12 -20
View File
@@ -2,48 +2,42 @@ name: Code Style
on: [push, pull_request]
jobs:
clang-format:
runs-on: ubuntu-20.04
nGraph:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Install clang-format-9
run: |
sudo apt update
sudo apt --assume-yes install clang-format-9
- name: Install clang-format-3.9
run: sudo apt --assume-yes install clang-format-3.9
- name: Install dependencies
run: |
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install --upgrade pip
python3 -m pip install -r ./inference-engine/ie_bridges/python/requirements.txt
# Add for -DENABLE_PYTHON=ON, no cython
python3 -m pip install -r ./inference-engine/ie_bridges/python/src/requirements-dev.txt
# Run cmake with -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT in order to enable codestyle check for ITT collector
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_PYTHON=ON -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT ..
cmake ..
- name: Check code style
run: cmake --build build --target clang_format_check_all
run: make style-check
working-directory: build
- name: Create code style diff
if: failure()
run: |
cmake --build build --target clang_format_fix_all
git diff > code_style_diff.diff
ngraph/maint/apply-code-format.sh
git diff >ngraph_code_style_diff.patch
- uses: actions/upload-artifact@v2
if: failure()
with:
name: code_style_diff
path: code_style_diff.diff
name: ngraph_code_style_diff
path: ngraph_code_style_diff.patch
ShellCheck:
runs-on: ubuntu-18.04
@@ -53,9 +47,7 @@ jobs:
submodules: recursive
- name: Install ShellCheck
run: |
sudo apt update
sudo apt --assume-yes install shellcheck
run: sudo apt --assume-yes install shellcheck
- name: Install dependencies
run: |
+1 -2
View File
@@ -41,7 +41,6 @@ jobs:
pip install -r requirements.txt
pip install -r requirements_dev.txt
# requrements for CMake
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev
working-directory: model-optimizer
@@ -96,7 +95,7 @@ jobs:
wget -q http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz
tar -xf mobilenet_v1_1.0_224.tgz
python3 -m pip install model-optimizer/dist/*.whl
python3 -m mo --input_model mobilenet_v1_1.0_224_frozen.pb --input_shape "[1,224,224,3]"
python3 -c "import sys, subprocess, mo_tf; subprocess.run([sys.executable, mo_tf.__file__, '--input_model', 'mobilenet_v1_1.0_224_frozen.pb', '--input_shape', '[1,224,224,3]'], check=True)"
- uses: actions/upload-artifact@v2
with:
-75
View File
@@ -1,75 +0,0 @@
name: IE Python Checks
on:
workflow_dispatch:
push:
paths:
- 'inference-engine/ie_bridges/python/**'
pull_request:
paths:
- 'inference-engine/ie_bridges/python/**'
jobs:
linters:
runs-on: ubuntu-18.04
steps:
- name: Code checkout
uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.6'
- name: Install dependencies
run: python -m pip install -r inference-engine/ie_bridges/python/requirements_dev.txt
- name: Run Flake on samples
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/sample
- name: Create code style diff for samples
if: failure()
run: |
python -m black -l 160 -S ./
git diff > samples_diff.diff
working-directory: inference-engine/ie_bridges/python/sample
- uses: actions/upload-artifact@v2
if: failure()
with:
name: samples_diff
path: samples_diff.diff
- name: Run Flake on src
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/src
- name: Create code style diff for Python src
if: failure()
run: |
python -m black -l 160 -S ./
git diff > src_diff.diff
working-directory: inference-engine/ie_bridges/python/src
- uses: actions/upload-artifact@v2
if: failure()
with:
name: src_diff
path: src_diff.diff
- name: Run Flake on wheel
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/wheel
- name: Create code style diff for wheel
if: failure()
run: |
python -m black -l 160 -S ./
git diff > wheel_diff.diff
working-directory: inference-engine/ie_bridges/python/wheel
- uses: actions/upload-artifact@v2
if: failure()
with:
name: wheel_diff
path: wheel_diff.diff
- name: Run MyPy
run: python -m mypy ./ --config-file ./setup.cfg
working-directory: inference-engine/ie_bridges/python
- name: Run Bandit
run: python -m bandit -r ./ -f screen
working-directory: inference-engine/ie_bridges/python
-2
View File
@@ -2,8 +2,6 @@
_*
# but ensure we don't skip __init__.py
!__init__.py
# and sphinx documentation folders
!docs/_*
# developer tools
*.idea
-4
View File
@@ -18,7 +18,3 @@
path = thirdparty/xbyak
url = https://github.com/herumi/xbyak.git
ignore = dirty
[submodule "thirdparty/zlib/zlib"]
path = thirdparty/zlib/zlib
url = https://github.com/madler/zlib.git
ignore = dirty
+7 -13
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -30,7 +30,7 @@ message (STATUS "CMAKE_C_COMPILER_ID ................... " ${CMAKE_C_COMPILER_ID
message (STATUS "CMAKE_BUILD_TYPE ...................... " ${CMAKE_BUILD_TYPE})
# remove file with exported developer targets to force its regeneration
file(REMOVE "${CMAKE_BINARY_DIR}/InferenceEngineTargets.cmake")
file(REMOVE "${CMAKE_BINARY_DIR}/inference_engine_targets.cmake")
foreach(component IN LISTS openvino_export_components)
file(REMOVE "${CMAKE_BINARY_DIR}/${component}_dev_targets.cmake")
unset(${component} CACHE)
@@ -52,6 +52,7 @@ function(build_ngraph)
else ()
ngraph_set(NGRAPH_ADDRESS_SANITIZER OFF)
endif ()
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE OFF)
if(ENABLE_TESTS AND NOT ANDROID)
ngraph_set(NGRAPH_UNIT_TEST_ENABLE ON)
@@ -84,13 +85,7 @@ function(build_ngraph)
ngraph_set(NGRAPH_THREAD_SANITIZER_ENABLE OFF)
endif()
if(ENABLE_PYTHON)
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE ON)
else()
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE OFF)
endif()
if(OV_COMPILER_IS_CLANG)
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
ie_add_compiler_flags(-Wno-error=uninitialized -Wno-error=literal-conversion)
elseif(UNIX)
ie_add_compiler_flags(-Wno-error=maybe-uninitialized -Wno-error=return-type)
@@ -107,10 +102,10 @@ function(build_ngraph)
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION_RELEASE ON)
endif()
ie_cpack_add_component(ngraph REQUIRED)
ie_cpack_add_component(ngraph_dev REQUIRED DEPENDS ngraph)
ie_cpack_add_component(ngraph)
set(SDL_cmake_included ON)
set(NGRAPH_COMPONENT_PREFIX "deployment_tools/ngraph/")
add_subdirectory(ngraph)
set(NGRAPH_LIBRARIES ngraph PARENT_SCOPE)
set(NGRAPH_REF_LIBRARIES ngraph_reference PARENT_SCOPE)
@@ -154,7 +149,6 @@ build_ngraph()
add_subdirectory(inference-engine)
add_subdirectory(model-optimizer)
add_subdirectory(docs)
add_subdirectory(tools)
#
# Shellcheck
@@ -203,7 +197,7 @@ endif()
# install files for demo
ie_cpack_add_component(demo_scripts DEPENDS core)
ie_cpack_add_component(demo_scripts REQUIRED DEPENDS core)
if(UNIX)
install(DIRECTORY scripts/demo/
Vendored
-1
View File
@@ -13,7 +13,6 @@ properties([
name: 'library_version')
])
])
loadOpenVinoLibrary {
entrypoint(this)
}
+3 -4
View File
@@ -1,5 +1,5 @@
# OpenVINO™ Toolkit
[![Stable release](https://img.shields.io/badge/version-2021.4.2-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2021.4.2)
[![Stable release](https://img.shields.io/badge/version-2021.3-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2021.3)
[![Apache License Version 2.0](https://img.shields.io/badge/license-Apache_2.0-green.svg)](LICENSE)
![GitHub branch checks state](https://img.shields.io/github/checks-status/openvinotoolkit/openvino/master?label=GitHub%20checks)
![Azure DevOps builds (branch)](https://img.shields.io/azure-devops/build/openvinoci/b2bab62f-ab2f-4871-a538-86ea1be7d20f/13?label=Public%20CI)
@@ -42,9 +42,8 @@ Please report questions, issues and suggestions using:
---
\* Other names and brands may be claimed as the property of others.
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
[Open Model Zoo]:https://github.com/opencv/open_model_zoo
[Inference Engine]:https://software.intel.com/en-us/articles/OpenVINO-InferEngine
[Model Optimizer]:https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer
[nGraph]:https://docs.openvinotoolkit.org/latest/openvino_docs_nGraph_DG_DevGuide.html
[tag on StackOverflow]:https://stackoverflow.com/search?q=%23openvino
[nGraph]:https://docs.openvinotoolkit.org/latest/openvino_docs_nGraph_DG_DevGuide.html
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -12,7 +12,7 @@
# 2) ${TBBROOT} with IE own version of TBBConfig.cmake (actual for TBB < 2017.7)
#
# Path to IE own version of TBBConfig.cmake old TBB version without cmake config.
## Path to IE own version of TBBConfig.cmake old TBB version without cmake config.
if(APPLE)
set(IE_OWN_TBB_CONFIG tbb/mac)
elseif(UNIX)
@@ -27,7 +27,6 @@ find_package(TBB
CONFIG
PATHS ${TBBROOT}/cmake
${IEDevScripts_DIR}/${IE_OWN_TBB_CONFIG}
NO_CMAKE_FIND_ROOT_PATH
NO_DEFAULT_PATH
)
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -14,13 +14,7 @@ set(CMAKE_MODULE_PATH "${IEDevScripts_DIR}")
function(set_ci_build_number)
set(repo_root "${CMAKE_SOURCE_DIR}")
include(version)
foreach(var CI_BUILD_NUMBER IE_VERSION
IE_VERSION_MAJOR IE_VERSION_MINOR IE_VERSION_PATCH)
if(NOT DEFINED ${var})
message(FATAL_ERROR "${var} version component is not defined")
endif()
set(${var} "${${var}}" PARENT_SCOPE)
endforeach()
set(CI_BUILD_NUMBER "${CI_BUILD_NUMBER}" PARENT_SCOPE)
endfunction()
set_ci_build_number()
@@ -53,6 +47,9 @@ function(set_temp_directory temp_variable source_tree_dir)
if (DEFINED ENV{DL_SDK_TEMP} AND NOT $ENV{DL_SDK_TEMP} STREQUAL "")
message(STATUS "DL_SDK_TEMP environment is set : $ENV{DL_SDK_TEMP}")
file(TO_CMAKE_PATH $ENV{DL_SDK_TEMP} temp)
if (ENABLE_ALTERNATIVE_TEMP)
set(ALTERNATIVE_PATH ${source_tree_dir}/temp)
endif()
else ()
set(temp ${source_tree_dir}/temp)
endif()
@@ -205,9 +202,6 @@ endif()
# General flags
set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package(Threads REQUIRED)
include(compile_flags/sdl)
include(compile_flags/os_flags)
include(compile_flags/sanitizer)
@@ -223,7 +217,6 @@ include(api_validator/api_validator)
include(vs_version/vs_version)
include(plugins/plugins)
include(add_ie_target)
include(CMakePackageConfigHelpers)
if(ENABLE_FUZZING)
enable_fuzzing()
@@ -232,8 +225,6 @@ endif()
# macro to mark target as conditionally compiled
function(ie_mark_target_as_cc TARGET_NAME)
target_link_libraries(${TARGET_NAME} PRIVATE openvino::conditional_compilation)
if(NOT (SELECTIVE_BUILD STREQUAL "ON"))
return()
endif()
+2 -6
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -31,7 +31,6 @@ addIeTarget(
function(addIeTarget)
set(options
ADD_CPPLINT # Enables code style checks for the target
ADD_CLANG_FORMAT # Enables code style checks for the target
)
set(oneValueRequiredArgs
TYPE # type of target, SHARED|STATIC|EXECUTABLE. SHARED and STATIC correspond to add_library, EXECUTABLE to add_executable
@@ -120,10 +119,6 @@ function(addIeTarget)
# code style
add_cpplint_target(${ARG_NAME}_cpplint FOR_TARGETS ${ARG_NAME})
endif()
if (ARG_ADD_CLANG_FORMAT)
# code style
add_clang_format_target(${ARG_NAME}_clang FOR_TARGETS ${ARG_NAME})
endif()
if (ARG_DEVELOPER_PACKAGE)
# developer package
openvino_developer_export_targets(COMPONENT ${ARG_DEVELOPER_PACKAGE}
@@ -133,6 +128,7 @@ function(addIeTarget)
# Provide default compile pdb name equal to target name
set_target_properties(${ARG_NAME} PROPERTIES COMPILE_PDB_NAME ${ARG_NAME})
endif()
endfunction()
#[[
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,7 +1,6 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if (ENABLE_CLANG_FORMAT)
set(CLANG_FORMAT_FILENAME clang-format-9 clang-format)
find_program(CLANG_FORMAT NAMES ${CLANG_FORMAT_FILENAME} PATHS ENV PATH)
@@ -17,13 +16,10 @@ if (ENABLE_CLANG_FORMAT)
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
else()
message(WARNING "Supported clang-format version is not found!")
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
if(ENABLE_CLANG_FORMAT AND NOT TARGET clang_format_check_all)
if(ENABLE_CLANG_FORMAT)
add_custom_target(clang_format_check_all)
add_custom_target(clang_format_fix_all)
set_target_properties(clang_format_check_all clang_format_fix_all
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -7,11 +7,12 @@ file(REMOVE "${OUTPUT_FILE}")
execute_process(COMMAND ${CLANG_FORMAT} -style=file -output-replacements-xml ${INPUT_FILE}
OUTPUT_VARIABLE STYLE_CHECK_RESULT
)
# Display the cpplint output to console (to parse it form IDE)
message("${output}")
file(WRITE "${OUTPUT_FILE}" "${STYLE_CHECK_RESULT}")
if(NOT SKIP_RETURN_CODE)
if("${STYLE_CHECK_RESULT}" MATCHES ".*<replacement .*")
message(FATAL_ERROR "[clang-format] Code style check failed for: ${INPUT_FILE}")
message(FATAL_ERROR "[clang-format] Code style check failed for : ${INPUT_FILE}")
endif()
endif()
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -38,10 +38,9 @@ endmacro()
macro(ie_deprecated_no_errors)
if(WIN32)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(ie_c_cxx_deprecated_no_errors "/Qdiag-warning:1478,1786")
set(ie_c_cxx_deprecated "/Qdiag-warning:1478,1786")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# show 4996 only for /w4
set(ie_c_cxx_deprecated_no_errors "/w44996")
set(ie_c_cxx_deprecated "/wd4996")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
@@ -184,6 +183,7 @@ endfunction()
#
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(THREADS_PREFER_PTHREAD_FLAG ON)
# to allows to override CMAKE_CXX_STANDARD from command line
if(NOT DEFINED CMAKE_CXX_STANDARD)
@@ -211,16 +211,6 @@ set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_C_VISIBILITY_PRESET hidden)
set(CMAKE_VISIBILITY_INLINES_HIDDEN ON)
function(ie_python_minimal_api target)
# pybind11 uses a lot of API which is not a part of minimal python API subset
# Ref 1: https://docs.python.org/3.11/c-api/stable.html
# Ref 2: https://github.com/pybind/pybind11/issues/1755
# target_compile_definitions(${target} PRIVATE Py_LIMITED_API=0x03090000)
# if(NOT CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# target_compile_options(${target} PRIVATE "-Wno-unused-variable")
# endif()
endfunction()
if(WIN32)
ie_add_compiler_flags(-D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS)
ie_add_compiler_flags(/EHsc) # no asynchronous structured exception handling
@@ -1,9 +1,17 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(CheckCXXCompilerFlag)
if (ENABLE_SANITIZER OR ENABLE_THREAD_SANITIZER)
# This is workaround for https://gitlab.kitware.com/cmake/cmake/-/issues/16609.
# It ensures pthread is searched without ASAN linking.
# Line bellow must be before adding -fsanitize=address or -fsanitize=thread to
# build options for the trick to work.
find_package(Threads REQUIRED)
endif()
if (ENABLE_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=address -fno-omit-frame-pointer")
CHECK_CXX_COMPILER_FLAG("-fsanitize-recover=address" SANITIZE_RECOVER_SUPPORTED)
@@ -12,12 +20,9 @@ if (ENABLE_SANITIZER)
endif()
set(SANITIZER_LINKER_FLAGS "-fsanitize=address")
# prevent unloading libraries at runtime, so sanitizer can resolve their symbols
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=gold")
elseif(OV_COMPILER_IS_CLANG AND NOT WIN32)
elseif(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$" AND NOT WIN32)
if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=lld")
endif()
@@ -33,9 +38,7 @@ endif()
if (ENABLE_THREAD_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=thread -fno-omit-frame-pointer")
set(SANITIZER_LINKER_FLAGS "-fsanitize=thread")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(OV_COMPILER_IS_CLANG AND NOT WIN32)
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$" AND NOT WIN32)
if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=lld")
else()
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -23,7 +23,7 @@ if (CMAKE_BUILD_TYPE STREQUAL "Release")
if (NOT ENABLE_SANITIZER)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -s")
endif()
elseif(OV_COMPILER_IS_CLANG)
elseif(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-all")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
if (NOT ENABLE_SANITIZER)
@@ -1,7 +1,6 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(OV_COVERAGE_GCDA_DATA_DIRECTORY "${CMAKE_BINARY_DIR}")
if(NOT TARGET ie_coverage_clean)
add_custom_target(ie_coverage_clean)
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,17 +1,17 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_CPPLINT)
find_package(PythonInterp 3 QUIET)
find_package(Python3 COMPONENTS Interpreter)
if(NOT PYTHONINTERP_FOUND)
if(NOT Python3_Interpreter_FOUND)
message(WARNING "Python3 interpreter was not found (required for cpplint check)")
set(ENABLE_CPPLINT OFF)
endif()
endif()
if(ENABLE_CPPLINT AND NOT TARGET cpplint_all)
if(ENABLE_CPPLINT)
add_custom_target(cpplint_all ALL)
set_target_properties(cpplint_all PROPERTIES FOLDER cpplint)
set(CPPLINT_ALL_OUTPUT_FILES "" CACHE INTERNAL "All cpplint output files")
@@ -68,7 +68,6 @@ function(add_cpplint_target TARGET_NAME)
"${output_file}"
COMMAND
"${CMAKE_COMMAND}"
-D "PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE}"
-D "CPPLINT_SCRIPT=${IEDevScripts_DIR}/cpplint/cpplint.py"
-D "INPUT_FILE=${source_file}"
-D "OUTPUT_FILE=${output_file}"
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -25,7 +25,7 @@ set(FILTER "${DEFAULT_FILTER}${CUSTOM_FILTER}")
execute_process(
COMMAND
"${PYTHON_EXECUTABLE}"
python3
"${CPPLINT_SCRIPT}"
"--linelength=160"
"--counting=detailed"
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -10,7 +10,7 @@
# XARCH_FUNC_NAME -- name of function to dispatch
# XARCH_NAMESPACES -- full namespace used to keep ODR
# XARCH_DISP_FILE -- dispatcher file name to generate
# XARCH_SET -- set of ARCH supported by dispatcher. semicolon-delimited
# XARCH_SET -- set of ARCH supported by dispatcher. space delimited
#
# =================================================================
@@ -24,6 +24,7 @@ function(_generate_dispatcher)
_find_signature_in_file(${XARCH_API_HEADER} ${XARCH_FUNC_NAME} SIGNATURE)
_generate_call_line_from_signature("${SIGNATURE}" CALL_LINE)
string(REPLACE " " ";" XARCH_SET "${XARCH_SET}")
string(REPLACE "::" ";" XARCH_NAMESPACES "${XARCH_NAMESPACES}")
list(GET XARCH_NAMESPACES -1 XARCH_CURRENT_NAMESPACE)
@@ -1,4 +1,4 @@
# Copyright (C) 2020-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -13,10 +13,10 @@ set(_ACCEPTED_ARCHS_AVX512F "^(ANY|SSE42|AVX|AVX2|AVX512F)$")
## Arch specific definitions
set(_DEFINE_ANY "")
set(_DEFINE_SSE42 "HAVE_SSE42" ${_DEFINE_ANY})
set(_DEFINE_AVX "HAVE_AVX" ${_DEFINE_SSE42})
set(_DEFINE_AVX2 "HAVE_AVX2" ${_DEFINE_AVX})
set(_DEFINE_AVX512F "HAVE_AVX512F" ${_DEFINE_AVX2})
set(_DEFINE_SSE42 "-DHAVE_SSE42" ${_DEFINE_ANY})
set(_DEFINE_AVX "-DHAVE_AVX" ${_DEFINE_SSE42})
set(_DEFINE_AVX2 "-DHAVE_AVX2" ${_DEFINE_AVX})
set(_DEFINE_AVX512F "-DHAVE_AVX512F" ${_DEFINE_AVX2})
## Arch specific compile options
ie_avx512_optimization_flags(_FLAGS_AVX512F)
@@ -117,25 +117,21 @@ function(_clone_source_to_target TARGET SOURCE ARCH_SET)
${CMAKE_CURRENT_SOURCE_DIR}/${SOURCE}
${CMAKE_CURRENT_BINARY_DIR}/${ARCH_SOURCE}
DEPENDS ${SOURCE}
VERBATIM
)
set_property(SOURCE ${ARCH_SOURCE} APPEND_STRING PROPERTY COMPILE_FLAGS
" ${_FLAGS_${_arch}}")
set_property(SOURCE ${ARCH_SOURCE} APPEND PROPERTY COMPILE_DEFINITIONS
set(_ARCH_SPECIFIC_FLAGS
${_DEFINE_${_arch}}
"XARCH=${_arch}" ## to replace XARCH with direct ARCH name
${_FLAGS_${_arch}}
"-DXARCH=${_arch}" ## to replace XARCH with direct ARCH name
"-I${CMAKE_CURRENT_SOURCE_DIR}/${ARCH_INCLUDE_DIR}" ## To make valid #include "some.hpp"
)
## To make `#include "some.hpp"` valid
set_property(SOURCE ${ARCH_SOURCE} APPEND PROPERTY INCLUDE_DIRECTORIES
"${CMAKE_CURRENT_SOURCE_DIR}/${ARCH_INCLUDE_DIR}")
_add_source_compile_flags(${ARCH_SOURCE} ${_ARCH_SPECIFIC_FLAGS})
list(APPEND _ARCH_SOURCES ${ARCH_SOURCE})
endforeach()
target_sources(${TARGET} PRIVATE ${_ARCH_SOURCES})
_add_source_to_target(${TARGET} ${_ARCH_SOURCES})
endfunction()
@@ -150,27 +146,26 @@ function(_add_dispatcher_to_target TARGET HEADER FUNC_NAME NAMESPACE ARCH_SET)
set(DISPATCHER_SOURCE "cross-compiled/${DISPATCHER_NAME}_disp.cpp")
set(DISPATCHER_OPT_HOLDER "cross-compiled/${DISPATCHER_NAME}_holder.txt")
set(_GEN_ARGS_LIST
-DXARCH_FUNC_NAME="${X_NAME}"
-DXARCH_NAMESPACES="${NAMESPACE}"
-DXARCH_API_HEADER="${CMAKE_CURRENT_SOURCE_DIR}/${HEADER}"
-DXARCH_DISP_FILE="${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_SOURCE}"
-DXARCH_SET="${ARCH_SET}"
)
configure_file(${DISPATCHER_GEN_OPTIONS_HOLDER} ${DISPATCHER_OPT_HOLDER})
add_custom_command(
OUTPUT ${DISPATCHER_SOURCE}
COMMAND ${CMAKE_COMMAND}
-D "XARCH_FUNC_NAME=${X_NAME}"
-D "XARCH_NAMESPACES=${NAMESPACE}"
-D "XARCH_API_HEADER=${CMAKE_CURRENT_SOURCE_DIR}/${HEADER}"
-D "XARCH_DISP_FILE=${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_SOURCE}"
-D "XARCH_SET=${ARCH_SET}"
COMMAND ${CMAKE_COMMAND} ${_GEN_ARGS_LIST}
-P ${DISPATCHER_GEN_SCRIPT}
DEPENDS ${HEADER}
${DISPATCHER_GEN_SCRIPT}
${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_OPT_HOLDER} ## Just to make run dependency on args value
VERBATIM
)
set_property(SOURCE ${DISPATCHER_SOURCE} APPEND PROPERTY INCLUDE_DIRECTORIES
"${CMAKE_CURRENT_SOURCE_DIR}/${DISPATCHER_INCLUDE_DIR}")
target_sources(${TARGET} PRIVATE ${DISPATCHER_SOURCE})
_add_source_compile_flags(${DISPATCHER_SOURCE} "-I${DISPATCHER_INCLUDE_DIR}")
_add_source_to_target(${TARGET} ${DISPATCHER_SOURCE})
endfunction()
#######################################
@@ -204,3 +199,29 @@ function(_remove_source_from_target TARGET SOURCE_FILE)
PROPERTIES
SOURCES "${ORIGINAL_SOURCES}")
endfunction()
function(_add_source_to_target TARGET)
get_target_property(ORIGINAL_SOURCES ${TARGET} SOURCES)
list(APPEND ORIGINAL_SOURCES ${ARGN})
set_target_properties(${TARGET}
PROPERTIES
SOURCES "${ORIGINAL_SOURCES}")
endfunction()
function(_add_source_compile_flags SOURCE)
get_source_file_property(ORIGINAL_FLAGS ${SOURCE} COMPILE_FLAGS)
## Empty list of COMPILE_FLAGS represented as NOTFOUND
if(NOT ORIGINAL_FLAGS)
set(ORIGINAL_FLAGS "")
endif()
string(REPLACE ";" " " NEW_FLAGS "${ARGN}")
string(APPEND ORIGINAL_FLAGS " " ${NEW_FLAGS})
set_source_files_properties(${SOURCE}
PROPERTIES
COMPILE_FLAGS "${ORIGINAL_FLAGS}")
endfunction()
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,10 +1,10 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include (download/download)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256 FILES_TO_EXTRACT)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256)
if (ENVIRONMENT AND (DEFINED ${ENVIRONMENT} OR DEFINED ENV{${ENVIRONMENT}}))
set(HAS_ENV "TRUE")
endif()
@@ -12,9 +12,9 @@ function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHI
if (NOT DEFINED HAS_ENV)
if (ARCHIVE)
#TODO: check whether this is platform specific binary with same name per or it is in common folder
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256})
else()
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256})
endif()
set (${VAR} ${result_path} PARENT_SCOPE)
@@ -53,7 +53,7 @@ endfunction(read_version)
function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
list(REMOVE_AT ARGV 0)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT GITHUB_PULL_REQUEST VERSION_REGEX SHA256 FILES_TO_EXTRACT)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT GITHUB_PULL_REQUEST VERSION_REGEX SHA256)
#unnecessary vars
@@ -116,9 +116,6 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
message(FATAL_ERROR "SHA is not specified for: " ${NAME_OF_CMAKE_VAR})
endif()
if (NOT DEFINED FILES_TO_EXTRACT)
set (FILES_TO_EXTRACT FALSE)
endif()
#for each dependency type have to do separate things
if (ARCHIVE_WIN OR ARCHIVE_LIN OR ARCHIVE_MAC OR ARCHIVE_ANDROID OR ARCHIVE OR ARCHIVE_UNIFIED)
@@ -126,7 +123,7 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
message(FATAL_ERROR "TARGET_PATH should be defined for every dependency")
endif()
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256} ${FILES_TO_EXTRACT})
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256})
set(${NAME_OF_CMAKE_VAR} ${RESULT} PARENT_SCOPE)
if (VERSION_REGEX)
GetNameAndUrlToDownload(archive RELATIVE_URL ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID})
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -42,26 +42,25 @@ function (DownloadAndExtractPlatformSpecific
unpacked_path
result_path
folder
sha256
files_to_extract)
sha256)
GetNameAndUrlToDownload(archive_name RELATIVE_URL ${archive_name_unified} ${archive_name_win} ${archive_name_lin} ${archive_name_mac} ${archive_name_android} )
if (NOT archive_name OR NOT RELATIVE_URL)
return()
endif()
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256})
set (${result_path} ${result_path2} PARENT_SCOPE)
endfunction(DownloadAndExtractPlatformSpecific)
#download from common folder
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256 files_to_extract)
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256)
set (RELATIVE_URL "${archive_name}")
set(fattal TRUE)
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256})
if (NOT ${result})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder} ${sha256} ${files_to_extract})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder})
endif()
set (${result_path} ${result_path2} PARENT_SCOPE)
@@ -69,7 +68,7 @@ function (DownloadAndExtract component archive_name unpacked_path result_path fo
endfunction(DownloadAndExtract)
function (DownloadAndExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256 files_to_extract)
function (DownloadAndExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256)
set (status "ON")
DownloadAndCheck(${URL} ${archive_path} ${fattal} result1 ${sha256})
if ("${result1}" STREQUAL "ARCHIVE_DOWNLOAD_FAIL")
@@ -84,17 +83,17 @@ function (DownloadAndExtractInternal URL archive_path unpacked_path folder fatt
endif()
if("${status}" STREQUAL "ON")
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result ${files_to_extract})
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result)
endif()
set (${resultExt} ${status} PARENT_SCOPE)
endfunction(DownloadAndExtractInternal)
function (ExtractWithVersion URL archive_path unpacked_path folder result files_to_extract)
function (ExtractWithVersion URL archive_path unpacked_path folder result)
debug_message("ExtractWithVersion : ${archive_path} : ${unpacked_path} : ${folder} : ${files_to_extract}")
extract(${archive_path} ${unpacked_path} ${folder} ${files_to_extract} status)
debug_message("ExtractWithVersion : ${archive_path} : ${unpacked_path}")
extract(${archive_path} ${unpacked_path} ${folder} status)
#dont need archive actually after unpacking
file(REMOVE_RECURSE "${archive_path}")
if (${status})
@@ -107,20 +106,20 @@ function (ExtractWithVersion URL archive_path unpacked_path folder result files_
set (${result} ${status} PARENT_SCOPE)
endfunction (ExtractWithVersion)
function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256 files_to_extract)
function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256)
debug_message("checking wether archive downloaded : ${archive_path}")
set (downloadStatus "NOTOK")
if (NOT EXISTS ${archive_path})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
else()
if (ENABLE_UNSAFE_LOCATIONS)
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result ${files_to_extract})
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result)
if(NOT ${result})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
@@ -128,7 +127,7 @@ function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal
else()
debug_message("archive found on FS : ${archive_path}, however we cannot check it's checksum and think that it is invalid")
file(REMOVE_RECURSE "${archive_path}")
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
@@ -148,7 +147,7 @@ endfunction(DownloadOrExtractInternal)
file(REMOVE ${CMAKE_BINARY_DIR}/dependencies_64.txt)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256 files_to_extract)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256)
set (archive_path ${TEMP}/download/${archive_name})
set (status "ON")
@@ -170,12 +169,12 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
debug_message ("checking that unpacked directory exist: ${unpacked_path}")
if (NOT EXISTS ${unpacked_path})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256} ${files_to_extract})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256})
else(NOT EXISTS ${unpacked_path})
#path exists, so we would like to check what was unpacked version
set (version_file ${unpacked_path}/ie_dependency.info)
if (NOT EXISTS ${version_file})
if (NOT EXISTS ${version_file} AND NOT ${ENABLE_ALTERNATIVE_TEMP})
clean_message(FATAL_ERROR "error: Dependency doesn't contain version file. Please select actions: \n"
"if you are not sure about your FS dependency - remove it : \n"
"\trm -rf ${unpacked_path}\n"
@@ -201,12 +200,12 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
string(REPLACE ${TEMP} ${ALTERNATIVE_PATH} archive_path ${archive_path})
debug_message("dependency different: use local path for fetching updated version: ${alternative_path}")
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} ${result_path} ${folder} ${fattal} ${resultExt} FALSE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} ${result_path} ${folder} ${fattal} ${resultExt} FALSE ${sha256})
else()
debug_message("dependency updated: download it again")
file(REMOVE_RECURSE "${unpacked_path}")
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256} ${files_to_extract})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256})
endif()
endif ()
endif()
+6 -10
View File
@@ -1,8 +1,8 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
function (extract archive_path unpacked_path folder files_to_extract result)
function (extract archive_path unpacked_path folder result)
# Slurped from a generated extract-TARGET.cmake file.
get_filename_component(unpacked_dir ${unpacked_path} DIRECTORY)
@@ -24,24 +24,20 @@ function (extract archive_path unpacked_path folder files_to_extract result)
message("unpacked_dir= ${unpacked_dir}")
endif()
string(REGEX REPLACE ";" " " list_files_to_extract "${${files_to_extract}}")
message(STATUS "extracting... [tar xfz] ${list_files_to_extract}")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz ${archive_path} ${${files_to_extract}}
message(STATUS "extracting... [tar xfz]")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz ${archive_path}
WORKING_DIRECTORY ${unpacked_dir}
RESULT_VARIABLE rv
ERROR_VARIABLE err)
if (NOT (rv EQUAL 0))
if (NOT (err STREQUAL ""))
message(STATUS "error: extract of '${archive_path}' failed: ${err}")
#invalid archive
file(REMOVE_RECURSE "${unpacked_path}")
file(REMOVE_RECURSE "${archive_path}")
set(${result} 0 PARENT_SCOPE)
else()
if (NOT (err STREQUAL ""))
message(STATUS "${err}")
endif()
set(${result} 1 PARENT_SCOPE)
endif()
endfunction (extract)
endfunction (extract)
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+5 -3
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -50,13 +50,15 @@ if(NOT DEFINED ENABLE_CPPLINT_REPORT)
ie_dependent_option (ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF "ENABLE_CPPLINT" OFF)
endif()
ie_dependent_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ON "UNIX;NOT ANDROID" OFF)
ie_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ON)
ie_option (VERBOSE_BUILD "shows extra information about build" OFF)
ie_option (ENABLE_UNSAFE_LOCATIONS "skip check for MD5 for dependency" OFF)
ie_dependent_option (ENABLE_FUZZING "instrument build for fuzzing" OFF "OV_COMPILER_IS_CLANG; NOT WIN32" OFF)
ie_option (ENABLE_ALTERNATIVE_TEMP "in case of dependency conflict, to avoid modification in master, use local copy of dependency" ON)
ie_dependent_option (ENABLE_FUZZING "instrument build for fuzzing" OFF "CMAKE_CXX_COMPILER_ID MATCHES ^(Apple)?Clang$; NOT WIN32" OFF)
#
# Check features
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -3
View File
@@ -1,7 +1,6 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
# Usage: ie_option(<option_variable> "description" <initial value or boolean expression> [IF <condition>])
include (CMakeDependentOption)
@@ -29,7 +28,6 @@ macro (ie_option_enum variable description value)
list(APPEND IE_OPTIONS ${variable})
set(${variable} ${value} CACHE STRING "${description}")
set_property(CACHE ${variable} PROPERTY STRINGS ${IE_OPTION_ENUM_ALLOWED_VALUES})
endmacro()
function (print_enabled_features)
+1 -2
View File
@@ -1,6 +1,5 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(CPackComponent)
unset(IE_CPACK_COMPONENTS_ALL CACHE)
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+6 -24
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -27,10 +27,7 @@ endif()
# )
#
function(ie_add_plugin)
set(options
SKIP_INSTALL
ADD_CLANG_FORMAT
)
set(options SKIP_INSTALL)
set(oneValueArgs NAME DEVICE_NAME VERSION_DEFINES_FOR)
set(multiValueArgs SOURCES OBJECT_LIBRARIES CPPLINT_FILTERS)
cmake_parse_arguments(IE_PLUGIN "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
@@ -58,7 +55,7 @@ function(ie_add_plugin)
add_library(${IE_PLUGIN_NAME} MODULE ${input_files})
target_compile_definitions(${IE_PLUGIN_NAME} PRIVATE IMPLEMENT_INFERENCE_ENGINE_PLUGIN)
ie_add_vs_version_file(NAME ${IE_PLUGIN_NAME}
ie_add_vs_version_file(NAME ${TARGET_NAME}
FILEDESCRIPTION "Inference Engine ${IE_PLUGIN_DEVICE_NAME} device plugin library")
if(TARGET IE::inference_engine_plugin_api)
@@ -68,7 +65,7 @@ function(ie_add_plugin)
endif()
if(WIN32)
set_target_properties(${IE_PLUGIN_NAME} PROPERTIES COMPILE_PDB_NAME ${IE_PLUGIN_NAME})
set_target_properties(${IE_PLUGIN_NAME} PROPERTIES COMPILE_PDB_NAME ${TARGET_NAME})
endif()
set(custom_filter "")
@@ -76,27 +73,12 @@ function(ie_add_plugin)
string(CONCAT custom_filter "${custom_filter}" "," "${filter}")
endforeach()
if (IE_PLUGIN_ADD_CLANG_FORMAT)
add_clang_format_target(${IE_PLUGIN_NAME}_clang FOR_TARGETS ${IE_PLUGIN_NAME})
else()
add_cpplint_target(${IE_PLUGIN_NAME}_cpplint FOR_TARGETS ${IE_PLUGIN_NAME} CUSTOM_FILTERS ${custom_filter})
endif()
# check that plugin with such name is not registered
foreach(plugin_entry IN LISTS PLUGIN_FILES)
string(REPLACE ":" ";" plugin_entry "${plugin_entry}")
list(GET plugin_entry -1 library_name)
list(GET plugin_entry 0 plugin_name)
if(plugin_name STREQUAL "${IE_PLUGIN_DEVICE_NAME}" AND
NOT library_name STREQUAL ${IE_PLUGIN_NAME})
message(FATAL_ERROR "${IE_PLUGIN_NAME} and ${library_name} are both registered as ${plugin_name}")
endif()
endforeach()
add_cpplint_target(${IE_PLUGIN_NAME}_cpplint FOR_TARGETS ${IE_PLUGIN_NAME} CUSTOM_FILTERS ${custom_filter})
# append plugin to the list to register
list(APPEND PLUGIN_FILES "${IE_PLUGIN_DEVICE_NAME}:${IE_PLUGIN_NAME}")
list(REMOVE_DUPLICATES PLUGIN_FILES)
set(PLUGIN_FILES "${PLUGIN_FILES}" CACHE INTERNAL "" FORCE)
add_dependencies(ie_plugins ${IE_PLUGIN_NAME})
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+3 -9
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
# Target system specific flags
@@ -39,10 +39,10 @@ elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "amd64.*|x86_64.*|AMD64.*")
set(X86_64 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "i686.*|i386.*|x86.*|amd64.*|AMD64.*")
set(X86 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(arm64.*|aarch64.*|AARCH64.*)")
set(AARCH64 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(arm.*|ARM.*)")
set(ARM ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64.*|AARCH64.*)")
set(AARCH64 ON)
endif()
# in case of cross-compilation (or -m32) CMAKE_SYSTEM_PROCESSOR is equal to
@@ -55,9 +55,3 @@ endif()
if(UNIX AND NOT APPLE)
set(LINUX ON)
endif()
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
set(OV_COMPILER_IS_CLANG ON)
else()
set(OV_COMPILER_IS_CLANG OFF)
endif()
@@ -1,6 +1,19 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.
@@ -1,6 +1,18 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.
@@ -1,6 +1,18 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.
+1 -55
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -26,55 +26,6 @@ function (commitHash VAR)
set (${VAR} ${GIT_COMMIT_HASH} PARENT_SCOPE)
endfunction()
macro(ie_parse_ci_build_number)
if(CI_BUILD_NUMBER MATCHES "^([0-9]+)\.([0-9]+)\.([0-9]+)\-.*")
set(IE_VERSION_MAJOR ${CMAKE_MATCH_1})
set(IE_VERSION_MINOR ${CMAKE_MATCH_2})
set(IE_VERSION_PATCH ${CMAKE_MATCH_3})
set(has_ci_version ON)
else()
set(IE_VERSION_MAJOR 0)
set(IE_VERSION_MINOR 0)
set(IE_VERSION_PATCH 0)
endif()
if(NOT DEFINED repo_root)
message(FATAL_ERROR "repo_root is not defined")
endif()
if(DEFINED IEDevScripts_DIR AND DEFINED IE_MAIN_SOURCE_DIR AND NOT DEFINED custom_build)
set(ie_version_hpp "${IE_MAIN_SOURCE_DIR}/include/ie_version.hpp")
if(NOT EXISTS ${ie_version_hpp})
message(FATAL_ERROR "File ie_version.hpp with IE_VERSION definitions is not found")
endif()
file(STRINGS "${ie_version_hpp}" IE_VERSION_PARTS REGEX "#define IE_VERSION_[A-Z]+[ ]+" )
string(REGEX REPLACE ".+IE_VERSION_MAJOR[ ]+([0-9]+).*" "\\1"
IE_VERSION_MAJOR_HPP "${IE_VERSION_PARTS}")
string(REGEX REPLACE ".+IE_VERSION_MINOR[ ]+([0-9]+).*" "\\1"
IE_VERSION_MINOR_HPP "${IE_VERSION_PARTS}")
string(REGEX REPLACE ".+IE_VERSION_PATCH[ ]+([0-9]+).*" "\\1"
IE_VERSION_PATCH_HPP "${IE_VERSION_PARTS}")
foreach(var IE_VERSION_MAJOR IE_VERSION_MINOR IE_VERSION_PATCH)
if(DEFINED ${var} AND NOT ${var} EQUAL ${var}_HPP)
message(FATAL_ERROR "${var} parsed from CI_BUILD_NUMBER (${${var}}) \
and from ie_version.hpp (${${var}_HPP}) are different")
else()
# CI_BUILD_NUMBER is not defined well, take info from ie_verison.hpp as a baseline
set(${var} ${${var}_HPP})
endif()
endforeach()
elseif(has_ci_version)
message(WARNING "IE_MAIN_SOURCE_DIR is not defined. No way to compare versions")
else()
message(WARNING "No way to detect OpenVINO version. Supposing 0.0.0.0")
endif()
set(IE_VERSION "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}")
endmacro()
if (DEFINED ENV{CI_BUILD_NUMBER})
set(CI_BUILD_NUMBER $ENV{CI_BUILD_NUMBER})
else()
@@ -85,11 +36,6 @@ else()
set(CI_BUILD_NUMBER "${custom_build}")
endif()
# provides Inference Engine version
# 1. If CI_BUILD_NUMBER is defined, parses this information
# 2. Otherwise, parses ie_version.hpp
ie_parse_ci_build_number()
function (addVersionDefines FILE)
foreach (VAR ${ARGN})
if (DEFINED ${VAR} AND NOT "${${VAR}}" STREQUAL "")
@@ -1,15 +1,30 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(IE_VS_VER_FILEVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_PRODUCTVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_FILEVERSION_STR "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}.0")
macro(ie_parse_ci_build_number)
if(CI_BUILD_NUMBER MATCHES "^([0-9]+)\.([0-9]+)\.([0-9]+)\-.*")
set(IE_VERSION_MAJOR ${CMAKE_MATCH_1})
set(IE_VERSION_MINOR ${CMAKE_MATCH_2})
set(IE_VERSION_PATCH ${CMAKE_MATCH_3})
set(IE_VS_VER_HAS_VERSION 1)
else()
set(IE_VS_VER_HAS_VERSION 0)
endif()
endmacro()
ie_parse_ci_build_number()
if(IE_VS_VER_HAS_VERSION)
set(IE_VS_VER_FILEVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_PRODUCTVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_FILEVERSION_STR "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}.0")
endif()
set(IE_VS_VER_COMPANY_NAME_STR "Intel Corporation")
set(IE_VS_VER_PRODUCTVERSION_STR "${CI_BUILD_NUMBER}")
set(IE_VS_VER_PRODUCTNAME_STR "OpenVINO toolkit")
set(IE_VS_VER_COPYRIGHT_STR "Copyright (C) 2018-2021, Intel Corporation")
set(IE_VS_VER_COPYRIGHT_STR "Copyright (C) 2018-2020, Intel Corporation")
set(IE_VS_VER_COMMENTS_STR "https://docs.openvinotoolkit.org/")
#
@@ -36,11 +51,6 @@ function(ie_add_vs_version_file)
message(FATAL_ERROR "${VS_VER_NAME} must define a target")
endif()
get_target_property(target_type ${VS_VER_NAME} TYPE)
if(NOT target_type MATCHES "^(SHARED|MODULE)_LIBRARY$")
message(FATAL_ERROR "ie_add_vs_version_file can work only with dynamic libraries")
endif()
macro(_vs_ver_update_variable name)
if(VS_VER_NAME AND DEFINED IE_${VS_VER_NAME}_VS_VER_${name})
set(IE_VS_VER_${name} "${IE_${VS_VER_NAME}_VS_VER_${name}}")
@@ -1,8 +1,10 @@
#include <winver.h>
VS_VERSION_INFO VERSIONINFO
#if @IE_VS_VER_HAS_VERSION@
FILEVERSION @IE_VS_VER_FILEVERSION_QUAD@
PRODUCTVERSION @IE_VS_VER_PRODUCTVERSION_QUAD@
#endif
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
#ifdef _DEBUG
FILEFLAGS 1
@@ -19,7 +21,9 @@ BEGIN
BEGIN
VALUE "CompanyName", "@IE_VS_VER_COMPANY_NAME_STR@\0"
VALUE "FileDescription", "@IE_VS_VER_FILEDESCRIPTION_STR@\0"
#if @IE_VS_VER_HAS_VERSION@
VALUE "FileVersion", "@IE_VS_VER_FILEVERSION_STR@\0"
#endif
VALUE "InternalName", "@IE_VS_VER_INTERNALNAME_STR@\0"
VALUE "LegalCopyright", "@IE_VS_VER_COPYRIGHT_STR@\0"
VALUE "OriginalFilename", "@IE_VS_VER_ORIGINALFILENAME_STR@\0"
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+9 -26
View File
@@ -1,24 +1,22 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
ie_dependent_option (ENABLE_MKL_DNN "MKL-DNN plugin for inference engine" ON "X86_64" OFF)
if(X86_64)
set(ENABLE_MKL_DNN_DEFAULT ON)
else()
set(ENABLE_MKL_DNN_DEFAULT OFF)
endif()
ie_option (ENABLE_MKL_DNN "MKL-DNN plugin for inference engine" ${ENABLE_MKL_DNN_DEFAULT})
ie_option (ENABLE_TESTS "unit, behavior and functional tests" OFF)
ie_option (ENABLE_STRICT_DEPENDENCIES "Skip configuring \"convinient\" dependencies for efficient parallel builds" ON)
ie_dependent_option (ENABLE_CLDNN "clDnn based plugin for inference engine" ON "X86_64;NOT APPLE;NOT MINGW;NOT WINDOWS_STORE;NOT WINDOWS_PHONE" OFF)
ie_option (ENABLE_PROFILING_ITT "Build with ITT tracing. Optionally configure pre-built ittnotify library though INTEL_VTUNE_DIR variable." OFF)
ie_option_enum(ENABLE_PROFILING_FILTER "Enable or disable ITT counter groups.\
Supported values:\
ALL - enable all ITT counters (default value)\
FIRST_INFERENCE - enable only first inference time counters" ALL
ALLOWED_VALUES ALL FIRST_INFERENCE)
ie_option (ENABLE_PROFILING_FIRST_INFERENCE "Build with ITT tracing of first inference time." ON)
ie_option (ENABLE_DOCS "Build docs using Doxygen" OFF)
ie_option(ENABLE_TEMPLATE_PLUGIN "Register template plugin into plugins.xml" OFF)
@@ -29,21 +27,6 @@ Usage: -DSELECTIVE_BUILD=ON -DSELECTIVE_BUILD_STAT=/path/*.csv" OFF
ie_option(ENABLE_ERROR_HIGHLIGHT "Highlight errors and warnings during compile time" OFF)
# Try to find python3
find_package(PythonLibs 3 QUIET)
ie_dependent_option (ENABLE_PYTHON "enables ie python bridge build" OFF "PYTHONLIBS_FOUND" OFF)
find_package(PythonInterp 3 QUIET)
ie_dependent_option (ENABLE_DOCS "Build docs using Doxygen" OFF "PYTHONINTERP_FOUND" OFF)
#
# enable or disable output from NGRAPH_DEBUG statements
#
if(NGRAPH_DEBUG_ENABLE)
add_definitions(-DNGRAPH_DEBUG_ENABLE)
endif()
#
# Process options
#
+4 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -19,3 +19,6 @@ endmacro()
# need libusb 32-bits version
_set_if_not_defined(ENABLE_VPU OFF)
# fix conversion from uint64_t / int64_t to size_t
_set_if_not_defined(NGRAPH_ONNX_IMPORT_ENABLE OFF)
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
+1 -1
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
-25
View File
@@ -1,25 +0,0 @@
BasedOnStyle: Google
IndentWidth: 4
UseTab: Never
Language: Cpp
Standard: Cpp11
AccessModifierOffset: -4
AlignConsecutiveMacros: true
AllowAllArgumentsOnNextLine: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: Never
AllowShortLambdasOnASingleLine: Empty
AllowShortLoopsOnASingleLine: false
AlwaysBreakBeforeMultilineStrings: false
ColumnLimit: 160
# Specialize this comment pragma in order to avoid changes in SEA copyrights
CommentPragmas: '^#'
DerivePointerAlignment: false
FixNamespaceComments: true
IndentCaseLabels: false
IndentPPDirectives: BeforeHash
SpaceBeforeCpp11BracedList: true
SpaceBeforeCtorInitializerColon: false
+276 -149
View File
@@ -1,4 +1,4 @@
# Copyright (C) 2018-2023 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -44,212 +44,339 @@ if(NOT ENABLE_DOCKER)
endforeach()
endif()
set(LINKCHECKER_PY "" CACHE FILEPATH "Path to linkchecker.py for documentation check dir.")
set(ENABLE_OPENVINO_NOTEBOOKS OFF CACHE BOOL "Build with openvino notebooks")
set(OMZ_DOCS_DIR "" CACHE PATH "Path to open_model_zoo documentation dir.")
set(WORKBENCH_DOCS_DIR "" CACHE PATH "Path to workbench documentation dir.")
set(POT_DOCS_DIR "" CACHE PATH "Path to post-training-compression-tool documentation dir.")
set(OVMS_DOCS_DIR "" CACHE PATH "Path to model server documentation dir.")
set(GST_DOCS_DIR "" CACHE PATH "Path to gst-video-analytics documentation dir.")
set(GRAPH_CSV_DIR "" CACHE PATH "Path to the folder containing csv data for rendering graphs.")
set(LINKCHECKER_PY "" CACHE FILEPATH "Path to linkchecker.py for documentation check")
set(OMZ_DOCS_DIR "" CACHE PATH "Path to open_model_zoo documentation")
set(WORKBENCH_DOCS_DIR "" CACHE PATH "Path to workbench documentation")
set(POT_DOCS_DIR "" CACHE PATH "Path to post-training-compression-tool documentation")
set(GST_DOCS_DIR "" CACHE PATH "Path to gst-video-analytics documentation")
function(build_docs)
find_package(Doxygen REQUIRED dot)
find_package(PythonInterp 3 REQUIRED)
find_package(LATEX REQUIRED)
find_package(Python3 COMPONENTS Interpreter)
find_package(LATEX)
find_program(DOXYREST_EXECUTABLE NAMES doxyrest)
if (NOT DOXYREST_EXECUTABLE)
message(FATAL_ERROR "No doxyrest found. Documentation output is not available")
if(NOT DOXYGEN_FOUND)
message(FATAL_ERROR "Doxygen is required to build the documentation")
endif()
if(NOT Python3_FOUND)
message(FATAL_ERROR "Python3 is required to build the documentation")
endif()
execute_process(
COMMAND ${Python3_EXECUTABLE} -m pip show lxml
RESULT_VARIABLE PIP_EXIT_CODE
OUTPUT_QUIET
)
if (NOT ${PIP_EXIT_CODE} EQUAL 0)
message(FATAL_ERROR "lxml package is not installed. Please use \"pip install lxml\".")
endif()
if(NOT LATEX_FOUND)
message(FATAL_ERROR "LATEX is required to build the documentation")
endif()
set(DOCS_BUILD_DIR "${CMAKE_CURRENT_BINARY_DIR}")
set(DOCS_SOURCE_DIR "${OpenVINO_MAIN_SOURCE_DIR}/docs")
set(SCRIPTS_DIR "${DOCS_SOURCE_DIR}/scripts")
# API INPUT
set(NGRAPH_DIR "${OpenVINO_MAIN_SOURCE_DIR}/ngraph")
# markdown docs
set(MARKDOWN_INPUT "${DOCS_BUILD_DIR}")
# IE C++ API
set(DOXYGEN_DIR "${OpenVINO_MAIN_SOURCE_DIR}/docs/doxygen")
set(IE_SOURCE_DIR "${OpenVINO_MAIN_SOURCE_DIR}/inference-engine")
# IE C API
set(IE_C_API "${IE_SOURCE_DIR}/ie_bridges/c/include")
set(PYTHON_API_IN "${IE_SOURCE_DIR}/ie_bridges/python/src/openvino/inference_engine/ie_api.pyx")
set(PYTHON_API_OUT "${DOCS_BUILD_DIR}/python_api/ie_api.pyx")
set(C_API "${IE_SOURCE_DIR}/ie_bridges/c/include")
set(PLUGIN_API_DIR "${DOCS_BUILD_DIR}/IE_PLUGIN_DG")
set(NGRAPH_DIR "${OpenVINO_MAIN_SOURCE_DIR}/ngraph")
set(NGRAPH_PY_DIR "${NGRAPH_DIR}/python/src/ngraph/")
set(NGRAPH_CPP_DIR "${NGRAPH_DIR}/core/include/" "${NGRAPH_DIR}/frontend/onnx_import/include")
# Preprocessing scripts
set(DOXY_MD_FILTER "${SCRIPTS_DIR}/doxy_md_filter.py")
set(PYNGRAPH_REF_SCRIPT "${SCRIPTS_DIR}/pyngraph_ref.py")
set(DOXY_LOG_SCRIPT "${SCRIPTS_DIR}/log.py")
set(PYX_FILTER "${SCRIPTS_DIR}/pyx_filter.py")
set(PREPARE_XML_SCRIPT "${SCRIPTS_DIR}/prepare_xml.py")
set(REMOVE_XML_SCRIPT "${SCRIPTS_DIR}/remove_xml.py")
set(COPY_IMAGES_SCRIPT "${SCRIPTS_DIR}/copy_images.py")
set(DOC_TEST_DIR "${SCRIPTS_DIR}/tests")
set(DOXYGEN_MAPPING_SCRIPT "${SCRIPTS_DIR}/create_mapping.py")
set(DOXYGEN_MAPPING_FILE "${DOCS_BUILD_DIR}/mapping.json")
set(DOXY_MD_FILTER "${DOXYGEN_DIR}/doxy_md_filter.py")
set(DOXY_LAYOUT_SCRIPT "${DOXYGEN_DIR}/build_main_layout.py")
set(DOXY_LOG_SCRIPT "${DOXYGEN_DIR}/log.py")
set(PYX_FILTER "${DOXYGEN_DIR}/pyx_filter.py")
# assets dir
set(ASSETS_DIR "${DOXYGEN_DIR}/assets")
# header and footer
set(HEADER_SOURCE "${DOXYGEN_DIR}/header.html.in")
set(FOOTER_SOURCE "${DOXYGEN_DIR}/footer.html.in")
set(HEADER_BUILD "${DOCS_BUILD_DIR}/header.html")
set(FOOTER_BUILD "${DOCS_BUILD_DIR}/footer.html")
configure_file(${HEADER_SOURCE} ${HEADER_BUILD} @ONLY)
configure_file(${FOOTER_SOURCE} ${FOOTER_BUILD} @ONLY)
file(GLOB_RECURSE doc_source_files
LIST_DIRECTORIES true RELATIVE ${OpenVINO_MAIN_SOURCE_DIR}
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.md"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.png"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.gif"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.jpg"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.svg"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.md"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.png"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.gif"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.jpg"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.svg")
configure_file(${PYTHON_API_IN} ${PYTHON_API_OUT} @ONLY)
set(NGRAPH_CPP_CONFIG_SOURCE "${DOXYGEN_DIR}/ngraph_cpp_api.config")
set(NGRAPH_PY_CONFIG_SOURCE "${DOXYGEN_DIR}/ngraph_py_api.config")
set(IE_CONFIG_SOURCE "${DOXYGEN_DIR}/ie_docs.config")
set(C_CONFIG_SOURCE "${DOXYGEN_DIR}/ie_c_api.config")
set(PY_CONFIG_SOURCE "${DOXYGEN_DIR}/ie_py_api.config")
set(PLUGIN_CONFIG_SOURCE "${DOXYGEN_DIR}/ie_plugin_api.config")
set(NGRAPH_CPP_CONFIG_BUILD "${DOCS_BUILD_DIR}/ngraph_cpp_api.config")
set(NGRAPH_PY_CONFIG_BUILD "${DOCS_BUILD_DIR}/ngraph_py_api.config")
set(IE_CONFIG_BUILD "${DOCS_BUILD_DIR}/ie_docs.config")
set(C_CONFIG_BUILD "${DOCS_BUILD_DIR}/ie_c_api.config")
set(PY_CONFIG_BUILD "${DOCS_BUILD_DIR}/ie_py_api.config")
set(PLUGIN_CONFIG_BUILD "${DOCS_BUILD_DIR}/ie_plugin_api.config")
set(NGRAPH_CPP_LAYOUT_SOURCE "${DOXYGEN_DIR}/ngraph_cpp_api.xml")
set(NGRAPH_PY_LAYOUT_SOURCE "${DOXYGEN_DIR}/ngraph_py_api.xml")
set(IE_LAYOUT_SOURCE "${DOXYGEN_DIR}/ie_docs.xml")
set(OPENVINO_LAYOUT_SOURCE "${DOXYGEN_DIR}/openvino_docs.xml")
set(C_LAYOUT_SOURCE "${DOXYGEN_DIR}/ie_c_api.xml")
set(PY_LAYOUT_SOURCE "${DOXYGEN_DIR}/ie_py_api.xml")
set(PLUGIN_LAYOUT_SOURCE "${DOXYGEN_DIR}/ie_plugin_api.xml")
set(NGRAPH_CPP_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ngraph_cpp_api.xml")
set(NGRAPH_PY_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ngraph_py_api.xml")
set(IE_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ie_docs.xml")
set(OPENVINO_LAYOUT_BUILD "${DOCS_BUILD_DIR}/openvino_docs.xml")
set(C_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ie_c_api.xml")
set(PY_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ie_py_api.xml")
set(PLUGIN_LAYOUT_BUILD "${DOCS_BUILD_DIR}/ie_plugin_api.xml")
# out dirs
set(XML_OUTPUT "${DOCS_BUILD_DIR}/xml")
set(RST_OUTPUT "${DOCS_BUILD_DIR}/rst")
set(SPHINX_OUTPUT "${DOCS_BUILD_DIR}/_build")
set(OUTPUT_DIRECTORY "${DOCS_BUILD_DIR}/html")
set(IE_OUTPUT "${OUTPUT_DIRECTORY}")
set(C_OUTPUT "${OUTPUT_DIRECTORY}/ie_c_api")
set(PY_OUTPUT "${OUTPUT_DIRECTORY}/ie_python_api")
set(PLUGIN_OUTPUT "${OUTPUT_DIRECTORY}/ie_plugin_api")
set(NGRAPH_CPP_OUTPUT "${OUTPUT_DIRECTORY}/ngraph_cpp_api")
set(NGRAPH_PY_OUTPUT "${OUTPUT_DIRECTORY}/ngraph_python_api")
# Sphinx folders, doxyrest templates and config
set(SPHINX_CONF_IN "${DOCS_SOURCE_DIR}/conf.py")
set(SPHINX_CONF_OUT "${RST_OUTPUT}/conf.py")
set(SPHINX_STATIC_IN "${DOCS_SOURCE_DIR}/_static")
set(SPHINX_STATIC_OUT "${RST_OUTPUT}/_static")
set(SPHINX_INDEX_IN "${DOCS_SOURCE_DIR}/index.rst")
set(SPHINX_INDEX_OUT "${RST_OUTPUT}/index.rst")
set(API_DOCS_IN "${DOCS_SOURCE_DIR}/api")
set(API_DOCS_OUT "${RST_OUTPUT}/api")
set(DOXYREST_IN "${DOCS_SOURCE_DIR}/doxyrest")
set(DOXYREST_OUT "${DOCS_BUILD_DIR}/doxyrest")
set(DOXYREST_SPHINX_IN "${DOCS_SOURCE_DIR}/doxyrest-sphinx")
set(DOXYREST_SPHINX_OUT "${RST_OUTPUT}/doxyrest-sphinx")
set(DOXYREST_CONFIG_IN "${DOCS_SOURCE_DIR}/doxyrest-config.lua")
set(DOXYREST_CONFIG_OUT "${DOCS_BUILD_DIR}/doxyrest-config.lua")
configure_file(${DOXYREST_CONFIG_IN} ${DOXYREST_CONFIG_OUT} @ONLY)
configure_file(${SPHINX_CONF_IN} ${SPHINX_CONF_OUT} @ONLY)
# Tables of contents
configure_file(${NGRAPH_CPP_LAYOUT_SOURCE} ${NGRAPH_CPP_LAYOUT_BUILD} @ONLY)
configure_file(${NGRAPH_PY_LAYOUT_SOURCE} ${NGRAPH_PY_LAYOUT_BUILD} @ONLY)
configure_file(${IE_LAYOUT_SOURCE} ${IE_LAYOUT_BUILD} @ONLY)
configure_file(${OPENVINO_LAYOUT_SOURCE} ${OPENVINO_LAYOUT_BUILD} @ONLY)
configure_file(${C_LAYOUT_SOURCE} ${C_LAYOUT_BUILD} @ONLY)
configure_file(${PY_LAYOUT_SOURCE} ${PY_LAYOUT_BUILD} @ONLY)
configure_file(${PLUGIN_LAYOUT_SOURCE} ${PLUGIN_LAYOUT_BUILD} @ONLY)
# Doxygen config
set(DOXYFILE_SOURCE "${DOCS_SOURCE_DIR}/Doxyfile.config")
set(DOXYFILE_BUILD "${DOCS_BUILD_DIR}/Doxyfile.config")
configure_file(${DOXYFILE_SOURCE} ${DOXYFILE_BUILD} @ONLY)
# Doxygen config files
configure_file(${NGRAPH_CPP_CONFIG_SOURCE} ${NGRAPH_CPP_CONFIG_BUILD} @ONLY)
configure_file(${NGRAPH_PY_CONFIG_SOURCE} ${NGRAPH_PY_CONFIG_BUILD} @ONLY)
configure_file(${IE_CONFIG_SOURCE} ${IE_CONFIG_BUILD} @ONLY)
configure_file(${C_CONFIG_SOURCE} ${C_CONFIG_BUILD} @ONLY)
configure_file(${PY_CONFIG_SOURCE} ${PY_CONFIG_BUILD} @ONLY)
configure_file(${PLUGIN_CONFIG_SOURCE} ${PLUGIN_CONFIG_BUILD} @ONLY)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${OpenVINO_MAIN_SOURCE_DIR}
--output_dir=${DOCS_BUILD_DIR}/openvino
--exclude_dir=${DOCS_BUILD_DIR})
# Preprocessing scripts
set(DOXY_MD_FILTER "${DOXYGEN_DIR}/doxy_md_filter.py")
set(PYX_FILTER "${DOXYGEN_DIR}/pyx_filter.py")
# include additional repositories
# nGraph C++ API
# build with openvino notebooks
if(ENABLE_OPENVINO_NOTEBOOKS)
set(NBDOC_SCRIPT "${DOCS_SOURCE_DIR}/nbdoc/nbdoc.py")
list(APPEND commands
COMMAND ${PYTHON_EXECUTABLE} "${NBDOC_SCRIPT}" "${RST_OUTPUT}/notebooks"
)
endif()
add_custom_target(ngraph_cpp_api
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${NGRAPH_CPP_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${NGRAPH_CPP_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
VERBATIM)
if(GRAPH_CSV_DIR)
set(GRAPH_CSV_DIR_OUT "${RST_OUTPUT}/csv")
list(APPEND commands
COMMAND ${CMAKE_COMMAND} -E copy_directory "${GRAPH_CSV_DIR}" "${GRAPH_CSV_DIR_OUT}"
)
endif()
# nGraph Python API
list(APPEND commands
COMMAND ${CMAKE_COMMAND} -E copy ${API_DOCS_IN}/api_reference.rst ${API_DOCS_OUT}/api_reference.rst
)
add_custom_target(ngraph_py_api
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${NGRAPH_PY_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${NGRAPH_PY_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
VERBATIM)
if(ENABLE_PYTHON)
list(APPEND commands
COMMAND ${CMAKE_COMMAND} -E copy_directory ${API_DOCS_IN}/ie_python_api ${API_DOCS_OUT}/ie_python_api
)
list(APPEND commands
COMMAND ${CMAKE_COMMAND} -E copy_directory ${API_DOCS_IN}/ngraph_python_api ${API_DOCS_OUT}/ngraph_python_api
)
endif()
# C API
add_custom_target(c_api
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${C_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${C_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
COMMENT "Generating C API Reference"
VERBATIM)
# Python API
add_custom_target(py_api
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${PY_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${PY_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
COMMENT "Generating Python API Reference"
VERBATIM)
add_custom_command(TARGET py_api
PRE_BUILD
COMMAND ${Python3_EXECUTABLE} ${PYX_FILTER} ${PYTHON_API_OUT}
COMMENT "Pre-process Python API")
# Preprocess docs
add_custom_target(preprocess_docs
COMMENT "Pre-process docs"
VERBATIM)
# ovino doc files
file(GLOB_RECURSE ovino_doc_files
LIST_DIRECTORIES true RELATIVE ${OpenVINO_MAIN_SOURCE_DIR}
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.md"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.png"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.gif"
"${OpenVINO_MAIN_SOURCE_DIR}/docs/*.jpg"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.md"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.png"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.gif"
"${OpenVINO_MAIN_SOURCE_DIR}/inference-engine/*.jpg")
foreach(source_file ${ovino_doc_files})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E copy
"${OpenVINO_MAIN_SOURCE_DIR}/${source_file}" "${DOCS_BUILD_DIR}/openvino/${source_file}")
endforeach()
# omz doc files
if(EXISTS "${OMZ_DOCS_DIR}")
get_filename_component(OMZ_DOCS_DIR "${OMZ_DOCS_DIR}" ABSOLUTE)
list(APPEND commands
COMMAND ${PYTHON_EXECUTABLE} ${OMZ_DOCS_DIR}/ci/prepare-documentation.py ${CMAKE_BINARY_DIR}/open_model_zoo)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${CMAKE_BINARY_DIR}/open_model_zoo
--output_dir=${DOCS_BUILD_DIR}/open_model_zoo)
file(GLOB_RECURSE omz_doc_files
LIST_DIRECTORIES true RELATIVE ${OMZ_DOCS_DIR}
"${OMZ_DOCS_DIR}/*.md"
"${OMZ_DOCS_DIR}/*.png"
"${OMZ_DOCS_DIR}/*.gif"
"${OMZ_DOCS_DIR}/*.jpg")
foreach(source_file ${omz_doc_files})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E copy
"${OMZ_DOCS_DIR}/${source_file}" "${DOCS_BUILD_DIR}/omz/${source_file}")
endforeach()
configure_file("${OMZ_DOCS_DIR}/omz_docs.xml" "${DOCS_BUILD_DIR}/omz_docs.xml" @ONLY)
endif()
# workbench doc files
if(EXISTS "${WORKBENCH_DOCS_DIR}")
get_filename_component(WORKBENCH_DOCS_DIR "${WORKBENCH_DOCS_DIR}" ABSOLUTE)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${WORKBENCH_DOCS_DIR}
--output_dir=${DOCS_BUILD_DIR}/workbench)
file(GLOB_RECURSE workbench_doc_files
LIST_DIRECTORIES true RELATIVE ${WORKBENCH_DOCS_DIR}
"${WORKBENCH_DOCS_DIR}/*.md"
"${WORKBENCH_DOCS_DIR}/*.png"
"${WORKBENCH_DOCS_DIR}/*.gif"
"${WORKBENCH_DOCS_DIR}/*.jpg")
foreach(source_file ${workbench_doc_files})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E copy
"${WORKBENCH_DOCS_DIR}/${source_file}" "${DOCS_BUILD_DIR}/workbench/${source_file}")
endforeach()
configure_file("${WORKBENCH_DOCS_DIR}/docs/Workbench_DG/workbench_docs.xml" "${DOCS_BUILD_DIR}/workbench_docs.xml" @ONLY)
endif()
# pot doc files
if(EXISTS "${POT_DOCS_DIR}")
get_filename_component(POT_DOCS_DIR "${POT_DOCS_DIR}" ABSOLUTE)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${POT_DOCS_DIR}
--output_dir=${DOCS_BUILD_DIR}/pot)
endif()
file(GLOB_RECURSE pot_doc_files
LIST_DIRECTORIES true RELATIVE ${POT_DOCS_DIR}
"${POT_DOCS_DIR}/*.md"
"${POT_DOCS_DIR}/*.png"
"${POT_DOCS_DIR}/*.gif"
"${POT_DOCS_DIR}/*.jpg")
# ovms doc files
if(EXISTS "${OVMS_DOCS_DIR}")
get_filename_component(OVMS_DOCS_DIR "${OVMS_DOCS_DIR}" ABSOLUTE)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${OVMS_DOCS_DIR}
--output_dir=${DOCS_BUILD_DIR}/ovms)
foreach(source_file ${pot_doc_files})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E copy
"${POT_DOCS_DIR}/${source_file}" "${DOCS_BUILD_DIR}/pot/${source_file}")
endforeach()
configure_file("${POT_DOCS_DIR}/docs/pot_docs.xml" "${DOCS_BUILD_DIR}/pot_docs.xml" @ONLY)
endif()
# gst doc files
if(EXISTS "${GST_DOCS_DIR}")
get_filename_component(GST_DOCS_DIR "${GST_DOCS_DIR}" ABSOLUTE)
list(APPEND commands COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER}
--input_dir=${GST_DOCS_DIR}
--output_dir=${DOCS_BUILD_DIR}/gst)
file(GLOB_RECURSE gst_doc_files
LIST_DIRECTORIES true RELATIVE ${GST_DOCS_DIR}
"${GST_DOCS_DIR}/*.md"
"${GST_DOCS_DIR}/*.png"
"${GST_DOCS_DIR}/*.gif"
"${GST_DOCS_DIR}/*.jpg")
foreach(source_file ${gst_doc_files})
list(APPEND commands COMMAND ${CMAKE_COMMAND} -E copy
"${GST_DOCS_DIR}/${source_file}" "${DOCS_BUILD_DIR}/gst/${source_file}")
endforeach()
endif()
add_custom_target(preprocess_docs
COMMENT "Preprocess documentation"
VERBATIM)
# Preprocess docs
add_custom_command(TARGET preprocess_docs
POST_BUILD
PRE_BUILD
${commands}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
COMMENT "Preprocess documentation"
VERBATIM)
COMMAND ${Python3_EXECUTABLE} ${DOXY_LAYOUT_SCRIPT} --openvino ${OPENVINO_LAYOUT_BUILD}
COMMAND ${Python3_EXECUTABLE} ${DOXY_MD_FILTER} ${DOCS_BUILD_DIR}
COMMENT "Pre-process markdown and image links")
add_custom_target(doxygen_xml
DEPENDS preprocess_docs
COMMAND ${PYTHON_EXECUTABLE} ${REMOVE_XML_SCRIPT} ${XML_OUTPUT}
COMMAND ${DOXYGEN_EXECUTABLE} ${DOXYFILE_BUILD}
# IE dev guide and C++ API
add_custom_target(ie_docs
DEPENDS ngraph_cpp_api preprocess_docs
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${IE_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${IE_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
COMMENT "Generate doxygen XML output"
VERBATIM)
# Post-process docs
add_custom_command(TARGET doxygen_xml
POST_BUILD
COMMAND ${PYTHON_EXECUTABLE} ${PREPARE_XML_SCRIPT} ${XML_OUTPUT}
COMMAND ${CMAKE_COMMAND} -E copy_directory ${DOXYREST_IN} ${DOXYREST_OUT}
COMMAND ${DOXYREST_EXECUTABLE} -c ${DOXYREST_CONFIG_OUT}
COMMAND ${PYTHON_EXECUTABLE} ${COPY_IMAGES_SCRIPT} ${XML_OUTPUT} ${RST_OUTPUT}
COMMAND ${PYTHON_EXECUTABLE} ${DOXYGEN_MAPPING_SCRIPT} ${XML_OUTPUT} ${DOCS_BUILD_DIR} ${OpenVINO_MAIN_SOURCE_DIR}/../
COMMAND ${CMAKE_COMMAND} -E copy ${SPHINX_INDEX_IN} ${SPHINX_INDEX_OUT}
COMMAND ${CMAKE_COMMAND} -E copy_directory ${DOXYREST_IN} ${DOXYREST_OUT}
COMMAND ${CMAKE_COMMAND} -E copy_directory ${DOXYREST_SPHINX_IN} ${DOXYREST_SPHINX_OUT}
COMMAND ${CMAKE_COMMAND} -E copy_directory ${SPHINX_STATIC_IN} ${SPHINX_STATIC_OUT}
COMMENT "Prepare xml"
VERBATIM)
# Plugin API
add_custom_target(sphinx_docs
DEPENDS doxygen_xml
COMMAND sphinx-build -b html ${RST_OUTPUT} ${SPHINX_OUTPUT}
WORKING_DIRECTORY ${RST_OUTPUT}
add_custom_target(plugin_api
DEPENDS ngraph_cpp_api ie_docs
COMMAND ${CMAKE_COMMAND} -E copy_directory ${ASSETS_DIR} ${PLUGIN_OUTPUT}/assets
COMMAND ${DOXYGEN_EXECUTABLE} ${PLUGIN_CONFIG_BUILD}
WORKING_DIRECTORY ${DOCS_BUILD_DIR}
COMMENT "Generating Plugin API Reference"
VERBATIM)
set_target_properties(doxygen_xml sphinx_docs
# Umbrella OpenVINO target
add_custom_target(openvino_docs
DEPENDS ngraph_cpp_api ngraph_py_api c_api py_api ie_docs plugin_api
COMMENT "Generating OpenVINO documentation"
VERBATIM)
set_target_properties(openvino_docs ie_docs c_api py_api preprocess_docs plugin_api
ngraph_py_api ngraph_cpp_api
PROPERTIES FOLDER docs)
add_custom_command(TARGET openvino_docs
POST_BUILD
COMMAND ${Python3_EXECUTABLE} ${DOXY_LOG_SCRIPT} --log "${DOCS_BUILD_DIR}/ie_docs.log"
--include_omz $<BOOL:${OMZ_DOCS_DIR}>
--include_wb $<BOOL:${WORKBENCH_DOCS_DIR}>
--include_pot $<BOOL:${POT_DOCS_DIR}>
--include_gst $<BOOL:${GST_DOCS_DIR}>
COMMENT "Parse doxygen log to find errors."
VERBATIM)
# added linkcheker
if(EXISTS "${LINKCHECKER_PY}")
add_custom_target(docs_check
COMMAND ${Python3_EXECUTABLE} "${LINKCHECKER_PY}" -v "${DOCS_BUILD_DIR}/html/"
COMMENT "Check links in generated documentation"
WORKING_DIRECTORY "${DOCS_BUILD_DIR}"
VERBATIM)
set_target_properties(docs_check PROPERTIES FOLDER docs)
endif()
find_program(browser NAMES xdg-open)
if(browser)
add_custom_target(ie_docs_open
COMMAND ${browser} "${SPHINX_OUTPUT}/index.html"
DEPENDS sphinx_docs
COMMAND ${browser} "${OpenVINO_MAIN_SOURCE_DIR}/docs/html/index.html"
DEPENDS ie_docs
COMMENT "Open OpenVINO documentation"
VERBATIM)
set_target_properties(ie_docs_open PROPERTIES FOLDER docs)
+67 -66
View File
@@ -12,7 +12,7 @@ Representation (IR) for this model.
This guide illustrates the workflow for running inference on topologies featuring custom operations, allowing you to
plug in your own implementation for existing or completely new operation.
> **NOTE**: *Layer* — The legacy term for an *operation* which came from Caffe\* framework. Currently it is not used.
> **NOTE:** *Layer* — The legacy term for an *operation* which came from Caffe\* framework. Currently it is not used.
> Refer to the [Deep Learning Network Intermediate Representation and Operation Sets in OpenVINO™](../MO_DG/IR_and_opsets.md)
> for more information on the topic.
@@ -44,52 +44,72 @@ plugins to support inference of this operation using a particular target hardwar
To see the operations that are supported by each device plugin for the Inference Engine, refer to the
[Supported Devices](../IE_DG/supported_plugins/Supported_Devices.md).
> **NOTE**: If a device doesn't support a particular operation, an alternative to creating a new operation is to target
> **NOTE:** If a device doesn't support a particular operation, an alternative to creating a new operation is to target
> an additional device using the HETERO plugin. The [Heterogeneous Plugin](../IE_DG/supported_plugins/HETERO.md) may be
> used to run an inference model on multiple devices allowing the unsupported operations on one device to "fallback" to
> run on another device (e.g., CPU) that does support those operations.
### Custom Operation Support for the Model Optimizer
Model Optimizer model conversion pipeline is described in detail in "Model Conversion Pipeline" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). It is best to read that article first for a better understanding of the following material.
Model Optimizer model conversion pipeline is described in details in "Model Conversion Pipeline" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
It is recommended to read that article first for a better understanding of the following material.
Model Optimizer provides an extensions mechanism to support new operations and implement custom model transformations to generate optimized IR. This mechanism is described in the "Model Optimizer Extensions" section of
Model Optimizer provides extensions mechanism to support new operations and implement custom model transformations to
generate optimized IR. This mechanism is described in the "Model Optimizer Extensions" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
Two types of the Model Optimizer extensions should be implemented to support custom operations, at a minimum:
1. Operation class for a new operation. This class stores information about the operation, its attributes, shape inference function, attributes to be saved to an IR and some others internally used attributes. Refer to the "Model Optimizer Operation" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.
Two types of the Model Optimizer extensions should be implemented to support custom operation at minimum:
1. Operation class for a new operation. This class stores information about the operation, its attributes, shape
inference function, attributes to be saved to an IR and some others internally used attributes. Refer to the
"Model Optimizer Operation" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement it.
2. Operation attributes extractor. The extractor is responsible for parsing framework-specific representation of the
operation and uses corresponding operation class to update graph node attributes with necessary attributes of the
operation. Refer to the "Operation Extractor" section of
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.
operation. Refer to the "Operation Extractor" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement it.
> **NOTE**: In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
> **NOTE:** In some cases you may need to implement some transformation to support the operation. This topic is covered
> in the "Graph Transformation Extensions" section on the
> [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
## Custom Operations Extensions for the Inference Engine
Inference Engine provides extensions mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
Inference Engine provides extensions mechanism to support new operations. This mechanism is described in the
[Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
Each device plugin includes a library of optimized implementations to execute known operations which must be extended to execute a custom operation. The custom operation extension is implemented according to the target device:
Each device plugin includes a library of optimized implementations to execute known operations which must be extended to
execute a custom operation. The custom operation extension is implemented according to the target device:
- Custom Operation CPU Extension
- A compiled shared library (`.so` or `.dll`) needed by the CPU Plugin for executing the custom operation
- A compiled shared library (`.so`, `.dylib` or `.dll`) needed by the CPU Plugin for executing the custom operation
on a CPU. Refer to the [How to Implement Custom CPU Operations](../IE_DG/Extensibility_DG/CPU_Kernel.md) for more
details.
- Custom Operation GPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with an operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the [How to Implement Custom GPU Operations](../IE_DG/Extensibility_DG/GPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with a
operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the
[How to Implement Custom GPU Operations](../IE_DG/Extensibility_DG/GPU_Kernel.md) for more details.
- Custom Operation VPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with an operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to [How to Implement Custom Operations for VPU](../IE_DG/Extensibility_DG/VPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with a
operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to the
[How to Implement Custom Operations for VPU](../IE_DG/Extensibility_DG/VPU_Kernel.md) for more details.
Also, it is necessary to implement nGraph custom operation according to [Custom nGraph Operation](../IE_DG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
operation and correctly infer output tensor shape and type.
Also, it is necessary to implement nGraph custom operation according to the
[Custom nGraph Operation](../IE_DG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
operation and correctly infer output tensors shape and type.
## Enabling Magnetic Resonance Image Reconstruction Model
This chapter provides step-by-step instructions on how to enable the magnetic resonance image reconstruction model implemented in the [repository](https://github.com/rmsouza01/Hybrid-CS-Model-MRI/) using a custom operation on CPU. The example is prepared for a model generated from the repository with hash `2ede2f96161ce70dcdc922371fe6b6b254aafcc8`.
This chapter provides a step-by-step instruction on how to enable the magnetic resonance image reconstruction model
implemented in the [repository](https://github.com/rmsouza01/Hybrid-CS-Model-MRI/) using a custom operation on CPU. The
example is prepared for a model generated from the repository with hash `2ede2f96161ce70dcdc922371fe6b6b254aafcc8`.
### Download and Convert the Model to a Frozen TensorFlow\* Model Format
The original pre-trained model is provided in the hdf5 format which is not supported by OpenVINO directly and needs to be converted to TensorFlow\* frozen model format first.
The original pre-trained model is provided in the hdf5 format which is not supported by OpenVINO directly and needs to
be converted to TensorFlow\* frozen model format first.
1. Download repository `https://github.com/rmsouza01/Hybrid-CS-Model-MRI`:<br>
1. Download repository `https://github.com/rmsouza01/Hybrid-CS-Model-MRI`:<br
```bash
git clone https://github.com/rmsouza01/Hybrid-CS-Model-MRI
git checkout 2ede2f96161ce70dcdc922371fe6b6b254aafcc8
@@ -131,27 +151,14 @@ Firstly, open the model in the TensorBoard or other TensorFlow* model visualizat
batch dimension because the value for the batch dimension is not hardcoded in the model. Model Optimizer need to set all
dynamic dimensions to some specific value to create the IR, therefore specify the command line parameter `-b 1` to set
the batch dimension equal to 1. The actual batch size dimension can be changed at runtime using the Inference Engine API
described in the [Using Shape Inference](../IE_DG/ShapeInference.md). Also refer to the General Conversion Parameters section in [Converting a Model to Intermediate Representation (IR)](../MO_DG/prepare_model/convert_model/Converting_Model.md) and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
described in the [Using Shape Inference](../IE_DG/ShapeInference.md). Also refer to
[Converting a Model Using General Conversion Parameters](../MO_DG/prepare_model/convert_model/Converting_Model_General.md)
and [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
for more details and command line parameters used for the model conversion.
@sphinxdirective
.. tab:: Package, Docker, open-source installation
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
python3 mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1
.. tab:: pip installation
.. code-block:: sh
mo --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1
@endsphinxdirective
> **NOTE**: This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
> the OpenVINO supports this model out of the box.
```bash
./<MO_INSTALL_DIR>/mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1
```
Model Optimizer produces the following error:
```bash
@@ -224,15 +231,19 @@ model. The implementation of the Model Optimizer operation should be saved to `m
The attribute `inverse` is a flag specifying type of the FFT to apply: forward or inverse.
See the "Model Optimizer Operation" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement the operation.
See the "Model Optimizer Operation" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement the operation.
Now it is necessary to implement extractor for the "IFFT2D" operation according to the
"Operation Extractor" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). The
following snippet provides two extractors: one for "IFFT2D", another one for "FFT2D", however only on of them is used in this example. The implementation should be saved to the file `mo_extensions/front/tf/FFT_ext.py`.
"Operation Extractor" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). The
following snippet provides two extractors: one for "IFFT2D", another one for "FFT2D", however only on of them is used
in this example. The implementation should be saved to the file `mo_extensions/front/tf/FFT_ext.py`.
@snippet FFT_ext.py fft_ext:extractor
> **NOTE**: The graph is in inconsistent state after extracting node attributes because according to original operation
> **NOTE:** The graph is in inconsistent state after extracting node attributes because according to original operation
> "IFFT2D" semantic it should have an input consuming a tensor of complex numbers, but the extractor instantiated an
> operation "FFT" which expects a real tensor with specific layout. But the inconsistency will be resolved during
> applying front phase transformations discussed below.
@@ -244,13 +255,14 @@ consumed with the "Complex" operation to produce a tensor of complex numbers. Th
operations can be removed so the "FFT" operation will get a real value tensor encoding complex numbers. To achieve this
we implement the front phase transformation which searches for a pattern of two "StridedSlice" operations with specific
attributes producing data to "Complex" operation and removes it from the graph. Refer to the
"Pattern-Defined Front Phase Transformations" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for more
"Pattern-Defined Front Phase Transformations" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for more
information on how this type of transformation works. The code snippet should be saved to the file
`mo_extensions/front/tf/Complex.py`.
@snippet Complex.py complex:transformation
> **NOTE**: The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
> **NOTE:** The graph is in inconsistent state because the "ComplexAbs" operation consumes complex value tensor but
> "FFT" produces real value tensor.
Now lets implement a transformation which replace a "ComplexAbs" operation with a sub-graph of primitive operations
@@ -268,32 +280,21 @@ The implementation should be saved to the file `mo_extensions/front/tf/ComplexAb
@snippet ComplexAbs.py complex_abs:transformation
Now it is possible to convert the model using the following command line:
@sphinxdirective
.. tab:: Package, Docker, open-source installation
```bash
./<MO_INSTALL_DIR>/mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1 --extensions mo_extensions/
```
.. code-block:: sh
cd <INSTALL_DIR>/deployment_tools/model_optimizer/
python3 mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1 --extensions mo_extensions/
.. tab:: pip installation
.. code-block:: sh
mo --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1 --extensions mo_extensions/
@endsphinxdirective
The sub-graph corresponding to the originally non-supported one is depicted in the image below:
The sub-graph corresponding to the originally non-supported one is depicted on the image below:
![Converted sub-graph](img/converted_subgraph.png)
> **NOTE**: Model Optimizer performed conversion of the model from NHWC to NCHW layout that is why the dimension with
> **NOTE:** Model Optimizer performed conversion of the model from NHWC to NCHW layout that is why the dimension with
> the value 2 moved to another position.
### Inference Engine Extension Implementation
Now it is necessary to implement the extension for the CPU plugin with operation "FFT" introduced previously. The code
below is based on the template extension described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
below is based on the template extension described on the
[Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
#### CMake Build File
The first step is to create a CMake configuration file which builds the extension. The content of the "CMakeLists.txt"
@@ -333,7 +334,7 @@ The last step is to create an extension library "extension.cpp" and "extension.h
operation for the CPU plugin. The code of the library is described in the [Extension Library](../IE_DG/Extensibility_DG/Extension.md).
### Building and Running the Custom Extension
To build the extension, run the following:<br>
In order to build the extension run the following:<br>
```bash
mkdir build && cd build
source /opt/intel/openvino_2021/bin/setupvars.sh
@@ -341,9 +342,9 @@ cmake .. -DCMAKE_BUILD_TYPE=Release
make --jobs=$(nproc)
```
The result of this command is a compiled shared library (`.so` or `.dll`). It should be loaded in the
The result of this command is a compiled shared library (`.so`, `.dylib` or `.dll`). It should be loaded in the
application using `Core` class instance method `AddExtension` like this
`core.AddExtension(std::make_shared<Extension>(compiled_library_file_name), "CPU");`.
`core.AddExtension(make_so_pointer<IExtension>(compiled_library_file_name), "CPU");`.
To test that the extension is implemented correctly we can run the "mri_reconstruction_demo.py" with the following content:
@@ -373,7 +374,7 @@ python3 mri_reconstruction_demo.py \
## Converting Models:
- [Convert Your Caffe* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Caffe.md)
- [Convert Your Kaldi* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md)
- [Convert Your TensorFlow* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_TensorFlow.md)
- [Convert Your MXNet* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_MxNet.md)
- [Convert Your Kaldi* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_Kaldi.md)
- [Convert Your ONNX* Model](../MO_DG/prepare_model/convert_model/Convert_Model_From_ONNX.md)
+15 -2
View File
@@ -1,5 +1,18 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [complex:transformation]
import logging as log
@@ -1,5 +1,18 @@
# Copyright (C) 2018-2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [complex_abs:transformation]
import numpy as np

Some files were not shown because too many files have changed in this diff Show More