Compare commits

..

42 Commits

Author SHA1 Message Date
Alexey Suhov
45d159094b Fix license header in Movidius sources 2021-06-04 19:59:17 +03:00
Artemy Skrebkov
6f860ddcf2 Restore precision setting for benchmark (#4859)
* Add -ip,-op,-iop parameters to benchamrk_app

* Revert not related changes

* Revert extension usage
2021-03-19 10:20:18 +03:00
Andrew Bakalin
a86ae42aed [VPU] Limit dlclose() WA to be used for Ubuntu only (#4806) 2021-03-16 14:54:40 +03:00
Andrey Dmitriev
cef0696ef7 [DOCS] added iname/oname (#4735) 2021-03-16 12:41:54 +03:00
Andrey Zaytsev
e57a96474d Feature/vpu doc fixes 2021 3 (#4635)
* Documentation fixes and updates for VPU

* minor correction

* minor correction

* Fixed links

* updated supported layers list for vpu
2021-03-15 16:40:17 +03:00
Ilya Lavrenov
ed052022d3 Fixed some klockwork issues in C API samples (#4767) 2021-03-15 11:00:07 +03:00
Alina Alborova
6eda5c39c6 Inserted a disclaimer (#4760) 2021-03-12 18:38:58 +03:00
Mikhail Ryzhov
0100810dd6 Added mo.py to wheel packages (#4731) 2021-03-12 15:57:12 +03:00
Elizaveta Lobanova
882e377ef9 [GNA] Update documentation regarding splits and concatenations support (#4740) 2021-03-12 14:00:15 +03:00
Ilya Lavrenov
c9d5d95e2c Blocked dims hwc 2021/3 (#4729)
* Fix for BlockedDims

* Added test for HWC layout
2021-03-11 16:02:25 +03:00
Maxim Shevtsov
d77bc36dcd Docs update (#4626)
* Updated latency case desc to cover multi-socket machines

* updated opt guide a bit

* avoiding '#' which is interpreted as ref

* Update CPU.md

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

* Update docs/optimization_guide/dldt_optimization_guide.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>

Co-authored-by: Alina Alborova <alina.alborova@intel.com>
2021-03-11 15:33:08 +03:00
Andrey Somsikov
19e1b6002e Catch std::except in fuzz tests (#4695)
Fuzz tests must catch all expected exceptions from IE. IE is using C++ std
library which may raise standard exceptions which IE pass through.
2021-03-11 14:09:36 +03:00
Ivan Tikhonov
6bcd0f6072 Fix python API for Loop/TensorIterator/Assign/ReadValue operations 2021-03-11 13:22:50 +03:00
Anton Chetverikov
0e8534a4a9 Add STN to list of supported models (#4728) 2021-03-11 11:55:45 +03:00
Ilya Churaev
08d7c3e75f Restored folded Operations for QueryNetwork (#4685)
* Restored folded Operations for QueryNetwork

* Fixed comment

* Add unfolded constant operations to supported layers map
2021-03-11 10:41:57 +03:00
Evgeny Lazarev
821d513150 Updated documentation about the supported YOLOv3 model from ONNX (#4722) (#4726) 2021-03-11 10:32:13 +03:00
Mikhail Letavin
38a48b9cbf Fix NormalizeL2 creation in QueryNetwork (cherry pick from master PR 4310) (#4651) 2021-03-10 19:41:35 +03:00
Mikhail Letavin
c6d8905a88 [IE CLDNN] Fix missing variable initializations and types (#4669) 2021-03-10 16:23:10 +03:00
Ilya Churaev
3a80652d70 Updated nGraph custom op documentation (#4604)
* Updated nGraph custom op documentation

* Fixed comments
2021-03-10 16:16:57 +03:00
Roman Kazantsev
120d3a596d Document TensorFlow 2* Update: Layers Support and Remove Beta Status (#4474) (#4711)
* Document TensorFlow 2* Update: Layers Support and Remove Beta Status

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Update documentation based on latest test results and feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Remove ConvLSTM2D from supported layers list

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Document Dot layer without limitation

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Address feedback upon DenseFeatures and RNN operations

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Do a grammar correction

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>

* Do a grammar correction based on feedback

Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-03-10 13:35:46 +03:00
Andrew Bakalin
25af83db81 [VPU] WA for Segmentation fault on dlclose() issue (#4645) 2021-03-10 12:09:55 +03:00
Maksim Doronin
03c6f4e3fe [IE][VPU]: Fix empty output of CTCGreedyDecoderSeqLen (#4653)
* Allow the second output of CTCGreedyDecoderSeqLen to be nullptr in cases when it is not used but calculated in the Myriad plugin. In this case, parse the second output as FakeData
* It is a cherry-pick of #4652
* Update the firmware to release version
2021-03-10 11:48:13 +03:00
Mikhail Ryzhov
5d3d323bed [doc] Updated PyPI support OSes (#4643) (#4662)
* [doc] Updated PyPI support OSes (#4643)

* Updated PyPI support OSes

* Added python versions for win and mac

* Update pypi-openvino-dev.md

* Update pypi-openvino-dev.md

* Update pypi-openvino-rt.md

* Update pypi-openvino-dev.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-03-09 17:54:29 +03:00
Anastasia Popova
a53524a554 Fixed bug in assign elimination transformation. (#4644) 2021-03-09 14:58:12 +03:00
Maxim Vafin
02d2dbd0fa Add documentation on how to convert QuartzNet model (#4664)
* Add documentation on how to convert QuartzNet model (#4422)

* Add documentation on how to convert QuartzNet model

* Apply review feedback

* Small fix

* Apply review feedback

* Apply suggestions from code review

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Add reference to file

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
2021-03-09 13:55:46 +03:00
Ilya Lavrenov
bfe0748b4c Docs api (#4657)
* Updated API changes document

* Comment for CVS-49440
2021-03-09 12:50:37 +03:00
Nikolay Shchegolev
d78577aecb [CPU] Statically analyzed issues. (#4637) 2021-03-09 12:26:32 +03:00
Mikhail Ryzhov
e09f0e4808 Corrected PyYAML dependency (#4598) (#4620)
5.4.2 is absent on PyPI
2021-03-05 17:24:36 +03:00
Roman Kazantsev
ff73955354 Add workaround for control edges to support TF 2.4 RNN (#4634)
Signed-off-by: Roman Kazantsev <roman.kazantsev@intel.com>
2021-03-05 16:41:36 +03:00
Vladislav Volkov
18cb230af4 Fix for MKLDNN constant layers execution (#4642)
* Fix for MKLDNN constant layers execution

* Single mkldnn::engine for all MKLDNN graphs
2021-03-05 16:28:10 +03:00
Alina Kladieva
9067a25616 [.ci/azure] Add windows_conditional_compilation.yml (#4648) (#4655) 2021-03-05 15:59:59 +03:00
Gorokhov Dmitriy
c4ff0ffa9d [CPU] Supported ANY layout for inputs in inferRequest (#4621) 2021-03-05 12:30:26 +03:00
Ilya Churaev
4675a12c8f Fixed KW hits (#4638) 2021-03-05 11:13:11 +03:00
Evgeny Lazarev
3cd5da0797 Fixed transformation to pull constants into Loop body (cherry-pick of PR 4591) (#4607)
* Cherry-pick of PR 4591

* Fixed typo

* Moved a check into the parameter_unchanged_after_iteration function
2021-03-04 17:56:54 +03:00
Andrey Zaytsev
9b402f226f Formula fix (#4624) 2021-03-04 16:24:56 +03:00
Alina Kladieva
784adca70a [.ci/azure] Enable CC build (#4619) 2021-03-04 14:49:50 +03:00
Ilya Churaev
8e1603f7fd Fixed clone rt info (#4597) 2021-03-04 13:02:11 +03:00
azhogov
66ede40e4e Merge branch 'releases/2021/3' of https://github.com/openvinotoolkit/openvino into releases/2021/3 2021-03-04 12:22:59 +03:00
azhogov
40a29a7aa3 Azure CI: Add "ref: releases/2021/3" 2021-03-04 12:22:31 +03:00
Vladislav Volkov
a7e00dae54 Fix for broken CC in CPU plugin (#4595) 2021-03-04 12:22:15 +03:00
Andrey Zaytsev
4c40494605 Feature/azaytsev/gna model link fixes (#4599)
* Added info on DockerHub CI Framework

* Feature/azaytsev/change layout (#3295)

* Changes according to feedback comments

* Replaced @ref's with html links

* Fixed links, added a title page for installing from repos and images, fixed formatting issues

* Added links

* minor fix

* Added DL Streamer to the list of components installed by default

* Link fixes

* Link fixes

* ovms doc fix (#2988)

* added OpenVINO Model Server

* ovms doc fixes

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>

* Updated openvino_docs.xml

* Link Fixes

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
2021-03-04 12:00:07 +03:00
Andrey Zaytsev
3e2a4a5df1 Feature/azaytsev/cldnn doc fixes (#4600)
* Legal fixes, removed the Generating docs section

* Removed info regarding generating docs

Co-authored-by: Trawinski, Dariusz <dariusz.trawinski@intel.com>
2021-03-04 11:50:02 +03:00
9855 changed files with 468125 additions and 218624 deletions

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""

View File

@@ -1,6 +0,0 @@
TransposeOpTest.NHWC2NCHW
TransposeOpTest.NCHW2NHWC
TransposeOpTest.TwoDim_int16
GatherOpTest.Gather_axis1_indices2d_int16
SoftmaxOperator.ThreeDimsAxis1
SoftmaxOperator.ThreeDimsAxis0

View File

@@ -1 +0,0 @@
rel-1.7.1

View File

@@ -4,11 +4,13 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/3
jobs:
- job: Lin
@@ -29,7 +31,6 @@ jobs:
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
@@ -57,7 +58,6 @@ jobs:
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'Make dir'
@@ -81,11 +81,7 @@ jobs:
- script: |
sudo apt --assume-yes install libusb-1.0-0-dev
# For opencv-python: setuptools and upgrade
sudo apt-get install python3-setuptools patchelf
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/wheel/requirements-dev.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
@@ -107,12 +103,8 @@ jobs:
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/bin/python3.6
-DENABLE_WHEEL=ON
-DENABLE_TESTS=ON
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
@@ -122,28 +114,16 @@ jobs:
displayName: 'Build Lin'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/cpp/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/c/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
- script: $(BIN_DIR)/unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(BIN_DIR)/InferenceEngineUnitTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(BIN_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
@@ -180,6 +160,18 @@ jobs:
displayName: 'CPU FuncTests'
continueOnError: false
- script: $(BIN_DIR)/MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 $(WORK_DIR)/gtest-parallel/gtest-parallel $(BIN_DIR)/MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
workingDirectory: $(WORK_DIR)
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)

View File

@@ -1,158 +0,0 @@
jobs:
- job: onnxruntime
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 8
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
ONNXRUNTIME_REPO_DIR: $(REPO_DIR)/../onnxruntime
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
INSTALL_DIR: $(WORK_DIR)/install_pkg
BUILD_DIR: $(WORK_DIR)/build
ONNXRUNTIME_UTILS: $(REPO_DIR)/.ci/azure/ci_utils/onnxruntime
ONNXRUNTIME_BUILD_DIR: $(ONNXRUNTIME_REPO_DIR)/build
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
displayName: 'Make dirs'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- script: |
branch=`tr -s '\n ' < $(ONNXRUNTIME_UTILS)/version`
git clone --branch $branch --single-branch --recursive https://github.com/microsoft/onnxruntime.git $(ONNXRUNTIME_REPO_DIR)
displayName: 'Clone onnxruntime'
- script: |
sudo apt --assume-yes install libusb-1.0-0-dev
# For opencv-python: setuptools and upgrade
sudo apt-get install python3-setuptools
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/bin/python3.6
-DENABLE_VPU=OFF
-DENABLE_GNA=OFF
-DENABLE_OPENCV=OFF
-DENABLE_CPPLINT=OFF
-DENABLE_TESTS=OFF
-DENABLE_MKL_DNN=ON
-DENABLE_CLDNN=OFF
-DENABLE_PROFILING_ITT=OFF
-DENABLE_SAMPLES=OFF
-DENABLE_SPEECH_DEMO=OFF
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DNGRAPH_INTERPRETER_ENABLE=ON
-DNGRAPH_DEBUG_ENABLE=OFF
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
echo "2021.2" > $(INSTALL_DIR)/deployment_tools/inference_engine/version.txt
CXXFLAGS="-Wno-error=deprecated-declarations" ./build.sh --config RelWithDebInfo --use_openvino CPU_FP32 --build_shared_lib --parallel --skip_tests --build_dir $(ONNXRUNTIME_BUILD_DIR)
workingDirectory: $(ONNXRUNTIME_REPO_DIR)
displayName: 'Build ONNX Runtime'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
skip_tests=`tr -s '\n ' ':' < $(ONNXRUNTIME_UTILS)/skip_tests`
./onnxruntime_test_all --gtest_filter=-$skip_tests
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_test_all'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_shared_lib_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_shared_lib_test'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_global_thread_pools_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_global_thread_pools_test'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_api_tests_without_env
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_api_tests_without_env'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-converted"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-converted tests'
- script: |
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-operator"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-operator tests'

View File

@@ -4,11 +4,13 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/3
jobs:
- job: Mac
@@ -90,7 +92,7 @@ jobs:
# Disable errors with Ninja
export CXXFLAGS="-Wno-error=unused-command-line-argument"
export CFLAGS="-Wno-error=unused-command-line-argument"
cmake -GNinja -DVERBOSE_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules $(REPO_DIR)
cmake -GNinja -DVERBOSE_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
@@ -133,10 +135,21 @@ jobs:
displayName: 'IE FuncTests'
continueOnError: false
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke*:-smoke_LPT/ReduceMinTransformation.CompareWithRefImpl/f32_Shape* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
continueOnError: false
enabled: false
- script: $(BIN_DIR)/MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 $(WORK_DIR)/gtest-parallel/gtest-parallel $(BIN_DIR)/MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke*:-smoke_MobileNet/ModelTransformationsTest.LPT/mobilenet_v2_tf_depthwise_batch1_inPluginDisabled_inTestDisabled_asymmetric* -- --gtest_print_time=1
workingDirectory: $(WORK_DIR)
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)

View File

@@ -4,11 +4,13 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2021/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2021/3
jobs:
- job: Win
@@ -29,7 +31,6 @@ jobs:
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: D:\build
BUILD_SAMPLES_DIR: D:\build_samples
BIN_DIR: $(REPO_DIR)\bin\intel64
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
@@ -37,7 +38,7 @@ jobs:
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
IB_DIR: C:\Program Files (x86)\IncrediBuild
IB_TESTCONSOLE: $(IB_DIR)\IBTestConsole.exe
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;$(IB_DIR);%PATH%
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.1\opencv\bin;$(IB_DIR);%PATH%
steps:
- script: |
@@ -57,11 +58,10 @@ jobs:
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(BUILD_SAMPLES_DIR) & mkdir $(BUILD_SAMPLES_DIR)
displayName: 'Make dir'
- script: |
certutil -urlcache -split -f https://openvinoweb.z5.web.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
certutil -urlcache -split -f https://incredibuilddiag1wu2.blob.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
call install_ib_console.bat
workingDirectory: $(WORK_DIR)
displayName: 'Install IncrediBuild'
@@ -92,7 +92,7 @@ jobs:
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DENABLE_TEMPLATE_PLUGIN=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DENABLE_TEMPLATE_PLUGIN=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_TESTS=ON -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
@@ -103,23 +103,12 @@ jobs:
displayName: 'Build Win'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\cpp\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\c\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
@@ -175,11 +164,27 @@ jobs:
- script: |
set PATH=$(TEST_ENV_PATH)
rem $(BIN_DIR)\cpuFuncTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-cpuFuncTests.xml
"$(IB_TESTCONSOLE)" $(BIN_DIR)\cpuFuncTests.exe --gtest_filter=*smoke*:-*CompareWithRefs/base_size=16_pre_nms_topn=100_post_nms_topn=100_nms_thresh=0.7_feat_stride=1_min_size=1_ratio* --gtest_output=xml:TEST-cpuFuncTests-IB.xml /testlevel=24
displayName: 'CPU FuncTests - IB'
continueOnError: false
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\MklDnnBehaviorTests --gtest_output=xml:TEST-MklDnnBehaviorTests.xml
displayName: 'MklDnnBehaviorTests'
continueOnError: false
# Add for gtest-parallel, it hangs now (CVS-33386)
#python $(WORK_DIR)\gtest-parallel\gtest-parallel $(BIN_DIR)\MklDnnFunctionalTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=MklDnnFunctionalTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: |
set PATH=$(TEST_ENV_PATH)
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
rem "$(IB_TESTCONSOLE)" $(BIN_DIR)\MklDnnFunctionalTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-MklDnnFunctionalTests-IB.xml
$(BIN_DIR)\MklDnnFunctionalTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-MklDnnFunctionalTests.xml
displayName: 'MklDnnFunctionalTests'
continueOnError: false
- script: |
set PATH=$(TEST_ENV_PATH)
set DATA_PATH=$(MODELS_PATH)

View File

@@ -24,7 +24,7 @@ jobs:
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
IB_DIR: C:\Program Files (x86)\IncrediBuild
IB_TESTCONSOLE: $(IB_DIR)\IBTestConsole.exe
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;$(IB_DIR);%PATH%
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.1\opencv\bin;$(IB_DIR);%PATH%
steps:
- script: |
@@ -47,7 +47,7 @@ jobs:
displayName: 'Make dir'
- script: |
certutil -urlcache -split -f https://openvinoweb.z5.web.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
certutil -urlcache -split -f https://incredibuilddiag1wu2.blob.core.windows.net/incredibuild/install_ib_console.bat install_ib_console.bat
call install_ib_console.bat
workingDirectory: $(WORK_DIR)
displayName: 'Install IncrediBuild'

View File

@@ -1,10 +1,6 @@
FROM ubuntu:20.04
LABEL version=2021.03.30.1
# Build configuration arguments
ARG BUILD_TYPE=Release
ARG PROTOBUF_LITE=OFF
LABEL version=2020.07.09.1
ARG http_proxy
ARG https_proxy
@@ -55,7 +51,7 @@ RUN apt-get update && apt-get -y --no-install-recommends install \
COPY . /openvino/
WORKDIR /openvino/build
RUN cmake .. \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_VPU=OFF \
-DENABLE_GNA=OFF \
-DENABLE_OPENCV=OFF \
@@ -69,17 +65,17 @@ RUN cmake .. \
-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=/usr/bin/python3 \
-DNGRAPH_ONNX_IMPORT_ENABLE=ON \
-DNGRAPH_ONNX_EDITOR_ENABLE=ON \
-DNGRAPH_INTERPRETER_ENABLE=ON \
-DNGRAPH_DEBUG_ENABLE=OFF \
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON \
-DCMAKE_INSTALL_PREFIX=/openvino/dist \
-DNGRAPH_USE_PROTOBUF_LITE=${PROTOBUF_LITE}
-DCMAKE_INSTALL_PREFIX=/openvino/dist
RUN make -j $(nproc) install
# Run tests via tox
WORKDIR /openvino/ngraph/python
ENV ngraph_DIR=/openvino/dist/deployment_tools/ngraph
ENV NGRAPH_CPP_BUILD_PATH=/openvino/dist/deployment_tools/ngraph
ENV LD_LIBRARY_PATH=/openvino/dist/deployment_tools/ngraph/lib
ENV PYTHONPATH=/openvino/bin/intel64/${BUILD_TYPE}/lib/python_api/python3.8:${PYTHONPATH}
ENV NGRAPH_ONNX_IMPORT_ENABLE=TRUE
ENV PYTHONPATH=/openvino/bin/intel64/Release/lib/python_api/python3.8:${PYTHONPATH}
RUN git clone --recursive https://github.com/pybind/pybind11.git -b v2.5.0 --depth 1
CMD tox

View File

@@ -3,19 +3,12 @@
DOCKER_CONTAINER_NAME= "openvino-onnx-ci-container"
DOCKER_IMAGE_TAG = "openvino-onnx-ci-image"
ONNX_MODEL_ZOO_SHA = "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
BACKEND_CONFIGURATIONS = [
[ name: "Release", build_type: "Release", protobuf_lite : "OFF" ],
[ name: "Debug", build_type: "Debug", protobuf_lite : "OFF" ],
[ name: "Rel_Lite", build_type: "Release", protobuf_lite : "ON" ],
]
// workaround for aborting previous builds on PR update
@NonCPS
def stopPreviousRunningBuilds() {
def jobname = env.JOB_NAME
if (jobname.startsWith("onnx-ci/openvino onnx ci/openvino/PR")){
if (jobname.startsWith("onnx/openvino_ci/PR")){
def buildnum = env.BUILD_NUMBER.toInteger()
def job = Jenkins.instance.getItemByFullName(jobname)
def job_newest = job.builds.first()
@@ -30,7 +23,7 @@ BACKEND_CONFIGURATIONS = [
}
def getGitPrInfo(String project, String workdir) {
def getGitPrInfo(String project) {
def gitPrInfo = [
prAuthorEmail : "",
commitAuthorEmail : "",
@@ -38,7 +31,7 @@ def getGitPrInfo(String project, String workdir) {
commitSubject : ""
]
try {
dir ("${workdir}/${project}") {
dir ("${WORKDIR}/${project}") {
gitPrInfo.prAuthorEmail = sh (script: 'git log -1 --pretty="format:%ae" ', returnStdout: true).trim()
gitPrInfo.commitAuthorEmail = sh (script: 'git log -1 --pretty="format:%ce" ', returnStdout: true).trim()
gitPrInfo.commitSubject = sh (script: 'git log -1 --pretty="format:%H" ', returnStdout: true).trim()
@@ -71,146 +64,96 @@ def notifyByEmail(def gitPrInfo) {
}
}
def gitSubmoduleUpdate(String repository_name, String workdir) {
dir ("${workdir}/${repository_name}") {
def gitSubmoduleUpdate(String repository_name) {
dir ("${WORKDIR}/${repository_name}") {
sh label: "Init ${repository_name} submodules",
script:
"""
git submodule init && git submodule update \
--init \
--no-fetch \
--recursive
--recursive
"""
}
}
def prepare_repository(String workdir) {
dir("${workdir}") {
println "Preparing repository in directory: ${workdir}"
checkout scm
gitSubmoduleUpdate(PROJECT_NAME, workdir)
}
}
def updateModels() {
sh """
./ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d ${HOME}/ONNX_CI/models_data -o -s ${ONNX_MODEL_ZOO_SHA}
./ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d ${HOME}/ONNX_CI/data -o
"""
}
def get_docker_container_name(Map configuration){
println "RUN get_docker_container_name for ${configuration.name}"
String docker_container_name = "${DOCKER_CONTAINER_NAME}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
return docker_container_name
}
def buildDockerImage(Map configuration, String workdir) {
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
println "docker_image_tag: ${docker_image_tag}"
def buildDockerImage() {
updateModels()
sh """
docker build --tag=${docker_image_tag} \
--build-arg BUILD_TYPE=${configuration.build_type} \
--build-arg PROTOBUF_LITE=${configuration.protobuf_lite} \
--file=.ci/openvino-onnx/Dockerfile \
--build-arg http_proxy=http://proxy-ir.intel.com:911/ \
--build-arg https_proxy=http://proxy-ir.intel.com:911/ .
docker build --tag=${DOCKER_IMAGE_TAG} --file=.ci/openvino-onnx/Dockerfile \
--build-arg http_proxy=http://proxy-chain.intel.com:911/ \
--build-arg https_proxy=http://proxy-chain.intel.com:912/ .
"""
}
def runTests(Map configuration, String workdir) {
println "Run tests for ${configuration.name}"
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
String docker_container_name = get_docker_container_name(configuration)
// Run only basic unit tests in Debug configuration
if (configuration.build_type == "Debug") {
sh """
docker run --name ${docker_container_name} ${docker_image_tag}
"""
}
// Run unit-tests AND large model tests by default
else {
sh """
docker run --name ${docker_container_name} \
--volume ${HOME}/ONNX_CI/models_data/model_zoo/onnx_model_zoo_${ONNX_MODEL_ZOO_SHA}:/root/.onnx/model_zoo/onnx_model_zoo \
--volume ${HOME}/ONNX_CI/data/model_zoo/MSFT:/root/.onnx/model_zoo/MSFT \
${docker_image_tag} /bin/bash -c "tox && tox -e zoo_models"
"""
}
def runTests() {
sh """
docker run --name ${DOCKER_CONTAINER_NAME} \
--volume ${HOME}/ONNX_CI/data/model_zoo:/root/.onnx/model_zoo \
${DOCKER_IMAGE_TAG}
"""
}
def getConfigurationsMap() {
def configurationsMap = [:]
for (backend in BACKEND_CONFIGURATIONS) {
def configuration = backend.clone()
configurationsMap[configuration.name] = {
stage(configuration.name) { CONFIGURATION_WORKFLOW(configuration) }
}
pipeline {
agent {
label "OpenVino"
}
return configurationsMap
}
CONFIGURATION_WORKFLOW = { configuration ->
node("OpenVINO") {
try {
PROJECT_NAME = "openvino"
String workdir = "${HOME}/workspace/${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
stage("Clone repository") {
prepare_repository(workdir)
}
stage("Prepare Docker environment") {
dir("${workdir}") {
buildDockerImage(configuration, workdir)
environment {
PROJECT_NAME = "openvino"
WORKDIR = "${WORKSPACE}/${BUILD_NUMBER}"
}
options {
skipDefaultCheckout true
timeout(activity: true, time: 60, unit: 'MINUTES')
}
stages {
stage("Clone repository") {
steps{
stopPreviousRunningBuilds()
dir("${WORKDIR}") {
checkout scm
}
gitSubmoduleUpdate(PROJECT_NAME)
}
stage("Run tests") {
timeout(time: 60, unit: 'MINUTES') {
runTests(configuration, workdir)
}
stage("Prepare Docker environment") {
steps{
dir("${WORKDIR}") {
buildDockerImage()
}
}
}
catch(e) {
// Set result to ABORTED if exception contains exit code of a process interrupted by SIGTERM
if ("$e".contains("143")) {
currentBuild.result = "ABORTED"
} else {
currentBuild.result = "FAILURE"
}
def gitPrInfo = getGitPrInfo(PROJECT_NAME, workdir)
stage("Run tests") {
options {
timeout(time: 60, unit: 'MINUTES')
}
steps{
runTests()
}
}
}
post {
failure {
script {
gitPrInfo = getGitPrInfo(PROJECT_NAME)
notifyByEmail(gitPrInfo)
}
}
finally {
stage("Cleanup") {
cleanup {
dir("${WORKDIR}") {
deleteDir()
String docker_container_name = get_docker_container_name(configuration)
sh """
docker rm -f ${docker_container_name}
docker image prune -f
docker rm -f ${DOCKER_CONTAINER_NAME}
"""
}
}
}
}
pipeline {
agent none
options {
skipDefaultCheckout true
timeout(activity: true, time: 120, unit: 'MINUTES')
}
stages {
stage('Parallel CI') {
steps {
stopPreviousRunningBuilds()
script {
parallelStagesMap = getConfigurationsMap()
parallel parallelStagesMap
}
}
}
}
}

View File

@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging

View File

@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests

View File

@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse

View File

@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests

View File

@@ -1,6 +1,6 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import datetime

View File

@@ -1,17 +1,16 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Check GitHub organization and invite members
"""
# pylint: disable=fixme,no-member,too-many-locals
# pylint: disable=fixme,no-member
from argparse import ArgumentParser
import github_api
from configs import Config
from github_api import GithubOrgApi, get_dev_emails
from ldap_api import LdapApi, print_user_info, InfoLevel
def main():
@@ -20,74 +19,32 @@ def main():
arg_parser.add_argument("--cfg-file", metavar="PATH", default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}")
arg_parser.add_argument("--teams", action="store_true", help="Check GitHub teams")
arg_parser.add_argument("--no-ldap", action="store_true", help="Don't use LDAP info")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
gh_api = GithubOrgApi()
gh_api = github_api.GithubOrgApi()
if args.teams:
gh_api.get_org_teams()
return
else:
dev_emails = github_api.get_dev_emails()
print(f'\nDeveloper emails {len(dev_emails)}:', '; '.join(dev_emails))
cfg_emails = get_dev_emails()
print(f'\nCfg developer emails {len(cfg_emails)}:', '; '.join(sorted(cfg_emails)))
org_emails = gh_api.get_org_emails()
print(f'\nOrg emails {len(org_emails)}:', '; '.join(org_emails))
dev_emails = set()
dev_emails.update(cfg_emails)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
if not args.no_ldap:
ldap_api = LdapApi()
ldap_emails = ldap_api.get_user_emails()
dev_emails.update(ldap_emails)
print(f'\nLDAP developer emails {len(ldap_emails)}:', '; '.join(sorted(ldap_emails)))
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f'\nInvite emails {len(invite_emails)}:', '; '.join(invite_emails))
cfg_emails_no_in_ldap = ldap_api.get_absent_emails(cfg_emails)
print(f'\nCfg developer emails - absent in LDAP at all {len(cfg_emails_no_in_ldap)}:',
'; '.join(sorted(cfg_emails_no_in_ldap)))
no_in_dev_emails = org_emails.difference(dev_emails)
print(f'\nOrg members - no in developers list {len(no_in_dev_emails)}:',
'; '.join(no_in_dev_emails))
cfg_ldap_inters = cfg_emails.intersection(ldap_emails)
print(f'\nCfg developer emails - present in LDAP developers {len(cfg_ldap_inters)}:',
'; '.join(sorted(cfg_ldap_inters)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
org_emails, org_logins_no_intel_email = gh_api.get_org_emails()
print(f'\nOrg emails {len(org_emails)}:', '; '.join(sorted(org_emails)))
org_emails_no_in_ldap = set()
if not args.no_ldap:
org_ldap_diff = org_emails.difference(ldap_emails)
print(f'\nOrg member emails - absent in LDAP developers {len(org_ldap_diff)}:',
'; '.join(sorted(org_ldap_diff)))
for email in org_ldap_diff:
user_info = ldap_api.get_user_info_by_email(email)
if user_info:
print_user_info(user_info, InfoLevel.PDL)
else:
org_emails_no_in_ldap.add(email)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f'\nInvite emails {len(invite_emails)}:', '; '.join(sorted(invite_emails)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
gh_api.invite_users(valid_github_users)
print('\nCheck accounts below and remove from the GitHub organization and cfg list')
cfg_emails_no_in_org = sorted(cfg_emails.difference(org_emails))
print(f'\nCfg developer emails - absent in GitHub organization {len(cfg_emails_no_in_org)}:',
'; '.join(cfg_emails_no_in_org))
org_emails_no_in_dev = sorted(org_emails.difference(dev_emails))
print(f'\nOrg member emails - absent in cfg and LDAP developers {len(org_emails_no_in_dev)}:',
'; '.join(org_emails_no_in_dev))
print(f'\nOrg member emails - absent in LDAP at all {len(org_emails_no_in_ldap)}:',
'; '.join(sorted(org_emails_no_in_ldap)))
print(f'\nOrg member logins - absent Intel email {len(org_logins_no_intel_email)}:',
'; '.join(sorted(org_logins_no_intel_email)))
gh_api.invite_users(valid_github_users)
if __name__ == '__main__':

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -8,7 +8,6 @@ Check GitHub PRs and set labels by type and categories, e.g. 'ExternalPR', 'cate
# pylint: disable=fixme,no-member
import re
import sys
import datetime
from argparse import ArgumentParser
from enum import Enum
@@ -19,11 +18,10 @@ from configs import Config
class PrType(Enum):
"""Constants for type of GitHub pull request by author membership"""
EXTERNAL = "ExternalPR"
INTEL = "ExternalIntelPR"
ORG = "OpenvinoPR"
BAD = "BadPR"
EXTERNAL = 'ExternalPR'
INTEL = 'ExternalIntelPR'
ORG = 'OpenvinoPR'
BAD = 'BadPR'
def get_pr_labels(pull):
@@ -35,23 +33,13 @@ def get_pr_labels(pull):
def set_pr_labels(pull, labels):
"""Sets new PR labels (all previously set labels are removed)"""
"""Sets PR labels"""
if not labels or Config().DRY_RUN:
return
print("Set PR labels:", labels)
# set_labels() should accept list but fails with empty "AssertionError:"
print(f'Set PR labels:', labels)
pull.set_labels(labels)
def add_pr_labels(pull, labels):
"""Adds PR labels"""
if not labels or Config().DRY_RUN:
return
print("Add PR labels:", labels)
for label in labels:
pull.add_to_labels(label)
def get_pr_type_by_labels(pull):
"""Gets PR type using labels"""
pr_lables = get_pr_labels(pull)
@@ -60,19 +48,19 @@ def get_pr_type_by_labels(pull):
if not pr_types_labels:
return None
if len(pr_types_labels) > 1:
print(f"Duplicated labels: {pr_types_labels}")
print(f'Duplicated labels: {pr_types_labels}')
return PrType.BAD
return PrType(PrType(pr_types_labels.pop()))
def get_label_by_team_name_re(team_name):
"""Generates label by PR reviwer team name using regular expressions"""
if "admins" in team_name:
return "category: ci"
re_compile_label = re.compile(rf"{Config().GITHUB_REPO}-(.+)-maintainers")
if 'admins' in team_name:
return 'category: ci'
re_compile_label = re.compile(rf'{Config().GITHUB_REPO}-(.+)-maintainers')
re_label = re_compile_label.match(team_name)
if re_label:
return f"category: {re_label.group(1).strip()}"
return f'category: {re_label.group(1).strip()}'
return None
@@ -92,113 +80,17 @@ def get_category_labels(pull):
return labels
def get_pr_info_str(pull):
"""Gets info about PR using a few workarounds"""
pr_title = pull.title.encode("ASCII", "ignore").decode()
# Workaround for PyGithub issue: https://github.com/PyGithub/PyGithub/issues/512
pr_created_at = pull.created_at.replace(tzinfo=datetime.timezone.utc).astimezone()
return (
f"PR: {pull.number} - {pr_title} - Created: {pr_created_at} - "
f"Labels: {get_pr_labels(pull)} - Type: {get_pr_type_by_labels(pull)}"
)
def update_labels(gh_api, pull, non_org_intel_pr_users, non_org_pr_users):
"""Checks and updates labels"""
print("Check and update labels:")
pr_type_by_labels = get_pr_type_by_labels(pull)
add_labels = []
# Checks PR source type
if gh_api.is_org_user(pull.user):
print(" - Org user")
elif github_api.is_intel_email(pull.user.email) or github_api.is_intel_company(
pull.user.company
):
print(" - Non org user with Intel email or company")
non_org_intel_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.INTEL:
print(f'NO "{PrType.INTEL.value}" label: ', end="")
github_api.print_users(pull.user)
add_labels.append(PrType.INTEL.value)
elif github_api.is_user_ignored(pull.user):
print(" - IGNORED non org user with NO Intel email or company")
else:
print(" - Non org user with NO Intel email or company")
non_org_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.EXTERNAL:
print(f'NO "{PrType.EXTERNAL.value}" label: ', end="")
github_api.print_users(pull.user)
add_labels.append(PrType.EXTERNAL.value)
add_labels += get_category_labels(pull)
add_pr_labels(pull, add_labels)
def get_wrong_commits(pull):
"""Returns commits with incorrect user and email"""
pr_author_email = (pull.user.email or "").lower()
print("GitHub PR author email:", pr_author_email)
print("Check commits:")
wrong_commits = set()
for commit in pull.get_commits():
# import pprint; pprint.pprint(commit.raw_data)
print("Commit SHA:", commit.sha)
# Use raw data because commit author can be non GitHub user
commit_author_email = (commit.raw_data["commit"]["author"]["email"] or "").lower()
commit_committer_email = (commit.raw_data["commit"]["committer"]["email"] or "").lower()
print(" Commit author email:", commit_author_email)
print(" Commit committer email:", commit_committer_email)
if not github_api.is_valid_user(commit.author):
print(
" ERROR: User with the commit author email is absent in GitHub:",
commit.raw_data["commit"]["author"]["name"],
)
wrong_commits.add(commit.sha)
if not github_api.is_valid_user(commit.committer):
print(
" ERROR: User with the commit committer email is absent in GitHub:",
commit.raw_data["commit"]["committer"]["name"],
)
wrong_commits.add(commit.sha)
if not commit.raw_data["commit"]["verification"]["verified"]:
print(
" WARNING: The commit is not verified. Reason:",
commit.raw_data["commit"]["verification"]["reason"],
)
if pr_author_email != commit_author_email or pr_author_email != commit_committer_email:
print(" WARNING: Commit emails and GitHub PR author public email are differnt")
return wrong_commits
def main():
"""The main entry point function"""
arg_parser = ArgumentParser()
arg_parser.add_argument(
"--cfg-file",
metavar="PATH",
default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}",
)
arg_parser.add_argument(
"--pr", metavar="NUMBER", help="Get GitHub pull request with the number"
)
arg_parser.add_argument(
"--pr-state",
default="open",
choices=["open", "closed"],
help="Set GitHub pull request state",
)
arg_parser.add_argument(
"--newer", metavar="MINUTES", help="Get newly created GitHub pull request only"
)
arg_parser.add_argument(
"--check-commits",
action="store_true",
help="Check and compare git commit email with GitHub account email",
)
arg_parser.add_argument("--cfg-file", metavar="PATH", default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}")
arg_parser.add_argument("--pr", metavar="NUMBER",
help="Get GitHub pull request with the number")
arg_parser.add_argument("--pr-state", default="open", choices=["open", "closed"],
help="Set GitHub pull request state")
arg_parser.add_argument("--newer", metavar="MINUTES",
help="Get newly created GitHub pull request only")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
@@ -208,52 +100,50 @@ def main():
pulls = [gh_api.repo.get_pull(int(args.pr))]
else:
pulls = gh_api.repo.get_pulls(state=args.pr_state)
print(f"\nPRs count ({args.pr_state}):", pulls.totalCount)
print(f'\nPRs count ({args.pr_state}):', pulls.totalCount)
if args.newer:
pr_created_after = (
datetime.datetime.now() - datetime.timedelta(minutes=int(args.newer))
).astimezone()
print("Checking PRs created after:", pr_created_after)
pr_created_after = datetime.datetime.now() - datetime.timedelta(minutes=int(args.newer))
print('PRs created after:', pr_created_after)
non_org_intel_pr_users = set()
non_org_pr_users = set()
wrong_pulls = {}
for pull in pulls:
pr_created_at = pull.created_at.replace(tzinfo=datetime.timezone.utc).astimezone()
if args.newer and pr_created_at <= pr_created_after:
print(f"\nIGNORE: {get_pr_info_str(pull)}")
if args.newer and pull.created_at <= pr_created_after:
print(f'\nIGNORE: {pull} - Created: {pull.created_at}')
continue
pr_lables = get_pr_labels(pull)
pr_type_by_labels = get_pr_type_by_labels(pull)
set_labels = []
print(f'\n{pull} - Created: {pull.created_at} - Labels: {pr_lables} -',
f'Type: {pr_type_by_labels}', end='')
print(f"\n{get_pr_info_str(pull)}")
if args.check_commits:
wrong_commits = get_wrong_commits(pull)
if wrong_commits:
wrong_pulls[pull.number] = wrong_commits
# Checks PR source type
if gh_api.is_org_user(pull.user):
print(' - Org user')
elif github_api.is_intel_email(pull.user.email) or \
github_api.is_intel_company(pull.user.company):
print(' - Non org user with Intel email or company')
non_org_intel_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.INTEL:
print(f'NO "{PrType.INTEL.value}" label: ', end='')
github_api.print_users(pull.user)
set_labels.append(PrType.INTEL.value)
else:
update_labels(gh_api, pull, non_org_intel_pr_users, non_org_pr_users)
print(f' - Non org user with NO Intel email or company')
non_org_pr_users.add(pull.user)
if pr_type_by_labels is not PrType.EXTERNAL:
print(f'NO "{PrType.EXTERNAL.value}" label: ', end='')
github_api.print_users(pull.user)
set_labels.append(PrType.EXTERNAL.value)
if wrong_pulls:
for pull_number, wrong_commits in wrong_pulls.items():
print(
f"\nERROR: Remove or replace wrong commits in the PR {pull_number}:\n ",
"\n ".join(wrong_commits),
)
print(
"\nAbout commit signature verification:\n ",
"https://docs.github.com/en/github/authenticating-to-github/"
"managing-commit-signature-verification/about-commit-signature-verification",
)
sys.exit(1)
set_labels += get_category_labels(pull)
set_pr_labels(pull, set_labels)
if non_org_intel_pr_users:
print("\nNon org user with Intel email or company:")
github_api.print_users(non_org_intel_pr_users)
if non_org_pr_users:
print("\nNon org user with NO Intel email or company:")
github_api.print_users(non_org_pr_users)
print(f'\nNon org user with Intel email or company:')
github_api.print_users(non_org_intel_pr_users)
print(f'\nNon org user with NO Intel email or company:')
github_api.print_users(non_org_pr_users)
if __name__ == "__main__":
if __name__ == '__main__':
main()

View File

@@ -6,9 +6,7 @@
"openvino-ci",
"openvino-pushbot",
"lab-nerval",
"lab-nerval-onnx-ci",
"onnx-watchdog-agent",
"dependabot"
"lab-nerval-onnx-ci"
],
"EMAILS_FILE_PATH": "dev_emails-test.txt",
"PROXIES": {

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -57,19 +57,19 @@ class Config:
for name, value in self._json_cfg.items():
if hasattr(self, name):
raise ConfigException(f'Duplicating prosperity: {name}')
property_value = self._args.get(name) or os.getenv(name)
if property_value:
prosperity_value = self._args.get(name) or os.getenv(name)
if prosperity_value:
# Try to set prosperity_value as Python literal structures, e.g. DRY_RUN=False
try:
property_value = ast.literal_eval(property_value)
prosperity_value = ast.literal_eval(prosperity_value)
except Exception:
pass
if not isinstance(property_value, type(value)):
if not isinstance(prosperity_value, type(value)):
raise ConfigException(f'Python type of {name} parameter must be {type(value)}')
else:
property_value = value
setattr(self, name, property_value)
Config.properties[name] = property_value
prosperity_value = value
setattr(self, name, prosperity_value)
Config.properties[name] = prosperity_value
self.set_proxy()
@@ -78,7 +78,7 @@ class Config:
try:
with open(self._file_path) as conf:
self._json_cfg = json.load(conf)
except Exception:
except:
print('Failed to load configuration from:', self._file_path)
raise
@@ -105,7 +105,7 @@ class Config:
def _test():
"""Test and debug"""
print('Config.default_cfg_path:', Config.default_cfg_path)
cfg = Config(cli_args=['DRY_RUN', 'PROXIES={"NO_PROXY": "localhost"}'])
cfg = Config(cli_args=['DRY_RUN=True'])
print('Config.properties:', cfg.get_properties())

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
@@ -11,6 +11,7 @@ import re
import time
from github import Github, GithubException, RateLimitExceededException, IncompletableObject
from github import UnknownObjectException
from github.PaginatedList import PaginatedList
from configs import Config
@@ -109,13 +110,17 @@ class GithubOrgApi:
def is_org_user(self, user):
"""Checks that user is a member of GitHub organization"""
if is_valid_user(user):
# user.get_organization_membership(self.github_org) doesn't work with org members
# permissions, GITHUB_TOKEN must be org owner now
return self.github_org.has_in_members(user)
try:
membership = user.get_organization_membership(self.github_org)
# membership.role can be 'member' or 'admin'
if membership.state == 'active' and membership.role:
return True
except UnknownObjectException:
pass
return False
def get_org_emails(self):
"""Gets and prints emails of all GitHub organization members"""
"""Gets and prints all emails of GitHub organization members"""
org_members = self.github_org.get_members()
org_emails = set()
org_members_fix = set()
@@ -141,7 +146,7 @@ class GithubOrgApi:
'; '.join(org_logins_fix_intel_email))
print(f'\nOrg members - no real name {len(org_emails_fix_name)}:',
'; '.join(org_emails_fix_name))
return (org_emails, org_logins_fix_intel_email)
return org_emails
def get_org_invitation_emails(self):
"""Gets GitHub organization teams prints info"""

View File

@@ -1,236 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Gets info about users and groups via LDAP
"""
# pylint: disable=fixme,no-member
from enum import Enum
from ldap3 import Server, Connection, ALL, SUBTREE
from configs import Config
class LdapApiException(Exception):
"""Base LDAP API exception"""
class InfoLevel(Enum):
"""Constants for printing user info from LDAP"""
PDL = 'PDL' # Public Distribution List (group of e-mail addresses)
FULL = 'Full'
def print_user_info(info, info_level=None):
"""Pretty-print of a user info data structure (dict). info_level is the InfoLevel Enum"""
if not info or not info.get('mail'):
raise LdapApiException('ERROR: No info or absent mail')
def get_membership():
if info_level == InfoLevel.PDL:
membership_info = ' PDLs:'
elif info_level == InfoLevel.FULL:
membership_info = ' memberOf :'
else:
return ''
# Grouping groups by purpose
if info_level == InfoLevel.PDL:
sort_key = lambda i: i.split(',', 1)[0].lower()
else:
sort_key = lambda i: i.split(',', 1)[1] + i.split(',', 1)[0].lower()
for item in sorted(info['memberOf'], key=sort_key):
if info_level == InfoLevel.PDL and 'OU=Delegated' not in item:
continue
membership_info += f'\n {item}'
return membership_info
try:
text_info = \
f'\n{info["cn"]} <{info["mail"]}>; {info["sAMAccountName"]}; {info["employeeID"]}' \
f'\n Org group: {info["intelSuperGroupDescr"]} ({info["intelSuperGroupShortName"]}) /'\
f' {info["intelGroupDescr"]} ({info["intelGroupShortName"]}) /' \
f' {info["intelDivisionDescr"]} ({info["intelDivisionShortName"]}) /' \
f' {info["intelOrgUnitDescr"]}' \
f'\n Manager: {info["manager"]}' \
f'\n Location: {info["intelRegionCode"]} / {info["co"]} / {info["intelSiteCode"]} /' \
f' {info["intelBldgCode"]} ({info["intelSiteName"]}) /' \
f' {info["physicalDeliveryOfficeName"]}' \
f'\n Other: {info["employeeType"]} | {info["intelExportCountryGroup"]} |' \
f' {info["whenCreated"]} | {info["intelCostCenterDescr"]} | {info["jobDescription"]}'
except Exception as exc:
raise LdapApiException(f'ERROR: Failed to get info about "{info["mail"]}". ' \
f'Exception occurred:\n{repr(exc)}') from exc
print(text_info)
membership = get_membership()
if info_level == InfoLevel.PDL and membership:
print(membership)
elif info_level == InfoLevel.FULL:
for key in sorted(info):
if isinstance(info[key], list):
if key == 'memberOf':
print(membership)
else:
print(f' {key} :')
for item in info[key]:
print(' ', item)
else:
print(f' {key} : {info[key]}')
class LdapApi:
"""LDAP API for getting user info and emails"""
_binary_blobs = ['thumbnailPhoto', 'msExchUMSpokenName', 'msExchBlockedSendersHash']
_check_existing = [
'intelExportCountryGroup',
'physicalDeliveryOfficeName',
'intelSuperGroupShortName',
'intelGroupShortName',
'intelDivisionShortName',
]
null = '<null>'
def __init__(self):
self._cfg = Config()
self.server = Server(self._cfg.LDAP_SERVER, get_info=ALL)
self.connection = Connection(self.server,
user=self._cfg.LDAP_USER,
password=self._cfg.LDAP_PASSWORD,
auto_bind=True)
self.connection.bind()
def get_user_emails(self, groups=None):
"""Gets emails of LDAP groups and sub-groups"""
print('\nGet emails from LDAP groups:')
processed_ldap_members = {}
def process_group_members(member, parent_group):
if member in processed_ldap_members:
processed_ldap_members[member]['parent_groups'].append(parent_group)
print('\nWARNING: Ignore LDAP member to avoid duplication and recursive cycling '
f'of PDLs: {member}\n '
f'email: {processed_ldap_members[member].get("email")}\n parent_groups:')
for group in processed_ldap_members[member].get('parent_groups', []):
print(7 * ' ', group)
return
processed_ldap_members[member] = {'email': None, 'parent_groups': [parent_group]}
# AD moves terminated users to the boneyard OU in case the user returns,
# so it can be reactivated with little effort.
# After 30 days it is removed and the unix personality becomes unlinked.
if 'OU=Boneyard' in member:
return
self.connection.search(member, r'(objectClass=*)', SUBTREE,
attributes=['cn', 'member', 'mail'])
#print(self.connection.entries)
if not self.connection.response:
raise LdapApiException(f'ERROR: empty response. LDAP member: {member}')
# Check that the member is worker.
# The response can contain several items, but the first item is valid only
if 'OU=Workers' in member:
if self.connection.response[0]['attributes']['mail']:
processed_ldap_members[member]['email'] = \
self.connection.response[0]['attributes']['mail'].lower()
return
raise LdapApiException(f'ERROR: no mail. LDAP worker: {member}\n'
f'{self.connection.entries}')
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {member}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
if self.connection.response[0]['attributes']['member']:
for group_member in self.connection.response[0]['attributes']['member']:
process_group_members(group_member, member)
else:
print(f'\nERROR: no members in LDAP group: {member}\n{self.connection.entries}')
for group in groups or self._cfg.LDAP_PDLs:
print('\nProcess ROOT LDAP group:', group)
process_group_members(group, 'ROOT')
return {
member.get('email') for member in processed_ldap_members.values() if member.get('email')
}
def _get_user_info(self, query):
"""Gets user info from LDAP as dict matching key and values pairs from query"""
query_filter = ''.join(f'({key}={value})' for key, value in query.items())
for domain in self._cfg.LDAP_DOMAINS:
search_base = f'OU=Workers,DC={domain},DC=corp,DC=intel,DC=com'
self.connection.search(
search_base,
f'(&(objectcategory=person)(objectclass=user)(intelflags=1){query_filter})',
SUBTREE,
attributes=['*'])
if self.connection.response:
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {query_filter}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
info = self.connection.response[0]['attributes']
# remove long binary blobs
for blob in LdapApi._binary_blobs:
info[blob] = b''
for key in LdapApi._check_existing:
if not info.get(key):
info[key] = LdapApi.null
return info
return {}
def get_user_info_by_idsid(self, idsid):
"""Gets user info from LDAP as dict using account name for searching"""
return self._get_user_info({'sAMAccountName': idsid})
def get_user_info_by_name(self, name):
"""Gets user info from LDAP as dict using common name for searching"""
return self._get_user_info({'cn': name})
def get_user_info_by_email(self, email):
"""Gets user info from LDAP as dict using emails for searching"""
return self._get_user_info({'mail': email})
def get_absent_emails(self, emails):
"""Checks users by email in LDAP and returns absent emails"""
absent_emails = set()
for email in emails:
if not self.get_user_info_by_email(email):
absent_emails.add(email)
return absent_emails
def _test():
"""Test and debug"""
ldap = LdapApi()
emails = ldap.get_user_emails()
print(f'\nLDAP emails count: {len(emails)}\n{"; ".join(emails)}')
emails = ['foo@intel.com']
for email in emails:
info = ldap.get_user_info_by_email(email)
if info:
print_user_info(info, InfoLevel.PDL)
else:
print(f'\n{email} - not found')
if __name__ == '__main__':
_test()

View File

@@ -1 +0,0 @@
pylint==2.5.3

View File

@@ -1,2 +1 @@
PyGithub==1.51
ldap3==2.7

View File

@@ -0,0 +1 @@
pylint==2.3.0

View File

@@ -3,7 +3,6 @@ on: [push, pull_request]
jobs:
Build_Doc:
if: github.repository == 'openvinotoolkit/openvino'
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
@@ -39,7 +38,6 @@ jobs:
working-directory: build
- name: 'Upload doc'
if: github.event_name == 'push'
uses: actions/upload-artifact@v2
with:
name: openvino_doc

View File

@@ -1,17 +0,0 @@
name: PR Commits
on: [pull_request]
jobs:
Checks:
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@v2
- name: Install dependencies
run: python3 -m pip install -r ./.github/org_control/requirements.txt
- name: PR commits
run: python3 ./.github/org_control/check_pr.py --pr=${{ github.event.number }} --check-commits DRY_RUN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -2,45 +2,42 @@ name: Code Style
on: [push, pull_request]
jobs:
clang-format:
runs-on: ubuntu-20.04
nGraph:
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Install clang-format-9
run: sudo apt --assume-yes install clang-format-9
- name: Install clang-format-3.9
run: sudo apt --assume-yes install clang-format-3.9
- name: Install dependencies
run: |
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install --upgrade pip
python3 -m pip install -r ./inference-engine/ie_bridges/python/requirements.txt
# Add for -DENABLE_PYTHON=ON, no cython
python3 -m pip install -r ./inference-engine/ie_bridges/python/src/requirements-dev.txt
# Run cmake with -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT in order to enable codestyle check for ITT collector
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_PYTHON=ON -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT ..
cmake ..
- name: Check code style
run: cmake --build build --target clang_format_check_all
run: make style-check
working-directory: build
- name: Create code style diff
if: failure()
run: |
cmake --build build --target clang_format_fix_all
git diff > code_style_diff.diff
ngraph/maint/apply-code-format.sh
git diff >ngraph_code_style_diff.patch
- uses: actions/upload-artifact@v2
if: failure()
with:
name: code_style_diff
path: code_style_diff.diff
name: ngraph_code_style_diff
path: ngraph_code_style_diff.patch
ShellCheck:
runs-on: ubuntu-18.04

View File

@@ -80,22 +80,12 @@ jobs:
python3 setup.py sdist bdist_wheel
working-directory: model-optimizer
- name: Test package content
run: |
echo "src = open('openvino_mo.egg-info/SOURCES.txt', 'rt').read().split()" | tee -a test_wheel.py
echo "ref = open('automation/package_BOM.txt', 'rt').read().split()" | tee -a test_wheel.py
echo "for name in ref:" | tee -a test_wheel.py
echo " if name.endswith('.py'):" | tee -a test_wheel.py
echo " assert name in src or './' + name in src, name + ' file missed'" | tee -a test_wheel.py
python3 test_wheel.py
working-directory: model-optimizer
- name: Test conversion
- name: Test
run: |
wget -q http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz
tar -xf mobilenet_v1_1.0_224.tgz
python3 -m pip install model-optimizer/dist/*.whl
python3 -m mo --input_model mobilenet_v1_1.0_224_frozen.pb --input_shape "[1,224,224,3]"
python3 -c "import sys, subprocess, mo_tf; subprocess.run([sys.executable, mo_tf.__file__, '--input_model', 'mobilenet_v1_1.0_224_frozen.pb', '--input_shape', '[1,224,224,3]'], check=True)"
- uses: actions/upload-artifact@v2
with:

View File

@@ -1,75 +0,0 @@
name: IE Python Checks
on:
workflow_dispatch:
push:
paths:
- 'inference-engine/ie_bridges/python/**'
pull_request:
paths:
- 'inference-engine/ie_bridges/python/**'
jobs:
linters:
runs-on: ubuntu-18.04
steps:
- name: Code checkout
uses: actions/checkout@v2
with:
submodules: recursive
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.6'
- name: Install dependencies
run: python -m pip install -r inference-engine/ie_bridges/python/requirements_dev.txt
- name: Run Flake on samples
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/sample
- name: Create code style diff for samples
if: failure()
run: |
python -m black -l 160 -S ./
git diff > samples_diff.diff
working-directory: inference-engine/ie_bridges/python/sample
- uses: actions/upload-artifact@v2
if: failure()
with:
name: samples_diff
path: samples_diff.diff
- name: Run Flake on src
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/src
- name: Create code style diff for Python src
if: failure()
run: |
python -m black -l 160 -S ./
git diff > src_diff.diff
working-directory: inference-engine/ie_bridges/python/src
- uses: actions/upload-artifact@v2
if: failure()
with:
name: src_diff
path: src_diff.diff
- name: Run Flake on wheel
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/wheel
- name: Create code style diff for wheel
if: failure()
run: |
python -m black -l 160 -S ./
git diff > wheel_diff.diff
working-directory: inference-engine/ie_bridges/python/wheel
- uses: actions/upload-artifact@v2
if: failure()
with:
name: wheel_diff
path: wheel_diff.diff
- name: Run MyPy
run: python -m mypy ./ --config-file ./setup.cfg
working-directory: inference-engine/ie_bridges/python
- name: Run Bandit
run: python -m bandit -r ./ -f screen
working-directory: inference-engine/ie_bridges/python

4
.gitmodules vendored
View File

@@ -18,7 +18,3 @@
path = thirdparty/xbyak
url = https://github.com/herumi/xbyak.git
ignore = dirty
[submodule "thirdparty/zlib/zlib"]
path = thirdparty/zlib/zlib
url = https://github.com/madler/zlib.git
ignore = dirty

View File

@@ -30,7 +30,7 @@ message (STATUS "CMAKE_C_COMPILER_ID ................... " ${CMAKE_C_COMPILER_ID
message (STATUS "CMAKE_BUILD_TYPE ...................... " ${CMAKE_BUILD_TYPE})
# remove file with exported developer targets to force its regeneration
file(REMOVE "${CMAKE_BINARY_DIR}/InferenceEngineTargets.cmake")
file(REMOVE "${CMAKE_BINARY_DIR}/inference_engine_targets.cmake")
foreach(component IN LISTS openvino_export_components)
file(REMOVE "${CMAKE_BINARY_DIR}/${component}_dev_targets.cmake")
unset(${component} CACHE)
@@ -52,6 +52,7 @@ function(build_ngraph)
else ()
ngraph_set(NGRAPH_ADDRESS_SANITIZER OFF)
endif ()
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE OFF)
if(ENABLE_TESTS AND NOT ANDROID)
ngraph_set(NGRAPH_UNIT_TEST_ENABLE ON)
@@ -84,12 +85,6 @@ function(build_ngraph)
ngraph_set(NGRAPH_THREAD_SANITIZER_ENABLE OFF)
endif()
if(ENABLE_PYTHON)
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE ON)
else()
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE OFF)
endif()
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
ie_add_compiler_flags(-Wno-error=uninitialized -Wno-error=literal-conversion)
elseif(UNIX)
@@ -107,10 +102,10 @@ function(build_ngraph)
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION_RELEASE ON)
endif()
ie_cpack_add_component(ngraph REQUIRED)
ie_cpack_add_component(ngraph_dev REQUIRED DEPENDS ngraph)
ie_cpack_add_component(ngraph)
set(SDL_cmake_included ON)
set(NGRAPH_COMPONENT_PREFIX "deployment_tools/ngraph/")
add_subdirectory(ngraph)
set(NGRAPH_LIBRARIES ngraph PARENT_SCOPE)
set(NGRAPH_REF_LIBRARIES ngraph_reference PARENT_SCOPE)
@@ -154,7 +149,6 @@ build_ngraph()
add_subdirectory(inference-engine)
add_subdirectory(model-optimizer)
add_subdirectory(docs)
add_subdirectory(tools)
#
# Shellcheck
@@ -203,7 +197,7 @@ endif()
# install files for demo
ie_cpack_add_component(demo_scripts DEPENDS core)
ie_cpack_add_component(demo_scripts REQUIRED DEPENDS core)
if(UNIX)
install(DIRECTORY scripts/demo/

1
Jenkinsfile vendored
View File

@@ -13,7 +13,6 @@ properties([
name: 'library_version')
])
])
loadOpenVinoLibrary {
entrypoint(this)
}

View File

@@ -1,5 +1,5 @@
# OpenVINO™ Toolkit
[![Stable release](https://img.shields.io/badge/version-2021.3-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2021.3)
[![Stable release](https://img.shields.io/badge/version-2021.2-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2021.2)
[![Apache License Version 2.0](https://img.shields.io/badge/license-Apache_2.0-green.svg)](LICENSE)
![GitHub branch checks state](https://img.shields.io/github/checks-status/openvinotoolkit/openvino/master?label=GitHub%20checks)
![Azure DevOps builds (branch)](https://img.shields.io/azure-devops/build/openvinoci/b2bab62f-ab2f-4871-a538-86ea1be7d20f/13?label=Public%20CI)
@@ -42,9 +42,8 @@ Please report questions, issues and suggestions using:
---
\* Other names and brands may be claimed as the property of others.
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
[Open Model Zoo]:https://github.com/opencv/open_model_zoo
[Inference Engine]:https://software.intel.com/en-us/articles/OpenVINO-InferEngine
[Model Optimizer]:https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer
[nGraph]:https://docs.openvinotoolkit.org/latest/openvino_docs_nGraph_DG_DevGuide.html
[tag on StackOverflow]:https://stackoverflow.com/search?q=%23openvino
[nGraph]:https://docs.openvinotoolkit.org/latest/openvino_docs_nGraph_DG_DevGuide.html

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -12,7 +12,7 @@
# 2) ${TBBROOT} with IE own version of TBBConfig.cmake (actual for TBB < 2017.7)
#
# Path to IE own version of TBBConfig.cmake old TBB version without cmake config.
## Path to IE own version of TBBConfig.cmake old TBB version without cmake config.
if(APPLE)
set(IE_OWN_TBB_CONFIG tbb/mac)
elseif(UNIX)
@@ -27,7 +27,6 @@ find_package(TBB
CONFIG
PATHS ${TBBROOT}/cmake
${IEDevScripts_DIR}/${IE_OWN_TBB_CONFIG}
NO_CMAKE_FIND_ROOT_PATH
NO_DEFAULT_PATH
)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -14,13 +14,7 @@ set(CMAKE_MODULE_PATH "${IEDevScripts_DIR}")
function(set_ci_build_number)
set(repo_root "${CMAKE_SOURCE_DIR}")
include(version)
foreach(var CI_BUILD_NUMBER IE_VERSION
IE_VERSION_MAJOR IE_VERSION_MINOR IE_VERSION_PATCH)
if(NOT DEFINED ${var})
message(FATAL_ERROR "${var} version component is not defined")
endif()
set(${var} "${${var}}" PARENT_SCOPE)
endforeach()
set(CI_BUILD_NUMBER "${CI_BUILD_NUMBER}" PARENT_SCOPE)
endfunction()
set_ci_build_number()
@@ -53,6 +47,9 @@ function(set_temp_directory temp_variable source_tree_dir)
if (DEFINED ENV{DL_SDK_TEMP} AND NOT $ENV{DL_SDK_TEMP} STREQUAL "")
message(STATUS "DL_SDK_TEMP environment is set : $ENV{DL_SDK_TEMP}")
file(TO_CMAKE_PATH $ENV{DL_SDK_TEMP} temp)
if (ENABLE_ALTERNATIVE_TEMP)
set(ALTERNATIVE_PATH ${source_tree_dir}/temp)
endif()
else ()
set(temp ${source_tree_dir}/temp)
endif()
@@ -205,9 +202,6 @@ endif()
# General flags
set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package(Threads REQUIRED)
include(compile_flags/sdl)
include(compile_flags/os_flags)
include(compile_flags/sanitizer)
@@ -223,7 +217,6 @@ include(api_validator/api_validator)
include(vs_version/vs_version)
include(plugins/plugins)
include(add_ie_target)
include(CMakePackageConfigHelpers)
if(ENABLE_FUZZING)
enable_fuzzing()
@@ -232,8 +225,6 @@ endif()
# macro to mark target as conditionally compiled
function(ie_mark_target_as_cc TARGET_NAME)
target_link_libraries(${TARGET_NAME} PRIVATE openvino::conditional_compilation)
if(NOT (SELECTIVE_BUILD STREQUAL "ON"))
return()
endif()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -31,7 +31,6 @@ addIeTarget(
function(addIeTarget)
set(options
ADD_CPPLINT # Enables code style checks for the target
ADD_CLANG_FORMAT # Enables code style checks for the target
)
set(oneValueRequiredArgs
TYPE # type of target, SHARED|STATIC|EXECUTABLE. SHARED and STATIC correspond to add_library, EXECUTABLE to add_executable
@@ -120,10 +119,6 @@ function(addIeTarget)
# code style
add_cpplint_target(${ARG_NAME}_cpplint FOR_TARGETS ${ARG_NAME})
endif()
if (ARG_ADD_CLANG_FORMAT)
# code style
add_clang_format_target(${ARG_NAME}_clang FOR_TARGETS ${ARG_NAME})
endif()
if (ARG_DEVELOPER_PACKAGE)
# developer package
openvino_developer_export_targets(COMPONENT ${ARG_DEVELOPER_PACKAGE}
@@ -133,6 +128,7 @@ function(addIeTarget)
# Provide default compile pdb name equal to target name
set_target_properties(${ARG_NAME} PROPERTIES COMPILE_PDB_NAME ${ARG_NAME})
endif()
endfunction()
#[[

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,7 +1,6 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if (ENABLE_CLANG_FORMAT)
set(CLANG_FORMAT_FILENAME clang-format-9 clang-format)
find_program(CLANG_FORMAT NAMES ${CLANG_FORMAT_FILENAME} PATHS ENV PATH)
@@ -17,13 +16,10 @@ if (ENABLE_CLANG_FORMAT)
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
else()
message(WARNING "Supported clang-format version is not found!")
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
if(ENABLE_CLANG_FORMAT AND NOT TARGET clang_format_check_all)
if(ENABLE_CLANG_FORMAT)
add_custom_target(clang_format_check_all)
add_custom_target(clang_format_fix_all)
set_target_properties(clang_format_check_all clang_format_fix_all

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -7,11 +7,12 @@ file(REMOVE "${OUTPUT_FILE}")
execute_process(COMMAND ${CLANG_FORMAT} -style=file -output-replacements-xml ${INPUT_FILE}
OUTPUT_VARIABLE STYLE_CHECK_RESULT
)
# Display the cpplint output to console (to parse it form IDE)
message("${output}")
file(WRITE "${OUTPUT_FILE}" "${STYLE_CHECK_RESULT}")
if(NOT SKIP_RETURN_CODE)
if("${STYLE_CHECK_RESULT}" MATCHES ".*<replacement .*")
message(FATAL_ERROR "[clang-format] Code style check failed for: ${INPUT_FILE}")
message(FATAL_ERROR "[clang-format] Code style check failed for : ${INPUT_FILE}")
endif()
endif()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -38,10 +38,9 @@ endmacro()
macro(ie_deprecated_no_errors)
if(WIN32)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(ie_c_cxx_deprecated_no_errors "/Qdiag-warning:1478,1786")
set(ie_c_cxx_deprecated "/Qdiag-warning:1478,1786")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# show 4996 only for /w4
set(ie_c_cxx_deprecated_no_errors "/w44996")
set(ie_c_cxx_deprecated "/wd4996")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
@@ -184,6 +183,7 @@ endfunction()
#
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(THREADS_PREFER_PTHREAD_FLAG ON)
# to allows to override CMAKE_CXX_STANDARD from command line
if(NOT DEFINED CMAKE_CXX_STANDARD)
@@ -211,16 +211,6 @@ set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_C_VISIBILITY_PRESET hidden)
set(CMAKE_VISIBILITY_INLINES_HIDDEN ON)
function(ie_python_minimal_api target)
# pybind11 uses a lot of API which is not a part of minimal python API subset
# Ref 1: https://docs.python.org/3.11/c-api/stable.html
# Ref 2: https://github.com/pybind/pybind11/issues/1755
# target_compile_definitions(${target} PRIVATE Py_LIMITED_API=0x03090000)
# if(NOT CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# target_compile_options(${target} PRIVATE "-Wno-unused-variable")
# endif()
endfunction()
if(WIN32)
ie_add_compiler_flags(-D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS)
ie_add_compiler_flags(/EHsc) # no asynchronous structured exception handling

View File

@@ -1,9 +1,17 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(CheckCXXCompilerFlag)
if (ENABLE_SANITIZER OR ENABLE_THREAD_SANITIZER)
# This is workaround for https://gitlab.kitware.com/cmake/cmake/-/issues/16609.
# It ensures pthread is searched without ASAN linking.
# Line bellow must be before adding -fsanitize=address or -fsanitize=thread to
# build options for the trick to work.
find_package(Threads REQUIRED)
endif()
if (ENABLE_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=address -fno-omit-frame-pointer")
CHECK_CXX_COMPILER_FLAG("-fsanitize-recover=address" SANITIZE_RECOVER_SUPPORTED)
@@ -12,9 +20,6 @@ if (ENABLE_SANITIZER)
endif()
set(SANITIZER_LINKER_FLAGS "-fsanitize=address")
# prevent unloading libraries at runtime, so sanitizer can resolve their symbols
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=gold")
elseif(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$" AND NOT WIN32)
@@ -33,8 +38,6 @@ endif()
if (ENABLE_THREAD_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=thread -fno-omit-frame-pointer")
set(SANITIZER_LINKER_FLAGS "-fsanitize=thread")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$" AND NOT WIN32)
if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=lld")

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,7 +1,6 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(OV_COVERAGE_GCDA_DATA_DIRECTORY "${CMAKE_BINARY_DIR}")
if(NOT TARGET ie_coverage_clean)
add_custom_target(ie_coverage_clean)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,17 +1,17 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_CPPLINT)
find_package(PythonInterp 3 QUIET)
find_package(Python3 COMPONENTS Interpreter)
if(NOT PYTHONINTERP_FOUND)
if(NOT Python3_Interpreter_FOUND)
message(WARNING "Python3 interpreter was not found (required for cpplint check)")
set(ENABLE_CPPLINT OFF)
endif()
endif()
if(ENABLE_CPPLINT AND NOT TARGET cpplint_all)
if(ENABLE_CPPLINT)
add_custom_target(cpplint_all ALL)
set_target_properties(cpplint_all PROPERTIES FOLDER cpplint)
set(CPPLINT_ALL_OUTPUT_FILES "" CACHE INTERNAL "All cpplint output files")
@@ -68,7 +68,6 @@ function(add_cpplint_target TARGET_NAME)
"${output_file}"
COMMAND
"${CMAKE_COMMAND}"
-D "PYTHON_EXECUTABLE=${PYTHON_EXECUTABLE}"
-D "CPPLINT_SCRIPT=${IEDevScripts_DIR}/cpplint/cpplint.py"
-D "INPUT_FILE=${source_file}"
-D "OUTPUT_FILE=${output_file}"

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -25,7 +25,7 @@ set(FILTER "${DEFAULT_FILTER}${CUSTOM_FILTER}")
execute_process(
COMMAND
"${PYTHON_EXECUTABLE}"
python3
"${CPPLINT_SCRIPT}"
"--linelength=160"
"--counting=detailed"

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -10,7 +10,7 @@
# XARCH_FUNC_NAME -- name of function to dispatch
# XARCH_NAMESPACES -- full namespace used to keep ODR
# XARCH_DISP_FILE -- dispatcher file name to generate
# XARCH_SET -- set of ARCH supported by dispatcher. semicolon-delimited
# XARCH_SET -- set of ARCH supported by dispatcher. space delimited
#
# =================================================================
@@ -24,6 +24,7 @@ function(_generate_dispatcher)
_find_signature_in_file(${XARCH_API_HEADER} ${XARCH_FUNC_NAME} SIGNATURE)
_generate_call_line_from_signature("${SIGNATURE}" CALL_LINE)
string(REPLACE " " ";" XARCH_SET "${XARCH_SET}")
string(REPLACE "::" ";" XARCH_NAMESPACES "${XARCH_NAMESPACES}")
list(GET XARCH_NAMESPACES -1 XARCH_CURRENT_NAMESPACE)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -13,10 +13,10 @@ set(_ACCEPTED_ARCHS_AVX512F "^(ANY|SSE42|AVX|AVX2|AVX512F)$")
## Arch specific definitions
set(_DEFINE_ANY "")
set(_DEFINE_SSE42 "HAVE_SSE42" ${_DEFINE_ANY})
set(_DEFINE_AVX "HAVE_AVX" ${_DEFINE_SSE42})
set(_DEFINE_AVX2 "HAVE_AVX2" ${_DEFINE_AVX})
set(_DEFINE_AVX512F "HAVE_AVX512F" ${_DEFINE_AVX2})
set(_DEFINE_SSE42 "-DHAVE_SSE42" ${_DEFINE_ANY})
set(_DEFINE_AVX "-DHAVE_AVX" ${_DEFINE_SSE42})
set(_DEFINE_AVX2 "-DHAVE_AVX2" ${_DEFINE_AVX})
set(_DEFINE_AVX512F "-DHAVE_AVX512F" ${_DEFINE_AVX2})
## Arch specific compile options
ie_avx512_optimization_flags(_FLAGS_AVX512F)
@@ -117,25 +117,21 @@ function(_clone_source_to_target TARGET SOURCE ARCH_SET)
${CMAKE_CURRENT_SOURCE_DIR}/${SOURCE}
${CMAKE_CURRENT_BINARY_DIR}/${ARCH_SOURCE}
DEPENDS ${SOURCE}
VERBATIM
)
set_property(SOURCE ${ARCH_SOURCE} APPEND_STRING PROPERTY COMPILE_FLAGS
" ${_FLAGS_${_arch}}")
set_property(SOURCE ${ARCH_SOURCE} APPEND PROPERTY COMPILE_DEFINITIONS
set(_ARCH_SPECIFIC_FLAGS
${_DEFINE_${_arch}}
"XARCH=${_arch}" ## to replace XARCH with direct ARCH name
${_FLAGS_${_arch}}
"-DXARCH=${_arch}" ## to replace XARCH with direct ARCH name
"-I${CMAKE_CURRENT_SOURCE_DIR}/${ARCH_INCLUDE_DIR}" ## To make valid #include "some.hpp"
)
## To make `#include "some.hpp"` valid
set_property(SOURCE ${ARCH_SOURCE} APPEND PROPERTY INCLUDE_DIRECTORIES
"${CMAKE_CURRENT_SOURCE_DIR}/${ARCH_INCLUDE_DIR}")
_add_source_compile_flags(${ARCH_SOURCE} ${_ARCH_SPECIFIC_FLAGS})
list(APPEND _ARCH_SOURCES ${ARCH_SOURCE})
endforeach()
target_sources(${TARGET} PRIVATE ${_ARCH_SOURCES})
_add_source_to_target(${TARGET} ${_ARCH_SOURCES})
endfunction()
@@ -150,27 +146,26 @@ function(_add_dispatcher_to_target TARGET HEADER FUNC_NAME NAMESPACE ARCH_SET)
set(DISPATCHER_SOURCE "cross-compiled/${DISPATCHER_NAME}_disp.cpp")
set(DISPATCHER_OPT_HOLDER "cross-compiled/${DISPATCHER_NAME}_holder.txt")
set(_GEN_ARGS_LIST
-DXARCH_FUNC_NAME="${X_NAME}"
-DXARCH_NAMESPACES="${NAMESPACE}"
-DXARCH_API_HEADER="${CMAKE_CURRENT_SOURCE_DIR}/${HEADER}"
-DXARCH_DISP_FILE="${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_SOURCE}"
-DXARCH_SET="${ARCH_SET}"
)
configure_file(${DISPATCHER_GEN_OPTIONS_HOLDER} ${DISPATCHER_OPT_HOLDER})
add_custom_command(
OUTPUT ${DISPATCHER_SOURCE}
COMMAND ${CMAKE_COMMAND}
-D "XARCH_FUNC_NAME=${X_NAME}"
-D "XARCH_NAMESPACES=${NAMESPACE}"
-D "XARCH_API_HEADER=${CMAKE_CURRENT_SOURCE_DIR}/${HEADER}"
-D "XARCH_DISP_FILE=${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_SOURCE}"
-D "XARCH_SET=${ARCH_SET}"
COMMAND ${CMAKE_COMMAND} ${_GEN_ARGS_LIST}
-P ${DISPATCHER_GEN_SCRIPT}
DEPENDS ${HEADER}
${DISPATCHER_GEN_SCRIPT}
${CMAKE_CURRENT_BINARY_DIR}/${DISPATCHER_OPT_HOLDER} ## Just to make run dependency on args value
VERBATIM
)
set_property(SOURCE ${DISPATCHER_SOURCE} APPEND PROPERTY INCLUDE_DIRECTORIES
"${CMAKE_CURRENT_SOURCE_DIR}/${DISPATCHER_INCLUDE_DIR}")
target_sources(${TARGET} PRIVATE ${DISPATCHER_SOURCE})
_add_source_compile_flags(${DISPATCHER_SOURCE} "-I${DISPATCHER_INCLUDE_DIR}")
_add_source_to_target(${TARGET} ${DISPATCHER_SOURCE})
endfunction()
#######################################
@@ -204,3 +199,29 @@ function(_remove_source_from_target TARGET SOURCE_FILE)
PROPERTIES
SOURCES "${ORIGINAL_SOURCES}")
endfunction()
function(_add_source_to_target TARGET)
get_target_property(ORIGINAL_SOURCES ${TARGET} SOURCES)
list(APPEND ORIGINAL_SOURCES ${ARGN})
set_target_properties(${TARGET}
PROPERTIES
SOURCES "${ORIGINAL_SOURCES}")
endfunction()
function(_add_source_compile_flags SOURCE)
get_source_file_property(ORIGINAL_FLAGS ${SOURCE} COMPILE_FLAGS)
## Empty list of COMPILE_FLAGS represented as NOTFOUND
if(NOT ORIGINAL_FLAGS)
set(ORIGINAL_FLAGS "")
endif()
string(REPLACE ";" " " NEW_FLAGS "${ARGN}")
string(APPEND ORIGINAL_FLAGS " " ${NEW_FLAGS})
set_source_files_properties(${SOURCE}
PROPERTIES
COMPILE_FLAGS "${ORIGINAL_FLAGS}")
endfunction()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,10 +1,10 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include (download/download)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256 FILES_TO_EXTRACT)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256)
if (ENVIRONMENT AND (DEFINED ${ENVIRONMENT} OR DEFINED ENV{${ENVIRONMENT}}))
set(HAS_ENV "TRUE")
endif()
@@ -12,9 +12,9 @@ function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHI
if (NOT DEFINED HAS_ENV)
if (ARCHIVE)
#TODO: check whether this is platform specific binary with same name per or it is in common folder
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256})
else()
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256})
endif()
set (${VAR} ${result_path} PARENT_SCOPE)
@@ -53,7 +53,7 @@ endfunction(read_version)
function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
list(REMOVE_AT ARGV 0)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT GITHUB_PULL_REQUEST VERSION_REGEX SHA256 FILES_TO_EXTRACT)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT GITHUB_PULL_REQUEST VERSION_REGEX SHA256)
#unnecessary vars
@@ -116,9 +116,6 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
message(FATAL_ERROR "SHA is not specified for: " ${NAME_OF_CMAKE_VAR})
endif()
if (NOT DEFINED FILES_TO_EXTRACT)
set (FILES_TO_EXTRACT FALSE)
endif()
#for each dependency type have to do separate things
if (ARCHIVE_WIN OR ARCHIVE_LIN OR ARCHIVE_MAC OR ARCHIVE_ANDROID OR ARCHIVE OR ARCHIVE_UNIFIED)
@@ -126,7 +123,7 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
message(FATAL_ERROR "TARGET_PATH should be defined for every dependency")
endif()
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256} ${FILES_TO_EXTRACT})
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256})
set(${NAME_OF_CMAKE_VAR} ${RESULT} PARENT_SCOPE)
if (VERSION_REGEX)
GetNameAndUrlToDownload(archive RELATIVE_URL ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID})

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -42,26 +42,25 @@ function (DownloadAndExtractPlatformSpecific
unpacked_path
result_path
folder
sha256
files_to_extract)
sha256)
GetNameAndUrlToDownload(archive_name RELATIVE_URL ${archive_name_unified} ${archive_name_win} ${archive_name_lin} ${archive_name_mac} ${archive_name_android} )
if (NOT archive_name OR NOT RELATIVE_URL)
return()
endif()
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256})
set (${result_path} ${result_path2} PARENT_SCOPE)
endfunction(DownloadAndExtractPlatformSpecific)
#download from common folder
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256 files_to_extract)
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256)
set (RELATIVE_URL "${archive_name}")
set(fattal TRUE)
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256})
if (NOT ${result})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder} ${sha256} ${files_to_extract})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder})
endif()
set (${result_path} ${result_path2} PARENT_SCOPE)
@@ -69,7 +68,7 @@ function (DownloadAndExtract component archive_name unpacked_path result_path fo
endfunction(DownloadAndExtract)
function (DownloadAndExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256 files_to_extract)
function (DownloadAndExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256)
set (status "ON")
DownloadAndCheck(${URL} ${archive_path} ${fattal} result1 ${sha256})
if ("${result1}" STREQUAL "ARCHIVE_DOWNLOAD_FAIL")
@@ -84,17 +83,17 @@ function (DownloadAndExtractInternal URL archive_path unpacked_path folder fatt
endif()
if("${status}" STREQUAL "ON")
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result ${files_to_extract})
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result)
endif()
set (${resultExt} ${status} PARENT_SCOPE)
endfunction(DownloadAndExtractInternal)
function (ExtractWithVersion URL archive_path unpacked_path folder result files_to_extract)
function (ExtractWithVersion URL archive_path unpacked_path folder result)
debug_message("ExtractWithVersion : ${archive_path} : ${unpacked_path} : ${folder} : ${files_to_extract}")
extract(${archive_path} ${unpacked_path} ${folder} ${files_to_extract} status)
debug_message("ExtractWithVersion : ${archive_path} : ${unpacked_path}")
extract(${archive_path} ${unpacked_path} ${folder} status)
#dont need archive actually after unpacking
file(REMOVE_RECURSE "${archive_path}")
if (${status})
@@ -107,20 +106,20 @@ function (ExtractWithVersion URL archive_path unpacked_path folder result files_
set (${result} ${status} PARENT_SCOPE)
endfunction (ExtractWithVersion)
function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256 files_to_extract)
function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal resultExt sha256)
debug_message("checking wether archive downloaded : ${archive_path}")
set (downloadStatus "NOTOK")
if (NOT EXISTS ${archive_path})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
else()
if (ENABLE_UNSAFE_LOCATIONS)
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result ${files_to_extract})
ExtractWithVersion(${URL} ${archive_path} ${unpacked_path} ${folder} result)
if(NOT ${result})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
@@ -128,7 +127,7 @@ function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal
else()
debug_message("archive found on FS : ${archive_path}, however we cannot check it's checksum and think that it is invalid")
file(REMOVE_RECURSE "${archive_path}")
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256} ${files_to_extract})
DownloadAndExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} result ${sha256})
if (${result})
set (downloadStatus "OK")
endif()
@@ -148,7 +147,7 @@ endfunction(DownloadOrExtractInternal)
file(REMOVE ${CMAKE_BINARY_DIR}/dependencies_64.txt)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256 files_to_extract)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256)
set (archive_path ${TEMP}/download/${archive_name})
set (status "ON")
@@ -170,12 +169,12 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
debug_message ("checking that unpacked directory exist: ${unpacked_path}")
if (NOT EXISTS ${unpacked_path})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256} ${files_to_extract})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256})
else(NOT EXISTS ${unpacked_path})
#path exists, so we would like to check what was unpacked version
set (version_file ${unpacked_path}/ie_dependency.info)
if (NOT EXISTS ${version_file})
if (NOT EXISTS ${version_file} AND NOT ${ENABLE_ALTERNATIVE_TEMP})
clean_message(FATAL_ERROR "error: Dependency doesn't contain version file. Please select actions: \n"
"if you are not sure about your FS dependency - remove it : \n"
"\trm -rf ${unpacked_path}\n"
@@ -201,12 +200,12 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
string(REPLACE ${TEMP} ${ALTERNATIVE_PATH} archive_path ${archive_path})
debug_message("dependency different: use local path for fetching updated version: ${alternative_path}")
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} ${result_path} ${folder} ${fattal} ${resultExt} FALSE ${sha256} ${files_to_extract})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} ${result_path} ${folder} ${fattal} ${resultExt} FALSE ${sha256})
else()
debug_message("dependency updated: download it again")
file(REMOVE_RECURSE "${unpacked_path}")
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256} ${files_to_extract})
DownloadOrExtractInternal(${URL} ${archive_path} ${unpacked_path} ${folder} ${fattal} status ${sha256})
endif()
endif ()
endif()

View File

@@ -1,8 +1,8 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
function (extract archive_path unpacked_path folder files_to_extract result)
function (extract archive_path unpacked_path folder result)
# Slurped from a generated extract-TARGET.cmake file.
get_filename_component(unpacked_dir ${unpacked_path} DIRECTORY)
@@ -24,24 +24,20 @@ function (extract archive_path unpacked_path folder files_to_extract result)
message("unpacked_dir= ${unpacked_dir}")
endif()
string(REGEX REPLACE ";" " " list_files_to_extract "${${files_to_extract}}")
message(STATUS "extracting... [tar xfz] ${list_files_to_extract}")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz ${archive_path} ${${files_to_extract}}
message(STATUS "extracting... [tar xfz]")
execute_process(COMMAND ${CMAKE_COMMAND} -E tar xfz ${archive_path}
WORKING_DIRECTORY ${unpacked_dir}
RESULT_VARIABLE rv
ERROR_VARIABLE err)
if (NOT (rv EQUAL 0))
if (NOT (err STREQUAL ""))
message(STATUS "error: extract of '${archive_path}' failed: ${err}")
#invalid archive
file(REMOVE_RECURSE "${unpacked_path}")
file(REMOVE_RECURSE "${archive_path}")
set(${result} 0 PARENT_SCOPE)
else()
if (NOT (err STREQUAL ""))
message(STATUS "${err}")
endif()
set(${result} 1 PARENT_SCOPE)
endif()
endfunction (extract)
endfunction (extract)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -50,12 +50,14 @@ if(NOT DEFINED ENABLE_CPPLINT_REPORT)
ie_dependent_option (ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF "ENABLE_CPPLINT" OFF)
endif()
ie_dependent_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ON "UNIX;NOT ANDROID" OFF)
ie_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ON)
ie_option (VERBOSE_BUILD "shows extra information about build" OFF)
ie_option (ENABLE_UNSAFE_LOCATIONS "skip check for MD5 for dependency" OFF)
ie_option (ENABLE_ALTERNATIVE_TEMP "in case of dependency conflict, to avoid modification in master, use local copy of dependency" ON)
ie_dependent_option (ENABLE_FUZZING "instrument build for fuzzing" OFF "CMAKE_CXX_COMPILER_ID MATCHES ^(Apple)?Clang$; NOT WIN32" OFF)
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,7 +1,6 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
# Usage: ie_option(<option_variable> "description" <initial value or boolean expression> [IF <condition>])
include (CMakeDependentOption)
@@ -29,7 +28,6 @@ macro (ie_option_enum variable description value)
list(APPEND IE_OPTIONS ${variable})
set(${variable} ${value} CACHE STRING "${description}")
set_property(CACHE ${variable} PROPERTY STRINGS ${IE_OPTION_ENUM_ALLOWED_VALUES})
endmacro()
function (print_enabled_features)

View File

@@ -1,6 +1,5 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(CPackComponent)
unset(IE_CPACK_COMPONENTS_ALL CACHE)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -27,10 +27,7 @@ endif()
# )
#
function(ie_add_plugin)
set(options
SKIP_INSTALL
ADD_CLANG_FORMAT
)
set(options SKIP_INSTALL)
set(oneValueArgs NAME DEVICE_NAME VERSION_DEFINES_FOR)
set(multiValueArgs SOURCES OBJECT_LIBRARIES CPPLINT_FILTERS)
cmake_parse_arguments(IE_PLUGIN "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
@@ -58,7 +55,7 @@ function(ie_add_plugin)
add_library(${IE_PLUGIN_NAME} MODULE ${input_files})
target_compile_definitions(${IE_PLUGIN_NAME} PRIVATE IMPLEMENT_INFERENCE_ENGINE_PLUGIN)
ie_add_vs_version_file(NAME ${IE_PLUGIN_NAME}
ie_add_vs_version_file(NAME ${TARGET_NAME}
FILEDESCRIPTION "Inference Engine ${IE_PLUGIN_DEVICE_NAME} device plugin library")
if(TARGET IE::inference_engine_plugin_api)
@@ -68,7 +65,7 @@ function(ie_add_plugin)
endif()
if(WIN32)
set_target_properties(${IE_PLUGIN_NAME} PROPERTIES COMPILE_PDB_NAME ${IE_PLUGIN_NAME})
set_target_properties(${IE_PLUGIN_NAME} PROPERTIES COMPILE_PDB_NAME ${TARGET_NAME})
endif()
set(custom_filter "")
@@ -76,27 +73,12 @@ function(ie_add_plugin)
string(CONCAT custom_filter "${custom_filter}" "," "${filter}")
endforeach()
if (IE_PLUGIN_ADD_CLANG_FORMAT)
add_clang_format_target(${IE_PLUGIN_NAME}_clang FOR_TARGETS ${IE_PLUGIN_NAME})
else()
add_cpplint_target(${IE_PLUGIN_NAME}_cpplint FOR_TARGETS ${IE_PLUGIN_NAME} CUSTOM_FILTERS ${custom_filter})
endif()
# check that plugin with such name is not registered
foreach(plugin_entry IN LISTS PLUGIN_FILES)
string(REPLACE ":" ";" plugin_entry "${plugin_entry}")
list(GET plugin_entry -1 library_name)
list(GET plugin_entry 0 plugin_name)
if(plugin_name STREQUAL "${IE_PLUGIN_DEVICE_NAME}" AND
NOT library_name STREQUAL ${IE_PLUGIN_NAME})
message(FATAL_ERROR "${IE_PLUGIN_NAME} and ${library_name} are both registered as ${plugin_name}")
endif()
endforeach()
add_cpplint_target(${IE_PLUGIN_NAME}_cpplint FOR_TARGETS ${IE_PLUGIN_NAME} CUSTOM_FILTERS ${custom_filter})
# append plugin to the list to register
list(APPEND PLUGIN_FILES "${IE_PLUGIN_DEVICE_NAME}:${IE_PLUGIN_NAME}")
list(REMOVE_DUPLICATES PLUGIN_FILES)
set(PLUGIN_FILES "${PLUGIN_FILES}" CACHE INTERNAL "" FORCE)
add_dependencies(ie_plugins ${IE_PLUGIN_NAME})

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
# Target system specific flags
@@ -39,10 +39,10 @@ elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "amd64.*|x86_64.*|AMD64.*")
set(X86_64 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "i686.*|i386.*|x86.*|amd64.*|AMD64.*")
set(X86 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(arm64.*|aarch64.*|AARCH64.*)")
set(AARCH64 ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(arm.*|ARM.*)")
set(ARM ON)
elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "^(aarch64.*|AARCH64.*)")
set(AARCH64 ON)
endif()
# in case of cross-compilation (or -m32) CMAKE_SYSTEM_PROCESSOR is equal to

View File

@@ -1,6 +1,19 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.

View File

@@ -1,6 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.

View File

@@ -1,6 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#===============================================================================
# Copyright 2017-2020 Intel Corporation
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#===============================================================================
# TBB_FOUND should not be set explicitly. It is defined automatically by CMake.
# Handling of TBB_VERSION is in TBBConfigVersion.cmake.

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -26,55 +26,6 @@ function (commitHash VAR)
set (${VAR} ${GIT_COMMIT_HASH} PARENT_SCOPE)
endfunction()
macro(ie_parse_ci_build_number)
if(CI_BUILD_NUMBER MATCHES "^([0-9]+)\.([0-9]+)\.([0-9]+)\-.*")
set(IE_VERSION_MAJOR ${CMAKE_MATCH_1})
set(IE_VERSION_MINOR ${CMAKE_MATCH_2})
set(IE_VERSION_PATCH ${CMAKE_MATCH_3})
set(has_ci_version ON)
else()
set(IE_VERSION_MAJOR 0)
set(IE_VERSION_MINOR 0)
set(IE_VERSION_PATCH 0)
endif()
if(NOT DEFINED repo_root)
message(FATAL_ERROR "repo_root is not defined")
endif()
if(DEFINED IEDevScripts_DIR AND DEFINED IE_MAIN_SOURCE_DIR AND NOT DEFINED custom_build)
set(ie_version_hpp "${IE_MAIN_SOURCE_DIR}/include/ie_version.hpp")
if(NOT EXISTS ${ie_version_hpp})
message(FATAL_ERROR "File ie_version.hpp with IE_VERSION definitions is not found")
endif()
file(STRINGS "${ie_version_hpp}" IE_VERSION_PARTS REGEX "#define IE_VERSION_[A-Z]+[ ]+" )
string(REGEX REPLACE ".+IE_VERSION_MAJOR[ ]+([0-9]+).*" "\\1"
IE_VERSION_MAJOR_HPP "${IE_VERSION_PARTS}")
string(REGEX REPLACE ".+IE_VERSION_MINOR[ ]+([0-9]+).*" "\\1"
IE_VERSION_MINOR_HPP "${IE_VERSION_PARTS}")
string(REGEX REPLACE ".+IE_VERSION_PATCH[ ]+([0-9]+).*" "\\1"
IE_VERSION_PATCH_HPP "${IE_VERSION_PARTS}")
foreach(var IE_VERSION_MAJOR IE_VERSION_MINOR IE_VERSION_PATCH)
if(DEFINED ${var} AND NOT ${var} EQUAL ${var}_HPP)
message(FATAL_ERROR "${var} parsed from CI_BUILD_NUMBER (${${var}}) \
and from ie_version.hpp (${${var}_HPP}) are different")
else()
# CI_BUILD_NUMBER is not defined well, take info from ie_verison.hpp as a baseline
set(${var} ${${var}_HPP})
endif()
endforeach()
elseif(has_ci_version)
message(WARNING "IE_MAIN_SOURCE_DIR is not defined. No way to compare versions")
else()
message(WARNING "No way to detect OpenVINO version. Supposing 0.0.0.0")
endif()
set(IE_VERSION "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}")
endmacro()
if (DEFINED ENV{CI_BUILD_NUMBER})
set(CI_BUILD_NUMBER $ENV{CI_BUILD_NUMBER})
else()
@@ -85,11 +36,6 @@ else()
set(CI_BUILD_NUMBER "${custom_build}")
endif()
# provides Inference Engine version
# 1. If CI_BUILD_NUMBER is defined, parses this information
# 2. Otherwise, parses ie_version.hpp
ie_parse_ci_build_number()
function (addVersionDefines FILE)
foreach (VAR ${ARGN})
if (DEFINED ${VAR} AND NOT "${${VAR}}" STREQUAL "")

View File

@@ -1,15 +1,30 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(IE_VS_VER_FILEVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_PRODUCTVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_FILEVERSION_STR "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}.0")
macro(ie_parse_ci_build_number)
if(CI_BUILD_NUMBER MATCHES "^([0-9]+)\.([0-9]+)\.([0-9]+)\-.*")
set(IE_VERSION_MAJOR ${CMAKE_MATCH_1})
set(IE_VERSION_MINOR ${CMAKE_MATCH_2})
set(IE_VERSION_PATCH ${CMAKE_MATCH_3})
set(IE_VS_VER_HAS_VERSION 1)
else()
set(IE_VS_VER_HAS_VERSION 0)
endif()
endmacro()
ie_parse_ci_build_number()
if(IE_VS_VER_HAS_VERSION)
set(IE_VS_VER_FILEVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_PRODUCTVERSION_QUAD "${IE_VERSION_MAJOR},${IE_VERSION_MINOR},${IE_VERSION_PATCH},0")
set(IE_VS_VER_FILEVERSION_STR "${IE_VERSION_MAJOR}.${IE_VERSION_MINOR}.${IE_VERSION_PATCH}.0")
endif()
set(IE_VS_VER_COMPANY_NAME_STR "Intel Corporation")
set(IE_VS_VER_PRODUCTVERSION_STR "${CI_BUILD_NUMBER}")
set(IE_VS_VER_PRODUCTNAME_STR "OpenVINO toolkit")
set(IE_VS_VER_COPYRIGHT_STR "Copyright (C) 2018-2021, Intel Corporation")
set(IE_VS_VER_COPYRIGHT_STR "Copyright (C) 2018-2020, Intel Corporation")
set(IE_VS_VER_COMMENTS_STR "https://docs.openvinotoolkit.org/")
#
@@ -36,11 +51,6 @@ function(ie_add_vs_version_file)
message(FATAL_ERROR "${VS_VER_NAME} must define a target")
endif()
get_target_property(target_type ${VS_VER_NAME} TYPE)
if(NOT target_type MATCHES "^(SHARED|MODULE)_LIBRARY$")
message(FATAL_ERROR "ie_add_vs_version_file can work only with dynamic libraries")
endif()
macro(_vs_ver_update_variable name)
if(VS_VER_NAME AND DEFINED IE_${VS_VER_NAME}_VS_VER_${name})
set(IE_VS_VER_${name} "${IE_${VS_VER_NAME}_VS_VER_${name}}")

View File

@@ -1,8 +1,10 @@
#include <winver.h>
VS_VERSION_INFO VERSIONINFO
#if @IE_VS_VER_HAS_VERSION@
FILEVERSION @IE_VS_VER_FILEVERSION_QUAD@
PRODUCTVERSION @IE_VS_VER_PRODUCTVERSION_QUAD@
#endif
FILEFLAGSMASK VS_FFI_FILEFLAGSMASK
#ifdef _DEBUG
FILEFLAGS 1
@@ -19,7 +21,9 @@ BEGIN
BEGIN
VALUE "CompanyName", "@IE_VS_VER_COMPANY_NAME_STR@\0"
VALUE "FileDescription", "@IE_VS_VER_FILEDESCRIPTION_STR@\0"
#if @IE_VS_VER_HAS_VERSION@
VALUE "FileVersion", "@IE_VS_VER_FILEVERSION_STR@\0"
#endif
VALUE "InternalName", "@IE_VS_VER_INTERNALNAME_STR@\0"
VALUE "LegalCopyright", "@IE_VS_VER_COPYRIGHT_STR@\0"
VALUE "OriginalFilename", "@IE_VS_VER_ORIGINALFILENAME_STR@\0"

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,24 +1,22 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
ie_dependent_option (ENABLE_MKL_DNN "MKL-DNN plugin for inference engine" ON "X86_64" OFF)
if(X86_64)
set(ENABLE_MKL_DNN_DEFAULT ON)
else()
set(ENABLE_MKL_DNN_DEFAULT OFF)
endif()
ie_option (ENABLE_MKL_DNN "MKL-DNN plugin for inference engine" ${ENABLE_MKL_DNN_DEFAULT})
ie_option (ENABLE_TESTS "unit, behavior and functional tests" OFF)
ie_option (ENABLE_STRICT_DEPENDENCIES "Skip configuring \"convinient\" dependencies for efficient parallel builds" ON)
ie_dependent_option (ENABLE_CLDNN "clDnn based plugin for inference engine" ON "X86_64;NOT APPLE;NOT MINGW;NOT WINDOWS_STORE;NOT WINDOWS_PHONE" OFF)
ie_option (ENABLE_PROFILING_ITT "Build with ITT tracing. Optionally configure pre-built ittnotify library though INTEL_VTUNE_DIR variable." OFF)
ie_option_enum(ENABLE_PROFILING_FILTER "Enable or disable ITT counter groups.\
Supported values:\
ALL - enable all ITT counters (default value)\
FIRST_INFERENCE - enable only first inference time counters" ALL
ALLOWED_VALUES ALL FIRST_INFERENCE)
ie_option (ENABLE_PROFILING_FIRST_INFERENCE "Build with ITT tracing of first inference time." ON)
ie_option (ENABLE_DOCS "Build docs using Doxygen" OFF)
ie_option(ENABLE_TEMPLATE_PLUGIN "Register template plugin into plugins.xml" OFF)
@@ -29,21 +27,6 @@ Usage: -DSELECTIVE_BUILD=ON -DSELECTIVE_BUILD_STAT=/path/*.csv" OFF
ie_option(ENABLE_ERROR_HIGHLIGHT "Highlight errors and warnings during compile time" OFF)
# Try to find python3
find_package(PythonLibs 3 QUIET)
ie_dependent_option (ENABLE_PYTHON "enables ie python bridge build" OFF "PYTHONLIBS_FOUND" OFF)
find_package(PythonInterp 3 QUIET)
ie_dependent_option (ENABLE_DOCS "Build docs using Doxygen" OFF "PYTHONINTERP_FOUND" OFF)
#
# enable or disable output from NGRAPH_DEBUG statements
#
if(NGRAPH_DEBUG_ENABLE)
add_definitions(-DNGRAPH_DEBUG_ENABLE)
endif()
#
# Process options
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -19,3 +19,6 @@ endmacro()
# need libusb 32-bits version
_set_if_not_defined(ENABLE_VPU OFF)
# fix conversion from uint64_t / int64_t to size_t
_set_if_not_defined(NGRAPH_ONNX_IMPORT_ENABLE OFF)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,25 +0,0 @@
BasedOnStyle: Google
IndentWidth: 4
UseTab: Never
Language: Cpp
Standard: Cpp11
AccessModifierOffset: -4
AlignConsecutiveMacros: true
AllowAllArgumentsOnNextLine: false
AllowAllParametersOfDeclarationOnNextLine: false
AllowShortFunctionsOnASingleLine: Empty
AllowShortIfStatementsOnASingleLine: Never
AllowShortLambdasOnASingleLine: Empty
AllowShortLoopsOnASingleLine: false
AlwaysBreakBeforeMultilineStrings: false
ColumnLimit: 160
# Specialize this comment pragma in order to avoid changes in SEA copyrights
CommentPragmas: '^#'
DerivePointerAlignment: false
FixNamespaceComments: true
IndentCaseLabels: false
IndentPPDirectives: BeforeHash
SpaceBeforeCpp11BracedList: true
SpaceBeforeCtorInitializerColon: false

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2021 Intel Corporation
# Copyright (C) 2018-2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -52,11 +52,19 @@ set(GST_DOCS_DIR "" CACHE PATH "Path to gst-video-analytics documentation")
function(build_docs)
find_package(Doxygen REQUIRED dot)
find_package(PythonInterp 3 REQUIRED)
find_package(LATEX REQUIRED)
find_package(Python3 COMPONENTS Interpreter)
find_package(LATEX)
if(NOT DOXYGEN_FOUND)
message(FATAL_ERROR "Doxygen is required to build the documentation")
endif()
if(NOT Python3_FOUND)
message(FATAL_ERROR "Python3 is required to build the documentation")
endif()
execute_process(
COMMAND ${PYTHON_EXECUTABLE} -m pip show lxml
COMMAND ${Python3_EXECUTABLE} -m pip show lxml
RESULT_VARIABLE PIP_EXIT_CODE
OUTPUT_QUIET
)
@@ -65,6 +73,10 @@ function(build_docs)
message(FATAL_ERROR "lxml package is not installed. Please use \"pip install lxml\".")
endif()
if(NOT LATEX_FOUND)
message(FATAL_ERROR "LATEX is required to build the documentation")
endif()
set(DOCS_BUILD_DIR "${CMAKE_CURRENT_BINARY_DIR}")
set(DOXYGEN_DIR "${OpenVINO_MAIN_SOURCE_DIR}/docs/doxygen")
set(IE_SOURCE_DIR "${OpenVINO_MAIN_SOURCE_DIR}/inference-engine")
@@ -205,7 +217,7 @@ function(build_docs)
add_custom_command(TARGET py_api
PRE_BUILD
COMMAND ${PYTHON_EXECUTABLE} ${PYX_FILTER} ${PYTHON_API_OUT}
COMMAND ${Python3_EXECUTABLE} ${PYX_FILTER} ${PYTHON_API_OUT}
COMMENT "Pre-process Python API")
# Preprocess docs
@@ -305,8 +317,8 @@ function(build_docs)
add_custom_command(TARGET preprocess_docs
PRE_BUILD
${commands}
COMMAND ${PYTHON_EXECUTABLE} ${DOXY_LAYOUT_SCRIPT} --openvino ${OPENVINO_LAYOUT_BUILD}
COMMAND ${PYTHON_EXECUTABLE} ${DOXY_MD_FILTER} ${DOCS_BUILD_DIR}
COMMAND ${Python3_EXECUTABLE} ${DOXY_LAYOUT_SCRIPT} --openvino ${OPENVINO_LAYOUT_BUILD}
COMMAND ${Python3_EXECUTABLE} ${DOXY_MD_FILTER} ${DOCS_BUILD_DIR}
COMMENT "Pre-process markdown and image links")
# IE dev guide and C++ API
@@ -341,7 +353,7 @@ function(build_docs)
add_custom_command(TARGET openvino_docs
POST_BUILD
COMMAND ${PYTHON_EXECUTABLE} ${DOXY_LOG_SCRIPT} --log "${DOCS_BUILD_DIR}/ie_docs.log"
COMMAND ${Python3_EXECUTABLE} ${DOXY_LOG_SCRIPT} --log "${DOCS_BUILD_DIR}/ie_docs.log"
--include_omz $<BOOL:${OMZ_DOCS_DIR}>
--include_wb $<BOOL:${WORKBENCH_DOCS_DIR}>
--include_pot $<BOOL:${POT_DOCS_DIR}>
@@ -353,7 +365,7 @@ function(build_docs)
if(EXISTS "${LINKCHECKER_PY}")
add_custom_target(docs_check
COMMAND ${PYTHON_EXECUTABLE} "${LINKCHECKER_PY}" -v "${DOCS_BUILD_DIR}/html/"
COMMAND ${Python3_EXECUTABLE} "${LINKCHECKER_PY}" -v "${DOCS_BUILD_DIR}/html/"
COMMENT "Check links in generated documentation"
WORKING_DIRECTORY "${DOCS_BUILD_DIR}"
VERBATIM)

View File

@@ -51,45 +51,65 @@ To see the operations that are supported by each device plugin for the Inference
### Custom Operation Support for the Model Optimizer
Model Optimizer model conversion pipeline is described in detail in "Model Conversion Pipeline" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). It is best to read that article first for a better understanding of the following material.
Model Optimizer model conversion pipeline is described in details in "Model Conversion Pipeline" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
It is recommended to read that article first for a better understanding of the following material.
Model Optimizer provides an extensions mechanism to support new operations and implement custom model transformations to generate optimized IR. This mechanism is described in the "Model Optimizer Extensions" section of
Model Optimizer provides extensions mechanism to support new operations and implement custom model transformations to
generate optimized IR. This mechanism is described in the "Model Optimizer Extensions" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
Two types of the Model Optimizer extensions should be implemented to support custom operations, at a minimum:
1. Operation class for a new operation. This class stores information about the operation, its attributes, shape inference function, attributes to be saved to an IR and some others internally used attributes. Refer to the "Model Optimizer Operation" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.
Two types of the Model Optimizer extensions should be implemented to support custom operation at minimum:
1. Operation class for a new operation. This class stores information about the operation, its attributes, shape
inference function, attributes to be saved to an IR and some others internally used attributes. Refer to the
"Model Optimizer Operation" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement it.
2. Operation attributes extractor. The extractor is responsible for parsing framework-specific representation of the
operation and uses corresponding operation class to update graph node attributes with necessary attributes of the
operation. Refer to the "Operation Extractor" section of
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement it.
operation. Refer to the "Operation Extractor" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement it.
> **NOTE:** In some cases you may need to implement some transformation to support the operation. This topic is covered in the "Graph Transformation Extensions" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
> **NOTE:** In some cases you may need to implement some transformation to support the operation. This topic is covered
> in the "Graph Transformation Extensions" section on the
> [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md).
## Custom Operations Extensions for the Inference Engine
Inference Engine provides extensions mechanism to support new operations. This mechanism is described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
Inference Engine provides extensions mechanism to support new operations. This mechanism is described in the
[Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
Each device plugin includes a library of optimized implementations to execute known operations which must be extended to execute a custom operation. The custom operation extension is implemented according to the target device:
Each device plugin includes a library of optimized implementations to execute known operations which must be extended to
execute a custom operation. The custom operation extension is implemented according to the target device:
- Custom Operation CPU Extension
- A compiled shared library (`.so` or `.dll`) needed by the CPU Plugin for executing the custom operation
- A compiled shared library (`.so`, `.dylib` or `.dll`) needed by the CPU Plugin for executing the custom operation
on a CPU. Refer to the [How to Implement Custom CPU Operations](../IE_DG/Extensibility_DG/CPU_Kernel.md) for more
details.
- Custom Operation GPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with an operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the [How to Implement Custom GPU Operations](../IE_DG/Extensibility_DG/GPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the GPU along with a
operation description file (.xml) needed by the GPU Plugin for the custom operation kernel. Refer to the
[How to Implement Custom GPU Operations](../IE_DG/Extensibility_DG/GPU_Kernel.md) for more details.
- Custom Operation VPU Extension
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with an operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to [How to Implement Custom Operations for VPU](../IE_DG/Extensibility_DG/VPU_Kernel.md) for more details.
- OpenCL source code (.cl) for the custom operation kernel that will be compiled to execute on the VPU along with a
operation description file (.xml) needed by the VPU Plugin for the custom operation kernel. Refer to the
[How to Implement Custom Operations for VPU](../IE_DG/Extensibility_DG/VPU_Kernel.md) for more details.
Also, it is necessary to implement nGraph custom operation according to [Custom nGraph Operation](../IE_DG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
operation and correctly infer output tensor shape and type.
Also, it is necessary to implement nGraph custom operation according to the
[Custom nGraph Operation](../IE_DG/Extensibility_DG/AddingNGraphOps.md) so the Inference Engine can read an IR with this
operation and correctly infer output tensors shape and type.
## Enabling Magnetic Resonance Image Reconstruction Model
This chapter provides step-by-step instructions on how to enable the magnetic resonance image reconstruction model implemented in the [repository](https://github.com/rmsouza01/Hybrid-CS-Model-MRI/) using a custom operation on CPU. The example is prepared for a model generated from the repository with hash `2ede2f96161ce70dcdc922371fe6b6b254aafcc8`.
This chapter provides a step-by-step instruction on how to enable the magnetic resonance image reconstruction model
implemented in the [repository](https://github.com/rmsouza01/Hybrid-CS-Model-MRI/) using a custom operation on CPU. The
example is prepared for a model generated from the repository with hash `2ede2f96161ce70dcdc922371fe6b6b254aafcc8`.
### Download and Convert the Model to a Frozen TensorFlow\* Model Format
The original pre-trained model is provided in the hdf5 format which is not supported by OpenVINO directly and needs to be converted to TensorFlow\* frozen model format first.
The original pre-trained model is provided in the hdf5 format which is not supported by OpenVINO directly and needs to
be converted to TensorFlow\* frozen model format first.
1. Download repository `https://github.com/rmsouza01/Hybrid-CS-Model-MRI`:<br>
1. Download repository `https://github.com/rmsouza01/Hybrid-CS-Model-MRI`:<br
```bash
git clone https://github.com/rmsouza01/Hybrid-CS-Model-MRI
git checkout 2ede2f96161ce70dcdc922371fe6b6b254aafcc8
@@ -139,8 +159,6 @@ for more details and command line parameters used for the model conversion.
```bash
./<MO_INSTALL_DIR>/mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1
```
> **NOTE:** This conversion guide is applicable for the 2021.3 release of OpenVINO and that starting from 2021.4
> the OpenVINO supports this model out of the box.
Model Optimizer produces the following error:
```bash
@@ -213,11 +231,15 @@ model. The implementation of the Model Optimizer operation should be saved to `m
The attribute `inverse` is a flag specifying type of the FFT to apply: forward or inverse.
See the "Model Optimizer Operation" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for detailed instructions on how to implement the operation.
See the "Model Optimizer Operation" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for the
detailed instruction on how to implement the operation.
Now it is necessary to implement extractor for the "IFFT2D" operation according to the
"Operation Extractor" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). The
following snippet provides two extractors: one for "IFFT2D", another one for "FFT2D", however only on of them is used in this example. The implementation should be saved to the file `mo_extensions/front/tf/FFT_ext.py`.
"Operation Extractor" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md). The
following snippet provides two extractors: one for "IFFT2D", another one for "FFT2D", however only on of them is used
in this example. The implementation should be saved to the file `mo_extensions/front/tf/FFT_ext.py`.
@snippet FFT_ext.py fft_ext:extractor
@@ -233,7 +255,8 @@ consumed with the "Complex" operation to produce a tensor of complex numbers. Th
operations can be removed so the "FFT" operation will get a real value tensor encoding complex numbers. To achieve this
we implement the front phase transformation which searches for a pattern of two "StridedSlice" operations with specific
attributes producing data to "Complex" operation and removes it from the graph. Refer to the
"Pattern-Defined Front Phase Transformations" section of [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for more
"Pattern-Defined Front Phase Transformations" section on the
[Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md) for more
information on how this type of transformation works. The code snippet should be saved to the file
`mo_extensions/front/tf/Complex.py`.
@@ -261,7 +284,7 @@ Now it is possible to convert the model using the following command line:
./<MO_INSTALL_DIR>/mo.py --input_model <PATH_TO_MODEL>/wnet_20.pb -b 1 --extensions mo_extensions/
```
The sub-graph corresponding to the originally non-supported one is depicted in the image below:
The sub-graph corresponding to the originally non-supported one is depicted on the image below:
![Converted sub-graph](img/converted_subgraph.png)
@@ -270,7 +293,8 @@ The sub-graph corresponding to the originally non-supported one is depicted in t
### Inference Engine Extension Implementation
Now it is necessary to implement the extension for the CPU plugin with operation "FFT" introduced previously. The code
below is based on the template extension described in [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
below is based on the template extension described on the
[Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md).
#### CMake Build File
The first step is to create a CMake configuration file which builds the extension. The content of the "CMakeLists.txt"
@@ -310,17 +334,17 @@ The last step is to create an extension library "extension.cpp" and "extension.h
operation for the CPU plugin. The code of the library is described in the [Extension Library](../IE_DG/Extensibility_DG/Extension.md).
### Building and Running the Custom Extension
To build the extension, run the following:<br>
In order to build the extension run the following:<br>
```bash
mkdir build && cd build
source /opt/intel/openvino_2021/bin/setupvars.sh
source /opt/intel/openvino/bin/setupvars.sh
cmake .. -DCMAKE_BUILD_TYPE=Release
make --jobs=$(nproc)
```
The result of this command is a compiled shared library (`.so` or `.dll`). It should be loaded in the
The result of this command is a compiled shared library (`.so`, `.dylib` or `.dll`). It should be loaded in the
application using `Core` class instance method `AddExtension` like this
`core.AddExtension(std::make_shared<Extension>(compiled_library_file_name), "CPU");`.
`core.AddExtension(make_so_pointer<IExtension>(compiled_library_file_name), "CPU");`.
To test that the extension is implemented correctly we can run the "mri_reconstruction_demo.py" with the following content:
@@ -344,7 +368,7 @@ python3 mri_reconstruction_demo.py \
- [Model Optimizer Extensibility](../MO_DG/prepare_model/customize_model_optimizer/Customize_Model_Optimizer.md)
- [Inference Engine Extensibility Mechanism](../IE_DG/Extensibility_DG/Intro.md)
- [Inference Engine Samples Overview](../IE_DG/Samples_Overview.md)
- [Overview of OpenVINO™ Toolkit Pre-Trained Models](@ref omz_models_group_intel)
- [Overview of OpenVINO™ Toolkit Pre-Trained Models](@ref omz_models_intel_index)
- For IoT Libraries and Code Samples see the [Intel® IoT Developer Kit](https://github.com/intel-iot-devkit).
## Converting Models:

View File

@@ -1,5 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [complex:transformation]
import logging as log

View File

@@ -1,5 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [complex_abs:transformation]
import numpy as np

View File

@@ -1,5 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
# ! [fft_ext:extractor]
from ...ops.FFT import FFT

View File

@@ -1,5 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [fft:operation]
from mo.front.common.partial_infer.elemental import copy_shape_infer

View File

@@ -1,5 +1,18 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Copyright (C) 2018-2020 Intel Corporation
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
"""
#! [mri_demo:demo]
import numpy as np

View File

@@ -2,64 +2,6 @@
The sections below contain detailed list of changes made to the Inference Engine API in recent releases.
## 2021.4
### New API
* InferenceEngine::Core::LoadNetwork(modelPath, deviceName, config) simplified API to read and load network in one call
### Deprecated API
**InferenceEngine::Parameter**
* InferenceEngine::Parameter(const std::shared_ptr<ngraph::Variant>&)
* InferenceEngine::Parameter(std::shared_ptr<ngraph::Variant>& var)
* std::shared_ptr<ngraph::Variant> InferenceEngine::Parameter::asVariant() const
* InferenceEngine::Parameter::operator std::shared_ptr<ngraph::Variant>() const
**GPU plugin configuration keys**
* KEY_CLDNN_NV12_TWO_INPUTS GPU plugin option. Use KEY_GPU_NV12_TWO_INPUTS instead
* KEY_CLDNN_PLUGIN_PRIORITY GPU plugin option. Use KEY_GPU_PLUGIN_PRIORITY instead
* KEY_CLDNN_PLUGIN_THROTTLE GPU plugin option. Use KEY_GPU_PLUGIN_THROTTLE instead
* KEY_CLDNN_MEM_POOL GPU plugin option
* KEY_CLDNN_GRAPH_DUMPS_DIR GPU plugin option
* KEY_CLDNN_SOURCES_DUMPS_DIR GPU plugin option
* KEY_DUMP_KERNELS GPU plugin option
* KEY_TUNING_MODE GPU plugin option
* KEY_TUNING_FILE GPU plugin option
**InferenceEngine::IInferRequest**
* IInferRequest interface is deprecated, use InferRequest wrapper:
* Constructor for InferRequest from IInferRequest:: Ptr is deprecated
* Cast operator for InferRequest to IInferRequest shared pointer is deprecated
**InferenceEngine::ICNNNetwork**
* ICNNNetwork interface is deprecated by means of deprecation of all its methods, use CNNNetwork wrapper
* CNNNetwork methods working with ICNNNetwork are deprecated:
* Cast to ICNNNetwork shared pointer
* Cast to reference to ICNNNetwork interface
* Constructor from ICNNNetwork shared pointer
**InferenceEngine::IExecutableNetwork**
* IExecutableNetwork is deprecated, use ExecutableNetwork wrappers:
* Constructor of ExecutableNetwork from IExecutableNetwork shared pointer is deprecated
* The following ExecutableNetwork methods are deprecated:
* ExecutableNetwork::reset
* Cast operator to IExecutableNetwork shared pointer
* ExecutableNetwork::CreateInferRequestPtr - use ExecutableNetwork::CreateInferRequest instead
**Extensions API**
* InferenceEngine::make_so_pointer which is used to create Extensions library is replaced by std::make_shared<Extension>(..)
* InferenceEngine::IExtension::Release is deprecated with no replacement
* Use IE_DEFINE_EXTENSION_CREATE_FUNCTION helper macro instead of explicit declaration of CreateExtension function, which create extension.
**Other changes**
* Version::ApiVersion structure is deprecated, Inference Engine does not have API version anymore
* LowLatency - use lowLatency2 instead
* CONFIG_KEY(DUMP_EXEC_GRAPH_AS_DOT) - use InferenceEngine::ExecutableNetwork::GetExecGraphInfo::serialize() instead
* Core::ImportNetwork with no device - pass device name explicitly.
* details::InferenceEngineException - use InferenceEngine::Exception and its derivatives instead.
## 2021.3
### New API
@@ -573,7 +515,7 @@ The sections below contain detailed list of changes made to the Inference Engine
* DLIA_CONFIG_KEY(ENABLE_STREAMING) config key
### Removed API
* InferenceEngine::EltwiseLayer::Select from InferenceEngine::EltwiseLayer::eOperation enumeration
## 2019 R2
@@ -622,7 +564,7 @@ The sections below contain detailed list of changes made to the Inference Engine
* DLIA_CONFIG_KEY(IO_TRANSFORMATIONS_NATIVE) config key
* DLIA_CONFIG_KEY(DUMP_SUPPORTED_LAYERS_INFORMATION) config key
* GNA_CONFIG_VALUE(SW_FP32) config value for GNA_CONFIG_KEY(DEVICE_MODE) key
* MULTI_CONFIG_KEY(DEVICE_PRIORITIES) config key for `MULTI` device
* MULTI_CONFIG_KEY(DEVICE_PRIORITIES) config key for `MULTI` device
* InferenceEngine::CNNNetReader::ReadNetwork(const std::wstring &filepath) new method
* InferenceEngine::CNNNetReader::ReadWeights(const std::wstring &filepath) new method
* InferenceEngine::ExecutableNetwork::ExecutableNetwork(IExecutableNetwork::Ptr actual, InferenceEnginePluginPtr plg) constructor with additional `plg` parameter
@@ -638,7 +580,7 @@ The sections below contain detailed list of changes made to the Inference Engine
* InferenceEngine::EltwiseLayer::Logical_NOT, InferenceEngine::EltwiseLayer::Mean, InferenceEngine::EltwiseLayer::Select extensions to InferenceEngine::EltwiseLayer::eOperation enumeration
* InferenceEngine::OneHotLayer new class
* InferenceEngine::SelectLayer new class
* InferenceEngine::BroadcastLayer new class
* InferenceEngine::BroadcastLayer new class
* InferenceEngine::MathLayer new class
* InferenceEngine::ReduceLayer new class
* InferenceEngine::TopKLayer new class

View File

@@ -15,7 +15,7 @@ Preserving the exponent bits keeps BF16 to the same range as the FP32 (~1e-38 to
Truncated mantissa leads to occasionally less precision, but according to [investigations](https://cloud.google.com/blog/products/ai-machine-learning/bfloat16-the-secret-to-high-performance-on-cloud-tpus), neural networks are more sensitive to the size of the exponent than the mantissa size. Also, in lots of models, precision is needed close to zero but not so much at the maximum range.
Another useful feature of BF16 is possibility to encode INT8 in BF16 without loss of accuracy, because INT8 range completely fits in BF16 mantissa field. It reduces data flow in conversion from INT8 input image data to BF16 directly without intermediate representation in FP32, or in combination of [INT8 inference](Int8Inference.md) and BF16 layers.
See the ["BFLOAT16 Hardware Numerics Definition" white paper"](https://software.intel.com/sites/default/files/managed/40/8b/bf16-hardware-numerics-definition-white-paper.pdf) for more bfloat16 format details.
See the [Intel's site](https://software.intel.com/sites/default/files/managed/40/8b/bf16-hardware-numerics-definition-white-paper.pdf) for more bfloat16 format details.
There are two ways to check if CPU device can support bfloat16 computations for models:
1. Query the instruction set via system `lscpu | grep avx512_bf16` or `cat /proc/cpuinfo | grep avx512_bf16`.

Some files were not shown because too many files have changed in this diff Show More