Compare commits

..

167 Commits

Author SHA1 Message Date
Alexey Suhov
6c4462759e [README.md] change latest release to 2021.4.2 2021-11-16 20:55:28 +03:00
Andrey Zaytsev
a28d93b2bf Feature/azaytsev/doc updates gna 2021 4 2 (#8567)
* Various doc changes

* Reformatted C++/Pythob sections. Updated with info from PR8490

* additional fix

* Gemini Lake replaced with Elkhart Lake

* Fixed links in IGs, Added 12th Gen
2021-11-16 12:36:17 +00:00
Nikolay Tyukaev
400e0657cd doc script changes (#8568)
* fix openvino-sphinx-theme

* add linkcheck target

* fix

* change version

* add doxygen-xfail.txt

* fix

* AA

* fix

* fix

* fix

* fix

* fix
2021-11-13 00:17:59 +03:00
Aleksandr Korolev
e2a469a345 [IE][VPU] heap-use-after-free fix v2 (#8509)
* [VPU] use-after-free fix v2
2021-11-12 02:31:47 +03:00
Mikhail Ryzhov
de85202b29 Added Import/Export functions to C_API #8353 (#8458)
* initial commit

# Conflicts:
#	inference-engine/src/inference_engine/src/ie_core.cpp
#	inference-engine/thirdparty/ade

* [GNA] added export/import for c_api (need fixes)

# Conflicts:
#	inference-engine/thirdparty/ade

* [GNA] import/export with file

# Conflicts:
#	inference-engine/thirdparty/ade

* [GNA] fixed tests and function with memory

# Conflicts:
#	inference-engine/thirdparty/ade

* deleted unnecessary testing changes

# Conflicts:
#	inference-engine/src/inference_engine/src/ie_core.cpp
#	inference-engine/thirdparty/ade

* fixed bug with const

# Conflicts:
#	inference-engine/thirdparty/ade

* fixed review comments

# Conflicts:
#	inference-engine/thirdparty/ade

* [GNA] changed testing model

# Conflicts:
#	inference-engine/thirdparty/ade

* Put memory buffer to istream directly

# Conflicts:
#	inference-engine/thirdparty/ade

* Replaced blob with array of bytes

# Conflicts:
#	inference-engine/thirdparty/ade

* Make config optional

# Conflicts:
#	inference-engine/thirdparty/ade

* Reverted load_network change

# Conflicts:
#	inference-engine/thirdparty/ade

* Fixed tests with null parameters

# Conflicts:
#	inference-engine/thirdparty/ade

* Changed path to models

# Conflicts:
#	inference-engine/thirdparty/ade

* Made config optional for other methods

# Conflicts:
#	inference-engine/thirdparty/ade

* Update ie_c_api.h
# Conflicts:
#	inference-engine/thirdparty/ade

* Merge branch 'c_api' of https://github.com/a-noskov/openvino into c_api

# Conflicts:
#	inference-engine/thirdparty/ade

* Changed export signature

# Conflicts:
#	inference-engine/thirdparty/ade

* Reverted import changes(not needed)

# Conflicts:
#	inference-engine/thirdparty/ade

* fixed test

# Conflicts:
#	inference-engine/thirdparty/ade

* Replaced stringbuf by streambuf

# Conflicts:
#	inference-engine/thirdparty/ade

* fixed code style issues

# Conflicts:
#	inference-engine/thirdparty/ade

* Added condition for the tests

# Conflicts:
#	inference-engine/thirdparty/ade

* Enabled skipping of tests

# Conflicts:
#	inference-engine/thirdparty/ade

Rebase commit

Revert "Rebase commit"

This reverts commit aa717c076a9901637bc525ba6ef9960e012073e9.

* Changed test model path

* skipped tests for old gna lib v1

Co-authored-by: Andrey <andrey.noskov@intel.com>
Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-11-10 14:13:13 +03:00
Daria Mityagina
5b283ee6d4 [IE][VPU][XLink] - Fix Mac build (#8479)
* Fix Mac build - XLink_sem_trywait problems - trywait

* Fix Mac build - XLink_sem_trywait problems

* Fix Mac build - XLink_sem_trywait - small changes

Co-authored-by: Maksim Doronin <maksim.doronin@intel.com>
2021-11-10 10:31:58 +03:00
Sergey Lyubimtsev
ca36a3335f Set scikit-image~=0.18.3 for python versions >=3.7 (#7910) (#8444)
* Set scikit-image~=0.18.3 for python versions >=3.7

* Set scikit-image>=0.17.2 (0.17.2 is not available for >=3.7)

(cherry picked from commit a10f40d6d4)
2021-11-09 19:27:53 +03:00
Aleksandr Korolev
b1985ff5fd [VPU] heap-use-after-free fix (#8466)
* [VPU]heap-use-after-free fix

* Empty commit
2021-11-09 18:14:10 +03:00
Alexander Zhogov
dd0f845a18 Azure CI: exclude docs from triggers (#8475) 2021-11-09 16:29:11 +03:00
Andrey Zaytsev
3a0d7a28ed Feature/azaytsev/doc updates gna (#8445)
* Various doc changes

* Intel® Celeron® J4125 Processor is added to the list
2021-11-08 15:47:24 +03:00
Egor Duplensky
3cdebfcb7c Revert "[CPU] Fix mixing VEX and non-VEX instructions (#7238)" (#8381)
Because few issues were detected on AVX (unknown instruction)
and AVX2 (accuracy) platforms.
Reverting for now.
2021-11-03 13:51:26 +03:00
Vladislav Volkov
6fdc6b4c16 Reverted "Migrating to the new tbbbind library version 2.5 (#8270)" (#8362) 2021-11-03 13:51:10 +03:00
Aleksandr Korolev
7096c01127 [VPU] vpu scale update (#8180)
* [VPU] vpu scale update

* fix ubuntu20 build
2021-11-03 12:58:41 +03:00
Denis Orlov
2d28a05421 [GNA] Fix comments in GNA public header and speech sample (#8373) 2021-11-03 10:45:01 +03:00
Daria Mityagina
e2ace1da40 [ICV][XLink] - Port changes (#8203)
* [ICV][XLink] - Port - Add mutex for event queues

* [ICV][XLink] - Port - XLinkReleaseSpecificPacket implementation

* [ICV][XLink] - Port - Exposed the configuration of maximum resources for the XLink streams at application build level

* [ICV][XLink] - Port - Expose the setting of the xLink threads priority to the application level

* [ICV][XLink] - Port - Changed calling of sem_wait() to account for EINTR error code

* [ICV][XLink] - Port - Kw issues fixes for R17

* [ICV][XLink] - Port - USB_VSC: moved HW init and datapump startup to upper layer

* [ICV][XLink] - Port - Additional mutexes for XLinkDispatcher

* [ICV][XLink] - Port - Add XLink read/write timeouts

* [ICV][XLink] - precommit failures fix

* [ICV][XLink] - win_semaphore

* [ICV][XLink] - review fixes

* [ICV][XLink] - review fixes

* [ICV][XLink] - review fixes

* [ICV][XLink] - review fixes - Write -> Read

* [ICV][XLink] - review fixes - Windows issues - sem_trywait

* [ICV][XLink] - review fixes - Windows issues - sem_trywait - space

* [ICV][XLink] - review fixes - Windows issues - sem_trywait

* [ICV][XLink] - removed one lock

* [ICV][XLink] - small change - else

* [ICV][XLink] - small change - remove unlock

* [ICV][XLink] - errors fixed - new semaphore

* [ICV][XLink] - review comments

* [ICV][XLink] - review comments

* [ICV][XLink] - review comments

* [ICV][XLink] - remove pthread_attr_destroy(&attr);

* [ICV][XLink] - fw update

* [ICV][XLink] - fw update
2021-11-02 14:25:41 +03:00
Nikolay Tyukaev
f5e9b699ce doc updates (#8268)
* Various doc changes

* theme changes

* remove font-family (#8211)

* fix  css

* Update uninstalling-openvino.md

* fix css

* fix

* Fixes for Installation Guides

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
Co-authored-by: kblaszczak-intel <karol.blaszczak@intel.com>
2021-11-02 11:27:03 +03:00
Alexey Lebedev
5ec50585df fix sporadic test (#8366) 2021-11-02 10:05:45 +03:00
Anastasia Kuporosova
279961332b [IE Python API] Add set_config for executable_network (#7796) (#8351)
* [IE Python API] Add set_config for executable_network

* fix test

* fix test comment
2021-11-02 03:13:04 +03:00
Elizaveta Lobanova
0d5f86ae33 [GNA] Fixed Log segments calculation for internal algorithm (#8318) 2021-11-01 16:57:03 +03:00
Yury Gorbachev
9b9f5e3eee Update get_started_scripts.md (#8338) 2021-11-01 11:22:11 +03:00
Vladislav Volkov
c885b1933d Migrating to the new tbbbind library version 2.5 (#8270) 2021-10-29 15:47:08 +03:00
Anastasia Kazantaeva
1b6dc06cf0 Update message 2021.4.2 (#8130)
* Upgrated MO message

* fix
2021-10-28 22:41:46 +03:00
Daria Mityagina
a3afa57bd9 [ICV][XLink] - Port some changes (#8284)
* [ICV][XLink] - mvnc_data 0xd0

* [ICV][XLink] - fw update

* [ICV][XLink] - Port - XLinkOpenStream now allocates streamId only on the host side

* [ICV][XLink] - Port - enum updated
2021-10-28 22:14:46 +03:00
kblaszczak-intel
ad1efb7f16 remove font-family (#8211) 2021-10-27 12:49:15 +03:00
Elizaveta Lobanova
7eed1d910b [GNA] Limit activation output scale factor (#8106) 2021-10-26 15:38:12 +03:00
Elizaveta Lobanova
2278100258 [GNA] Fixed segments creation for multi segment pwl functions (#8044) (#8165)
* [GNA] Fixed extreme segments creation for tanh pwl

* [GNA] Tests added

* [GNA] Added tests for different activations

* [GNA] Comments apply
2021-10-25 11:57:40 +03:00
Nikolay Tyukaev
361dec7362 Docs to Sphinx (#8151)
* docs to sphinx

* Update GPU.md

* Update CPU.md

* Update AUTO.md

* Update performance_int8_vs_fp32.md

* update

* update md

* updates

* disable doc ci

* disable ci

* fix index.rst

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-10-24 17:43:00 +03:00
Alexey Lebedev
d7a917d92c [IE PYTHON] fix gil (#8145)
* remove nogil

* Add test
2021-10-22 16:13:52 +03:00
Roman Lyamin
326b8ad009 [GPU] Fixed matmul handling for some shapes (#6642) (#8103)
Co-authored-by: Vladimir Paramuzov <vladimir.paramuzov@intel.com>
2021-10-20 09:34:32 +03:00
Vladislav Golubev
cc6321f6d3 [LPT] MatmulTransformation: canBeTransformed fix (#7816) 2021-10-19 23:50:20 +03:00
Elizaveta Lobanova
5c2a39009e [GNA] Fixed insertion of delayed copy error (#7944) (#8037)
* [GNA] Fixed error with delayed copy insertion

* [GNA] Added test
2021-10-18 15:34:55 +03:00
Polina Brzezinskaya
a0b4394423 [VPU] Changed calling of sem_wait() to account for EINTR error code (#7815)
Changed calling of sem_wait() to account for EINTR error code
2021-10-15 16:18:14 +03:00
Dmitry Pigasin
85378ce176 Add --ignore-installed to the downloader requirement installation (#8025) 2021-10-15 10:42:23 +03:00
Gleb Kazantaev
e2fd335b70 Fix FrameworkNodeAttr Deserialization (#7943)
* Fix FrameworkNodeAttr Deserialization

* Fix
2021-10-14 13:27:50 +03:00
Elizaveta Lobanova
c5175063c8 [GNA] More precized calculation of Log pwl and pwl with sf < 1 (#7884) (#7982)
* [GNA] More precized calculation of Log pwl and pwl with sf < 1

* [GNA] Added tests

* [GNA] Types correction

* [GNA] Added comment for absolute threshold calculation in the test
2021-10-14 12:38:08 +03:00
Krzysztof Bruniecki
5fb5057cb6 [GNA] Fix pooling output backward compatibility (#7889)
* [GNA] Fix pooling output backward compatibility

* Apply review
2021-10-12 11:43:12 +03:00
Roman Lyamin
485bea73b1 [GPU] Fixed code gen with custom locale (#7548) 2021-10-08 12:54:34 +03:00
Maxim Vafin
105e67573a [MO] Fix axes in FusedBatchNorm -> MVN transformation (#7679)
* [MO] Fix axes in FusedBatchNorm -> MVN transformation

* Improve testing

* Fix eps_mode attribute
2021-10-08 11:21:50 +03:00
Alexander Zhogov
2282b0ee5c Change IE version to 2021.4.2 2021-10-05 13:47:30 +03:00
Anton Dudchenko
f00dc87a92 [IE][VPU] Fix execTimeMcs for VPU (#7442) (#7712)
When serializing execGraph, milliseconds were actually written to the execTimeMcs field. It's just cherry-pick commit from master branch
2021-09-29 21:16:33 +03:00
Mikhail Kozlov
e932a2eda5 Add virtual inheritance to shared tests (#7649) 2021-09-28 17:20:04 +03:00
Andrey Zaytsev
bd989cee67 [doc] Improved description of OpenVINO workflow (#6128) (#7703)
* Update get_started_dl_workbench.md

POToolkit => POTool

* Update QuantizedNetworks.md

POToolkit => POTool

* Moving POT to optimizations section

Moving POT to optimizations section

* structure

* links

* step 1

* step 1 diagram

* step 2

* typo

* step 4

* step 4 diagram

* step 3

* minor corrections

* Applied comments

* Applied comments from Tatiana

* Applied comments from Alex
# Conflicts:
#	docs/index.md
#	inference-engine/thirdparty/ade

Co-authored-by: Maksim Proshin <mvproshin@gmail.com>
2021-09-28 15:32:19 +03:00
Krzysztof Bruniecki
bdf72bcd88 [GNA] Fix KEY_EXEC_TARGET (#7671)
* Use Gna2DeviceCreateForExport when GNA_EXEC_TARGET is != detected

* Update detected GNA device version field in GNA Device helper

   * Use EXEC instead of COMPILE TARGET to append
   CNN Legacy enforcement (GNA1)

* Apply review
2021-09-28 11:55:28 +03:00
Egor Duplensky
6d634d09a4 [CPU] Fix mixing VEX and non-VEX instructions (#7238)
Quote: The Skylake microarchitecture implements a different state
machine than prior generations to manage the YMM state transition
associated with mixing SSE and AVX instructions.
It no longer saves the entire upper YMM state when executing
an SSE instruction when in “Modified and Unsaved” state,
but saves the upper bits of individual register.
As a result, mixing SSE and AVX instructions will experience
a penalty associated with partial register dependency of
the destination registers being used and additional blend
operation on the upper bits of the destination registers.

Such type of penalties have a huge impact on openvino's and oneDNN's kernels.
Basically the mixing of VEX and non-VEX instructions should be avoided.
2021-09-22 10:46:37 +03:00
Rafal Blaczkowski
05c641ff7c Update azure onnx linux 2021/4 (#7534)
* update azure pipeline

* save
2021-09-16 14:50:10 +03:00
Vitaliy Urusovskij
05a768e357 Bump up MKLDNN version to get fix of c26453 warning (#7090) 2021-09-16 12:57:39 +03:00
Szymon Irzabek
a0b2200408 Gna 2dconv30 releases (#7387)
* [GNA] Port padding and 2d convolution support from master

* [GNA] Add separate fixes required after porting

* [GNA] Add support for activation without bias in padding and 2d convolution decomposition
2021-09-15 14:43:42 +03:00
Krzysztof Bruniecki
e339afe375 IE: Fix Windows compilation on MSVC (adapt PR: #7206 from master branch) (#7486)
`Win32` may be undefined when MSVC is used, see:
https://docs.microsoft.com/en-us/cpp/preprocessor/predefined-macros?redirectedfrom=MSDN&view=msvc-160

Signed-off-by: Karol Trzcinski <karolx.trzcinski@intel.com>
2021-09-14 17:49:01 +03:00
Evgeny Talanin
7f47108bb3 Add apt update (#7484) 2021-09-13 13:52:21 +03:00
Elizaveta Lobanova
b13bcbcd0d [GNA] Fixed scale factors propagation for Eltwise with very different inputs ranges (#7305) (#7445)
* [GNA] Fix scale factors propogation for Eltwise with very different inputs ranges

* [GNA] Added test

* [GNA] Added exception for scale factor <= 0

* [GNA] Disable tests with integer weights

* [GNA] Added assert for CNNLayer in getScaleFactor()

* [GNA] Added check if scale factor is inf

* [GNA] Fixed legacy tests
2021-09-13 13:01:37 +03:00
Alexey Suhov
c2bfbf29fb [README.md] change latest release to 2021.4.1 2021-09-10 00:08:38 +03:00
Artyom Tugaryov
14e67d8663 Check installed intel-opencl-icd to find version (#7427) 2021-09-08 17:22:18 +03:00
Artyom Tugaryov
d98cb7bdf8 Use dpkg-info to check installed driver (#7426) 2021-09-08 17:17:35 +03:00
Evgeny Talanin
fa18eecfb7 Revert "[IE CLDNN] Fixed code gen with custom locale (#6243)" (#7370)
This reverts commit fadeaecb6d.
2021-09-08 10:44:49 +03:00
Kate Generalova
3aa2f02240 doc: fix docker win links (#7402) 2021-09-07 15:17:14 +03:00
Sergey Lyubimtsev
a4fdc1c947 Fix for missed OMZ public models in openvino-dev package (#6568) (#7324)
(cherry picked from commit b893eae9e7)
2021-09-01 22:24:04 +03:00
Artyom Anokhov
f14f28f32b [Scripts] Detecting newest drivers on ubu20 (#7322)
* install_NEO_OCL_driver: Added detecting current driver via intel-opencl-icd package in case of newest one on ubuntu20. Added removing intel-opencl-icd package for ubuntu.

* install_NEO_OCL_driver: Fixed pattern for parsing driver version for newest drivers
2021-09-01 15:00:24 +03:00
Artyom Anokhov
e3baff25a6 install_NEO_OCL_driver: Added ocl-icd-libopencl1 for ubuntu18 case (#7314) 2021-08-31 19:46:51 +03:00
Anton Voronov
708825b439 [CPU] fixed convolution outputShape in ConvDWConv fusing (#7290) 2021-08-31 15:21:15 +03:00
Mikhail Letavin
91d85a88a1 [GPU] Remove unused copy constructor (#7221) 2021-08-31 14:14:51 +03:00
Denis Orlov
f77c3d7fdc [GNA] Include documentation for GNA3 QoS (#7046)
* [GNA] Include documentation for GNA3 QoS

* Fix according to review

* Fix the driver version
2021-08-31 12:54:05 +03:00
Artyom Anokhov
36f2e63c9c Update install neo driver script with 21.29.20389 (#7244)
* install_NEO_OCL_driver: Added installing 21.29.20389

* install_NEO_OCL_driver: Updated CMDs for installing 21.29.20389 on Ubuntu

* Update scripts/install_dependencies/install_NEO_OCL_driver.sh

Co-authored-by: Kate Generalova <kate.generalova@intel.com>

* Update scripts/install_dependencies/install_NEO_OCL_driver.sh

Co-authored-by: Kate Generalova <kate.generalova@intel.com>

* install_NEO_OCL_driver: Updated getting values from dictionaries

* install_NEO_OCL_driver: Updated descriptions

* install_NEO_OCL_driver: Added more packages to install

* install_NEO_OCL_driver: Added ocl-icd-2.2.12-1.el8.x86_64 for RHEL

* install_NEO_OCL_driver: Updated error message. Replaced `dnf update --refresh` with --refresh option for dnf install. Added software-properties-common in case if user doesn't have apt-add-repository.

* install_NEO_OCL_driver.sh: Fixed syntax

Co-authored-by: Kate Generalova <kate.generalova@intel.com>
2021-08-31 11:45:02 +03:00
Mikhail Ryzhov
d474617d12 [GNA] Fixed import of model with several inputs (#7277) (#7278)
* [GNA] Fixed import of model with several intputs

* Fixed copyright year
2021-08-31 11:09:57 +03:00
Elizaveta Lobanova
64a896c22e [GNA] Fixed accuracy degradation caused by the input quantization restriction (#7260) 2021-08-27 11:48:57 +03:00
Elizaveta Lobanova
2dcd09055f [GNA] Fixed calculation of input scale factor and search of the next layer for FQ (#7246)
* [GNA] Fixed search of the next layer for FQ

* [GNA] Fixed calculation of input scale factor for POT-quantized model in the case if the first layer after input is activation
2021-08-26 15:02:23 +03:00
Andrey Sapozhnikov
1a656f4e44 [GNA] Plugin transition to the library v3.0 (#7241)
* GNA Plugin transition to the library v3.0

* Remove Gna2RequestConfigEnableHardwareConsistency calls

Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
2021-08-26 01:00:07 +03:00
Mikhail Ryzhov
0702b44174 [Samples] Enabled oname for imported models (#7228) 2021-08-25 12:57:27 +03:00
Krzysztof Bruniecki
ce21344585 [GNA] Fixes for GNA 3.0 library (#7180)
* Pass compileTarget to am_intel_dnn

* Enable tests for GNA lib version prefix 3.0

* Fix conv split transform for 2d cnn tests

* Apply review
2021-08-25 10:06:10 +03:00
Ilya Sharikov
3a28ffaf57 Updated path to gflags (#7200) 2021-08-24 13:57:40 +03:00
Anton Dudchenko
f63be649eb Update FW (#7209) 2021-08-24 12:02:23 +03:00
Polina Brzezinskaya
32a9e98437 [VPU] Added ConvertGather7ToGather1 pass to frontend (#7184) 2021-08-24 11:03:23 +03:00
Victor Kuznetsov
b76c903745 add pytest html (#7169) 2021-08-20 16:58:39 +03:00
Mikhail Ryzhov
4714b8edb8 [GNA] Set input scale factors for imported model (#7139) (#7140)
* [GNA] Set custom scale factor for imported model
2021-08-19 14:48:37 +03:00
Andrey Sapozhnikov
47d1f2147a [GNA] Plugin preparation for transition to the library v3 (#7047)
* GNA Plugin transition to the library v3

* Embedded device support

* Revert transition, keep preparation for future transition
2021-08-19 14:46:43 +03:00
Elizaveta Lobanova
61aa366706 [GNA] Fix order of SwapMatMulInput transformations (#7138) 2021-08-19 09:13:55 +03:00
Sergey Shlyapnikov
b16ce268eb [GPU] Fix incorrect fusions indexes for eltwise ref kernel (#7026) 2021-08-18 18:23:19 +03:00
Daria Mityagina
629de56910 [IE][VPU] Firmware update (#7102)
Update FW version from 1717 to 1729
2021-08-18 16:35:18 +03:00
Victor Kuznetsov
dad76527d6 fixed several issues in time tests (#7059)
* Fixed return in run_timetest
* Fixed parse_stats() argument type notation
* Pass message from timetest exe to test error msg to be uploaded to DB
2021-08-18 13:15:58 +03:00
Yury Gaydaychuk
b86ab12f0f [CPU][Release 2021.4.1] Fix roipooling border proposal computation for 2021.4 (#7116)
* roi_pooling handles border props correctly

* fix adapted for old test infrastructure
2021-08-17 23:20:54 +03:00
Yury Gaydaychuk
170e4d2cce [CPU] Interpolate handles inplace child layout (#6961) 2021-08-17 23:10:09 +03:00
Gleb Kazantaev
2a9eec1c3f Enable SplitSqueezeConcatFusion + TransposeSinking in offline transformations (#7087) 2021-08-17 14:39:05 +03:00
Mikhail Ryzhov
92420cd0d5 [Merge] Gna split align convert to conv filter and dependent (#6347 and #5946) (#7083)
* [GNA] Use stride instead of window for pooling (#5946)

* Use pool stride instead of window size where applicable

* Add test for pooling stride not equal to wnd

* Add more tests and cleanup

* Fix SW_FP32 legacy cnn

* [WIP] Refactor CNN1D

* Remove unused (commented out) code

* Add tests

* Gna split align convert to conv filter (#6347)

* Make unaligned split based on Conv instead of Affine

* Dump Gna2Tensor.Data pointer for debugging

* Apply suggestions from code review

* Reuse conv helpers

* Cleanup CNN fields

* Disable weights reducer on ConvolutionFilter
# Conflicts:
#	inference-engine/src/gna_plugin/backend/am_intel_dnn.cpp
#	inference-engine/src/gna_plugin/optimizer/gna_pass_manager.cpp

Co-authored-by: Krzysztof Bruniecki <krzysztof.bruniecki@intel.com>
2021-08-17 10:26:40 +00:00
Elizaveta Lobanova
886254c5b9 Fixed Eltwise split and batch size selection during 2d reshape, transpose bias (#7099)
* [GNA] Transpose bias (#6759)

* transpose bias

* removed bias transpose; added bias validation predicate to pattern

* fixed after review; added handling of the case bias_output_shape.size() == 1 and bias_output_shape.at(0) > 1

* moved bias shape size check to matcher pattern; replaced loop with algorithm

* [GNA] Fixed Eltwise split and batch size selection during 2d reshape (#7042)

* [GNA] Fixed Eltwise split and batch size selection during 2d reshape

* [GNA] Added exception if memory isn't allocated for concat filter

* Added assert for minZeroDimSize

* [GNA] Added unit test for GetAlignedSplitSizes()

Co-authored-by: Dmitrii Khurtin <dmitrii.khurtin@intel.com>
2021-08-17 10:19:07 +03:00
Vladislav Volkov
e75e647ebe [CPU] Memory leaks in gemm module (#7093) 2021-08-17 10:07:59 +03:00
Maksim Kutakov
6e45b62be6 [CPU] Add reorder if the constant memory is not aligned, and isa is SSE (#7095) 2021-08-17 09:01:14 +03:00
Alexander Zhogov
4a70806d10 Azure CI: Remove IB on Windows (#7097) 2021-08-16 21:36:38 +03:00
Dmitrii Khurtin
761c645f14 [GNA] Convolution to matmul (#7029)
* [GNA] Remove transposes around MatMul

* Added tests for transformation HandleTransposesAroundMatMul

* Move IsTransposeSupported function to GNA limitations file

* added TransposeAfterMatmul tests and moved InsertTransposeBeforeMatmul tests to handle_transposes_around_matmul.cpp

* added inifitiry loop checker and memory concat test

* fixed build errors

* changed the conditions for selecting an input of Concat for ScaleFactor calculation when entering an infinite loop

* fixed after review

* s/INSTANTIATE_TEST_SUITE_P/INSTANTIATE_TEST_CASE_P

* .ignore

Co-authored-by: Elizaveta Lobanova <elizaveta.lobanova@intel.com>
2021-08-16 20:41:14 +03:00
Mikhail Ryzhov
e19b3befb7 [GNA] Disabled TransposeReduction (#7011) (#7062)
* [GNA] Disabled TransposeReduction  (#7011)

* Rebase master

* [gna] Fixed export/import precision

* Revert "[gna] Fixed export/import precision"

This reverts commit d381a2e216.

* Rebase master

* [gna] Fixed export/import precision

* Revert "[gna] Fixed export/import precision"

This reverts commit d381a2e216.

* Fixed transposition error

* [GNA] Added tests for conv wrapped to transpose

* Code review fixes

* Fixed copyright year

* Replaced test suite with case
2021-08-16 14:04:43 +03:00
Mikhail Ryzhov
fb3ceb6aa4 [GNA] Fixed issue for concat connection to memory layer (#6985) (#7058)
* [GNA] Fixed issue for concat connection to memory layer (#6985)

* Fix for concat connection to memory layer

* reverted merge files

* Replaced opset8
2021-08-16 11:04:29 +03:00
Paul Youngsoo Ahn
9a8d8440a5 [GPU] Set TBB affinity in load_network (#7049) 2021-08-16 09:14:22 +03:00
Mikhail Ryzhov
543ea75813 [GNA] FQ accuracy fixes (#6924) (#7061) 2021-08-15 22:36:15 +03:00
Maria Kaglinskaya
f03763defe Pruning for 2021.4 release (#6987)
* Fix Pruning for case with INT8 GroupConvolution operation (#6872)

* Added i8, u8 precisions in Gather constant folding (#6195)

Co-authored-by: Gleb Kazantaev <gleb.kazantaev@intel.com>
2021-08-13 18:23:35 +03:00
Elizaveta Lobanova
114ed1cb4b [GNA] Support bias and FQ in SwapInputMatMul transformation (#6996) (#7027) 2021-08-13 12:07:08 +03:00
Mikhail Ryzhov
3117879c54 [GNA] Added support of FQ layers for outputs (#6999)
* [GNA] Added support of FQ layers for outputs (#6905)

* [GNA] Fixed FQ pass for several outputs

* Added tests
2021-08-12 22:54:02 +03:00
Anton Voronov
2f48787fc4 [CPU] gemm inner product - memory access fix (#7009) 2021-08-12 10:15:40 +03:00
Anton Voronov
7848ac7a74 [CPU] fixed conv + dw conv fusing (#6975) 2021-08-12 09:42:06 +03:00
Alexey Lebedev
62f126cdd2 [IE PYTHON] release GIL (#6968)
* Release GIL in load_network

* release gil in infer, wait and get_idle_request_id

* release gil in read_network and IECore.__cinit__

* release GIL in properties

* Release GIL in infer_async

* Add test

* Fix test

* Fix test
2021-08-09 22:34:11 +03:00
Kate Generalova
4d1c358aa3 doc: refactor docker install guide (#6988)
* doc: refactor docker install guide

* doc: refactor docker install guide windows

* Update installing-openvino-docker-linux.md

* Update installing-openvino-docker-windows.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-08-09 17:56:25 +03:00
Maksim Kutakov
bf51d49ad1 [CPU] Get/Set Blob overhead has been eliminated. (#6737) (#6933)
(cherry picked from commit e47a85b427)
2021-08-06 16:59:00 +03:00
Kate Generalova
6d9699681f doc: fix 58710 issue (#6911) 2021-08-06 12:05:33 +03:00
Vladimir Gavrilov
0b248b68dd Fix for summarize_graph.py. (#6904) 2021-08-05 12:09:31 +03:00
Daria Mityagina
d286e0a9ad [IE][VPU] Added support for 2 axis for MVN layer - duplicate (#6748) (#6778)
Co-authored-by: Polina <polina.brzezinskaya@intel.com>
This PR adds support for Person-reidentification-retail model on VPU device by adding support for {2} axis in MVN layer
2021-08-04 10:29:40 +03:00
Tatiana Savina
21ed761569 [59449][DOCS] GPU table layout change (#6789)
* changed argument display

* added br tag to more arguments

* changed argument display in GPU table

* changed more arguments

* changed Quantized_ models display
2021-08-02 20:00:46 +03:00
Dmitrii Khurtin
2639f35543 fixed crash related to loading model with fq and sigmoid (#6866)
* fixed crash related with loading model with fq and sigmoid

* renamed multiple_input.* to multiple_input_fq.*; removed two unnecessary FQ layers from smoke_fq_fusion_with_sigmoid test; moved FQ params to test params

* s/INSTANTIATE_TEST_CASE_P/INSTANTIATE_TEST_SUITE_P/
2021-08-02 12:20:50 +03:00
iliya mironov
1bbd91506b Fixing Split-Concat reverse input channel (#6765)
* Fix split-concut revers input chennel

* Fix comments
2021-07-28 15:47:46 +03:00
Alina Kladieva
9a31a3d821 Bump Jenkins library (#6817) 2021-07-27 13:24:53 +03:00
Evgenya Stepyreva
9acc3dfe68 Zero point optimization (#6628)
* Zero point optimization

* Expand the equality to zero criteria
2021-07-25 13:33:29 +03:00
Alina Kladieva
205c23b382 Commit to bump Jenkins library (#6781) 2021-07-23 17:16:49 +03:00
Maxim Vafin
e48965683b Fix node name issue introduced by #5854 (#6709) (#6736)
* Fix node name issue introduced by #5854
* Compare names in TransposeFuse tests
2021-07-23 03:28:34 +03:00
Ilya Lavrenov
eaa5a22979 Fixed build with cmake 3.21 (#6757) 2021-07-22 19:55:38 +03:00
Vitaliy Urusovskij
bfdd1a199f Develop installation rules for time and stress tests (#6649) (#6738)
* Develop installation rules for time and stress tests (#6649)

* Prepare `install` rule for time_tests

* Prepare `install` rule for stress tests

* Update path to `gflags`
2021-07-22 10:43:15 +03:00
Ilya Churaev
096a92dcb3 Port extension fix (#6725) 2021-07-21 11:05:14 +03:00
Andrey Zaytsev
7a05a12190 Feature/azaytsev/changes from baychub revise demos samples (#6651)
* Edits to MO

Per findings spreadsheet

* macOS changes

per issue spreadsheet

* Fixes from review spreadsheet

Mostly IE_DG fixes

* Consistency changes

* Make doc fixes from last round of review

* Add GSG build-all details

* Fix links to samples and demos pages

* Make MO_DG v2 changes

* Add image view step to classify demo

* Put MO dependency with others

* Edit docs per issues spreadsheet

* Add file to pytorch_specific

* More fixes per spreadsheet

* Prototype sample page

* Add build section

* Update README.md

* Batch download/convert by default

* Add detail to How It Works

* Minor change

* Temporary restored topics

* corrected layout

* Resized

* Added white background into the picture

* fixed link to omz_tools_downloader

* fixed title in the layout

Co-authored-by: baychub <cbay@yahoo.com>
Co-authored-by: baychub <31420038+baychub@users.noreply.github.com>
2021-07-16 11:41:04 +03:00
iliya mironov
5d39724934 Imironov/add missing descriptions for transformations configs releases (#6521)
* Add retinanet doc

* Update doc

* Splite some text to several lines

* Update ie_docs
2021-07-15 18:24:15 +02:00
Dmitry Pigasin
7cec19fe6e Add information about model preparation (#6661) 2021-07-15 10:04:11 +00:00
Andrey Zaytsev
568096ddeb Install guides improvements (#6418) (#6648)
* Install guides improvements

* add bullet to conda  System Requirements

* fix formating

* - Add conda install command for Ubuntu20

- fix typo /tmp

* added conda prerequisites

* Update installing-openvino-apt.md

* Update installing-openvino-conda.md

* Update installing-openvino-conda.md

CentOS 7.6

* Update installing-openvino-apt.md

APT Repository

* Update installing-openvino-conda.md

Added Introduction & notice about runtime package

* Update installing-openvino-conda.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
# Conflicts:
#	docs/install_guides/installing-openvino-conda.md
#	inference-engine/thirdparty/ade

Co-authored-by: Sergey Lyubimtsev <sergey.lyubimtsev@intel.com>
2021-07-14 22:31:16 +03:00
Krzysztof Bruniecki
34bda79333 [GNA] Introduce an option to invoke the QoS feature (#6604)
* cherry picked (#5827)
* Issue 56759
* Introduce HW_WITH_SW_FBACK
* Add unit test for HW_WITH_SW_FBACK
* Enable HW_WITH_SW_FBACK in speech_sample cpp
* Use perf counters to report number of HW delivered frames to the user (eg speech_sample)
* Add GNA frequency for 6/151 CPU family/model
* Update inference-engine/samples/speech_sample/main.cpp
* Co-authored-by: Mikhail Ryzhov <mikhail.ryzhov@intel.com>
2021-07-14 19:14:38 +03:00
Dmitry Pigasin
0da68d9c70 Add a check for existence of output images (#6640) 2021-07-14 12:28:50 +03:00
Pavel Esir
a82011199a [doc] corrections for Yolo V1 and V2 coversion (#6284)
* added corrections for Yolo V1 and V2

* changed order of conversion commands : v1 goes first general last

* aligned line endings

* added commit hash

* clarified about downloaded files location

* added missing <br>

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2021-07-13 18:39:02 +03:00
Edward Shogulin
6bbec510b0 [Runtime] INT8 inference documentation update (#6419)
* [Runtime] INT8 inference documentation update

* [Runtime] INT8 inference documentation: typo was fixed

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Update docs/IE_DG/Int8Inference.md

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Table of Contents was removed

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
2021-07-12 20:15:22 +03:00
Tatiana Savina
90eaa2666a [58825][58829][DOCS] Installation docs changes (#6548)
* install docs fixes

* changed video width

* CMake reference added

* fixed table

* added backtics and table formating

* new table changes

* GPU table changes

* added more backtics and changed table format

* gpu table changes

* Update get_started_dl_workbench.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-07-12 14:52:17 +03:00
Alexander Zhogov
4eb4ee1882 Change IE version to 2021.4.1 2021-07-07 17:11:42 +03:00
Vladimir Paramuzov
fadeaecb6d [IE CLDNN] Fixed code gen with custom locale (#6243) 2021-07-06 10:03:57 +03:00
Anastasiya Ageeva
a5c930eeaa Fixed CVS-58871 (#6519) 2021-07-05 17:51:24 +03:00
Nikolay Tyukaev
5135425bb9 automatic insertion of htmlonly (#6466) 2021-06-30 20:07:31 +03:00
Victor Kuznetsov
204c4ba79a add target branch condition (fix null value) (#6467) 2021-06-30 17:52:55 +03:00
Dmitry Pigasin
640ab71b6a [IE Python Sample Docs 2021.4] Use @ref for validated model links (#6465)
* Use @ref for validated model field

* Add link to googlenet-v1
2021-06-30 16:49:50 +03:00
Dmitry Pigasin
0361fc8e2d Fix other language realization links (#6464) 2021-06-30 12:39:44 +03:00
Alexey Suhov
ccae439943 [README.md] change latest release to 2021.4 2021-06-29 21:49:40 +03:00
Nikolay Tyukaev
5cee8bbf29 omz layout fix (#6450)
* fix layout

* 4
2021-06-29 18:20:39 +00:00
Andrey Zaytsev
a220a0a7af Feature/azaytsev/docs 2021 4 (#6447)
* Added benchmark page changes

* Make the picture smaller

* Added Intel® Iris® Xe MAX Graphics

* Changed the TIP about DL WB

* Added Note on the driver for Intel® Iris® Xe MAX Graphics

* Fixed formatting

* Added the link to Intel® software for general purpose GPU capabilities

* OVSA ovsa_get_started updates

* Fixed link
2021-06-29 20:38:51 +03:00
Andrey Zaytsev
af2fec9a00 Feature/azaytsev/changes from baychub colin q2 (#6437)
* Q2 changes

* Changed Convert_RNNT.md

Co-authored-by: baychub <cbay@yahoo.com>
2021-06-29 18:16:46 +03:00
Nikolay Tyukaev
cca57782ce doc fixes (#6438)
* doc fixes

* doc fix

* doc fix
2021-06-29 13:04:12 +00:00
Andrey Zaytsev
c2e8c3bd92 Feature/azaytsev/mo devguide changes (#6405)
* MO devguide edits

* MO devguide edits

* MO devguide edits

* MO devguide edits

* MO devguide edits

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Experimenting with videos

* Additional edits

* Additional edits

* Updated the workflow diagram

* Minor fix

* Experimenting with videos

* Updated the workflow diagram

* Removed  Prepare_Trained_Model, changed the title for Config_Model_Optimizer

* Rolled back

* Revert "Rolled back"

This reverts commit 6a4a3e1765.

* Revert "Removed  Prepare_Trained_Model, changed the title for Config_Model_Optimizer"

This reverts commit 0810bd534f.

* Fixed ie_docs.xml, Removed  Prepare_Trained_Model, changed the title for Config_Model_Optimizer

* Fixed ie_docs.xml

* Minor fix

* <details> tag issue

* <details> tag issue

* Fix <details> tag issue

* Fix <details> tag issue

* Fix <details> tag issue
2021-06-29 03:59:24 +03:00
Tatiana Savina
4833c8db72 [DOCS]Changed DL WB related docs and tips (#6318)
* changed DL WB related docs and tips

* added two tips to benchmark and changed layout

* changed layout

* changed links

* page title added

* changed tips

* ie layout fixed

* updated diagram and hints

* changed tooltip and ref link

* changet tooltip link

* changed DL WB description

* typo fix
2021-06-28 16:39:15 +03:00
Andrey Zaytsev
3352b483b9 Updated legal info (#6409) 2021-06-28 16:13:25 +03:00
iliya mironov
c40da68a2b Update ylov4 docs (#6270)
* Update ylov4 docs

* Update docs

* Update docs

* Update doc

* Update docs

* Update doc

* Update doc accordeng to review

* Update doc according to review

* Update Convert_YOLO_From_Tensorflow.md

Co-authored-by: Andrey Zaytsev <andrey.zaytsev@intel.com>
2021-06-23 10:01:33 +00:00
Yegor Kruglov
0a959ef8e5 script fix (#6306) 2021-06-23 09:25:26 +03:00
Ivan Tikhonov
cd81789d29 Update LowLatency documentation (#6267)
* update LowLatency documentation

* fix missprint

* add deprecated attribute

* Apply suggestions from code review

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>

* Fix spelling mistakes

* resolve review comments

* refactoring

Co-authored-by: Anastasiya Ageeva <anastasiya.ageeva@intel.com>
2021-06-22 15:36:42 +03:00
Yegor Kruglov
55fb7c6663 [BERT-NER] Document model support (to release branch) (#6239)
* added docs

* update doc

* error correction
2021-06-22 11:46:03 +00:00
Mikhail Nosov
1aa89edbf3 Docs: Model caching feature overview (#6275) 2021-06-22 01:06:54 +03:00
Denis Orlov
6ab6983778 [Doc] Reference POT in documentation for GNA plugin (#6248) 2021-06-21 19:38:27 +03:00
Pavel Esir
fb4d52068b [doc] Update DeepSpeech (#6249)
* corrected output names in DeepSpeech conversion doc

* mo args correction

* changed instruction for DeepSpeech version 0.8.2

* added venv activate; removed redundant ending

* added picture and squashed MO graph input args into one

* Apply suggestions from code review

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>

* applied review comments

Co-authored-by: Tatiana Savina <tatiana.savina@intel.com>
2021-06-21 19:14:42 +03:00
Xie Zhengtian
21514fa9d5 [Doc] Update auto device plugin doc for 2021.4 release (#6271)
* Update auto-device plugin doc

* Add openvino_docs_IE_DG_supported_plugins_AUTO into web page

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>

* Update AUTO.md

Co-authored-by: Maxim Shevtsov <maxim.y.shevtsov@intel.com>
2021-06-21 17:58:19 +03:00
Anton Chetverikov
bb8e2c3137 Add documentation on how to convert RCAN model (#6259)
* Add documentation on how to convert RCAN model

* Apply review feedback
2021-06-21 13:36:44 +00:00
Ilya Lavrenov
7a316dcde3 Fixed links to OMZ / DL Streamer (#6257) 2021-06-21 11:29:54 +03:00
Eugeny Volosenkov
abe9005ffb [Attention OCR] Document model support (#6244)
* Add doc How to convert AttentionOCR

* Add converting

* Add converting 2

* Add converting 3

* Fix document

* Fix document1

* Fix document1

* Fix document1

* Fix ie_docs

* Fix model/path

* Add link to Convert_Model_From_TensorFlow.md

* fix doc

* Fix documentation
2021-06-21 09:49:06 +03:00
Ilya Lavrenov
c6654b9c81 Deprecated API updates (#6252) 2021-06-21 08:26:24 +03:00
Maria Kaglinskaya
58dd421d58 Fix klocwork issues in pruning transformation (#6175)
* Fix klockwork issues in pruning transformation

* Fixed tabs
2021-06-17 16:11:31 +03:00
Maxim Vafin
64bc081abc Add iSeeBetter to supported PyTorch models list (#6177)
* Add iSeeBetter to supported PyTorch models list

* Apply feedback

* Apply feedback
2021-06-17 15:37:02 +03:00
Taylor Yeonbok Lee
c5b65f2cb1 [IE CLDNN] Disable crop optimization only when the node is inside a loop body program (#6198) 2021-06-16 17:46:33 +03:00
Xie Zhengtian
59ffa90724 [Doc] Add doc for Auto-Device Plugin 2021.4 (#6190)
* Add doc for Auto-Device Plugin

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>

* Update doc for auto-device plugin

Signed-off-by: Zhengtian Xie <zhengtian.xie@intel.com>
2021-06-16 16:51:01 +03:00
Edward Shogulin
cb4dcbce83 [LPT] Empty shape on weights handling (#6169)
* [LPT] empty shape on weights fix

* [LPT] SplitTransformation naming fix

Co-authored-by: Vladislav Golubev <vladislav.golubev@intel.com>
2021-06-16 16:45:43 +03:00
Vladislav Volkov
5670e9d8d0 [CPU] Memory leak in jit_uni_i8i8_pooling kernel (#6189) 2021-06-16 15:55:08 +03:00
Gleb Kazantaev
e47287264c Remove Pruning from MO (#6183) 2021-06-16 14:52:54 +03:00
Szymon Durawa
fe1563f0f0 Fix external_port_id serialization for loop op. (#6168) 2021-06-16 12:03:42 +03:00
Edward Shogulin
e87ab16e7c [LPT] Reshape folding extending (#6148)
* [LPT] Reshape folding extending

* [LPT] tests addition

* typo quick fix
2021-06-16 11:14:26 +03:00
Chenhu Wang
cf5c072cf4 [CPU] Change rounding type in load/store emitters (#6156) 2021-06-16 10:35:13 +03:00
Anton Romanov
6b3a652e54 Added copyrights note in CmakeLists (#6158) 2021-06-15 16:15:41 +03:00
Mikhail Nosov
66eef3c3d9 [Caching] Klocwork fixes (#6162) 2021-06-15 14:58:50 +03:00
Andrey Somsikov
0accd09c45 Add cnpy fuzz test and fix issues (#6109) (#6159) 2021-06-15 11:05:10 +00:00
Edward Shogulin
f339cf70c6 [LPT] FakeQuantize folding fix to support ConvolutionBackpropData with FQ on weights (#6118) 2021-06-15 12:36:41 +03:00
Elena Gvozdeva
2ec6d9590c add single_layer_test for Interpolate-1 (#6133) 2021-06-12 10:47:24 +03:00
Zlobin Vladimir
ca116ab8d1 Fix InferRequest::operator!() (#6140) 2021-06-12 10:46:46 +03:00
Ilya Lavrenov
84e935c0f2 Fixed InferenceEngineConfig.cmake usage in include() (#6136) 2021-06-11 13:10:35 +03:00
Alexander Zhogov
5859d44abc GitHub CI: Check committer email 2021-06-11 11:19:50 +03:00
Alexander Zhogov
7b67a83d8c GitHub CI: Fix checking commits in case of no email 2021-06-10 12:52:22 +03:00
22557 changed files with 1045506 additions and 1601352 deletions

View File

@@ -1,163 +0,0 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: android_arm64
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
OPENVINO_REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(OPENVINO_REPO_DIR)/../openvino_contrib
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
ANDROID_TOOLS: $(WORK_DIR)/android_tools
ANDROID_SDK_VERSION: 29
ANDROID_ABI_CONFIG: arm64-v8a
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/android_arm64
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System information'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(ANDROID_TOOLS) ; mkdir $(ANDROID_TOOLS)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(OPENVINO_REPO_DIR)/install_build_dependencies.sh
# Move into contrib install_build_dependencies.sh
sudo apt --assume-yes install scons crossbuild-essential-arm64 libprotoc-dev protobuf-compiler default-jdk
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Install Android SDK, NDK and TOOLS
wget https://dl.google.com/android/repository/commandlinetools-linux-7583922_latest.zip
unzip commandlinetools-linux-7583922_latest.zip
yes | ./cmdline-tools/bin/sdkmanager --sdk_root=$(ANDROID_TOOLS) --licenses
./cmdline-tools/bin/sdkmanager --sdk_root=$(ANDROID_TOOLS) --install "ndk-bundle" "platform-tools" "platforms;android-$(ANDROID_SDK_VERSION)"
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_TOOLCHAIN_FILE=$(ANDROID_TOOLS)/ndk-bundle/build/cmake/android.toolchain.cmake
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DANDROID_ABI=$(ANDROID_ABI_CONFIG)
-DANDROID_STL=c++_shared
-DANDROID_PLATFORM=$(ANDROID_SDK_VERSION)
-DENABLE_TESTS=ON
-DBUILD_java_api=ON
-DBUILD_nvidia_plugin=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
$(OPENVINO_REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Android ARM64'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(OPENVINO_REPO_DIR)/bin/
displayName: 'List binary files'

View File

@@ -1 +1 @@
rel-1.8.1
rel-1.7.1

View File

@@ -5,24 +5,7 @@ trigger:
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- docs/*
resources:
repositories:
@@ -30,51 +13,25 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Lin
strategy:
matrix:
# Dynamic:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
# PYTHON_STATIC_ARGS:
# CMAKE_CPACK_GENERATOR:
# SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/samples
# PYTHON_SAMPLES_INSTALL_DIR: $(SAMPLES_INSTALL_DIR)/python
# RUN_PREFIX: . $(SETUPVARS) -pyver 3.8 &&
# Debian:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
# PYTHON_STATIC_ARGS:
# CMAKE_CPACK_GENERATOR: 'DEB'
# SAMPLES_INSTALL_DIR: /usr/share/openvino/samples
# PYTHON_SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/share/openvino/samples/python
# RUN_PREFIX: LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$(INSTALL_DIR)/opencv/lib:$LD_LIBRARY_PATH
Static:
CMAKE_BUILD_SHARED_LIBS: 'OFF'
PYTHON_STATIC_ARGS: -m "not dynamic_library and not template_plugin"
CMAKE_CPACK_GENERATOR:
SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/samples
PYTHON_SAMPLES_INSTALL_DIR: $(SAMPLES_INSTALL_DIR)/python
RUN_PREFIX: . $(SETUPVARS) &&
maxParallel: 2
# About 150% of total time
timeoutInMinutes: 120
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
name: LIN_VMSS_VENV_F16S_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 16
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
@@ -82,18 +39,9 @@ jobs:
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BUILD_LAYER_TESTS_DIR: $(WORK_DIR)/build_layer_tests
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)/build_samples_tests
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
LAYER_TESTS_DIR: $(INSTALL_TEST_DIR)/layer_tests
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux
CMAKE_VERSION: 3.24.0
BUILD_PYTHON: $(WORK_DIR)/build_python
INSTALL_PYTHON: $(INSTALL_OPENVINO)/extras/python
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
steps:
- script: |
@@ -101,9 +49,9 @@ jobs:
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -113,20 +61,14 @@ jobs:
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
- script: |
set -e
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'Make dir'
- checkout: self
@@ -141,30 +83,22 @@ jobs:
submodules: recursive
path: openvino_contrib
- checkout: testdata
clean: true
lfs: true
path: testdata
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
# 'clang' compiler is to check that samples can be built using it
sudo apt --assume-yes install openjdk-11-jdk libbz2-dev clang
# For Python API
sudo apt --assume-yes install libusb-1.0-0-dev
# For opencv-python: setuptools and upgrade
sudo apt-get install python3-setuptools patchelf
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/requirements.txt
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/wheel/requirements-dev.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements-dev.txt
# For running Paddle frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/paddle/tests/requirements.txt
# For running ONNX frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements.txt
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements_dev.txt
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
@@ -172,384 +106,107 @@ jobs:
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
# Should be after 'Install dependencies' because Git lfs is not installed
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DVERBOSE_BUILD=ON
-DENABLE_TEMPLATE_PLUGIN=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_PYTHON=ON
-DBUILD_SHARED_LIBS=$(CMAKE_BUILD_SHARED_LIBS)
-DENABLE_ONEDNN_FOR_GPU=$(CMAKE_BUILD_SHARED_LIBS)
-DPYTHON_EXECUTABLE=/usr/bin/python3.6
-DENABLE_WHEEL=ON
-DENABLE_TESTS=ON
-DENABLE_OV_ONNX_FRONTEND=ON
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCPACK_GENERATOR=$(CMAKE_CPACK_GENERATOR)
-DBUILD_nvidia_plugin=OFF
-S $(REPO_DIR)
-B $(BUILD_DIR)
-DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja -C $(BUILD_DIR)
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
$(REPO_DIR)/tests/layer_tests
workingDirectory: $(BUILD_LAYER_TESTS_DIR)
- script: ninja -C $(BUILD_LAYER_TESTS_DIR)
displayName: 'Build Layer Tests'
- script: sudo apt-get remove libtbb2 -y
displayName: 'Remove debian dependencies'
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
continueOnError: false
- script: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install wheel packages'
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_LAYER_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Layer Tests'
- script: python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
displayName: 'Install python wheels'
- script: |
set -e
cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install test files'
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# because of static build libgna is needed for python binary
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$LD_LIBRARY_PATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph $(PYTHON_STATIC_ARGS) \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph and IE Python Bindings Tests'
continueOnError: false
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# because of static build libgna is needed for python binary and mock_py frontend library
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
# For python imports to import pybind_mock_frontend
export PYTHONPATH=$(INSTALL_TEST_DIR):$(INSTALL_DIR)/python/python3.8:$PYTHONPATH
# For frontend manager to load mock_py library
export OV_FRONTEND_PATH=$(INSTALL_TEST_DIR)
python3 -m pytest -sv $(INSTALL_TEST_DIR)/pyopenvino $(PYTHON_STATIC_ARGS) \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_utils/test_utils.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_backend.py
displayName: 'Python API 2.0 Tests'
continueOnError: false
- script: |
# because of static build libgna is needed for python binary
export LD_LIBRARY_PATH=$(REPO_DIR)/temp/gna_03.00.00.1910/linux/x64:$LD_LIBRARY_PATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/mo/unit_tests --junitxml=$(INSTALL_TEST_DIR)/TEST-ModelOptimizer.xml
displayName: 'Model Optimizer UT'
continueOnError: false
- script: |
sudo apt-get install libtbb-dev libpugixml-dev -y
cmake --build $(BUILD_DIR) --target package -j8
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Build Debian packages'
continueOnError: false
- script: |
# install debian packages from previous release
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-get install --no-install-recommends gnupg -y
echo "deb https://apt.repos.intel.com/openvino/2022 focal main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/intel-openvino-2022.list
# sudo apt-get install openvino -y
# install our local one and make sure the conflicts are resolved
sudo apt-get install --no-install-recommends dpkg-dev -y
rm -r _CPack_Packages
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
echo "deb [trusted=yes] file:$(BUILD_DIR) ./" | sudo tee /etc/apt/sources.list.d/openvino-local.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/openvino-local.list
sudo apt-get install openvino -y
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
condition: eq(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Install Debian packages'
continueOnError: false
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
condition: ne(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'Install openvino'
continueOnError: false
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
condition: ne(variables['CMAKE_CPACK_GENERATOR'], 'DEB')
displayName: 'List install files'
- script: $(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR) -b $(BUILD_DIR)/cpp_samples
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/cpp/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
continueOnError: false
- script: |
export CC=clang
export CXX=clang++
$(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -b $(BUILD_DIR)/cpp_samples_clang
displayName: 'Build cpp samples - clang'
continueOnError: false
- script: |
export VERBOSE=1
$(SAMPLES_INSTALL_DIR)/c/build_samples.sh -i $(INSTALL_DIR) -b $(BUILD_DIR)/c_samples
- script: $(INSTALL_DIR)/deployment_tools/inference_engine/samples/c/build_samples.sh
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
- script: $(BIN_DIR)/unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
- script: rm -fr $(BUILD_DIR)
displayName: 'Clean build dir'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_inference_functional_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceFunc.xml
displayName: 'Inference Func Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_inference_unit_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceUnit.xml
displayName: 'Inference Unit Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_conditional_compilation_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ConditionalCompilation.xml
displayName: 'Conditional Compilation Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_lp_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-LpTransformations.xml
displayName: 'Low Precision Transformations Tests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Transformations.xml
displayName: 'Transformations Tests'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --workers=16 --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(BIN_DIR)/InferenceEngineUnitTests --workers=$(WORKERS_NUMBER) --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(BIN_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
- script: $(BIN_DIR)/ieUnitTests --gtest_output=xml:TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
- script: $(BIN_DIR)/cpuUnitTests --gtest_output=xml:TEST-cpuUnitTests.xml
displayName: 'CPU UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_gna_unit_tests.xml
- script: $(BIN_DIR)/gnaUnitTests --gtest_output=xml:TEST-gnaUnitTests.xml
displayName: 'GNA UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
- script: $(BIN_DIR)/vpuUnitTests --gtest_output=xml:TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-XLinkTests.xml
displayName: 'XLink Tests'
- script: $(BIN_DIR)/onnxImporterUnitTests --gtest_output=xml:TEST-onnxImporterUnitTests.xml
displayName: 'ONNX Importer UT'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: |
# because of IR reader v7
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
- script: $(BIN_DIR)/ieFuncTests --gtest_output=xml:TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_template_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-templateFuncTests.xml
- script: $(BIN_DIR)/templateFuncTests --gtest_filter=*smoke* --gtest_output=xml:TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
- script: $(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
continueOnError: false
condition: and(succeeded(), eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'OFF'))
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
$(BIN_DIR)/InferenceEngineCAPITests --gtest_output=xml:TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(RUN_PREFIX) $(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
$(REPO_DIR)/tests/samples_tests
workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'Install Samples Tests'
- script: python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
workingDirectory: $(INSTALL_TEST_DIR)
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
export PATH=$HOME/.local/bin:$PATH
export IE_APP_PATH=$(INSTALL_DIR)/samples_bin
export LD_LIBRARY_PATH=$IE_APP_PATH:$LD_LIBRARY_PATH
export IE_APP_PYTHON_PATH=$(PYTHON_SAMPLES_INSTALL_DIR)/
export SHARE=$(INSTALL_TEST_DIR)/smoke_tests/samples_smoke_tests_data/
export WORKSPACE=$(INSTALL_DIR)
$(RUN_PREFIX) python3 -m pytest $(INSTALL_TEST_DIR)/smoke_tests/ --env_conf $(INSTALL_TEST_DIR)/smoke_tests/env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/ --use_new_frontend -m precommit_tf_fe --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_fe.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - TF FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(REPO_DIR)/tools/mo/:$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow2_keras_tests/ --use_new_frontend -m precommit_tf_fe --junitxml=$(INSTALL_TEST_DIR)/TEST-tf2_fe.xmlTEST
displayName: 'TensorFlow 2 Layer Tests - TF FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_Roll.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - Legacy FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow2_keras_tests/test_tf2_keras_activation.py --ir_version=11 --junitxml=./TEST-tf2_Activation.xmlTEST -k "sigmoid"
displayName: 'TensorFlow 2 Layer Tests - Legacy FE'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_complex_params.py --ir_version=11 --junitxml=./TEST-test_mo_convert_complex_params.xmlTEST
displayName: 'MO Python API Tests - Complex Python params'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_tf.py --ir_version=11 --junitxml=./TEST-test_mo_convert_tf.xmlTEST
displayName: 'MO Python API Tests - Import TF model from memory'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
export TEST_DEVICE=CPU
$(RUN_PREFIX) python3 -m pytest $(LAYER_TESTS_DIR)/mo_python_api_tests/test_mo_convert_pytorch.py --ir_version=11 --junitxml=./TEST-test_mo_convert_pytorch.xmlTEST
displayName: 'MO Python API Tests - Import PyTorch model from memory'
export LD_LIBRARY_PATH=$(BIN_DIR)/lib
export PYTHONPATH=$(BIN_DIR)/lib/python_api/python3.6
env
cd $(REPO_DIR)/inference-engine/ie_bridges/python/tests
pytest pytest --junitxml=TEST-PythonAPI.xml
displayName: 'Python API Tests'
continueOnError: false
enabled: false
- task: PublishTestResults@2
condition: always()

View File

@@ -1,275 +0,0 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: linux_arm64
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
PYTHON_ARM_VERSION: "3.10.6"
PYTHON_EXEC: "python3.10"
OPENVINO_ARCH: 'aarch64'
NUM_PROC: 1
BUILD_TYPE: Release
OPENVINO_REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(OPENVINO_REPO_DIR)/../openvino_contrib
OPENCV_REPO_DIR: $(OPENVINO_REPO_DIR)/../opencv
ONETBB_REPO_DIR: $(OPENVINO_CONTRIB_REPO_DIR)/../oneTBB
BUILD_PYTHON: $(WORK_DIR)/build_python
BUILD_OPENCV: $(WORK_DIR)/build_opencv
BUILD_ONETBB: $(WORK_DIR)/build_onetbb
BUILD_OPENVINO: $(WORK_DIR)/build
BUILD_OPENVINO_PYTHON: $(WORK_DIR)/build_python
BUILD_OPEN_MODEL_ZOO: $(WORK_DIR)/build_open_model_zoo
CROSSENV_DIR: $(WORK_DIR)/cross_env
INSTALL_OPENVINO: $(WORK_DIR)/install_openvino
INSTALL_PYTHON: $(INSTALL_OPENVINO)/extras/python
INSTALL_ONETBB: $(WORK_DIR)/build/extras/oneTBB
INSTALL_ONETBB_PACKAGE: $(INSTALL_OPENVINO)/extras/oneTBB
INSTALL_OPENCV: $(INSTALL_OPENVINO)/extras/opencv
INSTALL_OPEN_MODEL_ZOO: $(INSTALL_OPENVINO)/extras/open_model_zoo
WORK_DIR: $(Pipeline.Workspace)/_w
SHARE_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
OPENVINO_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64
OPENCV_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64_opencv
ONETBB_CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_arm64_onetbb
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3 echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo "##vso[task.setvariable variable=NUM_PROC]$(nproc --all)"
echo "NUM_PROC=$(NUM_PROC)"
displayName: 'System information'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
mkdir -p $(BUILD_ONETBB) $(BUILD_OPENCV) $(BUILD_OPENVINO) $(BUILD_OPENVINO_PYTHON) $(BUILD_PYTHON) $(BUILD_OPEN_MODEL_ZOO)
mkdir -p $(INSTALL_ONETBB) $(INSTALL_ONETBB_PACKAGE) $(INSTALL_OPENVINO) $(INSTALL_PYTHON) $(INSTALL_OPENCV) $(INSTALL_OPEN_MODEL_ZOO)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(OPENVINO_CCACHE_DIR)
mkdir -p $(OPENCV_CCACHE_DIR)
mkdir -p $(ONETBB_CCACHE_DIR)
displayName: 'Make directories'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(OPENVINO_REPO_DIR)/install_build_dependencies.sh
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
export OPENCV_CCACHE_DIR=$(OPENCV_CCACHE_DIR)
export ONETBB_CCACHE_DIR=$(ONETBB_CCACHE_DIR)
export PYTHON_ARM_VERSION=$(PYTHON_ARM_VERSION)
export NUM_PROC=$(NUM_PROC)
export BUILD_PYTHON=$(BUILD_PYTHON)
export WORK_DIR=$(WORK_DIR)
export INSTALL_PYTHON=$(INSTALL_PYTHON)
export BUILD_TYPE=$(BUILD_TYPE)
export OPENVINO_REPO_DIR=$(OPENVINO_REPO_DIR)
export BUILD_ONETBB=$(BUILD_ONETBB)
export INSTALL_ONETBB=$(INSTALL_ONETBB)
export INSTALL_OPENCV=$(INSTALL_OPENCV)
export PYTHON_EXEC=$(PYTHON_EXEC)
export ONETBB_REPO_DIR=$(ONETBB_REPO_DIR)
export OPENCV_REPO_DIR=$(OPENCV_REPO_DIR)
export BUILD_OPENCV=$(BUILD_OPENCV)
export INSTALL_OPENVINO=$(INSTALL_OPENVINO)
$(OPENVINO_CONTRIB_REPO_DIR)/modules/arm_plugin/scripts/install_build_dependencies.sh
python3 -m pip install --upgrade pip
python3 -m pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Install dependencies'
- script: |
set -e
/usr/local/bin/$(PYTHON_EXEC) -m pip install -U pip
/usr/local/bin/$(PYTHON_EXEC) -m pip install crossenv
/usr/local/bin/$(PYTHON_EXEC) -m crossenv $(INSTALL_PYTHON)/bin/$(PYTHON_EXEC) $(CROSSENV_DIR)
source $(CROSSENV_DIR)/bin/activate
build-pip3 install -U pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
cross-pip3 install -U pip install -r $(OPENVINO_REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Create crossenv'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_COMPILE_WARNING_AS_ERROR=OFF
-DOpenCV_DIR=$(INSTALL_OPENCV)/cmake
-DENABLE_PYTHON=OFF
-DENABLE_TESTS=ON
-DENABLE_DATA=OFF
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_INTEL_MYRIAD=OFF
-DTHREADING=TBB
-DTBB_DIR=$(INSTALL_ONETBB)/lib/cmake/TBB
-DCMAKE_VERBOSE_MAKEFILE=ON
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules/arm_plugin
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DARM_COMPUTE_SCONS_JOBS=$(NUM_PROC)
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPENVINO)
$(OPENVINO_REPO_DIR)
workingDirectory: $(BUILD_OPENVINO)
displayName: 'CMake OpenVINO ARM plugin'
- script: |
export CCACHE_DIR=$(OPENVINO_CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
ninja
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Build OpenVINO ARM plugin'
- script: ninja install
workingDirectory: $(BUILD_OPENVINO)
displayName: 'Install OpenVINO ARM plugin'
- script: |
source $(CROSSENV_DIR)/bin/activate
cmake \
-GNinja \
-DENABLE_PYTHON=ON \
-DENABLE_WHEEL=ON \
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake \
-DOpenVINODeveloperPackage_DIR=$(BUILD_OPENVINO) \
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPENVINO) \
-S $(OPENVINO_REPO_DIR)/src/bindings/python \
-B $(BUILD_OPENVINO_PYTHON)
deactivate
displayName: 'CMake OpenVINO python binding'
- script: |
export CCACHE_DIR=$(OPENVINO_CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
export USE_CCACHE=1
ninja
workingDirectory: $(BUILD_OPENVINO_PYTHON)
displayName: 'Build OpenVINO python binding'
- script: ninja install
workingDirectory: $(BUILD_OPENVINO_PYTHON)
displayName: 'Install OpenVINO python binding'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/local/bin/$(PYTHON_EXEC)
-DPYTHON_INCLUDE_DIR=$(INSTALL_PYTHON)/include/$(PYTHON_EXEC)
-DPYTHON_LIBRARY=$(INSTALL_PYTHON)/lib
-DCMAKE_TOOLCHAIN_FILE=$(OPENVINO_REPO_DIR)/cmake/arm64.toolchain.cmake
-DOpenVINO_DIR=$(BUILD_OPENVINO)
-DOpenCV_DIR=$(INSTALL_OPENCV)/cmake
-DCMAKE_INSTALL_PREFIX=$(INSTALL_OPEN_MODEL_ZOO)
-S $(OPENVINO_REPO_DIR)/thirdparty/open_model_zoo/demos
-B $(BUILD_OPEN_MODEL_ZOO)
displayName: 'CMake Open Model Zoo demos'
- script: ninja
workingDirectory: $(BUILD_OPEN_MODEL_ZOO)
displayName: 'Build Open Model Zoo demos'
- script: ninja install
workingDirectory: $(BUILD_OPEN_MODEL_ZOO)
displayName: 'Install Open Model Zoo demos'
- script: |
cp -r $(BUILD_OPEN_MODEL_ZOO)/$(OPENVINO_ARCH)/$(BUILD_TYPE)/* $(INSTALL_OPEN_MODEL_ZOO)/
cp -r $(INSTALL_ONETBB)/* $(INSTALL_ONETBB_PACKAGE)/
zip -9 -r $(Build.ArtifactStagingDirectory)/openvino_$(OPENVINO_ARCH)_linux.zip ./*
workingDirectory: $(INSTALL_OPENVINO)
displayName: 'Create OpenVINO ARM64 linux package'
- task: PublishBuildArtifacts@1
inputs:
pathToPublish: $(Build.ArtifactStagingDirectory)
artifactName: 'openvino_aarch64_linux'
displayName: 'Publish OpenVINO AArch64 linux package'

View File

@@ -4,32 +4,8 @@ trigger:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
exclude:
- docs/*
jobs:
- job: LinCC
@@ -37,19 +13,22 @@ jobs:
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
name: LIN_VMSS_VENV_F16S_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
WORKERS_NUMBER: 16
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
steps:
- script: |
@@ -60,7 +39,6 @@ jobs:
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -84,73 +62,35 @@ jobs:
path: openvino
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DVERBOSE_BUILD=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_FASTER_BUILD=ON
-DENABLE_PROFILING_ITT=ON
-DSELECTIVE_BUILD=COLLECT
-DENABLE_INTEL_GPU=OFF
-DENABLE_MULTI=OFF
-DENABLE_AUTO=OFF
-DENABLE_AUTO_BATCH=OFF
-DENABLE_HETERO=OFF
-DENABLE_TEMPLATE=OFF
-DENABLE_INTEL_MYRIAD_COMMON=OFF
-DENABLE_OV_ONNX_FRONTEND=OFF
-DENABLE_OV_PADDLE_FRONTEND=OFF
-DENABLE_OV_TF_FRONTEND=OFF
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: |
ninja
cmake --build . --target sea_itt_lib
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build LinCC COLLECT'
displayName: 'Build'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: |
python3 thirdparty/itt_collector/runtool/sea_runtool.py --bindir $(REPO_DIR)/bin/intel64/Release -o $(BUILD_DIR)/itt_stat ! $(REPO_DIR)/bin/intel64/Release/benchmark_app -niter 1 -nireq 1 -m $(MODELS_PATH)/models/test_model/test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Code usage analysis'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-DSELECTIVE_BUILD=ON
-DSELECTIVE_BUILD_STAT=$(BUILD_DIR)/*.csv
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: cmake --build $(BUILD_DIR)
displayName: 'Build LinCC ON'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files ON'
# TODO: ebable after the fix on CPU side
- script: |
$(REPO_DIR)/bin/intel64/Release/benchmark_app -niter 1 -nireq 1 -m $(MODELS_PATH)/models/test_model/test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Use OpenVINO after CC'

View File

@@ -1,150 +0,0 @@
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
jobs:
- job: Lin
# About 150% of total time
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux_coverity
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo ls /usr/bin/python3.10
rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
sudo apt --assume-yes install openjdk-11-jdk
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- task: CMake@1
inputs:
# Coverity has too many PARSE_ERROR errors with ENABLE_FASTER_BUILD=ON. Disabling FASTER_BUILD.
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DENABLE_FASTER_BUILD=OFF
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DBUILD_nvidia_plugin=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
set -e
wget https://scan.coverity.com/download/linux64 --post-data "token=$(COVERITY_TOKEN)&project=openvino" -O coverity_tool.tgz
tar xvf coverity_tool.tgz
rm coverity_tool.tgz
workingDirectory: $(WORK_DIR)
displayName: 'Install coverity tool'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
$(WORK_DIR)/cov-analysis*/bin/cov-build --dir $(BUILD_DIR)/cov-int ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin with Coverity'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- script: tar -C $(BUILD_DIR) -czvf openvino.tgz cov-int
workingDirectory: $(BUILD_DIR)
displayName: 'Pack cov-int folder for submission'
- script: |
curl --form token=$(COVERITY_TOKEN) \
--form email=$(COVERITY_USER) \
--form file=@openvino.tgz \
--form version="$(Build.SourceVersion)" \
--form description="https://github.com/openvinotoolkit/openvino/runs/$(Build.BuildNumber)" \
https://scan.coverity.com/builds?project=openvino
workingDirectory: $(BUILD_DIR)
displayName: 'Submit for analysis'

View File

@@ -1,145 +0,0 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
resources:
repositories:
- repository: openvino
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
jobs:
- job: CUDAPlugin_Lin
timeoutInMinutes: 60
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
HOME_DIR: $(Agent.HomeDirectory)
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_REPO_DIR: $(REPO_DIR)/../openvino
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BIN_DIR: $(OPENVINO_REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
GRADLE_VER: 7.1.1
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
echo # prev line output doesn't end with eol
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
displayName: 'System info'
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
# Speed up build
sudo apt --assume-yes install unzip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script: >
sudo docker pull openvino.azurecr.io/openvino_ci/cuda-ubuntu2004:2022.1 &&
sudo docker run --volume $(REPO_DIR)/../:/root/repos --volume $(WORK_DIR):/root/w
openvino.azurecr.io/openvino_ci/cuda-ubuntu2004:2022.1
bash -c "
sudo -E /root/repos/openvino/install_build_dependencies.sh &&
python3 -m pip install -r /root/repos/openvino/src/bindings/python/requirements.txt &&
cd /root/w/build &&
cmake -GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DOPENVINO_EXTRA_MODULES=/root/repos/openvino_contrib/modules
-DENABLE_INTEL_CPU=OFF
-DENABLE_INTEL_GPU=OFF
-DENABLE_INTEL_GNA=OFF
-DBUILD_arm_plugin=OFF
-DBUILD_java_api=OFF
-DENABLE_PYTHON=OFF
-DENABLE_NVIDIA=ON
-DENABLE_TESTS=ON
/root/repos/openvino &&
/root/w/ninja -v CudaFuncTests CudaUnitTests"
workingDirectory: $(WORK_DIR)
displayName: 'Docker build Lin'
- script: ls -alR $(OPENVINO_REPO_DIR)/bin/
displayName: 'List bin files'

View File

@@ -1,451 +0,0 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: openvino_contrib
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Lin_Debian
# About 150% of total time
timeoutInMinutes: 120
pool:
name: LIN_VMSS_VENV_F16S_U20_WU2
variables:
system.debug: true
VSTS_HTTP_RETRY: 5
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)/../openvino_contrib
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BUILD_SAMPLES_DIR: $(WORK_DIR)/build_samples
BUILD_LAYER_TESTS_DIR: $(WORK_DIR)/build_layer_tests
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)/build_samples_tests
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
LAYER_TESTS_DIR: $(INSTALL_TEST_DIR)/layer_tests
SAMPLES_INSTALL_DIR: /usr/share/openvino/samples
PYTHON_SAMPLES_INSTALL_DIR: $(INSTALL_DIR)/share/openvino/samples/python
PYTHON_WHEEL_INSTALL_DIR: $HOME/.local/lib/python3.8/site-packages
TMP_DIR: /mnt/tmp
SHARE_DIR: /mount/cinfsshare/onnxtestdata
CCACHE_DIR: $(SHARE_DIR)/ccache/master/linux
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
whoami
uname -a
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
cat /proc/meminfo
cat /etc/fstab
vmstat -s
df
lsblk -o NAME,HCTL,SIZE,MOUNTPOINT | grep -i "sd"
free -h
echo TargetBranch: $(System.PullRequest.TargetBranch)
echo SourceBranch: $(Build.SourceBranch)
displayName: 'System info'
- script: |
set -e
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
rm -rf $(BUILD_DIR) ; mkdir $(BUILD_DIR)
rm -rf $(BUILD_SAMPLES_DIR) ; mkdir $(BUILD_SAMPLES_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(SHARE_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(SHARE_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(CCACHE_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
lfs: false
submodules: recursive
path: openvino
- checkout: openvino_contrib
clean: true
lfs: false
submodules: recursive
path: openvino_contrib
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
# Move jdk into contrib
# 'clang' compiler is to check that samples can be built using it
sudo apt --assume-yes install openjdk-11-jdk clang
sudo apt --assume-yes install --no-install-recommends libopencv-imgproc-dev libopencv-imgcodecs-dev
# For opencv-python: python3-setuptools and pip upgrade
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/wheel/requirements-dev.txt
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/src/bindings/python/src/compatibility/openvino/requirements-dev.txt
# For running Paddle frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/paddle/tests/requirements.txt
# For running ONNX frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# For running TensorFlow frontend unit tests
python3 -m pip install -r $(REPO_DIR)/src/frontends/tensorflow/tests/requirements.txt
# For MO unit tests
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements.txt
python3 -m pip install -r $(REPO_DIR)/tools/mo/requirements_dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
# Should be after 'Install dependencies' because Git lfs is not installed
- checkout: testdata
clean: true
lfs: true
path: testdata
- task: CMake@1
inputs:
# CMake must get Python 3.x version by default
cmakeArgs: >
-GNinja
-DCMAKE_VERBOSE_MAKEFILE=ON
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_PYTHON=ON
-DENABLE_OPENCV=OFF
-DPYTHON_EXECUTABLE=/usr/bin/python3.8
-DENABLE_TESTS=ON
-DENABLE_FASTER_BUILD=ON
-DENABLE_STRICT_DEPENDENCIES=OFF
-DENABLE_REQUIREMENTS_INSTALL=OFF
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache
-DCMAKE_C_COMPILER_LAUNCHER=ccache
-DCMAKE_CXX_LINKER_LAUNCHER=ccache
-DCMAKE_C_LINKER_LAUNCHER=ccache
-DCPACK_GENERATOR=DEB
-DBUILD_nvidia_plugin=OFF
-S $(REPO_DIR)
-B $(BUILD_DIR)
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- script: ccache --zero-stats --max-size=50G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=50G
ninja -C $(BUILD_DIR)
displayName: 'Build Lin'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-S $(REPO_DIR)/tests/layer_tests
-B $(BUILD_LAYER_TESTS_DIR)
- script: ninja -C $(BUILD_LAYER_TESTS_DIR)
displayName: 'Build Layer Tests'
- script: sudo apt-get remove libtbb2 libpugixml1v5 -y
displayName: 'Remove debian dependencies'
continueOnError: false
- script: cmake -DCOMPONENT=python_wheels -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install wheel packages'
- script: cmake -DCOMPONENT=python_samples -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install Python Samples'
- script: cmake -DCOMPONENT=gna -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install GNA plugin'
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_LAYER_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Layer Tests'
- script: python3 -m pip install openvino-dev --find-links=$(INSTALL_DIR)/tools
displayName: 'Install python wheels'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install test files'
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyngraph \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyngraph/tests/test_onnx/test_backend.py
displayName: 'nGraph and IE Python Bindings Tests'
continueOnError: false
# Skip test_onnx/test_zoo_models and test_onnx/test_backend due to long execution time
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
# Required by python imports to load requires libraries
# - tests install dir for mock_py
# - OpenVINO wheel installation dir for others frontend required by mock_py (is not part of wheel pkg)
export LD_LIBRARY_PATH=$(PYTHON_WHEEL_INSTALL_DIR)/openvino/libs:$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
# For frontend manager to load frontend libraries:
# - tests install dir for mock_py
# - OpenVINO wheel installation dir for others frontend (not required if this variable is not set)
export OV_FRONTEND_PATH=$(PYTHON_WHEEL_INSTALL_DIR)/openvino/libs:$(INSTALL_TEST_DIR)
# For python imports to import pybind_mock_frontend
export PYTHONPATH=$(INSTALL_TEST_DIR):$PYTHONPATH
python3 -m pytest -s $(INSTALL_TEST_DIR)/pyopenvino \
--junitxml=$(INSTALL_TEST_DIR)/TEST-Pyngraph.xml \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_utils/test_utils.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_zoo_models.py \
--ignore=$(INSTALL_TEST_DIR)/pyopenvino/tests/test_onnx/test_backend.py -v
displayName: 'Python API 2.0 Tests'
continueOnError: false
- script: python3 -m pytest -s $(INSTALL_TEST_DIR)/mo/unit_tests --junitxml=$(INSTALL_TEST_DIR)/TEST-ModelOptimizer.xml
displayName: 'Model Optimizer UT'
continueOnError: false
- script: |
sudo apt-get install libtbb-dev libpugixml-dev -y
cmake --build $(BUILD_DIR) --target package -j8
displayName: 'Build Debian packages'
continueOnError: false
- script: |
# install debian packages from previous release
wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
sudo apt-get install --no-install-recommends gnupg -y
echo "deb https://apt.repos.intel.com/openvino/2022 focal main" | sudo tee /etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/intel-openvino-2022.list
sudo apt-get install openvino -y
# install our local one and make sure the conflicts are resolved
sudo apt-get install --no-install-recommends dpkg-dev -y
rm -r _CPack_Packages
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
echo "deb [trusted=yes] file:$(BUILD_DIR) ./" | sudo tee /etc/apt/sources.list.d/openvino-local.list
sudo apt-get update -o Dir::Etc::sourcelist=/etc/apt/sources.list.d/openvino-local.list
sudo apt-get install openvino -y
workingDirectory: $(BUILD_DIR)
displayName: 'Install Debian packages'
continueOnError: false
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: rm -fr $(BUILD_DIR)
displayName: 'Clean build dir'
continueOnError: false
- script: $(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build cpp samples'
continueOnError: false
- script: |
export CC=clang
export CXX=clang++
$(SAMPLES_INSTALL_DIR)/cpp/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build cpp samples - clang'
continueOnError: false
- script: $(SAMPLES_INSTALL_DIR)/c/build_samples.sh -i $(INSTALL_DIR)
displayName: 'Build c samples'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
$(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
# python3 $(WORK_DIR)/gtest-parallel/gtest_parallel.py $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --workers=16 --dump_json_test_results=InferenceEngineUnitTests.json --gtest_filter=*smoke* -- --gtest_print_time=1
- script: $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_gna_unit_tests.xml
displayName: 'GNA UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-XLinkTests.xml
displayName: 'XLink Tests'
continueOnError: false
- script: $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: |
# because of IR reader v7
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR)
$(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: |
export LD_LIBRARY_PATH=$(INSTALL_TEST_DIR):$LD_LIBRARY_PATH
$(INSTALL_TEST_DIR)/ov_template_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
# run not all smoke filter to save time in post-commit
- script: $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*OVCLass*:*CoreThreadingTests* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
displayName: 'CPU FuncTests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
$(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: CMake@1
inputs:
cmakeArgs: >
-GNinja
-S $(REPO_DIR)/tests/samples_tests
-B $(BUILD_SAMPLES_TESTS_DIR)
- script: cmake -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_SAMPLES_TESTS_DIR)/cmake_install.cmake
displayName: 'Install Samples Tests'
- script: python3 -m pip install -r $(INSTALL_TEST_DIR)/smoke_tests/requirements.txt
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
export PATH=$HOME/.local/bin:$PATH
export IE_APP_PATH=$(INSTALL_DIR)/samples_bin
export LD_LIBRARY_PATH=$IE_APP_PATH:$LD_LIBRARY_PATH
# for gna plugin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$(INSTALL_DIR)/lib/x86_64-linux-gnu:$(INSTALL_DIR)/lib/x86_64-linux-gnu/openvino-2022.3.0
export IE_APP_PYTHON_PATH=$(PYTHON_SAMPLES_INSTALL_DIR)/
export SHARE=$(INSTALL_TEST_DIR)/smoke_tests/samples_smoke_tests_data/
export WORKSPACE=$(INSTALL_DIR)
python3 -m pytest $(INSTALL_TEST_DIR)/smoke_tests/ --env_conf $(INSTALL_TEST_DIR)/smoke_tests/env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: |
python3 -m pip install -r $(LAYER_TESTS_DIR)/requirements.txt
export PYTHONPATH=$(LAYER_TESTS_DIR):$PYTHONPATH
python3 -m pytest $(LAYER_TESTS_DIR)/tensorflow_tests/test_tf_Roll.py --ir_version=10 --junitxml=$(INSTALL_TEST_DIR)/TEST-tf_Roll.xmlTEST
displayName: 'TensorFlow 1 Layer Tests - Legacy FE'
continueOnError: false
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit' # Options: JUnit, NUnit, VSTest, xUnit, cTest
testResultsFiles: '**/TEST-*.xml'
#searchFolder: '$(BUILD_DIR)'
mergeTestResults: false # Optional
#failTaskOnFailedTests: false # Optional
#testRunTitle: 'Pre/Post-Commit' # Optional
buildPlatform: 'x64' # Optional
buildConfiguration: 'Linux' # Optional
#publishRunAttachments: true # Optional

View File

@@ -1,80 +0,0 @@
#resources:
# repositories:
# - repository: testdata
# type: github
# endpoint: openvinotoolkit
# name: openvinotoolkit/testdata
# ref: releases/2022/3
jobs:
- job: Lin_lohika
# About 150% of total time
timeoutInMinutes: 90
pool:
name: LIN_LOHIKA
variables:
system.debug: true
# VSTS_HTTP_RETRY: 5
# VSTS_HTTP_TIMEOUT: 200
# BUILD_TYPE: Release
# REPO_DIR: $(Build.Repository.LocalPath)
# MODELS_PATH: $(REPO_DIR)/../testdata
# WORK_DIR: $(Pipeline.Workspace)/_w
# BUILD_DIR: $(WORK_DIR)/build
steps:
- script: git -C ~/work/openvino fetch origin $(Build.SourceBranch)
displayName: fetch
# - checkout: self
# clean: true
# lfs: false
# submodules: recursive
# path: openvino
- checkout: none
- script: git -C ~/work/openvino checkout -m $(Build.SourceVersion) && git -C ~/work/openvino submodule update --init --recursive
displayName: checkout
# Should be after 'Install dependencies' because Git lfs is not installed
# - checkout: testdata
# clean: true
# lfs: true
# path: testdata
- script: env -C ~/work ./configreleasenolto.sh
displayName: CMake
# - task: CMake@1
# inputs:
# # CMake must get Python 3.x version by default
# cmakeArgs: >
# -GNinja
# -DCMAKE_VERBOSE_MAKEFILE=ON
# -DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
# -DENABLE_PYTHON=ON
# -DPYTHON_EXECUTABLE=/usr/bin/python3.8
# -DENABLE_TESTS=ON
# -DENABLE_OV_ONNX_FRONTEND=ON
# -DENABLE_FASTER_BUILD=ON
# -DENABLE_STRICT_DEPENDENCIES=OFF
# -DENABLE_REQUIREMENTS_INSTALL=OFF
# -DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules
# $(REPO_DIR)
# workingDirectory: $(BUILD_DIR)
- script: >
env -C ~/work
./buildreleasenolto.sh
libopenvino_gapi_preproc.so
openvino_intel_cpu_plugin
openvino_intel_gpu_plugin
clDNN_unit_tests64
gpuFuncTests
displayName: Build Lin
- script: ~/work/testreleasenolto.sh
displayName: cldnn tests

View File

@@ -4,27 +4,8 @@ trigger:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
exclude:
- docs/*
jobs:
- job: OpenVINO_ONNX_CI
@@ -32,9 +13,11 @@ jobs:
matrix:
Release:
BUILD_TYPE: 'Release'
PROTOBUF_LITE: 'ON'
TOX_COMMAND: 'tox && tox -e zoo_models'
Debug:
BUILD_TYPE: 'Debug'
PROTOBUF_LITE: 'ON'
TOX_COMMAND: 'tox'
maxParallel: 2
@@ -79,8 +62,7 @@ jobs:
- script: |
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo apt install nfs-common -y
sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
mkdir -p $(MODELS_DIR)/models_data
displayName: 'Make dirs'
@@ -93,31 +75,21 @@ jobs:
- script: |
set -e
apt-get update && apt-get install -y lsb-release && apt-get clean all
sudo apt --assume-yes install git-lfs uidmap
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- script:
src/bindings/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
- script: ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d $(MODELS_DIR)/models_data -o -s "$(ONNX_MODEL_ZOO_SHA)"
displayName: 'Update models'
condition: ne(variables['BUILD_TYPE'], 'Debug')
- script: |
sudo docker build \
--tag=openvino-onnx-ci-image \
--file=.ci/openvino-onnx/Dockerfile \
--build-arg BUILD_TYPE=$(BUILD_TYPE) .
displayName: 'Docker build $(BUILD_TYPE)'
- script: sudo docker build --tag=openvino-onnx-ci-image --file=.ci/openvino-onnx/Dockerfile --build-arg BUILD_TYPE=$(BUILD_TYPE) --build-arg PROTOBUF_LITE=$(PROTOBUF_LITE) .
displayName: 'Docker build $(BUILD_TYPE) protobuf-lite: $(PROTOBUF_LITE)'
- script: sudo fallocate -l 64G /swapfile ; sudo mkswap /swapfile ; sudo swapon /swapfile ; df ; free -h
displayName: 'Create swap'
- script: |
sudo docker run \
--name openvino-onnx-ci-container \
--volume $(MODELS_DIR)/models_data/model_zoo/onnx_model_zoo_$(ONNX_MODEL_ZOO_SHA):/root/.onnx/model_zoo/onnx_model_zoo \
--volume $(MODELS_DIR)/msft:/root/.onnx/model_zoo/MSFT openvino-onnx-ci-image \
/bin/bash -c "$(TOX_COMMAND)"
displayName: 'Docker run $(BUILD_TYPE)'
- script: sudo docker run --name openvino-onnx-ci-container --volume $(MODELS_DIR)/models_data/model_zoo/onnx_model_zoo_$(ONNX_MODEL_ZOO_SHA):/root/.onnx/model_zoo/onnx_model_zoo --volume $(MODELS_DIR)/msft:/root/.onnx/model_zoo/MSFT openvino-onnx-ci-image /bin/bash -c "$(TOX_COMMAND)"
displayName: 'Docker run $(BUILD_TYPE) protobuf-lite: $(PROTOBUF_LITE)'

View File

@@ -4,34 +4,15 @@ trigger:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
- src/tests/functional/plugin/conformance/**
exclude:
- docs/*
jobs:
- job: onnxruntime
timeoutInMinutes: 90
pool:
name: LIN_VMSS_VENV_ONNX_U20_WU2
name: LIN_VMSS_VENV_ONNX_WU2
variables:
system.debug: true
@@ -43,11 +24,10 @@ jobs:
WORK_DIR: $(Pipeline.Workspace)/_w
MODELS_DIR: /mount/cinfsshare/onnxtestdata
TMP_DIR: /mnt/tmp
INSTALL_DIR: $(WORK_DIR)/install_pkg/openvino
INSTALL_DIR: $(WORK_DIR)/install_pkg
BUILD_DIR: $(WORK_DIR)/build
ONNXRUNTIME_UTILS: $(REPO_DIR)/.ci/azure/ci_utils/onnxruntime
ONNXRUNTIME_BUILD_DIR: $(ONNXRUNTIME_REPO_DIR)/build
steps:
- script: |
curl -H Metadata:true --noproxy "*" "http://169.254.169.254/metadata/instance?api-version=2019-06-01"
@@ -57,7 +37,6 @@ jobs:
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
lsb_release
env
cat /proc/cpuinfo
@@ -73,7 +52,7 @@ jobs:
rm -rf $(WORK_DIR) ; mkdir $(WORK_DIR)
sudo rm -rf $(TMP_DIR) ; sudo mkdir $(TMP_DIR) ; sudo chmod 777 -R $(TMP_DIR)
sudo mkdir -p $(MODELS_DIR)
sudo apt --assume-yes update && sudo apt --assume-yes install nfs-common
sudo apt --assume-yes install nfs-common
sudo mount -vvv -t nfs cinfsshare.file.core.windows.net:/cinfsshare/onnxtestdata $(MODELS_DIR) -o vers=4,minorversion=1,sec=sys
displayName: 'Make dirs'
@@ -89,10 +68,15 @@ jobs:
displayName: 'Clone onnxruntime'
- script: |
set -e
sudo -E $(REPO_DIR)/install_build_dependencies.sh
sudo apt --assume-yes install libusb-1.0-0-dev
# For opencv-python: setuptools and upgrade
sudo apt-get install python3-setuptools
python3 -m pip install --upgrade pip
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/requirements.txt
# For running Python API tests
python3 -m pip install -r $(REPO_DIR)/inference-engine/ie_bridges/python/src/requirements-dev.txt
# Speed up build
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
wget https://github.com/ninja-build/ninja/releases/download/v1.10.0/ninja-linux.zip
unzip ninja-linux.zip
sudo cp -v ninja /usr/local/bin/
# Speed up tests
@@ -106,70 +90,77 @@ jobs:
cmakeArgs: >
-GNinja
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE)
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON
-DENABLE_INTEL_MYRIAD_COMMON=OFF
-DENABLE_INTEL_GNA=OFF
-DENABLE_INTEL_GPU=OFF
-DENABLE_PYTHON=ON
-DPYTHON_EXECUTABLE=/usr/bin/python3.6
-DENABLE_VPU=OFF
-DENABLE_GNA=OFF
-DENABLE_OPENCV=OFF
-DENABLE_CPPLINT=OFF
-DENABLE_TESTS=OFF
-DENABLE_MKL_DNN=ON
-DENABLE_CLDNN=OFF
-DENABLE_PROFILING_ITT=OFF
-DENABLE_SAMPLES=OFF
-DENABLE_COMPILE_TOOL=OFF
-DENABLE_OV_TF_FRONTEND=OFF
-DENABLE_OV_PADDLE_FRONTEND=OFF
-DENABLE_OPENVINO_DEBUG=OFF
-DENABLE_SPEECH_DEMO=OFF
-DNGRAPH_ONNX_IMPORT_ENABLE=ON
-DNGRAPH_ONNX_EDITOR_ENABLE=ON
-DNGRAPH_INTERPRETER_ENABLE=ON
-DNGRAPH_DEBUG_ENABLE=OFF
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON
$(REPO_DIR)
workingDirectory: $(BUILD_DIR)
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Lin ONNX'
displayName: 'Build Lin'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install OpenVINO'
displayName: 'Install'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
echo "2021.2" > $(INSTALL_DIR)/deployment_tools/inference_engine/version.txt
CXXFLAGS="-Wno-error=deprecated-declarations" ./build.sh --config RelWithDebInfo --use_openvino CPU_FP32 --build_shared_lib --parallel --skip_tests --build_dir $(ONNXRUNTIME_BUILD_DIR)
workingDirectory: $(ONNXRUNTIME_REPO_DIR)
displayName: 'Build Lin ONNX Runtime'
displayName: 'Build ONNX Runtime'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
skip_tests=`tr -s '\n ' ':' < $(ONNXRUNTIME_UTILS)/skip_tests`
./onnxruntime_test_all --gtest_filter=-$skip_tests
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_test_all'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_shared_lib_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_shared_lib_test'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_global_thread_pools_test
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_global_thread_pools_test'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
./onnxruntime_api_tests_without_env
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run onnxruntime_api_tests_without_env'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-converted"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-converted tests'
- script: |
source $(INSTALL_DIR)/setupvars.sh
source $(INSTALL_DIR)/bin/setupvars.sh
./onnx_test_runner "$(ONNXRUNTIME_REPO_DIR)/cmake/external/onnx/onnx/backend/test/data/pytorch-operator"
workingDirectory: $(ONNXRUNTIME_BUILD_DIR)/RelWithDebInfo
displayName: 'Run pytorch-operator tests'

View File

@@ -4,25 +4,8 @@ trigger:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
exclude:
- docs/*
resources:
repositories:
@@ -30,13 +13,11 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Mac
@@ -44,7 +25,7 @@ jobs:
timeoutInMinutes: 360
pool:
vmImage: 'macOS-11'
vmImage: 'macOS-10.15'
variables:
system.debug: true
@@ -56,25 +37,19 @@ jobs:
MODELS_PATH: $(REPO_DIR)/../testdata
WORK_DIR: $(Pipeline.Workspace)/_w
BUILD_DIR: $(WORK_DIR)/build
BIN_DIR: $(REPO_DIR)/bin/intel64/$(BUILD_TYPE)
INSTALL_DIR: $(WORK_DIR)/install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)/tests
SETUPVARS: $(INSTALL_DIR)/setupvars.sh
TMP_DIR: /tmp
CCACHE_DIR: $(WORK_DIR)/ccache/mac
SETUPVARS: $(INSTALL_DIR)/bin/setupvars.sh
steps:
- script: |
whoami
uname -a
echo test_debug ; which python3
echo ls /usr/bin/python3.10
sudo rm -rf /usr/bin/python3
sudo ln -s /usr/bin/python3.10 /usr/bin/python3
echo Python3 info ; which python3 ; python3 --version
echo Python info ; which python ; python --version
echo Java info ; which java ; java -version
echo gcc info ; which gcc ; gcc --version
echo cmake info ; which cmake ; cmake --version
which python3
python3 --version
which java
java -version
gcc --version
xcrun --sdk macosx --show-sdk-version
env
sysctl -a
@@ -104,15 +79,13 @@ jobs:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.10'
versionSpec: '3.7'
- script: |
brew install cython
brew install automake
python3 -m pip install -r $(REPO_DIR)/src/frontends/onnx/tests/requirements.txt
# Speed up build
brew install ninja
brew install ccache
# Speed up tests
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
@@ -122,112 +95,53 @@ jobs:
export PATH="/usr/local/opt/cython/bin:$PATH"
export CC=gcc
export CXX=g++
cmake -GNinja \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DENABLE_REQUIREMENTS_INSTALL=OFF \
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) \
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
-DENABLE_PYTHON=ON \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_LINKER_LAUNCHER=ccache \
-DCMAKE_C_LINKER_LAUNCHER=ccache \
-DBUILD_nvidia_plugin=OFF \
-S $(REPO_DIR) \
-B $(BUILD_DIR)
# Disable errors with Ninja
export CXXFLAGS="-Wno-error=unused-command-line-argument"
export CFLAGS="-Wno-error=unused-command-line-argument"
cmake -GNinja -DVERBOSE_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)/modules $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: ls -alR $(REPO_DIR)/temp/
displayName: 'List temp SDKs'
- task: Cache@2
inputs:
key: 'ccache | "$(Agent.OS)"'
path: $(CCACHE_DIR)
restoreKeys: |
ccache | "$(Agent.OS)"
displayName: Cache
- script: ccache --zero-stats --max-size=10G --show-config
displayName: 'Clean ccache stats'
- script: |
export CCACHE_DIR=$(CCACHE_DIR)
export CCACHE_TEMPDIR=$(TMP_DIR)/ccache
export CCACHE_BASEDIR=$(Pipeline.Workspace)
export CCACHE_MAXSIZE=10G
ninja
- script: ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Mac'
- script: ccache --show-stats
displayName: 'Show ccache stats'
- script: ls -alR $(REPO_DIR)/bin/
displayName: 'List bin files'
displayName: 'List files'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: |
set -e
cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)/cmake_install.cmake
displayName: 'Install tests'
- script: ls -alR $(INSTALL_DIR)
displayName: 'List install files'
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-NGraphUT.xml
displayName: 'OV Core UT'
- script: $(BIN_DIR)/unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU*:IE_CPU.onnx_model_sigmoid:IE_CPU/GRUSequenceOp.onnx_model_gru* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU*--gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineUnitTests.xml
- script: $(BIN_DIR)/InferenceEngineUnitTests --gtest_print_time=1 --gtest_filter=-MKLDNNGraphStructureTests.TestNoRedundantReordersBeforeDWConvolution:TestConvolution/MKLDNNGraphConvolutionTests.TestsConvolution/0:TestConvolutionDefaultPrimitivesPriority/MKLDNNGraphConvolutionTests.TestsConvolution/0 --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieUnitTests.xml
- script: $(BIN_DIR)/ieUnitTests --gtest_output=xml:TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
- script: $(BIN_DIR)/cpuUnitTests --gtest_output=xml:TEST-cpuUnitTests.xml
displayName: 'CPU UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-vpuUnitTests.xml
- script: $(BIN_DIR)/vpuUnitTests --gtest_output=xml:TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
- script: $(BIN_DIR)/onnxImporterUnitTests --gtest_output=xml:TEST-onnxImporterUnitTests.xml
displayName: 'ONNX Importer UT'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ieFuncTests.xml
- script: $(BIN_DIR)/ieFuncTests --gtest_output=xml:TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
enabled: false
- script: . $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_cpu_func_tests --gtest_filter=*smoke* --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_cpu_func_tests.xml
- script: $(BIN_DIR)/cpuFuncTests --gtest_filter=*smoke*:-smoke_LPT/ReduceMinTransformation.CompareWithRefImpl/f32_Shape* --gtest_print_time=1 --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
continueOnError: false
enabled: false
@@ -235,18 +149,9 @@ jobs:
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) && $(INSTALL_TEST_DIR)/InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-InferenceEngineCAPITests.xml
$(BIN_DIR)/InferenceEngineCAPITests --gtest_output=xml:TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
enabled: false
- script: |
export DATA_PATH=$(MODELS_PATH)
export MODELS_PATH=$(MODELS_PATH)
. $(SETUPVARS) && $(INSTALL_TEST_DIR)/ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)/TEST-ov_capi_test.xml
displayName: 'IE CAPITests'
continueOnError: false
enabled: false
- task: PublishTestResults@2
condition: always()

View File

@@ -4,25 +4,8 @@ trigger:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
exclude:
- docs/*
resources:
repositories:
@@ -30,29 +13,19 @@ resources:
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/openvino_contrib
ref: releases/2022/3
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
ref: releases/2022/3
jobs:
- job: Win
strategy:
matrix:
Static:
CMAKE_BUILD_SHARED_LIBS: 'OFF'
# Dynamic:
# CMAKE_BUILD_SHARED_LIBS: 'ON'
maxParallel: 2
# About 150% of total time
timeoutInMinutes: 270 #Temporary change
timeoutInMinutes: 120
pool:
name: WIN_VMSS_VENV_D8S_WU2
name: WIN_VMSS_VENV_F16S_WU2
variables:
system.debug: true
@@ -63,54 +36,20 @@ jobs:
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)\..\openvino_contrib
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: $(WORK_DIR)\build
BUILD_SAMPLES_DIR: $(WORK_DIR)\build_samples
BUILD_SAMPLES_TESTS_DIR: $(WORK_DIR)\build_samples_tests
BUILD_DIR: D:\build
BUILD_SAMPLES_DIR: D:\build_samples
BIN_DIR: $(REPO_DIR)\bin\intel64
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
INSTALL_TEST_DIR: $(INSTALL_DIR)\tests
SETUPVARS: $(INSTALL_DIR)\setupvars.bat
IB_DIR: C:\Program Files (x86)\IncrediBuild
IB_TESTCONSOLE: $(IB_DIR)\IBTestConsole.exe
PYTHON_DIR: C:\hostedtoolcache\windows\Python\3.10.7\x64
CMAKE_VERSION: 3.24.0
CMAKE_CMD: $(WORK_DIR)\cmake-$(CMAKE_VERSION)-windows-x86_64\cmake-$(CMAKE_VERSION)-windows-x86_64\bin\cmake.exe
OV_CMAKE_TOOLCHAIN_FILE: $(REPO_DIR)\cmake\toolchains\mt.runtime.win32.toolchain.cmake
PYTHON_VENV_DIR: $(WORK_DIR)\.venv
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
TEST_ENV_PATH: $(REPO_DIR)\inference-engine\temp\tbb\bin;$(REPO_DIR)\inference-engine\temp\opencv_4.5.2\opencv\bin;%PATH%
steps:
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(WORK_DIR) & mkdir C:\hostedtoolcache\windows\Python\3.10.7
rd /Q /S $(BUILD_DIR) & mkdir C:\hostedtoolcache\windows\Python\3.10.7\x64
rd /Q /S $(BUILD_SAMPLES_DIR) & mkdir $(BUILD_SAMPLES_DIR)
rd /Q /S $(BUILD_SAMPLES_TESTS_DIR) & mkdir $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'Make dir'
- script: curl -O https://www.python.org/ftp/python/3.10.7/python-3.10.7-amd64.exe
displayName: 'Download Python'
workingDirectory: $(WORK_DIR)
continueOnError: false
- script: |
python-3.10.7-amd64.exe /passive InstallAllUsers=0 Include_launcher=0 TargetDir=C:\hostedtoolcache\windows\Python\3.10.7\x64
cp C:\hostedtoolcache\windows\Python\3.8.2\x64.complete C:\hostedtoolcache\windows\Python\3.10.7\x64.complete
displayName: 'Install Python'
workingDirectory: $(WORK_DIR)
continueOnError: false
- task: UsePythonVersion@0
displayName: 'Use Python'
inputs:
versionSpec: '3.10.7'
disableDownloadFromRegistry: true
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
tree C:\hostedtoolcache\windows\Python
where python3
python3 --version
where python
python --version
where java
@@ -123,10 +62,10 @@ jobs:
displayName: 'System info'
- script: |
curl -O https://openvinoweb.z5.web.core.windows.net/incredibuild/install_ib_console.bat
call install_ib_console.bat
workingDirectory: $(WORK_DIR)
displayName: 'Install IncrediBuild'
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(BUILD_SAMPLES_DIR) & mkdir $(BUILD_SAMPLES_DIR)
displayName: 'Make dir'
- checkout: self
clean: true
@@ -146,221 +85,114 @@ jobs:
path: testdata
- script: |
python -m pip install --upgrade pip
rem For running Python API tests
python -m pip install -r $(REPO_DIR)\src\bindings\python\src\compatibility\openvino\requirements-dev.txt
python -m pip install -r $(REPO_DIR)\src\bindings\python\wheel\requirements-dev.txt
python -m pip install -r $(REPO_DIR)\src\bindings\python\requirements.txt
rem For running Paddle frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\paddle\tests\requirements.txt
rem For running ONNX frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\onnx\tests\requirements.txt
rem For running TensorFlow frontend unit tests
python -m pip install -r $(REPO_DIR)\src\frontends\tensorflow\tests\requirements.txt
rem For MO unit tests
python -m pip install -r $(REPO_DIR)\tools\mo\requirements.txt
python -m pip install -r $(REPO_DIR)\tools\mo\requirements_dev.txt
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/Kitware/CMake/releases/download/v$(CMAKE_VERSION)/cmake-$(CMAKE_VERSION)-windows-x86_64.zip -OutFile cmake-$(CMAKE_VERSION)-windows-x86_64.zip"
powershell -command "Expand-Archive -Force cmake-$(CMAKE_VERSION)-windows-x86_64.zip"
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
certutil -urlcache -split -f https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip ninja-win.zip
powershell -command "Expand-Archive -Force ninja-win.zip"
git clone https://github.com/google/gtest-parallel.git
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- powershell: |
Write-Host "##vso[task.setvariable variable=CMAKE_TOOLCHAIN_FILE]$(OV_CMAKE_TOOLCHAIN_FILE)"
condition: eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'ON')
displayName: "Set cmake toolchain"
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && $(CMAKE_CMD) -G "Ninja Multi-Config" ^
-DENABLE_ONEDNN_FOR_GPU=$(CMAKE_BUILD_SHARED_LIBS) ^
-DBUILD_SHARED_LIBS=$(CMAKE_BUILD_SHARED_LIBS) ^
-DENABLE_REQUIREMENTS_INSTALL=OFF ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_TESTS=ON ^
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON ^
-DENABLE_STRICT_DEPENDENCIES=OFF ^
-DENABLE_PYTHON=ON ^
-DBUILD_nvidia_plugin=OFF ^
-DPYTHON_EXECUTABLE="C:\hostedtoolcache\windows\Python\3.10.7\x64\python.exe" ^
-DPYTHON_INCLUDE_DIR="C:\hostedtoolcache\windows\Python\3.10.7\x64\include" ^
-DPYTHON_LIBRARY="C:\hostedtoolcache\windows\Python\3.10.7\x64\libs\python310.lib" ^
-DOPENVINO_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-S $(REPO_DIR) -B $(BUILD_DIR)
displayName: 'CMake OpenVINO'
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DENABLE_TEMPLATE_PLUGIN=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_TESTS=ON -DENABLE_STRICT_DEPENDENCIES=OFF -DIE_EXTRA_MODULES=$(OPENVINO_CONTRIB_REPO_DIR)\modules -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: dir $(REPO_DIR)\temp\ /s
- script: dir $(REPO_DIR)\inference-engine\temp\ /s
displayName: 'List temp SDKs'
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && "C:\Program Files (x86)\IncrediBuild\BuildConsole.exe" /COMMAND="$(CMAKE_CMD) --build $(BUILD_DIR) --config Release"
displayName: 'Build Win - IB'
- script: call "$(MSVS_VARS_PATH)" && $(WORK_DIR)\ninja-win\ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Win'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
- script: |
$(CMAKE_CMD) -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_DIR)/cmake_install.cmake
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: |
python -m pip install openvino-dev --find-links=$(INSTALL_DIR)\tools
displayName: 'Install Wheels'
- script: |
call "$(MSVS_VARS_PATH)" && $(CMAKE_CMD) ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-S $(REPO_DIR)\tests\samples_tests ^
-B $(BUILD_SAMPLES_TESTS_DIR)
displayName: 'CMake Samples Tests'
- script: |
$(CMAKE_CMD) -DCOMPONENT=tests -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P $(BUILD_SAMPLES_TESTS_DIR)\cmake_install.cmake
displayName: 'Install Samples Tests'
- script: $(INSTALL_DIR)\samples\cpp\build_samples_msvc.bat -i $(INSTALL_DIR)
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\cpp\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build cpp samples'
continueOnError: false
- script: $(INSTALL_DIR)\samples\c\build_samples_msvc.bat -i $(INSTALL_DIR)
- script: $(INSTALL_DIR)\deployment_tools\inference_engine\samples\c\build_samples_msvc.bat
workingDirectory: $(BUILD_SAMPLES_DIR)
displayName: 'Build c samples'
continueOnError: false
- script: |
python -m pip install -r $(INSTALL_TEST_DIR)\smoke_tests\requirements.txt
displayName: 'Install dependencies for samples smoke tests'
continueOnError: false
- script: |
call $(SETUPVARS) && set IE_APP_PATH=$(INSTALL_DIR)\samples_bin
set IE_APP_PYTHON_PATH=$(INSTALL_DIR)\samples\python\
set SHARE=$(INSTALL_DIR)\tests\smoke_tests\samples_smoke_tests_data\
set WORKSPACE=$(INSTALL_DIR)
python -m pytest $(INSTALL_DIR)\tests\smoke_tests\ --env_conf $(INSTALL_TEST_DIR)\smoke_tests\env_config.yml -s --junitxml=$(INSTALL_TEST_DIR)/TEST-SamplesSmokeTests.xml
displayName: 'Samples Smoke Tests'
continueOnError: false
- script: $(CMAKE_CMD) -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -DCOMPONENT=tests -P $(BUILD_DIR)\cmake_install.cmake
displayName: 'Install tests'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'
- script: rd /Q /S $(BUILD_DIR)
displayName: 'Clean build dir'
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\unit-test --gtest_print_time=1 --gtest_filter=-backend_api.config_unsupported:*IE_GPU* --gtest_output=xml:TEST-NGraphUT.xml
displayName: 'nGraph UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_core_unit_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-NGraphUT.xml
displayName: 'OV Core UT'
continueOnError: false
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\InferenceEngineUnitTests.exe --gtest_output=xml:TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old - IB'
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_inference_functional_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceFunc.xml
displayName: 'Inference Func Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_inference_unit_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceUnit.xml
displayName: 'Inference Unit Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_conditional_compilation_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ConditionalCompilation.xml
displayName: 'Conditional Compilation Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_ir_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-IRFrontend.xml
displayName: 'IR Frontend Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_onnx_frontend_tests --gtest_print_time=1 --gtest_filter=-*IE_GPU* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ONNXFrontend.xml
displayName: 'ONNX Frontend Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\paddle_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-Paddle.xml
displayName: 'Paddle Frontend UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_tensorflow_frontend_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-Tensorflow.xml
displayName: 'TensorFlow Frontend Unit Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_lp_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\LpTransformations.xml
displayName: 'Low Precision Transformations Tests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_transformations_tests --gtest_print_time=1 --gtest_output=xml:$(INSTALL_TEST_DIR)\Transformations.xml
displayName: 'Transformations Tests'
continueOnError: false
# set PATH=$(IB_DIR);%PATH%
# call $(SETUPVARS) && "$(IB_TESTCONSOLE)" $(INSTALL_TEST_DIR)\InferenceEngineUnitTests.exe --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceEngineUnitTests-IB.xml
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\InferenceEngineUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceEngineUnitTests.xml
displayName: 'IE UT old'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieUnitTests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\ieUnitTests --gtest_output=xml:TEST-ieUnitTests.xml
displayName: 'IE UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_cpu_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_cpu_unit_tests.xml
displayName: 'Intel CPU Unit Tests'
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\cpuUnitTests --gtest_output=xml:TEST-cpuUnitTests.xml
displayName: 'CPU UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_gna_unit_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_gna_unit_tests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\gnaUnitTests --gtest_output=xml:TEST-gnaUnitTests.xml
displayName: 'GNA UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\vpuUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-vpuUnitTests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\vpuUnitTests --gtest_output=xml:TEST-vpuUnitTests.xml
displayName: 'VPU UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\XLinkTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-XLinkTests.xml
displayName: 'XLink Tests'
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\onnxImporterUnitTests --gtest_output=xml:TEST-onnxImporterUnitTests.xml
displayName: 'ONNX Importer UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieMultiPluginUnitTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieMultiPluginUnitTests.xml
displayName: 'MULTI UT'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ieFuncTests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ieFuncTests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\ieFuncTests --gtest_output=xml:TEST-ieFuncTests.xml
displayName: 'IE FuncTests'
continueOnError: false
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_template_func_tests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-templateFuncTests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\templateFuncTests --gtest_output=xml:TEST-templateFuncTests.xml
displayName: 'TEMPLATE FuncTests'
continueOnError: false
# set PATH=$(IB_DIR);%PATH%
# call $(SETUPVARS) && "$(IB_TESTCONSOLE)" $(INSTALL_TEST_DIR)\ov_cpu_func_tests.exe --gtest_filter=*smoke*:-*CompareWithRefs/base_size=16_pre_nms_topn=100_post_nms_topn=100_nms_thresh=0.7_feat_stride=1_min_size=1_ratio*:*smoke_GRUSequenceCommonZeroClip/GRUSequenceTest.CompareWithRefs/mode=CONVERT_TO_TI_MAX_SEQ_LEN_CONST_seq_lengths* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_cpu_func_tests-IB.xml /testlevel=24
- script: call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_cpu_func_tests --gtest_filter=*smoke* --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_cpu_func_tests.xml
- script: |
set PATH=$(TEST_ENV_PATH)
$(BIN_DIR)\cpuFuncTests.exe --gtest_filter=*smoke* --gtest_output=xml:TEST-cpuFuncTests.xml
displayName: 'CPU FuncTests'
continueOnError: false
condition: and(succeeded(), eq(variables['CMAKE_BUILD_SHARED_LIBS'], 'OFF'))
- script: |
set PATH=$(TEST_ENV_PATH)
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
call $(SETUPVARS) && $(INSTALL_TEST_DIR)\InferenceEngineCAPITests --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-InferenceEngineCAPITests.xml
$(BIN_DIR)\InferenceEngineCAPITests --gtest_output=xml:TEST-InferenceEngineCAPITests.xml
displayName: 'IE CAPITests'
continueOnError: false
- script: |
set DATA_PATH=$(MODELS_PATH)
set MODELS_PATH=$(MODELS_PATH)
call $(SETUPVARS) && $(INSTALL_TEST_DIR)\ov_capi_test --gtest_output=xml:$(INSTALL_TEST_DIR)\TEST-ov_capi_test.xml
displayName: 'OV CAPITests'
continueOnError: false
- task: PublishTestResults@2
condition: always()
inputs:
@@ -373,8 +205,3 @@ jobs:
buildPlatform: 'x64' # Optional
buildConfiguration: 'Windows' # Optional
#publishRunAttachments: true # Optional
- script: echo Stop IncrediBuild_Agent && net stop IncrediBuild_Agent
displayName: Stop IncrediBuild
continueOnError: true
enabled: false

View File

@@ -1,40 +1,16 @@
trigger:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
pr:
branches:
include:
- master
- releases/*
paths:
exclude:
- docs/
- /**/docs/*
- /**/*.md
- README.md
- /**/layer_tests_summary/**
resources:
repositories:
- repository: testdata
type: github
endpoint: openvinotoolkit
name: openvinotoolkit/testdata
exclude:
- docs/*
jobs:
- job: WinCC
# About 150% of total time
timeoutInMinutes: 120
timeoutInMinutes: 60
pool:
name: WIN_VMSS_VENV_F8S_WU2
@@ -45,18 +21,20 @@ jobs:
VSTS_HTTP_TIMEOUT: 200
BUILD_TYPE: Release
REPO_DIR: $(Build.Repository.LocalPath)
OPENVINO_CONTRIB_REPO_DIR: $(REPO_DIR)\..\openvino_contrib
MODELS_PATH: $(REPO_DIR)\..\testdata
WORK_DIR: $(Pipeline.Workspace)\_w
BUILD_DIR: $(WORK_DIR)\build
BUILD_DIR_2: $(WORK_DIR)\build_s
BUILD_DIR: D:\build
MSVS_VARS_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvars64.bat
MSVC_COMPILER_PATH: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.24.28314\bin\Hostx64\x64\cl.exe
INSTALL_DIR: $(WORK_DIR)\install_pkg
SETUPVARS: $(INSTALL_DIR)\setupvars.bat
SETUPVARS: $(INSTALL_DIR)\bin\setupvars.bat
steps:
- script: |
powershell -command "Invoke-RestMethod -Headers @{\"Metadata\"=\"true\"} -Method GET -Uri http://169.254.169.254/metadata/instance/compute?api-version=2019-06-01 | format-custom"
where python3
python3 --version
where python
python --version
where java
@@ -71,7 +49,6 @@ jobs:
- script: |
rd /Q /S $(WORK_DIR) & mkdir $(WORK_DIR)
rd /Q /S $(BUILD_DIR) & mkdir $(BUILD_DIR)
rd /Q /S $(BUILD_DIR_2) & mkdir $(BUILD_DIR_2)
displayName: 'Make dir'
- checkout: self
@@ -82,95 +59,30 @@ jobs:
- script: |
rem Speed up build
powershell -command "Invoke-WebRequest https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip -OutFile ninja-win.zip"
certutil -urlcache -split -f https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-win.zip ninja-win.zip
powershell -command "Expand-Archive -Force ninja-win.zip"
workingDirectory: $(WORK_DIR)
displayName: 'Install dependencies'
- checkout: testdata
clean: true
lfs: true
path: testdata
- script: |
set PATH=$(WORK_DIR)\ninja-win;%PATH%
call "$(MSVS_VARS_PATH)" && cmake -GNinja ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_PROFILING_ITT=ON ^
-DSELECTIVE_BUILD=COLLECT ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DENABLE_CLDNN=OFF ^
-DENABLE_INTEL_GPU=OFF ^
-DENABLE_MULTI=OFF ^
-DENABLE_AUTO=OFF ^
-DENABLE_AUTO_BATCH=OFF ^
-DENABLE_HETERO=OFF ^
-DENABLE_TEMPLATE=OFF ^
-DENABLE_INTEL_MYRIAD=OFF ^
-DENABLE_INTEL_MYRIAD_COMMON=OFF ^
-DENABLE_TESTS=OFF ^
-DENABLE_OV_ONNX_FRONTEND=OFF ^
-DENABLE_OV_PADDLE_FRONTEND=OFF ^
-DENABLE_OV_TF_FRONTEND=OFF ^
$(REPO_DIR)
call "$(MSVS_VARS_PATH)" && cmake -GNinja -DENABLE_FASTER_BUILD=ON -DCMAKE_BUILD_TYPE=$(BUILD_TYPE) -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT -DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" -DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" $(REPO_DIR)
workingDirectory: $(BUILD_DIR)
displayName: 'CMake'
- script: dir $(REPO_DIR)\temp\ /s
- script: dir $(REPO_DIR)\inference-engine\temp\ /s
displayName: 'List temp SDKs'
- script: |
call "$(MSVS_VARS_PATH)" && cmake --build . --config $(BUILD_TYPE) && cmake --build . --config $(BUILD_TYPE) --target sea_itt_lib
- script: call "$(MSVS_VARS_PATH)" && $(WORK_DIR)\ninja-win\ninja
workingDirectory: $(BUILD_DIR)
displayName: 'Build Win CC COLLECT'
displayName: 'Build Win CC'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files'
- script: |
call "$(MSVS_VARS_PATH)" && python thirdparty\itt_collector\runtool\sea_runtool.py --bindir $(REPO_DIR)\bin\intel64\$(BUILD_TYPE) -o $(BUILD_DIR)\itt_stat ! $(REPO_DIR)\bin\intel64\$(BUILD_TYPE)\benchmark_app.exe -niter 1 -nireq 1 -m $(MODELS_PATH)\models\test_model\test_model_fp32.xml -d CPU
workingDirectory: $(REPO_DIR)
displayName: 'Code usage analysis'
- script: cmake -DCMAKE_INSTALL_PREFIX=$(INSTALL_DIR) -P cmake_install.cmake
workingDirectory: $(BUILD_DIR)
displayName: 'Install'
- script: |
call "$(MSVS_VARS_PATH)" && cmake -G"Visual Studio 16 2019" ^
-DVERBOSE_BUILD=ON ^
-DENABLE_FASTER_BUILD=ON ^
-DCMAKE_BUILD_TYPE=$(BUILD_TYPE) ^
-DENABLE_PROFILING_ITT=OFF ^
-DSELECTIVE_BUILD=ON ^
-DSELECTIVE_BUILD_STAT=$(BUILD_DIR)\*.csv ^
-DCMAKE_C_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DCMAKE_CXX_COMPILER:PATH="$(MSVC_COMPILER_PATH)" ^
-DENABLE_CLDNN=OFF ^
-DENABLE_INTEL_GPU=OFF ^
-DENABLE_MULTI=OFF ^
-DENABLE_AUTO=OFF ^
-DENABLE_AUTO_BATCH=OFF ^
-DENABLE_HETERO=OFF ^
-DENABLE_TEMPLATE=OFF ^
-DENABLE_INTEL_MYRIAD=OFF ^
-DENABLE_INTEL_MYRIAD_COMMON=OFF ^
-DENABLE_TESTS=OFF ^
-DENABLE_OV_ONNX_FRONTEND=OFF ^
-DENABLE_OV_PADDLE_FRONTEND=OFF ^
-DENABLE_OV_TF_FRONTEND=OFF ^
$(REPO_DIR)
workingDirectory: $(BUILD_DIR_2)
displayName: 'CMake2'
- script: cmake --build . --config $(BUILD_TYPE)
workingDirectory: $(BUILD_DIR_2)
displayName: 'Build WinCC ON'
- script: dir $(REPO_DIR)\bin\ /s
displayName: 'List bin files ON'
# TODO: Enable after the fix on CPU side
# - script: |
# $(REPO_DIR)\bin\intel64\$(BUILD_TYPE)\benchmark_app.exe -niter 1 -nireq 1 -m $(MODELS_PATH)\models\test_model\test_model_fp32.xml -d CPU
# workingDirectory: $(REPO_DIR)
# displayName: 'Use OpenVINO after CC'
- script: dir $(INSTALL_DIR) /s
displayName: 'List install files'

View File

@@ -1,9 +1,10 @@
FROM ubuntu:22.04
FROM ubuntu:20.04
LABEL version=2021.03.30.1
# Build configuration arguments
ARG BUILD_TYPE=Release
ARG PROTOBUF_LITE=OFF
ARG http_proxy
ARG https_proxy
@@ -25,14 +26,16 @@ ENV LC_ALL en_US.UTF-8
RUN apt-get update && apt-get -y --no-install-recommends install \
# OpenVINO dependencies
autoconf \
automake \
build-essential \
ninja-build \
cmake \
curl \
git \
libtool \
ocl-icd-opencl-dev \
pkg-config \
unzip \
libtbb-dev \
libpugixml-dev \
wget \
# Python dependencies
python3 \
@@ -40,7 +43,11 @@ RUN apt-get update && apt-get -y --no-install-recommends install \
python3-dev \
python3-virtualenv \
cython3 \
tox && \
tox \
# ONNX dependencies
git-lfs \
protobuf-compiler \
libprotobuf-dev && \
apt-get clean autoclean && \
apt-get autoremove -y
@@ -48,28 +55,31 @@ RUN apt-get update && apt-get -y --no-install-recommends install \
COPY . /openvino/
WORKDIR /openvino/build
RUN cmake .. \
-G Ninja \
-DCMAKE_BUILD_TYPE=${BUILD_TYPE} \
-DENABLE_INTEL_MYRIAD_COMMON=OFF \
-DENABLE_INTEL_GNA=OFF \
-DENABLE_INTEL_GPU=OFF \
-DENABLE_HETERO=OFF \
-DENABLE_MULTI=OFF \
-DENABLE_BATCH=OFF \
-DENABLE_GAPI_PREPROCESSING=OFF \
-DENABLE_VPU=OFF \
-DENABLE_GNA=OFF \
-DENABLE_OPENCV=OFF \
-DENABLE_CPPLINT=OFF \
-DENABLE_NCC_STYLE=OFF \
-DENABLE_TESTS=OFF \
-DENABLE_MKL_DNN=ON \
-DENABLE_CLDNN=OFF \
-DENABLE_PROFILING_ITT=OFF \
-DENABLE_SAMPLES=OFF \
-DENABLE_OV_PADDLE_FRONTEND=OFF \
-DENABLE_OV_TF_FRONTEND=OFF \
-DENABLE_OPENVINO_DEBUG=OFF \
-DCMAKE_INSTALL_PREFIX=/openvino/dist
RUN ninja install
-DENABLE_SPEECH_DEMO=OFF \
-DENABLE_PYTHON=ON \
-DPYTHON_EXECUTABLE=/usr/bin/python3 \
-DNGRAPH_ONNX_IMPORT_ENABLE=ON \
-DNGRAPH_ONNX_EDITOR_ENABLE=ON \
-DNGRAPH_INTERPRETER_ENABLE=ON \
-DNGRAPH_DEBUG_ENABLE=OFF \
-DNGRAPH_DYNAMIC_COMPONENTS_ENABLE=ON \
-DCMAKE_INSTALL_PREFIX=/openvino/dist \
-DNGRAPH_USE_PROTOBUF_LITE=${PROTOBUF_LITE}
RUN make -j $(nproc) install
# Run tests via tox
WORKDIR /openvino/src/bindings/python
ENV OpenVINO_DIR=/openvino/dist/runtime/cmake
ENV LD_LIBRARY_PATH=/openvino/dist/runtime/lib:/openvino/dist/runtime/3rdparty/tbb/lib
ENV PYTHONPATH=/openvino/bin/intel64/${BUILD_TYPE}/python_api/python3.10:${PYTHONPATH}
WORKDIR /openvino/ngraph/python
ENV ngraph_DIR=/openvino/dist/deployment_tools/ngraph
ENV LD_LIBRARY_PATH=/openvino/dist/deployment_tools/ngraph/lib
ENV PYTHONPATH=/openvino/bin/intel64/${BUILD_TYPE}/lib/python_api/python3.8:${PYTHONPATH}
CMD tox

216
.ci/openvino-onnx/Jenkinsfile vendored Normal file
View File

@@ -0,0 +1,216 @@
// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
DOCKER_CONTAINER_NAME= "openvino-onnx-ci-container"
DOCKER_IMAGE_TAG = "openvino-onnx-ci-image"
ONNX_MODEL_ZOO_SHA = "d58213534f2a4d1c4b19ba62b3bb5f544353256e"
BACKEND_CONFIGURATIONS = [
[ name: "Release", build_type: "Release", protobuf_lite : "OFF" ],
[ name: "Debug", build_type: "Debug", protobuf_lite : "OFF" ],
[ name: "Rel_Lite", build_type: "Release", protobuf_lite : "ON" ],
]
// workaround for aborting previous builds on PR update
@NonCPS
def stopPreviousRunningBuilds() {
def jobname = env.JOB_NAME
if (jobname.startsWith("onnx-ci/openvino onnx ci/openvino/PR")){
def buildnum = env.BUILD_NUMBER.toInteger()
def job = Jenkins.instance.getItemByFullName(jobname)
def job_newest = job.builds.first()
for (build in job.builds.reverse()[0..<-1]) {
if (build.isBuilding()){
echo "Stop task = ${build} because newest #${job_newest} is on the way"
build.doStop();
continue;
}
}
}
}
def getGitPrInfo(String project, String workdir) {
def gitPrInfo = [
prAuthorEmail : "",
commitAuthorEmail : "",
commitHash : "",
commitSubject : ""
]
try {
dir ("${workdir}/${project}") {
gitPrInfo.prAuthorEmail = sh (script: 'git log -1 --pretty="format:%ae" ', returnStdout: true).trim()
gitPrInfo.commitAuthorEmail = sh (script: 'git log -1 --pretty="format:%ce" ', returnStdout: true).trim()
gitPrInfo.commitSubject = sh (script: 'git log -1 --pretty="format:%H" ', returnStdout: true).trim()
gitPrInfo.commitHash = sh (script: 'git log -1 --pretty="format:%s" ', returnStdout: true).trim()
}
}
catch(e) {
echo "Failed to retrieve ${project} git repository information!"
echo "ERROR: ${e}"
}
return gitPrInfo
}
def notifyByEmail(def gitPrInfo) {
stage('Notify') {
String notifyPeople = "${gitPrInfo.prAuthorEmail}, ${gitPrInfo.commitAuthorEmail}"
emailext (
subject: "OpenVino CI: PR ${CHANGE_ID} ${currentBuild.result}!",
body: """
Status: ${currentBuild.result}
Pull Request Title: ${CHANGE_TITLE}
Pull Request: ${CHANGE_URL}
Branch: ${CHANGE_BRANCH}
Commit Hash: ${gitPrInfo.commitSubject}
Commit Subject: ${gitPrInfo.commitHash}
Jenkins Build: ${RUN_DISPLAY_URL}
""",
to: "${notifyPeople}"
)
}
}
def gitSubmoduleUpdate(String repository_name, String workdir) {
dir ("${workdir}/${repository_name}") {
sh label: "Init ${repository_name} submodules",
script:
"""
git submodule init && git submodule update \
--init \
--no-fetch \
--recursive
"""
}
}
def prepare_repository(String workdir) {
dir("${workdir}") {
println "Preparing repository in directory: ${workdir}"
checkout scm
gitSubmoduleUpdate(PROJECT_NAME, workdir)
}
}
def updateModels() {
sh """
./ngraph/python/tests/test_onnx/model_zoo_preprocess.sh -d ${HOME}/ONNX_CI/models_data -o -s ${ONNX_MODEL_ZOO_SHA}
"""
}
def get_docker_container_name(Map configuration){
println "RUN get_docker_container_name for ${configuration.name}"
String docker_container_name = "${DOCKER_CONTAINER_NAME}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
return docker_container_name
}
def buildDockerImage(Map configuration, String workdir) {
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
println "docker_image_tag: ${docker_image_tag}"
updateModels()
sh """
docker build --tag=${docker_image_tag} \
--build-arg BUILD_TYPE=${configuration.build_type} \
--build-arg PROTOBUF_LITE=${configuration.protobuf_lite} \
--file=.ci/openvino-onnx/Dockerfile \
--build-arg http_proxy=http://proxy-ir.intel.com:911/ \
--build-arg https_proxy=http://proxy-ir.intel.com:911/ .
"""
}
def runTests(Map configuration, String workdir) {
println "Run tests for ${configuration.name}"
String docker_image_tag = "${DOCKER_IMAGE_TAG}_${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}".toLowerCase()
String docker_container_name = get_docker_container_name(configuration)
// Run only basic unit tests in Debug configuration
if (configuration.build_type == "Debug") {
sh """
docker run --name ${docker_container_name} ${docker_image_tag}
"""
}
// Run unit-tests AND large model tests by default
else {
sh """
docker run --name ${docker_container_name} \
--volume ${HOME}/ONNX_CI/models_data/model_zoo/onnx_model_zoo_${ONNX_MODEL_ZOO_SHA}:/root/.onnx/model_zoo/onnx_model_zoo \
--volume ${HOME}/ONNX_CI/data/model_zoo/MSFT:/root/.onnx/model_zoo/MSFT \
${docker_image_tag} /bin/bash -c "tox && tox -e zoo_models"
"""
}
}
def getConfigurationsMap() {
def configurationsMap = [:]
for (backend in BACKEND_CONFIGURATIONS) {
def configuration = backend.clone()
configurationsMap[configuration.name] = {
stage(configuration.name) { CONFIGURATION_WORKFLOW(configuration) }
}
}
return configurationsMap
}
CONFIGURATION_WORKFLOW = { configuration ->
node("OpenVINO") {
try {
PROJECT_NAME = "openvino"
String workdir = "${HOME}/workspace/${BUILD_NUMBER}_${env.CHANGE_ID}_${configuration.name}"
stage("Clone repository") {
prepare_repository(workdir)
}
stage("Prepare Docker environment") {
dir("${workdir}") {
buildDockerImage(configuration, workdir)
}
}
stage("Run tests") {
timeout(time: 60, unit: 'MINUTES') {
runTests(configuration, workdir)
}
}
}
catch(e) {
// Set result to ABORTED if exception contains exit code of a process interrupted by SIGTERM
if ("$e".contains("143")) {
currentBuild.result = "ABORTED"
} else {
currentBuild.result = "FAILURE"
}
def gitPrInfo = getGitPrInfo(PROJECT_NAME, workdir)
notifyByEmail(gitPrInfo)
}
finally {
stage("Cleanup") {
deleteDir()
String docker_container_name = get_docker_container_name(configuration)
sh """
docker rm -f ${docker_container_name}
"""
}
}
}
}
pipeline {
agent none
options {
skipDefaultCheckout true
timeout(activity: true, time: 120, unit: 'MINUTES')
}
stages {
stage('Parallel CI') {
steps {
stopPreviousRunningBuilds()
script {
parallelStagesMap = getConfigurationsMap()
parallel parallelStagesMap
}
}
}
}
}

65
.ci/openvino-onnx/watchdog/Jenkinsfile vendored Normal file
View File

@@ -0,0 +1,65 @@
// Copyright (C) 2018-2020 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
timeout(30)
{
node(LABEL) {
BUILD_WORKSPACE = "$WORKSPACE/$BUILD_NUMBER"
WATCHDOG_ROOT = "$BUILD_WORKSPACE/.ci/openvino-onnx/watchdog"
VENV_PATH = "${BUILD_WORKSPACE}/.wdvenv"
try {
stage("Clone repository") {
dir ("$BUILD_WORKSPACE") {
checkout([$class: 'GitSCM', branches: [[name: "*/$BRANCH"]],
doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'CloneOption', timeout: 30]], submoduleCfg: [],
userRemoteConfigs: [[credentialsId: "${GITHUB_KEY}", url: "${OPEN_VINO_URL}"]]])
}
}
stage("Prepare environment") {
sh """#!/bin/bash
if [ ! -d ${VENV_PATH} ]; then
python3 -m venv ${VENV_PATH}
source ${VENV_PATH}/bin/activate
pip install -r ${WATCHDOG_ROOT}/requirements.txt
fi
"""
}
stage("Run script") {
withCredentials([
usernamePassword(credentialsId: '7157091e-bc04-42f0-99fd-dc4da2922a55',
usernameVariable: 'username',
passwordVariable: 'password')])
{
dir ("$BUILD_WORKSPACE") {
sh """#!/bin/bash
source ${VENV_PATH}/bin/activate
export PYTHONHTTPSVERIFY=0
python ${WATCHDOG_ROOT}/src/main.py \
--msteams-url=${MSTEAMS_URL_FILE} \
--github-credentials '${username}' '${password}' \
--github-org=${GITHUB_ORG} \
--github-project=${GITHUB_PROJECT} \
--jenkins-token=${JENKINS_TOKEN_FILE} \
--jenkins-server=${JENKINS_SERVER} \
--jenkins-user=${JENKINS_USER} \
--ci-job=${CI_JOB_NAME} \
--watchdog-job=${WATCHDOG_JOB_NAME}
"""
}
}
}
} catch (e) {
echo "$e"
currentBuild.result = "FAILURE"
} finally {
stage("Cleanup") {
sh """
cd $BUILD_WORKSPACE
rm -rf ..?* .[!.]* *
"""
}
}
}
}

View File

@@ -0,0 +1,6 @@
python-jenkins==1.7.0
retrying==1.3.3
pygithub==1.51
timeout-decorator==0.4.1
requests==2.23.0
wheel

View File

@@ -0,0 +1,108 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import logging
import timeout_decorator
from datetime import datetime
from retrying import retry
from github import Github, GithubException
# Logging
logging.basicConfig(format='%(name)s - %(levelname)s - %(message)s')
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
_RETRY_LIMIT = 3
_RETRY_COOLDOWN_MS = 2000
_REQUEST_TIMEOUT_S = 10
class GitWrapper:
"""Class wrapping PyGithub API.
The purpose of this class is to wrap methods from PyGithub API used in Watchdog, for less error-prone and
more convenient use. Docs for used API, including wrapped methods can be found at:
https://pygithub.readthedocs.io/en/latest/introduction.html
:param github_credentials: Credentials used for GitHub
:param repository: GitHub repository name
:param project: GitHub project name
:type github_credentials: String
:type repository: String
:type project: String
"""
def __init__(self, github_credentials, repository, project):
self.git = Github(*github_credentials)
self.repository = repository
self.project = project
self.github_credentials = github_credentials
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_git_time(self):
"""Retrieve time from GitHub.
Used to reliably determine time during Watchdog run.
:return: Datetime object describing current time
:rtype: datetime
"""
try:
datetime_object = self._get_git_time()
except ValueError as e:
raise GitWrapperError(str(e))
except GithubException as e:
message = 'GitHub Exception during API status retrieval. Exception: {}'.format(str(e))
raise GitWrapperError(message)
except timeout_decorator.TimeoutError:
message = 'GitHub Exception during API status retrieval. Timeout during API request.'
raise GitWrapperError(message)
return datetime_object
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_pull_requests(self):
"""Retrieve paginated list of pull requests from GitHub.
:return: Paginated list of Pull Requests in GitHub repo
:rtype: github.PaginatedList.PaginatedList of github.PullRequest.PullRequest
"""
try:
prs = self._get_pull_requests()
except GithubException as e:
message = 'GitHub Exception during API status retrieval. Exception: {}'.format(str(e))
raise GitWrapperError(message)
return prs
@timeout_decorator.timeout(_REQUEST_TIMEOUT_S)
def _get_git_time(self):
"""Private method retrieving time from GitHub.
:return: Datetime object describing current time
:rtype: datetime
"""
datetime_string = self.git.get_api_status().raw_headers.get('date', '')
datetime_format = '%a, %d %b %Y %H:%M:%S %Z'
datetime_object = datetime.strptime(datetime_string, datetime_format)
return datetime_object
@timeout_decorator.timeout(_REQUEST_TIMEOUT_S)
def _get_pull_requests(self):
"""Private method retrieving pull requests from GitHub.
:return: Paginated list of Pull Requests in GitHub repo
:rtype: github.PaginatedList.PaginatedList of github.PullRequest.PullRequest
"""
return self.git.get_organization(self.repository).get_repo(self.project).get_pulls()
class GitWrapperError(Exception):
"""Base class for exceptions raised in GitWrapper.
:param message Explanation of the error
"""
def __init__(self, message):
self.message = message
log.exception(message)

View File

@@ -0,0 +1,91 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests
import jenkins
import logging
from retrying import retry
# Logging
logging.basicConfig(format='%(name)s - %(levelname)s - %(message)s')
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
_RETRY_LIMIT = 3
_RETRY_COOLDOWN_MS = 5000
class JenkinsWrapper:
"""Class wrapping Python-Jenkins API.
The purpose of this class is to wrap methods from Python-Jenkins API used in Watchdog, for less error-prone and
more convenient use. Docs for used API, including wrapped methods can be found at:
https://python-jenkins.readthedocs.io/en/latest/
:param jenkins_token: Token used for Jenkins
:param jenkins_user: Username used to connect to Jenkins
:param jenkins_server: Jenkins server address
:type jenkins_token: String
:type jenkins_user: String
:type jenkins_server: String
"""
def __init__(self, jenkins_token, jenkins_user, jenkins_server):
self.jenkins_server = jenkins_server
self.jenkins = jenkins.Jenkins(jenkins_server, username=jenkins_user,
password=jenkins_token)
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_build_console_output(self, job_name, build_number):
return self.jenkins.get_build_console_output(job_name, build_number)
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_job_info(self, job_name):
return self.jenkins.get_job_info(job_name)
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_build_info(self, job_name, build_number):
return self.jenkins.get_build_info(job_name, build_number)
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_queue_item(self, queue_id):
"""Attempt to retrieve Jenkins job queue item.
Exception communicating queue doesn't exist is expected,
in that case method returns empty dict.
:param queue_id: Jenkins job queue ID number
:type queue_id: int
:return: Dictionary representing Jenkins job queue item
:rtype: dict
"""
try:
return self.jenkins.get_queue_item(queue_id)
except Exception as e:
# Exception 'queue does not exist' is expected behaviour when job is running
if 'queue' in str(e) and 'does not exist' in str(e):
return {}
else:
raise
@retry(stop_max_attempt_number=_RETRY_LIMIT, wait_fixed=_RETRY_COOLDOWN_MS)
def get_idle_ci_hosts(self):
"""Query Jenkins for idle servers.
Send GET request to Jenkins server, querying for idle servers labeled
for OpenVino-ONNX CI job.
:return: Number of idle hosts delegated to OpenVino-ONNX CI
:rtype: int
"""
jenkins_request_url = self.jenkins_server + 'label/ci&&onnx/api/json?pretty=true'
try:
log.info('Sending request to Jenkins: %s', jenkins_request_url)
r = requests.Request(method='GET', url=jenkins_request_url, verify=False)
response = self.jenkins.jenkins_request(r).json()
return int(response['totalExecutors']) - int(response['busyExecutors'])
except Exception as e:
log.exception('Failed to send request to Jenkins!\nException message: %s', str(e))
raise

View File

@@ -0,0 +1,89 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import argparse
import sys
from watchdog import Watchdog
DEFAULT_MSTEAMS_URL_FILE = '/home/lab_nerval/tokens/msteams_url'
DEFAULT_GITHUB_ORGANIZATION = 'openvinotoolkit'
DEFAULT_GITHUB_PROJECT = 'openvino'
DEFAULT_JENKINS_TOKEN_FILE = '/home/lab_nerval/tokens/crackerjack'
DEFAULT_JENKINS_SERVER = 'https://crackerjack.intel.com/'
DEFAULT_JENKINS_USER = 'lab_nerval'
DEFAULT_CI_JOB_NAME = 'onnx/OpenVino_CI'
DEFAULT_WATCHDOG_JOB_NAME = 'onnx/ci_watchdog'
def main(args):
"""
Read args passed to script, load tokens and run watchdog.
Keyword arguments:
:param args: arguments parsed by argparse ArgumentParser
:return: returns status code 0 on successful completion
"""
jenkins_server = args.jenkins_server.strip()
jenkins_user = args.jenkins_user.strip()
jenkins_token = open(args.jenkins_token).read().replace('\n', '').strip()
msteams_url = open(args.msteams_url).read().replace('\n', '').strip()
github_credentials = args.github_credentials
github_org = args.github_org
github_project = args.github_project
ci_job = args.ci_job.strip()
watchdog_job = args.watchdog_job.strip()
quiet = args.quiet
wd = Watchdog(jenkins_token=jenkins_token,
jenkins_server=jenkins_server,
jenkins_user=jenkins_user,
github_credentials=github_credentials,
git_org=github_org,
git_project=github_project,
msteams_url=msteams_url,
ci_job_name=ci_job,
watchdog_job_name=watchdog_job)
wd.run(quiet=quiet)
return 0
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--msteams-url', help='Path to MS Teams channel url to communicate messages.',
default=DEFAULT_MSTEAMS_URL_FILE, action='store', required=False)
parser.add_argument('--github-credentials', help='GitHub user credentials to access repo.',
nargs="+", required=True)
parser.add_argument('--github-org', help='Name of organization on GitHub.',
default=DEFAULT_GITHUB_ORGANIZATION, action='store', required=False)
parser.add_argument('--github-project', help='Name of project on GitHub.',
default=DEFAULT_GITHUB_PROJECT, action='store', required=False)
parser.add_argument('--jenkins-token', help='Path to Jenkins user token to access build info.',
default=DEFAULT_JENKINS_TOKEN_FILE, action='store', required=False)
parser.add_argument('--jenkins-server', help='Jenkins server address.',
default=DEFAULT_JENKINS_SERVER, action='store', required=False)
parser.add_argument('--jenkins-user', help='Jenkins user used to log in.',
default=DEFAULT_JENKINS_USER, action='store', required=False)
parser.add_argument('--ci-job', help='Jenkins CI job name.',
default=DEFAULT_CI_JOB_NAME, action='store', required=False)
parser.add_argument('--watchdog-job', help='Jenkins CI Watchdog job name.',
default=DEFAULT_WATCHDOG_JOB_NAME, action='store', required=False)
parser.add_argument('--quiet', help="Quiet mode - doesn\'t send message to communicator.",
action='store_true', required=False)
args = parser.parse_args()
sys.exit(main(args))

View File

@@ -0,0 +1,128 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import requests
class MSTeamsCommunicator:
"""Class communicating with MSTeams using Incoming Webhook.
The purpose of this class is to use MSTeams API to send message.
Docs for used API, including wrapped methods can be found at:
https://docs.microsoft.com/en-us/outlook/actionable-messages/send-via-connectors
"""
def __init__(self, _ci_alerts_channel_url):
self._ci_alerts_channel_url = _ci_alerts_channel_url
self._queued_messages = {
self._ci_alerts_channel_url: [],
}
@property
def messages(self):
"""
Get list of queued messages.
:return: List of queued messages
:return type: List[String]
"""
return self._queued_messages.values()
def queue_message(self, message):
"""
Queue message to be sent later.
:param message: Message content
:type message: String
"""
self._queued_messages[self._ci_alerts_channel_url].append(message)
def _parse_text(self, watchdog_log, message):
"""
Parse text to display as alert.
:param watchdog_log: Watchdog log content
:param message: Unparsed message content
:type watchdog_log: String
:type message: String
"""
message_split = message.split('\n')
log_url = None
if len(message_split) == 3:
log_url = message_split[-1]
title = message_split[0]
text = message_split[1]
header = watchdog_log.split(' - ')
header_formatted = '{} - [Watchdog Log]({})'.format(header[0], header[1])
return title, log_url, '{}\n\n{}'.format(header_formatted, text)
def _json_request_content(self, title, log_url, text_formatted):
"""
Create final json request to send message to MS Teams channel.
:param title: Title of alert
:param log_url: URL to PR
:param text_formatted: General content of alert - finally formatted
:type title: String
:type title: String
:type title: String
"""
data = {
'@context': 'https://schema.org/extensions',
'@type': 'MessageCard',
'themeColor': '0072C6',
'title': title,
'text': text_formatted,
'potentialAction':
[
{
'@type': 'OpenUri',
'name': 'Open PR',
'targets':
[
{
'os': 'default',
'uri': log_url,
},
],
},
],
}
return data
def _send_to_channel(self, watchdog_log, message_queue, channel_url):
"""
Send MSTeams message to specified channel.
:param watchdog_log: Watchdog log content
:param message_queue: Queued messages to send
:param channel_url: Channel url
:type watchdog_log: String
:type message_queue: String
:type channel_url: String
"""
for message in message_queue:
title, log_url, text_formatted = self._parse_text(watchdog_log, message)
data = self._json_request_content(title, log_url, text_formatted)
try:
requests.post(url=channel_url, json=data)
except Exception as ex:
raise Exception('!!CRITICAL!! MSTeamsCommunicator: Could not send message '
'due to {}'.format(ex))
def send_message(self, watchdog_log, quiet=False):
"""
Send queued messages as single communication.
:param watchdog_log: Watchdog log content
:param quiet: Flag for disabling sending report through MS Teams
:type watchdog_log: String
:type quiet: Boolean
"""
for channel, message_queue in self._queued_messages.items():
if not quiet and message_queue:
self._send_to_channel(watchdog_log, message_queue, channel)

View File

@@ -0,0 +1,505 @@
#!/usr/bin/python3
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
import datetime
import time
import re
import logging
import requests
from ms_teams_communicator import MSTeamsCommunicator
from jenkins_wrapper import JenkinsWrapper
from jenkins import NotFoundException
from git_wrapper import GitWrapper, GitWrapperError
import os
import json
# Logging
logging.basicConfig(format='%(name)s - %(levelname)s - %(message)s')
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
# Watchdog static constant variables
_SCRIPT_DIR = os.path.dirname(os.path.realpath(__file__))
_BUILD_DURATION_THRESHOLD = datetime.timedelta(minutes=60)
_CI_START_THRESHOLD = datetime.timedelta(minutes=30)
_AWAITING_JENKINS_THRESHOLD = datetime.timedelta(minutes=5)
_WATCHDOG_DIR = os.path.expanduser('~')
_PR_REPORTS_CONFIG_KEY = 'pr_reports'
_CI_BUILD_FAIL_MESSAGE = 'ERROR: py3: commands failed'
_CI_BUILD_SUCCESS_MESSAGE = 'py3: commands succeeded'
_GITHUB_CI_CHECK_NAME = 'OpenVINO-ONNX'
INTERNAL_ERROR_MESSAGE_HEADER = '!!! --- !!! INTERNAL WATCHDOG ERROR !!! --- !!!'
ERROR_MESSAGE_HEADER = '!!! OpenVino-ONNX CI Error !!!'
WARNING_MESSAGE_HEADER = 'OpenVino-ONNX CI WARNING'
INFO_MESSAGE_HEADER = 'OpenVino-ONNX CI INFO'
class Watchdog:
"""Class describing OpenVino-ONNX-CI Watchdog.
Watchdog connects to GitHub and retrieves the list of current pull requests (PRs) in
OpenVino repository. Then it connects to specified Jenkins server to
check CI jobs associated with every PR. Watchdog verifies time durations for Jenkins
initial response, job queue and execution against time treshold constants. Every fail
is logged and reported through MS Teams communicators.
:param jenkins_token: Token used for Jenkins
:param jenkins_server: Jenkins server address
:param jenkins_user: Username used to connect to Jenkins
:param github_credentials: Credentials used to connect to GitHub
:param msteams_url: URL used to connect to MS Teams channel
:param ci_job_name: OpenVino-ONNX CI job name used in Jenkins
:param watchdog_job_name: Watchdog job name used in Jenkins
:type jenkins_token: String
:type jenkins_server: String
:type jenkins_user: String
:type github_credentials: String
:type msteams_url: String
:type ci_job_name: String
:type watchdog_job_name: String
.. note::
Watchdog and OpenVino-ONNX CI job must be placed on the same Jenkins server.
"""
def __init__(self, jenkins_token, jenkins_server, jenkins_user, github_credentials, git_org,
git_project, msteams_url, ci_job_name, watchdog_job_name):
self._config_path = os.path.join(_WATCHDOG_DIR, '{}/.{}_ci_watchdog.json'.format(_WATCHDOG_DIR, git_project))
# Jenkins Wrapper object for CI job
self._jenkins = JenkinsWrapper(jenkins_token,
jenkins_user=jenkins_user,
jenkins_server=jenkins_server)
# Load GitHub token and log in, retrieve pull requests
self._git = GitWrapper(github_credentials, repository=git_org, project=git_project)
# Create MS Teams api object
self._msteams_hook = MSTeamsCommunicator(msteams_url)
self._ci_job_name = ci_job_name.lower()
self._watchdog_job_name = watchdog_job_name
# Read config file
self._config = self._read_config_file()
# Time at Watchdog initiation
self._now_time = datetime.datetime.now()
self._current_prs = {}
self._ms_teams_enabled = True
def run(self, quiet=False):
"""Run main watchdog logic.
Retrieve list of pull requests and pass it to the method responsible for checking them.
:param quiet: Flag for disabling sending report through communicator
:type quiet: Boolean
"""
try:
pull_requests = self._git.get_pull_requests()
except GitWrapperError:
message = 'Failed to retrieve Pull Requests!'
log.exception(message)
self._queue_message(message, message_severity='internal')
# Check all pull requests
for pr in pull_requests:
try:
self._check_pr(pr)
except Exception as e:
log.exception(str(e))
self._queue_message(str(e), message_severity='internal', pr=pr)
self._update_config()
self._send_message(quiet=quiet)
def _read_config_file(self):
"""Read Watchdog config file stored on the system.
The file stores every fail already reported along with timestamp. This
mechanism is used to prevent Watchdog from reporting same failure
multiple times. In case there's no config under the expected path,
appropriate data structure is created and returned.
:return: Returns dict of dicts with reported fails with their timestamps
:rtype: dict of dicts
"""
if os.path.isfile(self._config_path):
log.info('Reading config file in: {}'.format(self._config_path))
file = open(self._config_path, 'r')
data = json.load(file)
else:
log.info('No config file found in: {}'.format(self._config_path))
data = {_PR_REPORTS_CONFIG_KEY: {}}
return data
def _check_pr(self, pr):
"""Check pull request (if there's no reason to skip).
Retrieve list of statuses for every PR's last commit and interpret them. Filters out statuses
unrelated to OpenVino-ONNX Jenkins CI and passes relevant statuses to method that interprets them.
If no commit statuses related to Jenkins are available after time defined by
**_AWAITING_JENKINS_THRESHOLD** calls appropriate method to check for builds waiting in queue.
:param pr: GitHub Pull Requests
:type pr: github.PullRequest.PullRequest
"""
log.info('===============================================')
log.info('Checking PR#{}'.format(pr.number))
# Get last Jenkins status
last_status = self._get_last_status(pr)
# Append PR checked in current run for Watchdog config
self._current_prs[str(pr.number)] = self._get_pr_timestamps(pr, last_status)
if self._should_ignore(pr) or self._updated_since_last_run(pr):
log.info('Ignoring PR#{}'.format(pr.number))
return
# Calculate time passed since PR update (any commit, merge or comment)
pr_time_delta = self._now_time - pr.updated_at
if last_status:
# Interpret found CI statuses
log.info('Last status: {} at {}'.format(last_status.description, last_status.updated_at))
self._interpret_status(last_status, pr)
elif pr_time_delta > _CI_START_THRESHOLD:
# If there's no status after assumed time - check if build is waiting in queue
log.info('CI for PR {}: NO JENKINS STATUS YET'.format(pr.number))
self._check_missing_status(pr)
@staticmethod
def _get_pr_timestamps(pr, last_status):
"""Get dict containing PR timestamp and last status timestamp.
:param pr: Single PR being currently checked
:type pr: github.PullRequest.PullRequest
:return: Dictionary with PR and last status update timestamps
:rtype: dict
"""
pr_timestamp = time.mktime(pr.updated_at.timetuple())
if last_status:
status_timestamp = time.mktime(last_status.updated_at.timetuple())
else:
status_timestamp = None
pr_dict = {'pr_timestamp': pr_timestamp,
'status_timestamp': status_timestamp}
return pr_dict
@staticmethod
def _get_last_status(pr):
"""Get last commit status posted from Jenkins.
:param pr: Single PR being currently checked
:type pr: github.PullRequest.PullRequest
:return: Either last PR status posted from Jenkins or None
:rtype: github.CommitStatus.CommitStatus
"""
# Find last commit in PR
last_commit = pr.get_commits().reversed[0]
# Get statuses and filter them to contain only those related to Jenkins CI
# and check if CI in Jenkins started
statuses = last_commit.get_statuses()
jenk_statuses = [stat for stat in statuses if
_GITHUB_CI_CHECK_NAME in stat.context]
try:
last_status = jenk_statuses[0]
except IndexError:
last_status = None
return last_status
@staticmethod
def _should_ignore(pr):
"""Determine if PR should be ignored.
:param pr: Single PR being currently checked
:type pr: github.PullRequest.PullRequest
:return: Returns True if PR should be ignored
:rtype: Bool
"""
# Ignore PR if it has WIP label or WIP in title
if 'WIP' in pr.title:
log.info('PR#{} should be ignored. WIP tag in title.'.format(pr.number))
return True
label_names = [label.name for label in pr.labels]
if 'WIP' in label_names:
log.info('PR#{} should be ignored. WIP label present.'.format(pr.number))
return True
# Ignore PR if base ref is not master
if 'master' not in pr.base.ref:
log.info('PR#{} should be ignored. Base ref is not master'.format(pr.number))
return True
# Ignore PR if mergeable state is 'dirty' or 'behind'.
# Practically this ignores PR in case of merge conflicts
ignored_mergeable_states = ['behind', 'dirty', 'draft']
if pr.mergeable_state in ignored_mergeable_states:
log.info('PR#{} should be ignored. Mergeable state is {}. '.format(pr.number, pr.mergeable_state))
return True
# If no criteria for ignoring PR are met - return false
return False
def _updated_since_last_run(self, pr):
# Ignore if PR was already checked and there was no update in meantime
pr_number = str(pr.number)
current_pr_timestamps = self._current_prs.get(pr_number)
last_pr_timestamps = self._config[_PR_REPORTS_CONFIG_KEY].get(pr_number)
if current_pr_timestamps == last_pr_timestamps:
log.info('PR#{} - No update since last check'.format(pr.number))
return True
else:
return False
def _check_missing_status(self, pr):
"""Verify if missing status is expected.
This method checks if CI build for last was scheduled and still waits in queue for
executor.
:param pr: Single PR being currently checked
:type pr: github.PullRequest.PullRequest
"""
pr_time_delta = self._now_time - pr.updated_at
try:
build_number = self._build_scheduled(pr)
if self._build_in_queue(pr, build_number):
message = ('PR# {}: build waiting in queue after {} minutes.'
.format(pr.number, pr_time_delta.seconds / 60))
severity = 'warning'
else:
message = ('PR# {}: missing status on GitHub after {} minutes.'
.format(pr.number, pr_time_delta.seconds / 60))
severity = 'error'
self._queue_message(message, message_severity=severity, pr=pr)
except TypeError:
log.info('Committer outside of OpenVino organization')
def _build_scheduled(self, pr):
"""Check if Jenkins build corresponding to PR was scheduled.
This method takes last Jenkins build for given PR and compares hash from Jenkins console output
and sha from PR object to determine if CI build for appropriate commit was scheduled.
:param pr: Single PR being currently checked
:type pr: github.PullRequest.PullRequest
:return: Returns build number or -1 if no build found
:rtype: int
"""
pr_number = str(pr.number)
project_name_full = self._ci_job_name + '/PR-' + pr_number
try:
# Retrieve console output from last Jenkins build for job corresponding to this PR
last_build_number = self._jenkins.get_job_info(project_name_full)['lastBuild']['number']
console_output = self._jenkins.get_build_console_output(project_name_full, last_build_number)
# Check if CI build was scheduled - commit hash on GH must match hash in last Jenkins build console output
# Retrieve hash from Jenkins output
match_string = '(?:Obtained .ci/[a-zA-Z/]+Jenkinsfile from ([a-z0-9]{40}))'
retrieved_sha = re.search(match_string, console_output).group(1)
if retrieved_sha == pr.get_commits().reversed[0].sha:
return last_build_number
else:
return -1
except (NotFoundException, AttributeError, requests.exceptions.HTTPError):
message = ('PR #{}: Jenkins build corresponding to commit {} not found!'
.format(pr_number, pr.get_commits().reversed[0].sha))
self._queue_message(message, message_severity='error', pr=pr)
return -1
def _build_in_queue(self, pr, build_number):
"""Check if Jenkins build waits in queue.
This method verifies if CI build is waiting in queue based on console output.
:param pr: Single PR being currently checked
:param build_number: Jenkins build number to retrieve console output from
:type pr: github.PullRequest.PullRequest
:type build_number: int
:return: Returns True if CI build is waiting in queue
:rtype: Bool
"""
pr_number = str(pr.number)
project_name_full = self._ci_job_name + '/PR-' + pr_number
# Retrieve console output
try:
console_output = self._jenkins.get_build_console_output(project_name_full, build_number)
except NotFoundException:
return False
# Check if build is waiting in queue (and not already running on an executor)
if 'Waiting for next available executor on' in console_output \
and 'Running on' not in console_output:
log.info('CI for PR %s: WAITING IN QUEUE', pr_number)
return True
else:
return False
def _interpret_status(self, status, pr):
"""
Verify GitHub status passed to the method.
This method verifies last commit status for given PR, calling appropriate methods
to further validate the status.
:param status: GitHub commit status
:param pr: Single PR being currently checked
:type status: github.CommitStatus.CommitStatus
:type pr: github.PullRequest.PullRequest
"""
try:
# Retrieve build number for Jenkins build related to this PR
build_number = self._retrieve_build_number(status.target_url)
# CI build finished - verify if expected output is present
finished_statuses = ['Build finished', 'This commit cannot be built', 'This commit looks good']
pending_statuses = ['This commit is being built', 'Testing in progress',
'This commit is scheduled to be built']
if any(phrase in status.description for phrase in finished_statuses):
self._check_finished(pr, build_number)
# CI build in progress - verify timeouts for build queue and duration
elif any(phrase in status.description for phrase in pending_statuses):
self._check_in_progress(pr, build_number)
else:
message = 'ONNX CI job for PR# {}: unrecognized status: {}'.format(pr.number, status.description)
self._queue_message(message, message_severity='error', pr=pr)
except Exception:
# Log Watchdog internal error in case any status can't be properly verified
message = 'Failed to verify status "{}" for PR# {}'.format(status.description, pr.number)
log.exception(message)
self._queue_message(message, message_severity='internal', pr=pr)
def _retrieve_build_number(self, url):
"""Retrieve Jenkins CI job build number from URL address coming from GitHub commit status.
:param url: URL address from GitHub commit status
:type url: String
:return: Returns build number
:rtype: int
"""
# Retrieve the build number from url string
match_obj = re.search('(?:/PR-[0-9]+/)([0-9]+)', url)
try:
number = int(match_obj.group(1))
return number
except Exception:
log.exception('Failed to retrieve build number from url link: %s', url)
raise
def _queue_message(self, message, message_severity='info', pr=None):
"""Add a message to message queue in communicator object.
The queued message is constructed based on message string passed as
a method argument and message header. Message header is mapped to message severity
also passed as an argument.
:param message: Message content
:param message_severity: Message severity level
:type message: String
:type message_severity: int
"""
log.info(message)
internal = False
if 'internal' in message_severity:
message_header = INTERNAL_ERROR_MESSAGE_HEADER
internal = True
elif 'error' in message_severity:
message_header = ERROR_MESSAGE_HEADER
elif 'warning' in message_severity:
message_header = WARNING_MESSAGE_HEADER
else:
message_header = INFO_MESSAGE_HEADER
# If message is related to PR attatch url
if pr:
message = message + '\n' + pr.html_url
send = message_header + '\n' + message
if self._ms_teams_enabled:
self._msteams_hook.queue_message(send)
def _check_finished(self, pr, build_number):
"""Verify if finished build output contains expected string for either fail or success.
:param pr: Single PR being currently checked
:param build_number: Jenkins CI job build number
:type pr: github.PullRequest.PullRequest
:type build_number: int
"""
pr_number = str(pr.number)
log.info('CI for PR %s: FINISHED', pr_number)
# Check if FINISH was valid FAIL / SUCCESS
project_name_full = self._ci_job_name + '/PR-' + pr_number
build_output = self._jenkins.get_build_console_output(project_name_full, build_number)
if _CI_BUILD_FAIL_MESSAGE not in build_output \
and _CI_BUILD_SUCCESS_MESSAGE not in build_output:
message = ('ONNX CI job for PR #{}: finished but no tests success or fail '
'confirmation is present in console output!'.format(pr_number))
self._queue_message(message, message_severity='error', pr=pr)
def _send_message(self, quiet=False):
"""Send messages queued in MS Teams objects to designated channel.
Queued messages are being sent as a single communication.
:param quiet: Flag for disabling sending report through communicator
:type quiet: Boolean
"""
if any(messages for messages in self._msteams_hook.messages):
try:
watchdog_build = self._jenkins.get_job_info(self._watchdog_job_name)['lastBuild']
watchdog_build_number = watchdog_build['number']
watchdog_build_link = watchdog_build['url']
except Exception:
watchdog_build_number = 'UNKNOWN'
watchdog_build_link = self._jenkins.jenkins_server
send = self._watchdog_job_name + '- build ' + str(
watchdog_build_number) + ' - ' + watchdog_build_link
if self._ms_teams_enabled:
self._msteams_hook.send_message(send, quiet=quiet)
else:
log.info('Nothing to report.')
def _check_in_progress(self, pr, build_number):
"""Check if CI build succesfully started.
Checks if build started within designated time threshold, and job is
currently running - it didn't cross the time threshold.
:param pr: Single PR being currently checked
:param build_number: Jenkins CI job build number
:type pr: github.PullRequest.PullRequest
:type build_number: int
"""
pr_number = str(pr.number)
log.info('CI for PR %s: TESTING IN PROGRESS', pr_number)
project_name_full = self._ci_job_name + '/PR-' + pr_number
build_info = self._jenkins.get_build_info(project_name_full, build_number)
build_datetime = datetime.datetime.fromtimestamp(build_info['timestamp'] / 1000.0)
build_delta = self._now_time - build_datetime
log.info('Build %s: IN PROGRESS, started: %s minutes ago', str(build_number),
str(build_delta))
# If build still waiting in queue
if build_delta > _CI_START_THRESHOLD and self._build_in_queue(pr, build_number):
message = ('ONNX CI job build #{}, for PR #{} waiting in queue after {} '
'minutes'.format(build_number, pr_number, str(build_delta.seconds / 60)))
self._queue_message(message, message_severity='warning', pr=pr)
elif build_delta > _BUILD_DURATION_THRESHOLD:
# CI job take too long, possibly froze - communicate failure
message = ('ONNX CI job build #{}, for PR #{} started,'
'but did not finish in designated time of {} '
'minutes!'.format(build_number, pr_number,
str(_BUILD_DURATION_THRESHOLD.seconds / 60)))
self._queue_message(message, message_severity='error', pr=pr)
def _update_config(self):
"""Update Watchdog config file with PRs checked in current Watchdog run, remove old entries.
:param current_prs: List of PR numbers checked during current Watchdog run
:type current_prs: list of ints
"""
# Cleanup config of old reports
log.info('Writing to config file at: {}'.format(self._config_path))
new_config = {_PR_REPORTS_CONFIG_KEY: self._current_prs}
file = open(self._config_path, 'w+')
json.dump(new_config, file)

14
.ci/pot/Jenkinsfile vendored
View File

@@ -1,14 +0,0 @@
#!groovy
properties([
parameters([
string(defaultValue: '',
description: 'Pipeline shared library version (branch/tag/commit). Determined automatically if empty',
name: 'library_version')
])
])
loadOpenVinoLibrary {
potEntrypoint(this)
}

19
.gitattributes vendored
View File

@@ -2,6 +2,7 @@
# Set default behavior to automatically normalize line endings.
###############################################################################
* text=auto
###############################################################################
# Set default behavior for command prompt diff.
#
@@ -10,7 +11,9 @@
# Note: This is only used by command line
###############################################################################
#*.cs diff=csharp
*.py text eol=lf
###############################################################################
# Set the merge driver for project and solution files
#
@@ -33,6 +36,7 @@
#*.modelproj merge=binary
#*.sqlproj merge=binary
#*.wwaproj merge=binary
###############################################################################
# behavior for image files
#
@@ -41,6 +45,7 @@
#*.jpg binary
#*.png binary
#*.gif binary
###############################################################################
# diff behavior for common document formats
#
@@ -58,21 +63,9 @@
#*.PDF diff=astextplain
#*.rtf diff=astextplain
#*.RTF diff=astextplain
*.PNG filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text
*.jpg filter=lfs diff=lfs merge=lfs -text
*.gif filter=lfs diff=lfs merge=lfs -text
*.vsdx filter=lfs diff=lfs merge=lfs -text
*.bmp filter=lfs diff=lfs merge=lfs -text
#POT attributes
tools/pot/tests/data/test_cases_refs/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/models/*/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/reference_models/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/video/* filter=lfs diff=lfs merge=lfs -text
tools/pot/tests/data/reference_fake_quantize_conf/* filter=lfs diff=lfs merge=lfs -text
/tools/pot/tests/** -pot_package
/tools/pot/tools/auxilary/** -pot_package
/tools/pot/tools/run_series_experiments.py -pot_package
/tools/pot/.pylintrc -pot_package
/tools/pot/README_dev.md -pot_package

View File

@@ -47,7 +47,7 @@ assignees: ''
- [ ] I checked the problem with documentation, FAQ, open issues, Stack Overflow, etc and have not found solution
<!--
Places to check:
* OpenVINO documentation: https://docs.openvino.ai/
* OpenVINO documentation: https://docs.openvinotoolkit.org/
* OpenVINO forum: https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/bd-p/distribution-openvino-toolkit
* OpenVINO issue tracker: https://github.com/openvinotoolkit/openvino/issues?q=is%3Aissue
* Stack Overflow branch: https://stackoverflow.com/questions/tagged/openvino

View File

@@ -2,16 +2,16 @@
version: 2
updates:
# Enable version updates for Python API
# Enable version updates for nGraph Python API
- package-ecosystem: pip
directory: "/src/bindings/python"
directory: "/ngraph/python"
schedule:
interval: weekly
day: sunday
day: monday
time: "13:00"
open-pull-requests-limit: 0
open-pull-requests-limit: 10
reviewers:
- jiwaszki
- akuporos
- postrational
labels:
- "category: dependencies"

View File

@@ -1,139 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Check GitHub organization and invite members
"""
# pylint: disable=fixme,no-member,too-many-locals
import sys
from pathlib import Path
from argparse import ArgumentParser
sys.path.append(str(Path(__file__).resolve().parents[1]))
from github_org_control.configs import Config
from github_org_control.github_api import GithubOrgApi, get_dev_emails, print_users
from github_org_control.ldap_api import LdapApi, print_user_info, InfoLevel
def remove_members(gh_api, cfg_emails, org_emails, dev_emails, org_emails_no_in_ldap):
"""Checks and remove members"""
print(
f"\n{'=' * 10} Check accounts below and remove from the GitHub organization or "
f"configuration {'=' * 10}"
)
cfg_emails_no_in_org = sorted(cfg_emails.difference(org_emails))
print(
f"\nCfg developer emails - absent in GitHub organization {len(cfg_emails_no_in_org)}:",
"; ".join(cfg_emails_no_in_org),
)
non_member_ignored_logins = set(Config().IGNORE_LOGINS).difference(
set(gh_api.org_members_by_login.keys())
)
print(
f"\nIgnored logins - absent in GitHub organization {len(non_member_ignored_logins)}:\n",
"\n".join(non_member_ignored_logins),
)
org_emails_no_in_dev = sorted(org_emails.difference(dev_emails))
print(
f"\nOrg member emails - absent in cfg and LDAP PDLs {len(org_emails_no_in_dev)}:",
"; ".join(org_emails_no_in_dev),
)
print(
f"\nOrg member emails - absent in LDAP at all {len(org_emails_no_in_ldap)}:",
"; ".join(sorted(org_emails_no_in_ldap)),
)
print("\nOrg members - no real name:")
members_to_fix_name = sorted(gh_api.members_to_fix_name, key=lambda member: member.email)
print_users(members_to_fix_name)
print(
"\nOrg member emails - no real name:",
"; ".join([member.email.lower() for member in members_to_fix_name]),
)
print("\nOrg members - no Intel emails:")
print_users(gh_api.members_to_remove)
gh_api.remove_users(org_emails_no_in_ldap | gh_api.members_to_remove)
def main():
"""The main entry point function"""
arg_parser = ArgumentParser()
arg_parser.add_argument(
"--cfg-file",
metavar="PATH",
default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}",
)
arg_parser.add_argument("--teams", action="store_true", help="Check GitHub teams")
arg_parser.add_argument("--no-ldap", action="store_true", help="Don't use LDAP info")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
gh_api = GithubOrgApi()
if args.teams:
gh_api.get_org_teams()
return
cfg_emails = get_dev_emails()
print(f"\nCfg developer emails {len(cfg_emails)}:", "; ".join(sorted(cfg_emails)))
dev_emails = set()
dev_emails.update(cfg_emails)
if not args.no_ldap:
ldap_api = LdapApi()
ldap_emails = ldap_api.get_user_emails()
dev_emails.update(ldap_emails)
print(f"\nLDAP developer emails {len(ldap_emails)}:", "; ".join(sorted(ldap_emails)))
cfg_emails_no_in_ldap = ldap_api.get_absent_emails(cfg_emails)
print(
f"\nCfg developer emails - absent in LDAP at all {len(cfg_emails_no_in_ldap)}:",
"; ".join(sorted(cfg_emails_no_in_ldap)),
)
cfg_ldap_inters = cfg_emails.intersection(ldap_emails)
print(
f"\nCfg developer emails - present in LDAP developers {len(cfg_ldap_inters)}:",
"; ".join(sorted(cfg_ldap_inters)),
)
org_emails = gh_api.get_org_emails()
print(f"\nOrg emails {len(org_emails)}:", "; ".join(sorted(org_emails)))
org_emails_no_in_ldap = set()
if not args.no_ldap:
org_ldap_diff = org_emails.difference(ldap_emails)
print(
f"\nOrg member emails - absent in LDAP developers {len(org_ldap_diff)}:",
"; ".join(sorted(org_ldap_diff)),
)
for email in org_ldap_diff:
user_info = ldap_api.get_user_info_by_email(email)
if user_info:
print_user_info(user_info, InfoLevel.PDL)
else:
org_emails_no_in_ldap.add(email)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f"\nInvite emails {len(invite_emails)}:", "; ".join(sorted(invite_emails)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
gh_api.invite_users(valid_github_users)
remove_members(gh_api, cfg_emails, org_emails, dev_emails, org_emails_no_in_ldap)
if __name__ == "__main__":
main()

View File

@@ -1,44 +0,0 @@
{
"GITHUB_TOKEN": "<Put token here or set as arg or as env variable>",
"GITHUB_ORGANIZATION": "openvinotoolkit",
"GITHUB_REPO": "openvino",
"IGNORE_LOGINS": [
"openvino-ci",
"openvino-pushbot",
"workbench-ci-bot",
"openvino-pot-ci",
"sysicvvpux",
"ote-ci-bot"
],
"MAX_MEMBERS_TO_REMOVE": 15,
"EMAILS_FILE_PATH": "dev_emails-test.txt",
"PROXIES": {
"HTTP_PROXY": null,
"HTTPS_PROXY": null,
"NO_PROXY": "localhost,127.0.0.1,.intel.com"
},
"DRY_RUN": false,
"TEAM_TO_LABEL": {
"openvino-admins": "category: CI",
"openvino-maintainers": "category: IE common",
"openvino-docs-maintainers": "category: docs",
"openvino-ie-maintainers": "category: IE common",
"openvino-ie-cpu-maintainers": "category: CPU",
"openvino-ie-gna-maintainers": "category: GNA",
"openvino-ie-gpu-maintainers": "category: GPU",
"openvino-ie-lpt-maintainers": "category: LP transformations",
"openvino-ie-auto-multi-maintainers": "category: MULTI",
"openvino-ie-python-api-maintainers": "category: python api",
"openvino-ie-template-maintainers": "category: TEMPLATE",
"openvino-ie-tests-maintainers": "category: IE Tests",
"openvino-ie-vpu-maintainers": "category: VPU",
"openvino-mo-maintainers": "category: MO",
"openvino-ngraph-maintainers": "category: nGraph",
"openvino-scripts-maintainers": "category: build",
"openvino-tests-maintainers": "category: IE Tests",
"openvino-tools-maintainers": "category: tools",
"openvino-pot-maintainers": "category: POT",
"openvino-configuration-mgmt": "category: dependency_changes",
"openvino-samples-maintainers": "category: samples"
}
}

View File

@@ -1,120 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Configurations management
"""
# pylint: disable=fixme,broad-except
import os
import sys
import ast
import json
from pathlib import Path
if sys.version_info[:2] < (3, 7):
raise Exception("Python version must be >= 3.7")
class ConfigException(Exception):
"""Base configuration exception"""
class Config:
"""Configuration wrapper"""
_instance = None
_properties = None
default_cfg_path = Path(__file__).resolve().parent / "config.json"
def __new__(cls, *_args, **_kwargs):
if not Config._instance:
Config._instance = super(Config, cls).__new__(cls)
return Config._instance
def __init__(self, file_path=None, cli_args=None):
"""
:param file_path: Path to json configuration file
:type file_path: String
:param args: List of argparse arguments with patterns: 'name=value' or 'name'
:type args: list
"""
if Config._properties:
return
self._file_path = file_path or Config.default_cfg_path
self._cli_args = cli_args or []
self._json_cfg = {}
self._args = {}
self._load_cfg()
self._parse_cli_args()
Config._properties = {}
for name, value in self._json_cfg.items():
if hasattr(self, name):
raise ConfigException(f"Duplicating prosperity: {name}")
property_value = self._args.get(name) or os.getenv(name)
if property_value:
# Try to set prosperity_value as Python literal structures, e.g. DRY_RUN=False
try:
property_value = ast.literal_eval(property_value)
except Exception:
pass
if not isinstance(property_value, type(value)):
raise ConfigException(f"Python type of {name} parameter must be {type(value)}")
else:
property_value = value
Config._properties[name] = property_value
self.set_proxy()
def __getattr__(self, attr_name):
if attr_name in self._properties:
return self._properties.get(attr_name)
raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{attr_name}'")
def _load_cfg(self):
"""Load the json configuration file"""
try:
with open(self._file_path, encoding="utf-8") as conf:
self._json_cfg = json.load(conf)
except Exception as exc:
raise ConfigException("Failed to load configuration from:", self._file_path) from exc
def _parse_cli_args(self):
"""Parse argparse arguments with patterns: 'name=value' or 'name'"""
for cli_arg in self._cli_args:
arg = cli_arg.split("=")
if arg[0] not in self._json_cfg:
raise ConfigException(f"Unsupported argument: {arg}")
self._args[arg[0]] = True if len(arg) == 1 else "=".join(arg[1:])
@property
def properties(self):
"""Get all properties as Dict"""
return self._properties
def set_proxy(self):
"""Set proxies"""
for proxy_name, url in self._properties["PROXIES"].items():
if url is not None:
print(f"Set proxy: {proxy_name}={url}")
os.environ[proxy_name] = url
def _test():
"""Test and debug"""
print("Config.default_cfg_path:", Config.default_cfg_path)
cfg = Config(cli_args=["DRY_RUN", 'PROXIES={"NO_PROXY": "localhost"}'])
print("Config.properties:", cfg.properties)
print("cfg.PROXIES:", cfg.PROXIES)
if __name__ == "__main__":
_test()

View File

@@ -1,384 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
GitHub API for controlling organization
"""
# pylint: disable=fixme,no-member
import re
import sys
import time
import typing
from pathlib import Path
from github import Github, GithubException, RateLimitExceededException, IncompletableObject
from github.PaginatedList import PaginatedList
sys.path.append(str(Path(__file__).resolve().parents[1]))
from github_org_control.configs import Config
class GithubApiException(Exception):
"""Base GitHub API exception"""
def is_valid_user(user):
"""Checks that user is valid github.Github object"""
try:
return user and user.login
except IncompletableObject:
return False
def is_user_ignored(user):
"""Checks that user should be ignored"""
if is_valid_user(user) and user.login.lower() not in Config().IGNORE_LOGINS:
return False
return True
def is_valid_name(name):
"""Checks that GitHub user's name is valid"""
return name and len(name) >= 3 and " " in name
def is_intel_email(email):
"""Checks that email is valid Intel email"""
return email and len(email) > 10 and " " not in email and email.lower().endswith("@intel.com")
def is_intel_company(company):
"""Checks that company contains intel"""
return company and "intel" in company.lower()
def is_valid_intel_user(user):
"""Checks that user is valid GitHub and Intel user"""
try:
return is_valid_user(user) and is_valid_name(user.name) and is_intel_email(user.email)
except IncompletableObject:
return False
def print_users(users):
"""Print list of users in different formats: list, set, PaginatedList"""
if isinstance(users, (list, set, PaginatedList)):
users_count = users.totalCount if isinstance(users, PaginatedList) else len(users)
print(f"GitHub users {users_count} (login - name - company - email - valid):")
else:
users = [users]
for user in users:
if not is_valid_user(user):
print("WRONG GitHub user: ???")
continue
try:
name = user.name
except IncompletableObject:
name = "???"
try:
company = user.company
except IncompletableObject:
company = "???"
try:
email = user.email
except IncompletableObject:
email = "???"
valid_check = "OK" if is_valid_intel_user(user) else "FIX"
if not is_intel_email(email):
valid_check += " email"
if not is_valid_name(name):
valid_check += " name"
print(f'{user.login} - "{name}" - "{company}" - {email} - {valid_check}')
def get_dev_emails():
"""
Read a file with developer emails. Supported email formats
first_name.last_name@intel.com
Import from Outlook: Last_name, First_name <first_name.last_name@intel.com>
"""
re_email = re.compile(r".+<(.+)>")
emails = set()
cfg = Config()
with open(cfg.properties["EMAILS_FILE_PATH"]) as file_obj:
for line in file_obj:
line = line.strip().lower()
if not line or line.startswith("#"):
continue
re_outlook_email = re_email.match(line)
if re_outlook_email:
line = re_outlook_email.group(1).strip()
if not is_intel_email(line):
print(f'Wrong email in {cfg.properties["EMAILS_FILE_PATH"]}: {line}')
continue
emails.add(line)
return emails
class GithubOrgApi:
"""Common API for GitHub organization"""
def __init__(self):
self._cfg = Config()
self.github = Github(self._cfg.GITHUB_TOKEN)
self.github_org = self.github.get_organization(self._cfg.GITHUB_ORGANIZATION)
self.repo = self.github.get_repo(f"{self._cfg.GITHUB_ORGANIZATION}/{self._cfg.GITHUB_REPO}")
self.github_users_by_email = {}
self.org_members_by_login = {}
self.members_to_remove = set()
self.members_to_fix_name = set()
def is_org_user(self, user):
"""Checks that user is a member of GitHub organization"""
if is_valid_user(user):
# user.get_organization_membership(self.github_org) doesn't work with org members
# permissions, GITHUB_TOKEN must be org owner now
return self.github_org.has_in_members(user)
return False
def get_org_emails(self):
"""Gets and prints emails of all GitHub organization members"""
org_members = self.github_org.get_members()
org_emails = set()
print(f"\nOrg members {org_members.totalCount} (login - name - company - email - valid):")
for org_member in org_members:
self.org_members_by_login[org_member.login.lower()] = org_member
print_users(org_member)
if is_intel_email(org_member.email):
email = org_member.email.lower()
org_emails.add(email)
self.github_users_by_email[email] = org_member
if not is_valid_name(org_member.name):
self.members_to_fix_name.add(org_member)
else:
self.members_to_remove.add(org_member)
print("\nOrg members - no Intel emails:")
print_users(self.members_to_remove)
print("\nOrg members - no real name:")
print_users(self.members_to_fix_name)
print(
"\nOrg member emails - no real name:",
"; ".join([member.email.lower() for member in self.members_to_fix_name]),
)
return org_emails
def get_org_invitation_emails(self):
"""Gets GitHub organization teams prints info"""
org_invitations = self.github_org.invitations()
org_invitation_emails = set()
print(
f"\nOrg invitations {org_invitations.totalCount} "
"(login - name - company - email - valid):"
)
for org_invitation in org_invitations:
print_users(org_invitation)
if is_user_ignored(org_invitation):
continue
if is_intel_email(org_invitation.email):
org_invitation_emails.add(org_invitation.email.lower())
else:
print("Strange org invitation:", org_invitation)
print(
f"\nOrg invitation emails {len(org_invitation_emails)}:",
"; ".join(org_invitation_emails),
)
return org_invitation_emails
def get_org_teams(self):
"""Gets GitHub organization teams prints info"""
teams = []
org_teams = self.github_org.get_teams()
print("\nOrg teams count:", org_teams.totalCount)
for team in org_teams:
teams.append(team.name)
print(f"\nTeam: {team.name} - parent: {team.parent}")
repos = team.get_repos()
print("Repos:")
for repo in repos:
print(f" {repo.name} -", team.get_repo_permission(repo))
team_maintainers = team.get_members(role="maintainer")
team_maintainer_logins = set()
for maintainer in team_maintainers:
team_maintainer_logins.add(maintainer.login)
team_members = team.get_members(role="member")
team_member_logins = set()
for member in team_members:
team_member_logins.add(member.login)
members = team.get_members(role="all")
member_emails = []
print("Members (role - login - name - company - email - valid):")
for user in members:
if user.login in team_maintainer_logins:
print(" Maintainer - ", end="")
elif user.login in team_member_logins:
print(" Member - ", end="")
else:
# It is not possible to check child teams members
print(" ??? - ", end="")
print_users(user)
if is_intel_email(user.email) and not is_user_ignored(user):
member_emails.append(user.email.lower())
print(f"Intel emails {len(member_emails)}:", "; ".join(member_emails))
return teams
def get_github_user_by_email(self, email):
"""Gets GitHub user by email"""
if email in self.github_users_by_email:
return self.github_users_by_email.get(email)
def search_users():
paginated_users = self.github.search_users(f"{email} in:email")
# Minimize the GitHub Rate Limit
users = []
for user in paginated_users:
users.append(user)
if len(users) == 1:
return users[0]
if len(users) == 0:
return None
raise GithubApiException(
f"ERROR: Found {len(users)} GitHub accounts with the same email {email}"
)
try:
user = search_users()
except RateLimitExceededException:
print("WARNING: RateLimitExceededException")
time.sleep(30)
user = search_users()
self.github_users_by_email[email] = user
return user
def get_valid_github_users(self, emails):
"""Gets valid GitHub users by email and prints status"""
valid_users = set()
wrong_emails = set()
no_account_emails = set()
no_account_names = set()
print(f"\nGitHub users from {len(emails)} invite emails (email - status):")
for email in emails:
if not is_intel_email(email):
print(f"{email} - Non Intel email")
wrong_emails.add(email)
continue
# You can make up to 30 requests per minute; https://developer.github.com/v3/search/
time.sleep(2)
user = self.get_github_user_by_email(email)
if not user:
print(f"{email} - No valid GitHub account")
no_account_emails.add(email)
continue
if user.email and user.email.lower() == email:
if is_valid_name(user.name):
print(f"{email} - OK")
valid_users.add(user)
else:
print(f"{email} - No valid name in GitHub account: ", end="")
print_users(user)
no_account_names.add(email)
else:
print(f"{email} - Non public or wrong email in GitHub account: ", end="")
print_users(user)
no_account_emails.add(email)
print("\nValid users:")
print_users(valid_users)
print(f"\nWrong emails {len(wrong_emails)}:", "; ".join(wrong_emails))
print(
f"\nIntel emails - No valid GitHub account {len(no_account_emails)}:",
"; ".join(no_account_emails),
)
print(
f"\nIntel emails - No valid name in GitHub account {len(no_account_names)}:",
"; ".join(no_account_names),
)
return valid_users
def invite_users(self, users):
"""Invites users to GitHub organization and prints status"""
if not isinstance(users, typing.Iterable):
users = [users]
print(f"\nInvite {len(users)} users:")
for user in users:
if isinstance(user, str):
print(f"Email: {user}")
self.github_org.invite_user(email=user)
else:
print(f'{user.login} - "{user.name}" - {user.email} - ', end="")
try:
if is_user_ignored(user):
print("Ignored")
continue
if self._cfg.DRY_RUN:
print("Dry run")
continue
self.github_org.invite_user(user=user)
print("OK")
except GithubException as exc:
print(f'FAIL: {exc.data["errors"][0]["message"]}')
def remove_users(self, users):
"""Removes users from GitHub organization"""
if not isinstance(users, typing.Iterable):
users = [users]
print(f"\nRemove {len(users)} users:")
dry_run = self._cfg.DRY_RUN
if not dry_run and len(users) > self._cfg.MAX_MEMBERS_TO_REMOVE:
print(
"WARNING: Review is required for removing members more than "
f"{self._cfg.MAX_MEMBERS_TO_REMOVE}"
)
# TODO: Add notification
dry_run = True
for user in users:
member = self.get_github_user_by_email(user) if isinstance(user, str) else user
print(f'{member.login} - "{member.name}" - {member.email} - ', end="")
try:
if is_user_ignored(member):
print("Ignored")
continue
if dry_run:
print("Dry run")
continue
self.github_org.remove_from_membership(member)
print("OK")
except GithubException as exc:
print(f'FAIL: {exc.data["errors"][0]["message"]}')
def _test():
"""Test and debug"""
Config(cli_args=["DRY_RUN=True"])
dev_emails = get_dev_emails()
print("dev_emails:", dev_emails)
gh_api = GithubOrgApi()
gh_api.get_org_emails()
if __name__ == "__main__":
_test()

View File

@@ -1,247 +0,0 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Gets info about users and groups via LDAP
"""
# pylint: disable=fixme,no-member
import sys
from enum import Enum
from pathlib import Path
from ldap3 import Server, Connection, ALL, SUBTREE
sys.path.append(str(Path(__file__).resolve().parents[1]))
from github_org_control.configs import Config
class LdapApiException(Exception):
"""Base LDAP API exception"""
class InfoLevel(Enum):
"""Constants for printing user info from LDAP"""
PDL = "PDL" # Public Distribution List (group of e-mail addresses)
FULL = "Full"
def print_user_info(info, info_level=None):
"""Pretty-print of a user info data structure (dict). info_level is the InfoLevel Enum"""
if not info or not info.get("mail"):
raise LdapApiException("ERROR: No info or absent mail")
def get_membership():
if info_level == InfoLevel.PDL:
membership_info = " PDLs:"
elif info_level == InfoLevel.FULL:
membership_info = " memberOf :"
else:
return ""
# Grouping groups by purpose
if info_level == InfoLevel.PDL:
sort_key = lambda i: i.split(",", 1)[0].lower()
else:
sort_key = lambda i: i.split(",", 1)[1] + i.split(",", 1)[0].lower()
for item in sorted(info["memberOf"], key=sort_key):
if info_level == InfoLevel.PDL and "OU=Delegated" not in item:
continue
membership_info += f"\n {item}"
return membership_info
try:
text_info = (
f'\n{info["cn"]} <{info["mail"]}>; {info["sAMAccountName"]}; {info["employeeID"]}'
f'\n Org group: {info["intelSuperGroupDescr"]} ({info["intelSuperGroupShortName"]}) /'
f' {info["intelGroupDescr"]} ({info["intelGroupShortName"]}) /'
f' {info["intelDivisionDescr"]} ({info["intelDivisionShortName"]}) /'
f' {info["intelOrgUnitDescr"]}'
f'\n Manager: {info.get("manager")}'
f'\n Location: {info["intelRegionCode"]} / {info["co"]} / {info["intelSiteCode"]} /'
f' {info["intelBldgCode"]} ({info.get("intelSiteName")}) /'
f' {info["physicalDeliveryOfficeName"]}'
f'\n Other: {info["employeeType"]} | {info["intelExportCountryGroup"]} |'
f' {info["whenCreated"]} | {info["intelCostCenterDescr"]} | {info["jobDescription"]}'
)
except Exception as exc:
raise LdapApiException(
f'ERROR: Failed to get info about "{info["mail"]}". '
f"Exception occurred:\n{repr(exc)}"
) from exc
print(text_info)
membership = get_membership()
if info_level == InfoLevel.PDL and membership:
print(membership)
elif info_level == InfoLevel.FULL:
for key in sorted(info):
if isinstance(info[key], list):
if key == "memberOf":
print(membership)
else:
print(f" {key} :")
for item in info[key]:
print(" ", item)
else:
print(f" {key} : {info[key]}")
class LdapApi:
"""LDAP API for getting user info and emails"""
_binary_blobs = ["thumbnailPhoto", "msExchUMSpokenName", "msExchBlockedSendersHash"]
_check_existing = [
"intelExportCountryGroup",
"physicalDeliveryOfficeName",
"intelSuperGroupShortName",
"intelGroupShortName",
"intelDivisionShortName",
]
null = "<null>"
def __init__(self):
self._cfg = Config()
self.server = Server(self._cfg.LDAP_SERVER, get_info=ALL)
self.connection = Connection(
self.server, user=self._cfg.LDAP_USER, password=self._cfg.LDAP_PASSWORD, auto_bind=True
)
self.connection.bind()
def get_user_emails(self, groups=None):
"""Gets emails of LDAP groups and sub-groups"""
print("\nGet emails from LDAP groups:")
processed_ldap_members = {}
def process_group_members(member, parent_group):
if member in processed_ldap_members:
processed_ldap_members[member]["parent_groups"].append(parent_group)
print(
"\nWARNING: Ignore LDAP member to avoid duplication and recursive cycling "
f"of PDLs: {member}\n "
f'email: {processed_ldap_members[member].get("email")}\n parent_groups:'
)
for group in processed_ldap_members[member].get("parent_groups", []):
print(7 * " ", group)
return
processed_ldap_members[member] = {"email": None, "parent_groups": [parent_group]}
# AD moves terminated users to the boneyard OU in case the user returns,
# so it can be reactivated with little effort.
# After 30 days it is removed and the unix personality becomes unlinked.
if "OU=Boneyard" in member:
return
self.connection.search(
member, r"(objectClass=*)", SUBTREE, attributes=["cn", "member", "mail"]
)
# print(self.connection.entries)
if not self.connection.response:
raise LdapApiException(f"ERROR: empty response. LDAP member: {member}")
# Check that the member is worker.
# The response can contain several items, but the first item is valid only
if "OU=Workers" in member:
if self.connection.response[0]["attributes"]["mail"]:
processed_ldap_members[member]["email"] = self.connection.response[0][
"attributes"
]["mail"].lower()
return
raise LdapApiException(
f"ERROR: no mail. LDAP worker: {member}\n" f"{self.connection.entries}"
)
if len(self.connection.response) > 1:
raise LdapApiException(
f"ERROR: multiple responses for {member}: "
f"{len(self.connection.response)}\n"
f"{self.connection.entries}"
)
if self.connection.response[0]["attributes"]["member"]:
for group_member in self.connection.response[0]["attributes"]["member"]:
process_group_members(group_member, member)
else:
print(f"\nERROR: no members in LDAP group: {member}\n{self.connection.entries}")
for group in groups or self._cfg.LDAP_PDLs:
print("\nProcess ROOT LDAP group:", group)
process_group_members(group, "ROOT")
return {
member.get("email") for member in processed_ldap_members.values() if member.get("email")
}
def _get_user_info(self, query):
"""Gets user info from LDAP as dict matching key and values pairs from query"""
query_filter = "".join(f"({key}={value})" for key, value in query.items())
for domain in self._cfg.LDAP_DOMAINS:
search_base = f"OU=Workers,DC={domain},DC=corp,DC=intel,DC=com"
self.connection.search(
search_base,
f"(&(objectcategory=person)(objectclass=user)(intelflags=1){query_filter})",
SUBTREE,
attributes=["*"],
)
if self.connection.response:
if len(self.connection.response) > 1:
raise LdapApiException(
f"ERROR: multiple responses for {query_filter}: "
f"{len(self.connection.response)}\n"
f"{self.connection.entries}"
)
info = self.connection.response[0]["attributes"]
# remove long binary blobs
for blob in LdapApi._binary_blobs:
info[blob] = b""
for key in LdapApi._check_existing:
if not info.get(key):
info[key] = LdapApi.null
return info
return {}
def get_user_info_by_idsid(self, idsid):
"""Gets user info from LDAP as dict using account name for searching"""
return self._get_user_info({"sAMAccountName": idsid})
def get_user_info_by_name(self, name):
"""Gets user info from LDAP as dict using common name for searching"""
return self._get_user_info({"cn": name})
def get_user_info_by_email(self, email):
"""Gets user info from LDAP as dict using emails for searching"""
return self._get_user_info({"mail": email})
def get_absent_emails(self, emails):
"""Checks users by email in LDAP and returns absent emails"""
absent_emails = set()
for email in emails:
if not self.get_user_info_by_email(email):
absent_emails.add(email)
return absent_emails
def _test():
"""Test and debug"""
ldap = LdapApi()
emails = ldap.get_user_emails()
print(f'\nLDAP emails count: {len(emails)}\n{"; ".join(emails)}')
emails = ["foo@intel.com"]
for email in emails:
info = ldap.get_user_info_by_email(email)
if info:
print_user_info(info, InfoLevel.PDL)
else:
print(f"\n{email} - not found")
if __name__ == "__main__":
_test()

View File

@@ -1 +0,0 @@
pylint==2.11.1

View File

@@ -1,2 +0,0 @@
PyGithub==1.55
ldap3==2.7

94
.github/org_control/check_org.py vendored Normal file
View File

@@ -0,0 +1,94 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Check GitHub organization and invite members
"""
# pylint: disable=fixme,no-member,too-many-locals
from argparse import ArgumentParser
from configs import Config
from github_api import GithubOrgApi, get_dev_emails
from ldap_api import LdapApi, print_user_info, InfoLevel
def main():
"""The main entry point function"""
arg_parser = ArgumentParser()
arg_parser.add_argument("--cfg-file", metavar="PATH", default=Config.default_cfg_path,
help=f"Path to json configuration file, e.g. {Config.default_cfg_path}")
arg_parser.add_argument("--teams", action="store_true", help="Check GitHub teams")
arg_parser.add_argument("--no-ldap", action="store_true", help="Don't use LDAP info")
args, unknown_args = arg_parser.parse_known_args()
Config(args.cfg_file, unknown_args)
gh_api = GithubOrgApi()
if args.teams:
gh_api.get_org_teams()
return
cfg_emails = get_dev_emails()
print(f'\nCfg developer emails {len(cfg_emails)}:', '; '.join(sorted(cfg_emails)))
dev_emails = set()
dev_emails.update(cfg_emails)
if not args.no_ldap:
ldap_api = LdapApi()
ldap_emails = ldap_api.get_user_emails()
dev_emails.update(ldap_emails)
print(f'\nLDAP developer emails {len(ldap_emails)}:', '; '.join(sorted(ldap_emails)))
cfg_emails_no_in_ldap = ldap_api.get_absent_emails(cfg_emails)
print(f'\nCfg developer emails - absent in LDAP at all {len(cfg_emails_no_in_ldap)}:',
'; '.join(sorted(cfg_emails_no_in_ldap)))
cfg_ldap_inters = cfg_emails.intersection(ldap_emails)
print(f'\nCfg developer emails - present in LDAP developers {len(cfg_ldap_inters)}:',
'; '.join(sorted(cfg_ldap_inters)))
org_emails, org_logins_no_intel_email = gh_api.get_org_emails()
print(f'\nOrg emails {len(org_emails)}:', '; '.join(sorted(org_emails)))
org_emails_no_in_ldap = set()
if not args.no_ldap:
org_ldap_diff = org_emails.difference(ldap_emails)
print(f'\nOrg member emails - absent in LDAP developers {len(org_ldap_diff)}:',
'; '.join(sorted(org_ldap_diff)))
for email in org_ldap_diff:
user_info = ldap_api.get_user_info_by_email(email)
if user_info:
print_user_info(user_info, InfoLevel.PDL)
else:
org_emails_no_in_ldap.add(email)
org_pendig_invitation_emails = gh_api.get_org_invitation_emails()
invite_emails = dev_emails.difference(org_emails).difference(org_pendig_invitation_emails)
print(f'\nInvite emails {len(invite_emails)}:', '; '.join(sorted(invite_emails)))
valid_github_users = gh_api.get_valid_github_users(invite_emails)
gh_api.invite_users(valid_github_users)
print('\nCheck accounts below and remove from the GitHub organization and cfg list')
cfg_emails_no_in_org = sorted(cfg_emails.difference(org_emails))
print(f'\nCfg developer emails - absent in GitHub organization {len(cfg_emails_no_in_org)}:',
'; '.join(cfg_emails_no_in_org))
org_emails_no_in_dev = sorted(org_emails.difference(dev_emails))
print(f'\nOrg member emails - absent in cfg and LDAP developers {len(org_emails_no_in_dev)}:',
'; '.join(org_emails_no_in_dev))
print(f'\nOrg member emails - absent in LDAP at all {len(org_emails_no_in_ldap)}:',
'; '.join(sorted(org_emails_no_in_ldap)))
print(f'\nOrg member logins - absent Intel email {len(org_logins_no_intel_email)}:',
'; '.join(sorted(org_logins_no_intel_email)))
if __name__ == '__main__':
main()

View File

@@ -10,13 +10,11 @@ Check GitHub PRs and set labels by type and categories, e.g. 'ExternalPR', 'cate
import re
import sys
import datetime
from enum import Enum
from pathlib import Path
from argparse import ArgumentParser
from enum import Enum
sys.path.append(str(Path(__file__).resolve().parents[1]))
from github_org_control import github_api
from github_org_control.configs import Config
import github_api
from configs import Config
class PrType(Enum):

38
.github/org_control/config.json vendored Normal file
View File

@@ -0,0 +1,38 @@
{
"GITHUB_TOKEN": "<Put token here or set as arg or as env variable>",
"GITHUB_ORGANIZATION": "openvinotoolkit",
"GITHUB_REPO": "openvino",
"IGNORE_LOGINS": [
"openvino-ci",
"openvino-pushbot",
"lab-nerval",
"lab-nerval-onnx-ci",
"onnx-watchdog-agent",
"dependabot"
],
"EMAILS_FILE_PATH": "dev_emails-test.txt",
"PROXIES": {
"HTTP_PROXY": null,
"HTTPS_PROXY": null,
"NO_PROXY": "localhost,127.0.0.1,.intel.com"
},
"DRY_RUN": false,
"TEAM_TO_LABEL": {
"openvino-admins": "category: CI",
"openvino-maintainers": "category: IE common",
"openvino-docs-maintainers": "category: docs",
"openvino-ie-maintainers": "category: IE common",
"openvino-ie-cpu-maintainers": "category: CPU",
"openvino-ie-gna-maintainers": "category: GNA",
"openvino-ie-gpu-maintainers": "category: GPU",
"openvino-ie-lpt-maintainers": "category: LP transformations",
"openvino-ie-multi-maintainers": "category: MULTI",
"openvino-ie-python-api-maintainers": "category: python api",
"openvino-ie-tests-maintainers": "category: IE Tests",
"openvino-ie-vpu-maintainers": "category: VPU",
"openvino-mo-maintainers": "category: MO",
"openvino-ngraph-maintainers": "category: nGraph",
"openvino-tests-maintainers": "category: IE Tests",
"openvino-tools-maintainers": "category: tools"
}
}

113
.github/org_control/configs.py vendored Normal file
View File

@@ -0,0 +1,113 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Configurations management
"""
# pylint: disable=fixme,broad-except
import os
import sys
import ast
import json
from pathlib import Path
if sys.hexversion < 0x3060000:
raise Exception('Python version must be >= 3.6')
class ConfigException(Exception):
"""Base configuration exception"""
class Config:
"""Configuration wrapper"""
_instance = None
properties = None
default_cfg_path = Path(__file__).resolve().parent / 'config.json'
def __new__(cls, *_args, **_kwargs):
if not Config._instance:
Config._instance = super(Config, cls).__new__(cls)
return Config._instance
def __init__(self, file_path=None, cli_args=None):
"""
:param file_path: Path to json configuration file
:type file_path: String
:param args: List of argparse arguments with patterns: 'name=value' or 'name'
:type args: list
"""
if Config.properties:
return
self._file_path = file_path or Config.default_cfg_path
self._cli_args = cli_args or []
self._json_cfg = {}
self._args = {}
self._load_cfg()
self._parse_cli_args()
Config.properties = {}
for name, value in self._json_cfg.items():
if hasattr(self, name):
raise ConfigException(f'Duplicating prosperity: {name}')
property_value = self._args.get(name) or os.getenv(name)
if property_value:
# Try to set prosperity_value as Python literal structures, e.g. DRY_RUN=False
try:
property_value = ast.literal_eval(property_value)
except Exception:
pass
if not isinstance(property_value, type(value)):
raise ConfigException(f'Python type of {name} parameter must be {type(value)}')
else:
property_value = value
setattr(self, name, property_value)
Config.properties[name] = property_value
self.set_proxy()
def _load_cfg(self):
"""Load the json configuration file"""
try:
with open(self._file_path) as conf:
self._json_cfg = json.load(conf)
except Exception:
print('Failed to load configuration from:', self._file_path)
raise
def _parse_cli_args(self):
"""Parse argparse arguments with patterns: 'name=value' or 'name'"""
for cli_arg in self._cli_args:
arg = cli_arg.split('=')
if arg[0] not in self._json_cfg:
raise ConfigException(f'Unsupported argument: {arg}')
self._args[arg[0]] = True if len(arg) == 1 else '='.join(arg[1:])
def get_properties(self):
"""Get all properties as Dict"""
return self.properties
def set_proxy(self):
"""Set proxies"""
for proxy_name, url in self.properties['PROXIES'].items():
if url is not None:
print(f'Set proxy: {proxy_name}={url}')
os.environ[proxy_name] = url
def _test():
"""Test and debug"""
print('Config.default_cfg_path:', Config.default_cfg_path)
cfg = Config(cli_args=['DRY_RUN', 'PROXIES={"NO_PROXY": "localhost"}'])
print('Config.properties:', cfg.get_properties())
if __name__ == '__main__':
_test()

282
.github/org_control/github_api.py vendored Normal file
View File

@@ -0,0 +1,282 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
GitHub API for controlling organization
"""
# pylint: disable=fixme,no-member
import re
import time
from github import Github, GithubException, RateLimitExceededException, IncompletableObject
from github.PaginatedList import PaginatedList
from configs import Config
def is_valid_user(user):
"""Checks that user is valid github.Github object"""
try:
return user and user.login
except IncompletableObject:
return False
def is_user_ignored(user):
"""Checks that user should be ignored"""
cfg = Config()
if is_valid_user(user) and user.login.lower() not in cfg.properties['IGNORE_LOGINS']:
return False
return True
def is_valid_name(name):
"""Checks that GitHub user's name is valid"""
return name and len(name) >= 3 and ' ' in name
def is_intel_email(email):
"""Checks that email is valid Intel email"""
return email and len(email) > 10 and ' ' not in email and email.lower().endswith('@intel.com')
def is_intel_company(company):
"""Checks that company contains intel"""
return company and 'intel' in company.lower()
def is_valid_intel_user(user):
"""Checks that user is valid GitHub and Intel user"""
return is_valid_user(user) and (is_valid_name(user.name) and is_intel_email(user.email) or
is_user_ignored(user))
def print_users(users):
"""Print list of users in different formats: list, set, PaginatedList"""
if isinstance(users, (list, set, PaginatedList)):
users_count = users.totalCount if isinstance(users, PaginatedList) else len(users)
print(f'\nGitHub users {users_count} (login - name - company - email - valid):')
else:
users = [users]
for user in users:
if not is_valid_user(user):
print('WRONG GitHub user: ???')
continue
valid_check = 'OK' if is_valid_intel_user(user) else 'FIX'
if not is_intel_email(user.email):
valid_check += ' email'
if not is_valid_name(user.name):
valid_check += ' name'
print(f'{user.login} - "{user.name}" - "{user.company}" - {user.email} - {valid_check}')
def get_dev_emails():
"""
Read a file with developer emails. Supported email formats
first_name.last_name@intel.com
Import from Outlook: Last_name, First_name <first_name.last_name@intel.com>
"""
re_email = re.compile(r'.+<(.+)>')
emails = set()
cfg = Config()
with open(cfg.properties['EMAILS_FILE_PATH']) as file_obj:
for line in file_obj:
line = line.strip().lower()
if not line or line.startswith('#'):
continue
re_outlook_email = re_email.match(line)
if re_outlook_email:
line = re_outlook_email.group(1).strip()
if not is_intel_email(line):
print(f'Wrong email in {cfg.properties["EMAILS_FILE_PATH"]}: {line}')
continue
emails.add(line)
return emails
class GithubOrgApi:
"""Common API for GitHub organization"""
def __init__(self):
self._cfg = Config()
self.github = Github(self._cfg.GITHUB_TOKEN)
self.github_org = self.github.get_organization(self._cfg.GITHUB_ORGANIZATION)
self.repo = self.github.get_repo(f'{self._cfg.GITHUB_ORGANIZATION}/'
f'{self._cfg.GITHUB_REPO}')
def is_org_user(self, user):
"""Checks that user is a member of GitHub organization"""
if is_valid_user(user):
# user.get_organization_membership(self.github_org) doesn't work with org members
# permissions, GITHUB_TOKEN must be org owner now
return self.github_org.has_in_members(user)
return False
def get_org_emails(self):
"""Gets and prints emails of all GitHub organization members"""
org_members = self.github_org.get_members()
org_emails = set()
org_members_fix = set()
org_emails_fix_name = set()
org_logins_fix_intel_email = set()
print(f'\nOrg members {org_members.totalCount} (login - name - company - email - valid):')
for org_member in org_members:
print_users(org_member)
if is_user_ignored(org_member):
continue
if is_intel_email(org_member.email):
org_emails.add(org_member.email.lower())
if not is_valid_name(org_member.name):
org_members_fix.add(org_member)
org_emails_fix_name.add(org_member.email.lower())
else:
org_members_fix.add(org_member)
org_logins_fix_intel_email.add(org_member.login.lower())
print_users(org_members_fix)
print(f'\nOrg members - no Intel emails {len(org_logins_fix_intel_email)}:',
'; '.join(org_logins_fix_intel_email))
print(f'\nOrg members - no real name {len(org_emails_fix_name)}:',
'; '.join(org_emails_fix_name))
return (org_emails, org_logins_fix_intel_email)
def get_org_invitation_emails(self):
"""Gets GitHub organization teams prints info"""
org_invitations = self.github_org.invitations()
org_invitation_emails = set()
print(f'\nOrg invitations {org_invitations.totalCount} (login - name - email - valid):')
for org_invitation in org_invitations:
# TODO: investigate GithubException while access to user name and enable print_users()
# github.GithubException.IncompletableObject: 400 "Returned object contains no URL"
#print_users(org_invitation)
print(f'{org_invitation.login} - ??? - {org_invitation.email} - ???')
if is_user_ignored(org_invitation):
continue
if is_intel_email(org_invitation.email):
org_invitation_emails.add(org_invitation.email.lower())
else:
print('Strange org invitation:', org_invitation)
print(f'\nOrg invitation emails {len(org_invitation_emails)}:',
'; '.join(org_invitation_emails))
return org_invitation_emails
def get_org_teams(self):
"""Gets GitHub organization teams prints info"""
teams = []
org_teams = self.github_org.get_teams()
print('\nOrg teams count:', org_teams.totalCount)
for team in org_teams:
teams.append(team.name)
print(f'\nTeam: {team.name} - parent: {team.parent}')
repos = team.get_repos()
print('Repos:')
for repo in repos:
print(f' {repo.name} -', team.get_repo_permission(repo))
team_maintainers = team.get_members(role='maintainer')
team_maintainer_logins = set()
for maintainer in team_maintainers:
team_maintainer_logins.add(maintainer.login)
team_members = team.get_members(role='member')
team_member_logins = set()
for member in team_members:
team_member_logins.add(member.login)
members = team.get_members(role='all')
member_emails = []
print('Members (role - login - name - company - email - valid):')
for user in members:
if user.login in team_maintainer_logins:
print(' Maintainer - ', end='')
elif user.login in team_member_logins:
print(' Member - ', end='')
else:
# It is not possible to check child teams members
print(' ??? - ', end='')
print_users(user)
if is_intel_email(user.email) and not is_user_ignored(user):
member_emails.append(user.email.lower())
print(f'Intel emails {len(member_emails)}:', '; '.join(member_emails))
return teams
def get_valid_github_users(self, emails):
"""Gets valid GitHub users by email and prints status"""
valid_users = set()
no_account_emails = set()
print(f'\nGitHub users from {len(emails)} invite emails (email - status):')
for email in emails:
if not is_intel_email(email):
print(f'{email} - Non Intel email')
continue
# You can make up to 30 requests per minute; https://developer.github.com/v3/search/
# Sleep 2.4 sec is about 25 requests per minute
time.sleep(2.4)
try:
users = self.github.search_users(f'{email} in:email')
except RateLimitExceededException:
time.sleep(5)
users = self.github.search_users(f'{email} in:email')
if users.totalCount == 0:
print(f'{email} - No valid GitHub account')
no_account_emails.add(email)
continue
if users.totalCount > 1:
print(f'{email} - Found {users.totalCount} GitHub accounts')
for user in users:
if user.email and user.email.lower() == email:
print(f'{email} - OK')
valid_users.add(user)
else:
print(f'{email} - Non public or wrong email - login: {user.login} - '
f'email: {user.email}')
print('Valid users count:', len(valid_users))
print_users(valid_users)
print(f'\nIntel emails - No valid GitHub account {len(no_account_emails)}:',
'; '.join(no_account_emails))
return valid_users
def invite_users(self, users):
"""Invites users and prints status"""
if isinstance(users, (list, set)):
print(f'\nInvite {len(users)} users:')
else:
users = [users]
for user in users:
if isinstance(user, str):
print(f'Email: {user}')
self.github_org.invite_user(email=user)
else:
print(f'{user.login} - "{user.name}" - {user.email} - ', end='')
try:
if is_user_ignored(user):
print('Ignored')
continue
if not self._cfg.DRY_RUN:
self.github_org.invite_user(user=user)
print('OK')
else:
print('Dry run')
except GithubException as exc:
print(f'FAIL: {exc.data["errors"][0]["message"]}')
def _test():
"""Test and debug"""
Config(cli_args=['DRY_RUN=True'])
dev_emails = get_dev_emails()
print('dev_emails:', dev_emails)
gh_api = GithubOrgApi()
gh_api.get_org_emails()
if __name__ == '__main__':
_test()

236
.github/org_control/ldap_api.py vendored Normal file
View File

@@ -0,0 +1,236 @@
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
"""
Gets info about users and groups via LDAP
"""
# pylint: disable=fixme,no-member
from enum import Enum
from ldap3 import Server, Connection, ALL, SUBTREE
from configs import Config
class LdapApiException(Exception):
"""Base LDAP API exception"""
class InfoLevel(Enum):
"""Constants for printing user info from LDAP"""
PDL = 'PDL' # Public Distribution List (group of e-mail addresses)
FULL = 'Full'
def print_user_info(info, info_level=None):
"""Pretty-print of a user info data structure (dict). info_level is the InfoLevel Enum"""
if not info or not info.get('mail'):
raise LdapApiException('ERROR: No info or absent mail')
def get_membership():
if info_level == InfoLevel.PDL:
membership_info = ' PDLs:'
elif info_level == InfoLevel.FULL:
membership_info = ' memberOf :'
else:
return ''
# Grouping groups by purpose
if info_level == InfoLevel.PDL:
sort_key = lambda i: i.split(',', 1)[0].lower()
else:
sort_key = lambda i: i.split(',', 1)[1] + i.split(',', 1)[0].lower()
for item in sorted(info['memberOf'], key=sort_key):
if info_level == InfoLevel.PDL and 'OU=Delegated' not in item:
continue
membership_info += f'\n {item}'
return membership_info
try:
text_info = \
f'\n{info["cn"]} <{info["mail"]}>; {info["sAMAccountName"]}; {info["employeeID"]}' \
f'\n Org group: {info["intelSuperGroupDescr"]} ({info["intelSuperGroupShortName"]}) /'\
f' {info["intelGroupDescr"]} ({info["intelGroupShortName"]}) /' \
f' {info["intelDivisionDescr"]} ({info["intelDivisionShortName"]}) /' \
f' {info["intelOrgUnitDescr"]}' \
f'\n Manager: {info["manager"]}' \
f'\n Location: {info["intelRegionCode"]} / {info["co"]} / {info["intelSiteCode"]} /' \
f' {info["intelBldgCode"]} ({info["intelSiteName"]}) /' \
f' {info["physicalDeliveryOfficeName"]}' \
f'\n Other: {info["employeeType"]} | {info["intelExportCountryGroup"]} |' \
f' {info["whenCreated"]} | {info["intelCostCenterDescr"]} | {info["jobDescription"]}'
except Exception as exc:
raise LdapApiException(f'ERROR: Failed to get info about "{info["mail"]}". ' \
f'Exception occurred:\n{repr(exc)}') from exc
print(text_info)
membership = get_membership()
if info_level == InfoLevel.PDL and membership:
print(membership)
elif info_level == InfoLevel.FULL:
for key in sorted(info):
if isinstance(info[key], list):
if key == 'memberOf':
print(membership)
else:
print(f' {key} :')
for item in info[key]:
print(' ', item)
else:
print(f' {key} : {info[key]}')
class LdapApi:
"""LDAP API for getting user info and emails"""
_binary_blobs = ['thumbnailPhoto', 'msExchUMSpokenName', 'msExchBlockedSendersHash']
_check_existing = [
'intelExportCountryGroup',
'physicalDeliveryOfficeName',
'intelSuperGroupShortName',
'intelGroupShortName',
'intelDivisionShortName',
]
null = '<null>'
def __init__(self):
self._cfg = Config()
self.server = Server(self._cfg.LDAP_SERVER, get_info=ALL)
self.connection = Connection(self.server,
user=self._cfg.LDAP_USER,
password=self._cfg.LDAP_PASSWORD,
auto_bind=True)
self.connection.bind()
def get_user_emails(self, groups=None):
"""Gets emails of LDAP groups and sub-groups"""
print('\nGet emails from LDAP groups:')
processed_ldap_members = {}
def process_group_members(member, parent_group):
if member in processed_ldap_members:
processed_ldap_members[member]['parent_groups'].append(parent_group)
print('\nWARNING: Ignore LDAP member to avoid duplication and recursive cycling '
f'of PDLs: {member}\n '
f'email: {processed_ldap_members[member].get("email")}\n parent_groups:')
for group in processed_ldap_members[member].get('parent_groups', []):
print(7 * ' ', group)
return
processed_ldap_members[member] = {'email': None, 'parent_groups': [parent_group]}
# AD moves terminated users to the boneyard OU in case the user returns,
# so it can be reactivated with little effort.
# After 30 days it is removed and the unix personality becomes unlinked.
if 'OU=Boneyard' in member:
return
self.connection.search(member, r'(objectClass=*)', SUBTREE,
attributes=['cn', 'member', 'mail'])
#print(self.connection.entries)
if not self.connection.response:
raise LdapApiException(f'ERROR: empty response. LDAP member: {member}')
# Check that the member is worker.
# The response can contain several items, but the first item is valid only
if 'OU=Workers' in member:
if self.connection.response[0]['attributes']['mail']:
processed_ldap_members[member]['email'] = \
self.connection.response[0]['attributes']['mail'].lower()
return
raise LdapApiException(f'ERROR: no mail. LDAP worker: {member}\n'
f'{self.connection.entries}')
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {member}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
if self.connection.response[0]['attributes']['member']:
for group_member in self.connection.response[0]['attributes']['member']:
process_group_members(group_member, member)
else:
print(f'\nERROR: no members in LDAP group: {member}\n{self.connection.entries}')
for group in groups or self._cfg.LDAP_PDLs:
print('\nProcess ROOT LDAP group:', group)
process_group_members(group, 'ROOT')
return {
member.get('email') for member in processed_ldap_members.values() if member.get('email')
}
def _get_user_info(self, query):
"""Gets user info from LDAP as dict matching key and values pairs from query"""
query_filter = ''.join(f'({key}={value})' for key, value in query.items())
for domain in self._cfg.LDAP_DOMAINS:
search_base = f'OU=Workers,DC={domain},DC=corp,DC=intel,DC=com'
self.connection.search(
search_base,
f'(&(objectcategory=person)(objectclass=user)(intelflags=1){query_filter})',
SUBTREE,
attributes=['*'])
if self.connection.response:
if len(self.connection.response) > 1:
raise LdapApiException(f'ERROR: multiple responses for {query_filter}: '
f'{len(self.connection.response)}\n'
f'{self.connection.entries}')
info = self.connection.response[0]['attributes']
# remove long binary blobs
for blob in LdapApi._binary_blobs:
info[blob] = b''
for key in LdapApi._check_existing:
if not info.get(key):
info[key] = LdapApi.null
return info
return {}
def get_user_info_by_idsid(self, idsid):
"""Gets user info from LDAP as dict using account name for searching"""
return self._get_user_info({'sAMAccountName': idsid})
def get_user_info_by_name(self, name):
"""Gets user info from LDAP as dict using common name for searching"""
return self._get_user_info({'cn': name})
def get_user_info_by_email(self, email):
"""Gets user info from LDAP as dict using emails for searching"""
return self._get_user_info({'mail': email})
def get_absent_emails(self, emails):
"""Checks users by email in LDAP and returns absent emails"""
absent_emails = set()
for email in emails:
if not self.get_user_info_by_email(email):
absent_emails.add(email)
return absent_emails
def _test():
"""Test and debug"""
ldap = LdapApi()
emails = ldap.get_user_emails()
print(f'\nLDAP emails count: {len(emails)}\n{"; ".join(emails)}')
emails = ['foo@intel.com']
for email in emails:
info = ldap.get_user_info_by_email(email)
if info:
print_user_info(info, InfoLevel.PDL)
else:
print(f'\n{email} - not found')
if __name__ == '__main__':
_test()

View File

@@ -0,0 +1 @@
pylint==2.5.3

2
.github/org_control/requirements.txt vendored Normal file
View File

@@ -0,0 +1,2 @@
PyGithub==1.51
ldap3==2.7

View File

@@ -1,144 +1,47 @@
name: Documentation
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
Build_Doc:
if: github.repository == 'openvinotoolkit/openvino'
runs-on: ubuntu-22.04
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@v2
with:
submodules: true
submodules: recursive
lfs: true
# cache from ccache
- name: Cache ccache
id: cache_restore
uses: actions/cache@v3
with:
path: .ccache
key: doc-ccache-cache
- name: Install ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: doc-ccache-cache
save: false
# cache from doxygen build
- name: Cache doxygen make
id: cache_doxygen_make
uses: actions/cache@v3
with:
path: doxygen-build
key: build-doxygen-cache
# cache from python packages
- name: Cache python requirements
id: cache_python_requirements
uses: actions/cache@v3
with:
path: ~/.cache/pip
key: doc-python-requirements
- name: Install apt-get tools
uses: awalsh128/cache-apt-pkgs-action@v1.1.3
# install doc dependencies
with:
packages: graphviz texlive liblua5.2-0
version: 3.0
- name: Install dependencies
run: |
set -e
python3 -m pip install -r docs/requirements.txt --user
cd docs/openvino_sphinx_theme
python3 setup.py install --user
cd ../..
# install doxyrest
wget https://github.com/vovkos/doxyrest/releases/download/doxyrest-2.1.3/doxyrest-2.1.3-linux-amd64.tar.xz
tar -xf doxyrest-2.1.3-linux-amd64.tar.xz
echo "$(pwd)/doxyrest-2.1.3-linux-amd64/bin/" >> $GITHUB_PATH
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev graphviz texlive
python3 -m pip install lxml
# install doxygen
mkdir doxygen
cd doxygen
git clone --depth 1 https://github.com/doxygen/doxygen.git -b Release_1_9_2
cmake \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-S doxygen \
-B ../doxygen-build
cmake --build ../doxygen-build -j`nproc`
sudo cmake --build ../doxygen-build --target install
git clone https://github.com/doxygen/doxygen.git
cd doxygen
git checkout Release_1_9_1
mkdir build
cd build
cmake ..
cmake --build . -j`nproc`
sudo make install
- name: CMake doc
run: |
cmake \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DENABLE_INTEL_MYRIAD_COMMON=OFF \
-DENABLE_DOCS=ON \
-DCMAKE_BUILD_TYPE=Release \
-B build
mkdir build
cd build
echo "TBD"
- name: Cache documentation
id: cache_sphinx_docs
uses: actions/cache@v3
with:
path: build/docs/_build/.doctrees
key: sphinx-docs-cache
- name: Build doc
run: |
cmake --build build --target sphinx_docs -j`nproc`
- name: Archive HTML
run: |
zip -r openvino_html.zip _build
working-directory: build/docs
- name: Run Pytest
run: |
pytest --doxygen="./build/docs/doxygen.log" \
--include_pot \
--sphinx="./build/docs/sphinx.log" \
--suppress-warnings="./docs/suppress_warnings.txt" \
--confcutdir="./docs/scripts/tests/" \
--html="./build/docs/_artifacts/doc-generation.html" \
--doxygen-strip="$(pwd)" \
--sphinx-strip="$(pwd)/build/docs/rst" \
--doxygen-xfail="./docs/doxygen-xfail.txt" \
--self-contained-html ./docs/scripts/tests/test_docs.py
run: echo "TBD"
working-directory: build
- name: 'Upload test results'
if: always()
- name: 'Upload doc'
if: github.event_name == 'push'
uses: actions/upload-artifact@v2
with:
name: openvino_doc_pytest
path: build/docs/_artifacts/
- name: 'Upload doxygen.log'
if: always()
uses: actions/upload-artifact@v2
with:
name: doxygen_log
path: build/docs/doxygen.log
- name: 'Upload sphinx.log'
if: always()
uses: actions/upload-artifact@v2
with:
name: sphinx_log
path: build/docs/sphinx.log
- name: 'Upload html'
if: always()
uses: actions/upload-artifact@v2
with:
name: openvino_html
path: build/docs/openvino_html.zip
name: openvino_doc
path: build/docs/html/

View File

@@ -1,21 +1,17 @@
name: PR Commits
on: [pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
Checks:
runs-on: ubuntu-22.04
runs-on: ubuntu-20.04
steps:
- name: Clone OpenVINO
uses: actions/checkout@v2
- name: Install dependencies
run: python3 -m pip install -r ./.github/github_org_control/requirements.txt
run: python3 -m pip install -r ./.github/org_control/requirements.txt
- name: PR commits
run: python3 ./.github/github_org_control/check_pr.py --pr=${{ github.event.number }} --check-commits DRY_RUN
run: python3 ./.github/org_control/check_pr.py --pr=${{ github.event.number }} --check-commits DRY_RUN
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -1,48 +0,0 @@
name: Code snippets
on:
push:
paths:
- '.github/workflows/code_snippets.yml'
- 'docs/snippets/**'
pull_request:
paths:
- '.github/workflows/code_snippets.yml'
- 'docs/snippets/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
Build:
strategy:
fail-fast: false
matrix:
os: ['ubuntu-22.04', 'macos-latest', 'windows-latest']
runs-on: ${{ matrix.os }}
steps:
- name: Clone OpenVINO
uses: actions/checkout@v2
with:
submodules: recursive
lfs: true
- name: Check cmake
run: |
which cmake
cmake --version
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_INTEL_MYRIAD_COMMON=OFF -DCMAKE_BUILD_TYPE=Release ..
- name: Get number of CPU cores
uses: SimenB/github-actions-cpu-cores@v1
id: cpu-cores
- name: Build snippets
run: |
cmake --build . --target ie_docs_snippets -j${{ steps.cpu-cores.outputs.count }}
working-directory: build

View File

@@ -1,10 +1,6 @@
name: Code Style
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
clang-format:
runs-on: ubuntu-20.04
@@ -23,34 +19,26 @@ jobs:
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install --upgrade pip
python3 -m pip install -r ./src/bindings/python/src/compatibility/openvino/requirements.txt
python3 -m pip install -r ./inference-engine/ie_bridges/python/requirements.txt
# Add for -DENABLE_PYTHON=ON, no cython
python3 -m pip install -r ./src/bindings/python/src/compatibility/openvino/requirements-dev.txt
python3 -m pip install -r ./inference-engine/ie_bridges/python/src/requirements-dev.txt
# Run cmake with -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT in order to enable codestyle check for ITT collector
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_PYTHON=ON -DENABLE_TESTS=ON -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT ..
cmake -DENABLE_PYTHON=ON -DENABLE_PROFILING_ITT=ON -DSELECTIVE_BUILD=COLLECT ..
- name: Check code style
run: cmake --build . --target clang_format_check_all -j8
working-directory: build
run: cmake --build build --target clang_format_check_all
- name: Create code style diff
if: failure()
run: |
cmake --build build --target clang_format_fix_all -j8
cmake --build build --target clang_format_fix_all
git diff > code_style_diff.diff
- name: suggester / clang-format
if: failure() && startsWith(github.event_name, 'pull_request')
uses: reviewdog/action-suggester@v1
with:
level: error
tool_name: clang-format
- uses: actions/upload-artifact@v2
if: failure()
with:
@@ -58,7 +46,7 @@ jobs:
path: code_style_diff.diff
ShellCheck:
runs-on: ubuntu-22.04
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
with:
@@ -70,40 +58,16 @@ jobs:
sudo apt --assume-yes install shellcheck
- name: Install dependencies
run: python3 -m pip install -r ./src/bindings/python/src/compatibility/openvino/requirements.txt
run: |
sudo apt --assume-yes install libusb-1.0-0-dev
python3 -m pip install -r ./inference-engine/ie_bridges/python/requirements.txt
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_INTEL_MYRIAD_COMMON=OFF ..
cmake ..
- name: ShellCheck
run: cmake --build . --target ie_shellcheck -j8
working-directory: build
NamingConventionCheck:
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v2
with:
submodules: recursive
- name: Install Clang dependency
run: |
sudo apt update
sudo apt --assume-yes remove clang-7 clang-8 clang-9 clang-10 clang-11 clang-12 clang-13
sudo apt --assume-yes install libclang-14-dev
- name: Install Python-based dependencies
run: python3 -m pip install -r cmake/developer_package/ncc_naming_style/requirements_dev.txt
- name: CMake
run: |
mkdir build
cd build
cmake -DENABLE_INTEL_MYRIAD_COMMON=OFF ..
- name: Naming convention check
run: cmake --build . --target ncc_all -j8
run: make ie_shellcheck
working-directory: build

View File

@@ -1,13 +1,9 @@
name: Files Size
on: [push, pull_request]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
Check_Files_Size:
runs-on: ubuntu-22.04
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2

View File

@@ -2,18 +2,14 @@ name: MO
on:
push:
paths:
- 'openvino/tools/mo/**'
- 'model-optimizer/**'
pull_request:
paths:
- 'openvino/tools/mo/**'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
- 'model-optimizer/**'
jobs:
Pylint-UT:
runs-on: ubuntu-22.04
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
with:
@@ -22,13 +18,13 @@ jobs:
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:
python-version: 3.7
python-version: 3.6
- name: Cache pip
uses: actions/cache@v1
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('openvino/tools/mo/requirements*.txt') }}
key: ${{ runner.os }}-pip-${{ hashFiles('model-optimizer/requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-
@@ -47,11 +43,11 @@ jobs:
# requrements for CMake
sudo apt update
sudo apt --assume-yes install libusb-1.0-0-dev
working-directory: openvino/tools/mo
working-directory: model-optimizer
- name: Pylint
run: pylint -d C,R,W openvino/tools/mo/ openvino/tools/mo/mo.py
working-directory: openvino/tools/mo
run: pylint -d C,R,W mo/ mo.py extensions/
working-directory: model-optimizer
- name: CMake
run: |
@@ -66,4 +62,43 @@ jobs:
env
mkdir ../mo-ut-logs
python3 -m xmlrunner discover -p *_test.py --output=../mo-ut-logs
working-directory: openvino/tools/mo
working-directory: model-optimizer
build_wheel:
name: Build Python wheel
runs-on: ubuntu-18.04
steps:
- uses: actions/checkout@v2
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install wheel setuptools
python3 -m pip install tensorflow==2.3.0
- name: Build
run: |
python3 setup.py sdist bdist_wheel
working-directory: model-optimizer
- name: Test package content
run: |
echo "src = open('openvino_mo.egg-info/SOURCES.txt', 'rt').read().split()" | tee -a test_wheel.py
echo "ref = open('automation/package_BOM.txt', 'rt').read().split()" | tee -a test_wheel.py
echo "for name in ref:" | tee -a test_wheel.py
echo " if name.endswith('.py'):" | tee -a test_wheel.py
echo " assert name in src or './' + name in src, name + ' file missed'" | tee -a test_wheel.py
python3 test_wheel.py
working-directory: model-optimizer
- name: Test conversion
run: |
wget -q http://download.tensorflow.org/models/mobilenet_v1_2018_08_02/mobilenet_v1_1.0_224.tgz
tar -xf mobilenet_v1_1.0_224.tgz
python3 -m pip install model-optimizer/dist/*.whl
python3 -m mo --input_model mobilenet_v1_1.0_224_frozen.pb --input_shape "[1,224,224,3]"
- uses: actions/upload-artifact@v2
with:
name: mo_wheel
path: "model-optimizer/dist/*.whl"

View File

@@ -1,25 +1,16 @@
name: Python API Checks
name: IE Python Checks
on:
workflow_dispatch:
push:
paths:
- 'src/bindings/python/**'
- 'samples/python/**'
- '.github/workflows/py_checks.yml'
- 'inference-engine/ie_bridges/python/**'
pull_request:
paths:
- 'src/bindings/python/**'
- 'samples/python/**'
- '.github/workflows/py_checks.yml'
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
- 'inference-engine/ie_bridges/python/**'
jobs:
linters:
runs-on: ubuntu-20.04
runs-on: ubuntu-18.04
steps:
- name: Code checkout
uses: actions/checkout@v2
@@ -28,102 +19,57 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: '3.7'
python-version: '3.6'
- name: Install dependencies
run: python -m pip install -r src/bindings/python/requirements_test.txt
# samples code-style
- name: Run flake8 on samples
run: python -m flake8 ./ --config=setup.cfg
working-directory: samples/python
run: python -m pip install -r inference-engine/ie_bridges/python/requirements_dev.txt
- name: Run Flake on samples
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/sample
- name: Create code style diff for samples
if: failure()
run: |
python -m black -l 160 -S ./
git diff > samples_diff.diff
working-directory: samples/python
working-directory: inference-engine/ie_bridges/python/sample
- uses: actions/upload-artifact@v2
if: failure()
with:
name: samples_diff
path: samples_diff.diff
# IE Python API Flake code-style
- name: Run flake8 on IE Python API
run: python -m flake8 ./ --config=setup.cfg
working-directory: src/bindings/python/src/compatibility/openvino
- name: Create code style diff for IE Python API
if: failure()
run: |
python -m black -l 160 -S ./
git diff > ie_python_diff.diff
working-directory: src/bindings/python/src/compatibility/openvino
- uses: actions/upload-artifact@v2
if: failure()
with:
name: ie_python_diff
path: ie_python_diff.diff
# nGraph Python API Flake code-style
- name: Run flake8 on nGraph Python API
run: python -m flake8 ./src/compatibility/ngraph --config=setup.cfg
working-directory: src/bindings/python
- name: Create code style diff for nGraph Python API
if: failure()
run: |
python -m black -l 160 -S ./
git diff > pyngraph_diff.diff
working-directory: src/bindings/python/src/compatibility/ngraph
- uses: actions/upload-artifact@v2
if: failure()
with:
name: pyngraph_diff
path: pyngraph_diff.diff
# Python API 2.0 Flake code-style
- name: Run flake8 on Python API 2.0
run: python -m flake8 ./src/openvino --config=setup.cfg
working-directory: src/bindings/python
- name: Create code style diff for Python API 2.0
if: failure()
run: |
python -m black -l 160 -S ./
git diff > pyopenvino_diff.diff
working-directory: src/bindings/python/src/openvino
- uses: actions/upload-artifact@v2
if: failure()
with:
name: pyopenvino_diff
path: pyopenvino_diff.diff
# wheel Flake code-style
- name: Run flake8 on wheel
- name: Run Flake on src
run: python -m flake8 ./ --config=../setup.cfg
working-directory: src/bindings/python/wheel
working-directory: inference-engine/ie_bridges/python/src
- name: Create code style diff for Python src
if: failure()
run: |
python -m black -l 160 -S ./
git diff > src_diff.diff
working-directory: inference-engine/ie_bridges/python/src
- uses: actions/upload-artifact@v2
if: failure()
with:
name: src_diff
path: src_diff.diff
- name: Run Flake on wheel
run: python -m flake8 ./ --config=../setup.cfg
working-directory: inference-engine/ie_bridges/python/wheel
- name: Create code style diff for wheel
if: failure()
run: |
python -m black -l 160 -S ./
git diff > wheel_diff.diff
working-directory: src/bindings/python/wheel
working-directory: inference-engine/ie_bridges/python/wheel
- uses: actions/upload-artifact@v2
if: failure()
with:
name: wheel_diff
path: wheel_diff.diff
# Python API 2.0 tests Flake code-style
- name: Run flake8 on python tests
# ignore lack of docs in tests
run: python -m flake8 tests/ --config=setup.cfg
working-directory: src/bindings/python
# IE Python API mypy check
- name: Run mypy on IE Python API
- name: Run MyPy
run: python -m mypy ./ --config-file ./setup.cfg
working-directory: src/bindings/python/src/compatibility/openvino
# nGraph Python API mypy check
- name: Run mypy on nGraph Python API
run: python -m mypy ./src/compatibility/ngraph --config-file ./setup.cfg
working-directory: src/bindings/python
# Python API 2.0 mypy check
- name: Run mypy on Python API 2.0
run: python -m mypy ./src/openvino --config-file ./setup.cfg
working-directory: src/bindings/python
working-directory: inference-engine/ie_bridges/python
- name: Run Bandit
run: python -m bandit -r ./ -f screen
working-directory: src/bindings/python/src/compatibility/openvino
working-directory: inference-engine/ie_bridges/python

54
.gitignore vendored
View File

@@ -1,29 +1,30 @@
# build/artifact dirs
_*
[Bb]uild*/
cmake-build*
# but ensure we don't skip __init__.py and __main__.py
# but ensure we don't skip __init__.py
!__init__.py
!__main__.py
# and sphinx documentation folders
!docs/_*
# developer tools
*.idea
.vscode
.vs/
cmake-build-*
.DS_Store
**/tags
compile_commands.json
bin/
build/
.local_vimrc
.gdb_history
.vimspector.json
doc/
!ngraph/doc
docs/build_documentation/work_dir/
temp/
inference-engine/plugins/
inference-engine/temp
inference-engine/report
.repo/
docs/template_plugin/html/
CMakeLists.txt.user
docs/IE_PLUGIN_DG/html/
@@ -32,13 +33,14 @@ docs/IE_PLUGIN_DG/html/
*.pydevproject
*.settings
*/gen/
__pycache__
*.swp
/config.xml
# Python-specific
*.?env*
*.env3
*.pyc
__pycache__
# Tests-specific
*.coverage
*htmlcov
@@ -46,14 +48,26 @@ __pycache__
*pylint_report_comments.txt
# Artifacts
/tools/mo/*.bin
/tools/mo/*.xml
/tools/mo/*.json
/tools/mo/*.so
/tools/mo/*.txt
/tools/mo/*.pb
/tools/mo/*.pbtxt
/tools/mo/!CMakeLists.txt
/tools/mo/*.mapping
/tools/mo/*.dat
/tools/mo/*.svg
/model-optimizer/*.bin
/model-optimizer/*.xml
/model-optimizer/*.json
/model-optimizer/*.so
/model-optimizer/*.txt
/model-optimizer/*.pb
/model-optimizer/*.pbtxt
/model-optimizer/!CMakeLists.txt
/model-optimizer/*.mapping
/model-optimizer/*.dat
/model-optimizer/*.svg
# ngraph
ngraph/src/CPackConfig.cmake
ngraph/src/CPackSourceConfig.cmake
ngraph/src/VERSION
ngraph/src/gtest/
ngraph/src/json/
ngraph/src/ngraphConfig.cmake
ngraph/src/ngraphConfigVersion.cmake
ngraph/src/protobuf/
ngraph/src/src/
ngraph/src/test/

69
.gitmodules vendored
View File

@@ -1,7 +1,19 @@
[submodule "src/plugins/intel_cpu/thirdparty/onednn"]
path = src/plugins/intel_cpu/thirdparty/onednn
[submodule "inference-engine/thirdparty/ade"]
path = inference-engine/thirdparty/ade
url = https://github.com/opencv/ade.git
ignore = dirty
[submodule "inference-engine/thirdparty/mkl-dnn"]
path = inference-engine/thirdparty/mkl-dnn
url = https://github.com/openvinotoolkit/oneDNN.git
ignore = dirty
[submodule "inference-engine/tests/ie_test_utils/common_test_utils/gtest"]
path = inference-engine/tests/ie_test_utils/common_test_utils/gtest
url = https://github.com/openvinotoolkit/googletest.git
ignore = dirty
[submodule "inference-engine/samples/thirdparty/gflags"]
path = inference-engine/samples/thirdparty/gflags
url = https://github.com/gflags/gflags.git
ignore = dirty
[submodule "thirdparty/xbyak"]
path = thirdparty/xbyak
url = https://github.com/herumi/xbyak.git
@@ -10,56 +22,3 @@
path = thirdparty/zlib/zlib
url = https://github.com/madler/zlib.git
ignore = dirty
[submodule "thirdparty/pugixml"]
path = thirdparty/pugixml
url = https://github.com/zeux/pugixml.git
ignore = dirty
[submodule "thirdparty/ade"]
path = thirdparty/ade
url = https://github.com/opencv/ade.git
ignore = dirty
[submodule "thirdparty/gflags/gflags"]
path = thirdparty/gflags/gflags
url = https://github.com/gflags/gflags.git
ignore = dirty
[submodule "thirdparty/gtest/gtest"]
path = thirdparty/gtest/gtest
url = https://github.com/openvinotoolkit/googletest.git
ignore = dirty
[submodule "thirdparty/ocl/icd_loader"]
path = thirdparty/ocl/icd_loader
url = https://github.com/KhronosGroup/OpenCL-ICD-Loader.git
ignore = dirty
[submodule "thirdparty/ocl/cl_headers"]
path = thirdparty/ocl/cl_headers
url = https://github.com/KhronosGroup/OpenCL-Headers.git
ignore = dirty
[submodule "thirdparty/ocl/clhpp_headers"]
path = thirdparty/ocl/clhpp_headers
url = https://github.com/KhronosGroup/OpenCL-CLHPP.git
ignore = dirty
[submodule "thirdparty/onnx"]
path = thirdparty/onnx/onnx
url = https://github.com/onnx/onnx.git
[submodule "thirdparty/protobuf"]
path = thirdparty/protobuf/protobuf
url = https://github.com/protocolbuffers/protobuf.git
[submodule "src/bindings/python/thirdparty/pybind11"]
path = src/bindings/python/thirdparty/pybind11
url = https://github.com/pybind/pybind11.git
[submodule "thirdparty/ittapi/ittapi"]
path = thirdparty/ittapi/ittapi
url = https://github.com/intel/ittapi.git
[submodule "ncc"]
path = cmake/developer_package/ncc_naming_style/ncc
url = https://github.com/nithinn/ncc.git
[submodule "thirdparty/onednn_gpu"]
path = src/plugins/intel_gpu/thirdparty/onednn_gpu
url = https://github.com/oneapi-src/oneDNN.git
[submodule "tools/pot/thirdparty/open_model_zoo"]
path = thirdparty/open_model_zoo
url = https://github.com/openvinotoolkit/open_model_zoo.git
[submodule "thirdparty/json/nlohmann_json"]
path = thirdparty/json/nlohmann_json
url = https://github.com/nlohmann/json.git
shallow = true

View File

@@ -1,30 +1,16 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(DEFINED BUILD_SHARED_LIBS AND NOT BUILD_SHARED_LIBS)
# 'target_link_libraries' does not work correctly when called from
# different directory where 'add_library' is called: CMake generates
# incorrect OpenVINOConfig.cmake in this case
cmake_minimum_required(VERSION 3.17)
else()
if(CPACK_GENERATOR STREQUAL "DEB")
# we have to use CPACK_DEBIAN_PACKAGE_SHLIBDEPS_PRIVATE_DIRS variable
cmake_minimum_required(VERSION 3.20)
else()
# default choice
cmake_minimum_required(VERSION 3.13)
endif()
endif()
cmake_minimum_required(VERSION 3.13)
project(OpenVINO DESCRIPTION "OpenVINO toolkit")
project(OpenVINO)
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "CMake build type" FORCE)
endif()
set(OpenVINO_MAIN_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
set(IE_MAIN_SOURCE_DIR ${OpenVINO_MAIN_SOURCE_DIR}/inference-engine)
find_package(IEDevScripts REQUIRED
PATHS "${OpenVINO_SOURCE_DIR}/cmake/developer_package"
PATHS "${OpenVINO_MAIN_SOURCE_DIR}/cmake/developer_package"
NO_CMAKE_FIND_ROOT_PATH
NO_DEFAULT_PATH)
@@ -34,40 +20,102 @@ include(cmake/features.cmake)
# These options are shared with 3rdparty plugins by means of developer package
include(cmake/dependencies.cmake)
if(ENABLE_COVERAGE)
include(cmake/coverage.cmake)
endif()
# resolving dependencies for the project
message (STATUS "PROJECT ............................... " ${PROJECT_NAME})
message (STATUS "CMAKE_VERSION ......................... " ${CMAKE_VERSION})
message (STATUS "CMAKE_BINARY_DIR ...................... " ${CMAKE_BINARY_DIR})
message (STATUS "CMAKE_SOURCE_DIR ...................... " ${CMAKE_SOURCE_DIR})
message (STATUS "OpenVINO_SOURCE_DIR ................... " ${OpenVINO_SOURCE_DIR})
message (STATUS "OpenVINO_BINARY_DIR ................... " ${OpenVINO_BINARY_DIR})
message (STATUS "OpenVINO_MAIN_SOURCE_DIR .............. " ${OpenVINO_MAIN_SOURCE_DIR})
message (STATUS "IE_MAIN_SOURCE_DIR .................... " ${IE_MAIN_SOURCE_DIR})
message (STATUS "CMAKE_GENERATOR ....................... " ${CMAKE_GENERATOR})
message (STATUS "CMAKE_C_COMPILER_ID ................... " ${CMAKE_C_COMPILER_ID})
message (STATUS "CMAKE_CXX_COMPILER_ID ................. " ${CMAKE_CXX_COMPILER_ID})
message (STATUS "CMAKE_BUILD_TYPE ...................... " ${CMAKE_BUILD_TYPE})
message (STATUS "CMAKE_TOOLCHAIN_FILE .................. " ${CMAKE_TOOLCHAIN_FILE})
message (STATUS "GLIBC_VERSION....... .................. " ${OV_GLIBC_VERSION})
# remove file with exported developer targets to force its regeneration
file(REMOVE "${CMAKE_BINARY_DIR}/ngraphTargets.cmake")
file(REMOVE "${CMAKE_BINARY_DIR}/InferenceEngineTargets.cmake")
file(REMOVE "${CMAKE_BINARY_DIR}/OpenVINOTargets.cmake")
foreach(component IN LISTS openvino_export_components)
file(REMOVE "${CMAKE_BINARY_DIR}/${component}_dev_targets.cmake")
file(REMOVE "${CMAKE_BINARY_DIR}/ov_${component}_dev_targets.cmake")
unset(${component} CACHE)
endforeach()
unset(openvino_export_components CACHE)
#
# Build
#
function(build_ngraph)
function(ngraph_set option value)
if(NOT DEFINED ${option})
set(${option} ${value} CACHE BOOL "" FORCE)
endif()
endfunction()
if (ENABLE_SANITIZER)
ngraph_set(NGRAPH_ADDRESS_SANITIZER ON)
else ()
ngraph_set(NGRAPH_ADDRESS_SANITIZER OFF)
endif ()
if(ENABLE_TESTS AND NOT ANDROID)
ngraph_set(NGRAPH_UNIT_TEST_ENABLE ON)
else()
ngraph_set(NGRAPH_UNIT_TEST_ENABLE OFF)
endif()
if(NOT (ANDROID OR WINDOWS_STORE OR (MSVC AND (ARM OR AARCH64)) ))
ngraph_set(NGRAPH_ONNX_IMPORT_ENABLE ON)
else()
ngraph_set(NGRAPH_ONNX_IMPORT_ENABLE OFF)
endif()
ngraph_set(NGRAPH_INTERPRETER_ENABLE ON)
if(TREAT_WARNING_AS_ERROR)
ngraph_set(NGRAPH_WARNINGS_AS_ERRORS ON)
else()
ngraph_set(NGRAPH_WARNINGS_AS_ERRORS OFF)
endif()
if(ENABLE_SANITIZER)
ngraph_set(NGRAPH_ADDRESS_SANITIZER_ENABLE ON)
else()
ngraph_set(NGRAPH_ADDRESS_SANITIZER_ENABLE OFF)
endif()
if(ENABLE_THREAD_SANITIZER)
ngraph_set(NGRAPH_THREAD_SANITIZER_ENABLE ON)
else()
ngraph_set(NGRAPH_THREAD_SANITIZER_ENABLE OFF)
endif()
if(ENABLE_PYTHON)
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE ON)
else()
ngraph_set(NGRAPH_PYTHON_BUILD_ENABLE OFF)
endif()
if(OV_COMPILER_IS_CLANG)
ie_add_compiler_flags(-Wno-error=uninitialized -Wno-error=literal-conversion)
elseif(UNIX)
ie_add_compiler_flags(-Wno-error=maybe-uninitialized -Wno-error=return-type)
endif()
# WA for GCC 7.0
if (UNIX)
ie_add_compiler_flags(-Wno-error=return-type -Wno-undef)
elseif(WIN32)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /wd4308 /wd4146 /wd4703 /wd4244 /wd4819")
endif()
if(ENABLE_LTO)
set(CMAKE_INTERPROCEDURAL_OPTIMIZATION_RELEASE ON)
endif()
ie_cpack_add_component(ngraph REQUIRED)
ie_cpack_add_component(ngraph_dev REQUIRED DEPENDS ngraph)
set(SDL_cmake_included ON)
add_subdirectory(ngraph)
set(NGRAPH_LIBRARIES ngraph PARENT_SCOPE)
set(NGRAPH_REF_LIBRARIES ngraph_reference PARENT_SCOPE)
endfunction()
function(openvino_developer_export_targets)
cmake_parse_arguments(EXPORT "" "COMPONENT" "TARGETS" ${ARGN})
@@ -83,7 +131,7 @@ function(openvino_developer_export_targets)
get_target_property(original_name ${target_name} ALIASED_TARGET)
if(TARGET "${original_name}")
message(STATUS "The name ${target_name} is an ALIAS for ${original_name}. "
"It will be exported to the OpenVINODeveloperPackage with the original name.")
"It will be exported to the InferenceEngineDeveloperPackage with the original name.")
list(REMOVE_ITEM ${EXPORT_COMPONENT} ${target_name})
list(APPEND ${EXPORT_COMPONENT} ${original_name})
endif()
@@ -100,30 +148,75 @@ function(openvino_developer_export_targets)
"A list of OpenVINO exported components" FORCE)
endfunction()
# add target with processed tests model zoo
if (ENABLE_TESTS)
include(cmake/test_model_zoo.cmake)
endif()
add_subdirectory(thirdparty)
add_subdirectory(src)
add_subdirectory(samples)
# Enable interpreter backend for tests
if (ENABLE_TESTS OR ENABLE_TEMPLATE)
add_subdirectory(src/plugins/template/backend)
endif()
include(cmake/extra_modules.cmake)
add_subdirectory(openvino)
build_ngraph()
add_subdirectory(inference-engine)
add_subdirectory(model-optimizer)
add_subdirectory(docs)
add_subdirectory(tools)
add_subdirectory(scripts)
add_subdirectory(licensing)
#
# CPack
# Shellcheck
#
# provides a callback function to describe each component in repo
include(cmake/packaging/packaging.cmake)
ie_shellcheck_process(DIRECTORY "${OpenVINO_MAIN_SOURCE_DIR}"
SKIP "${OpenVINO_MAIN_SOURCE_DIR}/bin"
"${OpenVINO_MAIN_SOURCE_DIR}/build"
"${OpenVINO_MAIN_SOURCE_DIR}/thirdparty"
"${IE_MAIN_SOURCE_DIR}/tests/ie_test_utils/common_test_utils/gtest"
"${IE_MAIN_SOURCE_DIR}/samples/thirdparty"
"${IE_MAIN_SOURCE_DIR}/thirdparty"
"${IE_MAIN_SOURCE_DIR}/temp"
# TODO fix and enable back:
"${OpenVINO_MAIN_SOURCE_DIR}/scripts/install_dependencies"
"${OpenVINO_MAIN_SOURCE_DIR}/scripts/demo"
"${OpenVINO_MAIN_SOURCE_DIR}/ngraph"
"${IE_MAIN_SOURCE_DIR}/scripts")
#
# cpack
#
# install setupvars
ie_cpack_add_component(setupvars REQUIRED)
if(UNIX)
install(PROGRAMS scripts/setupvars/setupvars.sh
DESTINATION bin
COMPONENT setupvars)
elseif(WIN32)
install(PROGRAMS scripts/setupvars/setupvars.bat
DESTINATION bin
COMPONENT setupvars)
endif()
# install install_dependencies
if(UNIX)
ie_cpack_add_component(install_dependencies REQUIRED)
install(DIRECTORY scripts/install_dependencies/
DESTINATION install_dependencies
COMPONENT install_dependencies)
endif()
# install files for demo
ie_cpack_add_component(demo_scripts DEPENDS core)
if(UNIX)
install(DIRECTORY scripts/demo/
DESTINATION deployment_tools/demo
COMPONENT demo_scripts
USE_SOURCE_PERMISSIONS
PATTERN *.bat EXCLUDE)
elseif(WIN32)
install(DIRECTORY scripts/demo/
DESTINATION deployment_tools/demo
COMPONENT demo_scripts
USE_SOURCE_PERMISSIONS
PATTERN *.sh EXCLUDE)
endif()
ie_cpack(${IE_CPACK_COMPONENTS_ALL})

View File

@@ -6,105 +6,70 @@ CODEOWNERS @openvinotoolkit/openvino-admins @openvinotoolkit/openvino-maintaine
# CI:
Jenkinsfile @openvinotoolkit/openvino-admins
azure-pipelines.yml @openvinotoolkit/openvino-admins
/.github/ @openvinotoolkit/openvino-admins
/.ci/ @openvinotoolkit/openvino-admins
# OpenVINO Samples:
/samples/ @openvinotoolkit/openvino-samples-maintainers
# OpenVINO Scripts:
/scripts/ @openvinotoolkit/openvino-scripts-maintainers
# QA Tests:
/tests/ @openvinotoolkit/openvino-tests-maintainers
/tests/layer_tests @openvinotoolkit/openvino-tests-maintainers @openvinotoolkit/openvino-mo-maintainers
# Tools:
# OpenVINO Scripts
/scripts/ @openvinotoolkit/openvino-admins @openvinotoolkit/openvino-scripts-maintainers
# IE Core:
/inference-engine/ @openvinotoolkit/openvino-ie-maintainers
/inference-engine/ie_bridges/python @openvinotoolkit/openvino-ie-python-api-maintainers
/inference-engine/src/transformations/ @GlebKazantaev @ilyachur
/inference-engine/src/legacy_api/ @openvinotoolkit/openvino-ngraph-maintainers
/inference-engine/src/readers/ @openvinotoolkit/openvino-ngraph-maintainers
# IE CPU:
/inference-engine/src/mkldnn_plugin/ @openvinotoolkit/openvino-ie-cpu-maintainers @openvinotoolkit/openvino-ie-cpu-developers
/inference-engine/src/low_precision_transformations/ @openvinotoolkit/openvino-ie-cpu-maintainers @openvinotoolkit/openvino-ie-cpu-developers
/inference-engine/thirdparty/mkl-dnn/ @openvinotoolkit/openvino-ie-cpu-maintainers @openvinotoolkit/openvino-ie-cpu-developers
# IE GPU:
/inference-engine/src/cldnn_engine/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/inference-engine/include/gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/inference-engine/include/cldnn/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/inference-engine/thirdparty/clDNN/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
# IE VPU:
/inference-engine/src/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers
/inference-engine/include/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers
/inference-engine/thirdparty/movidius/ @openvinotoolkit/openvino-ie-vpu-maintainers
/inference-engine/tests_deprecated/unit/engines/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests_deprecated/functional/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests_deprecated/behavior/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests/functional/plugin/myriad/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests/unit/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests/unit/engines/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tools/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers
/inference-engine/scripts/run_tests_myriad_multistick.sh @openvinotoolkit/openvino-ie-vpu-maintainers
# IE GNA:
/inference-engine/src/gna_plugin/ @openvinotoolkit/openvino-ie-gna-maintainers
/inference-engine/include/gna/ @openvinotoolkit/openvino-ie-gna-maintainers
# IE MULTI:
/inference-engine/src/multi_device/ @openvinotoolkit/openvino-ie-multi-maintainers
/inference-engine/include/multi-device/ @openvinotoolkit/openvino-ie-multi-maintainers
# IE Tests:
/inference-engine/tests/ @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests_deprecated/ @openvinotoolkit/openvino-ie-tests-maintainers
/inference-engine/tests/functional/inference_engine/ngraph_reader/ @openvinotoolkit/openvino-ie-tests-maintainers @openvinotoolkit/openvino-ngraph-maintainers
/inference-engine/tests/functional/inference_engine/transformations/ @openvinotoolkit/openvino-ie-tests-maintainers @openvinotoolkit/openvino-ngraph-maintainers
# MO:
/model-optimizer/ @openvinotoolkit/openvino-mo-maintainers
# nGraph:
/ngraph/ @openvinotoolkit/openvino-ngraph-maintainers
# Tools
/tools/ @openvinotoolkit/openvino-tools-maintainers
# Model Optimizer:
/tools/mo/ @openvinotoolkit/openvino-mo-maintainers
# POT:
/tools/pot/ @openvinotoolkit/openvino-pot-maintainers
# OpenVINO Core:
/src/inference/ @openvinotoolkit/openvino-ie-maintainers
/cmake/ @openvinotoolkit/openvino-ie-maintainers
/src/cmake/ @openvinotoolkit/openvino-ie-maintainers
CMakeLists.txt @openvinotoolkit/openvino-ie-maintainers
*cmake @openvinotoolkit/openvino-ie-maintainers
/src/common/cc/ @openvinotoolkit/openvino-ie-maintainers
/src/common/itt/ @openvinotoolkit/openvino-ie-maintainers
/src/common/legacy/ @openvinotoolkit/openvino-ie-maintainers
/src/common/preprocessing/ @openvinotoolkit/openvino-ie-maintainers
/src/common/util/ @openvinotoolkit/openvino-ie-maintainers
/thirdparty/ @openvinotoolkit/openvino-ie-maintainers
/scripts/ @openvinotoolkit/openvino-ie-maintainers
/src/bindings/python/ @openvinotoolkit/openvino-ie-python-api-maintainers
/src/bindings/c/ @openvinotoolkit/openvino-c-api-maintainers
/src/common/*transformations/ @openvinotoolkit/openvino-ie-transformations-maintainers
/src/core/ @openvinotoolkit/openvino-ngraph-maintainers
# OpenVINO CPU:
/src/plugins/intel_cpu/ @openvinotoolkit/openvino-ie-cpu-maintainers @openvinotoolkit/openvino-ie-cpu-developers
/src/common/snippets/ @openvinotoolkit/openvino-ie-cpu-maintainers
/src/tests/**cpu** @openvinotoolkit/openvino-ie-cpu-maintainers
# OpenVINO LPT
/src/common/low_precision_transformations/ @openvinotoolkit/openvino-ie-lpt-maintainers
# OpenVINO GPU:
/src/plugins/intel_gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/src/tests/**gpu** @openvinotoolkit/openvino-ie-gpu-maintainers
/src/inference/include/ie/gpu/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
/src/inference/include/ie/cldnn/ @openvinotoolkit/openvino-ie-gpu-maintainers @openvinotoolkit/openvino-ie-gpu-developers
# OpenVINO VPU:
/src/plugins/intel_myriad @openvinotoolkit/openvino-ie-vpu-maintainers
/src/tests/**myriad** @openvinotoolkit/openvino-ie-vpu-maintainers
/src/inference/include/ie/vpu/ @openvinotoolkit/openvino-ie-vpu-maintainers
# OpenVINO GNA:
/src/plugins/intel_gna/ @openvinotoolkit/openvino-ie-gna-maintainers
/src/tests/**gna** @openvinotoolkit/openvino-ie-gna-maintainers
# OpenVINO ARM CPU:
/docs/OV_Runtime_UG/supported_plugins/ARM_CPU.md @openvinotoolkit/openvino_contrib-arm_plugin-maintainers
# OpenVINO Auto (MULTI) plugin:
/src/plugins/auto/ @openvinotoolkit/openvino-ie-auto-multi-maintainers
# OpenVINO Auto (Batch) plugin:
/src/plugins/auto_batch/ @openvinotoolkit/openvino-auto-batch-maintainers
# OpenVINO Hetero plugin:
/src/plugins/hetero/ @openvinotoolkit/openvino-hetero-maintainers
# OpenVINO Frontends:
/src/frontends/common/ @openvinotoolkit/openvino-common-frontend-maintainers
/src/frontends/ir/ @openvinotoolkit/openvino-ir-frontend-maintainers
/src/frontends/onnx/ @openvinotoolkit/openvino-onnx-frontend-maintainers
/src/frontends/tests/onnx/ @openvinotoolkit/openvino-onnx-frontend-maintainers
/src/frontends/paddle/ @openvinotoolkit/openvino-ie-paddle-maintainers
/src/frontends/tensorflow/ @openvinotoolkit/openvino-tf-frontend-maintainers
# OpenVINO Tests:
/src/tests/ @openvinotoolkit/openvino-ie-tests-maintainers @openvinotoolkit/openvino-ie-test-developers
/src/tests_deprecated/ @openvinotoolkit/openvino-ie-tests-maintainers @openvinotoolkit/openvino-ie-test-developers
/src/frontends/tests/frontend/shared/ @openvinotoolkit/openvino-ie-tests-maintainers
# Documentation:
# Documentation
/docs/ @openvinotoolkit/openvino-docs-maintainers
/docs/MO_DG @openvinotoolkit/openvino-mo-maintainers @openvinotoolkit/openvino-docs-maintainers
*.md @openvinotoolkit/openvino-docs-maintainers
README @openvinotoolkit/openvino-docs-maintainers
/src/plugins/template/ @openvinotoolkit/openvino-ie-template-maintainers
# Control 3d party dependencies
**/*requirements*.* @openvinotoolkit/openvino-configuration-mgmt
**/setup.py @openvinotoolkit/openvino-configuration-mgmt
/scripts/install_dependencies/ @openvinotoolkit/openvino-configuration-mgmt
*.md @openvinotoolkit/openvino-docs-maintainers

View File

@@ -1,55 +0,0 @@
# How to contribute to the OpenVINO repository
We welcome community contributions to OpenVINO™. Please read the following guide to learn how to find ideas for contribution, practices for good pull requests, checking your changes with our tests and more.
## Before you start contributing you should
- Make sure you agree to contribute your code under [OpenVINO™ (Apache 2.0)](https://github.com/openvinotoolkit/openvino/blob/master/LICENSE) license.
- Figure out what youre going to contribute. If you dont know what you are going to work on, navigate to the [Github "Issues" tab](https://github.com/openvinotoolkit/openvino/issues). Make sure that there isn't someone working on it. In the latter case you might provide support or suggestion in the issue or in the linked pull request.
- If you are going to fix a bug, check that it's still exists in the latest release. This can be done by building the latest master branch, and make sure that the error is still reproducible there. We do not fix bugs that only affect older non-LTS releases like 2020.2 for example (more details about [branching strategy](https://github.com/openvinotoolkit/openvino/wiki/Branches)).
## "Fork & Pull Request model" for code contribution
### [](https://github.com/openvinotoolkit/openvino/blob/master/CONTRIBUTING.md#the-instruction-in-brief)The instruction in brief
- Register at GitHub. Create your fork of OpenVINO™ repository [https://github.com/openvinotoolkit/openvino](https://github.com/openvinotoolkit/openvino) (see [https://help.github.com/articles/fork-a-repo](https://help.github.com/articles/fork-a-repo) for details).
- Install Git.
- Set your user name and email address in a Git configuration according to GitHub account (see [https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) for details).
- Choose a task for yourself. It could be a bugfix or some new code.
- Choose a base branch for your work. More details about branches and policies are here: [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches)
- Clone your fork to your computer.
- Create a new branch (with a meaningful name) from the base branch you chose.
- Modify / add the code following our [Coding Style Guide](./docs/dev/coding_style.md).
- If you want to add a new sample, please look at this [Guide for contributing to C++/C/Python IE samples](https://github.com/openvinotoolkit/openvino/wiki/SampleContribute)
- If you want to contribute to the documentation and want to add a new guide, follow that instruction [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation)
- Run testsuite locally:
- execute each test binary from the artifacts directory, e.g. `<source dir>/bin/intel64/Release/ieFuncTests`
- When you are done, make sure that your branch is to date with latest state of the branch you want to contribute to (e.g. `git fetch upstream && git merge upstream/master`), push your branch to your GitHub fork; then create a pull request from your branch to the base branch (see [https://help.github.com/articles/using-pull-requests](https://help.github.com/articles/using-pull-requests) for details).
## Making a good pull request
Following these guidelines will increase the likelihood of your pull request being accepted:
- One PR one issue.
- Build perfectly on your local system.
- Choose the right base branch [Branches](https://github.com/openvinotoolkit/openvino/wiki/Branches).
- Follow the [Coding Style Guide](./docs/dev/coding_style.md) for your code.
- Update documentation using [Documentation guidelines](https://github.com/openvinotoolkit/openvino/wiki/CodingStyleGuideLinesDocumentation) if needed.
- Cover your changes with test.
- Add license at the top of new files [C++ example](https://github.com/openvinotoolkit/openvino/blob/master/samples/cpp/classification_sample_async/main.cpp#L1-L2), [Python example](https://github.com/openvinotoolkit/openvino/blob/master/samples/python/hello_classification/hello_classification.py#L3-L4).
- Add enough information: a meaningful title, the reason why you made the commit and a link to the issue page if exists.
- Remove unrelated to PR changes.
- If it is still WIP and you want to check CI test results early then use _Draft_ PR.
- Submit your PR and become an OpenVINO™ contributor!
## Testing and merging pull requests
Your pull request will be automatically tested by OpenVINO™'s precommit (testing status are automatically reported as "green" or "red" circles in precommit steps on PR's page). If any builders have failed, you need fix the issue. To rerun the automatic builds just push changes to your branch on GitHub. No need to close pull request and open a new one!
## Merging PR
When the reviewer accepts the pull request and the pre-commit shows a "green" status, the review status is set to "Approved", which signals to the OpenVINO™ maintainers that they can merge your pull request.

226
README.md
View File

@@ -1,208 +1,50 @@
<div align="center">
<img src="docs/img/openvino-logo-purple-black.png" width="400px">
[![Stable release](https://img.shields.io/badge/version-2022.2-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2022.2.0)
# OpenVINO™ Toolkit
[![Stable release](https://img.shields.io/badge/version-2021.4.2-green.svg)](https://github.com/openvinotoolkit/openvino/releases/tag/2021.4.2)
[![Apache License Version 2.0](https://img.shields.io/badge/license-Apache_2.0-green.svg)](LICENSE)
![GitHub branch checks state](https://img.shields.io/github/checks-status/openvinotoolkit/openvino/master?label=GitHub%20checks)
![Azure DevOps builds (branch)](https://img.shields.io/azure-devops/build/openvinoci/b2bab62f-ab2f-4871-a538-86ea1be7d20f/13?label=Public%20CI)
[![PyPI Status](https://badge.fury.io/py/openvino.svg)](https://badge.fury.io/py/openvino)
[![PyPI Downloads](https://pepy.tech/badge/openvino)](https://pepy.tech/project/openvino)
</div>
## Contents:
This toolkit allows developers to deploy pre-trained deep learning models
through a high-level C++ Inference Engine API integrated with application logic.
- [What is OpenVINO?](#what-is-openvino-toolkit)
- [Components](#components)
- [Supported Hardware matrix](#supported-hardware-matrix)
- [License](#license)
- [Documentation](#documentation)
- [Tutorials](#tutorials)
- [Products which use OpenVINO](#products-which-use-openvino)
- [System requirements](#system-requirements)
- [How to build](#how-to-build)
- [How to contribute](#how-to-contribute)
- [Get a support](#get-a-support)
- [See also](#see-also)
This open source version includes several components: namely [Model Optimizer], [nGraph] and
[Inference Engine], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inferencing on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from the [Open Model Zoo], along with 100+ open
source and public models in popular formats such as Caffe\*, TensorFlow\*,
MXNet\* and ONNX\*.
## What is OpenVINO toolkit?
OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference.
- Boost deep learning performance in computer vision, automatic speech recognition, natural language processing and other common tasks
- Use models trained with popular frameworks like TensorFlow, PyTorch and more
- Reduce resource demands and efficiently deploy on a range of Intel® platforms from edge to cloud
This open-source version includes several components: namely [Model Optimizer], [OpenVINO™ Runtime], [Post-Training Optimization Tool], as well as CPU, GPU, MYRIAD, multi device and heterogeneous plugins to accelerate deep learning inference on Intel® CPUs and Intel® Processor Graphics.
It supports pre-trained models from [Open Model Zoo], along with 100+ open
source and public models in popular formats such as TensorFlow, ONNX, PaddlePaddle, MXNet, Caffe, Kaldi.
### Components
* [OpenVINO™ Runtime] - is a set of C++ libraries with C and Python bindings providing a common API to deliver inference solutions on the platform of your choice.
* [core](./src/core) - provides the base API for model representation and modification.
* [inference](./src/inference) - provides an API to infer models on the device.
* [transformations](./src/common/transformations) - contains the set of common transformations which are used in OpenVINO plugins.
* [low precision transformations](./src/common/low_precision_transformations) - contains the set of transformations that are used in low precision models
* [bindings](./src/bindings) - contains all available OpenVINO bindings which are maintained by the OpenVINO team.
* [c](./src/bindings/c) - C API for OpenVINO™ Runtime
* [python](./src/bindings/python) - Python API for OpenVINO™ Runtime
* [Plugins](./src/plugins) - contains OpenVINO plugins which are maintained in open-source by the OpenVINO team. For more information, take a look at the [list of supported devices](#supported-hardware-matrix).
* [Frontends](./src/frontends) - contains available OpenVINO frontends that allow reading models from the native framework format.
* [Model Optimizer] - is a cross-platform command-line tool that facilitates the transition between training and deployment environments, performs static model analysis, and adjusts deep learning models for optimal execution on end-point target devices.
* [Post-Training Optimization Tool] - is designed to accelerate the inference of deep learning models by applying special methods without model retraining or fine-tuning, for example, post-training 8-bit quantization.
* [Samples] - applications in C, C++ and Python languages that show basic OpenVINO use cases.
## Supported Hardware matrix
The OpenVINO™ Runtime can infer models on different hardware devices. This section provides the list of supported devices.
<table>
<thead>
<tr>
<th>Device</th>
<th>Plugin</th>
<th>Library</th>
<th>ShortDescription</th>
</tr>
</thead>
<tbody>
<tr>
<td rowspan=2>CPU</td>
<td> <a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_supported_plugins_CPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-c-p-u">Intel CPU</a></tb>
<td><b><i><a href="./src/plugins/intel_cpu">openvino_intel_cpu_plugin</a></i></b></td>
<td>Intel Xeon with Intel® Advanced Vector Extensions 2 (Intel® AVX2), Intel® Advanced Vector Extensions 512 (Intel® AVX-512), and AVX512_BF16, Intel Core Processors with Intel AVX2, Intel Atom Processors with Intel® Streaming SIMD Extensions (Intel® SSE)</td>
</tr>
<tr>
<td> <a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_supported_plugins_ARM_CPU.html">ARM CPU</a></tb>
<td><b><i><a href="https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/arm_plugin">openvino_arm_cpu_plugin</a></i></b></td>
<td>Raspberry Pi™ 4 Model B, Apple® Mac mini with M1 chip, NVIDIA® Jetson Nano™, Android™ devices
</tr>
<tr>
<td>GPU</td>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_supported_plugins_GPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-g-p-u">Intel GPU</a></td>
<td><b><i><a href="./src/plugins/intel_gpu">openvino_intel_gpu_plugin</a></i></b></td>
<td>Intel Processor Graphics, including Intel HD Graphics and Intel Iris Graphics</td>
</tr>
<tr>
<td>GNA</td>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_supported_plugins_GNA.html#doxid-openvino-docs-o-v-u-g-supported-plugins-g-n-a">Intel GNA</a></td>
<td><b><i><a href="./src/plugins/intel_gna">openvino_intel_gna_plugin</a></i></b></td>
<td>Intel Speech Enabling Developer Kit, Amazon Alexa* Premium Far-Field Developer Kit, Intel Pentium Silver J5005 Processor, Intel Pentium Silver N5000 Processor, Intel Celeron J4005 Processor, Intel Celeron J4105 Processor, Intel Celeron Processor N4100, Intel Celeron Processor N4000, Intel Core i3-8121U Processor, Intel Core i7-1065G7 Processor, Intel Core i7-1060G7 Processor, Intel Core i5-1035G4 Processor, Intel Core i5-1035G7 Processor, Intel Core i5-1035G1 Processor, Intel Core i5-1030G7 Processor, Intel Core i5-1030G4 Processor, Intel Core i3-1005G1 Processor, Intel Core i3-1000G1 Processor, Intel Core i3-1000G4 Processor</td>
</tr>
<tr>
<td>VPU</td>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_IE_DG_supported_plugins_VPU.html#doxid-openvino-docs-i-e-d-g-supported-plugins-v-p-u">Myriad plugin</a></td>
<td><b><i><a href="./src/plugins/intel_myriad">openvino_intel_myriad_plugin</a></i></b></td>
<td>Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X</td>
</tr>
</tbody>
</table>
OpenVINO™ Toolkit also contains several plugins which simplify loading models on several hardware devices:
<table>
<thead>
<tr>
<th>Plugin</th>
<th>Library</th>
<th>ShortDescription</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_IE_DG_supported_plugins_AUTO.html#doxid-openvino-docs-i-e-d-g-supported-plugins-a-u-t-o">Auto</a></td>
<td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td>
<td>Auto plugin enables selecting Intel device for inference automatically</td>
</tr>
<tr>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_Automatic_Batching.html">Auto Batch</a></td>
<td><b><i><a href="./src/plugins/auto_batch">openvino_auto_batch_plugin</a></i></b></td>
<td>Auto batch plugin performs on-the-fly automatic batching (i.e. grouping inference requests together) to improve device utilization, with no programming effort from the user</td>
</tr>
<tr>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_Hetero_execution.html#doxid-openvino-docs-o-v-u-g-hetero-execution">Hetero</a></td>
<td><b><i><a href="./src/plugins/hetero">openvino_hetero_plugin</a></i></b></td>
<td>Heterogeneous execution enables automatic inference splitting between several devices</td>
</tr>
<tr>
<td><a href="https://docs.openvino.ai/nightly/openvino_docs_OV_UG_Running_on_multiple_devices.html#doxid-openvino-docs-o-v-u-g-running-on-multiple-devices">Multi</a></td>
<td><b><i><a href="./src/plugins/auto">openvino_auto_plugin</a></i></b></td>
<td>Multi plugin enables simultaneous inference of the same model on several devices in parallel</td>
</tr>
</tbody>
</table>
## Repository components:
* [Inference Engine]
* [nGraph]
* [Model Optimizer]
## License
OpenVINO™ Toolkit is licensed under [Apache License Version 2.0](LICENSE).
By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.
Deep Learning Deployment Toolkit is licensed under [Apache License Version 2.0](LICENSE).
By contributing to the project, you agree to the license and copyright terms therein
and release your contribution under these terms.
## Documentation
### User documentation
The latest documentation for OpenVINO™ Toolkit is available [here](https://docs.openvino.ai/). This documentation contains detailed information about all OpenVINO components and provides all the important information you may need to create an application based on binary OpenVINO distribution or own OpenVINO version without source code modification.
### Developer documentation
[Developer documentation](./docs/dev/index.md) contains information about architectural decisions which are applied inside the OpenVINO components. This documentation has all necessary information which could be needed in order to contribute to OpenVINO.
## Tutorials
The list of OpenVINO tutorials:
- [Jupiter notebooks](https://github.com/openvinotoolkit/openvino_notebooks)
## Products which use OpenVINO
- [OpenCV](https://opencv.org/)
- [ONNX Runtime](https://onnxruntime.ai/)
- [OpenVINO™ Integration with TensorFlow](https://www.intel.com/content/www/us/en/developer/tools/devcloud/edge/build/ovtfoverview.html)
- [TNN](https://github.com/Tencent/TNN/tree/master)
## System requirements
The system requirements vary depending on platform and are available on dedicated pages:
- [Linux](https://docs.openvino.ai/nightly/openvino_docs_install_guides_installing_openvino_linux_header.html)
- [Windows](https://docs.openvino.ai/nightly/openvino_docs_install_guides_installing_openvino_windows_header.html)
- [macOS](https://docs.openvino.ai/nightly/openvino_docs_install_guides_installing_openvino_macos_header.html)
- [Raspbian](https://docs.openvino.ai/nightly/openvino_docs_install_guides_installing_openvino_raspbian.html)
## How to build
See the [OpenVINO Wiki](https://github.com/openvinotoolkit/openvino/wiki#how-to-build) to get more information about the OpenVINO build process.
## How to contribute
See [CONTRIBUTING](./CONTRIBUTING.md) for details. Thank you!
## Get a support
Report questions, issues and suggestions, using:
* [GitHub* Issues](https://github.com/openvinotoolkit/openvino/issues)
* The [`openvino`](https://stackoverflow.com/questions/tagged/openvino) tag on StackOverflow\*
* [Forum](https://software.intel.com/en-us/forums/computer-vision)
## Additional Resources
* [OpenVINO Wiki](https://github.com/openvinotoolkit/openvino/wiki)
* [OpenVINO Storage](https://storage.openvinotoolkit.org/)
* Additional OpenVINO™ toolkit modules:
* [openvino_contrib](https://github.com/openvinotoolkit/openvino_contrib)
## Resources:
* Docs: https://docs.openvinotoolkit.org/
* Wiki: https://github.com/openvinotoolkit/openvino/wiki
* Issue tracking: https://github.com/openvinotoolkit/openvino/issues
* Storage: https://storage.openvinotoolkit.org/
* Additional OpenVINO™ modules: https://github.com/openvinotoolkit/openvino_contrib
* [Intel® Distribution of OpenVINO™ toolkit Product Page](https://software.intel.com/content/www/us/en/develop/tools/openvino-toolkit.html)
* [Intel® Distribution of OpenVINO™ toolkit Release Notes](https://software.intel.com/en-us/articles/OpenVINO-RelNotes)
* [Neural Network Compression Framework (NNCF)](https://github.com/openvinotoolkit/nncf) - a suite of advanced algorithms for model inference optimization including quantization, filter pruning, binarization and sparsity
* [OpenVINO™ Training Extensions (OTE)](https://github.com/openvinotoolkit/training_extensions) - convenient environment to train Deep Learning models and convert them using OpenVINO for optimized inference.
* [OpenVINO™ Model Server (OVMS)](https://github.com/openvinotoolkit/model_server) - a scalable, high-performance solution for serving deep learning models optimized for Intel architectures
* [DL Workbench](https://docs.openvino.ai/nightly/workbench_docs_Workbench_DG_Introduction.html) - an alternative, web-based version of OpenVINO designed to facilitate optimization and compression of pre-trained deep learning models.
* [Computer Vision Annotation Tool (CVAT)](https://github.com/opencv/cvat) - an online, interactive video and image annotation tool for computer vision purposes.
* [Dataset Management Framework (Datumaro)](https://github.com/openvinotoolkit/datumaro) - a framework and CLI tool to build, transform, and analyze datasets.
## Support
Please report questions, issues and suggestions using:
* The [`openvino`](https://stackoverflow.com/questions/tagged/openvino) tag on StackOverflow\*
* [GitHub* Issues](https://github.com/openvinotoolkit/openvino/issues)
* [Forum](https://software.intel.com/en-us/forums/computer-vision)
---
\* Other names and brands may be claimed as the property of others.
[Open Model Zoo]:https://github.com/openvinotoolkit/open_model_zoo
[OpenVINO™ Runtime]:https://docs.openvino.ai/nightly/openvino_docs_OV_UG_OV_Runtime_User_Guide.html
[Model Optimizer]:https://docs.openvino.ai/nightly/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html
[Post-Training Optimization Tool]:https://docs.openvino.ai/nightly/pot_introduction.html
[Samples]:https://github.com/openvinotoolkit/openvino/tree/master/samples
[Inference Engine]:https://software.intel.com/en-us/articles/OpenVINO-InferEngine
[Model Optimizer]:https://software.intel.com/en-us/articles/OpenVINO-ModelOptimizer
[nGraph]:https://docs.openvinotoolkit.org/latest/openvino_docs_nGraph_DG_DevGuide.html
[tag on StackOverflow]:https://stackoverflow.com/search?q=%23openvino

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -7,8 +7,6 @@ set(CMAKE_SYSTEM_PROCESSOR armv7l)
set(CMAKE_C_COMPILER arm-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER arm-linux-gnueabihf-g++)
set(CMAKE_STRIP arm-linux-gnueabihf-strip)
set(PKG_CONFIG_EXECUTABLE arm-linux-gnueabihf-pkg-config CACHE PATH "Path to ARM pkg-config")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -7,8 +7,6 @@ set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_C_COMPILER aarch64-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++)
set(CMAKE_STRIP aarch64-linux-gnu-strip)
set(PKG_CONFIG_EXECUTABLE aarch64-linux-gnu-pkg-config CACHE PATH "Path to ARM64 pkg-config")
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)

View File

@@ -1,152 +0,0 @@
# Copyright (C) 2018-2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(OV_COVERAGE_BASE_DIRECTORY "${OpenVINO_SOURCE_DIR}")
ov_coverage_clean(REPOSITORY "openvino"
DIRECTORY "${OV_COVERAGE_GCDA_DATA_DIRECTORY}")
ov_coverage_capture(INFO_FILE "openvino"
BASE_DIRECTORY "${OV_COVERAGE_BASE_DIRECTORY}"
DIRECTORY "${OV_COVERAGE_GCDA_DATA_DIRECTORY}"
EXCLUDE_PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/*.pb.cc"
"${OV_COVERAGE_BASE_DIRECTORY}/*.pb.h"
"${OV_COVERAGE_BASE_DIRECTORY}/*/tests/*"
"${OV_COVERAGE_BASE_DIRECTORY}/*/tests_deprecated/*"
"${OV_COVERAGE_BASE_DIRECTORY}/thirdparty/*") # Skip some pb files, tests and thirdparty
# Common report
ov_coverage_genhtml(INFO_FILE "openvino"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
# Generate reports
##################### Core Components #####################
ov_coverage_extract(INPUT "openvino" OUTPUT "inference"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/inference/*")
ov_coverage_genhtml(INFO_FILE "inference"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "core"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/core/*")
ov_coverage_genhtml(INFO_FILE "core"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "transformations"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/common/transformations/*")
ov_coverage_genhtml(INFO_FILE "transformations"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "low_precision_transformations"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/common/low_precision_transformations/*")
ov_coverage_genhtml(INFO_FILE "low_precision_transformations"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "legacy"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/common/legacy/*")
ov_coverage_genhtml(INFO_FILE "legacy"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "preprocessing"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}src/common/preprocessing/*")
ov_coverage_genhtml(INFO_FILE "preprocessing"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "snippets"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/common/snippets/*")
ov_coverage_genhtml(INFO_FILE "snippets"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
ov_coverage_extract(INPUT "openvino" OUTPUT "frontend_common"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/frontends/common/*")
ov_coverage_genhtml(INFO_FILE "frontend_common"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
##################### Core Components #####################
##################### Plugins #####################
if(ENABLE_AUTO OR ENABLE_MULTI)
ov_coverage_extract(INPUT "openvino" OUTPUT "auto_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/auto/*")
ov_coverage_genhtml(INFO_FILE "auto_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_AUTO_BATCH)
ov_coverage_extract(INPUT "openvino" OUTPUT "auto_batch_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/auto_batch/*")
ov_coverage_genhtml(INFO_FILE "auto_batch_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_HETERO)
ov_coverage_extract(INPUT "openvino" OUTPUT "hetero_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/hetero/*")
ov_coverage_genhtml(INFO_FILE "hetero_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_INTEL_CPU)
ov_coverage_extract(INPUT "openvino" OUTPUT "intel_cpu_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/intel_cpu/*")
ov_coverage_genhtml(INFO_FILE "intel_cpu_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_INTEL_GNA)
ov_coverage_extract(INPUT "openvino" OUTPUT "intel_gna_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/intel_gna/*")
ov_coverage_genhtml(INFO_FILE "intel_gna_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if (ENABLE_INTEL_GPU)
ov_coverage_extract(INPUT "openvino" OUTPUT "intel_gpu_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/intel_gpu/*")
ov_coverage_genhtml(INFO_FILE "intel_gpu_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_INTEL_MYRIAD)
ov_coverage_extract(INPUT "openvino" OUTPUT "intel_myriad_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/intel_myriad/*")
ov_coverage_genhtml(INFO_FILE "intel_myriad_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_TEMPLATE)
ov_coverage_extract(INPUT "openvino" OUTPUT "template_plugin"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/plugins/template/*")
ov_coverage_genhtml(INFO_FILE "template_plugin"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
##################### Plugins #####################
##################### Frontends #####################
if(ENABLE_OV_IR_FRONTEND)
ov_coverage_extract(INPUT "openvino" OUTPUT "ir_frontend"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/frontends/ir/*")
ov_coverage_genhtml(INFO_FILE "ir_frontend"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_OV_ONNX_FRONTEND)
ov_coverage_extract(INPUT "openvino" OUTPUT "onnx_frontend"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/frontends/onnx/*")
ov_coverage_genhtml(INFO_FILE "onnx_frontend"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_OV_PADDLE_FRONTEND)
ov_coverage_extract(INPUT "openvino" OUTPUT "paddle_frontend"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/frontends/paddle/*")
ov_coverage_genhtml(INFO_FILE "paddle_frontend"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
if(ENABLE_OV_TF_FRONTEND)
ov_coverage_extract(INPUT "openvino" OUTPUT "tf_frontend"
PATTERNS "${OV_COVERAGE_BASE_DIRECTORY}/src/frontends/tensorflow/*")
ov_coverage_genhtml(INFO_FILE "tf_frontend"
PREFIX "${OV_COVERAGE_BASE_DIRECTORY}")
endif()
##################### Frontends #####################

View File

@@ -1,342 +1,29 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
cmake_policy(SET CMP0054 NEW)
set_temp_directory(TEMP "${IE_MAIN_SOURCE_DIR}")
# TODO: fix it, outside of source dir MO cannot find TBB dependency
set_temp_directory(TEMP "${CMAKE_SOURCE_DIR}")
if(CMAKE_CROSSCOMPILING AND CMAKE_HOST_SYSTEM_NAME MATCHES Linux AND CMAKE_HOST_SYSTEM_PROCESSOR MATCHES "amd64.*|x86_64.*|AMD64.*")
set(protoc_version "3.7.1")
if(ENABLE_SAME_BRANCH_FOR_MODELS)
branchName(MODELS_BRANCH)
else()
set(MODELS_BRANCH "master")
endif()
if(ENABLE_DATA)
add_models_repo(${ENABLE_DATA} "data:https://github.com/openvinotoolkit/testdata.git")
set(MODELS_PATH "${TEMP}/models/src/data")
set(DATA_PATH "${MODELS_PATH}")
endif()
message(STATUS "MODELS_PATH=" ${MODELS_PATH})
fetch_models_and_validation_set()
if(ENABLE_INTEL_MYRIAD)
include(${OpenVINO_SOURCE_DIR}/src/plugins/intel_myriad/myriad_dependencies.cmake)
endif()
## Intel OMP package
if(THREADING STREQUAL "OMP")
reset_deps_cache(OMP)
if(WIN32 AND X86_64)
RESOLVE_DEPENDENCY(OMP
ARCHIVE_WIN "iomp.zip"
TARGET_PATH "${TEMP}/omp"
ENVIRONMENT "OMP"
VERSION_REGEX ".*_([a-z]*_([a-z0-9]+\\.)*[0-9]+).*"
SHA256 "62c68646747fb10f19b53217cb04a1e10ff93606f992e6b35eb8c31187c68fbf")
elseif(LINUX AND X86_64)
RESOLVE_DEPENDENCY(OMP
ARCHIVE_LIN "iomp.tgz"
TARGET_PATH "${TEMP}/omp"
ENVIRONMENT "OMP"
VERSION_REGEX ".*_([a-z]*_([a-z0-9]+\\.)*[0-9]+).*"
SHA256 "7832b16d82513ee880d97c27c7626f9525ebd678decf6a8fe6c38550f73227d9")
elseif(APPLE AND X86_64)
RESOLVE_DEPENDENCY(OMP
ARCHIVE_MAC "iomp_20190130_mac.tgz"
TARGET_PATH "${TEMP}/omp"
ENVIRONMENT "OMP"
VERSION_REGEX ".*_([a-z]*_([a-z0-9]+\\.)*[0-9]+).*"
SHA256 "591ea4a7e08bbe0062648916f42bded71d24c27f00af30a8f31a29b5878ea0cc")
else()
message(FATAL_ERROR "Intel OMP is not available on current platform")
endif()
update_deps_cache(OMP "${OMP}" "Path to OMP root folder")
debug_message(STATUS "intel_omp=" ${OMP})
ie_cpack_add_component(omp HIDDEN)
file(GLOB_RECURSE source_list "${OMP}/*${CMAKE_SHARED_LIBRARY_SUFFIX}*")
install(FILES ${source_list}
DESTINATION ${OV_CPACK_RUNTIMEDIR}
COMPONENT omp)
endif()
## TBB package
unset(_ov_download_tbb_done CACHE)
#
# The function downloads prebuilt TBB package
# NOTE: the function should be used if system TBB is not found
# or ENABLE_SYSTEM_TBB is OFF
#
function(ov_download_tbb)
if(_ov_download_tbb_done OR NOT THREADING MATCHES "^(TBB|TBB_AUTO)$")
return()
endif()
set(_ov_download_tbb_done ON CACHE INTERNAL "Whether prebuilt TBB is already downloaded")
reset_deps_cache(TBBROOT TBB_DIR)
if(DEFINED ENV{THIRDPARTY_SERVER_PATH})
set(IE_PATH_TO_DEPS "$ENV{THIRDPARTY_SERVER_PATH}")
elseif(DEFINED THIRDPARTY_SERVER_PATH)
set(IE_PATH_TO_DEPS "${THIRDPARTY_SERVER_PATH}")
endif()
if(NOT DEFINED ENV{TBBROOT} AND (DEFINED ENV{TBB_DIR} OR DEFINED TBB_DIR))
if(DEFINED ENV{TBB_DIR})
set(TEMP_ROOT $ENV{TBB_DIR})
elseif (DEFINED TBB_DIR)
set(TEMP_ROOT ${TBB_DIR})
endif()
while(NOT EXISTS "${TEMP_ROOT}/include")
get_filename_component(TEMP_ROOT ${TEMP_ROOT} PATH)
endwhile()
set(TBBROOT ${TEMP_ROOT})
endif()
if(WIN32 AND X86_64)
# TODO: add target_path to be platform specific as well, to avoid following if
RESOLVE_DEPENDENCY(TBB
ARCHIVE_WIN "tbb2020_617e9a71_win.zip"
TARGET_PATH "${TEMP}/tbb"
ENVIRONMENT "TBBROOT"
SHA256 "01cac3cc48705bd52b83a6e1fa1ed95c708928be76160f5b9c5c37f954d56df4"
USE_NEW_LOCATION TRUE)
elseif(ANDROID AND X86_64)
RESOLVE_DEPENDENCY(TBB
ARCHIVE_ANDROID "tbb2020_20200404_android.tgz"
TARGET_PATH "${TEMP}/tbb"
ENVIRONMENT "TBBROOT"
SHA256 "f42d084224cc2d643314bd483ad180b081774608844000f132859fca3e9bf0ce")
elseif(LINUX AND X86_64)
RESOLVE_DEPENDENCY(TBB
ARCHIVE_LIN "tbb2020_617e9a71_lin_strip.tgz"
TARGET_PATH "${TEMP}/tbb"
ENVIRONMENT "TBBROOT"
SHA256 "e7a38f68059fb36de8b59d40b283a849f26275e34a58d2acadfdb84d49e31b9b"
USE_NEW_LOCATION TRUE)
elseif(YOCTO_AARCH64)
RESOLVE_DEPENDENCY(TBB
ARCHIVE_LIN "keembay/tbb2020_38404_kmb_lic.tgz"
TARGET_PATH "${TEMP}/tbb_yocto"
ENVIRONMENT "TBBROOT"
SHA256 "321261ff2eda6d4568a473cb883262bce77a93dac599f7bd65d2918bdee4d75b")
elseif(APPLE AND X86_64)
RESOLVE_DEPENDENCY(TBB
ARCHIVE_MAC "tbb2020_617e9a71_mac.tgz"
TARGET_PATH "${TEMP}/tbb"
ENVIRONMENT "TBBROOT"
SHA256 "67a44b695bef3348416eaf5bf2baca2b1401576c0e09c394304eba1e0eee96cd"
USE_NEW_LOCATION TRUE)
else()
message(WARNING "Prebuilt TBB is not available on current platform")
endif()
update_deps_cache(TBBROOT "${TBB}" "Path to TBB root folder")
if(EXISTS "${TBBROOT}/lib/cmake/TBB/TBBConfig.cmake")
# oneTBB case
update_deps_cache(TBB_DIR "${TBBROOT}/lib/cmake/TBB" "Path to TBB cmake folder")
elseif(EXISTS "${TBBROOT}/lib/cmake/tbb/TBBConfig.cmake")
# oneTBB release package version less than 2021.6.0
update_deps_cache(TBB_DIR "${TBBROOT}/lib/cmake/tbb" "Path to TBB cmake folder")
elseif(EXISTS "${TBBROOT}/lib64/cmake/TBB/TBBConfig.cmake")
# 64-bits oneTBB case
update_deps_cache(TBB_DIR "${TBBROOT}/lib64/cmake/TBB" "Path to TBB cmake folder")
elseif(EXISTS "${TBBROOT}/cmake/TBBConfig.cmake")
# custom downloaded or user provided TBB
update_deps_cache(TBB_DIR "${TBBROOT}/cmake" "Path to TBB cmake folder")
else()
message(WARNING "Failed to find TBBConfig.cmake in ${TBBROOT} tree. Custom TBBConfig.cmake will be used")
endif()
debug_message(STATUS "tbb=" ${TBB})
debug_message(STATUS "tbb_dir=" ${TBB_DIR})
debug_message(STATUS "tbbroot=" ${TBBROOT})
endfunction()
## TBBBind_2_5 package
unset(_ov_download_tbbbind_2_5_done CACHE)
#
# The function downloads static prebuilt TBBBind_2_5 package
# NOTE: the function should be called only we have TBB with version less 2021
#
function(ov_download_tbbbind_2_5)
if(_ov_download_tbbbind_2_5_done OR NOT ENABLE_TBBBIND_2_5)
return()
endif()
set(_ov_download_tbbbind_2_5_done ON CACHE INTERNAL "Whether prebuilt TBBBind_2_5 is already downloaded")
reset_deps_cache(TBBBIND_2_5_ROOT TBBBIND_2_5_DIR)
if(DEFINED ENV{THIRDPARTY_SERVER_PATH})
set(IE_PATH_TO_DEPS "$ENV{THIRDPARTY_SERVER_PATH}")
elseif(DEFINED THIRDPARTY_SERVER_PATH)
set(IE_PATH_TO_DEPS "${THIRDPARTY_SERVER_PATH}")
endif()
if(WIN32 AND X86_64)
RESOLVE_DEPENDENCY(TBBBIND_2_5
ARCHIVE_WIN "tbbbind_2_5_static_win_v1.zip"
TARGET_PATH "${TEMP}/tbbbind_2_5"
ENVIRONMENT "TBBBIND_2_5_ROOT"
SHA256 "a67afeea8cf194f97968c800dab5b5459972908295242e282045d6b8953573c1")
elseif(LINUX AND X86_64)
RESOLVE_DEPENDENCY(TBBBIND_2_5
ARCHIVE_LIN "tbbbind_2_5_static_lin_v2.tgz"
TARGET_PATH "${TEMP}/tbbbind_2_5"
ENVIRONMENT "TBBBIND_2_5_ROOT"
SHA256 "865e7894c58402233caf0d1b288056e0e6ab2bf7c9d00c9dc60561c484bc90f4")
else()
# TMP: for Apple Silicon TBB does not provide TBBBind
if(NOT (APPLE AND AARCH64))
message(WARNING "prebuilt TBBBIND_2_5 is not available.
Build oneTBB from sources and set TBBROOT environment var before OpenVINO cmake configure")
endif()
return()
endif()
update_deps_cache(TBBBIND_2_5_ROOT "${TBBBIND_2_5}" "Path to TBBBIND_2_5 root folder")
update_deps_cache(TBBBIND_2_5_DIR "${TBBBIND_2_5}/cmake" "Path to TBBBIND_2_5 cmake folder")
endfunction()
## OpenCV
if(ENABLE_OPENCV)
reset_deps_cache(OpenCV_DIR)
set(OPENCV_VERSION "4.5.2")
set(OPENCV_BUILD "076")
set(OPENCV_BUILD_YOCTO "772")
if(YOCTO_AARCH64)
if(DEFINED ENV{THIRDPARTY_SERVER_PATH})
set(IE_PATH_TO_DEPS "$ENV{THIRDPARTY_SERVER_PATH}")
elseif(DEFINED THIRDPARTY_SERVER_PATH)
set(IE_PATH_TO_DEPS "${THIRDPARTY_SERVER_PATH}")
else()
message(WARNING "OpenCV is not found!")
endif()
if(DEFINED IE_PATH_TO_DEPS)
set(OPENCV_SUFFIX "yocto_kmb")
set(OPENCV_BUILD "${OPENCV_BUILD_YOCTO}")
RESOLVE_DEPENDENCY(OPENCV
ARCHIVE_LIN "opencv/opencv_${OPENCV_VERSION}-${OPENCV_BUILD}_${OPENCV_SUFFIX}.txz"
TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}_${OPENCV_SUFFIX}/opencv"
ENVIRONMENT "OpenCV_DIR"
VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*"
SHA256 "23c250796ad5fc9db810e1680ccdb32c45dc0e50cace5e0f02b30faf652fe343")
unset(IE_PATH_TO_DEPS)
endif()
else()
if(WIN32 AND X86_64)
RESOLVE_DEPENDENCY(OPENCV
ARCHIVE_WIN "opencv/opencv_${OPENCV_VERSION}-${OPENCV_BUILD}.txz"
TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}/opencv"
ENVIRONMENT "OpenCV_DIR"
VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*"
SHA256 "a14f872e6b63b6ac12c7ff47fa49e578d14c14433b57f5d85ab5dd48a079938c")
elseif(APPLE AND X86_64)
RESOLVE_DEPENDENCY(OPENCV
ARCHIVE_MAC "opencv/opencv_${OPENCV_VERSION}-${OPENCV_BUILD}_osx.txz"
TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}_osx/opencv"
ENVIRONMENT "OpenCV_DIR"
VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*"
SHA256 "3e162f96e86cba8836618134831d9cf76df0438778b3e27e261dedad9254c514")
elseif(LINUX)
if(YOCTO_AARCH64)
set(OPENCV_SUFFIX "yocto_kmb")
set(OPENCV_BUILD "${OPENCV_BUILD_YOCTO}")
elseif((OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.17 AND
CMAKE_CXX_COMPILER_VERSION VERSION_LESS "4.9") AND X86_64)
set(OPENCV_SUFFIX "centos7")
set(OPENCV_HASH "5fa76985c84fe7c64531682ef0b272510c51ac0d0565622514edf1c88b33404a")
elseif(OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.31 AND X86_64)
set(OPENCV_SUFFIX "ubuntu20")
set(OPENCV_HASH "2fe7bbc40e1186eb8d099822038cae2821abf617ac7a16fadf98f377c723e268")
elseif(OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.27 AND X86_64)
set(OPENCV_SUFFIX "ubuntu18")
set(OPENCV_HASH "db087dfd412eedb8161636ec083ada85ff278109948d1d62a06b0f52e1f04202")
elseif(OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.24 AND ARM)
set(OPENCV_SUFFIX "debian9arm")
set(OPENCV_HASH "4274f8c40b17215f4049096b524e4a330519f3e76813c5a3639b69c48633d34e")
elseif(OV_GLIBC_VERSION VERSION_GREATER_EQUAL 2.23 AND X86_64)
set(OPENCV_SUFFIX "ubuntu16")
set(OPENCV_HASH "cd46831b4d8d1c0891d8d22ff5b2670d0a465a8a8285243059659a50ceeae2c3")
# elseif((LINUX_OS_NAME STREQUAL "CentOS 8" OR
# LINUX_OS_NAME STREQUAL "CentOS 9") AND X86_64)
# set(OPENCV_SUFFIX "centos8")
# set(OPENCV_HASH "db087dfd412eedb8161636ec083ada85ff278109948d1d62a06b0f52e1f04202")
elseif(NOT DEFINED OpenCV_DIR AND NOT DEFINED ENV{OpenCV_DIR})
message(FATAL_ERROR "OpenCV is not available on current platform (OS = ${CMAKE_SYSTEM_NAME}, glibc ${OV_GLIBC_VERSION})")
endif()
RESOLVE_DEPENDENCY(OPENCV
ARCHIVE_LIN "opencv/opencv_${OPENCV_VERSION}-${OPENCV_BUILD}_${OPENCV_SUFFIX}.txz"
TARGET_PATH "${TEMP}/opencv_${OPENCV_VERSION}_${OPENCV_SUFFIX}/opencv"
ENVIRONMENT "OpenCV_DIR"
VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+).*"
SHA256 ${OPENCV_HASH})
endif()
endif()
if(ANDROID)
set(ocv_cmake_path "${OPENCV}/sdk/native/jni/")
else()
set(ocv_cmake_path "${OPENCV}/cmake")
endif()
update_deps_cache(OpenCV_DIR "${ocv_cmake_path}" "Path to OpenCV package folder")
debug_message(STATUS "opencv=" ${OPENCV})
else()
reset_deps_cache(OpenCV_DIR)
endif()
if(ENABLE_INTEL_GNA)
reset_deps_cache(
GNA_EXT_DIR
GNA_PLATFORM_DIR
GNA_KERNEL_LIB_NAME
GNA_LIBS_LIST
GNA_LIB_DIR
libGNA_INCLUDE_DIRS
libGNA_LIBRARIES_BASE_PATH)
set(GNA_VERSION "03.00.00.1910")
set(GNA_HASH "894ddbc0ae3459f04513b853b0cabc32890dd4ea37228a022b6a32101bdbb7f8")
set(FILES_TO_EXTRACT_LIST gna_${GNA_VERSION}/include)
if(WIN32)
LIST(APPEND FILES_TO_EXTRACT_LIST gna_${GNA_VERSION}/win64)
else()
LIST(APPEND FILES_TO_EXTRACT_LIST gna_${GNA_VERSION}/linux)
endif()
RESOLVE_DEPENDENCY(GNA_EXT_DIR
ARCHIVE_UNIFIED "gna/gna_${GNA_VERSION}.zip"
TARGET_PATH "${TEMP}/gna_${GNA_VERSION}"
VERSION_REGEX ".*_([0-9]+.[0-9]+.[0-9]+.[0-9]+).*"
FILES_TO_EXTRACT FILES_TO_EXTRACT_LIST
SHA256 ${GNA_HASH}
USE_NEW_LOCATION TRUE)
update_deps_cache(GNA_EXT_DIR "${GNA_EXT_DIR}" "Path to GNA root folder")
debug_message(STATUS "gna=" ${GNA_EXT_DIR})
if (WIN32)
set(GNA_PLATFORM_DIR win64 CACHE STRING "" FORCE)
elseif (UNIX)
set(GNA_PLATFORM_DIR linux CACHE STRING "" FORCE)
else ()
message(FATAL_ERROR "GNA not supported on this platform, only linux, and windows")
endif ()
set(GNA_LIB_DIR x64 CACHE STRING "" FORCE)
set(GNA_PATH ${GNA_EXT_DIR}/${GNA_PLATFORM_DIR}/${GNA_LIB_DIR} CACHE STRING "" FORCE)
if(NOT BUILD_SHARED_LIBS)
list(APPEND PATH_VARS "GNA_PATH")
endif()
RESOLVE_DEPENDENCY(SYSTEM_PROTOC_ROOT
ARCHIVE_LIN "protoc-${protoc_version}-linux-x86_64.tar.gz"
TARGET_PATH "${TEMP}/protoc-${protoc_version}-linux-x86_64"
SHA256 "a1bedd5c05ca51e49f8f254faa3d7331e05b3a806c151fb111d582f154d0fee8"
)
debug_message(STATUS "host protoc-${protoc_version} root path = " ${SYSTEM_PROTOC_ROOT})
reset_deps_cache(SYSTEM_PROTOC)
find_program(
SYSTEM_PROTOC
NAMES protoc
PATHS "${SYSTEM_PROTOC_ROOT}/bin"
NO_DEFAULT_PATH)
if(NOT SYSTEM_PROTOC)
message(FATAL_ERROR "[ONNX IMPORTER] Missing host protoc binary")
endif()
update_deps_cache(SYSTEM_PROTOC "${SYSTEM_PROTOC}" "Path to host protoc for ONNX Importer")
endif()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -14,8 +14,8 @@ set(CMAKE_MODULE_PATH "${IEDevScripts_DIR}")
function(set_ci_build_number)
set(repo_root "${CMAKE_SOURCE_DIR}")
include(version)
foreach(var CI_BUILD_NUMBER OpenVINO_VERSION OpenVINO_SOVERSION OpenVINO_VERSION_SUFFIX OpenVINO_VERSION_BUILD
OpenVINO_VERSION_MAJOR OpenVINO_VERSION_MINOR OpenVINO_VERSION_PATCH)
foreach(var CI_BUILD_NUMBER IE_VERSION
IE_VERSION_MAJOR IE_VERSION_MINOR IE_VERSION_PATCH)
if(NOT DEFINED ${var})
message(FATAL_ERROR "${var} version component is not defined")
endif()
@@ -23,28 +23,26 @@ function(set_ci_build_number)
endforeach()
endfunction()
set_ci_build_number()
include(features)
include(message)
set_ci_build_number()
#
# Detect target
#
include(target_flags)
string(TOLOWER "${CMAKE_SYSTEM_PROCESSOR}" ARCH_FOLDER)
string(TOLOWER ${CMAKE_SYSTEM_PROCESSOR} ARCH_FOLDER)
if(X86_64)
set(ARCH_FOLDER intel64)
elseif(X86)
set(ARCH_FOLDER ia32)
elseif(MSVC AND ARM)
set(ARCH_FOLDER arm)
elseif((MSVC OR APPLE) AND AARCH64)
elseif(MSVC AND AARCH64)
set(ARCH_FOLDER arm64)
elseif(UNIVERSAL2)
set(ARCH_FOLDER universal2)
endif()
#
@@ -52,10 +50,7 @@ endif()
#
function(set_temp_directory temp_variable source_tree_dir)
if(DEFINED OV_TEMP)
message(STATUS "OV_TEMP cmake variable is set : ${OV_TEMP}")
file(TO_CMAKE_PATH ${OV_TEMP} temp)
elseif (DEFINED ENV{DL_SDK_TEMP} AND NOT $ENV{DL_SDK_TEMP} STREQUAL "")
if (DEFINED ENV{DL_SDK_TEMP} AND NOT $ENV{DL_SDK_TEMP} STREQUAL "")
message(STATUS "DL_SDK_TEMP environment is set : $ENV{DL_SDK_TEMP}")
file(TO_CMAKE_PATH $ENV{DL_SDK_TEMP} temp)
else ()
@@ -79,11 +74,6 @@ if(NOT COMMAND find_host_package)
find_package(${ARGN})
endmacro()
endif()
if(NOT COMMAND find_host_library)
macro(find_host_library)
find_library(${ARGN})
endmacro()
endif()
if(NOT COMMAND find_host_program)
macro(find_host_program)
find_program(${ARGN})
@@ -94,6 +84,7 @@ endif()
# Common scripts
#
include(packaging)
include(coverage/coverage)
include(shellcheck/shellcheck)
@@ -112,11 +103,17 @@ else()
set(BIN_FOLDER "bin/${ARCH_FOLDER}")
endif()
set(CMAKE_BUILD_TYPE "Release" CACHE STRING "CMake build type")
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Release;Debug;RelWithDebInfo;MinSizeRel")
if(CMAKE_GENERATOR MATCHES "^Ninja Multi-Config$")
set(CMAKE_DEFAULT_BUILD_TYPE "Release" CACHE STRING "CMake default build type")
if(NOT DEFINED CMAKE_BUILD_TYPE OR CMAKE_BUILD_TYPE STREQUAL "")
message(STATUS "CMAKE_BUILD_TYPE not defined, 'Release' will be used")
set(CMAKE_BUILD_TYPE "Release")
else()
set(RELEASE_TYPES "Debug" "Release" "RelWithDebInfo" "MinSizeRel")
list(FIND RELEASE_TYPES ${CMAKE_BUILD_TYPE} INDEX_FOUND)
if (INDEX_FOUND EQUAL -1)
message(FATAL_ERROR "CMAKE_BUILD_TYPE must be one of Debug, Release, RelWithDebInfo, or MinSizeRel")
endif()
endif()
message(STATUS "CMAKE_BUILD_TYPE: ${CMAKE_BUILD_TYPE}")
if(USE_BUILD_TYPE_SUBFOLDER)
set(BIN_FOLDER "${BIN_FOLDER}/${CMAKE_BUILD_TYPE}")
@@ -124,10 +121,10 @@ endif()
# allow to override default OUTPUT_ROOT root
if(NOT DEFINED OUTPUT_ROOT)
if(NOT DEFINED OpenVINO_SOURCE_DIR)
message(FATAL_ERROR "OpenVINO_SOURCE_DIR is not defined")
if(NOT DEFINED OpenVINO_MAIN_SOURCE_DIR)
message(FATAL_ERROR "OpenVINO_MAIN_SOURCE_DIR is not defined")
endif()
set(OUTPUT_ROOT ${OpenVINO_SOURCE_DIR})
set(OUTPUT_ROOT ${OpenVINO_MAIN_SOURCE_DIR})
endif()
# Enable postfixes for Debug/Release builds
@@ -152,8 +149,8 @@ endif()
set(CMAKE_DEBUG_POSTFIX ${IE_DEBUG_POSTFIX})
set(CMAKE_RELEASE_POSTFIX ${IE_RELEASE_POSTFIX})
# Support CMake multi-configuration for Visual Studio / Ninja or Xcode build
if (OV_GENERATOR_MULTI_CONFIG)
if (MSVC OR CMAKE_GENERATOR STREQUAL "Xcode")
# Support CMake multiconfiguration for Visual Studio or Xcode build
set(IE_BUILD_POSTFIX $<$<CONFIG:Debug>:${IE_DEBUG_POSTFIX}>$<$<CONFIG:Release>:${IE_RELEASE_POSTFIX}>)
else ()
if (CMAKE_BUILD_TYPE STREQUAL "Debug")
@@ -162,42 +159,22 @@ else ()
set(IE_BUILD_POSTFIX ${IE_RELEASE_POSTFIX})
endif()
endif()
add_definitions(-DIE_BUILD_POSTFIX=\"${IE_BUILD_POSTFIX}\")
macro(ov_set_if_not_defined var value)
if(NOT DEFINED ${var})
set(${var} ${value})
endif()
endmacro()
ov_set_if_not_defined(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
ov_set_if_not_defined(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
ov_set_if_not_defined(CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
ov_set_if_not_defined(CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
ov_set_if_not_defined(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
if(CPACK_GENERATOR MATCHES "^(DEB|RPM)$")
# to make sure that lib/<multiarch-tuple> is created on Debian
set(CMAKE_INSTALL_PREFIX "/usr" CACHE PATH "Cmake install prefix" FORCE)
if(NOT UNIX)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
else()
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER}/lib)
endif()
include(packaging/packaging)
set(CMAKE_SKIP_INSTALL_RPATH ON)
set(CMAKE_COMPILE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
set(CMAKE_PDB_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${OUTPUT_ROOT}/${BIN_FOLDER})
if(APPLE)
set(CMAKE_INSTALL_RPATH_USE_LINK_PATH ON)
if(DEFINED OV_CPACK_LIBRARYDIR)
set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/${OV_CPACK_LIBRARYDIR}")
else()
message(FATAL_ERROR "Internal error: OV_CPACK_LIBRARYDIR is not defined, while it's required to initialize RPATH")
endif()
if(CPACK_GENERATOR STREQUAL "BREW")
set(CMAKE_SKIP_INSTALL_RPATH OFF)
endif()
set(CMAKE_MACOSX_RPATH ON)
# WA for Xcode generator + object libraries issue:
# https://gitlab.kitware.com/cmake/cmake/issues/20260
# http://cmake.3232098.n2.nabble.com/XCODE-DEPEND-HELPER-make-Deletes-Targets-Before-and-While-They-re-Built-td7598277.html
@@ -207,21 +184,8 @@ endif()
# Use solution folders
set_property(GLOBAL PROPERTY USE_FOLDERS ON)
# CMake 3.0+: Enable CMAKE_<LANG>_COMPILER_ID AppleClang
# Enable CMAKE_<LANG>_COMPILER_ID AppleClang
set(CMAKE_POLICY_DEFAULT_CMP0025 NEW)
# CMake 3.0+: MACOSX_RPATH is enabled by default.
set(CMAKE_POLICY_DEFAULT_CMP0026 NEW)
# CMake 3.0+ (2.8.12): MacOS "@rpath" in target's install name
set(CMAKE_POLICY_DEFAULT_CMP0042 NEW)
# CMake 3.9+: `RPATH` settings on macOS do not affect `install_name`.
set(CMAKE_POLICY_DEFAULT_CMP0068 NEW)
# CMake 3.13+: option() honors normal variables.
set(CMAKE_POLICY_DEFAULT_CMP0077 NEW)
# CMake 3.22+ :cmake_dependent_option() supports full Condition Syntax
set(CMAKE_POLICY_DEFAULT_CMP0127 NEW)
set(CMAKE_WARN_DEPRECATED OFF CACHE BOOL "Don't warn about obsolete cmake versions in 3rdparty")
set(CMAKE_WARN_ON_ABSOLUTE_INSTALL_DESTINATION ON CACHE BOOL "Warn about absolute paths in destination")
# LTO
@@ -241,17 +205,6 @@ endif()
# General flags
macro(ov_install_static_lib target comp)
if(NOT BUILD_SHARED_LIBS)
get_target_property(target_type ${target} TYPE)
if(${target_type} STREQUAL "STATIC_LIBRARY")
set_target_properties(${target} PROPERTIES EXCLUDE_FROM_ALL FALSE)
endif()
install(TARGETS ${target} EXPORT OpenVINOTargets
ARCHIVE DESTINATION ${OV_CPACK_ARCHIVEDIR} COMPONENT ${comp} ${ARGN})
endif()
endmacro()
set(THREADS_PREFER_PTHREAD_FLAG ON)
find_package(Threads REQUIRED)
@@ -269,7 +222,6 @@ include(api_validator/api_validator)
include(vs_version/vs_version)
include(plugins/plugins)
include(frontends/frontends)
include(add_ie_target)
include(CMakePackageConfigHelpers)
@@ -277,16 +229,10 @@ if(ENABLE_FUZZING)
enable_fuzzing()
endif()
get_linux_name(LINUX_OS_NAME)
# macro to mark target as conditionally compiled
function(ie_mark_target_as_cc TARGET_NAME)
set(cc_library openvino::conditional_compilation)
if(TARGET IE::conditional_compilation)
set(cc_library IE::conditional_compilation)
endif()
target_link_libraries(${TARGET_NAME} PRIVATE ${cc_library})
target_link_libraries(${TARGET_NAME} PRIVATE openvino::conditional_compilation)
if(NOT (SELECTIVE_BUILD STREQUAL "ON"))
return()
@@ -300,17 +246,10 @@ function(ie_mark_target_as_cc TARGET_NAME)
set_source_files_properties(${sources} PROPERTIES OBJECT_DEPENDS ${GENERATED_HEADER})
endfunction()
function(ov_mark_target_as_cc)
ie_mark_target_as_cc(${ARGN})
endfunction()
include(python_requirements)
# Code style utils
include(cpplint/cpplint)
include(clang_format/clang_format)
include(ncc_naming_style/ncc_naming_style)
# Restore state
set(CMAKE_MODULE_PATH ${OLD_CMAKE_MODULE_PATH})

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -23,14 +23,12 @@ else()
unset(IE_OWN_TBB_CONFIG)
endif()
unset(TBB_DIR)
unset(TBB_DIR CACHE)
find_package(TBB
2017.0
CONFIG
PATHS "${IEDevScripts_DIR}/${IE_OWN_TBB_CONFIG}"
PATHS ${TBBROOT}/cmake
${IEDevScripts_DIR}/${IE_OWN_TBB_CONFIG}
NO_CMAKE_FIND_ROOT_PATH
NO_DEFAULT_PATH
NO_CMAKE_FIND_ROOT_PATH)
)
find_package_handle_standard_args(TBB CONFIG_MODE)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -19,10 +19,11 @@ addIeTarget(
${SDL_INCLUDES}
/some/specific/path
LINK_LIBRARIES
link_dependencies
ie::important_plugin
EXPORT_DEPENDENCIES
dependency_lib_to_export
DEPENDENCIES
dependencies
ie::important_plugin
OBJECT_FILES
object libraries
)
@@ -50,6 +51,7 @@ function(addIeTarget)
EXCLUDED_SOURCE_PATHS # list of paths excluded from the global recursive search of source files
LINK_LIBRARIES_WHOLE_ARCHIVE # list of static libraries to link, each object file should be used and not discarded
LINK_FLAGS # list of extra commands to linker
EXPORT_DEPENDENCIES # list of the dependencies to be exported with the target through the developer package
)
cmake_parse_arguments(ARG "${options}" "${oneValueRequiredArgs};${oneValueOptionalArgs}" "${multiValueArgs}" ${ARGN} )
@@ -76,8 +78,8 @@ function(addIeTarget)
# remove unnecessary directories
foreach(excludedDir ${ARG_EXCLUDED_SOURCE_PATHS})
list(FILTER includes EXCLUDE REGEX "${excludedDir}.*")
list(FILTER sources EXCLUDE REGEX "${excludedDir}.*")
list(FILTER includes EXCLUDE REGEX "${excludedDir}*")
list(FILTER sources EXCLUDE REGEX "${excludedDir}*")
endforeach()
source_group("include" FILES ${includes})
@@ -125,7 +127,7 @@ function(addIeTarget)
if (ARG_DEVELOPER_PACKAGE)
# developer package
openvino_developer_export_targets(COMPONENT ${ARG_DEVELOPER_PACKAGE}
TARGETS ${ARG_NAME})
TARGETS ${ARG_NAME} ${ARG_EXPORT_DEPENDENCIES})
endif()
if(WIN32)
# Provide default compile pdb name equal to target name
@@ -133,10 +135,6 @@ function(addIeTarget)
endif()
endfunction()
function(ov_add_target)
addIeTarget(${ARGV})
endfunction()
#[[
Wrapper function over addIeTarget, that also adds a test with the same name.
You could use
@@ -151,27 +149,14 @@ function(addIeTargetTest)
NAME
)
set(oneValueOptionalArgs
COMPONENT
)
set(multiValueArgs
LABELS
)
cmake_parse_arguments(ARG "${options}" "${oneValueRequiredArgs};${oneValueOptionalArgs}" "${multiValueArgs}" ${ARGN} )
if (NOT DEFINED ARG_COMPONENT)
set(ARG_COMPONENT tests)
endif()
addIeTarget(TYPE EXECUTABLE NAME ${ARG_NAME} ${ARG_UNPARSED_ARGUMENTS})
add_test(NAME ${ARG_NAME} COMMAND ${ARG_NAME})
set_property(TEST ${ARG_NAME} PROPERTY LABELS ${ARG_LABELS})
install(TARGETS ${ARG_NAME}
RUNTIME DESTINATION tests
COMPONENT ${ARG_COMPONENT}
EXCLUDE_FROM_ALL)
endfunction()
function(ov_add_test_target)
addIeTargetTest(${ARGV})
endfunction()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -50,7 +50,7 @@ endfunction()
set(VALIDATED_LIBRARIES "" CACHE INTERNAL "")
function(_ov_add_api_validator_post_build_step)
function(_ie_add_api_validator_post_build_step)
set(UWP_API_VALIDATOR_APIS "${PROGRAMFILES}/Windows Kits/10/build/universalDDIs/x64/UniversalDDIs.xml")
set(UWP_API_VALIDATOR_EXCLUSION "${UWP_SDK_PATH}/BinaryExclusionlist.xml")
@@ -73,20 +73,8 @@ function(_ov_add_api_validator_post_build_step)
_ie_add_api_validator_post_build_step_recursive(TARGET ${API_VALIDATOR_TARGET})
# remove targets which were tested before
foreach(target IN LISTS API_VALIDATOR_TARGETS)
list(FIND VALIDATED_LIBRARIES ${target} index)
if (NOT index EQUAL -1)
list(APPEND VALIDATED_TARGETS ${target})
endif()
if(TARGET "${target}")
get_target_property(orig_target ${target} ALIASED_TARGET)
list(FIND VALIDATED_LIBRARIES ${orig_target} index)
if (NOT index EQUAL -1)
list(APPEND VALIDATED_TARGETS ${target})
endif()
endif()
endforeach()
foreach(item IN LISTS VALIDATED_TARGETS)
foreach(item IN LISTS VALIDATED_LIBRARIES)
list(REMOVE_ITEM API_VALIDATOR_TARGETS ${item})
endforeach()
@@ -100,12 +88,9 @@ function(_ov_add_api_validator_post_build_step)
macro(api_validator_get_target_name)
get_target_property(IS_IMPORTED ${target} IMPORTED)
get_target_property(orig_target ${target} ALIASED_TARGET)
if(IS_IMPORTED)
get_target_property(target_location ${target} LOCATION)
get_target_property(target_location ${target} LOCATION)
get_filename_component(target_name "${target_location}" NAME_WE)
elseif(TARGET "${orig_target}")
set(target_name ${orig_target})
else()
set(target_name ${target})
endif()
@@ -113,14 +98,10 @@ function(_ov_add_api_validator_post_build_step)
foreach(target IN LISTS API_VALIDATOR_TARGETS)
api_validator_get_target_name()
if(CMAKE_VERSION VERSION_GREATER_EQUAL 3.21 AND OV_GENERATOR_MULTI_CONFIG)
set(output_file "${CMAKE_BINARY_DIR}/api_validator/$<CONFIG>/${target_name}.txt")
else()
set(output_file "${CMAKE_BINARY_DIR}/api_validator/${target_name}.txt")
endif()
set(output_file "${CMAKE_BINARY_DIR}/api_validator/${target_name}.txt")
add_custom_command(TARGET ${API_VALIDATOR_TARGET} POST_BUILD
COMMAND ${CMAKE_COMMAND} --config $<CONFIG>
COMMAND ${CMAKE_COMMAND}
-D UWP_API_VALIDATOR=${UWP_API_VALIDATOR}
-D UWP_API_VALIDATOR_TARGET=$<TARGET_FILE:${target}>
-D UWP_API_VALIDATOR_APIS=${UWP_API_VALIDATOR_APIS}
@@ -143,5 +124,5 @@ endfunction()
# ie_add_api_validator_post_build_step(TARGET <name>)
#
macro(ie_add_api_validator_post_build_step)
_ov_add_api_validator_post_build_step(${ARGV})
_ie_add_api_validator_post_build_step(${ARGV})
endmacro()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,25 +1,24 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_CLANG_FORMAT)
set(clang_format_required_version 9)
set(CLANG_FORMAT_FILENAME clang-format-${clang_format_required_version} clang-format)
find_host_program(CLANG_FORMAT NAMES ${CLANG_FORMAT_FILENAME} PATHS ENV PATH)
if(CLANG_FORMAT)
if (ENABLE_CLANG_FORMAT)
set(CLANG_FORMAT_FILENAME clang-format-9 clang-format)
find_program(CLANG_FORMAT NAMES ${CLANG_FORMAT_FILENAME} PATHS ENV PATH)
if (CLANG_FORMAT)
execute_process(COMMAND ${CLANG_FORMAT} ${CMAKE_CURRENT_SOURCE_DIR} ARGS --version OUTPUT_VARIABLE CLANG_VERSION)
if(NOT CLANG_VERSION)
message(WARNING "Supported clang-format version is ${clang_format_required_version}!")
if (NOT CLANG_VERSION OR CLANG_VERSION STREQUAL "")
message(WARNING "Supported clang-format version is 9!")
set(ENABLE_CLANG_FORMAT OFF)
else()
string(REGEX REPLACE "[^0-9]+([0-9]+)\\..*" "\\1" CLANG_FORMAT_MAJOR_VERSION ${CLANG_VERSION})
if(NOT CLANG_FORMAT_MAJOR_VERSION EQUAL clang_format_required_version)
message(WARNING "Supported clang-format version is 9! Provided version ${CLANG_FORMAT_MAJOR_VERSION}")
if (NOT ${CLANG_FORMAT_MAJOR_VERSION} EQUAL "9")
message(WARNING "Supported clang-format version is 9!")
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
else()
message(WARNING "Supported clang-format-${clang_format_required_version} is not found!")
message(WARNING "Supported clang-format version is not found!")
set(ENABLE_CLANG_FORMAT OFF)
endif()
endif()
@@ -29,6 +28,7 @@ if(ENABLE_CLANG_FORMAT AND NOT TARGET clang_format_check_all)
add_custom_target(clang_format_fix_all)
set_target_properties(clang_format_check_all clang_format_fix_all
PROPERTIES FOLDER clang_format)
set(CLANG_FORMAT_ALL_OUTPUT_FILES "" CACHE INTERNAL "All clang-format output files")
endif()
function(add_clang_format_target TARGET_NAME)
@@ -88,10 +88,14 @@ function(add_clang_format_target TARGET_NAME)
"[clang-format] ${source_file}"
VERBATIM)
list(APPEND all_input_sources "${source_file}")
list(APPEND all_output_files "${output_file}")
endforeach()
set(CLANG_FORMAT_ALL_OUTPUT_FILES
${CLANG_FORMAT_ALL_OUTPUT_FILES} ${all_output_files}
CACHE INTERNAL
"All clang-format output files")
add_custom_target(${TARGET_NAME}
DEPENDS ${all_output_files}
COMMENT "[clang-format] ${TARGET_NAME}")
@@ -100,11 +104,11 @@ function(add_clang_format_target TARGET_NAME)
COMMAND
"${CMAKE_COMMAND}"
-D "CLANG_FORMAT=${CLANG_FORMAT}"
-D "INPUT_FILES=${all_input_sources}"
-D "INPUT_FILES=${CLANG_FORMAT_FOR_SOURCES}"
-D "EXCLUDE_PATTERNS=${CLANG_FORMAT_EXCLUDE_PATTERNS}"
-P "${IEDevScripts_DIR}/clang_format/clang_format_fix.cmake"
DEPENDS
"${all_input_sources}"
"${CLANG_FORMAT_FOR_SOURCES}"
"${IEDevScripts_DIR}/clang_format/clang_format_fix.cmake"
COMMENT
"[clang-format] ${TARGET_NAME}_fix"

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,9 +1,8 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(ProcessorCount)
include(CheckCXXCompilerFlag)
#
# Disables deprecated warnings generation
@@ -28,8 +27,6 @@ macro(disable_deprecated_warnings)
message(WARNING "Unsupported CXX compiler ${CMAKE_CXX_COMPILER_ID}")
endif()
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${ie_c_cxx_deprecated}")
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} ${ie_c_cxx_deprecated}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${ie_c_cxx_deprecated}")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${ie_c_cxx_deprecated}")
endmacro()
@@ -44,9 +41,7 @@ macro(ie_deprecated_no_errors)
set(ie_c_cxx_deprecated_no_errors "/Qdiag-warning:1478,1786")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# show 4996 only for /w4
set(ie_c_cxx_deprecated_no_errors "/wd4996")
# WA for VPUX plugin
set(ie_c_cxx_deprecated_no_errors "${ie_c_cxx_deprecated_no_errors} /wd4146 /wd4703")
set(ie_c_cxx_deprecated_no_errors "/w44996")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
@@ -60,8 +55,6 @@ macro(ie_deprecated_no_errors)
endif()
endif()
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${ie_c_cxx_deprecated_no_errors}")
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} ${ie_c_cxx_deprecated_no_errors}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${ie_c_cxx_deprecated_no_errors}")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${ie_c_cxx_deprecated_no_errors}")
endmacro()
@@ -69,119 +62,93 @@ endmacro()
#
# Provides SSE4.2 compilation flags depending on an OS and a compiler
#
macro(ie_sse42_optimization_flags flags)
function(ie_sse42_optimization_flags flags)
if(WIN32)
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# No such option for MSVC 2019
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} /QxSSE4.2)
set(${flags} "/arch:SSE4.2 /QxSSE4.2" PARENT_SCOPE)
else()
message(WARNING "Unsupported CXX compiler ${CMAKE_CXX_COMPILER_ID}")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} -xSSE4.2)
set(${flags} "-msse4.2 -xSSE4.2" PARENT_SCOPE)
else()
set(${flags} -msse4.2)
set(${flags} "-msse4.2" PARENT_SCOPE)
endif()
endif()
endmacro()
endfunction()
#
# Provides AVX2 compilation flags depending on an OS and a compiler
#
macro(ie_avx2_optimization_flags flags)
function(ie_avx2_optimization_flags flags)
if(WIN32)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} /QxCORE-AVX2)
set(${flags} "/QxCORE-AVX2" PARENT_SCOPE)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
set(${flags} /arch:AVX2)
set(${flags} "/arch:AVX2" PARENT_SCOPE)
else()
message(WARNING "Unsupported CXX compiler ${CMAKE_CXX_COMPILER_ID}")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} -xCORE-AVX2)
set(${flags} "-march=core-avx2 -xCORE-AVX2 -mtune=core-avx2" PARENT_SCOPE)
else()
set(${flags} -mavx2 -mfma)
set(${flags} "-mavx2 -mfma" PARENT_SCOPE)
endif()
endif()
endmacro()
endfunction()
#
# Provides common AVX512 compilation flags for AVX512F instruction set support
# depending on an OS and a compiler
#
macro(ie_avx512_optimization_flags flags)
function(ie_avx512_optimization_flags flags)
if(WIN32)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} /QxCOMMON-AVX512)
set(${flags} "/QxCOMMON-AVX512" PARENT_SCOPE)
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
set(${flags} /arch:AVX512)
set(${flags} "/arch:AVX512" PARENT_SCOPE)
else()
message(WARNING "Unsupported CXX compiler ${CMAKE_CXX_COMPILER_ID}")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
set(${flags} -xCOMMON-AVX512)
set(${flags} "-xCOMMON-AVX512" PARENT_SCOPE)
endif()
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(${flags} -mavx512f -mfma)
set(${flags} "-mavx512f -mfma" PARENT_SCOPE)
endif()
if(CMAKE_CXX_COMPILER_ID MATCHES "^(Clang|AppleClang)$")
set(${flags} -mavx512f -mfma)
set(${flags} "-mavx512f -mfma" PARENT_SCOPE)
endif()
endif()
endmacro()
endfunction()
macro(ie_arm_neon_optimization_flags flags)
function(ie_arm_neon_optimization_flags flags)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
message(WARNING "Unsupported CXX compiler ${CMAKE_CXX_COMPILER_ID}")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# nothing
elseif(ANDROID)
if(ANDROID_ABI STREQUAL "arm64-v8a")
set(${flags} -mfpu=neon -Wno-unused-command-line-argument)
set(${flags} "-mfpu=neon" PARENT_SCOPE)
elseif(ANDROID_ABI STREQUAL "armeabi-v7a-hard with NEON")
set(${flags} -march=armv7-a -mfloat-abi=hard -mhard-float -D_NDK_MATH_NO_SOFTFP=1 -mfpu=neon -Wno-unused-command-line-argument)
set(${flags} "-march=armv7-a -mfloat-abi=hard -mhard-float -D_NDK_MATH_NO_SOFTFP=1 -mfpu=neon" PARENT_SCOPE)
elseif((ANDROID_ABI STREQUAL "armeabi-v7a with NEON") OR
(ANDROID_ABI STREQUAL "armeabi-v7a" AND
DEFINED CMAKE_ANDROID_ARM_NEON AND CMAKE_ANDROID_ARM_NEON))
set(${flags} -march=armv7-a -mfloat-abi=softfp -mfpu=neon -Wno-unused-command-line-argument)
set(${flags} "-march=armv7-a -mfloat-abi=softfp -mfpu=neon" PARENT_SCOPE)
endif()
else()
if(AARCH64)
set(${flags} -O2 -ftree-vectorize)
set(${flags} "-O2 -ftree-vectorize" PARENT_SCOPE)
elseif(ARM)
set(${flags} -mfpu=neon -Wno-unused-command-line-argument)
set(${flags} "-mfpu=neon" PARENT_SCOPE)
endif()
endif()
endmacro()
#
# Disables all warnings for 3rd party targets
#
function(ov_disable_all_warnings)
foreach(target IN LISTS ARGN)
get_target_property(target_type ${target} TYPE)
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
target_compile_options(${target} PRIVATE /WX-)
elseif(CMAKE_COMPILER_IS_GNUCXX OR OV_COMPILER_IS_CLANG)
target_compile_options(${target} PRIVATE -w)
# required for LTO
set(link_interface INTERFACE_LINK_OPTIONS)
if(target_type STREQUAL "SHARED_LIBRARY" OR target_type STREQUAL "EXECUTABLE")
set(link_interface LINK_OPTIONS)
endif()
set_target_properties(${target} PROPERTIES ${link_interface} "-Wno-error=maybe-uninitialized;-Wno-maybe-uninitialized")
elseif(UNIX AND CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
# 193: zero used for undefined preprocessing identifier "XXX"
# 1011: missing return statement at end of non-void function "XXX"
# 2415: variable "xxx" of static storage duration was declared but never referenced
target_compile_options(${target} PRIVATE -diag-disable=warn,193,1011,2415)
endif()
endforeach()
endfunction()
#
@@ -201,10 +168,6 @@ macro(ie_add_compiler_flags)
endforeach()
endmacro()
function(ov_add_compiler_flags)
ie_add_compiler_flags(${ARGN})
endfunction()
#
# Forced includes certain header file to all target source files
#
@@ -264,15 +227,12 @@ if(WIN32)
ie_add_compiler_flags(/Gy) # remove unreferenced functions: function level linking
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /LARGEADDRESSAWARE")
if (CMAKE_COMPILE_WARNING_AS_ERROR)
if (CMAKE_VERSION VERSION_LESS 3.24)
ie_add_compiler_flags(/WX)
endif()
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} /WX")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} /WX")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} /WX")
if (TREAT_WARNING_AS_ERROR)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
ie_add_compiler_flags(/WX)
ie_add_compiler_flags(/Qdiag-warning:47,1740,1786)
elseif (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
# ie_add_compiler_flags(/WX) # Too many warnings
endif()
endif()
@@ -313,38 +273,35 @@ if(WIN32)
string(REPLACE "/Zi" "/Z7" CMAKE_C_FLAGS_RELWITHDEBINFO "${CMAKE_C_FLAGS_RELWITHDEBINFO}")
string(REPLACE "/Zi" "/Z7" CMAKE_CXX_FLAGS_RELWITHDEBINFO "${CMAKE_CXX_FLAGS_RELWITHDEBINFO}")
else()
if(CMAKE_COMPILE_WARNING_AS_ERROR AND CMAKE_VERSION VERSION_LESS 3.24)
# TODO: enable for C sources as well
# ie_add_compiler_flags(-Werror)
# TODO: enable for C sources as well
# ie_add_compiler_flags(-Werror)
if(TREAT_WARNING_AS_ERROR)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Werror")
endif()
ie_add_compiler_flags(-ffunction-sections -fdata-sections)
ie_add_compiler_flags(-fdiagnostics-show-option)
ie_add_compiler_flags(-Wundef)
ie_add_compiler_flags(-Wreturn-type)
ie_add_compiler_flags(-Wunused-variable)
if(OV_COMPILER_IS_APPLECLANG)
# Disable noisy warnings
if (CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang")
ie_add_compiler_flags(-Wswitch)
else()
elseif(UNIX)
ie_add_compiler_flags(-Wuninitialized -Winit-self)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
ie_add_compiler_flags(-Winconsistent-missing-override
-Wstring-plus-int)
ie_add_compiler_flags(-Wno-error=switch)
else()
ie_add_compiler_flags(-Wmaybe-uninitialized)
check_cxx_compiler_flag("-Wsuggest-override" SUGGEST_OVERRIDE_SUPPORTED)
if(SUGGEST_OVERRIDE_SUPPORTED)
set(CMAKE_CXX_FLAGS "-Wsuggest-override ${CMAKE_CXX_FLAGS}")
endif()
endif()
endif()
# Disable noisy warnings
if(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
# 177: function "XXX" was declared but never referenced
ie_add_compiler_flags(-diag-disable=remark,177,2196)
ie_add_compiler_flags(-diag-disable=remark)
# noisy warnings from Intel Compiler 19.1.1.217 20200306
ie_add_compiler_flags(-diag-disable=2196)
endif()
# Linker flags
@@ -353,41 +310,8 @@ else()
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,-dead_strip")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -Wl,-dead_strip")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-dead_strip")
else()
elseif(LINUX)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} -Wl,--gc-sections -Wl,--exclude-libs,ALL")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} -Wl,--gc-sections -Wl,--exclude-libs,ALL")
if(NOT ENABLE_FUZZING)
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--exclude-libs,ALL")
endif()
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,--gc-sections")
endif()
endif()
# Links provided libraries and include their INTERFACE_INCLUDE_DIRECTORIES as SYSTEM
function(link_system_libraries TARGET_NAME)
set(MODE PRIVATE)
foreach(arg IN LISTS ARGN)
if(arg MATCHES "(PRIVATE|PUBLIC|INTERFACE)")
set(MODE ${arg})
else()
if(TARGET "${arg}")
target_include_directories(${TARGET_NAME}
SYSTEM ${MODE}
$<TARGET_PROPERTY:${arg},INTERFACE_INCLUDE_DIRECTORIES>
$<TARGET_PROPERTY:${arg},INTERFACE_SYSTEM_INCLUDE_DIRECTORIES>
)
endif()
target_link_libraries(${TARGET_NAME} ${MODE} ${arg})
endif()
endforeach()
endfunction()
function(ov_try_use_gold_linker)
# gold linker on ubuntu20.04 may fail to link binaries build with sanitizer
if(CMAKE_COMPILER_IS_GNUCXX AND NOT ENABLE_SANITIZER AND NOT CMAKE_CROSSCOMPILING)
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fuse-ld=gold" PARENT_SCOPE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fuse-ld=gold" PARENT_SCOPE)
endif()
endfunction()

View File

@@ -1,88 +1,26 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(CheckCXXCompilerFlag)
if (ENABLE_SANITIZER)
if (WIN32)
check_cxx_compiler_flag("/fsanitize=address" SANITIZE_ADDRESS_SUPPORTED)
if (SANITIZE_ADDRESS_SUPPORTED)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} /fsanitize=address")
else()
message(FATAL_ERROR "Address sanitizer is not supported by current compiler.\n"
"Please, check requirements:\n"
"https://github.com/openvinotoolkit/openvino/wiki/AddressSanitizer-and-LeakSanitizer")
endif()
else()
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize=address")
check_cxx_compiler_flag("-fsanitize-recover=address" SANITIZE_RECOVER_ADDRESS_SUPPORTED)
if (SANITIZE_RECOVER_ADDRESS_SUPPORTED)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize-recover=address")
endif()
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fsanitize=address")
endif()
endif()
if (ENABLE_UB_SANITIZER)
if (WIN32)
message(FATAL_ERROR "UndefinedBehavior sanitizer is not supported in Windows")
endif()
# TODO: Remove -fno-sanitize=null as thirdparty/ocl/clhpp_headers UBSAN compatibility resolved:
# https://github.com/KhronosGroup/OpenCL-CLHPP/issues/17
# Mute -fsanitize=function Indirect call of a function through a function pointer of the wrong type.
# Sample cases:
# call to function GetAPIVersion through pointer to incorrect function type 'void *(*)()'
# Mute -fsanitize=alignment Use of a misaligned pointer or creation of a misaligned reference. Also sanitizes assume_aligned-like attributes.
# Sample cases:
# VPU_FixedMaxHeapTest.DefaultConstructor test case load of misaligned address 0x62000000187f for type 'const DataType', which requires 4 byte alignment
# Mute -fsanitize=bool Load of a bool value which is neither true nor false.
# Samples cases:
# ie_c_api_version.apiVersion test case load of value 32, which is not a valid value for type 'bool'
# Mute -fsanitize=enum Load of a value of an enumerated type which is not in the range of representable values for that enumerated type.
# Samples cases:
# load of value 4294967295, which is not a valid value for type 'const (anonymous namespace)::onnx::Field'
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize=undefined -fno-sanitize=null -fno-sanitize=alignment -fno-sanitize=bool -fno-sanitize=enum")
if(OV_COMPILER_IS_CLANG)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fno-sanitize=function")
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=address -fno-omit-frame-pointer")
CHECK_CXX_COMPILER_FLAG("-fsanitize-recover=address" SANITIZE_RECOVER_SUPPORTED)
if (SANITIZE_RECOVER_SUPPORTED)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize-recover=address")
endif()
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
# TODO: Remove -Wno-maybe-uninitialized after CVS-61143 fix
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -Wno-maybe-uninitialized")
endif()
check_cxx_compiler_flag("-fsanitize-recover=undefined" SANITIZE_RECOVER_UNDEFINED_SUPPORTED)
if (SANITIZE_RECOVER_UNDEFINED_SUPPORTED)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize-recover=undefined")
endif()
set(SANITIZER_LINKER_FLAGS "-fsanitize=address")
# prevent unloading libraries at runtime, so sanitizer can resolve their symbols
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fsanitize=undefined")
endif()
if (ENABLE_THREAD_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fsanitize=thread")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fsanitize=thread")
endif()
# common sanitizer options
if (DEFINED SANITIZER_COMPILER_FLAGS)
# ensure symbols are present
if (NOT WIN32)
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -g -fno-omit-frame-pointer")
if(NOT OV_COMPILER_IS_CLANG)
# GPU plugin tests compilation is slow with -fvar-tracking-assignments on GCC.
# Clang has no var-tracking-assignments.
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} -fno-var-tracking-assignments")
endif()
# prevent unloading libraries at runtime, so sanitizer can resolve their symbols
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(OV_COMPILER_IS_CLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=gold")
elseif(OV_COMPILER_IS_CLANG AND NOT WIN32)
if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=lld")
endif()
else()
set(SANITIZER_COMPILER_FLAGS "${SANITIZER_COMPILER_FLAGS} /Oy-")
endif()
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${SANITIZER_COMPILER_FLAGS}")
@@ -90,4 +28,23 @@ if (DEFINED SANITIZER_COMPILER_FLAGS)
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
endif()
endif()
if (ENABLE_THREAD_SANITIZER)
set(SANITIZER_COMPILER_FLAGS "-g -fsanitize=thread -fno-omit-frame-pointer")
set(SANITIZER_LINKER_FLAGS "-fsanitize=thread")
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -Wl,-z,nodelete")
if(OV_COMPILER_IS_CLANG AND NOT WIN32)
if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 8.0)
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -fuse-ld=lld")
else()
set(SANITIZER_LINKER_FLAGS "${SANITIZER_LINKER_FLAGS} -static-libsan")
endif()
endif()
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${SANITIZER_COMPILER_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${SANITIZER_COMPILER_FLAGS}")
set(CMAKE_SHARED_LINKER_FLAGS "${CMAKE_SHARED_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
set(CMAKE_MODULE_LINKER_FLAGS "${CMAKE_MODULE_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} ${SANITIZER_LINKER_FLAGS}")
endif()

View File

@@ -1,49 +1,50 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(UNIX)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -Wformat -Wformat-security")
if (NOT ENABLE_SANITIZER)
# ASan does not support fortification https://github.com/google/sanitizers/issues/247
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -D_FORTIFY_SOURCE=2")
endif()
if(NOT APPLE)
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} -pie")
endif()
if (CMAKE_BUILD_TYPE STREQUAL "Release")
if(UNIX)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -Wformat -Wformat-security")
if (NOT ENABLE_SANITIZER)
# ASan does not support fortification https://github.com/google/sanitizers/issues/247
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -D_FORTIFY_SOURCE=2")
endif()
if(NOT APPLE)
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} -pie")
endif()
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(IE_LINKER_FLAGS "${IE_LINKER_FLAGS} -z noexecstack -z relro -z now")
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fno-strict-overflow -fno-delete-null-pointer-checks -fwrapv")
if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.9)
if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
set(IE_LINKER_FLAGS "${IE_LINKER_FLAGS} -z noexecstack -z relro -z now")
if(CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.9)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-all")
else()
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-strong")
endif()
if (NOT ENABLE_SANITIZER)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -s")
endif()
elseif(OV_COMPILER_IS_CLANG)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-all")
else()
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
if (NOT ENABLE_SANITIZER)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -Wl,--strip-all")
endif()
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-strong")
set(IE_LINKER_FLAGS "${IE_LINKER_FLAGS} -z noexecstack -z relro -z now")
endif()
if (NOT ENABLE_SANITIZER)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -s")
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} /sdl")
endif()
elseif(OV_COMPILER_IS_CLANG)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-all")
elseif(CMAKE_CXX_COMPILER_ID STREQUAL "Intel")
if (NOT ENABLE_SANITIZER)
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -Wl,--strip-all")
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} /guard:cf")
if(ENABLE_INTEGRITYCHECK)
set(CMAKE_SHARED_LINKER_FLAGS_RELEASE "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} /INTEGRITYCHECK")
endif()
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} -fstack-protector-strong")
set(IE_LINKER_FLAGS "${IE_LINKER_FLAGS} -z noexecstack -z relro -z now")
endif()
else()
if(CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} /sdl")
endif()
set(IE_C_CXX_FLAGS "${IE_C_CXX_FLAGS} /guard:cf")
if(ENABLE_INTEGRITYCHECK)
set(CMAKE_SHARED_LINKER_FLAGS_RELEASE "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} /INTEGRITYCHECK")
endif()
endif()
set(CMAKE_C_FLAGS_RELEASE "${CMAKE_C_FLAGS_RELEASE} ${IE_C_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS_RELEASE "${CMAKE_CXX_FLAGS_RELEASE} ${IE_C_CXX_FLAGS}")
set(CMAKE_SHARED_LINKER_FLAGS_RELEASE "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
set(CMAKE_MODULE_LINKER_FLAGS_RELEASE "${CMAKE_MODULE_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${IE_C_CXX_FLAGS}")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${IE_C_CXX_FLAGS}")
set(CMAKE_SHARED_LINKER_FLAGS_RELEASE "${CMAKE_SHARED_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
set(CMAKE_MODULE_LINKER_FLAGS_RELEASE "${CMAKE_MODULE_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
set(CMAKE_EXE_LINKER_FLAGS_RELEASE "${CMAKE_EXE_LINKER_FLAGS_RELEASE} ${IE_LINKER_FLAGS}")
endif()

View File

@@ -1,234 +1,212 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(OV_COVERAGE_GCDA_DATA_DIRECTORY "${CMAKE_BINARY_DIR}")
if (NOT TARGET ov_coverage)
add_custom_target(ov_coverage)
set_target_properties(ov_coverage PROPERTIES FOLDER coverage)
if(NOT TARGET ie_coverage_clean)
add_custom_target(ie_coverage_clean)
set_target_properties(ie_coverage_clean PROPERTIES FOLDER coverage)
endif()
if(NOT TARGET ov_coverage_clean)
add_custom_target(ov_coverage_clean)
set_target_properties(ov_coverage_clean PROPERTIES FOLDER coverage)
if(NOT TARGET ie_coverage_init)
add_custom_target(ie_coverage_init)
set_target_properties(ie_coverage_init PROPERTIES FOLDER coverage)
endif()
if(NOT TARGET ie_coverage)
add_custom_target(ie_coverage)
set_target_properties(ie_coverage PROPERTIES FOLDER coverage)
endif()
set(OV_COVERAGE_REPORTS "${CMAKE_BINARY_DIR}/coverage")
set(OV_COVERAGE_SCRIPT_DIR "${IEDevScripts_DIR}/coverage")
set(IE_COVERAGE_REPORTS "${CMAKE_BINARY_DIR}/coverage")
set(IE_COVERAGE_SCRIPT_DIR "${IEDevScripts_DIR}/coverage")
include(CMakeParseArguments)
#
# ov_coverage_clean(REPOSITORY <repo> DIRECTORY <dir>)
# ie_coverage_clean(REPOSITORY <repo> DIRECTORY <dir>)
#
function(ov_coverage_clean)
cmake_parse_arguments(OV_COVERAGE "" "REPOSITORY;DIRECTORY" "" ${ARGN})
function(ie_coverage_clean)
cmake_parse_arguments(IE_COVERAGE "" "REPOSITORY;DIRECTORY" "" ${ARGN})
add_custom_target(ov_coverage_zerocounters_${OV_COVERAGE_REPOSITORY}
add_custom_target(ie_coverage_zerocounters_${IE_COVERAGE_REPOSITORY}
COMMAND lcov --zerocounters --quiet
--directory "${OV_COVERAGE_DIRECTORY}"
COMMENT "Add zero counters for coverage for ${OV_COVERAGE_REPOSITORY}"
--directory "${IE_COVERAGE_DIRECTORY}"
COMMENT "Add zero counters for coverage for ${IE_COVERAGE_REPOSITORY}"
VERBATIM)
add_custom_target(ov_coverage_clean_${OV_COVERAGE_REPOSITORY}
add_custom_target(ie_coverage_clean_${IE_COVERAGE_REPOSITORY}
COMMAND ${CMAKE_COMMAND}
-D "OV_COVERAGE_REPORTS=${OV_COVERAGE_REPORTS}"
-D "OV_COVERAGE_DIRECTORY=${OV_COVERAGE_DIRECTORY}"
-D "IE_COVERAGE_REPORTS=${IE_COVERAGE_REPORTS}"
-D "IE_COVERAGE_DIRECTORY=${IE_COVERAGE_DIRECTORY}"
-D "CMAKE_BINARY_DIRECTORY=${CMAKE_BINARY_DIR}"
-D "CMAKE_SOURCE_DIRECTORY=${CMAKE_SOURCE_DIR}"
-P "${OV_COVERAGE_SCRIPT_DIR}/coverage_clean.cmake"
COMMENT "Clean previously created HTML report files for ${OV_COVERAGE_REPOSITORY}"
DEPENDS "${OV_COVERAGE_SCRIPT_DIR}/coverage_clean.cmake"
-P "${IE_COVERAGE_SCRIPT_DIR}/coverage_clean.cmake"
COMMENT "Clean previously created HTML report files for ${IE_COVERAGE_REPOSITORY}"
DEPENDS "${IE_COVERAGE_SCRIPT_DIR}/coverage_clean.cmake"
VERBATIM)
set_target_properties(ov_coverage_zerocounters_${OV_COVERAGE_REPOSITORY}
ov_coverage_clean_${OV_COVERAGE_REPOSITORY}
set_target_properties(ie_coverage_zerocounters_${IE_COVERAGE_REPOSITORY}
ie_coverage_clean_${IE_COVERAGE_REPOSITORY}
PROPERTIES FOLDER coverage)
add_dependencies(ov_coverage_clean ov_coverage_zerocounters_${OV_COVERAGE_REPOSITORY}
ov_coverage_clean_${OV_COVERAGE_REPOSITORY})
add_dependencies(ie_coverage_clean ie_coverage_zerocounters_${IE_COVERAGE_REPOSITORY}
ie_coverage_clean_${IE_COVERAGE_REPOSITORY})
endfunction()
#
# ov_coverage_capture(INFO_FILE <info_file>
# ie_coverage_capture(INFO_FILE <info_file>
# BASE_DIRECTORY <base dir>
# DIRECTORY <gcda dir>
# EXCLUDE_PATTERNS exclude_patterns,...)
# DIRECTORY <gcda dir>)
#
function(ov_coverage_capture)
cmake_parse_arguments(OV_COVERAGE "" "INFO_FILE;BASE_DIRECTORY;DIRECTORY" "EXCLUDE_PATTERNS" ${ARGN})
function(ie_coverage_capture)
cmake_parse_arguments(IE_COVERAGE "" "INFO_FILE;BASE_DIRECTORY;DIRECTORY" "" ${ARGN})
set(output_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INFO_FILE}.info")
set(output_base_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INFO_FILE}_base.info")
set(output_tests_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INFO_FILE}_tests.info")
set(output_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INFO_FILE}.info")
set(output_base_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INFO_FILE}_base.info")
set(output_tests_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INFO_FILE}_tests.info")
add_custom_command(OUTPUT ${output_base_file}
COMMAND ${CMAKE_COMMAND} -E make_directory "${OV_COVERAGE_REPORTS}"
COMMAND ${CMAKE_COMMAND} -E make_directory "${IE_COVERAGE_REPORTS}"
COMMAND lcov --no-external --capture --initial --quiet
--directory "${OV_COVERAGE_DIRECTORY}"
--base-directory "${OV_COVERAGE_BASE_DIRECTORY}"
--directory "${IE_COVERAGE_DIRECTORY}"
--base-directory "${IE_COVERAGE_BASE_DIRECTORY}"
--output-file ${output_base_file}
COMMENT "Capture initial coverage data ${OV_COVERAGE_INFO_FILE}"
COMMENT "Capture initial coverage data ${IE_COVERAGE_INFO_FILE}"
VERBATIM)
add_custom_command(OUTPUT ${output_tests_file}
COMMAND ${CMAKE_COMMAND} -E make_directory "${OV_COVERAGE_REPORTS}"
COMMAND ${CMAKE_COMMAND} -E make_directory "${IE_COVERAGE_REPORTS}"
COMMAND lcov --no-external --capture --quiet
--directory "${OV_COVERAGE_DIRECTORY}"
--base-directory "${OV_COVERAGE_BASE_DIRECTORY}"
--directory "${IE_COVERAGE_DIRECTORY}"
--base-directory "${IE_COVERAGE_BASE_DIRECTORY}"
--output-file ${output_tests_file}
COMMENT "Capture test coverage data ${OV_COVERAGE_INFO_FILE}"
COMMENT "Capture test coverage data ${IE_COVERAGE_INFO_FILE}"
VERBATIM)
if (OV_COVERAGE_EXCLUDE_PATTERNS)
set(out_suf ".tmp")
endif()
add_custom_command(OUTPUT ${output_file}${out_suf}
add_custom_command(OUTPUT ${output_file}
COMMAND ${CMAKE_COMMAND}
-D "OV_COVERAGE_OUTPUT_FILE=${output_file}${out_suf}"
-D "OV_COVERAGE_INPUT_FILES=${output_base_file};${output_tests_file}"
-P "${OV_COVERAGE_SCRIPT_DIR}/coverage_merge.cmake"
COMMENT "Generate total coverage data ${OV_COVERAGE_INFO_FILE}"
-D "IE_COVERAGE_OUTPUT_FILE=${output_file}"
-D "IE_COVERAGE_INPUT_FILES=${output_base_file};${output_tests_file}"
-P "${IE_COVERAGE_SCRIPT_DIR}/coverage_merge.cmake"
COMMENT "Generate total coverage data ${IE_COVERAGE_INFO_FILE}"
DEPENDS ${output_base_file} ${output_tests_file}
VERBATIM)
if (OV_COVERAGE_EXCLUDE_PATTERNS)
set(commands lcov --quiet)
foreach(pattern IN LISTS OV_COVERAGE_EXCLUDE_PATTERNS)
list(APPEND commands --remove ${output_file}${out_suf} ${pattern})
endforeach()
list(APPEND commands --output-file ${output_file})
add_custom_command(OUTPUT ${output_file}
COMMAND ${commands}
COMMENT "Exclude patterns from report ${OV_COVERAGE_OUTPUT}"
DEPENDS ${output_file}${out_suf}
VERBATIM)
endif()
add_custom_target(ov_coverage_${OV_COVERAGE_INFO_FILE}_info
add_custom_target(ie_coverage_${IE_COVERAGE_INFO_FILE}_info
DEPENDS ${output_file})
set_target_properties(ov_coverage_${OV_COVERAGE_INFO_FILE}_info
set_target_properties(ie_coverage_${IE_COVERAGE_INFO_FILE}_info
PROPERTIES FOLDER coverage)
endfunction()
#
# ov_coverage_extract(INPUT <info_file> OUTPUT <output_file> PATTERNS <patterns ...>)
# ie_coverage_extract(INPUT <info_file> OUTPUT <output_file> PATTERNS <patterns ...>)
#
function(ov_coverage_extract)
cmake_parse_arguments(OV_COVERAGE "" "INPUT;OUTPUT" "PATTERNS" ${ARGN})
function(ie_coverage_extract)
cmake_parse_arguments(IE_COVERAGE "" "INPUT;OUTPUT" "PATTERNS" ${ARGN})
set(input_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INPUT}.info")
set(output_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_OUTPUT}.info")
set(input_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INPUT}.info")
set(output_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_OUTPUT}.info")
set(commands lcov --quiet)
foreach(pattern IN LISTS OV_COVERAGE_PATTERNS)
foreach(pattern IN LISTS IE_COVERAGE_PATTERNS)
list(APPEND commands --extract ${input_file} ${pattern})
endforeach()
list(APPEND commands --output-file ${output_file})
add_custom_command(OUTPUT ${output_file}
COMMAND ${commands}
COMMENT "Generate coverage data ${OV_COVERAGE_OUTPUT}"
COMMENT "Generate coverage data ${IE_COVERAGE_OUTPUT}"
DEPENDS ${input_file}
VERBATIM)
add_custom_target(ov_coverage_${OV_COVERAGE_OUTPUT}_info
add_custom_target(ie_coverage_${IE_COVERAGE_OUTPUT}_info
DEPENDS ${output_file})
set_target_properties(ov_coverage_${OV_COVERAGE_OUTPUT}_info
set_target_properties(ie_coverage_${IE_COVERAGE_OUTPUT}_info
PROPERTIES FOLDER coverage)
add_dependencies(ov_coverage_${OV_COVERAGE_OUTPUT}_info ov_coverage_${OV_COVERAGE_INPUT}_info)
endfunction()
#
# ov_coverage_genhtml(INFO_FILE <info_file> PREFIX <prefix>)
#
function(ov_coverage_genhtml)
cmake_parse_arguments(OV_COVERAGE "" "INFO_FILE;PREFIX" "" ${ARGN})
set(input_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INFO_FILE}.info")
set(output_directory "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INFO_FILE}")
add_custom_command(OUTPUT "${output_directory}/index.html"
COMMAND [ -s ${input_file} ] && genhtml ${input_file} --title "${OV_COVERAGE_INFO_FILE}" --legend
--no-branch-coverage --demangle-cpp
--output-directory "${output_directory}"
--num-spaces 4 --quiet
--prefix "${OV_COVERAGE_PREFIX}" || echo "Skip ${input_file}"
DEPENDS ${input_file}
COMMENT "Generate HTML report for ${OV_COVERAGE_INFO_FILE}"
VERBATIM)
add_custom_target(ov_coverage_${OV_COVERAGE_INFO_FILE}_genhtml
DEPENDS "${output_directory}/index.html")
set_target_properties(ov_coverage_${OV_COVERAGE_INFO_FILE}_genhtml
PROPERTIES FOLDER coverage)
add_dependencies(ov_coverage_${OV_COVERAGE_INFO_FILE}_genhtml ov_coverage_${OV_COVERAGE_INFO_FILE}_info)
add_dependencies(ov_coverage ov_coverage_${OV_COVERAGE_INFO_FILE}_genhtml)
add_dependencies(ie_coverage_${IE_COVERAGE_OUTPUT}_info ie_coverage_${IE_COVERAGE_INPUT}_info)
endfunction()
#
# ie_coverage_remove(INPUT <info_file> OUTPUT <output_file> PATTERNS <patterns ...>)
#
function(ie_coverage_remove)
cmake_parse_arguments(OV_COVERAGE "" "INPUT;OUTPUT" "PATTERNS" ${ARGN})
cmake_parse_arguments(IE_COVERAGE "" "INPUT;OUTPUT" "PATTERNS" ${ARGN})
set(input_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_INPUT}.info")
set(output_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_OUTPUT}.info")
set(input_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INPUT}.info")
set(output_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_OUTPUT}.info")
set(commands lcov --quiet)
foreach(pattern IN LISTS OV_COVERAGE_PATTERNS)
foreach(pattern IN LISTS IE_COVERAGE_PATTERNS)
list(APPEND commands --remove ${input_file} ${pattern})
endforeach()
list(APPEND commands --output-file ${output_file})
add_custom_command(OUTPUT ${output_file}
COMMAND ${commands}
COMMENT "Generate coverage data ${OV_COVERAGE_OUTPUT}"
COMMENT "Generate coverage data ${IE_COVERAGE_OUTPUT}"
DEPENDS ${input_file}
VERBATIM)
add_custom_target(ov_coverage_${OV_COVERAGE_OUTPUT}_info
add_custom_target(ie_coverage_${IE_COVERAGE_OUTPUT}_info
DEPENDS ${output_file})
set_target_properties(ov_coverage_${OV_COVERAGE_OUTPUT}_info
set_target_properties(ie_coverage_${IE_COVERAGE_OUTPUT}_info
PROPERTIES FOLDER coverage)
add_dependencies(ov_coverage_${OV_COVERAGE_OUTPUT}_info ov_coverage_${OV_COVERAGE_INPUT}_info)
add_dependencies(ie_coverage_${IE_COVERAGE_OUTPUT}_info ie_coverage_${IE_COVERAGE_INPUT}_info)
endfunction()
#
# ie_coverage_merge(OUTPUT <output file> INPUTS <input files ...>)
#
function(ie_coverage_merge)
cmake_parse_arguments(OV_COVERAGE "" "OUTPUT" "INPUTS" ${ARGN})
cmake_parse_arguments(IE_COVERAGE "" "OUTPUT" "INPUTS" ${ARGN})
set(output_file "${OV_COVERAGE_REPORTS}/${OV_COVERAGE_OUTPUT}.info")
foreach(input_info_file IN LISTS OV_COVERAGE_INPUTS)
set(input_file ${OV_COVERAGE_REPORTS}/${input_info_file}.info)
list(APPEND dependencies ov_coverage_${input_info_file}_info)
set(output_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_OUTPUT}.info")
foreach(input_info_file IN LISTS IE_COVERAGE_INPUTS)
set(input_file ${IE_COVERAGE_REPORTS}/${input_info_file}.info)
list(APPEND dependencies ie_coverage_${input_info_file}_info)
list(APPEND input_files ${input_file})
endforeach()
add_custom_command(OUTPUT ${output_file}
COMMAND ${CMAKE_COMMAND}
-D "OV_COVERAGE_OUTPUT_FILE=${output_file}"
-D "OV_COVERAGE_INPUT_FILES=${input_files}"
-P "${OV_COVERAGE_SCRIPT_DIR}/coverage_merge.cmake"
COMMENT "Generate coverage data ${OV_COVERAGE_OUTPUT}"
-D "IE_COVERAGE_OUTPUT_FILE=${output_file}"
-D "IE_COVERAGE_INPUT_FILES=${input_files}"
-P "${IE_COVERAGE_SCRIPT_DIR}/coverage_merge.cmake"
COMMENT "Generate coverage data ${IE_COVERAGE_OUTPUT}"
DEPENDS ${input_files}
VERBATIM)
add_custom_target(ov_coverage_${OV_COVERAGE_OUTPUT}_info
add_custom_target(ie_coverage_${IE_COVERAGE_OUTPUT}_info
DEPENDS ${output_file})
set_target_properties(ov_coverage_${OV_COVERAGE_OUTPUT}_info
set_target_properties(ie_coverage_${IE_COVERAGE_OUTPUT}_info
PROPERTIES FOLDER coverage)
add_dependencies(ov_coverage_${OV_COVERAGE_OUTPUT}_info ${dependencies})
add_dependencies(ie_coverage_${IE_COVERAGE_OUTPUT}_info ${dependencies})
endfunction()
if(NOT TARGET ie_coverage)
add_custom_target(ie_coverage)
set_target_properties(ie_coverage PROPERTIES FOLDER coverage)
add_dependencies(ie_coverage ov_coverage)
endif()
#
# ie_coverage_genhtml(INFO_FILE <info_file> PREFIX <prefix>)
#
function(ie_coverage_genhtml)
cmake_parse_arguments(IE_COVERAGE "" "INFO_FILE;PREFIX" "" ${ARGN})
set(input_file "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INFO_FILE}.info")
set(output_directory "${IE_COVERAGE_REPORTS}/${IE_COVERAGE_INFO_FILE}")
add_custom_command(OUTPUT "${output_directory}/index.html"
COMMAND genhtml ${input_file} --title "${IE_COVERAGE_INFO_FILE}" --legend
--no-branch-coverage --demangle-cpp
--output-directory "${output_directory}"
--num-spaces 4 --quiet
--prefix "${IE_COVERAGE_PREFIX}"
DEPENDS ${input_file}
COMMENT "Generate HTML report for ${IE_COVERAGE_INFO_FILE}"
VERBATIM)
add_custom_target(ie_coverage_${IE_COVERAGE_INFO_FILE}_genhtml
DEPENDS "${output_directory}/index.html")
set_target_properties(ie_coverage_${IE_COVERAGE_INFO_FILE}_genhtml
PROPERTIES FOLDER coverage)
add_dependencies(ie_coverage_${IE_COVERAGE_INFO_FILE}_genhtml ie_coverage_${IE_COVERAGE_INFO_FILE}_info)
add_dependencies(ie_coverage ie_coverage_${IE_COVERAGE_INFO_FILE}_genhtml)
endfunction()

View File

@@ -1,22 +1,22 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(NOT DEFINED OV_COVERAGE_REPORTS)
message(FATAL_ERROR "OV_COVERAGE_REPORTS variable is not defined")
if(NOT DEFINED IE_COVERAGE_REPORTS)
message(FATAL_ERROR "IE_COVERAGE_REPORTS variable is not defined")
return()
endif()
file(REMOVE_RECURSE "${OV_COVERAGE_REPORTS}")
file(REMOVE_RECURSE "${IE_COVERAGE_REPORTS}")
if(NOT DEFINED OV_COVERAGE_DIRECTORY)
message(FATAL_ERROR "OV_COVERAGE_DIRECTORY variable is not defined")
if(NOT DEFINED IE_COVERAGE_DIRECTORY)
message(FATAL_ERROR "IE_COVERAGE_DIRECTORY variable is not defined")
return()
endif()
# remove .gcno files which are kept from the previous build
file(GLOB_RECURSE gcno_files "${OV_COVERAGE_DIRECTORY}/*.gcno")
file(GLOB_RECURSE gcno_files "${IE_COVERAGE_DIRECTORY}/*.gcno")
foreach(file IN LISTS gcno_files)
string(REPLACE ".gcno" "" temp_file "${file}")
string(REGEX REPLACE "CMakeFiles/.+dir/" "" temp_file "${temp_file}")

View File

@@ -1,22 +1,22 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(NOT DEFINED OV_COVERAGE_OUTPUT_FILE)
message(FATAL_ERROR "OV_COVERAGE_OUTPUT_FILE is not defined")
if(NOT DEFINED IE_COVERAGE_OUTPUT_FILE)
message(FATAL_ERROR "IE_COVERAGE_OUTPUT_FILE is not defined")
endif()
if(NOT DEFINED OV_COVERAGE_INPUT_FILES)
message(FATAL_ERROR "OV_COVERAGE_INPUT_FILES is not defined")
if(NOT DEFINED IE_COVERAGE_INPUT_FILES)
message(FATAL_ERROR "IE_COVERAGE_INPUT_FILES is not defined")
endif()
set(command lcov --quiet)
foreach(input_info_file IN LISTS OV_COVERAGE_INPUT_FILES)
foreach(input_info_file IN LISTS IE_COVERAGE_INPUT_FILES)
file(SIZE ${input_info_file} size)
if(NOT size EQUAL 0)
list(APPEND command --add-tracefile "${input_info_file}")
endif()
endforeach()
list(APPEND command --output-file ${OV_COVERAGE_OUTPUT_FILE})
list(APPEND command --output-file ${IE_COVERAGE_OUTPUT_FILE})
execute_process(COMMAND ${command})

View File

@@ -1,9 +1,9 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_CPPLINT)
find_host_package(PythonInterp 3 QUIET)
find_package(PythonInterp 3 QUIET)
if(NOT PYTHONINTERP_FOUND)
message(WARNING "Python3 interpreter was not found (required for cpplint check)")

View File

@@ -3592,7 +3592,7 @@ def CheckOperatorSpacing(filename, clean_lines, linenum, error):
elif not Match(r'#.*include', line):
# Look for < that is not surrounded by spaces. This is only
# triggered if both sides are missing spaces, even though
# technically should flag if at least one side is missing a
# technically should should flag if at least one side is missing a
# space. This is done to avoid some false positives with shifts.
match = Match(r'^(.*[^\s<])<[^\s=<,]', line)
if match:

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2020 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -120,8 +120,8 @@ function(_clone_source_to_target TARGET SOURCE ARCH_SET)
VERBATIM
)
set_property(SOURCE ${ARCH_SOURCE} APPEND_STRING PROPERTY COMPILE_OPTIONS
"${_FLAGS_${_arch}}")
set_property(SOURCE ${ARCH_SOURCE} APPEND_STRING PROPERTY COMPILE_FLAGS
" ${_FLAGS_${_arch}}")
set_property(SOURCE ${ARCH_SOURCE} APPEND PROPERTY COMPILE_DEFINITIONS
${_DEFINE_${_arch}}

View File

@@ -1,17 +1,73 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
function(debug_message)
if(CMAKE_VERBOSE_MAKEFILE)
function (debug_message)
if (VERBOSE_BUILD)
message(${ARGV})
endif()
endfunction()
function(clean_message type)
string(REPLACE ";" "" output_string "${ARGN}")
execute_process(COMMAND ${CMAKE_COMMAND} -E echo "${output_string}")
if(${ARGV0} STREQUAL "FATAL_ERROR")
message (FATAL_ERROR)
endif()
string (REPLACE ";" "" output_string "${ARGN}")
execute_process(COMMAND ${CMAKE_COMMAND} -E echo "${output_string}")
if(${ARGV0} STREQUAL "FATAL_ERROR")
message (FATAL_ERROR)
endif()
endfunction()
file(REMOVE ${CMAKE_BINARY_DIR}/ld_library_rpath_64.txt)
# log relative path to shared library that has to be used in LD_LIBRARY_PATH
function (log_rpath_remove_top component component_remove_top lib lib_remove_top)
set(top_lib_dir ${${component}})
set(lib_dir ${lib})
# debug_message(STATUS "LIB-IN=${lib} ")
# debug_message(STATUS "TOPLIB-IN=${top_lib_dir} ")
get_filename_component(top_lib_dir "${${component}}" DIRECTORY)
if (${component_remove_top} AND ${component})
else()
get_filename_component(add_name "${${component}}" NAME)
set(top_lib_dir "${top_lib_dir}/${add_name}")
endif()
if (${lib_remove_top} AND lib)
get_filename_component(lib_dir ${lib} DIRECTORY)
endif()
string (REPLACE "//" "/" top_lib_dir "${top_lib_dir}")
string (REPLACE "//" "/" lib_dir "${lib_dir}")
string (REPLACE "\\\\" "/" top_lib_dir "${top_lib_dir}")
string (REPLACE "\\\\" "/" lib_dir "${lib_dir}")
# debug_message(STATUS "LIB-OUT=${lib_dir}")
# debug_message(STATUS "TOPLIB-OUT=${top_lib_dir}")
if (WIN32)
string (TOLOWER "${top_lib_dir}" top_lib_dir)
string (TOLOWER "${lib_dir}" lib_dir)
endif()
string (REPLACE "${top_lib_dir}" "" component_dir "${lib_dir}")
set(RPATH_INFO "${component}=${component_dir}")
debug_message(STATUS "LD_LIBRARY_RPATH: ${RPATH_INFO}")
file(APPEND ${CMAKE_BINARY_DIR}/ld_library_rpath_64.txt "${RPATH_INFO}\n")
endfunction()
function (log_rpath_from_dir component lib_dir)
log_rpath_remove_top("${component}" TRUE "${lib_dir}" FALSE)
endfunction()
function (log_rpath component lib_path)
log_rpath_remove_top(${component} TRUE ${lib_path} TRUE)
endfunction()
# Just wrapping of the original message() function to make this macro known during IE build.
# This macro is redefined (with additional checks) within the InferenceEngineConfig.cmake file.
macro(ext_message TRACE_LEVEL)
message(${TRACE_LEVEL} "${ARGN}")
endmacro()

View File

@@ -1,10 +1,10 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include (download/download)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256 FILES_TO_EXTRACT USE_NEW_LOCATION)
function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH FOLDER ENVIRONMENT SHA256 FILES_TO_EXTRACT)
if (ENVIRONMENT AND (DEFINED ${ENVIRONMENT} OR DEFINED ENV{${ENVIRONMENT}}))
set(HAS_ENV "TRUE")
endif()
@@ -12,9 +12,9 @@ function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHI
if (NOT DEFINED HAS_ENV)
if (ARCHIVE)
#TODO: check whether this is platform specific binary with same name per or it is in common folder
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT} ${USE_NEW_LOCATION})
DownloadAndExtract(${COMPONENT} ${ARCHIVE} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
else()
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT} ${USE_NEW_LOCATION})
DownloadAndExtractPlatformSpecific(${COMPONENT} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} result_path ${FOLDER} ${SHA256} ${FILES_TO_EXTRACT})
endif()
set (${VAR} ${result_path} PARENT_SCOPE)
@@ -25,8 +25,15 @@ function (resolve_archive_dependency VAR COMPONENT ARCHIVE ARCHIVE_UNIFIED ARCHI
set (${VAR} $ENV{${ENVIRONMENT}} PARENT_SCOPE)
endif ()
endif()
endfunction(resolve_archive_dependency)
function(resolve_pull_request GITHUB_PULL_REQUEST TARGET_PATH)
get_filename_component(FILE_NAME ${GITHUB_PULL_REQUEST} NAME)
set (PATCH_URL "")
DownloadAndApply("${PATCH_URL}/${GITHUB_PULL_REQUEST}" "${IE_MAIN_SOURCE_DIR}/${TARGET_PATH}/${FILE_NAME}")
endfunction(resolve_pull_request)
function(extract_version_from_filename filename regex version)
string(REGEX MATCH ${regex} match ${filename})
@@ -44,8 +51,10 @@ function(read_version archive regex version_var)
endfunction(read_version)
function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
list(REMOVE_AT ARGV 0)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT VERSION_REGEX SHA256 FILES_TO_EXTRACT USE_NEW_LOCATION)
set(SUPPORTED_ARGS FOLDER ARCHIVE ARCHIVE_UNIFIED ARCHIVE_WIN ARCHIVE_LIN ARCHIVE_MAC ARCHIVE_ANDROID TARGET_PATH ENVIRONMENT GITHUB_PULL_REQUEST VERSION_REGEX SHA256 FILES_TO_EXTRACT)
#unnecessary vars
foreach(arg ${ARGV})
@@ -111,26 +120,25 @@ function (RESOLVE_DEPENDENCY NAME_OF_CMAKE_VAR)
set (FILES_TO_EXTRACT FALSE)
endif()
if (NOT DEFINED USE_NEW_LOCATION)
set (USE_NEW_LOCATION FALSE)
endif()
#for each dependency type have to do separate things
if (ARCHIVE_WIN OR ARCHIVE_LIN OR ARCHIVE_MAC OR ARCHIVE_ANDROID OR ARCHIVE OR ARCHIVE_UNIFIED)
if (NOT DEFINED TARGET_PATH)
message(FATAL_ERROR "TARGET_PATH should be defined for every dependency")
endif()
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256} ${FILES_TO_EXTRACT} ${USE_NEW_LOCATION})
resolve_archive_dependency(RESULT ${NAME_OF_CMAKE_VAR} ${ARCHIVE} ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${TARGET_PATH} ${FOLDER} ${ENVIRONMENT} ${SHA256} ${FILES_TO_EXTRACT})
set(${NAME_OF_CMAKE_VAR} ${RESULT} PARENT_SCOPE)
if (VERSION_REGEX)
GetNameAndUrlToDownload(archive RELATIVE_URL ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID} ${USE_NEW_LOCATION})
GetNameAndUrlToDownload(archive RELATIVE_URL ${ARCHIVE_UNIFIED} ${ARCHIVE_WIN} ${ARCHIVE_LIN} ${ARCHIVE_MAC} ${ARCHIVE_ANDROID})
if (archive)
read_version(${archive} ${VERSION_REGEX} "${NAME_OF_CMAKE_VAR}_VERSION")
endif()
endif()
elseif (DEFINED GITHUB_PULL_REQUEST)
resolve_pull_request(${GITHUB_PULL_REQUEST} ${TARGET_PATH})
else()
message(FATAL_ERROR "Dependency of unknowntype, SHOULD set one of ARCHIVE_WIN, ARCHIVE, ARCHIVE_LIN, ARCHIVE_MAC, ARCHIVE_ANDROID")
message(FATAL_ERROR "Dependency of unknowntype, SHOULD set one of ARCHIVE_WIN, ARCHIVE, ARCHIVE_LIN, ARCHIVE_MAC, ARCHIVE_ANDROID, GITHUB_PULL_REQUEST")
endif()
endfunction(RESOLVE_DEPENDENCY)

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,17 +1,13 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(download/extract)
include(download/download_and_check)
function (GetNameAndUrlToDownload name url archive_name_unified archive_name_win archive_name_lin archive_name_mac archive_name_android USE_NEW_LOCATION)
function (GetNameAndUrlToDownload name url archive_name_unified archive_name_win archive_name_lin archive_name_mac archive_name_android)
if (archive_name_unified)
if (USE_NEW_LOCATION)
set (${url} "${archive_name_unified}" PARENT_SCOPE)
else()
set (${url} "thirdparty/unified/${archive_name_unified}" PARENT_SCOPE)
endif()
set (${url} "thirdparty/unified/${archive_name_unified}" PARENT_SCOPE)
set (${name} ${archive_name_unified} PARENT_SCOPE)
else()
if(archive_name_lin)
@@ -47,26 +43,25 @@ function (DownloadAndExtractPlatformSpecific
result_path
folder
sha256
files_to_extract
USE_NEW_LOCATION)
files_to_extract)
GetNameAndUrlToDownload(archive_name RELATIVE_URL ${archive_name_unified} ${archive_name_win} ${archive_name_lin} ${archive_name_mac} ${archive_name_android} ${USE_NEW_LOCATION})
GetNameAndUrlToDownload(archive_name RELATIVE_URL ${archive_name_unified} ${archive_name_win} ${archive_name_lin} ${archive_name_mac} ${archive_name_android} )
if (NOT archive_name OR NOT RELATIVE_URL)
return()
endif()
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256} ${files_to_extract} ${USE_NEW_LOCATION})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} TRUE FALSE TRUE ${sha256} ${files_to_extract})
set (${result_path} ${result_path2} PARENT_SCOPE)
endfunction(DownloadAndExtractPlatformSpecific)
#download from common folder
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256 files_to_extract USE_NEW_LOCATION)
function (DownloadAndExtract component archive_name unpacked_path result_path folder sha256 files_to_extract)
set (RELATIVE_URL "${archive_name}")
set(fattal TRUE)
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256} ${files_to_extract} ${USE_NEW_LOCATION})
CheckOrDownloadAndExtract(${component} ${RELATIVE_URL} ${archive_name} ${unpacked_path} result_path2 ${folder} ${fattal} result TRUE ${sha256} ${files_to_extract})
if (NOT ${result})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder} ${sha256} ${files_to_extract} ${USE_NEW_LOCATION})
DownloadAndExtractPlatformSpecific(${component} ${archive_name} ${archive_name} ${archive_name} ${unpacked_path} ${result_path2} ${folder} ${sha256} ${files_to_extract})
endif()
set (${result_path} ${result_path2} PARENT_SCOPE)
@@ -129,8 +124,6 @@ function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal
if (${result})
set (downloadStatus "OK")
endif()
else()
set (downloadStatus "OK")
endif()
else()
debug_message("archive found on FS : ${archive_path}, however we cannot check it's checksum and think that it is invalid")
@@ -153,7 +146,9 @@ function (DownloadOrExtractInternal URL archive_path unpacked_path folder fattal
endfunction(DownloadOrExtractInternal)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256 files_to_extract USE_NEW_LOCATION)
file(REMOVE ${CMAKE_BINARY_DIR}/dependencies_64.txt)
function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked_path result_path folder fattal resultExt use_alternatives sha256 files_to_extract)
set (archive_path ${TEMP}/download/${archive_name})
set (status "ON")
@@ -161,8 +156,6 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
set(URL "${IE_PATH_TO_DEPS}/${RELATIVE_URL}")
elseif(DEFINED ENV{IE_PATH_TO_DEPS})
set(URL "$ENV{IE_PATH_TO_DEPS}/${RELATIVE_URL}")
elseif(USE_NEW_LOCATION)
set(URL "https://storage.openvinotoolkit.org/dependencies/${RELATIVE_URL}")
else()
set(URL "https://download.01.org/opencv/master/openvinotoolkit/${RELATIVE_URL}")
endif()
@@ -171,6 +164,7 @@ function (CheckOrDownloadAndExtract component RELATIVE_URL archive_name unpacked
if (${use_alternatives})
set(DEP_INFO "${component}=${URL}")
debug_message (STATUS "DEPENDENCY_URL: ${DEP_INFO}")
file(APPEND ${CMAKE_BINARY_DIR}/dependencies_64.txt "${DEP_INFO}\n")
endif()
debug_message ("checking that unpacked directory exist: ${unpacked_path}")

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
@@ -7,38 +7,26 @@ include(target_flags)
# FIXME: there are compiler failures with LTO and Cross-Compile toolchains. Disabling for now, but
# this must be addressed in a proper way
ie_dependent_option (ENABLE_LTO "Enable Link Time Optimization" OFF
"LINUX OR (APPLE AND AARCH64);NOT CMAKE_CROSSCOMPILING;CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 4.9" OFF)
ie_dependent_option (ENABLE_LTO "Enable Link Time Optimization" OFF "LINUX;NOT CMAKE_CROSSCOMPILING; CMAKE_CXX_COMPILER_VERSION VERSION_GREATER 4.9" OFF)
ie_option (OS_FOLDER "create OS dedicated folder in output" OFF)
if(OV_GENERATOR_MULTI_CONFIG)
ie_option(USE_BUILD_TYPE_SUBFOLDER "Create dedicated sub-folder per build type for output binaries" OFF)
else()
if(UNIX)
ie_option(USE_BUILD_TYPE_SUBFOLDER "Create dedicated sub-folder per build type for output binaries" ON)
endif()
if(CI_BUILD_NUMBER)
set(TREAT_WARNING_AS_ERROR_DEFAULT ON)
else()
set(TREAT_WARNING_AS_ERROR_DEFAULT OFF)
ie_option(USE_BUILD_TYPE_SUBFOLDER "Create dedicated sub-folder per build type for output binaries" OFF)
endif()
ie_dependent_option (TREAT_WARNING_AS_ERROR "WILL BE REMOVED SOON, NEED TO FIX PRIVATE COMPONENTS" ON "X86_64 OR X86" OFF)
# FIXME: ARM cross-compiler generates several "false positive" warnings regarding __builtin_memcpy buffer overflow
ie_dependent_option (TREAT_WARNING_AS_ERROR "Treat build warnings as errors" ON "X86 OR X86_64" OFF)
if(NOT DEFINED CMAKE_COMPILE_WARNING_AS_ERROR)
set(CMAKE_COMPILE_WARNING_AS_ERROR ${TREAT_WARNING_AS_ERROR_DEFAULT})
endif()
ie_dependent_option (ENABLE_INTEGRITYCHECK "build DLLs with /INTEGRITYCHECK flag" OFF "CMAKE_CXX_COMPILER_ID STREQUAL MSVC" OFF)
ie_option (ENABLE_INTEGRITYCHECK "build DLLs with /INTEGRITYCHECK flag" OFF)
ie_option (ENABLE_SANITIZER "enable checking memory errors via AddressSanitizer" OFF)
ie_option (ENABLE_UB_SANITIZER "enable UndefinedBahavior sanitizer" OFF)
ie_option (ENABLE_THREAD_SANITIZER "enable checking data races via ThreadSanitizer" OFF)
ie_dependent_option (ENABLE_COVERAGE "enable code coverage" OFF "CMAKE_CXX_COMPILER_ID STREQUAL GNU OR OV_COMPILER_IS_CLANG" OFF)
ie_dependent_option (ENABLE_COVERAGE "enable code coverage" OFF "CMAKE_CXX_COMPILER_ID STREQUAL GNU" OFF)
# Defines CPU capabilities
@@ -48,33 +36,27 @@ ie_dependent_option (ENABLE_AVX2 "Enable AVX2 optimizations" ON "X86_64 OR X86"
ie_dependent_option (ENABLE_AVX512F "Enable AVX512 optimizations" ON "X86_64 OR X86" OFF)
# Type of build, we add this as an explicit option to default it to ON
# FIXME: Ah this moment setting this to OFF will only build ngraph a static library
ie_option (BUILD_SHARED_LIBS "Build as a shared library" ON)
# Android does not support SOVERSION
# see https://www.opengis.ch/2011/11/23/creating-non-versioned-shared-libraries-for-android/
ie_dependent_option (ENABLE_LIBRARY_VERSIONING "Enable libraries versioning" ON "NOT WIN32;NOT ANDROID" OFF)
ie_dependent_option (ENABLE_FASTER_BUILD "Enable build features (PCH, UNITY) to speed up build time" OFF "CMAKE_VERSION VERSION_GREATER_EQUAL 3.16" OFF)
if(UNIX AND NOT ANDROID)
set(STYLE_CHECKS_DEFAULT ON)
else()
set(STYLE_CHECKS_DEFAULT OFF)
if(NOT DEFINED ENABLE_CPPLINT)
ie_dependent_option (ENABLE_CPPLINT "Enable cpplint checks during the build" ON "UNIX;NOT ANDROID" OFF)
endif()
ie_option (ENABLE_CPPLINT "Enable cpplint checks during the build" ${STYLE_CHECKS_DEFAULT})
if(NOT DEFINED ENABLE_CPPLINT_REPORT)
ie_dependent_option (ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF "ENABLE_CPPLINT" OFF)
endif()
ie_dependent_option (ENABLE_CPPLINT_REPORT "Build cpplint report instead of failing the build" OFF "ENABLE_CPPLINT" OFF)
ie_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ${STYLE_CHECKS_DEFAULT})
ie_option (ENABLE_NCC_STYLE "Enable ncc style check" ${STYLE_CHECKS_DEFAULT})
ie_dependent_option (ENABLE_CLANG_FORMAT "Enable clang-format checks during the build" ON "UNIX;NOT ANDROID" OFF)
ie_option (VERBOSE_BUILD "shows extra information about build" OFF)
ie_option (ENABLE_UNSAFE_LOCATIONS "skip check for MD5 for dependency" OFF)
ie_dependent_option (ENABLE_FUZZING "instrument build for fuzzing" OFF "OV_COMPILER_IS_CLANG;NOT WIN32" OFF)
ie_dependent_option (ENABLE_FUZZING "instrument build for fuzzing" OFF "OV_COMPILER_IS_CLANG; NOT WIN32" OFF)
#
# Check features
@@ -88,8 +70,8 @@ if(ENABLE_AVX512F)
if ((CMAKE_CXX_COMPILER_ID STREQUAL "Clang") AND (CMAKE_CXX_COMPILER_VERSION VERSION_LESS 6))
set(ENABLE_AVX512F OFF CACHE BOOL "" FORCE)
endif()
if (OV_COMPILER_IS_APPLECLANG AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 10)
# TODO: clarify which AppleClang version supports avx512
if ((CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang") AND (CMAKE_CXX_COMPILER_VERSION VERSION_LESS 10))
# TBD: clarify which AppleClang version supports avx512
set(ENABLE_AVX512F OFF CACHE BOOL "" FORCE)
endif()
if ((CMAKE_CXX_COMPILER_ID STREQUAL "GNU") AND (CMAKE_CXX_COMPILER_VERSION VERSION_LESS 4.9))
@@ -97,4 +79,6 @@ if(ENABLE_AVX512F)
endif()
endif()
set(CMAKE_VERBOSE_MAKEFILE ${VERBOSE_BUILD} CACHE BOOL "" FORCE)
if (VERBOSE_BUILD)
set(CMAKE_VERBOSE_MAKEFILE ON CACHE BOOL "" FORCE)
endif()

View File

@@ -1,34 +0,0 @@
# Copyright (C) 2018-2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
foreach(var OV_FRONTENDS_HPP_HEADER_IN OV_FRONTENDS_HPP_HEADER FRONTEND_NAMES)
if(NOT DEFINED ${var})
message(FATAL_ERROR "${var} is required, but not defined")
endif()
endforeach()
# configure variables
set(OV_FRONTEND_DECLARATIONS "")
set(OV_FRONTEND_MAP_DEFINITION " FrontendsStaticRegistry registry = {")
foreach(frontend IN LISTS FRONTEND_NAMES)
# common
set(_OV_FRONTEND_DATA_FUNC "GetFrontEndData${frontend}")
set(_OV_VERSION_FUNC "GetAPIVersion${frontend}")
# declarations
set(OV_FRONTEND_DECLARATIONS "${OV_FRONTEND_DECLARATIONS}
ov::frontend::FrontEndVersion ${_OV_VERSION_FUNC}();
void* ${_OV_FRONTEND_DATA_FUNC}();")
set(OV_FRONTEND_MAP_DEFINITION "${OV_FRONTEND_MAP_DEFINITION}
{ Value { ${_OV_FRONTEND_DATA_FUNC}, ${_OV_VERSION_FUNC} } },")
endforeach()
set(OV_FRONTEND_MAP_DEFINITION "${OV_FRONTEND_MAP_DEFINITION}
};
return registry;")
configure_file("${OV_FRONTENDS_HPP_HEADER_IN}" "${OV_FRONTENDS_HPP_HEADER}" @ONLY)

View File

@@ -1,285 +0,0 @@
# Copyright (C) 2018-2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
set(FRONTEND_INSTALL_INCLUDE "${OV_CPACK_INCLUDEDIR}")
set(FRONTEND_NAME_PREFIX "openvino_")
set(FRONTEND_NAME_SUFFIX "_frontend")
set(FRONTEND_NAMES "" CACHE INTERNAL "")
if(NOT TARGET ov_frontends)
add_custom_target(ov_frontends)
endif()
#
# ov_target_link_frontends(<TARGET_NAME>)
#
function(ov_target_link_frontends TARGET_NAME)
if(BUILD_SHARED_LIBS)
return()
endif()
foreach(name IN LISTS FRONTEND_NAMES)
set(frontend_target_name "${FRONTEND_NAME_PREFIX}${name}${FRONTEND_NAME_SUFFIX}")
target_link_libraries(${TARGET_NAME} PRIVATE ${frontend_target_name})
endforeach()
endfunction()
#
# ov_generate_frontends_hpp()
#
function(ov_generate_frontends_hpp)
if(BUILD_SHARED_LIBS)
return()
endif()
# add frontends to libraries including ov_frontends.hpp
ov_target_link_frontends(openvino)
set(ov_frontends_hpp "${CMAKE_BINARY_DIR}/src/frontends/common/src/ov_frontends.hpp")
set(frontends_hpp_in "${IEDevScripts_DIR}/frontends/ov_frontends.hpp.in")
add_custom_command(OUTPUT "${ov_frontends_hpp}"
COMMAND
"${CMAKE_COMMAND}"
-D "OV_FRONTENDS_HPP_HEADER_IN=${frontends_hpp_in}"
-D "OV_FRONTENDS_HPP_HEADER=${ov_frontends_hpp}"
-D "FRONTEND_NAMES=${FRONTEND_NAMES}"
-P "${IEDevScripts_DIR}/frontends/create_frontends_hpp.cmake"
DEPENDS
"${frontends_hpp_in}"
"${IEDevScripts_DIR}/frontends/create_frontends_hpp.cmake"
COMMENT
"Generate ov_frontends.hpp for static build"
VERBATIM)
# for some reason dependency on source files does not work
# so, we have to use explicit target and make it dependency for frontend_common
add_custom_target(_ov_frontends_hpp DEPENDS ${ov_frontends_hpp})
add_dependencies(frontend_common_obj _ov_frontends_hpp)
# add dependency for object files
get_target_property(sources frontend_common_obj SOURCES)
foreach(source IN LISTS sources)
if("${source}" MATCHES "\\$\\<TARGET_OBJECTS\\:([A-Za-z0-9_]*)\\>")
# object library
set(obj_library ${CMAKE_MATCH_1})
get_target_property(obj_sources ${obj_library} SOURCES)
list(APPEND all_sources ${obj_sources})
else()
# usual source
list(APPEND all_sources ${source})
endif()
endforeach()
# add dependency on header file generation for all inference_engine source files
set_source_files_properties(${all_sources} PROPERTIES OBJECT_DEPENDS ${ov_frontends_hpp})
endfunction()
unset(protobuf_lite_installed CACHE)
unset(protobuf_installed CACHE)
#
# ov_add_frontend(NAME <IR|ONNX|...>
# FILEDESCRIPTION <description> # used on Windows to describe DLL file
# [LINKABLE_FRONTEND] # whether we can use FE API directly or via FEM only
# [SKIP_INSTALL] # private frontend, not for end users
# [PROTOBUF_LITE] # requires only libprotobuf-lite
# [SKIP_NCC_STYLE] # use custom NCC rules
# [LINK_LIBRARIES <lib1 lib2 ...>])
#
macro(ov_add_frontend)
set(options LINKABLE_FRONTEND PROTOBUF_LITE SKIP_NCC_STYLE SKIP_INSTALL)
set(oneValueArgs NAME FILEDESCRIPTION)
set(multiValueArgs LINK_LIBRARIES PROTO_FILES)
cmake_parse_arguments(OV_FRONTEND "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN})
foreach(prop NAME FILEDESCRIPTION)
if(NOT DEFINED OV_FRONTEND_${prop})
message(FATAL_ERROR "Frontend ${prop} property is not defined")
endif()
endforeach()
set(TARGET_NAME "${FRONTEND_NAME_PREFIX}${OV_FRONTEND_NAME}${FRONTEND_NAME_SUFFIX}")
list(APPEND FRONTEND_NAMES ${OV_FRONTEND_NAME})
set(FRONTEND_NAMES "${FRONTEND_NAMES}" CACHE INTERNAL "" FORCE)
set(frontend_root_dir "${CMAKE_CURRENT_SOURCE_DIR}")
if(frontend_root_dir MATCHES ".*src$")
get_filename_component(frontend_root_dir "${frontend_root_dir}" DIRECTORY)
endif()
file(GLOB_RECURSE LIBRARY_SRC ${frontend_root_dir}/src/*.cpp)
if (WIN32)
# Remove linux specific files
file(GLOB_RECURSE LIN_FILES ${frontend_root_dir}/src/os/lin/*.cpp
${frontend_root_dir}/src/os/lin/*.hpp)
list(REMOVE_ITEM LIBRARY_SRC "${LIN_FILES}")
else()
# Remove windows specific files
file(GLOB_RECURSE WIN_FILES ${frontend_root_dir}/src/os/win/*.cpp
${frontend_root_dir}/src/os/win/*.hpp)
list(REMOVE_ITEM LIBRARY_SRC "${WIN_FILES}")
endif()
file(GLOB_RECURSE LIBRARY_HEADERS ${frontend_root_dir}/src/*.hpp)
file(GLOB_RECURSE LIBRARY_PUBLIC_HEADERS ${frontend_root_dir}/include/*.hpp)
set(${TARGET_NAME}_INCLUDE_DIR ${frontend_root_dir}/include)
# Create named folders for the sources within the .vcproj
# Empty name lists them directly under the .vcproj
source_group("src" FILES ${LIBRARY_SRC})
source_group("include" FILES ${LIBRARY_HEADERS})
source_group("public include" FILES ${LIBRARY_PUBLIC_HEADERS})
# Generate protobuf file on build time for each '.proto' file in src/proto
file(GLOB proto_files ${frontend_root_dir}/src/proto/*.proto)
foreach(INFILE IN LISTS proto_files)
get_filename_component(FILE_DIR ${INFILE} DIRECTORY)
get_filename_component(FILE_WE ${INFILE} NAME_WE)
set(OUTPUT_PB_SRC ${CMAKE_CURRENT_BINARY_DIR}/${FILE_WE}.pb.cc)
set(OUTPUT_PB_HEADER ${CMAKE_CURRENT_BINARY_DIR}/${FILE_WE}.pb.h)
set(GENERATED_PROTO ${INFILE})
add_custom_command(
OUTPUT "${OUTPUT_PB_SRC}" "${OUTPUT_PB_HEADER}"
COMMAND ${PROTOC_EXECUTABLE} ARGS --cpp_out ${CMAKE_CURRENT_BINARY_DIR} -I ${FILE_DIR} ${FILE_WE}.proto
DEPENDS ${PROTOC_DEPENDENCY} ${GENERATED_PROTO}
COMMENT "Running C++ protocol buffer compiler (${PROTOC_EXECUTABLE}) on ${GENERATED_PROTO}"
VERBATIM
COMMAND_EXPAND_LISTS)
list(APPEND PROTO_SRCS "${OUTPUT_PB_SRC}")
list(APPEND PROTO_HDRS "${OUTPUT_PB_HEADER}")
endforeach()
# Disable all warnings for generated code
set_source_files_properties(${PROTO_SRCS} ${PROTO_HDRS} PROPERTIES COMPILE_OPTIONS -w GENERATED TRUE)
# Create library
add_library(${TARGET_NAME} ${LIBRARY_SRC} ${LIBRARY_HEADERS} ${LIBRARY_PUBLIC_HEADERS} ${PROTO_SRCS} ${PROTO_HDRS})
if(OV_FRONTEND_LINKABLE_FRONTEND)
# create beautiful alias
add_library(openvino::frontend::${OV_FRONTEND_NAME} ALIAS ${TARGET_NAME})
endif()
# Shutdown protobuf when unloading the front dynamic library
if(proto_files AND BUILD_SHARED_LIBS)
target_link_libraries(${TARGET_NAME} PRIVATE ov_protobuf_shutdown)
endif()
if(NOT BUILD_SHARED_LIBS)
# override default function names
target_compile_definitions(${TARGET_NAME} PRIVATE
"-DGetFrontEndData=GetFrontEndData${OV_FRONTEND_NAME}"
"-DGetAPIVersion=GetAPIVersion${OV_FRONTEND_NAME}")
endif()
# enable LTO
set_target_properties(${TARGET_NAME} PROPERTIES
INTERPROCEDURAL_OPTIMIZATION_RELEASE ${ENABLE_LTO})
if(OV_FRONTEND_SKIP_NCC_STYLE)
# frontend's CMakeLists.txt must define its own custom 'ov_ncc_naming_style' step
else()
ov_ncc_naming_style(FOR_TARGET ${TARGET_NAME}
SOURCE_DIRECTORY "${frontend_root_dir}/include"
ADDITIONAL_INCLUDE_DIRECTORIES
$<TARGET_PROPERTY:frontend_common::static,INTERFACE_INCLUDE_DIRECTORIES>)
endif()
target_include_directories(${TARGET_NAME}
PUBLIC
$<BUILD_INTERFACE:${${TARGET_NAME}_INCLUDE_DIR}>
PRIVATE
${frontend_root_dir}/src
${CMAKE_CURRENT_BINARY_DIR})
ie_add_vs_version_file(NAME ${TARGET_NAME}
FILEDESCRIPTION ${OV_FRONTEND_FILEDESCRIPTION})
ie_add_api_validator_post_build_step(TARGET ${TARGET_NAME})
target_link_libraries(${TARGET_NAME} PUBLIC openvino::runtime)
target_link_libraries(${TARGET_NAME} PRIVATE ${OV_FRONTEND_LINK_LIBRARIES})
ov_add_library_version(${TARGET_NAME})
# WA for TF frontends which always require protobuf (not protobuf-lite)
# if TF FE is built in static mode, use protobuf for all other FEs
if(FORCE_FRONTENDS_USE_PROTOBUF)
set(OV_FRONTEND_PROTOBUF_LITE OFF)
endif()
if(proto_files)
if(OV_FRONTEND_PROTOBUF_LITE)
if(NOT protobuf_lite_installed)
ov_install_static_lib(${Protobuf_LITE_LIBRARIES} ${OV_CPACK_COMP_CORE})
set(protobuf_lite_installed ON CACHE INTERNAL "" FORCE)
endif()
link_system_libraries(${TARGET_NAME} PRIVATE ${Protobuf_LITE_LIBRARIES})
else()
if(NOT protobuf_installed)
ov_install_static_lib(${Protobuf_LIBRARIES} ${OV_CPACK_COMP_CORE})
set(protobuf_installed ON CACHE INTERNAL "" FORCE)
endif()
link_system_libraries(${TARGET_NAME} PRIVATE ${Protobuf_LIBRARIES})
endif()
# prptobuf generated code emits -Wsuggest-override error
if(SUGGEST_OVERRIDE_SUPPORTED)
target_compile_options(${TARGET_NAME} PRIVATE -Wno-suggest-override)
endif()
endif()
add_clang_format_target(${TARGET_NAME}_clang FOR_TARGETS ${TARGET_NAME}
EXCLUDE_PATTERNS ${PROTO_SRCS} ${PROTO_HDRS})
add_dependencies(ov_frontends ${TARGET_NAME})
if(NOT OV_FRONTEND_SKIP_INSTALL)
if(BUILD_SHARED_LIBS)
# Note:
# we use 'framework' as component for deployment scenario, i.e. for libraries itself
# and use common 'core_dev' component for headers, cmake files and symlinks to versioned library
set(lib_component "${OV_FRONTEND_NAME}")
set(dev_component "${OV_CPACK_COMP_CORE_DEV}")
# TODO: whether we need to do it configuralbe on Windows installer?
ie_cpack_add_component(${lib_component} HIDDEN)
if(OV_FRONTEND_LINKABLE_FRONTEND)
set(export_set EXPORT OpenVINOTargets)
set(archive_dest ARCHIVE DESTINATION ${OV_CPACK_ARCHIVEDIR}
COMPONENT ${lib_component})
set(namelink NAMELINK_COMPONENT ${dev_component})
else()
set(namelink NAMELINK_SKIP)
endif()
install(TARGETS ${TARGET_NAME} ${export_set}
RUNTIME DESTINATION ${OV_CPACK_RUNTIMEDIR} COMPONENT ${lib_component}
${archive_dest}
LIBRARY DESTINATION ${OV_CPACK_LIBRARYDIR} COMPONENT ${lib_component}
${namelink})
else()
ov_install_static_lib(${TARGET_NAME} ${OV_CPACK_COMP_CORE})
endif()
if(OV_FRONTEND_LINKABLE_FRONTEND)
# install library development files
install(DIRECTORY ${${TARGET_NAME}_INCLUDE_DIR}/openvino
DESTINATION ${FRONTEND_INSTALL_INCLUDE}
COMPONENT ${dev_component}
FILES_MATCHING PATTERN "*.hpp")
set_target_properties(${TARGET_NAME} PROPERTIES EXPORT_NAME frontend::${OV_FRONTEND_NAME})
export(TARGETS ${TARGET_NAME} NAMESPACE openvino::
APPEND FILE "${CMAKE_BINARY_DIR}/OpenVINOTargets.cmake")
endif()
else()
# skipped frontend has to be installed in static libraries case
ov_install_static_lib(${TARGET_NAME} ${OV_CPACK_COMP_CORE})
endif()
endmacro()

View File

@@ -1,27 +0,0 @@
// Copyright (C) 2018-2022 Intel Corporation
// SPDX-License-Identifier: Apache-2.0
//
#pragma once
#include "openvino/frontend/frontend.hpp"
@OV_FRONTEND_DECLARATIONS@
namespace {
using GetFrontEndDataFunc = void*();
using GetAPIVersionFunc = ov::frontend::FrontEndVersion();
struct Value {
GetFrontEndDataFunc* m_dataFunc;
GetAPIVersionFunc* m_versionFunc;
};
using FrontendsStaticRegistry = std::vector<Value>;
const FrontendsStaticRegistry getStaticFrontendsRegistry() {
@OV_FRONTEND_MAP_DEFINITION@
}
} // namespace

View File

@@ -1,24 +1,23 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
include(target_flags)
if(LINUX)
if (LINUX)
function(get_linux_name res_var)
if(EXISTS "/etc/lsb-release")
if (NOT EXISTS "/etc/lsb-release")
execute_process(COMMAND find -L /etc/ -maxdepth 1 -type f -name *-release -exec cat {} \;
OUTPUT_VARIABLE release_data RESULT_VARIABLE result)
string(REPLACE "Red Hat" "CentOS" release_data "${release_data}")
set(name_regex "NAME=\"([^ \"\n]*).*\"\n")
set(version_regex "VERSION=\"([0-9]+(\\.[0-9]+)?)[^\n]*\"")
else ()
# linux version detection using cat /etc/lsb-release
file(READ "/etc/lsb-release" release_data)
set(name_regex "DISTRIB_ID=([^ \n]*)\n")
set(version_regex "DISTRIB_RELEASE=([0-9]+(\\.[0-9]+)?)")
else()
execute_process(COMMAND find -L /etc/ -maxdepth 1 -type f -name *-release -exec cat {} \;
OUTPUT_VARIABLE release_data
RESULT_VARIABLE result)
string(REPLACE "Red Hat" "CentOS" release_data "${release_data}")
set(name_regex "NAME=\"([^ \"\n]*).*\"\n")
set(version_regex "VERSION=\"([0-9]+(\\.[0-9]+)?)[^\n]*\"")
endif()
endif ()
string(REGEX MATCH ${name_regex} name ${release_data})
set(os_name ${CMAKE_MATCH_1})
@@ -26,14 +25,10 @@ if(LINUX)
string(REGEX MATCH ${version_regex} version ${release_data})
set(os_name "${os_name} ${CMAKE_MATCH_1}")
if(os_name)
if (os_name)
set(${res_var} ${os_name} PARENT_SCOPE)
else ()
set(${res_var} NOTFOUND PARENT_SCOPE)
endif ()
endfunction()
else()
function(get_linux_name res_var)
set(${res_var} NOTFOUND PARENT_SCOPE)
endfunction()
endif ()

View File

@@ -1,4 +1,4 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#

View File

@@ -1,10 +1,8 @@
# Copyright (C) 2018-2022 Intel Corporation
# Copyright (C) 2018-2021 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
#
if(ENABLE_DATA)
find_package(Git REQUIRED)
endif()
find_package(Git REQUIRED)
set(MODELS_LST "")
set(MODELS_LST_TO_FETCH "")
@@ -32,7 +30,6 @@ function(add_lfs_repo name prefix url tag)
return()
endif()
include(ExternalProject)
ExternalProject_Add(${name}
PREFIX ${prefix}
GIT_REPOSITORY ${url}

Some files were not shown because too many files have changed in this diff Show More