IntenseWebs/openvino
3
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1,440 Commits 38 Branches 54 Tags
12b690952c5b9ba283e570d04cd10018dfbc29e9
Commit Graph
47 Commits
Author SHA1 Message Date
Vitaliy Urusovskij 12b690952c Time tests improvements (#2642) 
* Remove extra functions from run_timetest.py

* Add `log.debug` of raw and aggregated statistics in run_timetest.py

* Implement storing of models locally for test_timetest.py
 2020-10-14 20:05:52 +03:00
Vitaliy Urusovskij a4fe59ba0a Specify cl_cache_dir env var: (#2581) 
1. General behaviour doesn't work for some compute-runtime versions.
Specification of `cl_cache_dir` fixes it
 2020-10-08 15:02:31 +03:00
Vitaliy Urusovskij 474dcc277f Prepare new test configuration for test_timetest.py (#2561) 
* Add logging of DB upload to tconf with refs saving

* Prepare `.automation/tgl_test_config.yml` with models from VPUX package

* Add restriction on `exe` and `model` existence in run_timetest.py
 2020-10-08 10:30:43 +03:00
Vitaliy Urusovskij c3ff2948a4 Timetests test_runner improvements (#2552) 
* Remove `generate_tmp_path` as unnecessary after refactoring

* Add `check_positive_int` check for `-niter` CLI key

* Replace `TestConfDumper` with number of fixtures:
1. Save all test info in global `request` and `pytestconfig` fixtures
2. Add `test_info` fixture for retrieving test info from test
3. Add `prepare_tconf_with_refs` fixture for test conf dump
 2020-10-07 17:15:02 +03:00
Vitaliy Urusovskij 6949d93e42 Add "cl_cache" preparation in test_timetest.py (#2490) 2020-10-06 14:16:50 +03:00
Vitaliy Urusovskij 05d70cfa43 Remove --collect_results_only (#2509) 
* Remove CLI keys from README

* Remove `--collect_results_only` from MemCheckTests
 2020-10-05 17:43:24 +03:00
Vitaliy Urusovskij 639052d0b8 Support references in test_timetest.py (#2431) 2020-10-01 11:35:57 +03:00
Vitaliy Urusovskij c8530201dd Support DB reporting for test_timetest.py (#2359) 2020-10-01 08:39:19 +03:00
Vitaliy Urusovskij 0bd966ad08 Add fill_inputs step in timetests (#2413) 2020-09-25 14:08:03 +03:00
Vitaliy Urusovskij d48d2109dc Test timetest MVP (#2345) 2020-09-21 21:33:42 +03:00
Vitaliy Urusovskij babff1fd15 Extend timetest_infer pipeline with infer track and blob's support (#2298) 2020-09-21 02:09:41 +03:00
Evgeny Talanin fe99e055f2 Bump cmake version to 3.13 (#2258) 2020-09-18 18:58:12 +03:00
Vitaliy Urusovskij 20266276ec New layout for time_tests (#2321) 
* Change folders' layout for `time_tests`

* Refactoring of namings in `time_tests`
 2020-09-18 18:53:49 +03:00
Vitaliy Urusovskij 383152db28 Fix TimeTests build and README (#2275) 
* Fix searching of pipelines for `time_tests` build

* Add `realpath` use in README for IEDevPackage because of limitation

* Add `mkdir build && cd build` commands in README

* Rename `time-tests` to `time_tests` in README
 2020-09-17 13:01:40 +03:00
Andrey Somsikov e35b720e8f Fix time_tests gcc 4.8.5 build error (#2208) 2020-09-17 11:50:40 +03:00
Andrey SomsikovandAlina Alborova f84a6d97ac Split time-tests common library (#2173) 
Split time-tests common library

Add a README.md with workflow description.
Defined "timetest_" suffix for all time tests.
Applied clang-format-9 and added a README.md

Co-authored-by: Alina Alborova <alina.alborova@intel.com>
 2020-09-14 09:04:49 +03:00
Vitaliy Urusovskij a8f8ab8c12 Implement run_executable.py to run TimeTests several times (#2125) 2020-09-10 19:41:23 +03:00
Vitaliy Urusovskij 4bd05c5364 Implement statistics collection: (#2056) 
1. Add `-s` CLI key to get statistics file path
2. Implement `StatisticsWriter` singleton to manage handle to this file
 2020-09-09 15:30:23 +03:00
Andrey Somsikov b8a8fe7c0d Sort memcheck timeline report for failures (#2088) 2020-09-09 10:10:22 +03:00
Vitaliy Urusovskij 867340e8f1 Add runPipeline wrapper to mandatory track full run (#2100) 2020-09-09 00:13:07 +03:00
Vitaliy Urusovskij 0b61a7568f First time to inference POC (#1964) 2020-09-03 22:08:37 +03:00
Vitaliy Urusovskij 2386c5d376 [Stress] Define --timeout in run_memcheck.py used in gtest-parallel (#1576) 2020-08-05 12:33:01 +03:00
Vitaliy Urusovskij f8513d8fd3 [Stress] Remove --env_conf at all after deprecation (#1582) 2020-08-03 12:58:27 +03:00
Vitaliy Urusovskij 25cadb661b [Stress] Enable OMZ info_dumper.py in get_testdata.py (#1485) 
* [Stress] Support OMZ model_info.py in get_testdata.py

* [Stress] Copy IRs from OMZ models folder to IRs folder

* [Stress] Support modified configs in C++ tests

* [Stress] Deprecate support of --env_conf due refactoring of configs

* [Stress] Update configs:
1. Removed env configs due deprecation
2. Moved test configs to a new format

* [Stress] Extend MemCheck records with info from test config
 2020-07-31 12:13:12 +03:00
Andrey Somsikov 9299e32df0 Add product version to memcheck records (#1508) 2020-07-30 10:17:56 +03:00
Vitaliy Urusovskij 6fcad95c71 [Stress] Redesign of MemCheckTests (#650) 
* [Stress] Redesigned MemCheckTests: 1. Added MemCheckPipeline to incapsulate measures and logging. 2. Moved references to array

* [Stress] Added tracking of THREADS in MemCheckTests
 2020-07-23 11:07:13 +03:00
Vitaliy Urusovskij 5665ec9b26 [Stress] Small improvement of scripts (#1302) 
* [Stress] Extend CLI with arguments checks for compare_memcheck_2_runs.py

* [Stress] Extend requirements.txt with PyYAML
 2020-07-22 10:24:15 +03:00
Vitaliy Urusovskij 57779511d6 [Stress] Integrate compare_memcheck_2_runs call in run_memcheck.py (#1036) 
* [Stress] Define Database constant arguments in memcheck_upload.py only

* [Stress] Simplify computations using HashableDict in `compare_memcheck_2_runs`

* [Stress] Add comparison using pandas
 2020-07-20 10:23:51 +03:00
Vitaliy Urusovskij 930d687ed9 [Stress] Specify OMZ develop branch in get_testdata.py (#1318) 
Since in validation OMZ is used from a package (and OMZ is from develop branch), align OMZ default for get_testdata.py
 2020-07-16 15:28:25 +03:00
Vitaliy Urusovskij c0b28daf9c Fix MemCheckTests failures caused by change in OMZ models scope (#1214) 
Run only models available in OMZ to prevent failures
Remove person-reidentification-retail-0031.xml from configs
 2020-07-08 12:33:39 +03:00
Vitaliy Urusovskij e085a04d8c [Stress] Move logging after all computations of progress_str (#1226) 2020-07-07 14:23:26 +03:00
Vitaliy Urusovskij a17366f621 [Stress] Enable StressMemleaksTests on GPU in precommit (#1079) 
* [Stress] Fix missing retry for StressMemLeaksTests

* [Stress] Add smoothing with sliding average for StressMemleaksTests

* [Stress] Enable GPU in StressMemleaksTests precommit scope
 2020-07-03 11:57:16 +03:00
Vitaliy Urusovskij 0e904405f7 [Stress] Add --ref_db_collection in compare_memcheck_2_runs.py (#1157) 2020-07-02 13:14:07 +03:00
Ilya Lavrenov a7579d5c35 Removed ICNNNetReader interface (#1042) 
* Removed ICNNNetReader interface

* Fixed stress tests

* Fixed comments in VPU plugin

* Removed duplicated stress tests

* Fixed watchdog tests
 2020-06-23 22:34:26 +03:00
Andrey Somsikov 52a42624e9 Add memcheck runner script (#1031) 
Script executes measurement isolated with gtest-parallel,
handles database uploading and reports generation.
 2020-06-22 16:23:09 +03:00
Vitaliy Urusovskij 3417004e6d Add memcheck comparison script (#935) 
Add compare_memcheck_2_runs.py compares two runs.
Add handling of broken files for `parse_memcheck_log`
 2020-06-19 15:56:32 +03:00
Andrey Somsikov 626bc4f3d4 Add commit links to memcheck report (#820) 2020-06-08 15:08:58 +03:00
Vitaliy Urusovskij 72d9a9fae7 Use pre-defined DB collection names in memcheck_upload.py CLI (#651) 
Use argparses `choices` for `--db_collection` option.

Also removed unnecessary redefinition of `db_collection` in memcheck_upload.py
 2020-06-03 13:54:38 +03:00
Vitaliy Urusovskij 1d179fdb39 Add parallel downloads to stress tests (#678) 2020-06-02 17:24:22 +03:00
Vitaliy Urusovskij 4a44f84dab [Stress] Updated test_configs with new path to OMZ mtcnn models (#602) 2020-05-26 20:11:46 +03:00
Alexey Suhov deb008a26f publish master branch snapshot, revision 8d31237e2c3f673cbb0f0ba110fc10f5cce1d2bb 2020-05-22 02:23:12 +03:00
Alexey Suhov f30dcc218c publish master branch snapshot, revision 9df5eb1f84e13a35720a918f88324561222ab114 2020-05-20 01:13:06 +03:00
Alexey Suhov ba0a339888 publish master branch snapshot, revision 59af1853ca21ea08acf17b177da0b239753deb46 2020-05-18 17:21:58 +03:00
Alexey Suhov 645641e87d add execute permissions to get_testdata.py 2020-05-14 23:17:54 +03:00
Alexey Suhov 5b428f0655 publish master branch snapshot, revision 49482ae3bea0cbaa07474f86f36db11943142687 2020-05-13 21:12:22 +03:00
Alexey Suhov 357cc7eb4c publish master branch snapshot, revision 0110d9c98fd7209589d06344f0d836f61d81f4b3 2020-04-27 21:21:29 +03:00
Alexey Suhov 127cbac5bc publish master branch snapshot, revision cdcab9d7ab48ffb0ee5629fabbfa06cb45debd9b 2020-04-15 19:01:57 +03:00