Files

Maxim Shevtsov d0eef043fd [MULTI]Data affinity remote context and blobs (#3342 )

* zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these

* Optimized Infer Request Scheduling

* remoteblob checks in the conventional SetBlob

* correctly (with status) reporting NOT_IMPLEMENTED

* SetBlob to accomodate for the RemoteBobs

* Tests for remote blobs support via MULTI: creating the shared_test in case the other (closed source) plugins would want to use that (in the private shared_tests instantiations).
Also instantiating the remote blobs tests for the some basic combinations to test the MULTI supports them

* macos compilation (and general plugin platform support) fix

* shuffled files, so that the MULTI tests are now part of the ieFuncTests (and need no separate target). Also brushed the macro that handales the NOT_IMPLEMENTED as bit

* further shuffled files, so that the initial MULTI tests are now part of the IE tests, yet specific instances do need separate targets

* Fixed misprint

* Brushing the code and comments a bit

* further brushing of the ScheduleToWorkerRequest: moving the task execution directly into the loop over devices (avoids pointers and 'else' clause)

* 1) zero-copy (assuming determenistic app-level scheduling) for the multi-device, via "borrowing" the corresponding device-specific blobs and letting the app to implicitly use these

2) Initial MULTI section in the opt guide (primarily to document a tip on helping the MULTI to keep the zero-copy path)

* [MULTI] remote context support and associated scheduling (respecting the remote data affinity)

* fix CentOS (old) gcc issue: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81880
since the intriduced therad_local string is template the bug manifests itself (and the string is not allocated/initialized).
the QA is to wrap the std::string into the function

* further fix for the old gcc versions issue, now with non-trivial thread_local destruction sefault: switching from the std::string to the plain const char*

* additional tests for the MULTI and remote blobs (no remote context and multi GPUs cases)

* fix for the tests (that now can check for more specific NotImplemented exeption).
Alos couple of line endings

2020-12-09 09:52:19 +03:00

functional

[MULTI]Data affinity remote context and blobs (#3342 )

2020-12-09 09:52:19 +03:00

ie_test_utils

[MULTI]Data affinity remote context and blobs (#3342 )

2020-12-09 09:52:19 +03:00

ngraph_helpers

Reference Implementation of ROIPooling op (#2903 )

2020-11-30 06:59:31 +03:00

unit

Remove opset0 support and undesired passes from Interpreter backend (#1469 )

2020-12-03 12:36:34 +03:00

CMakeLists.txt

[IE TESTS] Remove extra dependencies from NgraphFunction lib (#3222 )

2020-11-27 16:48:39 +03:00

ie_tsan.supp

publish master branch snapshot, revision 9df5eb1f84e13a35720a918f88324561222ab114

2020-05-20 01:13:06 +03:00

README.md

Publishing 2020.2 content

2020-04-13 21:17:23 +03:00

README.md

Inference Engine Test Infrastructure

This is OpenVINO Inference Engine testing framework. OpenVINO Inference Engine test system contains:

Unit tests
This test type is used for detailed testing of each software instance (including internal classes with their methods) within the tested modules (Inference Engine and Plugins). There are following rules which are required for Unit Tests development:
- All unit tests are separated into different executables per each tested module.
- Unit test folder for a particular module should replicate SRC folder layout of the corresponding tested module to allow further developers get better understanding which part of software is already covered by unit tests and where to add new tests if needed.
  
  Example: We have network_serializer.h and network_serializer.cpp files within the src folder of the tested Inference Engine module. Then, new network_serializer_test.cpp file should be created within the root of the Unit Test folder for this module. This test file should cover all the classes and methods from the original files.
  
  Example: We have ie_reshaper.cpp within the src/shape_infer subfolder of the tested module. In this case new shape_infer subfolder should be created within the the root of the Unit Test folder for this module. And new ie_reshaper_test.cpp file should be created within this newly created subfolder. This test file should cover all the classes and methods from the original file.
- Each Unit Test should cover the only target classes and methods. If needed, all external interface components should be mocked. There are common mock objects provided within the common Unit Test Utilities to stub the general Inference Engine API classes.
  
  Example: We have cnn_network_impl.hpp and cnn_network_impl.cpp files within the src folder of the tested module. In this case, new cnn_network_impl_test.cpp file should be created and it should contain tests on CNNNetworkImpl class only.
- It's not prohibited to have several test files for the same file from the tested module.
- It's not prohibited to create a separate test file for a specific classes or functions (not for the whole file).
Functional tests
This test type is used to verify public Inference Engine API. There are following types of functional tests:
- inference_engine_tests are plugin-independent tests. Used to verify Inference Engine API methods which don't involve any plugin runtime. E.g. network_reader, network_serializer, precision tests.
- plugin_tests are plugin-dependent tests. These tests require plugin runtime to be executed during testing. E.g. any tests using ExecutableNetwork, InferRequest API can only be implemented within this test group.
Example: Any new test on creating of a CNNNetwork object and checking of its output info should be included to to the Inference Engine Functional tests suite. But any new test containing reading of a network and loading it to a specified plugin is always the plugin test.

There are following rules which are required for Functional Tests development:
- All Functional tests are separated into different executables for the Inference Engine and each plugin.
- Pre-converted IR files must not be used within the new Functional Tests. Tested models should be generated during the tests execution. The main method to generate a required model is building of the required NGraph function and creating of a CNNNetwork using it. If a required layer is not covered by Ngraph it's allowed to build IR file using xml_net_builder utility (please refer to the ir_net.hpp file). IR XML files hardcoded as strings within the test code should not be used.
- All the plugin test cases are parametrized with (at least) the device name and included to the common funcSharedTests static library. This library is linked to the Plugin Test binaries. And all the plugin developers just add required test instantiations based on the linked test definitions to own test binary. It should be done to make all the shared test cases always visible and available to instantiate by other plugins.
  
  Note
  
  : Any new plugin test case should be added to the common test definitions library (funcSharedTests) within the DLDT repository first. And then this test case can be instantiated with the required parameters inside own plugin's test binary which links this shared tests library.
  
  Note
  
  : funcSharedTests library is added to the developer package and available for closed source development.
- All the inference engine functional test cases are defined and instantiated within the single test binary. These test cases are not implemented as a separate library and not available for instantiations outside this binary.
Inference Engine tests utilities
The set of utilities which are used by the Inference Engine Functional and Unit tests. Different helper functions, blob comparators, OS specific constants, etc are implemented within the utilities.
Internal namespaces (for example, CommonTestUtils::, FuncTestUtils:: or UnitTestUtils::) must be used to separate utilities by domains.

Note

: All the utilities libraries are added to the developer package and available for closed source development.