[Docs] Add Python dependencies management docs (#17439)

Co-authored-by: Karol Blaszczak <karol.blaszczak@intel.com>
This commit is contained in:
Przemyslaw WysockiandKarol Blaszczak authored and GitHub committed 2023-05-11 11:17:03 +00:00
1 parent d99a5ab1ba
commit aaf4fdb84f
2 files changed
+95

No files matched your search

+3
View File
@@ -60,6 +60,9 @@ OpenVINO can be built based on specific virtual environments such as [venv](http
7. Follow the rest of building and installation steps from ["How to build OpenVINO" developer documentation](../../../../docs/dev/build.md).
## Project dependencies management
For details please refer to [Python requirements and version constraints management](./requirements_management.md).
## Run tests to verify OpenVINO™ Python API
Follow instructions in [How to test OpenVINO™ Python API?](./test_examples.md#Running_OpenVINO™_Python_API_tests) to verify the build.
@@ -0,0 +1,92 @@
# Python requirements and version constraints management
OpenVINO uses a pip built-in feature called "constraints" in order to reduce the complexity of requirements.
## What are constraints files?
Official constraints [documentation](https://pip.pypa.io/en/stable/user_guide/#constraints-files) describes them as:
> Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is a subset of Requirements Files, with several kinds of syntax not allowed: constraints must have a name, they cannot be editable, and they cannot specify extras. In terms of semantics, there is one key difference: Including a package in a constraints file does not trigger installation of the package.
Constraints files have the `.txt` extension and are addons to regular `requirements.txt` files. They are useful when the project has multiple components, each with its own dependencies.
This means we can specify only package names for different components and keep their versions in a centralized constraints file, which is significantly easier to maintain.
### Example
Two requirements files are linked to a centralized constraints file using the `-c` flag.
```text
# main/constraints.txt
coverage>=4.4.2
astroid>=2.9.0
pylint>=2.7.0
pyenchant>=3.0.0
test-generator==0.1.1
```
```text
# requirements_pytorch.txt
-c main/constraints.txt # versions are already defined here!
coverage
astroid
pylint
pyenchant
```
```text
# requirements_tensorflow.txt
-c main/constraints.txt # versions are already defined here!
coverage
pylint
pyenchant
test-generator
```
Note that `astroid` is not required by `requirements_tensorflow.txt`, so it's not going to be installed even though its version has been specified in `main/constraints.txt`.
## Benefits of this approach
Using `constraints.txt` files brings many benefits, some of which are:
- Lower project complexity
- Significantly fewer requirements files to maintain
- Centralization of project requirements
- Easier pip conflict management
- Upgrading package versions is now easier
- Package versions are soft-forced to be aligned across the project
- Groundwork for future package managers and utilities
## Known limitations and best practices
There are several known limitations to the `constraints.txt` approach. The most notable are:
#### 1. Environment markers
Placing [environment markers](https://peps.python.org/pep-0508/) in `constraints.txt` files can be prone to bugs due to how pip resolver interprets them. Markers should be placed only in `requirements.txt` files unless it is certain they will work as intended.
#### 2. Inconsistent version requirements
If a package version differs between `requirements.txt` files, it can't be unified in the common `constraints.txt` file. The workarounds, in order of preference, are:
- Align the package versions between `requirements.txt` files
- Exclude this package from `constraints.txt` and keep its version in `requirements.txt`
- Exclude this `requirements.txt` file from the constraints system
#### 3. Lack of support in distutils
`distutils` is a package used for building Python wheels. It does not offer support for constraints files, so custom requirement parsers had to be added. A parser example can be found [here](https://github.com/openvinotoolkit/openvino/blob/master/tools/mo/setup.py) - functions `read_constraints()` and `read_requirements()`.
## Implementation in OpenVINO
The implementation in OpenVINO is a subject to change. At the time of writing, there are three `constraints.txt` files with the following requirement coverage:
|Constraints file |Coverage |
|-----------------------------------------------|---------------------------|
|`openvino/src/bindings/python/constraints.txt` |Python bindings, frontends |
|`openvino/tests/constraints.txt` |tests |
|`openvino/tools/constraints.txt` |tools, openvino_dev |
## See also
 * [OpenVINO™ README](../../../../README.md)
 * [OpenVINO™ bindings README](../../README.md)
 * [Developer documentation](../../../../docs/dev/index.md)