IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[PyOV] Update BUILDING.md and minor dev docs fixes (#14357)

Browse Source
This commit is contained in:
Jan Iwaszkiewicz
2022-12-19 16:03:54 +01:00
committed by GitHub
parent 4d44c70c32
commit c8493bd47b
5 changed files with 89 additions and 289 deletions
Download Patch File Download Diff File Expand all files Collapse all files

65
src/bindings/python/docs/build.md Normal file
View File

@@ -0,0 +1,65 @@
# Building the OpenVINO™ Python API
**Refer to ["How to build OpenVINO" in OpenVINO™ developer documentation](../../../../docs/dev/build.md) for general building instructions.**
For each platform, you can build and install the API as a part of OpenVINO™ Toolkit or as a Python wheel.
A Python wheel is a portable package that allows you to install OpenVINO™ in either your Python distribution or a dedicated virtual environment.
## Virtual environments
OpenVINO can be built based on specific virtual environments such as [venv](https://docs.python.org/3/tutorial/venv.html), [virtualenv](https://virtualenv.pypa.io/en/latest/) or [pyenv](https://github.com/pyenv/pyenv). It is highly recommended to use virtual environments during development. They improve development process and allow better management of Python versions and packages.
*Note: Supported Python versions can be found in ["System Requirements" section](../../../../docs/install_guides/pypi-openvino-dev.md#system-requirements).*
### Example: using pyenv with OpenVINO™ on Linux based system
1. First, set up the `pyenv` project. Please follow [official instructions of the pyenv project](https://github.com/pyenv/pyenv#installation) for any additional information.
2. Install a desired Python version. Following example will use Python in version 3.10.7. To correctly link libraries, an installed Python version must match OpenVINO™:
* Python with a shared library for a dynamically linked OpenVINO™:
```shell
env PYTHON_CONFIGURE_OPTS="--enable-shared" pyenv install --verbose 3.10.7
```
* Python with a static library version for a static build of OpenVINO™:
```shell
pyenv install --verbose 3.10.7
```
3. Create a virtual environment based on the installed Python version:
```
pyenv virtualenv 3.10.7 ov-py310
```
4. Activate the environment:
```bash
pyenv activate ov-py310
```
5. Install developer requirements for OpenVINO™ Python API while inside virtual environment:
```shell
cd <openvino_repo>
pip install -r src/bindings/python/requirements.txt
pip install -r src/bindings/python/requirements_test.txt
pip install -r src/bindings/python/src/compatibility/openvino/requirements-dev.txt
```
If `-DENABLE_WHEEL=ON` flag is present in `cmake` command, additionally install wheel requirements:
```
pip install -r src/bindings/python/wheel/requirements-dev.txt
```
6. Add following flags to the main `cmake` command to use specific virtual environment:
```shell
-DPYTHON_EXECUTABLE=`which python` \
-DPYTHON_LIBRARY=/home/user/.pyenv/versions/3.10.7/lib/libpython3.10.so \
-DPYTHON_INCLUDE_DIR=/home/user/.pyenv/versions/3.10.7/include/python3.10
```
*Note: In order to use `which python`, specific virtual environment has to be activated in a current shell.*
*Note: `libpython3.10.so` is created with `--enable-shared` flag while installing specific Python version. For static build name of library may differ.*
7. Follow the rest of building and installation steps from ["How to build OpenVINO" developer documentation](../../../../docs/dev/build.md).
## Run tests to verify OpenVINO™ Python API
Follow instructions in [How to test OpenVINO™ Python API?](./test_examples.md#Running_OpenVINO™_Python_API_tests) to verify the build.

34
src/bindings/python/docs/code_examples.md
View File

@@ -1,17 +1,9 @@
# Examples of OpenVINO:tm: Python API code
# Examples of OpenVINO™ Python API code
#### Prerequisites
*To be added...*
#### Building and environment
Instructions can be found in ["Building the OpenVINO™ Python API"](./build.md).
##### Enviroment
<!-- TODO: Link to enviroment setup -->
*To be added...*
##### Building
<!-- TODO: Link to building instructions -->
*To be added...*
### Different ways of extending OpenVINO:tm: Python API
### Different ways of extending OpenVINO™ Python API
###### Before start: Project's naming conventions
General guide:
@@ -24,13 +16,13 @@ General guide:
One of the simplest ways to extend the existing codebase is by writing it in pure Python.
###### Before start: Layout of the project
How does OpenVINO:tm: packaging work? It is strictly connected to the layout of the Python API itself and reused in different supporting packages like tools and extensions. The main namespace of `openvino` provides a unified place that connects all packages together during import, **which is the required part**. However, it is up to the developer how to organize the rest of the package. There are also other common namespaces which follow the same rules:
How does OpenVINO™ packaging work? It is strictly connected to the layout of the Python API itself and reused in different supporting packages like tools and extensions. The main namespace of `openvino` provides a unified place that connects all packages together during import, **which is the required part**. However, it is up to the developer how to organize the rest of the package. There are also other common namespaces which follow the same rules:
* `openvino.tools`
* ...
For further reading, please refer to: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/
##### Creating new package that extends OpenVINO:tm: project namespace
##### Creating new package that extends OpenVINO™ project namespace
Let's go over the example available in `openvino/src/bindings/python/docs/examples/openvino`:
```
@@ -42,7 +34,7 @@ openvino/ <-- Main package/namespace
└── myclass.py
```
Now let's add it to your exisiting `PYTHONPATH` (replace `[your_path]` with correct path to the OpenVINO:tm: project):
Now let's add it to your exisiting `PYTHONPATH` (replace `[your_path]` with correct path to the OpenVINO™ project):
export PYTHONPATH=$PYTHONPATH:[your_path]/openvino/src/bindings/python/docs/examples/
@@ -106,7 +98,7 @@ Following this method, developers can add new modules and adjust existing ones,
<!-- Pure pybind11 solution describes C++ based approach -->
#### Pure pybind11 solution
The second approach to extend OpenVINO:tm: codebase is utilizing the *pybind11* library. It allows to write C++ based code, thus creating so-called Python bindings.
The second approach to extend OpenVINO™ codebase is utilizing the *pybind11* library. It allows to write C++ based code, thus creating so-called Python bindings.
**The example in this section covers the scenario of adding new features to a newly created submodule. Extending existing codebase can be done in a similar fashion by working on already implemented classes and modules.**
@@ -127,7 +119,7 @@ Add a new submodule by writing:
```cpp
py::module mymodule = m.def_submodule("mymodule", "My first feature - openvino.runtime.mymodule");
```
This is a shorthand way of adding new submodules which can later be used to extend the package. The mysterious `m` is actaully the main OpenVINO:tm: module called `pyopenvino` -- it is registered with `PYBIND11_MODULE(pyopenvino, m)` at the top of the "registering-point" file. Later imports from it are done by calling upon the `openvino._pyopenvino` package.
This is a shorthand way of adding new submodules which can later be used to extend the package. The mysterious `m` is actaully the main OpenVINO™ module called `pyopenvino` -- it is registered with `PYBIND11_MODULE(pyopenvino, m)` at the top of the "registering-point" file. Later imports from it are done by calling upon the `openvino._pyopenvino` package.
Keep in mind that in most real-life scenarios, modules and classes are registered in different files. The general idea is to create a helper function that will hold all of the registered modules, classes, and functions. This function needs to be exposed within a separate header file and included in "registering-point". The project's common guideline suggests to use names in the following convention: `regmodule_[domain]_[name_of_the_module]` or `regclass_[domain]_[name_of_the_class]`. Where optional `[domain]` generally points to parts of the API such as graph or frontend, or stay empty in the case of core runtime. Examples can be found in the "registering-point" file, `openvino/src/bindings/python/src/pyopenvino/pyopenvino.cpp`.
@@ -167,7 +159,7 @@ py::class_<MyTensor, std::shared_ptr<MyTensor>> cls(mymodule, "MyTensor");
cls.doc() = "These are my first class bindings!"
```
Create `__init__` and functions for the class. `py::arg` stand for actual arguments of the given function. Remember, these should match their C++ equivalents both in order and number. However, argument names are not required to be exact copies of the C++ ones, as different restricted keywords or shortnames could appear. Pick them tastefully :) A very unusual construct of `R"(...)"` adds a documentation string to the function. The OpenVINO:tm: project follows the *reST (reStructuredText)* format of docstrings.
Create `__init__` and functions for the class. `py::arg` stand for actual arguments of the given function. Remember, these should match their C++ equivalents both in order and number. However, argument names are not required to be exact copies of the C++ ones, as different restricted keywords or shortnames could appear. Pick them tastefully :) A very unusual construct of `R"(...)"` adds a documentation string to the function. The OpenVINO™ project follows the *reST (reStructuredText)* format of docstrings.
```cpp
// This initialize use already implemented C++ constructor
cls.def(py::init<ov::Tensor&>(),
@@ -253,7 +245,7 @@ Note: **bindings** that are created for classes are sometimes called **wrappers*
MyTensor wraps (around) Tensor class.
However, in OpenVINO:tm: there is an unwritten distinction between "everyday" wrappers and more complex ones (with this article published... it is now a written one ;) ). An example may be found in `openvino/src/bindings/python/src/pyopenvino/core/infer_request.hpp`, where `InferRequest` is actually wrapped inside `InferRequestWrapper`, similarly to the `Tensor` and `MyTensor` scenario. It helps to extend original object capabilities with members and functions that do not necessarily belong to the C++ API. Thus, explicitly calling something a **wrapper** in the project indicates that binding is probably inheriting or using the composition technique to include the original class, later extending it in some way.
However, in OpenVINO™ there is an unwritten distinction between "everyday" wrappers and more complex ones (with this article published... it is now a written one ;) ). An example may be found in `openvino/src/bindings/python/src/pyopenvino/core/infer_request.hpp`, where `InferRequest` is actually wrapped inside `InferRequestWrapper`, similarly to the `Tensor` and `MyTensor` scenario. It helps to extend original object capabilities with members and functions that do not necessarily belong to the C++ API. Thus, explicitly calling something a **wrapper** in the project indicates that binding is probably inheriting or using the composition technique to include the original class, later extending it in some way.
##### Overloads of functions
One of the main advantages of *pybind11* is the ability to resolve overloaded functions. Let's assume that a previously created function is extended to print any message passed by the user.
@@ -310,7 +302,7 @@ Notice that only functions with correct arguments are **not** throwing exception
#### Mix between Python and pybind11
Although *pybind11* is a powerful tool, it is sometimes required (or simply easier and more efficent) to combine both approaches and utilize both languages to achive best results.
##### Making pybind11-based module/class visible in OpenVINO:tm: package
##### Making pybind11-based module/class visible in OpenVINO™ package
Let's move a new class from `openvino._pyopenvino.mymodule` to the actual package. Simply introduce a new import statement in the desired file. Let it be `openvino/src/bindings/python/src/openvino/runtime/__init__.py`:
```python
from openvino._pyopenvino.mymodule import MyTensor
@@ -408,7 +400,7 @@ a.say_hello([1,2,3])
Great! Now the class has reached its destination, from C++, to Python, to Python once more. Such aliasing is a common technique in the project and gives a lot of power to the developer. With easy-to-understand code, the `say_hello` function is now able to dispatch arguments based on their type and apply necessary preprocessing to feed data into the function. However, you might say that this could be done with "a few more lines of C++" as well. Where is the tricky part? The answer is, the difficult feat of returning different types based on the same argument type is achieved here (look at the dispatching of integer arguments).
This concludes developer work on OpenVINO:tm: Python API. Don't forget to recompile your builds and have a good time while writing your code!:)
This concludes developer work on OpenVINO™ Python API. Don't forget to recompile your builds and have a good time while writing your code!:)
#### Testing out new code
All of the code is now written. Let's move on to testing.

9
src/bindings/python/docs/contributing.md
View File

@@ -1,4 +1,4 @@
# Contributing to OpenVINO:tm: Python API
# Contributing to OpenVINO™ Python API
#### Prerequisites
*To be added...*
@@ -9,13 +9,12 @@ In case the Python version you have is not supported by OpenVINO, you can refer
*To be added...*
##### Building
<!-- TODO: Link to building instructions -->
*To be added...*
Building instructions can be found in [BUILDING.md](../BUILDING.md)
## Contribution guidelines and best practices
#### How to contribute to Python API?
It is nothing special... :) First, make sure that all prerequisites are met and focus on writing the code itself. A good starting point is to have some knowledge of the Python language. C++ is also a vital language for OpenVINO:tm:, so it is not a surprise that it is used in this part of the project as well.
It is nothing special... :) First, make sure that all prerequisites are met and focus on writing the code itself. A good starting point is to have some knowledge of the Python language. C++ is also a vital language for OpenVINO™, so it is not a surprise that it is used in this part of the project as well.
Code snippets and detailed explanations can be found here:
<!-- Link to EXAMPLES -->
@@ -27,7 +26,7 @@ Please refer to Test Guide available here:
openvino/src/bindings/python/docs/test_examples.md
Moreover, the project utilizes *flake8* and *mypy* packages to run codestyle checks. Additionally OpenVINO:tm: uses the custom configuration file to exclude some strict rules. To run codestyle checks, navigate to the main Python API folder first and use following commands:
Moreover, the project utilizes *flake8* and *mypy* packages to run codestyle checks. Additionally OpenVINO™ uses the custom configuration file to exclude some strict rules. To run codestyle checks, navigate to the main Python API folder first and use following commands:
```shell
cd .../openvino/src/bindings/python/

22
src/bindings/python/docs/test_examples.md
View File

@@ -1,17 +1,9 @@
# How to test OpenVINO:tm: Python API?
# How to test OpenVINO™ Python API?
#### Prerequisites
*To be added...*
#### Building and environment
Instructions can be found in ["Building the OpenVINO™ Python API"](./build.md).
##### Enviroment
<!-- TODO: Link to enviroment setup -->
*To be added...*
##### Building
<!-- TODO: Link to building instructions -->
*To be added...*
### Running OpenVINO:tm: Python API tests
### Running OpenVINO™ Python API tests
*For simplicity, all of these commands require to navigate to the main Python API folder first:*
```shell
cd .../openvino/src/bindings/python/
@@ -43,12 +35,12 @@ To run full test suite one can utilize `tox` command:
tox
```
### Writing OpenVINO:tm: Python API tests
### Writing OpenVINO™ Python API tests
###### Before start
Follow and complete `openvino/src/bindings/python/docs/code_examples.md`.
##### Adding new test-case in the correct place
Let's add a new test for OpenVINO:tm: Python API.
Let's add a new test for OpenVINO™ Python API.
First, the test should confirm that the new pybind11-based class of `MyTensor` is behaving correctly. Navigate to tests folder and create a new file that describes tests within it. It should be along the lines of:
@@ -112,4 +104,4 @@ Notice that the test name is shared between cases. In a real-life pull request,
###### Difference between *tests* and *tests_compatibility* directories
<!-- TO-DELETE when compatibility layer is no longer supported in the project -->
Someone could notice two similar folders `tests` and `tests_compatibility`. First one is the desired place for all upcoming features and tests. Compatibility layer is only supported in specific cases and any updates to it should be explicitly approved by OpenVINO:tm: reviewers. Please do not duplicate tests in both directories if not necessary.
Someone could notice two similar folders `tests` and `tests_compatibility`. First one is the desired place for all upcoming features and tests. Compatibility layer is only supported in specific cases and any updates to it should be explicitly approved by OpenVINO™ reviewers. Please do not duplicate tests in both directories if not necessary.