[PyOV] Adjust style in dev docs (#17440)
This commit is contained in:
Jan Iwaszkiewicz
GitHub
@@ -63,3 +63,8 @@ OpenVINO can be built based on specific virtual environments such as [venv](http
|
||||
## Run tests to verify OpenVINO™ Python API
|
||||
|
||||
Follow instructions in [How to test OpenVINO™ Python API?](./test_examples.md#Running_OpenVINO™_Python_API_tests) to verify the build.
|
||||
|
||||
## See also
|
||||
* [OpenVINO™ README](../../../../README.md)
|
||||
* [OpenVINO™ bindings README](../../README.md)
|
||||
* [Developer documentation](../../../../docs/dev/index.md)
|
||||
|
||||
@@ -1,34 +1,34 @@
|
||||
# Examples of OpenVINO™ Python API code
|
||||
|
||||
#### Building and environment
|
||||
### Building and environment
|
||||
Instructions can be found in ["Building the OpenVINO™ Python API"](./build.md).
|
||||
|
||||
### Different ways of extending OpenVINO™ Python API
|
||||
## Different ways of extending OpenVINO™ Python API
|
||||
|
||||
###### Before start: Project's naming conventions
|
||||
### Before start: Project's naming conventions
|
||||
General guide:
|
||||
* Snake case (also known as the *lower_case_with_underscores* style) is used across the codebase. That includes modules (`runtime`, `offline_transformations`), function names, and arguments/variables (`async_infer`, `wait`, `path_to_xml`).
|
||||
* Naming of classes is an exception to the above rule. The *CamelCase* style is used in this case, for example: `Core`, `InferRequest` or `AsyncInferQueue`.
|
||||
* If bindings (explained later in the [Pure pybind11 solution](#pure-pybind11-solution) section) are created to expose existing C++ code, make them similar to their C++ counterparts, regarding both names and placement, for example, C++'s `ov::InferRequest` and Python's `openvino.runtime.InferRequest`. If alignment is not possible, try to describe your class/function/module as well as possible, such as the pair of `openvino.runtime.ConstOutput/openvino.runtime.Output` which relates to `ov::Output<const ov::Node>/ov::Output<ov::Node>`. This naming points out the functional difference between both classes - one is an immutable and the other a mutable version.
|
||||
|
||||
<!-- Pure Python solution describes Python based approach -->
|
||||
#### Pure Python solution
|
||||
## Pure Python solution
|
||||
One of the simplest ways to extend the existing codebase is by writing it in pure Python.
|
||||
|
||||
###### Before start: Layout of the project
|
||||
### Layout of the project
|
||||
How does OpenVINO™ packaging work? It is strictly connected to the layout of the Python API itself and reused in different supporting packages like tools and extensions. The main namespace of `openvino` provides a unified place that connects all packages together during import, **which is the required part**. However, it is up to the developer how to organize the rest of the package. There are also other common namespaces which follow the same rules:
|
||||
* `openvino.tools`
|
||||
* ...
|
||||
|
||||
For further reading, please refer to: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/
|
||||
|
||||
##### Creating new package that extends OpenVINO™ project namespace
|
||||
Let's go over the example available in `openvino/src/bindings/python/docs/examples/openvino`:
|
||||
### Creating new package that extends OpenVINO™ project namespace
|
||||
Let's go over the example available in [examples folder](./examples/openvino):
|
||||
|
||||
```
|
||||
openvino/ <-- Main package/namespace
|
||||
├── __init__.py <-- Unified file between all packages
|
||||
└── mymodule/ <-- This is your new module and it's contents:)
|
||||
└── mymodule/ <-- This is your new module and its contents
|
||||
├── __init__.py
|
||||
├── ...
|
||||
└── myclass.py
|
||||
@@ -48,8 +48,8 @@ obj.say_hello()
|
||||
>>> "Hello! Let's work on OV together!"
|
||||
```
|
||||
|
||||
##### Extending of existing API (sub)modules
|
||||
But how to extend existing API? Let's navigate to `openvino/src/bindings/python/src/openvino` and add something to project helpers. Create new directory and fill it's contents:
|
||||
### Extending of existing API (sub)modules
|
||||
But how to extend existing API? Let's navigate to [main bindings folder](./../src/openvino/) and add something to project helpers. Create new directory and fill it's contents:
|
||||
|
||||
```
|
||||
openvino/
|
||||
@@ -67,18 +67,20 @@ openvino/
|
||||
└── utils.py
|
||||
```
|
||||
|
||||
Let's add in `custom_module/custom_helpers.py`:
|
||||
Let's add in [`custom_helpers.py`](./examples/custom_module/custom_helpers.py):
|
||||
<!-- TODO: Link with code -->
|
||||
```python
|
||||
def top1_index(results: list) -> int:
|
||||
return results.index(max(results))
|
||||
```
|
||||
|
||||
Import it to a new module in `custom_module/__init__.py`:
|
||||
Import it to a new module in [`custom_module/__init__.py`](./examples/custom_module/__init__.py):
|
||||
<!-- TODO: Link with code -->
|
||||
```python
|
||||
from openvino.helpers.custom_module.custom_helpers import top1_index
|
||||
```
|
||||
|
||||
Follow it with correct import in `helpers/__init__.py`:
|
||||
Follow it with correct import in [`helpers/__init__.py`](./../src/openvino/helpers/__init__.py):
|
||||
```python
|
||||
from openvino.helpers.custom_module import top1_index
|
||||
```
|
||||
@@ -97,23 +99,21 @@ Following this method, developers can add new modules and adjust existing ones,
|
||||
|
||||
<!-- Pure pybind11 solution describes C++ based approach -->
|
||||
|
||||
#### Pure pybind11 solution
|
||||
## Pure pybind11 solution
|
||||
The second approach to extend OpenVINO™ codebase is utilizing the *pybind11* library. It allows to write C++ based code, thus creating so-called Python bindings.
|
||||
|
||||
**The example in this section covers the scenario of adding new features to a newly created submodule. Extending existing codebase can be done in a similar fashion by working on already implemented classes and modules.**
|
||||
|
||||
###### Before start: What is pybind11?
|
||||
### Before start: What is pybind11?
|
||||
It is a thridparty project that allows to expose C++ code as a Python library.
|
||||
|
||||
Link to offical documentation: https://pybind11.readthedocs.io/en/stable/
|
||||
Link to project repository: https://github.com/pybind/pybind11
|
||||
|
||||
##### Adding new (sub)module
|
||||
### Adding new (sub)module
|
||||
Adding a new module could be done only by using *pybind11* built-in capabilities.
|
||||
|
||||
Navigate to the main project file responsible for creation of the whole package, let's call it "registering-point":
|
||||
|
||||
openvino/src/bindings/python/src/pyopenvino/pyopenvino.cpp
|
||||
Navigate to the main project file responsible for creation of the whole package [`pyopenvino.cpp`](./../src/pyopenvino/pyopenvino.cpp), let's call it "registering-point".
|
||||
|
||||
Add a new submodule by writing:
|
||||
```cpp
|
||||
@@ -121,11 +121,11 @@ py::module mymodule = m.def_submodule("mymodule", "My first feature - openvino.r
|
||||
```
|
||||
This is a shorthand way of adding new submodules which can later be used to extend the package. The mysterious `m` is actaully the main OpenVINO™ module called `pyopenvino` -- it is registered with `PYBIND11_MODULE(pyopenvino, m)` at the top of the "registering-point" file. Later imports from it are done by calling upon the `openvino._pyopenvino` package.
|
||||
|
||||
Keep in mind that in most real-life scenarios, modules and classes are registered in different files. The general idea is to create a helper function that will hold all of the registered modules, classes, and functions. This function needs to be exposed within a separate header file and included in "registering-point". The project's common guideline suggests to use names in the following convention: `regmodule_[domain]_[name_of_the_module]` or `regclass_[domain]_[name_of_the_class]`. Where optional `[domain]` generally points to parts of the API such as graph or frontend, or stay empty in the case of core runtime. Examples can be found in the "registering-point" file, `openvino/src/bindings/python/src/pyopenvino/pyopenvino.cpp`.
|
||||
Keep in mind that in most real-life scenarios, modules and classes are registered in different files. The general idea is to create a helper function that will hold all of the registered modules, classes, and functions. This function needs to be exposed within a separate header file and included in "registering-point". The project's common guideline suggests to use names in the following convention: `regmodule_[domain]_[name_of_the_module]` or `regclass_[domain]_[name_of_the_class]`. Where optional `[domain]` generally points to parts of the API such as graph or frontend, or stay empty in the case of core runtime. Examples can be found in the "registering-point" file [`pyopenvino.cpp`](../src/pyopenvino/pyopenvino.cpp).
|
||||
|
||||
*Note: Submodules can be "chained" as well. Refer to the official documentation for more details: https://pybind11.readthedocs.io/en/stable/reference.html#_CPPv4N7module_13def_submoduleEPKcPKc*
|
||||
|
||||
##### Binding of classes and functions
|
||||
### Binding of classes and functions
|
||||
When the module is created, classes can be added to it. Let's assume a class called `MyTensor` needs to be added in a new module. Here is a list of required features:
|
||||
|
||||
* Handle construction from the `ov::Tensor` class and 1-D initialization from Python built-in list.
|
||||
@@ -245,9 +245,9 @@ Note: **bindings** that are created for classes are sometimes called **wrappers*
|
||||
|
||||
MyTensor wraps (around) Tensor class.
|
||||
|
||||
However, in OpenVINO™ there is an unwritten distinction between "everyday" wrappers and more complex ones (with this article published... it is now a written one ;) ). An example may be found in `openvino/src/bindings/python/src/pyopenvino/core/infer_request.hpp`, where `InferRequest` is actually wrapped inside `InferRequestWrapper`, similarly to the `Tensor` and `MyTensor` scenario. It helps to extend original object capabilities with members and functions that do not necessarily belong to the C++ API. Thus, explicitly calling something a **wrapper** in the project indicates that binding is probably inheriting or using the composition technique to include the original class, later extending it in some way.
|
||||
However, in OpenVINO™ there is an unwritten distinction between "everyday" wrappers and more complex ones (with this article published... it is now a written one ;) ). An example may be found in [`core/infer_request.hpp`](./../src/pyopenvino/core/infer_request.hpp), where `InferRequest` is actually wrapped inside `InferRequestWrapper`, similarly to the `Tensor` and `MyTensor` scenario. It helps to extend original object capabilities with members and functions that do not necessarily belong to the C++ API. Thus, explicitly calling something a **wrapper** in the project indicates that binding is probably inheriting or using the composition technique to include the original class, later extending it in some way.
|
||||
|
||||
##### Overloads of functions
|
||||
### Overloads of functions
|
||||
One of the main advantages of *pybind11* is the ability to resolve overloaded functions. Let's assume that a previously created function is extended to print any message passed by the user.
|
||||
|
||||
```cpp
|
||||
@@ -299,11 +299,11 @@ Notice that only functions with correct arguments are **not** throwing exception
|
||||
|
||||
<!-- Mixed solution describes both approaches combined -->
|
||||
|
||||
#### Mix between Python and pybind11
|
||||
## Mix between Python and pybind11
|
||||
Although *pybind11* is a powerful tool, it is sometimes required (or simply easier and more efficent) to combine both approaches and utilize both languages to achive best results.
|
||||
|
||||
##### Making pybind11-based module/class visible in OpenVINO™ package
|
||||
Let's move a new class from `openvino._pyopenvino.mymodule` to the actual package. Simply introduce a new import statement in the desired file. Let it be `openvino/src/bindings/python/src/openvino/runtime/__init__.py`:
|
||||
### Making pybind11-based module/class visible in OpenVINO™ package
|
||||
Let's move a new class from `openvino._pyopenvino.mymodule` to the actual package. Simply introduce a new import statement in the desired file. Let it be [`openvino/runtime/__init__.py`](./../src/openvino/runtime/__init__.py):
|
||||
```python
|
||||
from openvino._pyopenvino.mymodule import MyTensor
|
||||
```
|
||||
@@ -317,10 +317,10 @@ ov.MyTensor
|
||||
|
||||
Same rule applies to whole modules and free functions. **This is a required step when adding something to the public API**. Without exposing it, all of the work is hidden in the depths of the `pyopenvino` namespace, rendering it hard to access for the user.
|
||||
|
||||
##### Yet another Python layer
|
||||
### Yet another Python layer
|
||||
As mentioned earlier, it may be helpful to utilize Python in-between to achieve hard C++ feats in a more efficient way. Let's extend the previously created `say_hello` function a little bit.
|
||||
|
||||
First, create a new file in the `openvino/src/bindings/python/src/openvino/runtime` directory and call it `mymodule_ext.py`. There are no strict rules for naming, just make sure the names are in good taste. Import the class here:
|
||||
First, create a new file in the [runtime directory](./../src/openvino/runtime/) and call it `mymodule_ext.py`. There are no strict rules for naming, just make sure the names are in good taste. Import the class here:
|
||||
```python
|
||||
from openvino._pyopenvino.mymodule import MyTensor as MyTensorBase
|
||||
```
|
||||
@@ -402,8 +402,13 @@ Great! Now the class has reached its destination, from C++, to Python, to Python
|
||||
|
||||
This concludes developer work on OpenVINO™ Python API. Don't forget to recompile your builds and have a good time while writing your code!:)
|
||||
|
||||
### Testing the new code
|
||||
## Testing the new code
|
||||
|
||||
Coding is now finished. Let's move on to testing.
|
||||
|
||||
To learn how to test your code, refer to the guide on [how to test OpenVINO™ Python API?](./test_examples.md#Running_OpenVINO™_Python_API_tests)
|
||||
|
||||
## See also
|
||||
* [OpenVINO™ README](../../../../README.md)
|
||||
* [OpenVINO™ bindings README](../../README.md)
|
||||
* [Developer documentation](../../../../docs/dev/index.md)
|
||||
|
||||
@@ -1,25 +1,25 @@
|
||||
# Contributing to OpenVINO™ Python API
|
||||
|
||||
#### Prerequisites
|
||||
## Prerequisites
|
||||
|
||||
##### Enviroment
|
||||
In case the Python version you have is not supported by OpenVINO, you can refer to [openvino/src/bindings/python/docs/python_version_upgrade.md](https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/docs/python_version_upgrade.md) for instructions on how to download and build a newer, supported Python version.
|
||||
### Enviroment
|
||||
In case the Python version you have is not supported by OpenVINO, you can refer to ["Python version upgrade" guide](https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/docs/python_version_upgrade.md) for instructions on how to download and build a newer, supported Python version.
|
||||
|
||||
The environment setup is described as part of the pyenv example in [the build.md page](./build.md#Example:_using_pyenv_with_OpenVINO™_on_Linux_based_system).
|
||||
The environment setup is described as part of the pyenv example in [Building the OpenVINO™ Python API](./build.md#Example:_using_pyenv_with_OpenVINO™_on_Linux_based_system).
|
||||
|
||||
##### Building
|
||||
Building instructions can be found in [build.md](./build.md#_Building_the_OpenVINO™_Python_API).
|
||||
### Building
|
||||
Building instructions can be found in [Building the OpenVINO™ Python API](./build.md#_Building_the_OpenVINO™_Python_API).
|
||||
|
||||
## Contribution guidelines and best practices
|
||||
|
||||
#### How to contribute to Python API?
|
||||
### How to contribute to Python API?
|
||||
It is nothing special... :) First, make sure that all prerequisites are met and focus on writing the code itself. A good starting point is to have some knowledge of the Python language. C++ is also a vital language for OpenVINO™, so it is not a surprise that it is used in this part of the project as well.
|
||||
|
||||
Code snippets and detailed explanations can be found here:
|
||||
|
||||
[Examples of OpenVINO™ Python API code](./code_example.md)
|
||||
|
||||
##### Always test out our code! Don't forget about it before pushing and triggering CIs.
|
||||
### Always test out our code! Don't forget about it before pushing and triggering CIs.
|
||||
|
||||
To learn how to test your code, refer to the guide on [how to test OpenVINO™ Python API?](./test_examples.md#Running_OpenVINO™_Python_API_tests)
|
||||
|
||||
@@ -35,12 +35,12 @@ mypy src/openvino/ --config-file ./setup.cfg
|
||||
|
||||
**Python API CIs are composed of both functional tests and codestyle checks and may fail because of warnings/errors in both stages.**
|
||||
|
||||
##### Adding dependencies to the project
|
||||
### Adding dependencies to the project
|
||||
Remember that a new module/feature may be dependent on various third party modules. Please add a sufficient `requirements.txt` file and mention it in a Pull Request description. Consider other project requirements and dependencies of your module to make `requirements.txt` as compact as possible.
|
||||
|
||||
**Please look for current supported Python versions and check if packages are compatibile with them and not depreacated.**
|
||||
|
||||
#### Description of the Pull Request
|
||||
### Description of the Pull Request
|
||||
Please append all PR titles with a tag `[PyOV]` or `[PYTHON]`. Feel free to describe any level of relevant details in the PR, it helps a lot with the review process. The minimum requirement is a compact description of changes made, the form of a bullet-point list is really appreciated.
|
||||
|
||||
Template for external contributors:
|
||||
@@ -63,3 +63,8 @@ Requirements introduced/changed: <-- only if applicable
|
||||
Tickets:
|
||||
XXXX, YYYY <-- only numbers from tickets
|
||||
```
|
||||
|
||||
## See also
|
||||
* [OpenVINO™ README](../../../../README.md)
|
||||
* [OpenVINO™ bindings README](../../README.md)
|
||||
* [Developer documentation](../../../../docs/dev/index.md)
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright (C) 2018-2023 Intel Corporation
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
from openvino.helpers.custom_module.custom_helpers import top1_index
|
||||
@@ -0,0 +1,6 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
# Copyright (C) 2018-2023 Intel Corporation
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
def top1_index(results: list) -> int:
|
||||
return results.index(max(results))
|
||||
@@ -1,6 +1,6 @@
|
||||
# Python version upgrade
|
||||
|
||||
#### Notes
|
||||
### Notes
|
||||
Upgrade described in this documentation file can be useful when using a system such as Ubuntu18, which default Python is no longer supported (in this case Python 3.6). The recommended action is to use a newer system instead of upgrading Python.
|
||||
|
||||
*Warning: You make all changes at your own risk.*
|
||||
@@ -35,3 +35,8 @@ Verify your installation:
|
||||
python3.8 --version
|
||||
> Python 3.8.13
|
||||
```
|
||||
|
||||
## See also
|
||||
* [OpenVINO™ README](../../../../README.md)
|
||||
* [OpenVINO™ bindings README](../../README.md)
|
||||
* [Developer documentation](../../../../docs/dev/index.md)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# How to test OpenVINO™ Python API?
|
||||
|
||||
#### Building and environment
|
||||
## Building and environment
|
||||
Instructions can be found in ["Building the OpenVINO™ Python API"](./build.md).
|
||||
|
||||
Install the specific requirements file for testing:
|
||||
@@ -12,8 +12,8 @@ Make sure that Python libraries are added to the user environment variables:
|
||||
```
|
||||
export PYTHONPATH=PYTHONPATH:<openvino_repo>/bin/intel64/Release/python_api/python3.7
|
||||
```
|
||||
### Run OpenVINO™ Python API tests
|
||||
*For simplicity, all of these commands require to navigate to the main Python API folder first:*
|
||||
## Run OpenVINO™ Python API tests
|
||||
*For simplicity, all of these commands require to navigate to the [main Python API folder](./../) first:*
|
||||
```shell
|
||||
cd .../openvino/src/bindings/python/
|
||||
```
|
||||
@@ -55,9 +55,9 @@ To run full test suite one can utilize `tox` command:
|
||||
tox
|
||||
```
|
||||
|
||||
### Check the codestyle of Python API
|
||||
## Check the codestyle of Python API
|
||||
There are two packages used in the project to check the codestyle of python code: *mypy* and *flake8*.
|
||||
Besides, OpenVINO™ uses a custom configuration file to exclude some strict rules.
|
||||
Besides, OpenVINO™ uses a [custom configuration file](./../setup.cfg) to exclude some strict rules.
|
||||
|
||||
To check the codestyle of the Python API 2.0, run the following commands:
|
||||
```
|
||||
@@ -80,14 +80,14 @@ This check should be executed from the main Python API folder:
|
||||
```
|
||||
python -m flake8 ./tests/ --config=setup.cfg
|
||||
```
|
||||
### Writing OpenVINO™ Python API tests
|
||||
###### Before start
|
||||
Follow and complete `openvino/src/bindings/python/docs/code_examples.md`.
|
||||
## Writing OpenVINO™ Python API tests
|
||||
### Before start
|
||||
Follow and complete [Examples of OpenVINO™ Python API code](./code_examples.md).
|
||||
|
||||
##### Adding new test-case in the correct place
|
||||
### Adding new test-case in the correct place
|
||||
Let's add a new test for OpenVINO™ Python API.
|
||||
|
||||
First, the test should confirm that the new pybind11-based class of `MyTensor` is behaving correctly. Navigate to tests folder and create a new file that describes tests within it. It should be along the lines of:
|
||||
First, the test should confirm that the new pybind11-based class of `MyTensor` is behaving correctly. Navigate to [tests folder](./../tests/test_runtime/) and create a new file `test_mytensor.py` that describes tests within it. Final path should be along the lines of:
|
||||
|
||||
tests/test_runtime/test_mytensor.py
|
||||
|
||||
@@ -102,7 +102,7 @@ Note that name of the file is connected to the class/module to be tested. This i
|
||||
tests/test_onnx <-- ONNX Frontend tests and validation
|
||||
tests/test_transformations <-- optimization passes for OV Models
|
||||
|
||||
##### Writing test itself
|
||||
### Writing of the test itself
|
||||
Let's add a test case for new class. Start with imports and simple test of the creation of a class:
|
||||
```python
|
||||
import pytest
|
||||
@@ -147,6 +147,11 @@ Notice that the test name is shared between cases. In a real-life pull request,
|
||||
* ... or create reference values during runtime. Always use a good, thrust-worthy library for that!
|
||||
* Re-use common parts of the code (like multiple lines that create helper object) and move them out to make tests easier to read.
|
||||
|
||||
###### Difference between *tests* and *tests_compatibility* directories
|
||||
### Difference between *tests* and *tests_compatibility* directories
|
||||
<!-- TO-DELETE when compatibility layer is no longer supported in the project -->
|
||||
Someone could notice two similar folders `tests` and `tests_compatibility`. First one is the desired place for all upcoming features and tests. Compatibility layer is only supported in specific cases and any updates to it should be explicitly approved by OpenVINO™ reviewers. Please do not duplicate tests in both directories if not necessary.
|
||||
Someone could notice two similar folders [`tests`](./../tests/) and [`tests_compatibility`](./../tests_compatibility/). First one is the desired place for all upcoming features and tests. Compatibility layer is only supported in specific cases and any updates to it should be explicitly approved by OpenVINO™ reviewers. Please do not duplicate tests in both directories if not necessary.
|
||||
|
||||
## See also
|
||||
* [OpenVINO™ README](../../../../README.md)
|
||||
* [OpenVINO™ bindings README](../../README.md)
|
||||
* [Developer documentation](../../../../docs/dev/index.md)
|
||||
|
||||
Reference in New Issue
Block a user