IntenseWebs/openvino
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update nGraph Python API build instructions (#2610)

Browse Source
This commit is contained in:
Michał Karzyński 2020-10-15 22:15:33 +02:00 committed by GitHub
parent 6ae28bdd62
commit db85069713
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 140 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

193
ngraph/python/BUILDING.md
View File

@ -1,76 +1,158 @@
# Building the Python API for nGraph
## Building nGraph Python Wheels
You can build the nGraph Python API from sources by following instructions in this document. A Python wheel is a
portable package which will allow you to install nGraph in your Python distribution, or dedicated virtual environment.
If you want to try a newer version of nGraph's Python API than is available
from PyPI, you can build the latest version from source code. This process is
very similar to what is outlined in our [ngraph_build] instructions with two
important differences:
## Build nGraph Python Wheels on Linux or MacOS
1. You must specify: `-DNGRAPH_PYTHON_BUILD_ENABLE=ON` and `-DNGRAPH_ONNX_IMPORT_ENABLE=ON`
when running `cmake`.
### Prerequisites
2. Instead of running `make`, use the command `make python_wheel`.
In order to build the nGraph Python wheel, you will need to install a few packages.
`$ cmake ../ -DNGRAPH_PYTHON_BUILD_ENABLE=ON -DNGRAPH_ONNX_IMPORT_ENABLE=ON`
On Ubuntu 20.04 LTS you can use the following instructions to install the required packages, including Python and Cython.
`$ make python_wheel`
apt install git wget build-essential cmake
apt install python3 python3-dev python3-pip python3-virtualenv python-is-python3
After this procedure completes, the `ngraph/build/python/dist` directory should
contain the Python packages of the version you cloned. For example, if you
checked out and built `0.21` for Python 3.7, you might see something like:
You can see a full working example on an Ubuntu environment used in our continuous environment in this
[Dockerfile](https://github.com/openvinotoolkit/openvino/blob/master/.ci/openvino-onnx/Dockerfile).
$ ls python/dist/
ngraph-core-0.21.0rc0.tar.gz
ngraph_core-0.21.0rc0-cp37-cp37m-linux_x86_64.whl
On MacOS you can use [Homebrew](https://brew.sh) to install required packages:
## Building nGraph Python Wheels on Windows
brew install cmake
brew install automake
brew install libtool
brew install python3
The build process on Windows consists of 3 steps:
Install Cython in the Python installation, or virtualenv which you are planning to use:
pip3 install cython
### Configure, build and install OpenVINO
The following section will illustrate how to download, build and install OpenVINO in a workspace directory specified
by the `${MY_OPENVINO_BASEDIR}` variable. Let's start by setting this variable to a directory of your choice:
export MY_OPENVINO_BASEDIR=/path/to/my/workspace
Now we can clone OpenVINO, configure it using `cmake` and build using `make`. Please note that we're disabling
the building of a few modules by setting the `ENABLE_*` flag to `OFF`. In order to build the OpenVINO Python APIs
set the mentioned flags to `ON`. Note the `CMAKE_INSTALL_PREFIX`, which defaults to `/usr/local/` if not set.
cd "${MY_OPENVINO_BASEDIR}"
git clone --recursive https://github.com/openvinotoolkit/openvino.git
mkdir openvino/build
cd openvino/build
cmake .. \
-DENABLE_CLDNN=OFF \
-DENABLE_OPENCV=OFF \
-DENABLE_VPU=OFF \
-DENABLE_PYTHON=ON \
-DNGRAPH_PYTHON_BUILD_ENABLE=ON \
-DNGRAPH_ONNX_IMPORT_ENABLE=ON \
-DCMAKE_INSTALL_PREFIX="${MY_OPENVINO_BASEDIR}/openvino_dist"
make -j 4
make install
If you would like to use a specific version of Python, or use a virtual environment you can set the `PYTHON_EXECUTABLE`
variable. Examples:
```
-DPYTHON_EXECUTABLE=/path/to/venv/bin/python
-DPYTHON_EXECUTABLE=$(which python3.8)
```
### Build nGraph Python wheel
When OpenVINO is built and installed, we can build the Python wheel by issuing the following command:
make python_wheel
Once completed, the wheel package should be located under the following path:
$ ls "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl
You can now install the wheel in your Python environment:
cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
pip3 install ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl
#### What does `make python_wheel` do?
The `python_wheel` target automates a few steps, required to build the wheel package. You can recreate the process
manually by issuing the following commands:
cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python"
git clone --branch v2.5.0 https://github.com/pybind/pybind11.git
export NGRAPH_CPP_BUILD_PATH="${MY_OPENVINO_BASEDIR}/openvino_dist"
python3 setup.py bdist_wheel
## Build nGraph Python Wheels on Windows
### Prerequisites
In order to build OpenVINO and the nGraph Python wheel on Windows, you will need to install Visual Studio and Python.
Once Python is installed, you will also need to install Cython using `pip install cython`.
### Configure, build and install OpenVINO
Configure the build with a `cmake` invocation similar to the following. Note that you'll need to set the `-G` and
`-DCMAKE_CXX_COMPILER` to match the version and location of your Visual Studio installation.
```
cmake .. ^
-G"Visual Studio 16 2019" ^
-DCMAKE_BUILD_TYPE=Release ^
-DCMAKE_INSTALL_PREFIX="C:\temporary_install_dir" ^
-DENABLE_CLDNN=OFF ^
-DENABLE_OPENCV=OFF ^
-DENABLE_VPU=OFF ^
-DNGRAPH_PYTHON_BUILD_ENABLE=ON ^
-DNGRAPH_ONNX_IMPORT_ENABLE=ON ^
-DENABLE_PYTHON=ON ^
-DCMAKE_CXX_COMPILER="C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\Hostx64\x64"
```
1. Configure the build with the following `cmake` invocation:
~~~~
cmake ..
-G"Visual Studio 15 2017 Win64"
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX="C:\temporary_install_dir"
-DNGRAPH_PYTHON_BUILD_ENABLE=TRUE
-DNGRAPH_ONNX_IMPORT_ENABLE=TRUE
-DCMAKE_CXX_COMPILER="C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Tools\MSVC\14.16.27023\bin\Hostx64\x64"
~~~~
There are a couple of things to notice here. One is that the full path to the x64 version of
MSVC compiler has to be specified. This is because DNNL requires a 64-bit version and cmake may
fail to detect it correctly.
The other equally important thing to note is that the temporary directory where the build is to be installed can be specified.
If the installation directory is not specified, the default location is `C:\Program Files\OpenVINO`.
This examples uses `C:\temporary_install_dir` however, a subdirectory of `openvino\build` works as well.
The final Python wheel will contain the contents of this temporary directory so it's very important to set it.
The final Python wheel will contain the contents of this temporary directory so it's important to set it.
To specify an exact Python version, use the following options:
~~~~
-DPYTHON_EXECUTABLE="C:\Program Files\Python37\python.exe"
-DPYTHON_LIBRARY="C:\Program Files\Python37\libs\python37.lib"
-DPYTHON_INCLUDE_DIR="C:\Program Files\Python37\include"
~~~~
If you want to specify an exact Python version, use the following options:
```
-DPYTHON_EXECUTABLE="C:\Program Files\Python37\python.exe" ^
-DPYTHON_LIBRARY="C:\Program Files\Python37\libs\python37.lib" ^
-DPYTHON_INCLUDE_DIR="C:\Program Files\Python37\include" ^
```
2. Build the `install` target:
In order to build and install OpenVINO, build the `install` target:
`cmake --build . --target install --config Release -j 8`
cmake --build . --target install --config Release -j 8
In this step nGraph will be built and installed to the temporary directory specified above. You can
In this step OpenVINO will be built and installed to the directory specified above. You can
adjust the number of threads used in the building process to your machine's capabilities.
3. Build the Python wheel itself:
Build the Python wheel package:
`cmake --build . --target python_wheel --config Release -j 8`
cmake --build . --target python_wheel --config Release -j 8
The final wheel should be located in `ngraph\python\dist` directory.
4. Configure the environment for the Inference Engine Python API:
dir openvino\ngraph\python\dist\
10/09/2020 04:06 PM 4,010,943 ngraph_core-0.0.0-cp38-cp38-win_amd64.whl
`call <CMAKE_INSTALL_PREFIX>\bin\setupvars.bat`
**NOTE**: Skip this step if you want to use the nGraph Wheel by itself. This step is required for most usage scenarios, like running unit tests.
## Run tests
### Using a virtualenv (optional)
@ -80,21 +162,26 @@ You may wish to use a virutualenv for your installation.
$ source venv/bin/activate
(venv) $
### Installing the wheel
### Install the nGraph wheel and other requirements
You may wish to use a virutualenv for your installation.
(venv) $ cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python"
(venv) $ pip3 install -r requirements.txt
(venv) $ pip3 install -r requirements_test.txt
(venv) $ pip3 install dist/ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl
(venv) $ pip install ngraph/build/python/dist/ngraph_core-0.21.0rc0-cp37-cp37m-linux_x86_64.whl
### Run tests
## Running tests
You should now be able to run tests.
Unit tests require additional packages be installed:
You may need to run the `setupvars` script from OpenVINO to set paths to OpenVINO components.
(venv) $ cd ngraph/python
(venv) $ pip install -r test_requirements.txt
source ${MY_OPENVINO_BASEDIR}/openvino/scripts/setupvars/setupvars.sh
Then run tests:
The minimum requirement is to set the `PYTHONPATH` to include the Inference Engine Python API:
(venv) $ pytest tests
export PYTHONPATH="${MY_OPENVINO_BASEDIR}/openvino/bin/intel64/Release/lib/python_api/python3.8/":${PYTHONPATH}
pytest tests/
[ngraph_build]: http://ngraph.nervanasys.com/docs/latest/buildlb.html
Now you can run tests using `pytest`:
pytest tests