Docker Build System for OPM Python Wheels
This directory contains the Docker-based build system for generating PyPI-compatible wheels for the OPM Python bindings.
Directory Structure
docker/
├── Dockerfile # Main Docker build configuration
├── run-docker-build.sh # User-facing helper script for building wheels
├── python_versions.json # Python version definitions
├── README.md # This file (documentation)
├── scripts/ # Internal build scripts
│ ├── generate-pypi-package.sh # Wheel generation script
│ ├── setup-docker-image.sh # Docker environment setup
│ ├── build-boost.sh # Boost dependency build
│ ├── build-dune.sh # DUNE dependency build
│ ├── build-deps.sh # Additional dependencies
│ └── read_python_config.py # Python version config reader
├── test_wheels/ # Wheel testing framework
└── [output directories] # Generated by --output parameter (user-specified names)
Building Wheels
Use the provided helper script for simplified building with automatic logging:
# Static build with wheel extraction
./docker/run-docker-build.sh static --output wheelhouse
# Creates: docker/wheelhouse/ with wheels
# Shared libraries build
./docker/run-docker-build.sh shared --libtype shared --output wheelhouse-shared
# Creates: docker/wheelhouse-shared/ with wheels
# Build specific Python versions
./docker/run-docker-build.sh py312 --python-versions "3.12" --output test-wheels
# Creates: docker/test-wheels/ with wheels
# Debug build with detailed output (no wheel extraction)
./docker/run-docker-build.sh debug --libtype shared --no-buildx --jobs 8
# Absolute path output (advanced usage)
./docker/run-docker-build.sh release --output /tmp/release-wheels
# Creates: /tmp/release-wheels/ with wheels
# Show all options
./docker/run-docker-build.sh --help
# Test a PR from opm-simulators
./docker/run-docker-build.sh pr-test --version-simulators "pull/1234/merge" --output pr-wheels
# Test multiple PRs together
./docker/run-docker-build.sh multi-pr \
--version-common "pull/567/merge" \
--version-simulators "pull/1234/merge" \
--output multi-pr-wheels
# Use specific branches/tags for each repository
./docker/run-docker-build.sh release \
--version-common "release/2024.10/final" \
--version-grid "release/2024.10/final" \
--version-simulators "release/2024.10/final" \
--output release-wheels
# Test PR #6075 with custom build targets
./docker/run-docker-build.sh pr6075 \
--version-simulators "pull/6075/merge" \
--target-simulators "GasWater BlackOil" \
--output pr6075-wheels
Helper script features:
- Works from any directory (auto-navigates to
python/) - Smart output paths: Relative paths create directories in
docker/, absolute paths used as-is - Automatic timestamped logging to
docker/build-<tag>-<timestamp>.log - Reads Python versions from
docker/python_versions.jsonby default - Input validation and helpful error messages
- Progress indication and build summaries
Testing with Different Repository Versions
The build system supports testing different versions, branches, or pull requests from each OPM repository independently:
# Test a PR from opm-simulators against master branches
./docker/run-docker-build.sh pr-test \
--version-simulators "pull/1234/merge" \
--output pr-wheels
# Test compatibility between multiple PRs
./docker/run-docker-build.sh multi-pr \
--version-common "pull/567/merge" \
--version-grid "pull/890/merge" \
--version-simulators "pull/1234/merge" \
--output multi-pr-test
# Use specific release tags
./docker/run-docker-build.sh release \
--version-common "release/2024.10/final" \
--version-grid "release/2024.10/final" \
--version-simulators "release/2024.10/final" \
--output release-2024.10
The version parameters accept any valid Git reference:
- Branch names:
master,release/2024.10/final - Tags:
v2024.10,release-2024-10 - PR references:
pull/1234/merge(GitHub PR format) - Commit hashes:
abc123def456
The target parameters accept space-separated CMake target names:
- Default targets:
opmcommon_python(common),simulators(simulators) - PR #6075 targets:
GasWater BlackOil(split simulators module) - Multiple targets:
"target1 target2 target3"
Testing Wheels
See test_wheels/scripts/README.md for documentation on testing the generated wheels. The test framework installs the generated wheels in a clean Docker container and runs the complete test suites from both opm-common and opm-simulators to verify functionality.
For debugging failed tests, see the Debugging section for:
- Python version selection to speed up builds
- Focused test case execution for isolation
- Continue-on-error mode to see all failures at once
- Host test directory integration for faster iterations
Python Version Configuration
Python versions are configured in python_versions.json as the single source of truth.
Currently supported versions: Python 3.8, 3.9, 3.10, 3.11, 3.12, 3.13
Manylinux platform: manylinux_2_28_x86_64 (configured in docker/python_versions.json)
Adding a New Python Version
To add support for a new Python version (e.g., Python 3.14):
-
Edit
python_versions.json:"supported_versions": { "3.8": "3.8.19", "3.9": "3.9.21", "3.10": "3.10.16", "3.11": "3.11.11", "3.12": "3.12.9", "3.13": "3.13.1", "3.14": "3.14.0" // Add new version with exact patch version } -
Run the sync script:
./docker/scripts/sync_versions.shThis automatically updates:
docker/Dockerfile(ARG python_versions default)docker/test_wheels/scripts/README.md(supported versions list)python/setup.py.in(python_requires minimum version)- Dynamic files read the JSON directly
-
Manual update required: Edit
docker/scripts/generate-pypi-package.shlines 33-38 to add the new version mapping:all_python_versions[3.14]="cp314-cp314:/opt/python/cp314-cp314/bin/python" -
Test the new version:
./docker/run-docker-build.sh test --python-versions "3.14"
Removing/Deprecating a Python Version
To remove support for a Python version (e.g., Python 3.8):
- Edit
python_versions.json: Remove the version fromsupported_versions - Run the sync script:
./docker/scripts/sync_versions.sh - Manual cleanup: Remove the version mapping from
generate-pypi-package.sh - Test: Ensure builds work without the removed version
Changing Manylinux Platform Version
To update the manylinux platform (e.g., from manylinux_2_28 to manylinux_2_34):
-
Edit
python_versions.json:"manylinux_platform": "manylinux_2_34_x86_64" -
Run the sync script:
./docker/scripts/sync_versions.shThis automatically updates:
docker/Dockerfile(FROM line with correct manylinux Docker image)
-
Manual updates required:
- docker/Dockerfile line 16: Update the compatibility comment
- Example: Change "Debian 10+, Ubuntu 18.10+, CentOS/RHEL 8+" to match manylinux_2_34 requirements
- docker/README.md line 166: Update the platform compatibility description
- Verify and update the distribution compatibility information
- docker/Dockerfile line 16: Update the compatibility comment
-
Test the new platform:
./docker/run-docker-build.sh test
Note: Each manylinux version has specific glibc requirements that determine distribution compatibility. Always verify the compatibility information when changing versions.
Maintenance Workflow
The JSON file controls version support across all build components:
Auto-synced locations (updated by sync script):
docker/Dockerfile- Default ARG python_versionsdocker/test_wheels/scripts/README.md- Supported versions documentationpython/setup.py.in- Package python_requires minimum versiondocker/test_wheels/scripts/src/opm_wheels/constants.py- Python version enum (dynamic)docker/test_wheels/dockerfiles/common/install_python_versions.py- Installation script (dynamic)
Compatibility note (Windows checkout): directories under docker/test_wheels/dockerfiles
use hyphens instead of colons. Example mappings: ubuntu:22.04 → ubuntu-22.04,
ubuntu:24.04 → ubuntu-24.04, debian:11 → debian-11. The opm-wheels CLI
accepts both forms via --docker-os, but uses the hyphenated form for filesystem
paths and image tags.
Manual locations (requires manual update):
docker/scripts/generate-pypi-package.sh- Version to wheel tag mappings (lines 33-38)
Build Arguments
libtype: "static" (default) or "shared"python_versions: Comma-separated list of Python versions (default: "3.8,3.9,3.10,3.11,3.12,3.13")build_jobs: Number of parallel build jobs (default: 4)version_common: Git reference for opm-common (default: "master")version_grid: Git reference for opm-grid (default: "master")version_simulators: Git reference for opm-simulators (default: "master")target_common: CMake build targets for opm-common (default: "opmcommon_python")target_simulators: CMake build targets for opm-simulators (default: "simulators")version_tag: Version tag for wheels (optional)
Output
Built wheels follow the PEP 517 naming convention:
opm-{version}-{python}-{abi}-{platform}.whlopm_simulators-{version}-{python}-{abi}-{platform}.whl
Platform tags use manylinux_2_28 for broad compatibility (Debian 10+, Ubuntu 18.10+, CentOS/RHEL 8+).