The auto-detection wasn't working because the
CLI was always setting PYTHON_VERSIONS environment variable to all supported
versions when --python-versions was not provided.
Changes:
- click_helpers.py: Return None instead of all versions when option not provided
- helpers.py: Only set PYTHON_VERSIONS env var when explicitly specified
- main.py: Update type hints to allow None for python_versions parameter
Now the flow works correctly:
1. No --python-versions CLI option -> python_versions = None
2. None passed to Docker -> no PYTHON_VERSIONS env var set
3. Container auto-detects installed versions using pyenv
4. Tests run only on detected versions
When no --python-versions is specified at
runtime, auto-detect which Python versions are actually installed in the
container instead of defaulting to all versions from python_versions.json.
This resolves the issue where:
1. Building with --python-versions=3.12 installs only Python 3.12
2. Running without --python-versions would try to test all versions from JSON
3. Tests would fail for non-installed versions (3.8, 3.9, 3.10, 3.11, 3.13)
Now the runtime logic:
1. Uses `pyenv versions --bare` to detect installed versions
2. Maps full versions (3.12.9) to short versions (3.12)
3. Uses detected versions as default for testing
4. Falls back to JSON versions if detection fails
Example behavior:
- Build: --python-versions=3.12 (installs only 3.12.9)
- Runtime: auto-detects 3.12, tests only that version
- Explicit: --python-versions=3.11,3.12 still works as before
Adds a new CLI option to control test execution behavior when failures occur:
- Without --stop-on-error (default): Continue running all tests even if some fail
- With --stop-on-error: Stop execution on first test failure (previous behavior)
This enables comprehensive debugging by running both opm-common and omp-simulators
tests in a single run, showing all failures at once rather than stopping at the first one.
Changes:
- Add --stop-on-error flag to run-docker-image command
- Pass STOP_ON_ERROR environment variable to Docker container
- Update entrypoint.py to handle continue-on-error behavior
- Modify test functions to check return codes and continue when appropriate
- Maintain backward compatibility with existing error handling
This enhancement allows specifying which Python versions to install during
Docker image builds, significantly reducing build times when only testing
specific Python versions during debugging.
Changes:
- Add --python-versions option to build-docker-image command
- Pass BUILD_PYTHON_VERSIONS build argument to Docker
- Update install_python_versions.py to respect BUILD_PYTHON_VERSIONS env var
- Validate requested versions against supported versions in python_versions.json
- Maintain backward compatibility (installs all versions if not specified)
Usage examples:
# Build with only Python 3.12 (much faster for debugging)
opm-wheels build-docker-image --docker-os="ubuntu-24.04" --python-versions="3.12"
# Build with multiple specific versions
omp-wheels build-docker-image --python-versions="3.11,3.12"
# Build with all versions (original behavior)
opm-wheels build-docker-image
Benefits:
- Reduces build time from ~7 minutes to ~2 minutes for single Python version
- Enables faster iteration during debugging of Docker-specific issues
- Particularly valuable when combined with --test-cases-* for focused testing
This enhancement adds the ability to run specific test cases in the Docker
testing infrastructure, enabling more focused debugging and faster iteration
when investigating test failures.
Changes:
- Add --test-cases-common and --test-cases-simulators CLI options to opm-wheels
- Pass test selection as environment variables to Docker container
- Update entrypoint.py to support selective test execution:
* Run only opm-common tests when --test-cases-common is specified
* Run only opm-simulators tests when --test-cases-simulators is specified
* Maintain backward compatibility (run all tests if no selection specified)
- Support both host-tests-dir and cloned repo testing modes
Usage examples:
# Run only esmry tests from opm-common
opm-wheels run-docker-image --test-cases-common=esmry
# Run only basic tests from opm-simulators
opm-wheels run-docker-image --test-cases-simulators=basic
# Run specific tests from both suites
opm-wheels run-docker-image --test-cases-common=esmry --test-cases-simulators=mpi
This enables efficient debugging of Docker-specific test failures by isolating
problematic test cases without running the full test suite.
Move simulator instantiation to factory functions that automatically
disable async ECL output. This prevents future contributors from
forgetting the --enable-async-ecl-output=false flag.
- Add create_black_oil_simulator/create_gas_water_simulator to pytest_common
- Define ENABLE_ASYNC_ECL_OUTPUT_FLAG constant
- Update all tests to use factory functions
- No functional changes, pure refactoring
Add --enable-async-ecl-output=false to all Python simulator tests to prevent
race condition between async file writing tasklets and pushd context manager.
Problem:
- Tests use pushd() to change directory before running simulations
- Simulator dispatches async tasklets to write ECL output files
- pushd context exits before async writes complete
- Tasklets write files in wrong directory (build/python/ instead of test_data/)
- Causes confusing error messages and misplaced output files
Solution:
- Disable async ECL output during tests for deterministic behavior
- Files now created in correct locations within test_data/ directories
- Eliminates "Can not open EclFile" errors in logs
- Makes test execution predictable and synchronous
- Add 'BlackOil' as 5th parameter to generate_docstring_hpp.py
- Prepares for template-based docstring generation
- Will work with updated opm-common generate_docstring_hpp.py script
- Create unified template structure with 'simulators', 'constructors', and 'common_methods' sections
- Add support for both BlackOil and GasWater simulators
- Use {{name}} and {{class}} placeholders for template expansion
- Eliminates need for duplicate JSON files
Extend the Python blackoil simulator to support two-phase gas-water
simulations by extracting common functionality into a template base
class and implementing proper module separation.
- Extract PyBaseSimulator<TypeTag> template base class containing all
common simulator functionality (20+ methods including advance, run,
step, property access, and MPI handling)
- Add PyGasWaterSimulator for two-phase gas-water simulations using
FlowGasWaterProblem TypeTag with oil phase disabled
- Split Python modules to avoid ODR violations:
- BlackOil.so for PyBlackOilSimulator (FlowProblemTPFA)
- GasWater.so for PyGasWaterSimulator (FlowGasWaterProblem)
- Implement proper template instantiation control with explicit
instantiations per module
- Eliminate ~400 lines of code duplication between simulators
- Clean inheritance hierarchy: derived classes contain only constructors
- Type-safe specialization through OPM's TypeTag property system
- Separate docstring generation for each simulator type
- Independent build targets with shared infrastructure
- Update CMakeLists.txt to support multiple pybind11 modules
- Add docstrings_gw_simulators.json for gas-water simulator documentation
- Remove centralized Pybind11Exporter.cpp in favor of distributed exports
- Implement proper build dependencies and custom targets
This establishes a scalable foundation for extending OPM's Python
bindings to support additional simulator types (thermal, polymer,
solvent) while maintaining type safety and avoiding symbol conflicts.
Files changed: 21 files, 915 insertions, 428 deletions
- Export PIP_ROOT_USER_ACTION=ignore to silence root-user warning in manylinux.
- Remove pytest-runner from build-time pip installs; no longer needed after
removing setup_requires.
- Replace direct setup.py invocations with: python -m build --sdist --wheel
- Add comments explaining motivation (setuptools deprecations, build isolation)
and why we drop explicit --plat-name/--python-tag (auditwheel/env handles).
- Ensure the build frontend is present per-interpreter by installing 'build'.
Also add build-backend to pyproject.toml with notes and TODO for full modernization.
- Drop legacy setup_requires and test_suite to stop
setuptools _DeprecatedInstaller / fetch_build_eggs and
unknown distribution option warnings.
- Add inline comment explaining rationale and PEP 517 context.
lechevaa
Arne Morten Kvarving
David Landa-Marbán
Svenn Tveit
Håkon Hægland