mirror of
https://github.com/OPM/opm-simulators.git
synced 2026-10-24 22:57:42 -05:00
Håkon Hægland
6e2503cac8
Reorganize Python folder: separate Docker/PyPI from CMake/PyBind11 workflows and implement comprehensive build automation This commit represents a comprehensive restructuring and enhancement of the OPM Python build system, addressing maintainability, consistency, and usability issues through architectural improvements and automation. FILES CHANGED ============= Refactored: - python/Dockerfile -> python/docker/Dockerfile - python/generate-pypi-package.sh -> python/docker/scripts/generate-pypi-package.sh - python/setup-docker-image.sh -> python/docker/scripts/setup-docker-image.sh - python/build-deps.sh -> python/docker/scripts/build-boost.sh - python/build-deps.sh -> python/docker/scripts/build-dune.sh Added (24 files): - python/docker/python_versions.json (single source of truth) - python/docker/scripts/sync_versions.sh - python/docker/scripts/read_python_config.py - python/docker/run-docker-build.sh - python/docker/README.md - python/docker/test_wheels/* (complete testing framework) Modified: - python/setup.py.in - .gitignore MAJOR CHANGES ============= Architecture Restructure ------------------------ - Separated build workflows: Clearly distinguished Docker/PyPI wheel generation from CMake/PyBind11 compilation - Moved Docker-specific files: Relocated docker build system to appropriate locations - Streamlined organization: Created logical separation between different build contexts Single Source of Truth Configuration ------------------------------------- - Centralized Python version management: Created python_versions.json as authoritative configuration - Automated synchronization: Built sync_versions.sh script to maintain consistency across all components - Dynamic configuration: Components now read from JSON rather than hardcoded values Enhanced Docker Build System ---------------------------- - Multi-OS support: Added Ubuntu 22.04, Ubuntu 24.04, and Debian 11 Dockerfiles - Improved caching: Split dependencies into separate Docker layers for better build performance - Fixed PyPI wheel generation: Added LD_LIBRARY_PATH setup for proper library bundling, corrected auditwheel wheel copying from wheelhouse/ instead of dist/ - Optimized Docker layer structure: Delayed source code copying until final stage, preventing dependency rebuilds on source changes and reducing iteration time from 30+ to <5 minutes during development Build Automation & Tooling -------------------------- - Comprehensive wheel testing: Created opm-wheels CLI tool for building and testing Docker images - Automated validation: Integrated Python version validation and error handling - Enhanced scripts: Improved generate-pypi-package.sh with better error handling and version management Testing Framework ----------------- - Docker-based testing: Complete framework for testing wheels across multiple Python versions and OS configurations - Automated validation: Wheel installation and import testing with comprehensive error reporting - Multi-version support: Parallel testing across Python 3.8-3.13 - Platform verification: Validates manylinux wheels install and function correctly across Ubuntu 22.04/24.04 and Debian 11 Documentation & Usability ------------------------- - Comprehensive maintenance guides: Detailed instructions for adding/removing Python versions - Clear workflows: Step-by-step procedures for build system maintenance - Improved CLI experience: Better error messages, path handling, and user guidance - Updated documentation: Enhanced README files with practical examples and troubleshooting IMPACT ====== This restructure transforms the Python build system from a collection of loosely coupled scripts into a cohesive, maintainable, and reliable build automation system. The single source of truth configuration eliminates the primary source of maintenance overhead, while the enhanced testing framework provides confidence in the quality of generated wheels. The changes enable the OPM project to more easily support new Python versions, maintain consistency across platforms, and provide a better developer experience for both maintainers and users.
49 lines
605 B
Plaintext
49 lines
605 B
Plaintext
# Editor autosave files
|
|
*~
|
|
*.swp
|
|
|
|
# Compiled Object files
|
|
*.slo
|
|
*.lo
|
|
*.o
|
|
|
|
# Compiled Dynamic libraries
|
|
*.so
|
|
*.dylib
|
|
|
|
# Compiled Static libraries
|
|
*.lai
|
|
*.la
|
|
*.a
|
|
|
|
# Compiled executables
|
|
a.out
|
|
find_zero
|
|
sim_2p_incomp_adfi
|
|
sim_simple
|
|
test_ad
|
|
test_block
|
|
test_impestpfa_ad
|
|
test_syntax
|
|
test_vec
|
|
|
|
# Build directory in source
|
|
build/
|
|
|
|
# Mac OS X debug info
|
|
*.dSYM
|
|
|
|
# emacs directory setting:
|
|
.dir-locals.el
|
|
|
|
# Python sphinx build
|
|
python/sphinx_docs/docs/_build/
|
|
|
|
# Python cache directories
|
|
**/__pycache__/
|
|
|
|
# Python docker files
|
|
/python/docker/wheelhouse*
|
|
/python/docker/build*.log
|
|
/python/docker/extracted_wheels/
|