mirror of
https://github.com/OPM/ResInsight.git
synced 2026-10-25 14:40:26 -05:00
Magne Sjaastad
3052d3d554
* Select RHEL8 vcpkg-configuration.json via CMake instead of renaming the tracked file RHEL8 CI jobs previously ran 'cp vcpkg-configuration-rhel8.json vcpkg-configuration.json' before configuring, temporarily overwriting the tracked default configuration in the checkout. The vcpkg toolchain file only defaults VCPKG_MANIFEST_DIR to CMAKE_SOURCE_DIR when it is not already defined, and reads it while processing project(), so setting VCPKG_MANIFEST_DIR earlier -- before project() -- lets vcpkg read a different vcpkg-configuration.json without ever touching the tracked file. Add a RESINSIGHT_VCPKG_PROFILE cache variable, implemented in the new cmake/ResInsightVcpkgProfile.cmake (included from the root CMakeLists.txt before project(), which is required for VCPKG_MANIFEST_DIR to take effect). When set (e.g. to 'rhel8'), it copies vcpkg.json and vcpkg-configuration-<profile>.json into a profile-specific directory under the build tree and points VCPKG_MANIFEST_DIR there. The copy is regenerated on every configure, so it can never drift from the root vcpkg.json. overlay-ports/overlay-triplets entries (e.g. 'ThirdParty/vcpkg-overlay-ports') are resolved by vcpkg relative to the directory vcpkg-configuration.json lives in, so repo-relative 'ThirdParty/...' paths are rewritten to absolute paths right after the copy is generated, keeping them valid from the new location. Update the RHEL8 workflows and Dockerfile.rhel8 build-warmup stage to pass -DRESINSIGHT_VCPKG_PROFILE=rhel8 instead of the cp step, and update vcpkg-readme.md to describe the new mechanism. * #14733 Use x64-linux-release vcpkg triplet explicitly in RHEL8 CI The RHEL8 CI workflows and the Docker image's warmup/vcpkg-cache stages did not set VCPKG_TARGET_TRIPLET/VCPKG_HOST_TRIPLET, so vcpkg fell back to its plain x64-linux default triplet, which builds both debug and release variants of every dependency even though the CI build is Release-only. Set both to x64-linux-release consistently across the image's vcpkg install, the warmup CMake configure, and the two consuming workflows so the baked binary cache and the CI configure use matching ABI hashes and skip the unused debug builds.
216 lines
10 KiB
Docker
216 lines
10 KiB
Docker
# syntax=docker/dockerfile:1
|
|
|
|
# =============================================================================
|
|
# RHEL8 (Rocky Linux 8) CI image for ResInsight.
|
|
#
|
|
# Bakes in:
|
|
# * The toolchain (gcc-toolset-14, libstdc++exp, Qt)
|
|
# * A prebuilt vcpkg binary cache (populated from the manifest)
|
|
# * A warm ResInsight buildcache populated by a full ResInsight-tests compile
|
|
# at the fixed path /src -> /src/cmakebuild.
|
|
#
|
|
# The CI workflow runs inside this image, checks out into /src (path must
|
|
# match the warmup so buildcache hashes line up), and only recompiles
|
|
# translation units that actually changed.
|
|
#
|
|
# Rebuilt nightly by .github/workflows/build-rhel8-image.yml.
|
|
# =============================================================================
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Stage 1: base toolchain
|
|
# -----------------------------------------------------------------------------
|
|
FROM rockylinux:8 AS base
|
|
|
|
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
|
|
|
|
# Base development tools and libraries.
|
|
RUN dnf install -y epel-release \
|
|
&& dnf config-manager --set-enabled powertools \
|
|
&& dnf install -y \
|
|
gcc gcc-c++ make cmake ninja-build git curl zip unzip tar \
|
|
pkgconfig perl which xz flex bison \
|
|
python39 python39-devel python39-pip \
|
|
mesa-libGL-devel mesa-libGLU-devel mesa-libEGL-devel \
|
|
libxkbcommon-devel libxkbcommon-x11-devel \
|
|
xcb-util-keysyms-devel xcb-util-image-devel xcb-util-wm-devel \
|
|
xcb-util-renderutil-devel \
|
|
fontconfig-devel freetype-devel \
|
|
libX11-devel libXext-devel libXrender-devel \
|
|
&& dnf install -y \
|
|
gcc-toolset-14 gcc-toolset-14-gcc-c++ \
|
|
gcc-toolset-14-libatomic-devel gcc-toolset-14-libstdc++-devel \
|
|
&& dnf clean all
|
|
|
|
# Build libstdc++exp from GCC source to match gcc-toolset-14.
|
|
RUN source /opt/rh/gcc-toolset-14/enable \
|
|
&& GCC_FULL_VERSION=$(gcc -dumpfullversion) \
|
|
&& GCC_MAJOR_MINOR=$(echo "$GCC_FULL_VERSION" | cut -d. -f1,2) \
|
|
&& GCC_VERSION="${GCC_MAJOR_MINOR}.0" \
|
|
&& cd /tmp \
|
|
&& curl -LO "https://ftp.gnu.org/gnu/gcc/gcc-${GCC_VERSION}/gcc-${GCC_VERSION}.tar.xz" \
|
|
&& tar xf "gcc-${GCC_VERSION}.tar.xz" \
|
|
&& mkdir -p "gcc-${GCC_VERSION}/build-libstdcxx" \
|
|
&& cd "gcc-${GCC_VERSION}/build-libstdcxx" \
|
|
&& ../libstdc++-v3/configure \
|
|
--prefix=/opt/libstdcxx-exp \
|
|
--disable-multilib \
|
|
--disable-libstdcxx-pch \
|
|
--with-gxx-include-dir="/opt/rh/gcc-toolset-14/root/usr/include/c++/${GCC_VERSION}" \
|
|
&& { make -k -j"$(nproc)" || true; } \
|
|
&& test -f src/experimental/.libs/libstdc++exp.a \
|
|
&& mkdir -p /opt/libstdcxx-exp/lib \
|
|
&& cp src/experimental/.libs/libstdc++exp.a /opt/libstdcxx-exp/lib/ \
|
|
&& rm -rf /tmp/gcc-*
|
|
|
|
# Python and Qt.
|
|
ARG QT_VERSION=6.6.3
|
|
RUN alternatives --set python3 /usr/bin/python3.9 \
|
|
&& python3 -m pip install --upgrade pip \
|
|
&& python3 -m pip install grpcio-tools aqtinstall \
|
|
&& python3 -m aqt install-qt linux desktop "${QT_VERSION}" gcc_64 \
|
|
-O /opt/Qt --modules qtnetworkauth
|
|
|
|
ENV Qt6_DIR=/opt/Qt/${QT_VERSION}/gcc_64 \
|
|
CMAKE_PREFIX_PATH=/opt/Qt/${QT_VERSION}/gcc_64 \
|
|
PATH=/opt/Qt/${QT_VERSION}/gcc_64/bin:${PATH}
|
|
|
|
# buildcache (pinned, built from source). When this binary is on PATH at CMake
|
|
# configure time, the project's top-level CMakeLists.txt auto-detects it via
|
|
# find_program() and wires CMAKE_CXX_COMPILER_LAUNCHER. No further CMake
|
|
# plumbing is needed.
|
|
#
|
|
# Built from source rather than downloaded because the upstream
|
|
# `buildcache-linux.tar.gz` release binary is linked against glibc 2.29+, and
|
|
# Rocky Linux 8 ships glibc 2.28 -- the binary loads but crashes the first
|
|
# time the dynamic linker touches libm. Compiling here links against the
|
|
# image's own glibc and is portable (~1-2 minutes).
|
|
# Source from GitLab -- the canonical repo. The GitHub mirror is stale (last
|
|
# tag v0.28.4, 2024-03), while GitLab has progressed to v0.33.0 with the GCC
|
|
# 14 transitive-include fixes that v0.28.x lacks.
|
|
ARG BUILDCACHE_VERSION=v0.33.0
|
|
RUN source /opt/rh/gcc-toolset-14/enable \
|
|
&& git clone --depth 1 --branch "${BUILDCACHE_VERSION}" \
|
|
https://gitlab.com/bits-n-bites/buildcache.git /tmp/buildcache-src \
|
|
&& cmake -S /tmp/buildcache-src/src -B /tmp/buildcache-build -G Ninja \
|
|
-DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local \
|
|
&& cmake --build /tmp/buildcache-build --target install \
|
|
&& rm -rf /tmp/buildcache-src /tmp/buildcache-build \
|
|
&& buildcache --version
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Stage 2: populate the vcpkg binary cache
|
|
# -----------------------------------------------------------------------------
|
|
FROM base AS vcpkg-builder
|
|
|
|
ENV VCPKG_DEFAULT_BINARY_CACHE=/opt/vcpkg-cache
|
|
|
|
# Copy only the inputs that determine the dependency set instead of the whole
|
|
# source tree. This keys the expensive vcpkg compile below on just these
|
|
# paths, so the layer stays valid in the BuildKit registry cache across
|
|
# ordinary source commits -- nightly rebuilds reuse the previously compiled
|
|
# vcpkg binaries instead of recompiling every port.
|
|
COPY ThirdParty/vcpkg /src/ThirdParty/vcpkg
|
|
COPY ThirdParty/vcpkg-overlay-ports /src/ThirdParty/vcpkg-overlay-ports
|
|
COPY vcpkg.json /src/vcpkg.json
|
|
COPY vcpkg-configuration-rhel8.json /src/vcpkg-configuration.json
|
|
|
|
# A direct manifest install compiles every dependency into
|
|
# VCPKG_DEFAULT_BINARY_CACHE. vcpkg-configuration-rhel8.json's overlay-ports
|
|
# entry ("ThirdParty/vcpkg-overlay-ports") is resolved relative to the
|
|
# directory vcpkg-configuration.json lives in, i.e. /src here, so that
|
|
# directory must be copied in too, not just ThirdParty/vcpkg. CC/CXX below
|
|
# match the ResInsight configure in the warmup stage and in CI, so the vcpkg
|
|
# ABI hashes match and the cache is reused there. x64-linux-release matches
|
|
# the triplet the CI workflows and the warmup stage below pass via
|
|
# VCPKG_TARGET_TRIPLET/VCPKG_HOST_TRIPLET -- using the plain (debug+release)
|
|
# x64-linux triplet here would populate the cache under different ABI hashes
|
|
# and none of it would be reused downstream.
|
|
#
|
|
# After the install, verify the cache contains a plausible number of entries.
|
|
# vcpkg's `files` binary provider stores one .zip per package under
|
|
# <cache>/<2-hex>/<full-hash>.zip. ResInsight depends on ~30+ packages, so a
|
|
# threshold of 20 catches the failure mode where the install dies early
|
|
# (e.g. after building only a handful of leaf deps) while staying robust to
|
|
# small manifest churn.
|
|
RUN git config --global --add safe.directory '*' \
|
|
&& mkdir -p "${VCPKG_DEFAULT_BINARY_CACHE}" \
|
|
&& source /opt/rh/gcc-toolset-14/enable \
|
|
&& ( cd /src/ThirdParty/vcpkg && ./bootstrap-vcpkg.sh ) \
|
|
&& export CC=/opt/rh/gcc-toolset-14/root/usr/bin/gcc \
|
|
&& export CXX=/opt/rh/gcc-toolset-14/root/usr/bin/g++ \
|
|
&& ( cd /src && ./ThirdParty/vcpkg/vcpkg install --triplet x64-linux-release --host-triplet x64-linux-release ) \
|
|
&& vcpkg_zip_count=$(find "${VCPKG_DEFAULT_BINARY_CACHE}" -type f -name '*.zip' | wc -l) \
|
|
&& echo "vcpkg binary cache entries: ${vcpkg_zip_count}" \
|
|
&& test "${vcpkg_zip_count}" -ge 20 \
|
|
&& rm -rf /src
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Stage 3: warm the ResInsight buildcache by performing a full compile at
|
|
# /src -> /src/cmakebuild. CI runs at the same paths to get cache
|
|
# hits (buildcache hashes the absolute source path).
|
|
# -----------------------------------------------------------------------------
|
|
FROM base AS build-warmup
|
|
|
|
# Cap at 4 GB. The baked cache from this stage is what CI reads, and CI also
|
|
# writes back to the same /opt/buildcache, so the cap must be larger than
|
|
# (baked entries + per-run accumulation) or LRU eviction starts dropping baked
|
|
# entries between nightly rebuilds. 4 GB comfortably exceeds the observed
|
|
# warm cache size for a full ResInsight-tests compile.
|
|
ENV BUILDCACHE_ACCURACY=SLOPPY \
|
|
BUILDCACHE_MAX_CACHE_SIZE=4294967296
|
|
|
|
COPY --from=vcpkg-builder /opt/vcpkg-cache /opt/vcpkg-cache
|
|
COPY . /src
|
|
|
|
# The buildcache directory is mounted as a BuildKit cache mount so it persists
|
|
# across nightly image rebuilds (each rebuild only recompiles drift since the
|
|
# previous run). After the compile finishes the populated cache is copied into
|
|
# /opt/buildcache, a real path that is preserved in the image layer.
|
|
#
|
|
# Unity build is OFF here AND in the consumer workflow: with unity ON, a single
|
|
# .cpp change invalidates the whole unity blob's cache entry, defeating the
|
|
# point. The flag must match between this stage and CI or hit rate collapses
|
|
# (the command line is part of the hash).
|
|
RUN --mount=type=cache,id=ri-rhel8-buildcache,target=/var/cache/buildcache,sharing=locked \
|
|
git config --global --add safe.directory '*' \
|
|
&& source /opt/rh/gcc-toolset-14/enable \
|
|
&& ( cd /src/ThirdParty/vcpkg && ./bootstrap-vcpkg.sh ) \
|
|
&& export CC=/opt/rh/gcc-toolset-14/root/usr/bin/gcc \
|
|
&& export CXX=/opt/rh/gcc-toolset-14/root/usr/bin/g++ \
|
|
&& export VCPKG_FEATURE_FLAGS=binarycaching \
|
|
&& export VCPKG_BINARY_SOURCES="clear;files,/opt/vcpkg-cache,read" \
|
|
&& export BUILDCACHE_DIR=/var/cache/buildcache \
|
|
&& cmake -S /src -B /src/cmakebuild -G Ninja \
|
|
-DCMAKE_BUILD_TYPE=Release \
|
|
-DRESINSIGHT_INCLUDE_APPLICATION_UNIT_TESTS=true \
|
|
-DRESINSIGHT_ENABLE_UNITY_BUILD=false \
|
|
-DRESINSIGHT_ENABLE_HDF5=false \
|
|
-DRESINSIGHT_ENABLE_GRPC=false \
|
|
-DRESINSIGHT_GCC_STDCPP_EXP_PATH=/opt/libstdcxx-exp/lib \
|
|
-DRESINSIGHT_VCPKG_PROFILE=rhel8 \
|
|
-DVCPKG_TARGET_TRIPLET=x64-linux-release \
|
|
-DVCPKG_HOST_TRIPLET=x64-linux-release \
|
|
-DCMAKE_TOOLCHAIN_FILE=ThirdParty/vcpkg/scripts/buildsystems/vcpkg.cmake \
|
|
&& cmake --build /src/cmakebuild --target ResInsight-tests -- -j"$(nproc)" \
|
|
&& buildcache -s \
|
|
&& mkdir -p /opt/buildcache \
|
|
&& cp -a /var/cache/buildcache/. /opt/buildcache/ \
|
|
&& test -n "$(ls -A /opt/buildcache)" \
|
|
&& rm -rf /src
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Stage 4: final image
|
|
# -----------------------------------------------------------------------------
|
|
FROM base
|
|
|
|
LABEL org.opencontainers.image.source="https://github.com/OPM/ResInsight"
|
|
LABEL org.opencontainers.image.description="RHEL8 CI image for ResInsight with prebuilt toolchain, vcpkg binary cache, and warm ResInsight buildcache"
|
|
|
|
COPY --from=vcpkg-builder /opt/vcpkg-cache /opt/vcpkg-cache
|
|
COPY --from=build-warmup /opt/buildcache /opt/buildcache
|
|
|
|
ENV VCPKG_DEFAULT_BINARY_CACHE=/opt/vcpkg-cache \
|
|
BUILDCACHE_DIR=/opt/buildcache \
|
|
BUILDCACHE_ACCURACY=SLOPPY \
|
|
BUILDCACHE_MAX_CACHE_SIZE=4294967296
|