From 4ff02a9845fb097921f8815821a3420156ca3a62 Mon Sep 17 00:00:00 2001 From: Herbert Wolverson Date: Wed, 23 Sep 2026 13:53:28 -0500 Subject: [PATCH] Make the package check non-vacuous and finish the third-pass fixes - check-package asserts the .deb control fields against the expected dependency and recommends entries, so reverting the Debian dependency fix fails the check. - check-package --install installs the .deb in the running guest and verifies the postinst results: dpkg state, installed unit, venv, and no missing shared libraries. - Guest image facts (URL, checksum, tool, os-variant, expected os-release) come from one guest_os_facts helper; lab and the harness assertions share it. - up records the guest OS choice too, so later commands cannot disagree with the running lab. - wait_for_cloud_init is bounded with a guest-side timeout. - Prerequisite extras install one package at a time so a missing kernel-pinned linux-tools package cannot block bpftool. - Docs: renumber the update lists, fix the Spanish phrasing in the orphan update page, align the orphan troubleshooting page, and drop the repeated drift rationale from the front-end headers. --- docs-es/v2.0/TechnicalDocs/troubleshooting.md | 2 +- docs-es/v2.0/Updates/update.md | 4 +- docs/v2.0/update.md | 4 +- radius-harness/README.md | 4 +- radius-harness/lab | 55 +++++++++++-------- radius-harness/lib/common.sh | 30 +++++++--- src/build_pkg_debian.sh | 3 +- src/build_prereqs.sh | 15 ++--- src/build_rust_debian.sh | 3 +- 9 files changed, 72 insertions(+), 48 deletions(-) diff --git a/docs-es/v2.0/TechnicalDocs/troubleshooting.md b/docs-es/v2.0/TechnicalDocs/troubleshooting.md index 2f03daf7..03b0fc2b 100644 --- a/docs-es/v2.0/TechnicalDocs/troubleshooting.md +++ b/docs-es/v2.0/TechnicalDocs/troubleshooting.md @@ -67,7 +67,7 @@ sudo dpkg --configure -a sudo systemctl restart lqosd lqos_scheduler ``` -Git-based installs should use `./build_rust.sh` after pulling updates. It rebuilds the virtual environment before refreshing service files or restarting services. If systemd reports `status=203/EXEC` on `/opt/libreqos/venv/bin/python`, or a failed scheduler pre-start check, rebuild the virtual environment with the command above and restart `lqos_scheduler`. +Git-based installs should use `./build_rust.sh` after pulling updates (on Debian 13 or later, use `./build_rust_debian.sh`). It rebuilds the virtual environment before refreshing service files or restarting services. If systemd reports `status=203/EXEC` on `/opt/libreqos/venv/bin/python`, or a failed scheduler pre-start check, rebuild the virtual environment with the command above and restart `lqos_scheduler`. Older installs that predate the virtual environment may show `ModuleNotFoundError` and suggest system `pip` commands. Do not repair current installs with system `pip` or `--break-system-packages`; those packages are not used by the venv-backed scheduler service. Upgrade to a package that creates `/opt/libreqos/venv`, then use the repair command above. diff --git a/docs-es/v2.0/Updates/update.md b/docs-es/v2.0/Updates/update.md index 82810afc..3120071a 100644 --- a/docs-es/v2.0/Updates/update.md +++ b/docs-es/v2.0/Updates/update.md @@ -26,8 +26,8 @@ Esto eliminará los mapas eBPF antiguos y cargará la última versión de LibreQ 1. Cambie a su directorio `LibreQoS`(e.g. `cd /opt/LibreQoS`) 2.Actualización desde Git: `git pull` 3. ```git switch develop``` -5. Recompile: `./build_rust.sh` (on Debian 13 or later, `./build_rust_debian.sh`) -6. `sudo rust/remove_pinned_maps.sh` +4. Recompile: `./build_rust.sh` (en Debian 13 o posterior, `./build_rust_debian.sh`) +5. `sudo rust/remove_pinned_maps.sh` Ejecute los siguientes comandos para recargar los servicios de LibreQoS. diff --git a/docs/v2.0/update.md b/docs/v2.0/update.md index ddfd8149..83542b53 100644 --- a/docs/v2.0/update.md +++ b/docs/v2.0/update.md @@ -91,8 +91,8 @@ If checks fail, go directly to [Troubleshooting](troubleshooting.md) before furt 1. Change to your LibreQoS directory (e.g. `cd /opt/libreqos`) 2. Update from Git: `git pull` 3. ```git switch develop``` -5. Recompile: `./build_rust.sh` (use `./build_rust_debian.sh` on Debian 13 or later) -6. `sudo rust/remove_pinned_maps.sh` +4. Recompile: `./build_rust.sh` (use `./build_rust_debian.sh` on Debian 13 or later) +5. `sudo rust/remove_pinned_maps.sh` ### Ubuntu 24.04 hotfix for Git installs diff --git a/radius-harness/README.md b/radius-harness/README.md index 1f47954c..8941300e 100644 --- a/radius-harness/README.md +++ b/radius-harness/README.md @@ -33,7 +33,7 @@ firmware elsewhere, set `OVMF_CODE_PATH` and `OVMF_VARS_PATH` in the environment before `up`. The pinned image hashes are in [lab.env](lab.env). Review and update the URL, -version, and SHA-256 together when changing an image. Debian publishes SHA-512 +version, and checksum together when changing an image. Debian publishes SHA-512 digests in `SHA512SUMS`, so the Debian image pins a SHA-512 value instead. The harness builds the LibreQoS runtime bundle on the host and copies it into @@ -89,7 +89,7 @@ templates before running it. ./radius-harness/lab status Show domains and management addresses. ./radius-harness/lab configure Install fixtures and start services. ./radius-harness/lab test Run the lifecycle assertions. -./radius-harness/lab check-package Check .deb dependency resolution in the guest. +./radius-harness/lab check-package Check .deb dependencies; add --install to install it. ./radius-harness/lab down Remove VMs, networks, overlays, and secrets. ./radius-harness/lab purge Also remove cached images and runtime artifacts. ./radius-harness/lab console Open the RouterOS serial console. diff --git a/radius-harness/lab b/radius-harness/lab index af5b0483..59d4f001 100755 --- a/radius-harness/lab +++ b/radius-harness/lab @@ -6,23 +6,8 @@ source "$LAB_ROOT/lib/common.sh" source "$LAB_ROOT/lab.env" GUEST_OS=$(resolve_guest_os) -case "$GUEST_OS" in - ubuntu) - GUEST_IMAGE_URL=$UBUNTU_IMAGE_URL - GUEST_IMAGE_CHECKSUM=$UBUNTU_IMAGE_SHA256 - GUEST_IMAGE_CHECKSUM_TOOL=sha256sum - DEFAULT_GUEST_OS_VARIANT=ubuntu24.04 - ;; - debian) - GUEST_IMAGE_URL=$DEBIAN_IMAGE_URL - GUEST_IMAGE_CHECKSUM=$DEBIAN_IMAGE_SHA512 - GUEST_IMAGE_CHECKSUM_TOOL=sha512sum - DEFAULT_GUEST_OS_VARIANT=debian13 - ;; - *) - die "unsupported LAB_GUEST_OS '$GUEST_OS'; expected ubuntu or debian" - ;; -esac +GUEST_FACTS=$(guest_os_facts "$GUEST_OS") || die "unsupported LAB_GUEST_OS '$GUEST_OS'; expected ubuntu or debian" +IFS='|' read -r GUEST_IMAGE_URL GUEST_IMAGE_CHECKSUM GUEST_IMAGE_CHECKSUM_TOOL DEFAULT_GUEST_OS_VARIANT _ <<<"$GUEST_FACTS" GUEST_OS_VARIANT=${LAB_OS_VARIANT:-$DEFAULT_GUEST_OS_VARIANT} readonly GUEST_OS GUEST_IMAGE_URL GUEST_IMAGE_CHECKSUM GUEST_IMAGE_CHECKSUM_TOOL GUEST_OS_VARIANT @@ -46,7 +31,7 @@ EOF } require_host_tools() { - for tool in virsh virt-install qemu-img cloud-localds curl unzip sha256sum sha512sum rsync ssh scp; do require_command "$tool"; done + for tool in virsh virt-install qemu-img cloud-localds curl unzip sha256sum sha512sum rsync ssh scp dpkg-deb; do require_command "$tool"; done } ensure_ssh_key() { @@ -148,9 +133,11 @@ create_domains() { } destroy_domain() { local name=$1; if domain_exists "$name"; then virsh_lab destroy "$name" >/dev/null 2>&1 || true; virsh_lab undefine "$name" --nvram >/dev/null 2>&1 || virsh_lab undefine "$name"; fi; } -cmd_init() { require_host_tools; mkdir -p "$IMAGE_DIR" "$RUN_DIR"; printf '%s\n' "$GUEST_OS" >"$IMAGE_DIR/guest-os"; download_guest_image; download_routeros; build_runtime_bundle; note "lab bases and runtime bundle are ready"; } +record_guest_os() { printf '%s\n' "$GUEST_OS" >"$IMAGE_DIR/guest-os"; } +cmd_init() { require_host_tools; mkdir -p "$IMAGE_DIR" "$RUN_DIR"; record_guest_os; download_guest_image; download_routeros; build_runtime_bundle; note "lab bases and runtime bundle are ready"; } cmd_up() { require_host_tools; resolve_ovmf; [[ -f $GUEST_BASE && -f $ROUTER_BASE && -x $RUNTIME_DIR/bin/lqosd ]] || die "run ./radius-harness/lab init first"; ensure_lab_stopped + record_guest_os trap 'status=$?; trap - ERR; note "lab creation failed; removing partial resources"; cmd_down || true; exit "$status"' ERR safe_remove "$RUN_DIR"; mkdir -p "$RUN_DIR"; ensure_ssh_key for guest in lqos radius client; do cp "$LAB_ROOT/templates/$guest-network.yaml" "$RUN_DIR/$guest-network.yaml"; done @@ -158,17 +145,33 @@ cmd_up() { create_network "$CONTROL_NETWORK" "$LAB_ROOT/templates/network-control.xml"; create_network "$ACCESS_NETWORK" "$LAB_ROOT/templates/network-access.xml"; create_network "$TRANSIT_NETWORK" "$LAB_ROOT/templates/network-transit.xml"; create_domains trap - ERR; note "VMs created. Use ./radius-harness/lab console once to set RouterOS admin password, then run ./radius-harness/lab configure." } -cmd_status() { for item in "$LQOS_DOMAIN 52:54:00:10:00:10" "$RADIUS_DOMAIN 52:54:00:20:00:10" "$CLIENT_DOMAIN 52:54:00:30:00:10" "$ROUTER_DOMAIN 52:54:00:40:00:10"; do set -- $item; printf '%-20s %s\n' "$1" "$(management_ip "$2" || true)"; done; } +cmd_status() { for item in "$LQOS_DOMAIN 52:54:00:10:00:10" "$RADIUS_DOMAIN 52:54:00:20:00:10" "$CLIENT_DOMAIN 52:54:00:30:00:10" "$ROUTER_DOMAIN 52:54:00:40:00:10"; do set -- $item; printf '%-20s %s\n' "$1" "$(management_ip "$2")"; done; } cmd_console() { domain_exists "$ROUTER_DOMAIN" || die "RouterOS domain is not running; use ./radius-harness/lab up first"; virsh_lab console "$ROUTER_DOMAIN"; } cmd_configure() { "$LAB_ROOT/lib/configure.sh"; } cmd_test() { "$LAB_ROOT/lib/test-lifecycle.sh"; } +assert_package_control() { + local package_path=$1 depends recommends expected + depends=$(dpkg-deb -f "$package_path" Depends) + recommends=$(dpkg-deb -f "$package_path" Recommends) + for expected in libelf1t64 libssl3t64 zlib1g; do + [[ $depends == *"$expected"* ]] || die "package Depends is missing $expected" + done + for expected in netplan.io ethtool; do + [[ $recommends == *"$expected"* ]] || die "package Recommends is missing $expected" + done +} + cmd_check_package() { - local package_path=${1:-} lqos_ip simulate_output + local package_path=${1:-} install=${2:-} lqos_ip simulate_output require_host_tools + if [[ -n $install && $install != "--install" ]]; then + die "unknown check-package option '$install'; expected --install" + fi if [[ -z $package_path ]]; then package_path=$(ls -1t "$REPO_ROOT"/src/dist/libreqos_*_amd64.deb 2>/dev/null | head -n 1) || true fi [[ -n $package_path && -f $package_path ]] || die "no LibreQoS package found; build one with src/build_pkg_debian.sh or pass a path" + assert_package_control "$package_path" domain_exists "$LQOS_DOMAIN" || die "the lab is not running; use ./radius-harness/lab up first" lqos_ip=$(wait_for_guest_ready 52:54:00:10:00:10) note "verifying the LibreQoS guest operating system" @@ -179,9 +182,17 @@ cmd_check_package() { printf '%s\n' "$simulate_output" die "package check failed in the $GUEST_OS guest" fi + if [[ $install == "--install" ]]; then + note "installing the package inside the $GUEST_OS guest" + lab_ssh "$lqos_ip" 'sudo systemctl stop radius-lqosd.service 2>/dev/null || true' + lab_ssh "$lqos_ip" "sudo apt-get install -y '/tmp/$(basename "$package_path")'" || die "package installation failed in the $GUEST_OS guest" + lab_ssh "$lqos_ip" 'sudo dpkg -s libreqos >/dev/null && sudo test -e /etc/systemd/system/lqosd.service && sudo test -x /opt/libreqos/venv/bin/python' || die "installed package is missing expected files" + lab_ssh "$lqos_ip" '! ldd /opt/libreqos/src/bin/lqosd | grep -q "not found"' || die "installed lqosd has missing shared libraries" + note "package installed and postinst completed" + fi note "package dependency resolution passed" } cmd_down() { for domain in "$CLIENT_DOMAIN" "$RADIUS_DOMAIN" "$ROUTER_DOMAIN" "$LQOS_DOMAIN"; do destroy_domain "$domain"; done; for network in "$TRANSIT_NETWORK" "$ACCESS_NETWORK" "$CONTROL_NETWORK"; do destroy_network "$network"; done; [[ -d $RUN_DIR ]] && safe_remove "$RUN_DIR"; note "disposable lab resources removed"; } cmd_purge() { cmd_down; [[ -d $IMAGE_DIR ]] && safe_remove "$IMAGE_DIR"; [[ -d $RUNTIME_DIR ]] && safe_remove "$RUNTIME_DIR"; note "cached bases and runtime artifacts removed"; } -case ${1:-} in init) cmd_init ;; up) cmd_up ;; status) cmd_status ;; configure) cmd_configure ;; test) cmd_test ;; check-package) cmd_check_package "${2:-}" ;; down) cmd_down ;; purge) cmd_purge ;; console) cmd_console ;; *) usage; exit 2 ;; esac +case ${1:-} in init) cmd_init ;; up) cmd_up ;; status) cmd_status ;; configure) cmd_configure ;; test) cmd_test ;; check-package) cmd_check_package "${2:-}" "${3:-}" ;; down) cmd_down ;; purge) cmd_purge ;; console) cmd_console ;; *) usage; exit 2 ;; esac diff --git a/radius-harness/lib/common.sh b/radius-harness/lib/common.sh index a3ce397b..fb2c92f0 100755 --- a/radius-harness/lib/common.sh +++ b/radius-harness/lib/common.sh @@ -39,7 +39,7 @@ ensure_lab_stopped() { } # Prints the guest OS recorded for this lab: LAB_GUEST_OS, the images/guest-os -# marker written by init, or the ubuntu default. +# marker written by init and up, or the ubuntu default. resolve_guest_os() { local guest_os=${LAB_GUEST_OS:-} if [[ -z $guest_os && -f $IMAGE_DIR/guest-os ]]; then @@ -48,13 +48,27 @@ resolve_guest_os() { printf '%s\n' "${guest_os:-ubuntu}" } +# Prints "image_url|checksum|checksum_tool|os_variant|os_release" for a guest +# OS. Returns non-zero for unknown guest operating systems. +guest_os_facts() { + case "$1" in + ubuntu) + printf '%s|%s|%s|%s|%s\n' "$UBUNTU_IMAGE_URL" "$UBUNTU_IMAGE_SHA256" sha256sum ubuntu24.04 'ubuntu 24.04' + ;; + debian) + printf '%s|%s|%s|%s|%s\n' "$DEBIAN_IMAGE_URL" "$DEBIAN_IMAGE_SHA512" sha512sum debian13 'debian 13' + ;; + *) + return 1 + ;; + esac +} + # Prints the /etc/os-release "ID VERSION_ID" string expected for a guest OS. guest_expected_os_release() { - case "$1" in - ubuntu) printf 'ubuntu 24.04\n' ;; - debian) printf 'debian 13\n' ;; - *) return 1 ;; - esac + local facts + facts=$(guest_os_facts "$1") || return 1 + printf '%s\n' "${facts##*|}" } management_ip() { @@ -97,11 +111,11 @@ wait_for_ssh() { wait_for_cloud_init() { local host=$1 output - if output=$(lab_ssh "$host" 'sudo cloud-init status --wait --long' 2>&1); then + if output=$(lab_ssh "$host" 'sudo timeout 900 cloud-init status --wait --long' 2>&1); then return 0 fi printf '%s\n' "$output" - die "cloud-init did not complete successfully on $host" + die "cloud-init did not finish successfully on $host" } wait_for_guest_ready() { diff --git a/src/build_pkg_debian.sh b/src/build_pkg_debian.sh index 7a89b2ce..78a43721 100755 --- a/src/build_pkg_debian.sh +++ b/src/build_pkg_debian.sh @@ -4,8 +4,7 @@ # # This is the Debian front-end for build_dpkg.sh. It installs the Debian build # prerequisites (which include bpftool, needed by the lqos_sys build) and then -# runs the shared packaging implementation, so the Ubuntu and Debian packages -# cannot drift apart. +# runs the shared packaging implementation. # # Run build_rust_debian.sh first if you also want the local src/bin install. # diff --git a/src/build_prereqs.sh b/src/build_prereqs.sh index da3c11ae..de2bc788 100755 --- a/src/build_prereqs.sh +++ b/src/build_prereqs.sh @@ -92,14 +92,15 @@ lqos_install_build_prerequisites() { sudo apt-get install -y "${LQOS_COMMON_BUILD_PACKAGES[@]}" || return 1 # Kernel tools packages track the running kernel and can be unavailable on - # custom or HWE kernels. Keep them best-effort so their absence cannot - # abort the common install. - # shellcheck disable=SC2086 - if ! sudo apt-get install -y $extras; then - echo "Warning: failed to install: $extras" - echo "Install the distribution equivalents of bpftool (required by lqos_sys) and perf manually if the build fails." - fi + # custom or HWE kernels. Install each independently so a missing + # kernel-pinned package cannot block the rest. + while IFS= read -r package; do + if ! sudo apt-get install -y "$package"; then + echo "Warning: failed to install: $package" + fi + done <<<"$extras" if ! command -v bpftool >/dev/null 2>&1; then echo "Warning: bpftool was not found; lqos_sys cannot build without it." + echo "Install the distribution package that provides bpftool." fi } diff --git a/src/build_rust_debian.sh b/src/build_rust_debian.sh index 29c9caa1..25eca211 100755 --- a/src/build_rust_debian.sh +++ b/src/build_rust_debian.sh @@ -3,8 +3,7 @@ # # This is the Debian front-end for build_rust.sh. It selects the Debian # prerequisite package set (bpftool and linux-perf replace the Ubuntu -# linux-tools-* packages) and runs the shared build implementation, so the -# Ubuntu and Debian paths cannot drift apart. +# linux-tools-* packages) and runs the shared build implementation. # # Usage matches build_rust.sh: ./build_rust_debian.sh [--fast]