diff --git a/docs-es/v2.0/bridge-es.md b/docs-es/v2.0/bridge-es.md index 59364ef2..4c6d00f2 100644 --- a/docs-es/v2.0/bridge-es.md +++ b/docs-es/v2.0/bridge-es.md @@ -99,3 +99,23 @@ sudo netplan apply ``` Para usar el puente XDP, asegurese de establecer `use_xdp_bridge` como `true` en el archivo lqos.conf dentro de la sección [Configuración](configuration-es.md). + +## Capa de compatibilidad de interfaces + +Use la capa de compatibilidad solo cuando LibreQoS no pueda adjuntar XDP directamente a las interfaces seleccionadas. Los casos más comunes son las interfaces enlazadas mediante bonding y los controladores de NIC que no ofrecen la compatibilidad con XDP que LibreQoS necesita. Si funciona XDP directo o el puente normal de Linux, esa opción sigue siendo preferible. + +La capa conecta cada interfaz física a una interfaz veth multiqueue mediante un pequeño puente de Linux. LibreQoS adjunta a las interfaces veth su ruta existente de XDP y colas. Esto consume más CPU, pero mantiene activas en las interfaces físicas las descargas de sumas de comprobación, segmentación y VLAN. LibreQoS sigue aplicando los valores configurados de moderación de interrupciones cuando el controlador físico los acepta. + +Actívela en la página `Bridge & Interface Mode`, o configure: + +```toml +[bridge] +use_xdp_bridge = true +compatibility_shim = true +to_internet = "bond0" +to_network = "bond1" +``` + +Reinicie `lqosd` después de cambiar esta opción. LibreQoS elige la cantidad de colas veth según las CPU activas para shaping y cualquier override de colas configurado. Para la ruta de la capa usa el menor MTU de las dos interfaces físicas. + +La capa de compatibilidad no añade a las interfaces físicas un límite de velocidad, un limitador HTB ni `fq_codel`. Configure el shaping de suscriptores mediante los ajustes normales de colas de LibreQoS. diff --git a/docs-es/v2.0/configuration-advanced-es.md b/docs-es/v2.0/configuration-advanced-es.md index 0df8da6a..68afd35d 100644 --- a/docs-es/v2.0/configuration-advanced-es.md +++ b/docs-es/v2.0/configuration-advanced-es.md @@ -37,6 +37,8 @@ En la sección ```[bridge]```, cambie `to_internet` y `to_network` para que coin En la sección `[bridge]` del archivo lqos.conf, puede habilitar o deshabilitar el puente XDP con la opción `use_xdp_bridge`. El valor predeterminado es `false` - ya que la configuración por defecto asume un [Puente Linux](prereq-es.md). Si eligió usar el puente XDP durante la configuración de requisitos previos, establezca `use_xdp_bridge = true`. +Si una interfaz enlazada mediante bonding o el controlador de una NIC no permite adjuntar XDP de LibreQoS directamente, establezca `compatibility_shim = true` en la misma sección `[bridge]`. La capa requiere `use_xdp_bridge = true`. Proporciona interfaces veth multiqueue a LibreQoS sin añadir límites de velocidad ni qdiscs a las interfaces físicas. Consulte [Capa de compatibilidad de interfaces](bridge-es.md#capa-de-compatibilidad-de-interfaces). + - Configure downlink_bandwidth_mbps y uplink_bandwidth_mbps para que coincidan con el ancho de banda en Mbps de la conexión WAN/Upstream de su red. Lo mismo puede hacerse para generated_pn_download_mbps y generated_pn_upload_mbps. - to_internet es la interfaz que apunta hacia su router de borde (edge router) y el internet. - to_network es la interfaz que apunta hacia su router interno (core router) (o la red interna puenteada, si su red está configurada de esa manera). diff --git a/docs/v2.0/bridge.md b/docs/v2.0/bridge.md index 7f232bff..0b1641f1 100644 --- a/docs/v2.0/bridge.md +++ b/docs/v2.0/bridge.md @@ -99,3 +99,23 @@ sudo netplan apply ``` To use the XDP bridge, please be sure to set `use_xdp_bridge` to `true` in lqos.conf in the [Configuration](configuration.md) section. + +## Interface Compatibility Shim + +Use the interface compatibility shim only when LibreQoS cannot attach XDP directly to the selected interfaces. Common examples are bonded interfaces and NIC drivers without the XDP support LibreQoS requires. Direct XDP or the regular Linux bridge remains preferable when either works. + +The shim connects each physical interface to a multiqueue veth through a small Linux bridge. LibreQoS attaches its existing XDP and queueing path to the veth interfaces. This adds CPU overhead, but leaves checksum, segmentation, and VLAN offloads enabled on the physical interfaces. LibreQoS still applies the configured interrupt-coalescing values where the physical driver accepts them. + +Enable it on the `Bridge & Interface Mode` page, or set: + +```toml +[bridge] +use_xdp_bridge = true +compatibility_shim = true +to_internet = "bond0" +to_network = "bond1" +``` + +Restart `lqosd` after changing this setting. LibreQoS chooses the veth queue count from the active shaping CPU count and any configured queue override. It uses the smaller physical-interface MTU for the shim path. + +The compatibility shim does not add a link-speed cap, HTB limiter, or `fq_codel` to the physical interfaces. Configure subscriber shaping through the normal LibreQoS queue settings. diff --git a/docs/v2.0/configuration-advanced.md b/docs/v2.0/configuration-advanced.md index 7672585e..fbecf45c 100644 --- a/docs/v2.0/configuration-advanced.md +++ b/docs/v2.0/configuration-advanced.md @@ -39,6 +39,8 @@ In the ```[bridge]``` section, change `to_internet` and `to_network` to match yo In the `[bridge]` section of the lqos.conf file, you can enable or disable the XDP Bridge with the setting `use_xdp_bridge`. The default value is `false` - because the default setup assumes a [Linux Bridge](prereq.md). If you chose to use the XDP Bridge during that pre-requisites setup, please set `use_xdp_bridge = true` instead. +If a bonded interface or NIC driver cannot host LibreQoS XDP directly, set `compatibility_shim = true` in the same `[bridge]` section. The shim requires `use_xdp_bridge = true`. It supplies multiqueue veth interfaces to LibreQoS without adding rate limits or qdiscs to the physical interfaces. See [Interface Compatibility Shim](bridge.md#interface-compatibility-shim). + - Set downlink_bandwidth_mbps and uplink_bandwidth_mbps to match the bandwidth in Mbps of your network's upstream / WAN internet connection. The same can be done for generated_pn_download_mbps and generated_pn_upload_mbps. - to_internet would be the interface facing your edge router and the broader internet - to_network would be the interface facing your core router (or bridged internal network if your network is bridged) diff --git a/src/rust/lqos_bakery/src/lib.rs b/src/rust/lqos_bakery/src/lib.rs index a0cc5efe..b36b8bd8 100644 --- a/src/rust/lqos_bakery/src/lib.rs +++ b/src/rust/lqos_bakery/src/lib.rs @@ -11987,6 +11987,7 @@ mod tests { to_internet: "internet0".to_string(), to_network: "isp0".to_string(), mtu: None, + compatibility_shim: false, }), ..Config::default() }); @@ -12265,6 +12266,7 @@ mod tests { to_internet: "lo".to_string(), to_network: "lo".to_string(), mtu: None, + compatibility_shim: false, }), ..lqos_config::Config::default() }; @@ -14120,6 +14122,7 @@ mod tests { to_internet: "__bakery-missing-wan__".to_string(), to_network: "__bakery-missing-lan__".to_string(), mtu: None, + compatibility_shim: false, }), ..Config::default() }; diff --git a/src/rust/lqos_config/src/etc/migration.rs b/src/rust/lqos_config/src/etc/migration.rs index 1a6eec6a..554fcfc8 100644 --- a/src/rust/lqos_config/src/etc/migration.rs +++ b/src/rust/lqos_config/src/etc/migration.rs @@ -195,6 +195,7 @@ fn migrate_bridge( to_internet: python_config.interface_b.clone(), to_network: python_config.interface_a.clone(), mtu: None, + compatibility_shim: false, }); } Ok(()) diff --git a/src/rust/lqos_config/src/etc/v15/bridge.rs b/src/rust/lqos_config/src/etc/v15/bridge.rs index aebffa91..bccfe016 100644 --- a/src/rust/lqos_config/src/etc/v15/bridge.rs +++ b/src/rust/lqos_config/src/etc/v15/bridge.rs @@ -20,6 +20,15 @@ pub struct BridgeConfig { /// Optional MTU for LibreQoS-managed Linux bridge interfaces. #[serde(default, skip_serializing_if = "Option::is_none")] pub mtu: Option, + + /// Route traffic through LibreQoS-managed veth devices when the selected + /// physical interfaces cannot host the XDP programs directly. + #[serde(default, skip_serializing_if = "bool_is_false")] + pub compatibility_shim: bool, +} + +fn bool_is_false(value: &bool) -> bool { + !*value } impl Default for BridgeConfig { @@ -29,10 +38,26 @@ impl Default for BridgeConfig { to_internet: "eth0".to_string(), to_network: "eth1".to_string(), mtu: None, + compatibility_shim: false, } } } +impl BridgeConfig { + /// Returns whether the veth interface compatibility shim is enabled. + pub fn compatibility_shim_enabled(&self) -> bool { + self.compatibility_shim + } + + /// Checks invariants required by the veth interface compatibility shim. + pub fn validate_compatibility_shim(&self) -> Result<(), &'static str> { + if self.compatibility_shim_enabled() && !self.use_xdp_bridge { + return Err("bridge.compatibility_shim requires bridge.use_xdp_bridge = true"); + } + Ok(()) + } +} + /// Represents a single-interface bridge #[derive(Clone, Serialize, Deserialize, Debug, PartialEq, Allocative)] pub struct SingleInterfaceConfig { diff --git a/src/rust/lqos_config/src/etc/v15/top_config.rs b/src/rust/lqos_config/src/etc/v15/top_config.rs index cfc98f59..95d48add 100644 --- a/src/rust/lqos_config/src/etc/v15/top_config.rs +++ b/src/rust/lqos_config/src/etc/v15/top_config.rs @@ -301,6 +301,9 @@ impl Config { } if let Some(bridge) = &self.bridge { validate_interface_mtu("bridge.mtu", bridge.mtu)?; + bridge + .validate_compatibility_shim() + .map_err(str::to_string)?; } if let Some(single_interface) = &self.single_interface { validate_interface_mtu("single_interface.mtu", single_interface.mtu)?; @@ -625,7 +628,11 @@ impl Config { /// Calculate the unterface facing the Internet pub fn internet_interface(&self) -> String { if let Some(bridge) = &self.bridge { - bridge.to_internet.clone() + if bridge.compatibility_shim_enabled() { + crate::SHIM_INTERNET_LQOS.to_string() + } else { + bridge.to_internet.clone() + } } else if let Some(single_interface) = &self.single_interface { single_interface.interface.clone() } else { @@ -636,7 +643,11 @@ impl Config { /// Calculate the interface facing the ISP pub fn isp_interface(&self) -> String { if let Some(bridge) = &self.bridge { - bridge.to_network.clone() + if bridge.compatibility_shim_enabled() { + crate::SHIM_NETWORK_LQOS.to_string() + } else { + bridge.to_network.clone() + } } else if let Some(single_interface) = &self.single_interface { single_interface.interface.clone() } else { @@ -705,6 +716,36 @@ mod test { assert_eq!(config.version, "1.5"); } + #[test] + fn compatibility_shim_uses_managed_veth_interfaces() { + let raw = include_str!("example.toml").replace( + "use_xdp_bridge = true", + "use_xdp_bridge = true\ncompatibility_shim = true", + ); + let config = Config::load_from_string(&raw).expect("compatibility shim should load"); + + assert_eq!(config.internet_interface(), crate::SHIM_INTERNET_LQOS); + assert_eq!(config.isp_interface(), crate::SHIM_NETWORK_LQOS); + assert!( + config + .bridge + .as_ref() + .is_some_and(BridgeConfig::compatibility_shim_enabled) + ); + } + + #[test] + fn compatibility_shim_requires_xdp_bridge() { + let raw = include_str!("example.toml").replace( + "use_xdp_bridge = true", + "use_xdp_bridge = false\ncompatibility_shim = true", + ); + let error = Config::load_from_string(&raw) + .expect_err("compatibility shim without XDP bridge should fail"); + + assert!(error.contains("compatibility_shim requires bridge.use_xdp_bridge = true")); + } + #[test] fn local_api_config_defaults_when_section_is_absent() { let legacy = remove_sections(include_str!("example.toml"), &["local_api"]); diff --git a/src/rust/lqos_config/src/lib.rs b/src/rust/lqos_config/src/lib.rs index adf5bf5f..3a29e92f 100644 --- a/src/rust/lqos_config/src/lib.rs +++ b/src/rust/lqos_config/src/lib.rs @@ -157,3 +157,21 @@ pub fn topology_import_has_shaped_devices(config: &Config) -> bool { /// Used as a constant in determining buffer preallocation pub const SUPPORTED_CUSTOMERS: usize = 100_000; + +/// lqosd-facing veth on the Internet side of the interface compatibility shim. +pub const SHIM_INTERNET_LQOS: &str = "v_inet_lq"; + +/// Bridge-facing peer of [`SHIM_INTERNET_LQOS`]. +pub const SHIM_INTERNET_PEER: &str = "v_inet_phy"; + +/// lqosd-facing veth on the subscriber-network side of the compatibility shim. +pub const SHIM_NETWORK_LQOS: &str = "v_isp_lq"; + +/// Bridge-facing peer of [`SHIM_NETWORK_LQOS`]. +pub const SHIM_NETWORK_PEER: &str = "v_isp_phy"; + +/// Two-port Linux bridge joining the Internet interface to its veth peer. +pub const SHIM_INTERNET_BRIDGE: &str = "br_lq_inet"; + +/// Two-port Linux bridge joining the subscriber-network interface to its veth peer. +pub const SHIM_NETWORK_BRIDGE: &str = "br_lq_isp"; diff --git a/src/rust/lqos_netplan_helper/src/inspect.rs b/src/rust/lqos_netplan_helper/src/inspect.rs index 098ef02f..332f0d0e 100644 --- a/src/rust/lqos_netplan_helper/src/inspect.rs +++ b/src/rust/lqos_netplan_helper/src/inspect.rs @@ -58,6 +58,9 @@ pub struct InterfaceCandidate { #[serde(default)] pub details: Vec, pub bridge_eligible: bool, + /// Whether the interface can be used behind the veth compatibility shim. + #[serde(default)] + pub compatibility_shim_eligible: bool, pub single_interface_eligible: bool, #[serde(default)] pub current_selection: bool, @@ -496,27 +499,31 @@ fn collect_interface_restrictions( } } +fn is_plain_bond_interface(iface: &str) -> bool { + iface.starts_with("bond") && !iface.contains('.') && !iface.contains(':') +} + fn interface_name_role_reason(iface: &str) -> Option<&'static str> { if iface == "lo" { Some("Loopback interface.") } else if iface.contains('.') || iface.contains(':') { Some("VLAN or alias interface.") - } else if [ - "br", - "bond", - "docker", - "veth", - "virbr", - "ifb", - "wg", - "tun", - "tap", - "tailscale", - "zt", - "vmnet", - ] - .iter() - .any(|prefix| iface.starts_with(prefix)) + } else if is_plain_bond_interface(iface) + || [ + "br", + "docker", + "veth", + "virbr", + "ifb", + "wg", + "tun", + "tap", + "tailscale", + "zt", + "vmnet", + ] + .iter() + .any(|prefix| iface.starts_with(prefix)) { Some("Virtual, bridge, or tunnel interface.") } else { @@ -537,26 +544,34 @@ fn build_interface_candidates( .map(|iface| { let mut details = Vec::new(); - if let Some(reason) = interface_name_role_reason(iface) { + let role_reason = interface_name_role_reason(iface); + if let Some(reason) = role_reason { details.push(reason.to_string()); } - if default_interface_name.is_some_and(|default_iface| default_iface == iface) { + let is_management_interface = + default_interface_name.is_some_and(|default_iface| default_iface == iface); + if is_management_interface { details.push("Current management/default-route interface.".to_string()); } if !queue_caps.get(iface).copied().unwrap_or(false) { details.push("Does not appear to have multi-queue RX/TX support.".to_string()); } - if let Some(extra) = interface_restrictions.get(iface) { + let restrictions = interface_restrictions.get(iface); + if let Some(extra) = restrictions { details.extend(extra.iter().cloned()); } details.sort(); details.dedup(); let eligible = details.is_empty(); + let compatibility_shim_eligible = !is_management_interface + && restrictions.is_none_or(BTreeSet::is_empty) + && (role_reason.is_none() || is_plain_bond_interface(iface)); InterfaceCandidate { name: iface.clone(), details, bridge_eligible: eligible, + compatibility_shim_eligible, single_interface_eligible: eligible, current_selection: selected_set.contains(iface), } @@ -862,6 +877,10 @@ pub fn inspect_network_mode_with_paths( queue_caps: &BTreeMap, ) -> NetworkModeInspection { let mode = requested_mode(config); + let compatibility_shim_enabled = config + .bridge + .as_ref() + .is_some_and(|bridge| bridge.compatibility_shim_enabled()); let selected = selected_interfaces(&mode); let (preview, preview_note) = managed_preview_yaml(&mode); let mut warnings = Vec::new(); @@ -887,10 +906,14 @@ pub fn inspect_network_mode_with_paths( for iface in &selected { if !system_ifaces.contains(iface) { missing.push(iface.clone()); - } else if !queue_caps.get(iface).copied().unwrap_or(false) { + } else if !queue_caps.get(iface).copied().unwrap_or(false) && !compatibility_shim_enabled { warnings.push(format!( "Interface {iface} does not appear to have multi-queue RX/TX support." )); + } else if !queue_caps.get(iface).copied().unwrap_or(false) { + warnings.push(format!( + "Interface {iface} does not expose multi-queue RX/TX support; the compatibility shim will provide multiqueue veth interfaces to LibreQoS." + )); } } @@ -1128,7 +1151,9 @@ pub fn inspect_network_mode(config: &Config) -> NetworkModeInspection { #[cfg(test)] mod tests { - use super::{adoption_rewrite_for_path, inspect_network_mode_with_paths}; + use super::{ + adoption_rewrite_for_path, build_interface_candidates, inspect_network_mode_with_paths, + }; use std::collections::{BTreeMap, BTreeSet}; fn linux_bridge_config() -> lqos_config::Config { @@ -1138,6 +1163,7 @@ mod tests { to_internet: "ens19".to_string(), to_network: "ens20".to_string(), mtu: None, + compatibility_shim: false, }), single_interface: None, ..lqos_config::Config::default() @@ -1266,6 +1292,89 @@ mod tests { let _ = std::fs::remove_dir_all(root); } + #[test] + fn compatibility_shim_allows_bonds_without_multiqueue_support() { + let mut config = linux_bridge_config(); + let bridge = config.bridge.as_mut().expect("bridge"); + bridge.use_xdp_bridge = true; + bridge.compatibility_shim = true; + bridge.to_internet = "bond0".to_string(); + bridge.to_network = "bond1".to_string(); + let root = test_env("compatibility-shim-bonds"); + + let inspection = inspect_network_mode_with_paths( + &config, + &root.join("netplan"), + &root.join("pending"), + &BTreeSet::from(["bond0".to_string(), "bond1".to_string()]), + &BTreeMap::from([("bond0".to_string(), false), ("bond1".to_string(), false)]), + ); + + assert_eq!(inspection.interface_candidates.len(), 2); + assert!( + inspection + .interface_candidates + .iter() + .all(|candidate| candidate.compatibility_shim_eligible) + ); + assert!( + inspection + .interface_candidates + .iter() + .all(|candidate| !candidate.bridge_eligible) + ); + assert!( + inspection + .warnings + .iter() + .any(|warning| warning.contains("compatibility shim will provide multiqueue")) + ); + let _ = std::fs::remove_dir_all(root); + } + + #[test] + fn compatibility_shim_still_rejects_management_interface() { + let candidates = build_interface_candidates( + &BTreeSet::from(["bond0".to_string(), "bond1".to_string()]), + &BTreeMap::from([("bond0".to_string(), false), ("bond1".to_string(), false)]), + &[], + Some("bond0"), + &BTreeMap::new(), + ); + + let management = candidates + .iter() + .find(|candidate| candidate.name == "bond0") + .expect("bond0 candidate"); + assert!(!management.compatibility_shim_eligible); + assert!( + management + .details + .iter() + .any(|detail| detail.contains("management/default-route")) + ); + } + + #[test] + fn compatibility_shim_rejects_bond_vlan_and_alias_interfaces() { + let candidates = build_interface_candidates( + &BTreeSet::from(["bond0.100".to_string(), "bond0:1".to_string()]), + &BTreeMap::from([ + ("bond0.100".to_string(), false), + ("bond0:1".to_string(), false), + ]), + &[], + None, + &BTreeMap::new(), + ); + + assert!( + candidates + .iter() + .all(|candidate| !candidate.compatibility_shim_eligible) + ); + } + #[test] fn detects_external_compatible_linux_bridge() { let root = test_env("external-compatible"); diff --git a/src/rust/lqos_netplan_helper/src/transaction.rs b/src/rust/lqos_netplan_helper/src/transaction.rs index a785e6d9..c7fb449c 100644 --- a/src/rust/lqos_netplan_helper/src/transaction.rs +++ b/src/rust/lqos_netplan_helper/src/transaction.rs @@ -1061,6 +1061,7 @@ esac to_internet, to_network, mtu: None, + compatibility_shim: false, }), single_interface: None, ..Config::default() diff --git a/src/rust/lqos_setup/src/setup_actions.rs b/src/rust/lqos_setup/src/setup_actions.rs index aa0e2d21..c2ebb7c1 100644 --- a/src/rust/lqos_setup/src/setup_actions.rs +++ b/src/rust/lqos_setup/src/setup_actions.rs @@ -43,6 +43,10 @@ pub(crate) enum CommitOutcome { pub(crate) fn build_candidate_config(existing: Option) -> lqos_config::Config { let mut config = existing.unwrap_or_default(); let existing_bridge_mtu = config.bridge.as_ref().and_then(|bridge| bridge.mtu); + let existing_compatibility_shim = config + .bridge + .as_ref() + .is_some_and(|bridge| bridge.compatibility_shim_enabled()); let existing_single_interface_mtu = config .single_interface .as_ref() @@ -61,6 +65,7 @@ pub(crate) fn build_candidate_config(existing: Option) -> l to_internet: new_config.to_internet.clone(), to_network: new_config.to_network.clone(), mtu: existing_bridge_mtu, + compatibility_shim: false, }); } BridgeMode::XDP => { @@ -70,6 +75,7 @@ pub(crate) fn build_candidate_config(existing: Option) -> l to_internet: new_config.to_internet.clone(), to_network: new_config.to_network.clone(), mtu: existing_bridge_mtu, + compatibility_shim: existing_compatibility_shim, }); } BridgeMode::Single => { @@ -305,4 +311,49 @@ mod tests { *CURRENT_CONFIG.lock() = previous_builder; } + + #[test] + fn build_candidate_config_keeps_compatibility_shim_only_in_xdp_mode() { + let previous_builder = { + let mut builder = CURRENT_CONFIG.lock(); + let previous = builder.clone(); + builder.bridge_mode = BridgeMode::XDP; + builder.to_internet = "bond0".to_string(); + builder.to_network = "bond1".to_string(); + previous + }; + + let existing = lqos_config::Config { + bridge: Some(lqos_config::BridgeConfig { + use_xdp_bridge: true, + compatibility_shim: true, + ..lqos_config::BridgeConfig::default() + }), + ..lqos_config::Config::default() + }; + let candidate = build_candidate_config(Some(existing)); + + assert!( + candidate + .bridge + .as_ref() + .is_some_and(lqos_config::BridgeConfig::compatibility_shim_enabled) + ); + + CURRENT_CONFIG.lock().bridge_mode = BridgeMode::Linux; + let candidate = build_candidate_config(Some(candidate)); + assert!( + candidate + .bridge + .as_ref() + .is_some_and(|bridge| !bridge.compatibility_shim_enabled()) + ); + + CURRENT_CONFIG.lock().bridge_mode = BridgeMode::Single; + let candidate = build_candidate_config(Some(candidate)); + assert!(candidate.bridge.is_none()); + assert!(candidate.single_interface.is_some()); + + *CURRENT_CONFIG.lock() = previous_builder; + } } diff --git a/src/rust/lqos_sys/src/lqos_kernel.rs b/src/rust/lqos_sys/src/lqos_kernel.rs index 3010da5f..5a5d2133 100644 --- a/src/rust/lqos_sys/src/lqos_kernel.rs +++ b/src/rust/lqos_sys/src/lqos_kernel.rs @@ -616,19 +616,21 @@ pub fn attach_xdp_and_tc_to_interface( if let Some(bridge) = &etc.bridge && bridge.use_xdp_bridge { + let internet_interface = etc.internet_interface(); + let network_interface = etc.isp_interface(); // Enable "promiscuous" mode on interfaces - debug!("Enabling promiscuous mode on {}", &bridge.to_internet); + debug!("Enabling promiscuous mode on {}", &internet_interface); std::process::Command::new("/bin/ip") - .args(["link", "set", &bridge.to_internet, "promisc", "on"]) + .args(["link", "set", &internet_interface, "promisc", "on"]) .output()?; - debug!("Enabling promiscuous mode on {}", &bridge.to_network); + debug!("Enabling promiscuous mode on {}", &network_interface); std::process::Command::new("/bin/ip") - .args(["link", "set", &bridge.to_network, "promisc", "on"]) + .args(["link", "set", &network_interface, "promisc", "on"]) .output()?; // Build the interface and vlan map entries crate::bifrost_maps::clear_bifrost()?; - crate::bifrost_maps::map_multi_interface_mode(&bridge.to_internet, &bridge.to_network)?; + crate::bifrost_maps::map_multi_interface_mode(&internet_interface, &network_interface)?; // Actually attach the TC ingress program let error = unsafe { bpf::tc_attach_ingress(interface_index as i32, false, skeleton) }; diff --git a/src/rust/lqosd/src/interface_shim.rs b/src/rust/lqosd/src/interface_shim.rs new file mode 100644 index 00000000..beccd3db --- /dev/null +++ b/src/rust/lqosd/src/interface_shim.rs @@ -0,0 +1,616 @@ +//! Builds the veth and Linux-bridge compatibility shim used when the selected +//! physical interfaces cannot host LibreQoS XDP programs directly. + +use lqos_config::{ + Config, SHIM_INTERNET_BRIDGE, SHIM_INTERNET_LQOS, SHIM_INTERNET_PEER, SHIM_NETWORK_BRIDGE, + SHIM_NETWORK_LQOS, SHIM_NETWORK_PEER, +}; +use std::num::{ParseIntError, TryFromIntError}; +use std::path::PathBuf; +use std::process::{Command, Stdio}; +use thiserror::Error; +use tracing::{debug, info, warn}; + +const IP_COMMAND: &str = "/bin/ip"; + +#[derive(Clone, Debug, PartialEq, Eq)] +struct ShimTopology { + physical_to_internet: String, + physical_to_network: String, + queue_count: u32, + mtu: u32, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +struct ShimPhysicalInterfaces { + to_internet: String, + to_network: String, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +struct CommandSpec { + program: &'static str, + args: Vec, +} + +impl CommandSpec { + fn ip(args: impl IntoIterator>) -> Self { + Self { + program: IP_COMMAND, + args: args.into_iter().map(Into::into).collect(), + } + } + + fn display(&self) -> String { + format!("{} {}", self.program, self.args.join(" ")) + } +} + +/// Errors returned while planning or creating the interface compatibility shim. +#[derive(Debug, Error)] +pub(crate) enum CompatibilityShimError { + /// The bridge configuration cannot form a safe shim topology. + #[error("invalid interface compatibility shim configuration: {0}")] + InvalidConfig(String), + /// The selected shaping CPU count cannot be represented by iproute2. + #[error("interface compatibility shim queue count is too large")] + QueueCount(#[from] TryFromIntError), + /// A physical interface MTU could not be read. + #[error("unable to read MTU from {path}: {source}")] + ReadMtu { + /// Sysfs path that could not be read. + path: PathBuf, + /// Underlying filesystem error. + source: std::io::Error, + }, + /// A physical interface MTU was not a valid integer. + #[error("unable to parse MTU from {path}: {source}")] + ParseMtu { + /// Sysfs path containing the invalid value. + path: PathBuf, + /// Integer parsing error. + source: ParseIntError, + }, + /// iproute2 could not be started. + #[error("unable to run {command}: {source}")] + Launch { + /// Command that failed to launch. + command: String, + /// Underlying process-launch error. + source: std::io::Error, + }, + /// iproute2 returned a failure status. + #[error("{command} failed with status {status}: {stderr}")] + CommandFailed { + /// Command that returned a failure status. + command: String, + /// Process status code, if the process reported one. + status: String, + /// Standard error emitted by iproute2. + stderr: String, + }, +} + +/// Owns the active compatibility shim and removes it when startup unwinds. +pub(crate) struct CompatibilityShimGuard { + active: bool, +} + +impl CompatibilityShimGuard { + /// Returns whether this guard owns an active compatibility shim. + pub(crate) fn is_active(&self) -> bool { + self.active + } +} + +impl Drop for CompatibilityShimGuard { + fn drop(&mut self) { + if self.active { + cleanup(); + } + } +} + +fn read_interface_mtu(interface: &str) -> Result { + let path = PathBuf::from(format!("/sys/class/net/{interface}/mtu")); + let raw = std::fs::read_to_string(&path).map_err(|source| CompatibilityShimError::ReadMtu { + path: path.clone(), + source, + })?; + raw.trim() + .parse::() + .map_err(|source| CompatibilityShimError::ParseMtu { path, source }) +} + +fn reserved_interface_names() -> [&'static str; 6] { + [ + SHIM_INTERNET_LQOS, + SHIM_INTERNET_PEER, + SHIM_NETWORK_LQOS, + SHIM_NETWORK_PEER, + SHIM_INTERNET_BRIDGE, + SHIM_NETWORK_BRIDGE, + ] +} + +fn planned_queue_count( + config: &Config, + shaping_cpu_count: usize, +) -> Result { + let requested_queue_count = config + .queues + .override_available_queues + .map(|count| count as usize) + .unwrap_or(shaping_cpu_count); + Ok(u32::try_from( + requested_queue_count.min(shaping_cpu_count).max(2), + )?) +} + +fn physical_interfaces_from_config( + config: &Config, +) -> Result, CompatibilityShimError> { + let Some(bridge) = config.bridge.as_ref() else { + return Ok(None); + }; + if !bridge.compatibility_shim_enabled() { + return Ok(None); + } + bridge + .validate_compatibility_shim() + .map_err(|message| CompatibilityShimError::InvalidConfig(message.to_string()))?; + + let physical_to_internet = bridge.to_internet.trim(); + let physical_to_network = bridge.to_network.trim(); + if physical_to_internet.is_empty() || physical_to_network.is_empty() { + return Err(CompatibilityShimError::InvalidConfig( + "both physical interface names are required".to_string(), + )); + } + if physical_to_internet == physical_to_network { + return Err(CompatibilityShimError::InvalidConfig( + "physical interface names must be different".to_string(), + )); + } + if reserved_interface_names() + .iter() + .any(|reserved| *reserved == physical_to_internet || *reserved == physical_to_network) + { + return Err(CompatibilityShimError::InvalidConfig( + "physical interfaces cannot use LibreQoS shim device names".to_string(), + )); + } + + Ok(Some(ShimPhysicalInterfaces { + to_internet: physical_to_internet.to_string(), + to_network: physical_to_network.to_string(), + })) +} + +fn topology_with_mtu( + config: &Config, + physical: ShimPhysicalInterfaces, + mut read_mtu: impl FnMut(&str) -> Result, +) -> Result { + let shaping_cpu_count = lqos_config::detect_shaping_cpus(config) + .shaping + .len() + .max(1); + let queue_count = planned_queue_count(config, shaping_cpu_count)?; + let mtu = read_mtu(&physical.to_internet)?.min(read_mtu(&physical.to_network)?); + + Ok(ShimTopology { + physical_to_internet: physical.to_internet, + physical_to_network: physical.to_network, + queue_count, + mtu, + }) +} + +fn create_veth_commands(name: &str, peer: &str, queues: u32) -> Vec { + let queue_count = queues.to_string(); + vec![CommandSpec::ip([ + "link", + "add", + "name", + name, + "numrxqueues", + queue_count.as_str(), + "numtxqueues", + queue_count.as_str(), + "type", + "veth", + "peer", + "name", + peer, + "numrxqueues", + queue_count.as_str(), + "numtxqueues", + queue_count.as_str(), + ])] +} + +fn set_link_value(device: &str, field: &str, value: impl ToString) -> CommandSpec { + CommandSpec::ip([ + "link".to_string(), + "set".to_string(), + "dev".to_string(), + device.to_string(), + field.to_string(), + value.to_string(), + ]) +} + +fn set_link_up(device: &str) -> CommandSpec { + CommandSpec::ip(["link", "set", "dev", device, "up"]) +} + +fn create_bridge_commands(name: &str, members: [&str; 2], mtu: u32) -> Vec { + let mut commands = vec![ + CommandSpec::ip([ + "link", + "add", + "name", + name, + "type", + "bridge", + "stp_state", + "0", + "vlan_filtering", + "0", + "mcast_snooping", + "0", + ]), + set_link_value(name, "mtu", mtu), + ]; + for member in members { + commands.push(set_link_value(member, "master", name)); + commands.push(set_link_up(member)); + } + commands.push(set_link_up(name)); + commands +} + +fn setup_commands(topology: &ShimTopology) -> Vec { + let mut commands = Vec::new(); + commands.extend(create_veth_commands( + SHIM_INTERNET_LQOS, + SHIM_INTERNET_PEER, + topology.queue_count, + )); + commands.extend(create_veth_commands( + SHIM_NETWORK_LQOS, + SHIM_NETWORK_PEER, + topology.queue_count, + )); + for veth in [ + SHIM_INTERNET_LQOS, + SHIM_INTERNET_PEER, + SHIM_NETWORK_LQOS, + SHIM_NETWORK_PEER, + ] { + commands.push(set_link_value(veth, "mtu", topology.mtu)); + commands.push(set_link_up(veth)); + } + commands.extend(create_bridge_commands( + SHIM_INTERNET_BRIDGE, + [SHIM_INTERNET_PEER, topology.physical_to_internet.as_str()], + topology.mtu, + )); + commands.extend(create_bridge_commands( + SHIM_NETWORK_BRIDGE, + [SHIM_NETWORK_PEER, topology.physical_to_network.as_str()], + topology.mtu, + )); + commands +} + +fn cleanup_commands() -> Vec { + vec![ + CommandSpec::ip(["link", "delete", SHIM_INTERNET_BRIDGE, "type", "bridge"]), + CommandSpec::ip(["link", "delete", SHIM_NETWORK_BRIDGE, "type", "bridge"]), + CommandSpec::ip(["link", "delete", SHIM_INTERNET_LQOS, "type", "veth"]), + CommandSpec::ip(["link", "delete", SHIM_NETWORK_LQOS, "type", "veth"]), + ] +} + +fn run_checked(command: &CommandSpec) -> Result<(), CompatibilityShimError> { + let output = Command::new(command.program) + .args(&command.args) + .stdout(Stdio::null()) + .stderr(Stdio::piped()) + .output() + .map_err(|source| CompatibilityShimError::Launch { + command: command.display(), + source, + })?; + if output.status.success() { + return Ok(()); + } + + Err(CompatibilityShimError::CommandFailed { + command: command.display(), + status: output + .status + .code() + .map(|code| code.to_string()) + .unwrap_or_else(|| "terminated by signal".to_string()), + stderr: String::from_utf8_lossy(&output.stderr).trim().to_string(), + }) +} + +/// Removes any LibreQoS interface compatibility shim devices. +/// +/// This function changes host network-device state. Missing devices and other +/// cleanup failures are logged and ignored so a later setup can recover. +pub(crate) fn cleanup() { + cleanup_with(&mut run_checked); +} + +fn cleanup_with(run_command: &mut impl FnMut(&CommandSpec) -> Result<(), CompatibilityShimError>) { + for command in cleanup_commands() { + match run_command(&command) { + Ok(()) => debug!(command = %command.display(), "Removed compatibility shim device"), + Err(error) => debug!(%error, "Compatibility shim cleanup command did not succeed"), + } + } +} + +fn prepare_with( + config: &Config, + read_mtu: impl FnMut(&str) -> Result, + mut run_command: impl FnMut(&CommandSpec) -> Result<(), CompatibilityShimError>, +) -> Result { + let physical = physical_interfaces_from_config(config)?; + cleanup_with(&mut run_command); + let Some(physical) = physical else { + return Ok(false); + }; + let topology = topology_with_mtu(config, physical, read_mtu)?; + + info!( + internet_interface = %topology.physical_to_internet, + network_interface = %topology.physical_to_network, + queues = topology.queue_count, + mtu = topology.mtu, + "Creating LibreQoS interface compatibility shim" + ); + for command in setup_commands(&topology) { + if let Err(error) = run_command(&command) { + warn!(%error, "Unable to create interface compatibility shim; rolling back"); + cleanup_with(&mut run_command); + return Err(error); + } + } + + Ok(true) +} + +/// Removes stale shim devices, then creates the configured compatibility shim. +/// +/// This function changes host network-device state by creating veth pairs, +/// Linux bridges, and bridge membership. The returned guard removes those +/// devices if startup unwinds or the daemon shuts down normally. +pub(crate) fn prepare(config: &Config) -> Result { + let active = prepare_with(config, read_interface_mtu, run_checked)?; + Ok(CompatibilityShimGuard { active }) +} + +#[cfg(test)] +mod tests { + use super::{ + CommandSpec, CompatibilityShimError, ShimTopology, cleanup_commands, planned_queue_count, + prepare_with, setup_commands, + }; + use lqos_config::{ + BridgeConfig, Config, SHIM_INTERNET_BRIDGE, SHIM_INTERNET_LQOS, SHIM_NETWORK_BRIDGE, + SHIM_NETWORK_LQOS, + }; + + fn topology() -> ShimTopology { + ShimTopology { + physical_to_internet: "bond0".to_string(), + physical_to_network: "enp2s0".to_string(), + queue_count: 8, + mtu: 9000, + } + } + + fn command_text(command: &CommandSpec) -> String { + command.display() + } + + fn enabled_config() -> Config { + Config { + bridge: Some(BridgeConfig { + use_xdp_bridge: true, + to_internet: "bond0".to_string(), + to_network: "enp2s0".to_string(), + compatibility_shim: true, + ..BridgeConfig::default() + }), + ..Config::default() + } + } + + #[test] + fn setup_plan_uses_two_multiqueue_veth_pairs_and_two_bridges() { + let commands = setup_commands(&topology()); + let command_text = commands.iter().map(command_text).collect::>(); + + assert_eq!( + command_text + .iter() + .filter(|command| command.contains(" type veth peer ")) + .count(), + 2 + ); + assert_eq!( + command_text + .iter() + .filter(|command| command.contains(" type bridge ")) + .count(), + 2 + ); + assert!( + command_text + .iter() + .filter(|command| command.contains(" type veth peer ")) + .all(|command| command.contains("numrxqueues 8 numtxqueues 8")) + ); + assert!(command_text.iter().any(|command| { + command.contains(SHIM_INTERNET_BRIDGE) + && command.contains("stp_state 0") + && command.contains("mcast_snooping 0") + })); + assert!(command_text.iter().any(|command| { + command.contains(SHIM_NETWORK_BRIDGE) + && command.contains("stp_state 0") + && command.contains("mcast_snooping 0") + })); + assert!(command_text.iter().any(|command| { + command.contains(SHIM_INTERNET_LQOS) && command.ends_with("mtu 9000") + })); + assert!(command_text.iter().any(|command| { + command.contains(SHIM_NETWORK_LQOS) && command.ends_with("mtu 9000") + })); + } + + #[test] + fn cleanup_removes_bridges_before_veth_pairs() { + let commands = cleanup_commands(); + + assert!(commands[0].display().contains(SHIM_INTERNET_BRIDGE)); + assert!(commands[1].display().contains(SHIM_NETWORK_BRIDGE)); + assert!(commands[2].display().contains(SHIM_INTERNET_LQOS)); + assert!(commands[3].display().contains(SHIM_NETWORK_LQOS)); + } + + #[test] + fn queue_count_honors_the_smaller_of_override_and_shaping_cpus() { + let mut config = enabled_config(); + config.queues.override_available_queues = Some(8); + assert_eq!(planned_queue_count(&config, 4).expect("queue count"), 4); + + config.queues.override_available_queues = Some(3); + assert_eq!(planned_queue_count(&config, 8).expect("queue count"), 3); + + config.queues.override_available_queues = None; + assert_eq!(planned_queue_count(&config, 6).expect("queue count"), 6); + } + + #[test] + fn prepare_uses_the_smaller_physical_mtu() { + let mut commands = Vec::new(); + let active = prepare_with( + &enabled_config(), + |interface| match interface { + "bond0" => Ok(9000), + "enp2s0" => Ok(1500), + _ => unreachable!("unexpected interface"), + }, + |command| { + commands.push(command.display()); + Ok(()) + }, + ) + .expect("shim preparation should succeed"); + + assert!(active); + assert_eq!( + commands[..4], + cleanup_commands() + .iter() + .map(CommandSpec::display) + .collect::>() + ); + assert!( + commands + .iter() + .filter(|command| command.contains(" link set dev ") && command.contains(" mtu ")) + .all(|command| command.ends_with("mtu 1500")) + ); + } + + #[test] + fn prepare_rolls_back_after_partial_setup_failure() { + let mut commands = Vec::new(); + let error = prepare_with( + &enabled_config(), + |_| Ok(1500), + |command| { + let display = command.display(); + commands.push(display.clone()); + if display.contains("link add name v_isp_lq") { + return Err(CompatibilityShimError::CommandFailed { + command: display, + status: "1".to_string(), + stderr: "simulated failure".to_string(), + }); + } + Ok(()) + }, + ) + .expect_err("partial setup should fail"); + + assert!(error.to_string().contains("simulated failure")); + let expected_cleanup = cleanup_commands() + .iter() + .map(CommandSpec::display) + .collect::>(); + assert_eq!(&commands[..4], expected_cleanup.as_slice()); + assert_eq!(&commands[commands.len() - 4..], expected_cleanup.as_slice()); + } + + #[test] + fn invalid_config_is_rejected_before_host_cleanup() { + let mut config = enabled_config(); + config.bridge.as_mut().expect("bridge").to_internet = SHIM_INTERNET_LQOS.to_string(); + let mut commands = Vec::new(); + + let error = prepare_with( + &config, + |_| unreachable!("MTU should not be read for invalid config"), + |command| { + commands.push(command.display()); + Ok(()) + }, + ) + .expect_err("reserved physical name should fail"); + + assert!( + error + .to_string() + .contains("cannot use LibreQoS shim device names") + ); + assert!(commands.is_empty()); + } + + #[test] + fn mtu_read_failure_still_removes_stale_devices() { + let mut commands = Vec::new(); + let error = prepare_with( + &enabled_config(), + |interface| { + Err(CompatibilityShimError::ReadMtu { + path: format!("/sys/class/net/{interface}/mtu").into(), + source: std::io::Error::other("simulated MTU read failure"), + }) + }, + |command| { + commands.push(command.display()); + Ok(()) + }, + ) + .expect_err("MTU read should fail"); + + assert!(error.to_string().contains("simulated MTU read failure")); + assert_eq!( + commands, + cleanup_commands() + .iter() + .map(CommandSpec::display) + .collect::>() + ); + } +} diff --git a/src/rust/lqosd/src/main.rs b/src/rust/lqosd/src/main.rs index 1676fab6..76ba282a 100644 --- a/src/rust/lqosd/src/main.rs +++ b/src/rust/lqosd/src/main.rs @@ -6,6 +6,7 @@ mod blackboard; mod dynamic_circuits; mod file_lock; +mod interface_shim; mod ip_mapping; #[cfg(feature = "equinix_tests")] mod lqos_daht_test; @@ -243,13 +244,23 @@ fn main() -> Result<()> { license_cache_ready_tx, ); + // The compatibility shim must exist before interface tuning and preflight. + let mut compatibility_shim = Some(interface_shim::prepare(&config)?); + info!( + active = compatibility_shim + .as_ref() + .is_some_and(interface_shim::CompatibilityShimGuard::is_active), + "Interface compatibility shim state prepared" + ); + // Apply Tunings tuning::tune_lqosd_from_config_file()?; // Spawn tracking sub-systems let Ok(control_channel) = lts2_sys::control_channel::init_control_channel() else { - error!("Failed to initialize Insight control channel, exiting."); - std::process::exit(1); + return Err(anyhow::anyhow!( + "Failed to initialize the Insight control channel" + )); }; let control_tx_for_lts = control_channel.tx.clone(); let control_tx_for_web = control_channel.tx.clone(); @@ -371,6 +382,15 @@ fn main() -> Result<()> { }, }; + if kernels.is_none() + && compatibility_shim + .as_ref() + .is_some_and(interface_shim::CompatibilityShimGuard::is_active) + { + warn!("Removing the compatibility shim because shaping did not start"); + std::mem::drop(compatibility_shim.take()); + } + let control_tx_for_webserver = control_tx_for_web.clone(); let system_usage_tx_for_web = system_usage_tx.clone(); let shaping_started = stormguard_bakery_sender.is_some(); @@ -492,6 +512,7 @@ fn main() -> Result<()> { std::mem::drop(kernels); // Give kernel/driver a moment to finalize detach thread::sleep(Duration::from_millis(50)); + std::mem::drop(compatibility_shim); UnixSocketServer::signal_cleanup(); std::mem::drop(file_lock); std::process::exit(0); diff --git a/src/rust/lqosd/src/node_manager/js_build/src/config/config_interface_shim.test.mjs b/src/rust/lqosd/src/node_manager/js_build/src/config/config_interface_shim.test.mjs new file mode 100644 index 00000000..fa219db9 --- /dev/null +++ b/src/rust/lqosd/src/node_manager/js_build/src/config/config_interface_shim.test.mjs @@ -0,0 +1,112 @@ +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + bridgeEligibilityField, + normalizedBridgeFlags, + saveManualXdpConfiguration, + usesManualXdpWorkflow, +} from "./network_mode_shim.mjs"; + +test("compatibility shim selects relaxed bridge eligibility", () => { + assert.equal(bridgeEligibilityField("bridge", false), "bridge_eligible"); + assert.equal(bridgeEligibilityField("bridge", true), "compatibility_shim_eligible"); + assert.equal(bridgeEligibilityField("single", true), "single_interface_eligible"); +}); + +test("compatibility shim always enables the XDP bridge", () => { + assert.deepEqual(normalizedBridgeFlags(false, true), { + use_xdp_bridge: true, + compatibility_shim: true, + }); + assert.deepEqual(normalizedBridgeFlags(false, false), { + use_xdp_bridge: false, + compatibility_shim: false, + }); +}); + +test("XDP and compatibility-shim configs use config-only saves", () => { + assert.equal(usesManualXdpWorkflow({ bridge: { use_xdp_bridge: true } }), true); + assert.equal(usesManualXdpWorkflow({ bridge: { use_xdp_bridge: false } }), false); + assert.equal(usesManualXdpWorkflow({ single_interface: {} }), false); +}); + +test("manual XDP save completes the page workflow and keeps the saved candidate", async () => { + const previousConfig = { bridge: { use_xdp_bridge: false } }; + const candidate = { bridge: { use_xdp_bridge: true, compatibility_shim: true } }; + const configState = { config: previousConfig }; + let savedConfig = null; + const effects = []; + + const result = await saveManualXdpConfiguration({ + candidate, + configState, + persistConfig: (onSuccess) => { + savedConfig = configState.config; + effects.push("persist"); + onSuccess({ ok: true }); + }, + clearDraft: () => effects.push("clear-draft"), + notifySaved: () => effects.push("notify"), + refresh: async () => effects.push("refresh"), + }); + + assert.equal(savedConfig, candidate); + assert.equal(configState.config, candidate); + assert.deepEqual(effects, ["persist", "clear-draft", "notify", "refresh"]); + assert.equal(result.refreshError, null); +}); + +test("manual XDP save restores config and skips success effects after failure", async () => { + const previousConfig = { bridge: { use_xdp_bridge: false } }; + const candidate = { bridge: { use_xdp_bridge: true, compatibility_shim: true } }; + const expectedError = new Error("save failed"); + const configState = { config: previousConfig }; + const effects = []; + + await assert.rejects( + saveManualXdpConfiguration({ + candidate, + configState, + persistConfig: (_onSuccess, onError) => { + assert.equal(configState.config, candidate); + effects.push("persist"); + onError(expectedError); + }, + clearDraft: () => effects.push("clear-draft"), + notifySaved: () => effects.push("notify"), + refresh: async () => effects.push("refresh"), + }), + expectedError, + ); + + assert.equal(configState.config, previousConfig); + assert.deepEqual(effects, ["persist"]); +}); + +test("manual XDP save keeps saved config when the status refresh fails", async () => { + const previousConfig = { bridge: { use_xdp_bridge: false } }; + const candidate = { bridge: { use_xdp_bridge: true, compatibility_shim: true } }; + const configState = { config: previousConfig }; + const refreshError = new Error("refresh failed"); + const effects = []; + + const result = await saveManualXdpConfiguration({ + candidate, + configState, + persistConfig: (onSuccess) => { + effects.push("persist"); + onSuccess({ ok: true }); + }, + clearDraft: () => effects.push("clear-draft"), + notifySaved: () => effects.push("notify"), + refresh: async () => { + effects.push("refresh"); + throw refreshError; + }, + }); + + assert.equal(configState.config, candidate); + assert.equal(result.refreshError, refreshError); + assert.deepEqual(effects, ["persist", "clear-draft", "notify", "refresh"]); +}); diff --git a/src/rust/lqosd/src/node_manager/js_build/src/config/network_mode_shim.mjs b/src/rust/lqosd/src/node_manager/js_build/src/config/network_mode_shim.mjs new file mode 100644 index 00000000..20f6f8d9 --- /dev/null +++ b/src/rust/lqosd/src/node_manager/js_build/src/config/network_mode_shim.mjs @@ -0,0 +1,42 @@ +export function bridgeEligibilityField(modeKey, compatibilityShimEnabled) { + if (modeKey === "single") return "single_interface_eligible"; + return compatibilityShimEnabled ? "compatibility_shim_eligible" : "bridge_eligible"; +} + +export function normalizedBridgeFlags(useXdpBridge, compatibilityShim) { + const shimEnabled = Boolean(compatibilityShim); + return { + use_xdp_bridge: shimEnabled || Boolean(useXdpBridge), + compatibility_shim: shimEnabled, + }; +} + +export function usesManualXdpWorkflow(config) { + return Boolean(config?.bridge?.use_xdp_bridge); +} + +export async function saveManualXdpConfiguration({ + candidate, + configState, + persistConfig, + clearDraft, + notifySaved, + refresh, +}) { + const previousConfig = configState.config; + configState.config = candidate; + try { + await new Promise((resolve, reject) => persistConfig(resolve, reject)); + } catch (error) { + configState.config = previousConfig; + throw error; + } + clearDraft(); + notifySaved(); + try { + await refresh(); + return { refreshError: null }; + } catch (refreshError) { + return { refreshError }; + } +} diff --git a/src/rust/lqosd/src/node_manager/js_build/src/config_interface.js b/src/rust/lqosd/src/node_manager/js_build/src/config_interface.js index 833859c6..daf3753a 100644 --- a/src/rust/lqosd/src/node_manager/js_build/src/config_interface.js +++ b/src/rust/lqosd/src/node_manager/js_build/src/config_interface.js @@ -1,5 +1,11 @@ -import {loadConfig, renderConfigMenu} from "./config/config_helper"; +import {loadConfig, renderConfigMenu, saveConfig} from "./config/config_helper"; import {hydrateDraftMtu, MTU_MAX, MTU_MIN, parseOptionalMtu} from "./config/network_mode_mtu.mjs"; +import { + bridgeEligibilityField, + normalizedBridgeFlags, + saveManualXdpConfiguration, + usesManualXdpWorkflow, +} from "./config/network_mode_shim.mjs"; import { clearNetworkModeState, DRAFT_KEY, @@ -151,6 +157,20 @@ function updateBridgeMtuState() { } } +function compatibilityShimEnabled() { + return document.getElementById("compatibilityShim")?.checked ?? false; +} + +function updateCompatibilityShimState() { + const useXdpBridge = document.getElementById("useXdpBridge"); + if (!useXdpBridge) return; + const shimEnabled = compatibilityShimEnabled(); + if (shimEnabled) { + useXdpBridge.checked = true; + } + useXdpBridge.disabled = shimEnabled || Boolean(currentInspection?.editing_locked); +} + function updateSingleInterfaceMtuState() { const singleInterfaceMtu = document.getElementById("singleInterfaceMtu"); const singleInterfaceMtuHelp = document.getElementById("singleInterfaceMtuHelp"); @@ -167,6 +187,16 @@ function updateMtuState() { updateSingleInterfaceMtuState(); } +function updateApplyButtonState() { + const applyButton = document.getElementById("applyButton"); + if (!applyButton) return; + const manualXdpSave = usesManualXdpWorkflow(buildCandidateConfig()); + applyButton.disabled = manualXdpSave + ? Boolean(currentInspection?.editing_locked) + : !currentInspection?.can_apply; + applyButton.textContent = manualXdpSave ? "Save XDP Configuration" : "Apply Network Changes"; +} + function validateConfig() { clearMtuErrors(); if (document.getElementById("bridgeMode").checked) { @@ -217,8 +247,12 @@ function buildCandidateConfig() { next.single_interface = null; if (document.getElementById("bridgeMode").checked) { + const bridgeFlags = normalizedBridgeFlags( + document.getElementById("useXdpBridge").checked, + compatibilityShimEnabled(), + ); next.bridge = { - use_xdp_bridge: document.getElementById("useXdpBridge").checked, + ...bridgeFlags, to_internet: document.getElementById("toInternet").value.trim(), to_network: document.getElementById("toNetwork").value.trim(), mtu: currentBridgeMtuValue(), @@ -239,9 +273,9 @@ function interfaceCandidates() { return Array.isArray(currentInspection?.interface_candidates) ? currentInspection.interface_candidates : []; } -function optionLabel(candidate, selectedValue) { +function optionLabel(candidate, selectedValue, eligibilityField) { if (!candidate) return selectedValue; - if (candidate.bridge_eligible || candidate.single_interface_eligible) { + if (candidate[eligibilityField]) { return candidate.name; } if (candidate.current_selection || candidate.name === selectedValue) { @@ -252,7 +286,7 @@ function optionLabel(candidate, selectedValue) { function buildSelectOptions(selectElement, modeKey, selectedValue, excludedValue = null) { if (!selectElement) return; - const eligibilityField = modeKey === "single" ? "single_interface_eligible" : "bridge_eligible"; + const eligibilityField = bridgeEligibilityField(modeKey, compatibilityShimEnabled()); const candidates = interfaceCandidates(); const options = [``]; const seen = new Set(); @@ -263,7 +297,7 @@ function buildSelectOptions(selectElement, modeKey, selectedValue, excludedValue if (!eligible && !selected) return; if (excludedValue && candidate.name === excludedValue && !selected) return; options.push( - `` + `` ); seen.add(candidate.name); }); @@ -282,7 +316,7 @@ function renderInterfaceHelp(helpElementId, modeKey, selectedValues = {}) { const element = document.getElementById(helpElementId); if (!element) return; - const eligibilityField = modeKey === "single" ? "single_interface_eligible" : "bridge_eligible"; + const eligibilityField = bridgeEligibilityField(modeKey, compatibilityShimEnabled()); const selectedSet = new Set( Object.values(selectedValues) .map((value) => String(value || "").trim()) @@ -348,10 +382,10 @@ function renderInterfaceSelectors(config = null) { } function populateFormFromConfig(config) { - renderInterfaceSelectors(config); if (config?.bridge) { document.getElementById("bridgeMode").checked = true; document.getElementById("useXdpBridge").checked = config.bridge.use_xdp_bridge ?? true; + document.getElementById("compatibilityShim").checked = config.bridge.compatibility_shim ?? false; document.getElementById("toInternet").value = config.bridge.to_internet ?? ""; document.getElementById("toNetwork").value = config.bridge.to_network ?? ""; document.getElementById("bridgeMtu").value = config.bridge.mtu ?? ""; @@ -363,9 +397,12 @@ function populateFormFromConfig(config) { document.getElementById("singleInterfaceMtu").value = config.single_interface.mtu ?? ""; } + updateCompatibilityShimState(); + renderInterfaceSelectors(config); const event = new Event("change"); document.querySelector('input[name="networkMode"]:checked')?.dispatchEvent(event); updateMtuState(); + updateApplyButtonState(); } function loadDraft() { @@ -686,6 +723,7 @@ function renderInspection(inspection) { "bridgeMode", "singleInterfaceMode", "useXdpBridge", + "compatibilityShim", "toInternet", "toNetwork", "bridgeMtu", @@ -700,10 +738,11 @@ function renderInspection(inspection) { element.disabled = Boolean(inspection?.editing_locked); } }); - applyButton.disabled = !inspection?.can_apply; adoptButton.disabled = !inspection?.can_adopt; takeoverButton.disabled = !inspection?.can_take_over; + updateCompatibilityShimState(); updateMtuState(); + updateApplyButtonState(); } function renderHelperStatus(status) { @@ -810,6 +849,39 @@ function confirmDangerousChange(actionLabel, candidate) { function applyNetworkChanges(mode = "Apply") { if (!validateConfig()) return; const candidate = buildCandidateConfig(); + if (usesManualXdpWorkflow(candidate)) { + if (mode !== "Apply") return; + const confirmed = window.confirm( + "Save this XDP configuration to lqos.conf? LibreQoS will not change Netplan. Restart lqosd to rebuild the shaping path with these settings." + ); + if (!confirmed) return; + const button = document.getElementById("applyButton"); + if (button) { + button.disabled = true; + button.textContent = "Saving..."; + } + saveManualXdpConfiguration({ + candidate, + configState: window, + persistConfig: saveConfig, + clearDraft: () => clearNetworkModeState(DRAFT_KEY), + notifySaved: () => { + alert("XDP configuration saved. Netplan was not changed. Restart lqosd to apply the interface configuration."); + }, + refresh: () => { + updateApplyButtonState(); + return inspectCandidate(); + }, + }).then(({ refreshError }) => { + if (refreshError) { + alert("XDP configuration was saved, but the current network status could not be refreshed."); + } + }).catch((err) => { + updateApplyButtonState(); + alert(err?.message || "Unable to save XDP configuration"); + }); + return; + } const actionLabel = mode === "Adopt" ? "Adopt into libreqos.yaml" : mode === "TakeOver" ? "Take Over libreqos.yaml" : "Apply Network Changes"; if (!confirmDangerousChange(actionLabel, candidate)) { return; @@ -934,9 +1006,19 @@ function wireActions() { ].forEach((id) => { document.getElementById(id).addEventListener("change", () => { renderInterfaceSelectors(); + updateApplyButtonState(); }); }); - document.getElementById("useXdpBridge").addEventListener("change", updateMtuState); + document.getElementById("useXdpBridge").addEventListener("change", () => { + updateMtuState(); + updateApplyButtonState(); + }); + document.getElementById("compatibilityShim").addEventListener("change", () => { + updateCompatibilityShimState(); + updateMtuState(); + renderInterfaceSelectors(); + updateApplyButtonState(); + }); [ "bridgeMtu", "singleInterfaceMtu", @@ -953,9 +1035,13 @@ function wireActions() { }); document.getElementById("saveButton").addEventListener("click", () => { if (!validateConfig()) return; - saveDraft(); + const draft = saveDraft(); inspectCandidate(); - alert("Network mode draft saved for this browser tab. Use Apply Network Changes to commit both lqos.conf and netplan together."); + if (usesManualXdpWorkflow(draft)) { + alert("XDP draft saved for this browser tab. Use Save XDP Configuration to update lqos.conf; Netplan remains unchanged."); + } else { + alert("Network mode draft saved for this browser tab. Use Apply Network Changes to commit both lqos.conf and netplan together."); + } }); document.getElementById("inspectButton").addEventListener("click", inspectCandidate); document.getElementById("applyButton").addEventListener("click", () => applyNetworkChanges("Apply")); diff --git a/src/rust/lqosd/src/node_manager/js_build/test-node-manager.sh b/src/rust/lqosd/src/node_manager/js_build/test-node-manager.sh index c75f0496..63e624f5 100755 --- a/src/rust/lqosd/src/node_manager/js_build/test-node-manager.sh +++ b/src/rust/lqosd/src/node_manager/js_build/test-node-manager.sh @@ -20,6 +20,7 @@ node --test \ "${SCRIPT_DIR}/src/config/shaped_device_wire.test.mjs" \ "${SCRIPT_DIR}/src/config_radius_accounting_contract.test.mjs" \ "${SCRIPT_DIR}/src/config/config_interface_mtu.test.mjs" \ + "${SCRIPT_DIR}/src/config/config_interface_shim.test.mjs" \ "${SCRIPT_DIR}/src/config/local_api_token.test.mjs" \ "${SCRIPT_DIR}/src/config/network_mode_storage.test.mjs" \ "${SCRIPT_DIR}/src/config/ssl_redirect.test.mjs" \ diff --git a/src/rust/lqosd/src/node_manager/static2/config_interface.html b/src/rust/lqosd/src/node_manager/static2/config_interface.html index aa04f7b1..dd36f6c8 100644 --- a/src/rust/lqosd/src/node_manager/static2/config_interface.html +++ b/src/rust/lqosd/src/node_manager/static2/config_interface.html @@ -55,7 +55,7 @@
Bridge Mode Configuration
-
Choose the two shaping interfaces. Linux bridge mode can manage Netplan; XDP mode leaves Netplan alone.
+
Choose the two shaping interfaces. Linux bridge mode can manage Netplan; XDP and its interface compatibility shim leave Netplan alone.
@@ -63,12 +63,18 @@
XDP does not write Netplan or apply Bridge MTU.
+
+ + +
Use this only for bonded interfaces or NIC drivers that cannot host LibreQoS XDP directly. LibreQoS inserts two multiqueue veth pairs and two small Linux bridges, which adds CPU overhead.
+
+
-
Eligible non-management interfaces with LibreQoS-friendly queue support.
+
Eligible non-management interface for the selected direct or compatibility-shim path.
diff --git a/src/rust/lqosd/src/tuning/mod.rs b/src/rust/lqosd/src/tuning/mod.rs index e5c6ebbf..6bee0dc8 100644 --- a/src/rust/lqosd/src/tuning/mod.rs +++ b/src/rust/lqosd/src/tuning/mod.rs @@ -4,6 +4,24 @@ use lqos_bus::{BusRequest, BusResponse}; use lqos_config::{Config, Tunables}; use lqos_queue_tracker::set_queue_refresh_interval; +#[derive(Debug, PartialEq, Eq)] +struct InterfaceTuningPlan { + full: [String; 2], + coalescing_only: Option<[String; 2]>, +} + +fn interface_tuning_plan(config: &Config) -> InterfaceTuningPlan { + let coalescing_only = config + .bridge + .as_ref() + .filter(|bridge| bridge.compatibility_shim_enabled()) + .map(|bridge| [bridge.to_internet.clone(), bridge.to_network.clone()]); + InterfaceTuningPlan { + full: [config.internet_interface(), config.isp_interface()], + coalescing_only, + } +} + fn apply_non_interface_tuning(tuning: &Tunables) { offloads::bpf_sysctls(); if tuning.set_cpu_governor_performance { @@ -16,8 +34,17 @@ fn apply_non_interface_tuning(tuning: &Tunables) { } fn apply_interface_tuning(config: &Config, tuning: &Tunables) { - offloads::ethtool_tweaks(&config.internet_interface(), tuning); - offloads::ethtool_tweaks(&config.isp_interface(), tuning); + let plan = interface_tuning_plan(config); + for interface in &plan.full { + offloads::ethtool_tweaks(interface, tuning); + } + if let Some(physical_interfaces) = plan.coalescing_only { + // Keep checksum, segmentation, and VLAN offloads enabled on the + // physical path, but retain the configured interrupt-coalescing tune. + for interface in &physical_interfaces { + offloads::ethtool_coalescing_tweaks(interface, tuning); + } + } } pub fn tune_lqosd_from_config_file() -> Result<()> { @@ -41,3 +68,36 @@ pub fn tune_lqosd_from_bus(request: &BusRequest) -> BusResponse { _ => BusResponse::Fail("That wasn't a tuning request".to_string()), } } + +#[cfg(test)] +mod tests { + use super::interface_tuning_plan; + use lqos_config::{BridgeConfig, Config, SHIM_INTERNET_LQOS, SHIM_NETWORK_LQOS}; + + fn bridge_config(compatibility_shim: bool) -> Config { + Config { + bridge: Some(BridgeConfig { + use_xdp_bridge: true, + to_internet: "bond0".to_string(), + to_network: "enp2s0".to_string(), + compatibility_shim, + ..BridgeConfig::default() + }), + ..Config::default() + } + } + + #[test] + fn compatibility_shim_keeps_full_offload_tuning_on_veths() { + let direct = interface_tuning_plan(&bridge_config(false)); + assert_eq!(direct.full, ["bond0", "enp2s0"]); + assert_eq!(direct.coalescing_only, None); + + let shim = interface_tuning_plan(&bridge_config(true)); + assert_eq!(shim.full, [SHIM_INTERNET_LQOS, SHIM_NETWORK_LQOS]); + assert_eq!( + shim.coalescing_only, + Some(["bond0".to_string(), "enp2s0".to_string()]) + ); + } +} diff --git a/src/rust/lqosd/src/tuning/offloads.rs b/src/rust/lqosd/src/tuning/offloads.rs index 25879aa8..bd784d99 100644 --- a/src/rust/lqosd/src/tuning/offloads.rs +++ b/src/rust/lqosd/src/tuning/offloads.rs @@ -78,6 +78,25 @@ pub fn ethtool_tweaks(interface: &str, tuning: &Tunables) { disable_individual_offload(interface, feature); } + ethtool_coalescing_tweaks(interface, tuning); + + if tuning.disable_rxvlan { + let _ = Command::new("/sbin/ethtool") + .args(["-K", interface, "rxvlan", "off"]) + .output(); + } + + if tuning.disable_txvlan { + let _ = Command::new("/sbin/ethtool") + .args(["-K", interface, "txvlan", "off"]) + .output(); + } +} + +/// Applies interrupt-coalescing settings without changing interface offloads. +/// +/// This function invokes `ethtool` and changes host interface state. +pub(super) fn ethtool_coalescing_tweaks(interface: &str, tuning: &Tunables) { let _ = Command::new("/sbin/ethtool") .args([ "-C", @@ -95,16 +114,4 @@ pub fn ethtool_tweaks(interface: &str, tuning: &Tunables) { &format!("\"{}\"", tuning.tx_usecs), ]) .output(); - - if tuning.disable_rxvlan { - let _ = Command::new("/sbin/ethtool") - .args(["-K", interface, "rxvlan", "off"]) - .output(); - } - - if tuning.disable_txvlan { - let _ = Command::new("/sbin/ethtool") - .args(["-K", interface, "txvlan", "off"]) - .output(); - } }