diff --git a/docs-es/v2.0/integrations-uisp-es.md b/docs-es/v2.0/integrations-uisp-es.md index 22177b1a..acea1a3e 100644 --- a/docs-es/v2.0/integrations-uisp-es.md +++ b/docs-es/v2.0/integrations-uisp-es.md @@ -33,6 +33,7 @@ El ajuste `Root UISP Site` solo es obligatorio en modo de topología `full`. En - `network.json` queda para despliegues DIY o manuales. - Use la WebUI para confirmar que la importación y la profundidad del árbol son las esperadas. - En modo integración, trate ediciones de archivos como temporales. +- En modo `full`, los switches, routers y radios de UISP exportados como ramas AP/dispositivo pueden usar virtualización Queue Auto. Consulte [Nodos virtuales](configuration-advanced-es.md#nodos-virtuales-solo-lógicos). ## Validación en 5 minutos después de cambios UISP diff --git a/docs-es/v2.0/node-manager-ui-es.md b/docs-es/v2.0/node-manager-ui-es.md index 0826efba..015b19be 100644 --- a/docs-es/v2.0/node-manager-ui-es.md +++ b/docs-es/v2.0/node-manager-ui-es.md @@ -35,7 +35,7 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta - En compilaciones UISP `full`, un `integrationUISPbandwidths.csv` heredado se auto-migra a esos overrides en la siguiente ejecución de la integración cuando todavía no existen overrides operativos de tasa; en caso contrario, el CSV se ignora. - Las ediciones de tasa desde la página del árbol requieren sesión de administrador, un `node_id` estable y un nodo no generado. Los nodos generados/sintéticos de la integración permanecen en solo lectura en este editor. - La página del árbol mantiene `Node Details` como una cuadrícula compacta de resumen y coloca los editores de override como filas compactas directamente debajo. -- `Effective Now` muestra una insignia de origen estimado, como Parent, Override, Attachment o Mixed. Pase el cursor o enfoque la insignia para ver los detalles de descarga y subida. +- `Limited By` estima el origen de la velocidad efectiva. Los valores pueden incluir `Base rate`, `Est. Parent`, `Est. Override`, `Est. Attachment`, `Est. Queue` o `Est. Mixed`. Pase el cursor o enfoque el campo para ver los detalles de descarga y subida. - La página del árbol ahora muestra el estado de `Topology Override` solo en modo lectura. Los cambios de padre y adjunto se movieron a `Topology`, y el panel de detalles del árbol enlaza directamente a Topology Manager para el nodo seleccionado. - Los saltos de adjuntos seleccionados en runtime se colapsan fuera de la jerarquía principal del árbol. Cuando un sitio usa actualmente un backhaul o radio path específico, `Node Details` lo muestra como metadato `Active Attachment` en el sitio en lugar de exponer ese adjunto como un nodo propio del árbol. - Los adjuntos alternativos inactivos de backhaul PtP/cableado también se eliminan del árbol runtime. `tree.html` muestra solo la ruta efectiva; los alternativos inactivos quedan visibles en Topology Manager y no como nodos runtime independientes. diff --git a/docs/v2.0/integrations-uisp.md b/docs/v2.0/integrations-uisp.md index 5b0b2497..a6ea303f 100644 --- a/docs/v2.0/integrations-uisp.md +++ b/docs/v2.0/integrations-uisp.md @@ -32,6 +32,8 @@ LibreQoS supports multiple topology strategies for UISP integration to balance C **Performance Note:** When using `full` strategy with large networks, consider using `promote_to_root` to distribute CPU load across multiple cores. +In `full` mode, UISP switches, routers, and radios exported as AP/device branches can use Queue Auto virtualization. See [Advanced Configuration Reference](configuration-advanced.md#queueauto-decision-flow). + ## 5-Minute Validation After UISP Config Changes 1. Run the integration once: diff --git a/docs/v2.0/node-manager-ui.md b/docs/v2.0/node-manager-ui.md index 51ecd5b3..69c52692 100644 --- a/docs/v2.0/node-manager-ui.md +++ b/docs/v2.0/node-manager-ui.md @@ -38,7 +38,7 @@ For the full logical-topology versus queue-topology file flow, see [Topology Dat - On UISP `full` builds, legacy `integrationUISPbandwidths.csv` files are auto-migrated into those operator overrides on the next integration run when no operator rate overrides exist yet; otherwise the CSV is ignored. - Tree-page operator rate edits require an administrator session, a stable node ID, and a non-generated node. Generated/integration-synthetic nodes remain read-only in this editor. - The tree page keeps `Node Details` as a compact summary grid and places the override editors as compact inline rows directly beneath it. -- `Effective Now` shows an estimated source badge, such as Parent, Override, Attachment, or Mixed. Hover or focus the badge to see the download and upload details. +- `Limited By` estimates the source of the effective rate. Values can include `Base rate`, `Est. Parent`, `Est. Override`, `Est. Attachment`, `Est. Queue`, or `Est. Mixed`. Hover or focus it to see the download and upload details. - The tree page now shows topology override state as read-only summary only. Parent and attachment edits have moved to `Topology`, and the tree details panel links directly into Topology Manager for the selected node. - Runtime-selected attachment hops are collapsed out of the main tree hierarchy. When a site is currently using a specific backhaul or radio path, `Node Details` shows that as `Active Attachment` metadata on the site instead of exposing the attachment as its own branch node. - Inactive alternate PtP/wired backhaul attachment stubs are also pruned from the runtime tree. `tree.html` shows the effective path only; dormant alternates remain visible in Topology Manager rather than as standalone runtime nodes. diff --git a/src/rust/Cargo.lock b/src/rust/Cargo.lock index 1988d9ba..f948fa17 100644 --- a/src/rust/Cargo.lock +++ b/src/rust/Cargo.lock @@ -4677,6 +4677,7 @@ dependencies = [ "lqos_bus", "lqos_config", "lqos_overrides", + "lqos_topology", "lqos_topology_compile", "lqos_utils", "petgraph", diff --git a/src/rust/lqos_config/src/lib.rs b/src/rust/lqos_config/src/lib.rs index bb4d4fa7..76a8878d 100644 --- a/src/rust/lqos_config/src/lib.rs +++ b/src/rust/lqos_config/src/lib.rs @@ -81,7 +81,7 @@ pub use topology_editor_state::{ TOPOLOGY_ATTACHMENT_AUTO_ID, TOPOLOGY_EDITOR_STATE_FILENAME, TopologyAllowedParent, TopologyAttachmentHealthStatus, TopologyAttachmentOption, TopologyAttachmentRateSource, TopologyAttachmentRole, TopologyEditorNode, TopologyEditorStateError, TopologyEditorStateFile, - TopologyQueueVisibilityPolicy, topology_editor_state_path, + TopologyQueueVisibilityPolicy, topology_auto_attachment_option, topology_editor_state_path, }; pub use topology_parent_candidates::{ TOPOLOGY_PARENT_CANDIDATES_FILENAME, TopologyParentCandidate, TopologyParentCandidatesError, diff --git a/src/rust/lqos_config/src/topology_editor_state.rs b/src/rust/lqos_config/src/topology_editor_state.rs index defacff0..10320ad0 100644 --- a/src/rust/lqos_config/src/topology_editor_state.rs +++ b/src/rust/lqos_config/src/topology_editor_state.rs @@ -168,6 +168,36 @@ pub struct TopologyAttachmentOption { pub effective_selected: bool, } +/// Builds the standard dynamic attachment option used when LibreQoS selects the attachment. +pub fn topology_auto_attachment_option() -> TopologyAttachmentOption { + TopologyAttachmentOption { + attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(), + attachment_name: "Auto".to_string(), + attachment_kind: "auto".to_string(), + attachment_role: TopologyAttachmentRole::Unknown, + pair_id: None, + peer_attachment_id: None, + peer_attachment_name: None, + capacity_mbps: None, + download_bandwidth_mbps: None, + upload_bandwidth_mbps: None, + transport_cap_mbps: None, + transport_cap_reason: None, + rate_source: TopologyAttachmentRateSource::Unknown, + can_override_rate: false, + rate_override_disabled_reason: None, + has_rate_override: false, + local_probe_ip: None, + remote_probe_ip: None, + probe_enabled: false, + probeable: false, + health_status: TopologyAttachmentHealthStatus::Disabled, + health_reason: None, + suppressed_until_unix: None, + effective_selected: false, + } +} + /// One valid parent target for a topology node, plus allowed attachments below that parent. #[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, Default)] pub struct TopologyAllowedParent { @@ -402,32 +432,7 @@ impl TopologyEditorStateFile { .map(|candidate| TopologyAllowedParent { parent_node_id: candidate.node_id.clone(), parent_node_name: candidate.node_name.clone(), - attachment_options: vec![TopologyAttachmentOption { - attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(), - attachment_name: "Auto".to_string(), - attachment_kind: "auto".to_string(), - attachment_role: TopologyAttachmentRole::Unknown, - pair_id: None, - peer_attachment_id: None, - peer_attachment_name: None, - capacity_mbps: None, - download_bandwidth_mbps: None, - upload_bandwidth_mbps: None, - transport_cap_mbps: None, - transport_cap_reason: None, - rate_source: TopologyAttachmentRateSource::Unknown, - can_override_rate: false, - rate_override_disabled_reason: None, - has_rate_override: false, - local_probe_ip: None, - remote_probe_ip: None, - probe_enabled: false, - probeable: false, - health_status: TopologyAttachmentHealthStatus::Disabled, - health_reason: None, - suppressed_until_unix: None, - effective_selected: false, - }], + attachment_options: vec![topology_auto_attachment_option()], all_attachments_suppressed: false, has_probe_unavailable_attachments: false, }) diff --git a/src/rust/lqos_topology/src/effective_state.rs b/src/rust/lqos_topology/src/effective_state.rs index 2b56a9fe..a1edb94e 100644 --- a/src/rust/lqos_topology/src/effective_state.rs +++ b/src/rust/lqos_topology/src/effective_state.rs @@ -46,35 +46,6 @@ pub fn is_health_state_fresh(config: &Config, health: &TopologyAttachmentHealthS now.saturating_sub(generated_unix) <= health_state_stale_after_seconds(config) } -fn auto_attachment_option() -> TopologyAttachmentOption { - TopologyAttachmentOption { - attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(), - attachment_name: "Auto".to_string(), - attachment_kind: "auto".to_string(), - attachment_role: TopologyAttachmentRole::Unknown, - pair_id: None, - peer_attachment_id: None, - peer_attachment_name: None, - capacity_mbps: None, - download_bandwidth_mbps: None, - upload_bandwidth_mbps: None, - transport_cap_mbps: None, - transport_cap_reason: None, - rate_source: TopologyAttachmentRateSource::Unknown, - can_override_rate: false, - rate_override_disabled_reason: None, - has_rate_override: false, - local_probe_ip: None, - remote_probe_ip: None, - probe_enabled: false, - probeable: false, - health_status: TopologyAttachmentHealthStatus::Disabled, - health_reason: None, - suppressed_until_unix: None, - effective_selected: false, - } -} - fn overlay_manual_groups( canonical: &TopologyEditorStateFile, overrides: &TopologyOverridesFile, @@ -87,7 +58,7 @@ fn overlay_manual_groups( else { continue; }; - let mut options = vec![auto_attachment_option()]; + let mut options = vec![topology_auto_attachment_option()]; for attachment in &group.attachments { let local_probe_ip = parse_probe_ip(&attachment.local_probe_ip); let remote_probe_ip = parse_probe_ip(&attachment.remote_probe_ip); diff --git a/src/rust/lqos_topology/src/lib.rs b/src/rust/lqos_topology/src/lib.rs index dea76b55..cffbd1f8 100644 --- a/src/rust/lqos_topology/src/lib.rs +++ b/src/rust/lqos_topology/src/lib.rs @@ -17,7 +17,7 @@ use lqos_config::{ TopologyShapingInputsFile, TopologyShapingResolutionSource, circuit_anchors_path, compute_effective_network_generation, detect_shaping_cpus, plan_top_level_assignments, topology_effective_network_path, topology_effective_state_path, topology_runtime_status_path, - topology_shaping_inputs_path, + topology_auto_attachment_option, topology_shaping_inputs_path, }; use lqos_overrides::{ CircuitAdjustment, NetworkAdjustment, OverrideStore, TopologyAttachmentMode, diff --git a/src/rust/lqos_topology/src/tests/helpers.rs b/src/rust/lqos_topology/src/tests/helpers.rs index 0a60d3ce..1543731f 100644 --- a/src/rust/lqos_topology/src/tests/helpers.rs +++ b/src/rust/lqos_topology/src/tests/helpers.rs @@ -3,7 +3,7 @@ apply_effective_topology_to_canonical_state as try_apply_effective_topology_to_canonical_state, apply_effective_topology_to_network_json as try_apply_effective_topology_to_network_json, apply_effective_topology_to_network_json_from_canonical as try_apply_effective_topology_to_network_json_from_canonical, - apply_health_to_option, auto_attachment_option, build_effective_topology_artifacts, + apply_health_to_option, build_effective_topology_artifacts, build_effective_topology_artifacts_from_canonical, build_shaping_inputs, collect_direct_circuit_node_ids, collect_direct_circuit_node_names, compute_effective_state, parse_probe_ip, publish_effective_topology_artifacts, @@ -18,8 +18,8 @@ TopologyCanonicalRateInputSource, TopologyCanonicalStateFile, TopologyEditorNode, TopologyEditorStateFile, TopologyEffectiveAttachmentState, TopologyEffectiveNodeState, TopologyEffectiveStateFile, TopologyQueueVisibilityPolicy, TopologyRuntimeStatusFile, - topology_effective_network_path, topology_effective_state_path, - topology_runtime_status_path, topology_shaping_inputs_path, + topology_auto_attachment_option as auto_attachment_option, topology_effective_network_path, + topology_effective_state_path, topology_runtime_status_path, topology_shaping_inputs_path, }; use lqos_overrides::{TopologyAttachmentMode, TopologyOverridesFile}; use serde_json::{Value, json}; diff --git a/src/rust/lqos_topology_compile/src/compile_mode.rs b/src/rust/lqos_topology_compile/src/compile_mode.rs index 6432247c..ded849c9 100644 --- a/src/rust/lqos_topology_compile/src/compile_mode.rs +++ b/src/rust/lqos_topology_compile/src/compile_mode.rs @@ -4,8 +4,10 @@ use anyhow::Result; use lqos_config::{ CircuitAnchor, CircuitAnchorsFile, ConfigShapedDevices, ShapedDevice, TopologyCanonicalIngressKind, TopologyCanonicalNode, TopologyCanonicalStateFile, - TopologyEditorNode, TopologyEditorStateFile, TopologyParentCandidatesFile, - topology_ingress_identity_from_tokens, + TopologyAllowedParent, TopologyEditorNode, TopologyEditorStateFile, + TopologyParentCandidatesFile, + TopologyQueueVisibilityPolicy, + topology_auto_attachment_option, topology_ingress_identity_from_tokens, }; use serde_json::{Map, Number, Value}; use std::collections::{BTreeMap, HashMap, HashSet}; @@ -373,6 +375,7 @@ struct SanitizedTopologyIndex { exported_node_names_by_id: HashMap, exported_parent_by_node_id: HashMap>, node_names_by_id: HashMap, + queue_policy_by_node_id: HashMap, } fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopologyIndex { @@ -387,14 +390,19 @@ fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopol .map(|(node_id, node)| (node_id.clone(), node.parent_id.clone())) .collect::>(); + let mut queue_policy_by_node_id = HashMap::new(); if let Some(native) = imported.native_editor.as_ref() { for node in &native.nodes { node_names_by_id.insert(node.node_id.clone(), node.node_name.clone()); + queue_policy_by_node_id + .insert(node.node_id.clone(), node.queue_visibility_policy); } } if let Some(native) = imported.native_canonical.as_ref() { for node in &native.nodes { node_names_by_id.insert(node.node_id.clone(), node.node_name.clone()); + queue_policy_by_node_id + .insert(node.node_id.clone(), node.queue_visibility_policy); } } @@ -402,6 +410,36 @@ fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopol exported_node_names_by_id, exported_parent_by_node_id, node_names_by_id, + queue_policy_by_node_id, + } +} + +fn is_queue_auto_uisp_device_parent( + topology_index: &SanitizedTopologyIndex, + parent_id: &str, +) -> bool { + parent_id.starts_with("uisp:device:") + && topology_index + .queue_policy_by_node_id + .get(parent_id) + .is_some_and(|policy| policy == &TopologyQueueVisibilityPolicy::QueueAuto) +} + +fn preserved_exported_queue_auto_device_parent( + topology_index: &SanitizedTopologyIndex, + node: &TopologyEditorNode, +) -> Option { + let parent_id = topology_index + .exported_parent_by_node_id + .get(&node.node_id) + .and_then(|parent| parent.as_deref())?; + if is_queue_auto_uisp_device_parent(topology_index, parent_id) + && topology_index.node_names_by_id.contains_key(parent_id) + && node.current_attachment_id.as_deref() == Some(parent_id) + { + Some(parent_id.to_string()) + } else { + None } } @@ -424,6 +462,26 @@ fn sanitize_editor_nodes( .node_names_by_id .contains_key(parent.parent_node_id.as_str()) }); + let preserved_device_parent = + preserved_exported_queue_auto_device_parent(topology_index, &node); + if let Some(parent_id) = preserved_device_parent.as_ref() + && !node + .allowed_parents + .iter() + .any(|parent| parent.parent_node_id == *parent_id) + { + node.allowed_parents.push(TopologyAllowedParent { + parent_node_id: parent_id.clone(), + parent_node_name: topology_index + .node_names_by_id + .get(parent_id) + .cloned() + .unwrap_or_else(|| parent_id.clone()), + attachment_options: vec![topology_auto_attachment_option()], + all_attachments_suppressed: false, + has_probe_unavailable_attachments: false, + }); + } let legal_current_parent = node.current_parent_node_id .as_deref() @@ -445,6 +503,9 @@ fn sanitize_editor_nodes( .get(&node.node_id) .and_then(|parent| parent.as_deref()) .and_then(|parent_id| { + if preserved_device_parent.as_deref() == Some(parent_id) { + return Some(parent_id.to_string()); + } if (node.allowed_parents.is_empty() && topology_index.node_names_by_id.contains_key(parent_id)) || node @@ -457,7 +518,11 @@ fn sanitize_editor_nodes( None } }); - let selected_parent = legal_current_parent.or(exported_parent); + let selected_parent = if preserved_device_parent.is_some() { + exported_parent.or(legal_current_parent) + } else { + legal_current_parent.or(exported_parent) + }; node.current_parent_node_name = selected_parent.as_deref().and_then(|parent_id| { node.allowed_parents .iter() @@ -478,6 +543,9 @@ fn sanitize_editor_nodes( node.current_attachment_name = None; } + if let Some(policy) = topology_index.queue_policy_by_node_id.get(&node.node_id) { + node.queue_visibility_policy = *policy; + } node.node_name = exported_name.clone(); Some(node) }) @@ -780,9 +848,9 @@ mod tests { TopologyAttachmentRateSource, TopologyAttachmentRole, TopologyCanonicalIngressKind, TopologyCanonicalStateFile, TopologyEditorNode, TopologyEditorStateFile, TopologyParentCandidate, TopologyParentCandidatesFile, TopologyParentCandidatesNode, - TopologyQueueVisibilityPolicy, + TopologyQueueVisibilityPolicy, topology_auto_attachment_option, }; - use serde_json::json; + use serde_json::{Value, json}; fn shaped_devices(rows: Vec) -> ConfigShapedDevices { let mut shaped = ConfigShapedDevices::default(); @@ -814,6 +882,148 @@ mod tests { } } + fn queue_auto_device_parent_network_json() -> Value { + json!({ + "Root Site": { + "children": { + "Root Switch": { + "children": { + "Child Switch": { + "children": {}, + "id": "uisp:device:child-switch", + "name": "Child Switch", + "type": "AP" + } + }, + "id": "uisp:device:root-switch", + "name": "Root Switch", + "type": "AP" + } + }, + "id": "uisp:site:site-root", + "name": "Root Site", + "type": "Site" + } + }) + } + + fn root_site_editor_node() -> TopologyEditorNode { + TopologyEditorNode { + node_id: "uisp:site:site-root".to_string(), + node_name: "Root Site".to_string(), + latitude: None, + longitude: None, + current_parent_node_id: None, + current_parent_node_name: None, + current_attachment_id: None, + current_attachment_name: None, + can_move: false, + allowed_parents: Vec::new(), + queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueHiddenPromoteChildren, + preferred_attachment_id: None, + preferred_attachment_name: None, + effective_attachment_id: None, + effective_attachment_name: None, + } + } + + fn root_switch_editor_node( + queue_visibility_policy: TopologyQueueVisibilityPolicy, + ) -> TopologyEditorNode { + TopologyEditorNode { + node_id: "uisp:device:root-switch".to_string(), + node_name: "Root Switch".to_string(), + latitude: None, + longitude: None, + current_parent_node_id: Some("uisp:site:site-root".to_string()), + current_parent_node_name: Some("Root Site".to_string()), + current_attachment_id: Some("uisp:site:site-root".to_string()), + current_attachment_name: Some("Root Site".to_string()), + can_move: true, + allowed_parents: Vec::new(), + queue_visibility_policy, + preferred_attachment_id: None, + preferred_attachment_name: None, + effective_attachment_id: None, + effective_attachment_name: None, + } + } + + fn allowed_parent(parent_node_id: &str, parent_node_name: &str) -> TopologyAllowedParent { + TopologyAllowedParent { + parent_node_id: parent_node_id.to_string(), + parent_node_name: parent_node_name.to_string(), + attachment_options: vec![topology_auto_attachment_option()], + all_attachments_suppressed: false, + has_probe_unavailable_attachments: false, + } + } + + fn child_switch_editor_node( + current_attachment_id: &str, + current_attachment_name: &str, + allowed_parents: Vec, + ) -> TopologyEditorNode { + TopologyEditorNode { + node_id: "uisp:device:child-switch".to_string(), + node_name: "Child Switch".to_string(), + latitude: None, + longitude: None, + current_parent_node_id: Some("uisp:site:site-root".to_string()), + current_parent_node_name: Some("Root Site".to_string()), + current_attachment_id: Some(current_attachment_id.to_string()), + current_attachment_name: Some(current_attachment_name.to_string()), + can_move: true, + allowed_parents, + queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueAuto, + preferred_attachment_id: None, + preferred_attachment_name: None, + effective_attachment_id: None, + effective_attachment_name: None, + } + } + + fn queue_auto_device_parent_editor( + root_switch_policy: TopologyQueueVisibilityPolicy, + child_attachment_id: &str, + child_attachment_name: &str, + child_allowed_parents: Vec, + ) -> TopologyEditorStateFile { + TopologyEditorStateFile { + schema_version: 1, + source: "uisp/full2".to_string(), + generated_unix: Some(1), + ingress_identity: None, + nodes: vec![ + root_site_editor_node(), + root_switch_editor_node(root_switch_policy), + child_switch_editor_node( + child_attachment_id, + child_attachment_name, + child_allowed_parents, + ), + ], + } + } + + fn queue_auto_device_parent_bundle( + native_editor: Option, + native_canonical: Option, + ) -> ImportedTopologyBundle { + ImportedTopologyBundle { + source: "uisp/full2".to_string(), + generated_unix: Some(1), + ingress_identity: None, + native_canonical, + native_editor, + parent_candidates: None, + compatibility_network_json: queue_auto_device_parent_network_json(), + shaped_devices: shaped_devices(Vec::new()), + circuit_anchors: CircuitAnchorsFile::default(), + ethernet_advisories: Vec::new(), + } + } + fn imported_bundle() -> ImportedTopologyBundle { let network = json!({ "Site Alpha": { @@ -1117,6 +1327,186 @@ mod tests { assert_eq!(node.current_attachment_name.as_deref(), Some("AP Alpha")); } + #[test] + fn full_mode_preserves_queue_auto_device_parent_from_native_canonical() { + let editor = queue_auto_device_parent_editor( + TopologyQueueVisibilityPolicy::QueueAuto, + "uisp:device:root-switch", + "Root Switch", + vec![allowed_parent("uisp:site:site-root", "Root Site")], + ); + let canonical = TopologyCanonicalStateFile::from_editor_and_network( + &editor, + &queue_auto_device_parent_network_json(), + TopologyCanonicalIngressKind::NativeIntegration, + ); + let imported = queue_auto_device_parent_bundle(None, Some(canonical)); + + let compiled = compile_topology(imported, TopologyCompileMode::Full) + .expect("full compilation should preserve canonical queue-auto device parents"); + let child_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:child-switch") + .expect("child switch should remain in editor state"); + let root_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:root-switch") + .expect("root switch should remain in editor state"); + + assert_eq!( + root_switch.queue_visibility_policy, + TopologyQueueVisibilityPolicy::QueueAuto + ); + assert_eq!( + child_switch.current_parent_node_id.as_deref(), + Some("uisp:device:root-switch") + ); + assert_eq!( + child_switch.current_attachment_id.as_deref(), + Some("uisp:device:root-switch") + ); + assert_eq!( + child_switch.current_attachment_name.as_deref(), + Some("Root Switch") + ); + assert_eq!( + child_switch + .allowed_parents + .iter() + .filter(|parent| parent.parent_node_id == "uisp:device:root-switch") + .count(), + 1 + ); + } + + #[test] + fn full_mode_prefers_native_canonical_queue_policy_over_editor_policy() { + let editor = queue_auto_device_parent_editor( + TopologyQueueVisibilityPolicy::QueueVisible, + "uisp:device:root-switch", + "Root Switch", + vec![allowed_parent("uisp:site:site-root", "Root Site")], + ); + let mut canonical = TopologyCanonicalStateFile::from_editor_and_network( + &editor, + &queue_auto_device_parent_network_json(), + TopologyCanonicalIngressKind::NativeIntegration, + ); + let root_switch = canonical + .nodes + .iter_mut() + .find(|node| node.node_id == "uisp:device:root-switch") + .expect("canonical fixture should include root switch"); + root_switch.queue_visibility_policy = TopologyQueueVisibilityPolicy::QueueAuto; + let imported = queue_auto_device_parent_bundle(Some(editor), Some(canonical)); + + let compiled = compile_topology(imported, TopologyCompileMode::Full) + .expect("full compilation should prefer canonical queue policy"); + let child_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:child-switch") + .expect("child switch should remain in editor state"); + let root_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:root-switch") + .expect("root switch should remain in editor state"); + + assert_eq!( + root_switch.queue_visibility_policy, + TopologyQueueVisibilityPolicy::QueueAuto + ); + assert_eq!( + child_switch.current_parent_node_id.as_deref(), + Some("uisp:device:root-switch") + ); + assert!( + child_switch + .allowed_parents + .iter() + .any(|parent| parent.parent_node_id == "uisp:device:root-switch") + ); + } + + #[test] + fn full_mode_does_not_prefer_exported_device_parent_for_different_attachment() { + let editor = queue_auto_device_parent_editor( + TopologyQueueVisibilityPolicy::QueueAuto, + "uisp:site:site-root", + "Root Site", + vec![allowed_parent("uisp:site:site-root", "Root Site")], + ); + let canonical = TopologyCanonicalStateFile::from_editor_and_network( + &editor, + &queue_auto_device_parent_network_json(), + TopologyCanonicalIngressKind::NativeIntegration, + ); + let imported = queue_auto_device_parent_bundle(Some(editor), Some(canonical)); + + let compiled = compile_topology(imported, TopologyCompileMode::Full) + .expect("full compilation should preserve legal non-device parent"); + let child_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:child-switch") + .expect("child switch should remain in editor state"); + + assert_eq!( + child_switch.current_parent_node_id.as_deref(), + Some("uisp:site:site-root") + ); + assert!( + !child_switch + .allowed_parents + .iter() + .any(|parent| parent.parent_node_id == "uisp:device:root-switch") + ); + } + + #[test] + fn full_mode_uses_editor_queue_auto_device_parent_without_duplicate_allowed_parent() { + let editor = queue_auto_device_parent_editor( + TopologyQueueVisibilityPolicy::QueueAuto, + "uisp:device:root-switch", + "Root Switch", + vec![ + allowed_parent("uisp:site:site-root", "Root Site"), + allowed_parent("uisp:device:root-switch", "Root Switch"), + ], + ); + let imported = queue_auto_device_parent_bundle(Some(editor), None); + + let compiled = compile_topology(imported, TopologyCompileMode::Full) + .expect("full compilation should use editor-only queue-auto device parent"); + let child_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:child-switch") + .expect("child switch should remain in editor state"); + + assert_eq!( + child_switch.current_parent_node_id.as_deref(), + Some("uisp:device:root-switch") + ); + assert_eq!( + child_switch + .allowed_parents + .iter() + .filter(|parent| parent.parent_node_id == "uisp:device:root-switch") + .count(), + 1 + ); + } + #[test] fn full_mode_keeps_native_parent_when_compatibility_tree_flattens_backhaul_hops() { let imported = ImportedTopologyBundle { @@ -1160,32 +1550,7 @@ mod tests { allowed_parents: vec![TopologyAllowedParent { parent_node_id: "site-west".to_string(), parent_node_name: "WestRedd".to_string(), - attachment_options: vec![TopologyAttachmentOption { - attachment_id: "auto".to_string(), - attachment_name: "Auto".to_string(), - attachment_kind: "auto".to_string(), - attachment_role: TopologyAttachmentRole::Unknown, - pair_id: None, - peer_attachment_id: None, - peer_attachment_name: None, - capacity_mbps: None, - download_bandwidth_mbps: None, - upload_bandwidth_mbps: None, - transport_cap_mbps: None, - transport_cap_reason: None, - rate_source: TopologyAttachmentRateSource::Unknown, - can_override_rate: false, - rate_override_disabled_reason: None, - has_rate_override: false, - local_probe_ip: None, - remote_probe_ip: None, - probe_enabled: false, - probeable: false, - health_status: TopologyAttachmentHealthStatus::Disabled, - health_reason: None, - suppressed_until_unix: None, - effective_selected: false, - }], + attachment_options: vec![topology_auto_attachment_option()], all_attachments_suppressed: false, has_probe_unavailable_attachments: false, }], diff --git a/src/rust/uisp_integration/Cargo.toml b/src/rust/uisp_integration/Cargo.toml index 4d5c5407..b9e4c90b 100644 --- a/src/rust/uisp_integration/Cargo.toml +++ b/src/rust/uisp_integration/Cargo.toml @@ -24,3 +24,6 @@ lqos_bus = { path = "../lqos_bus" } petgraph = { version = "0.7.1", features = ["serde-1"] } lqos_overrides = { path = "../lqos_overrides" } lqos_topology_compile = { path = "../lqos_topology_compile" } + +[dev-dependencies] +lqos_topology = { path = "../lqos_topology" } diff --git a/src/rust/uisp_integration/src/strategies/full2.rs b/src/rust/uisp_integration/src/strategies/full2.rs index 63b2b6a6..abb66055 100644 --- a/src/rust/uisp_integration/src/strategies/full2.rs +++ b/src/rust/uisp_integration/src/strategies/full2.rs @@ -24,7 +24,7 @@ use lqos_config::{ TopologyAttachmentRateSource, TopologyAttachmentRole, TopologyCanonicalIngressKind, TopologyCanonicalStateFile, TopologyEditorNode, TopologyEditorStateFile, TopologyParentCandidate, TopologyParentCandidatesFile, TopologyParentCandidatesNode, - TopologyQueueVisibilityPolicy, + TopologyQueueVisibilityPolicy, topology_auto_attachment_option, }; #[cfg(test)] use lqos_overrides::{TopologyAttachmentMode, TopologyParentOverrideMode}; @@ -289,15 +289,13 @@ fn resolve_root_site( ) } -pub async fn build_imported_full2_bundle( +async fn build_imported_full2_bundle_from_data( config: Arc, - ip_ranges: IpRanges, + uisp_data: UispData, require_configured_root_site: bool, + build_started: Instant, ) -> Result { - let build_started = Instant::now(); let ethernet_policy = EthernetPortLimitPolicy::from(&config.integration_common); - // Fetch the data - let uisp_data = UispData::fetch_uisp_data(config.clone(), ip_ranges).await?; let bandwidth_overrides = crate::strategies::legacy_bandwidth_overrides::get_site_bandwidth_overrides(&config)?; @@ -633,12 +631,6 @@ pub async fn build_imported_full2_bundle( ); if is_real_topology_node(&graph, node) { - let queue_visibility_policy = - if matches!(&graph[node], GraphMapping::Site { .. }) { - TopologyQueueVisibilityPolicy::QueueAuto - } else { - TopologyQueueVisibilityPolicy::QueueVisible - }; topology_parent_candidates.push(TopologyParentCandidatesNode { node_id: graph[node].network_json_id(), node_name: name.to_owned(), @@ -669,7 +661,7 @@ pub async fn build_imported_full2_bundle( .map(|candidate| candidate.node_name.clone()), can_move: !allowed_parents.is_empty(), allowed_parents, - queue_visibility_policy, + queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueAuto, preferred_attachment_id: None, preferred_attachment_name: None, effective_attachment_id: None, @@ -954,6 +946,22 @@ pub async fn build_imported_full2_bundle( }) } +pub async fn build_imported_full2_bundle( + config: Arc, + ip_ranges: IpRanges, + require_configured_root_site: bool, +) -> Result { + let build_started = Instant::now(); + let uisp_data = UispData::fetch_uisp_data(config.clone(), ip_ranges).await?; + build_imported_full2_bundle_from_data( + config, + uisp_data, + require_configured_root_site, + build_started, + ) + .await +} + fn add_device_links_to_graph( uisp_data: &UispData, graph: &mut GraphType, @@ -1778,32 +1786,7 @@ fn topology_allowed_parents_from_groups_cached( let mut allowed_parents = Vec::with_capacity(groups.len()); for group in groups { let parent = { - let mut attachment_options = vec![TopologyAttachmentOption { - attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(), - attachment_name: "Auto".to_string(), - attachment_kind: "auto".to_string(), - attachment_role: TopologyAttachmentRole::Unknown, - pair_id: None, - peer_attachment_id: None, - peer_attachment_name: None, - capacity_mbps: None, - download_bandwidth_mbps: None, - upload_bandwidth_mbps: None, - transport_cap_mbps: None, - transport_cap_reason: None, - rate_source: TopologyAttachmentRateSource::Unknown, - can_override_rate: false, - rate_override_disabled_reason: None, - has_rate_override: false, - local_probe_ip: None, - remote_probe_ip: None, - probe_enabled: false, - probeable: false, - health_status: lqos_config::TopologyAttachmentHealthStatus::Disabled, - health_reason: None, - suppressed_until_unix: None, - effective_selected: false, - }]; + let mut attachment_options = vec![topology_auto_attachment_option()]; let mut seen_attachment_ids = HashSet::from([TOPOLOGY_ATTACHMENT_AUTO_ID.to_string()]); attachment_options.extend( @@ -2202,18 +2185,23 @@ mod topology_override_tests { use super::{ GraphType, TopologyAllowedParentGroup, TopologyAttachmentOverrideSelection, TopologyParentOverrideSelection, UispDevice, build_constrained_route, + build_imported_full2_bundle_from_data, export_parent_anchor_for_override, first_probe_ip_for_device, immediate_parent_from_route, is_upstream_parent_candidate, logical_parent_for_candidate, resolve_attachment_parent_candidate, resolve_parent_candidate, topology_allowed_parent_groups_for_node, topology_allowed_parents_from_groups, topology_parent_candidates_for_node, }; + use crate::strategies::common::UispData; use crate::strategies::full2::graph_mapping::GraphMapping; use crate::strategies::full2::link_mapping::LinkMapping; - use crate::uisp_types::UispAttachmentRateSource; - use lqos_config::TopologyAttachmentRole; + use crate::uisp_types::{UispAttachmentRateSource, UispSite, UispSiteType}; + use lqos_config::{TopologyAttachmentRole, TopologyQueueVisibilityPolicy}; use lqos_overrides::{TopologyAttachmentMode, TopologyParentOverrideMode}; + use serde_json::{Map, Value, json}; use std::collections::{HashMap, HashSet}; + use std::sync::Arc; + use uisp::{DataLink, Device, Site}; fn site(name: &str, id: &str) -> GraphMapping { GraphMapping::Site { @@ -2234,6 +2222,267 @@ mod topology_override_tests { } } + fn unique_temp_dir(name: &str) -> std::path::PathBuf { + let unique = format!( + "libreqos-uisp-full2-{name}-{}-{}", + std::process::id(), + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .unwrap_or_default() + .as_nanos() + ); + let path = std::env::temp_dir().join(unique); + std::fs::create_dir_all(&path).expect("temp dir should be created"); + path + } + + fn raw_site(id: &str, name: &str, site_type: &str) -> Site { + serde_json::from_value(json!({ + "id": id, + "identification": { + "id": id, + "name": name, + "type": site_type, + "suspended": false + } + })) + .expect("raw site fixture should deserialize") + } + + fn raw_device(id: &str, name: &str, site_id: &str) -> Device { + serde_json::from_value(json!({ + "identification": { + "id": id, + "hostname": name, + "site": { + "id": site_id + } + } + })) + .expect("raw device fixture should deserialize") + } + + struct LinkEndpoint<'a> { + site_id: &'a str, + site_name: &'a str, + device_id: &'a str, + device_name: &'a str, + } + + fn data_link_endpoint(endpoint: &LinkEndpoint<'_>) -> serde_json::Value { + json!({ + "site": { + "identification": { + "id": endpoint.site_id, + "name": endpoint.site_name + } + }, + "device": { + "identification": { + "id": endpoint.device_id, + "name": endpoint.device_name + } + } + }) + } + + fn device_data_link(id: &str, from: LinkEndpoint<'_>, to: LinkEndpoint<'_>) -> DataLink { + serde_json::from_value(json!({ + "id": id, + "from": data_link_endpoint(&from), + "to": data_link_endpoint(&to), + "canDelete": true + })) + .expect("data link fixture should deserialize") + } + + fn parsed_site(id: &str, name: &str) -> UispSite { + UispSite { + id: id.to_string(), + name: name.to_string(), + site_type: UispSiteType::Site, + max_down_mbps: 10_000, + max_up_mbps: 10_000, + ..Default::default() + } + } + + fn parsed_device(id: &str, name: &str, site_id: &str) -> UispDevice { + UispDevice { + id: id.to_string(), + name: name.to_string(), + mac: String::new(), + role: None, + wireless_mode: None, + site_id: site_id.to_string(), + raw_download: 10_000, + raw_upload: 10_000, + download: 10_000, + upload: 10_000, + ipv4: HashSet::new(), + ipv6: HashSet::new(), + probe_ipv4: HashSet::new(), + probe_ipv6: HashSet::new(), + negotiated_ethernet_mbps: None, + negotiated_ethernet_interface: None, + transport_cap_mbps: None, + transport_cap_reason: None, + attachment_rate_source: UispAttachmentRateSource::Static, + } + } + + fn find_network_node_by_id<'a>( + value: &'a Value, + target_id: &str, + ) -> Option<&'a Map> { + let map = value.as_object()?; + if map.get("id").and_then(Value::as_str) == Some(target_id) { + return Some(map); + } + + map.values() + .find_map(|child| find_network_node_by_id(child, target_id)) + } + + #[tokio::test] + async fn full2_import_emits_queue_auto_for_device_backed_branches() { + let temp_dir = unique_temp_dir("uisp-full2-queue-auto"); + let mut config = lqos_config::Config { + lqos_directory: temp_dir.to_string_lossy().to_string(), + state_directory: Some(temp_dir.join("state").to_string_lossy().to_string()), + ..Default::default() + }; + config.uisp_integration.enable_uisp = true; + config.uisp_integration.site = "Root Site".to_string(); + config.queues.generated_pn_download_mbps = 10_000; + config.queues.generated_pn_upload_mbps = 10_000; + config.topology.queue_auto_virtualize_threshold_mbps = 5_000; + let config_for_topology = config.clone(); + + let root_site = raw_site("site-root", "Root Site", "site"); + let child_site = raw_site("site-child", "Child Site", "site"); + let root_device = raw_device("device-root-switch", "Root Switch", "site-root"); + let child_device = raw_device("device-child-switch", "Child Switch", "site-child"); + let root_parsed_device = parsed_device("device-root-switch", "Root Switch", "site-root"); + let child_parsed_device = + parsed_device("device-child-switch", "Child Switch", "site-child"); + let uisp_data = UispData::from_parts( + vec![root_site, child_site], + vec![root_device, child_device], + vec![device_data_link( + "link-root-child", + LinkEndpoint { + site_id: "site-root", + site_name: "Root Site", + device_id: "device-root-switch", + device_name: "Root Switch", + }, + LinkEndpoint { + site_id: "site-child", + site_name: "Child Site", + device_id: "device-child-switch", + device_name: "Child Switch", + }, + )], + vec![ + parsed_site("site-root", "Root Site"), + parsed_site("site-child", "Child Site"), + ], + vec![root_parsed_device, child_parsed_device], + ); + + let bundle = build_imported_full2_bundle_from_data( + Arc::new(config), + uisp_data, + true, + std::time::Instant::now(), + ) + .await + .expect("in-memory UISP import should build"); + let editor = bundle + .native_editor + .as_ref() + .expect("full2 import should include native editor state"); + let root_switch = editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:device-root-switch") + .expect("root switch should be exported as a topology editor node"); + + assert_eq!( + root_switch.queue_visibility_policy, + TopologyQueueVisibilityPolicy::QueueAuto + ); + + let compiled = lqos_topology_compile::compile_topology( + bundle, + lqos_topology_compile::TopologyCompileMode::Full, + ) + .expect("full topology should compile from full2 import"); + let compiled_child_switch = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:device:device-child-switch") + .expect("compiled editor should include the child switch"); + assert_eq!( + compiled_child_switch.queue_visibility_policy, + TopologyQueueVisibilityPolicy::QueueAuto + ); + assert_eq!( + compiled_child_switch.current_parent_node_id.as_deref(), + Some("uisp:device:device-root-switch") + ); + assert!( + compiled_child_switch + .allowed_parents + .iter() + .any(|parent| parent.parent_node_id == "uisp:device:device-root-switch") + ); + let compiled_child_site = compiled + .editor + .nodes + .iter() + .find(|node| node.node_id == "uisp:site:site-child") + .expect("compiled editor should include the child site"); + assert_eq!( + compiled_child_site.queue_visibility_policy, + TopologyQueueVisibilityPolicy::QueueAuto + ); + assert_eq!( + compiled_child_site.current_parent_node_id.as_deref(), + Some("uisp:device:device-child-switch") + ); + assert!( + compiled_child_site + .allowed_parents + .iter() + .any(|parent| parent.parent_node_id == "uisp:device:device-child-switch") + ); + + let artifacts = lqos_topology::build_effective_topology_artifacts_from_canonical( + &config_for_topology, + &compiled.canonical, + &lqos_overrides::TopologyOverridesFile::default(), + &lqos_config::TopologyAttachmentHealthStateFile::default(), + ) + .expect("effective topology should compile from full2 canonical output"); + let effective_network = artifacts + .effective_network + .as_ref() + .expect("effective topology should include a network tree"); + let effective_root_switch = + find_network_node_by_id(effective_network, "uisp:device:device-root-switch") + .expect("root switch should remain visible in the effective tree"); + + assert_eq!( + effective_root_switch.get("virtual").and_then(Value::as_bool), + Some(true) + ); + + let _ = std::fs::remove_dir_all(temp_dir); + } + fn build_test_graph() -> ( GraphType, petgraph::graph::NodeIndex,