Merge pull request #1121 from LibreQoE/effective_circuit_rate

Preserve UISP transit device virtualization in full topology mode
This commit is contained in:
Robert Chacón authored and GitHub committed 2026-06-29 17:09:15 -06:00
commit bbd481d8ee
13 files changed
+732 -135

No files matched your search

+1
View File
@@ -33,6 +33,7 @@ El ajuste `Root UISP Site` solo es obligatorio en modo de topología `full`. En
- `network.json` queda para despliegues DIY o manuales.
- Use la WebUI para confirmar que la importación y la profundidad del árbol son las esperadas.
- En modo integración, trate ediciones de archivos como temporales.
- En modo `full`, los switches, routers y radios de UISP exportados como ramas AP/dispositivo pueden usar virtualización Queue Auto. Consulte [Nodos virtuales](configuration-advanced-es.md#nodos-virtuales-solo-lógicos).
## Validación en 5 minutos después de cambios UISP
+1 -1
View File
@@ -35,7 +35,7 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta
- En compilaciones UISP `full`, un `integrationUISPbandwidths.csv` heredado se auto-migra a esos overrides en la siguiente ejecución de la integración cuando todavía no existen overrides operativos de tasa; en caso contrario, el CSV se ignora.
- Las ediciones de tasa desde la página del árbol requieren sesión de administrador, un `node_id` estable y un nodo no generado. Los nodos generados/sintéticos de la integración permanecen en solo lectura en este editor.
- La página del árbol mantiene `Node Details` como una cuadrícula compacta de resumen y coloca los editores de override como filas compactas directamente debajo.
- `Effective Now` muestra una insignia de origen estimado, como Parent, Override, Attachment o Mixed. Pase el cursor o enfoque la insignia para ver los detalles de descarga y subida.
- `Limited By` estima el origen de la velocidad efectiva. Los valores pueden incluir `Base rate`, `Est. Parent`, `Est. Override`, `Est. Attachment`, `Est. Queue` o `Est. Mixed`. Pase el cursor o enfoque el campo para ver los detalles de descarga y subida.
- La página del árbol ahora muestra el estado de `Topology Override` solo en modo lectura. Los cambios de padre y adjunto se movieron a `Topology`, y el panel de detalles del árbol enlaza directamente a Topology Manager para el nodo seleccionado.
- Los saltos de adjuntos seleccionados en runtime se colapsan fuera de la jerarquía principal del árbol. Cuando un sitio usa actualmente un backhaul o radio path específico, `Node Details` lo muestra como metadato `Active Attachment` en el sitio en lugar de exponer ese adjunto como un nodo propio del árbol.
- Los adjuntos alternativos inactivos de backhaul PtP/cableado también se eliminan del árbol runtime. `tree.html` muestra solo la ruta efectiva; los alternativos inactivos quedan visibles en Topology Manager y no como nodos runtime independientes.
+2
View File
@@ -32,6 +32,8 @@ LibreQoS supports multiple topology strategies for UISP integration to balance C
**Performance Note:** When using `full` strategy with large networks, consider using `promote_to_root` to distribute CPU load across multiple cores.
In `full` mode, UISP switches, routers, and radios exported as AP/device branches can use Queue Auto virtualization. See [Advanced Configuration Reference](configuration-advanced.md#queueauto-decision-flow).
## 5-Minute Validation After UISP Config Changes
1. Run the integration once:
+1 -1
View File
@@ -38,7 +38,7 @@ For the full logical-topology versus queue-topology file flow, see [Topology Dat
- On UISP `full` builds, legacy `integrationUISPbandwidths.csv` files are auto-migrated into those operator overrides on the next integration run when no operator rate overrides exist yet; otherwise the CSV is ignored.
- Tree-page operator rate edits require an administrator session, a stable node ID, and a non-generated node. Generated/integration-synthetic nodes remain read-only in this editor.
- The tree page keeps `Node Details` as a compact summary grid and places the override editors as compact inline rows directly beneath it.
- `Effective Now` shows an estimated source badge, such as Parent, Override, Attachment, or Mixed. Hover or focus the badge to see the download and upload details.
- `Limited By` estimates the source of the effective rate. Values can include `Base rate`, `Est. Parent`, `Est. Override`, `Est. Attachment`, `Est. Queue`, or `Est. Mixed`. Hover or focus it to see the download and upload details.
- The tree page now shows topology override state as read-only summary only. Parent and attachment edits have moved to `Topology`, and the tree details panel links directly into Topology Manager for the selected node.
- Runtime-selected attachment hops are collapsed out of the main tree hierarchy. When a site is currently using a specific backhaul or radio path, `Node Details` shows that as `Active Attachment` metadata on the site instead of exposing the attachment as its own branch node.
- Inactive alternate PtP/wired backhaul attachment stubs are also pruned from the runtime tree. `tree.html` shows the effective path only; dormant alternates remain visible in Topology Manager rather than as standalone runtime nodes.
+1
View File
@@ -4677,6 +4677,7 @@ dependencies = [
"lqos_bus",
"lqos_config",
"lqos_overrides",
"lqos_topology",
"lqos_topology_compile",
"lqos_utils",
"petgraph",
+1 -1
View File
@@ -81,7 +81,7 @@ pub use topology_editor_state::{
TOPOLOGY_ATTACHMENT_AUTO_ID, TOPOLOGY_EDITOR_STATE_FILENAME, TopologyAllowedParent,
TopologyAttachmentHealthStatus, TopologyAttachmentOption, TopologyAttachmentRateSource,
TopologyAttachmentRole, TopologyEditorNode, TopologyEditorStateError, TopologyEditorStateFile,
TopologyQueueVisibilityPolicy, topology_editor_state_path,
TopologyQueueVisibilityPolicy, topology_auto_attachment_option, topology_editor_state_path,
};
pub use topology_parent_candidates::{
TOPOLOGY_PARENT_CANDIDATES_FILENAME, TopologyParentCandidate, TopologyParentCandidatesError,
@@ -168,6 +168,36 @@ pub struct TopologyAttachmentOption {
pub effective_selected: bool,
}
/// Builds the standard dynamic attachment option used when LibreQoS selects the attachment.
pub fn topology_auto_attachment_option() -> TopologyAttachmentOption {
TopologyAttachmentOption {
attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(),
attachment_name: "Auto".to_string(),
attachment_kind: "auto".to_string(),
attachment_role: TopologyAttachmentRole::Unknown,
pair_id: None,
peer_attachment_id: None,
peer_attachment_name: None,
capacity_mbps: None,
download_bandwidth_mbps: None,
upload_bandwidth_mbps: None,
transport_cap_mbps: None,
transport_cap_reason: None,
rate_source: TopologyAttachmentRateSource::Unknown,
can_override_rate: false,
rate_override_disabled_reason: None,
has_rate_override: false,
local_probe_ip: None,
remote_probe_ip: None,
probe_enabled: false,
probeable: false,
health_status: TopologyAttachmentHealthStatus::Disabled,
health_reason: None,
suppressed_until_unix: None,
effective_selected: false,
}
}
/// One valid parent target for a topology node, plus allowed attachments below that parent.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq, Default)]
pub struct TopologyAllowedParent {
@@ -402,32 +432,7 @@ impl TopologyEditorStateFile {
.map(|candidate| TopologyAllowedParent {
parent_node_id: candidate.node_id.clone(),
parent_node_name: candidate.node_name.clone(),
attachment_options: vec![TopologyAttachmentOption {
attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(),
attachment_name: "Auto".to_string(),
attachment_kind: "auto".to_string(),
attachment_role: TopologyAttachmentRole::Unknown,
pair_id: None,
peer_attachment_id: None,
peer_attachment_name: None,
capacity_mbps: None,
download_bandwidth_mbps: None,
upload_bandwidth_mbps: None,
transport_cap_mbps: None,
transport_cap_reason: None,
rate_source: TopologyAttachmentRateSource::Unknown,
can_override_rate: false,
rate_override_disabled_reason: None,
has_rate_override: false,
local_probe_ip: None,
remote_probe_ip: None,
probe_enabled: false,
probeable: false,
health_status: TopologyAttachmentHealthStatus::Disabled,
health_reason: None,
suppressed_until_unix: None,
effective_selected: false,
}],
attachment_options: vec![topology_auto_attachment_option()],
all_attachments_suppressed: false,
has_probe_unavailable_attachments: false,
})
+1 -30
View File
@@ -46,35 +46,6 @@ pub fn is_health_state_fresh(config: &Config, health: &TopologyAttachmentHealthS
now.saturating_sub(generated_unix) <= health_state_stale_after_seconds(config)
}
fn auto_attachment_option() -> TopologyAttachmentOption {
TopologyAttachmentOption {
attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(),
attachment_name: "Auto".to_string(),
attachment_kind: "auto".to_string(),
attachment_role: TopologyAttachmentRole::Unknown,
pair_id: None,
peer_attachment_id: None,
peer_attachment_name: None,
capacity_mbps: None,
download_bandwidth_mbps: None,
upload_bandwidth_mbps: None,
transport_cap_mbps: None,
transport_cap_reason: None,
rate_source: TopologyAttachmentRateSource::Unknown,
can_override_rate: false,
rate_override_disabled_reason: None,
has_rate_override: false,
local_probe_ip: None,
remote_probe_ip: None,
probe_enabled: false,
probeable: false,
health_status: TopologyAttachmentHealthStatus::Disabled,
health_reason: None,
suppressed_until_unix: None,
effective_selected: false,
}
}
fn overlay_manual_groups(
canonical: &TopologyEditorStateFile,
overrides: &TopologyOverridesFile,
@@ -87,7 +58,7 @@ fn overlay_manual_groups(
else {
continue;
};
let mut options = vec![auto_attachment_option()];
let mut options = vec![topology_auto_attachment_option()];
for attachment in &group.attachments {
let local_probe_ip = parse_probe_ip(&attachment.local_probe_ip);
let remote_probe_ip = parse_probe_ip(&attachment.remote_probe_ip);
+1 -1
View File
@@ -17,7 +17,7 @@ use lqos_config::{
TopologyShapingInputsFile, TopologyShapingResolutionSource, circuit_anchors_path,
compute_effective_network_generation, detect_shaping_cpus, plan_top_level_assignments,
topology_effective_network_path, topology_effective_state_path, topology_runtime_status_path,
topology_shaping_inputs_path,
topology_auto_attachment_option, topology_shaping_inputs_path,
};
use lqos_overrides::{
CircuitAdjustment, NetworkAdjustment, OverrideStore, TopologyAttachmentMode,
+3 -3
View File
@@ -3,7 +3,7 @@
apply_effective_topology_to_canonical_state as try_apply_effective_topology_to_canonical_state,
apply_effective_topology_to_network_json as try_apply_effective_topology_to_network_json,
apply_effective_topology_to_network_json_from_canonical as try_apply_effective_topology_to_network_json_from_canonical,
apply_health_to_option, auto_attachment_option, build_effective_topology_artifacts,
apply_health_to_option, build_effective_topology_artifacts,
build_effective_topology_artifacts_from_canonical, build_shaping_inputs,
collect_direct_circuit_node_ids, collect_direct_circuit_node_names,
compute_effective_state, parse_probe_ip, publish_effective_topology_artifacts,
@@ -18,8 +18,8 @@
TopologyCanonicalRateInputSource, TopologyCanonicalStateFile, TopologyEditorNode,
TopologyEditorStateFile, TopologyEffectiveAttachmentState, TopologyEffectiveNodeState,
TopologyEffectiveStateFile, TopologyQueueVisibilityPolicy, TopologyRuntimeStatusFile,
topology_effective_network_path, topology_effective_state_path,
topology_runtime_status_path, topology_shaping_inputs_path,
topology_auto_attachment_option as auto_attachment_option, topology_effective_network_path,
topology_effective_state_path, topology_runtime_status_path, topology_shaping_inputs_path,
};
use lqos_overrides::{TopologyAttachmentMode, TopologyOverridesFile};
use serde_json::{Value, json};
@@ -4,8 +4,10 @@ use anyhow::Result;
use lqos_config::{
CircuitAnchor, CircuitAnchorsFile, ConfigShapedDevices, ShapedDevice,
TopologyCanonicalIngressKind, TopologyCanonicalNode, TopologyCanonicalStateFile,
TopologyEditorNode, TopologyEditorStateFile, TopologyParentCandidatesFile,
topology_ingress_identity_from_tokens,
TopologyAllowedParent, TopologyEditorNode, TopologyEditorStateFile,
TopologyParentCandidatesFile,
TopologyQueueVisibilityPolicy,
topology_auto_attachment_option, topology_ingress_identity_from_tokens,
};
use serde_json::{Map, Number, Value};
use std::collections::{BTreeMap, HashMap, HashSet};
@@ -373,6 +375,7 @@ struct SanitizedTopologyIndex {
exported_node_names_by_id: HashMap<String, String>,
exported_parent_by_node_id: HashMap<String, Option<String>>,
node_names_by_id: HashMap<String, String>,
queue_policy_by_node_id: HashMap<String, TopologyQueueVisibilityPolicy>,
}
fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopologyIndex {
@@ -387,14 +390,19 @@ fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopol
.map(|(node_id, node)| (node_id.clone(), node.parent_id.clone()))
.collect::<HashMap<_, _>>();
let mut queue_policy_by_node_id = HashMap::new();
if let Some(native) = imported.native_editor.as_ref() {
for node in &native.nodes {
node_names_by_id.insert(node.node_id.clone(), node.node_name.clone());
queue_policy_by_node_id
.insert(node.node_id.clone(), node.queue_visibility_policy);
}
}
if let Some(native) = imported.native_canonical.as_ref() {
for node in &native.nodes {
node_names_by_id.insert(node.node_id.clone(), node.node_name.clone());
queue_policy_by_node_id
.insert(node.node_id.clone(), node.queue_visibility_policy);
}
}
@@ -402,6 +410,36 @@ fn sanitized_topology_index(imported: &ImportedTopologyBundle) -> SanitizedTopol
exported_node_names_by_id,
exported_parent_by_node_id,
node_names_by_id,
queue_policy_by_node_id,
}
}
fn is_queue_auto_uisp_device_parent(
topology_index: &SanitizedTopologyIndex,
parent_id: &str,
) -> bool {
parent_id.starts_with("uisp:device:")
&& topology_index
.queue_policy_by_node_id
.get(parent_id)
.is_some_and(|policy| policy == &TopologyQueueVisibilityPolicy::QueueAuto)
}
fn preserved_exported_queue_auto_device_parent(
topology_index: &SanitizedTopologyIndex,
node: &TopologyEditorNode,
) -> Option<String> {
let parent_id = topology_index
.exported_parent_by_node_id
.get(&node.node_id)
.and_then(|parent| parent.as_deref())?;
if is_queue_auto_uisp_device_parent(topology_index, parent_id)
&& topology_index.node_names_by_id.contains_key(parent_id)
&& node.current_attachment_id.as_deref() == Some(parent_id)
{
Some(parent_id.to_string())
} else {
None
}
}
@@ -424,6 +462,26 @@ fn sanitize_editor_nodes(
.node_names_by_id
.contains_key(parent.parent_node_id.as_str())
});
let preserved_device_parent =
preserved_exported_queue_auto_device_parent(topology_index, &node);
if let Some(parent_id) = preserved_device_parent.as_ref()
&& !node
.allowed_parents
.iter()
.any(|parent| parent.parent_node_id == *parent_id)
{
node.allowed_parents.push(TopologyAllowedParent {
parent_node_id: parent_id.clone(),
parent_node_name: topology_index
.node_names_by_id
.get(parent_id)
.cloned()
.unwrap_or_else(|| parent_id.clone()),
attachment_options: vec![topology_auto_attachment_option()],
all_attachments_suppressed: false,
has_probe_unavailable_attachments: false,
});
}
let legal_current_parent =
node.current_parent_node_id
.as_deref()
@@ -445,6 +503,9 @@ fn sanitize_editor_nodes(
.get(&node.node_id)
.and_then(|parent| parent.as_deref())
.and_then(|parent_id| {
if preserved_device_parent.as_deref() == Some(parent_id) {
return Some(parent_id.to_string());
}
if (node.allowed_parents.is_empty()
&& topology_index.node_names_by_id.contains_key(parent_id))
|| node
@@ -457,7 +518,11 @@ fn sanitize_editor_nodes(
None
}
});
let selected_parent = legal_current_parent.or(exported_parent);
let selected_parent = if preserved_device_parent.is_some() {
exported_parent.or(legal_current_parent)
} else {
legal_current_parent.or(exported_parent)
};
node.current_parent_node_name = selected_parent.as_deref().and_then(|parent_id| {
node.allowed_parents
.iter()
@@ -478,6 +543,9 @@ fn sanitize_editor_nodes(
node.current_attachment_name = None;
}
if let Some(policy) = topology_index.queue_policy_by_node_id.get(&node.node_id) {
node.queue_visibility_policy = *policy;
}
node.node_name = exported_name.clone();
Some(node)
})
@@ -780,9 +848,9 @@ mod tests {
TopologyAttachmentRateSource, TopologyAttachmentRole, TopologyCanonicalIngressKind,
TopologyCanonicalStateFile, TopologyEditorNode, TopologyEditorStateFile,
TopologyParentCandidate, TopologyParentCandidatesFile, TopologyParentCandidatesNode,
TopologyQueueVisibilityPolicy,
TopologyQueueVisibilityPolicy, topology_auto_attachment_option,
};
use serde_json::json;
use serde_json::{Value, json};
fn shaped_devices(rows: Vec<ShapedDevice>) -> ConfigShapedDevices {
let mut shaped = ConfigShapedDevices::default();
@@ -814,6 +882,148 @@ mod tests {
}
}
fn queue_auto_device_parent_network_json() -> Value {
json!({
"Root Site": {
"children": {
"Root Switch": {
"children": {
"Child Switch": {
"children": {},
"id": "uisp:device:child-switch",
"name": "Child Switch",
"type": "AP"
}
},
"id": "uisp:device:root-switch",
"name": "Root Switch",
"type": "AP"
}
},
"id": "uisp:site:site-root",
"name": "Root Site",
"type": "Site"
}
})
}
fn root_site_editor_node() -> TopologyEditorNode {
TopologyEditorNode {
node_id: "uisp:site:site-root".to_string(),
node_name: "Root Site".to_string(),
latitude: None,
longitude: None,
current_parent_node_id: None,
current_parent_node_name: None,
current_attachment_id: None,
current_attachment_name: None,
can_move: false,
allowed_parents: Vec::new(),
queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueHiddenPromoteChildren,
preferred_attachment_id: None,
preferred_attachment_name: None,
effective_attachment_id: None,
effective_attachment_name: None,
}
}
fn root_switch_editor_node(
queue_visibility_policy: TopologyQueueVisibilityPolicy,
) -> TopologyEditorNode {
TopologyEditorNode {
node_id: "uisp:device:root-switch".to_string(),
node_name: "Root Switch".to_string(),
latitude: None,
longitude: None,
current_parent_node_id: Some("uisp:site:site-root".to_string()),
current_parent_node_name: Some("Root Site".to_string()),
current_attachment_id: Some("uisp:site:site-root".to_string()),
current_attachment_name: Some("Root Site".to_string()),
can_move: true,
allowed_parents: Vec::new(),
queue_visibility_policy,
preferred_attachment_id: None,
preferred_attachment_name: None,
effective_attachment_id: None,
effective_attachment_name: None,
}
}
fn allowed_parent(parent_node_id: &str, parent_node_name: &str) -> TopologyAllowedParent {
TopologyAllowedParent {
parent_node_id: parent_node_id.to_string(),
parent_node_name: parent_node_name.to_string(),
attachment_options: vec![topology_auto_attachment_option()],
all_attachments_suppressed: false,
has_probe_unavailable_attachments: false,
}
}
fn child_switch_editor_node(
current_attachment_id: &str,
current_attachment_name: &str,
allowed_parents: Vec<TopologyAllowedParent>,
) -> TopologyEditorNode {
TopologyEditorNode {
node_id: "uisp:device:child-switch".to_string(),
node_name: "Child Switch".to_string(),
latitude: None,
longitude: None,
current_parent_node_id: Some("uisp:site:site-root".to_string()),
current_parent_node_name: Some("Root Site".to_string()),
current_attachment_id: Some(current_attachment_id.to_string()),
current_attachment_name: Some(current_attachment_name.to_string()),
can_move: true,
allowed_parents,
queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueAuto,
preferred_attachment_id: None,
preferred_attachment_name: None,
effective_attachment_id: None,
effective_attachment_name: None,
}
}
fn queue_auto_device_parent_editor(
root_switch_policy: TopologyQueueVisibilityPolicy,
child_attachment_id: &str,
child_attachment_name: &str,
child_allowed_parents: Vec<TopologyAllowedParent>,
) -> TopologyEditorStateFile {
TopologyEditorStateFile {
schema_version: 1,
source: "uisp/full2".to_string(),
generated_unix: Some(1),
ingress_identity: None,
nodes: vec![
root_site_editor_node(),
root_switch_editor_node(root_switch_policy),
child_switch_editor_node(
child_attachment_id,
child_attachment_name,
child_allowed_parents,
),
],
}
}
fn queue_auto_device_parent_bundle(
native_editor: Option<TopologyEditorStateFile>,
native_canonical: Option<TopologyCanonicalStateFile>,
) -> ImportedTopologyBundle {
ImportedTopologyBundle {
source: "uisp/full2".to_string(),
generated_unix: Some(1),
ingress_identity: None,
native_canonical,
native_editor,
parent_candidates: None,
compatibility_network_json: queue_auto_device_parent_network_json(),
shaped_devices: shaped_devices(Vec::new()),
circuit_anchors: CircuitAnchorsFile::default(),
ethernet_advisories: Vec::new(),
}
}
fn imported_bundle() -> ImportedTopologyBundle {
let network = json!({
"Site Alpha": {
@@ -1117,6 +1327,186 @@ mod tests {
assert_eq!(node.current_attachment_name.as_deref(), Some("AP Alpha"));
}
#[test]
fn full_mode_preserves_queue_auto_device_parent_from_native_canonical() {
let editor = queue_auto_device_parent_editor(
TopologyQueueVisibilityPolicy::QueueAuto,
"uisp:device:root-switch",
"Root Switch",
vec![allowed_parent("uisp:site:site-root", "Root Site")],
);
let canonical = TopologyCanonicalStateFile::from_editor_and_network(
&editor,
&queue_auto_device_parent_network_json(),
TopologyCanonicalIngressKind::NativeIntegration,
);
let imported = queue_auto_device_parent_bundle(None, Some(canonical));
let compiled = compile_topology(imported, TopologyCompileMode::Full)
.expect("full compilation should preserve canonical queue-auto device parents");
let child_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:child-switch")
.expect("child switch should remain in editor state");
let root_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:root-switch")
.expect("root switch should remain in editor state");
assert_eq!(
root_switch.queue_visibility_policy,
TopologyQueueVisibilityPolicy::QueueAuto
);
assert_eq!(
child_switch.current_parent_node_id.as_deref(),
Some("uisp:device:root-switch")
);
assert_eq!(
child_switch.current_attachment_id.as_deref(),
Some("uisp:device:root-switch")
);
assert_eq!(
child_switch.current_attachment_name.as_deref(),
Some("Root Switch")
);
assert_eq!(
child_switch
.allowed_parents
.iter()
.filter(|parent| parent.parent_node_id == "uisp:device:root-switch")
.count(),
1
);
}
#[test]
fn full_mode_prefers_native_canonical_queue_policy_over_editor_policy() {
let editor = queue_auto_device_parent_editor(
TopologyQueueVisibilityPolicy::QueueVisible,
"uisp:device:root-switch",
"Root Switch",
vec![allowed_parent("uisp:site:site-root", "Root Site")],
);
let mut canonical = TopologyCanonicalStateFile::from_editor_and_network(
&editor,
&queue_auto_device_parent_network_json(),
TopologyCanonicalIngressKind::NativeIntegration,
);
let root_switch = canonical
.nodes
.iter_mut()
.find(|node| node.node_id == "uisp:device:root-switch")
.expect("canonical fixture should include root switch");
root_switch.queue_visibility_policy = TopologyQueueVisibilityPolicy::QueueAuto;
let imported = queue_auto_device_parent_bundle(Some(editor), Some(canonical));
let compiled = compile_topology(imported, TopologyCompileMode::Full)
.expect("full compilation should prefer canonical queue policy");
let child_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:child-switch")
.expect("child switch should remain in editor state");
let root_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:root-switch")
.expect("root switch should remain in editor state");
assert_eq!(
root_switch.queue_visibility_policy,
TopologyQueueVisibilityPolicy::QueueAuto
);
assert_eq!(
child_switch.current_parent_node_id.as_deref(),
Some("uisp:device:root-switch")
);
assert!(
child_switch
.allowed_parents
.iter()
.any(|parent| parent.parent_node_id == "uisp:device:root-switch")
);
}
#[test]
fn full_mode_does_not_prefer_exported_device_parent_for_different_attachment() {
let editor = queue_auto_device_parent_editor(
TopologyQueueVisibilityPolicy::QueueAuto,
"uisp:site:site-root",
"Root Site",
vec![allowed_parent("uisp:site:site-root", "Root Site")],
);
let canonical = TopologyCanonicalStateFile::from_editor_and_network(
&editor,
&queue_auto_device_parent_network_json(),
TopologyCanonicalIngressKind::NativeIntegration,
);
let imported = queue_auto_device_parent_bundle(Some(editor), Some(canonical));
let compiled = compile_topology(imported, TopologyCompileMode::Full)
.expect("full compilation should preserve legal non-device parent");
let child_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:child-switch")
.expect("child switch should remain in editor state");
assert_eq!(
child_switch.current_parent_node_id.as_deref(),
Some("uisp:site:site-root")
);
assert!(
!child_switch
.allowed_parents
.iter()
.any(|parent| parent.parent_node_id == "uisp:device:root-switch")
);
}
#[test]
fn full_mode_uses_editor_queue_auto_device_parent_without_duplicate_allowed_parent() {
let editor = queue_auto_device_parent_editor(
TopologyQueueVisibilityPolicy::QueueAuto,
"uisp:device:root-switch",
"Root Switch",
vec![
allowed_parent("uisp:site:site-root", "Root Site"),
allowed_parent("uisp:device:root-switch", "Root Switch"),
],
);
let imported = queue_auto_device_parent_bundle(Some(editor), None);
let compiled = compile_topology(imported, TopologyCompileMode::Full)
.expect("full compilation should use editor-only queue-auto device parent");
let child_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:child-switch")
.expect("child switch should remain in editor state");
assert_eq!(
child_switch.current_parent_node_id.as_deref(),
Some("uisp:device:root-switch")
);
assert_eq!(
child_switch
.allowed_parents
.iter()
.filter(|parent| parent.parent_node_id == "uisp:device:root-switch")
.count(),
1
);
}
#[test]
fn full_mode_keeps_native_parent_when_compatibility_tree_flattens_backhaul_hops() {
let imported = ImportedTopologyBundle {
@@ -1160,32 +1550,7 @@ mod tests {
allowed_parents: vec![TopologyAllowedParent {
parent_node_id: "site-west".to_string(),
parent_node_name: "WestRedd".to_string(),
attachment_options: vec![TopologyAttachmentOption {
attachment_id: "auto".to_string(),
attachment_name: "Auto".to_string(),
attachment_kind: "auto".to_string(),
attachment_role: TopologyAttachmentRole::Unknown,
pair_id: None,
peer_attachment_id: None,
peer_attachment_name: None,
capacity_mbps: None,
download_bandwidth_mbps: None,
upload_bandwidth_mbps: None,
transport_cap_mbps: None,
transport_cap_reason: None,
rate_source: TopologyAttachmentRateSource::Unknown,
can_override_rate: false,
rate_override_disabled_reason: None,
has_rate_override: false,
local_probe_ip: None,
remote_probe_ip: None,
probe_enabled: false,
probeable: false,
health_status: TopologyAttachmentHealthStatus::Disabled,
health_reason: None,
suppressed_until_unix: None,
effective_selected: false,
}],
attachment_options: vec![topology_auto_attachment_option()],
all_attachments_suppressed: false,
has_probe_unavailable_attachments: false,
}],
+3
View File
@@ -24,3 +24,6 @@ lqos_bus = { path = "../lqos_bus" }
petgraph = { version = "0.7.1", features = ["serde-1"] }
lqos_overrides = { path = "../lqos_overrides" }
lqos_topology_compile = { path = "../lqos_topology_compile" }
[dev-dependencies]
lqos_topology = { path = "../lqos_topology" }
+290 -41
View File
@@ -24,7 +24,7 @@ use lqos_config::{
TopologyAttachmentRateSource, TopologyAttachmentRole, TopologyCanonicalIngressKind,
TopologyCanonicalStateFile, TopologyEditorNode, TopologyEditorStateFile,
TopologyParentCandidate, TopologyParentCandidatesFile, TopologyParentCandidatesNode,
TopologyQueueVisibilityPolicy,
TopologyQueueVisibilityPolicy, topology_auto_attachment_option,
};
#[cfg(test)]
use lqos_overrides::{TopologyAttachmentMode, TopologyParentOverrideMode};
@@ -289,15 +289,13 @@ fn resolve_root_site(
)
}
pub async fn build_imported_full2_bundle(
async fn build_imported_full2_bundle_from_data(
config: Arc<Config>,
ip_ranges: IpRanges,
uisp_data: UispData,
require_configured_root_site: bool,
build_started: Instant,
) -> Result<ImportedTopologyBundle, UispIntegrationError> {
let build_started = Instant::now();
let ethernet_policy = EthernetPortLimitPolicy::from(&config.integration_common);
// Fetch the data
let uisp_data = UispData::fetch_uisp_data(config.clone(), ip_ranges).await?;
let bandwidth_overrides =
crate::strategies::legacy_bandwidth_overrides::get_site_bandwidth_overrides(&config)?;
@@ -633,12 +631,6 @@ pub async fn build_imported_full2_bundle(
);
if is_real_topology_node(&graph, node) {
let queue_visibility_policy =
if matches!(&graph[node], GraphMapping::Site { .. }) {
TopologyQueueVisibilityPolicy::QueueAuto
} else {
TopologyQueueVisibilityPolicy::QueueVisible
};
topology_parent_candidates.push(TopologyParentCandidatesNode {
node_id: graph[node].network_json_id(),
node_name: name.to_owned(),
@@ -669,7 +661,7 @@ pub async fn build_imported_full2_bundle(
.map(|candidate| candidate.node_name.clone()),
can_move: !allowed_parents.is_empty(),
allowed_parents,
queue_visibility_policy,
queue_visibility_policy: TopologyQueueVisibilityPolicy::QueueAuto,
preferred_attachment_id: None,
preferred_attachment_name: None,
effective_attachment_id: None,
@@ -954,6 +946,22 @@ pub async fn build_imported_full2_bundle(
})
}
pub async fn build_imported_full2_bundle(
config: Arc<Config>,
ip_ranges: IpRanges,
require_configured_root_site: bool,
) -> Result<ImportedTopologyBundle, UispIntegrationError> {
let build_started = Instant::now();
let uisp_data = UispData::fetch_uisp_data(config.clone(), ip_ranges).await?;
build_imported_full2_bundle_from_data(
config,
uisp_data,
require_configured_root_site,
build_started,
)
.await
}
fn add_device_links_to_graph(
uisp_data: &UispData,
graph: &mut GraphType,
@@ -1778,32 +1786,7 @@ fn topology_allowed_parents_from_groups_cached(
let mut allowed_parents = Vec::with_capacity(groups.len());
for group in groups {
let parent = {
let mut attachment_options = vec![TopologyAttachmentOption {
attachment_id: TOPOLOGY_ATTACHMENT_AUTO_ID.to_string(),
attachment_name: "Auto".to_string(),
attachment_kind: "auto".to_string(),
attachment_role: TopologyAttachmentRole::Unknown,
pair_id: None,
peer_attachment_id: None,
peer_attachment_name: None,
capacity_mbps: None,
download_bandwidth_mbps: None,
upload_bandwidth_mbps: None,
transport_cap_mbps: None,
transport_cap_reason: None,
rate_source: TopologyAttachmentRateSource::Unknown,
can_override_rate: false,
rate_override_disabled_reason: None,
has_rate_override: false,
local_probe_ip: None,
remote_probe_ip: None,
probe_enabled: false,
probeable: false,
health_status: lqos_config::TopologyAttachmentHealthStatus::Disabled,
health_reason: None,
suppressed_until_unix: None,
effective_selected: false,
}];
let mut attachment_options = vec![topology_auto_attachment_option()];
let mut seen_attachment_ids = HashSet::from([TOPOLOGY_ATTACHMENT_AUTO_ID.to_string()]);
attachment_options.extend(
@@ -2202,18 +2185,23 @@ mod topology_override_tests {
use super::{
GraphType, TopologyAllowedParentGroup, TopologyAttachmentOverrideSelection,
TopologyParentOverrideSelection, UispDevice, build_constrained_route,
build_imported_full2_bundle_from_data,
export_parent_anchor_for_override, first_probe_ip_for_device, immediate_parent_from_route,
is_upstream_parent_candidate, logical_parent_for_candidate,
resolve_attachment_parent_candidate, resolve_parent_candidate,
topology_allowed_parent_groups_for_node, topology_allowed_parents_from_groups,
topology_parent_candidates_for_node,
};
use crate::strategies::common::UispData;
use crate::strategies::full2::graph_mapping::GraphMapping;
use crate::strategies::full2::link_mapping::LinkMapping;
use crate::uisp_types::UispAttachmentRateSource;
use lqos_config::TopologyAttachmentRole;
use crate::uisp_types::{UispAttachmentRateSource, UispSite, UispSiteType};
use lqos_config::{TopologyAttachmentRole, TopologyQueueVisibilityPolicy};
use lqos_overrides::{TopologyAttachmentMode, TopologyParentOverrideMode};
use serde_json::{Map, Value, json};
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
use uisp::{DataLink, Device, Site};
fn site(name: &str, id: &str) -> GraphMapping {
GraphMapping::Site {
@@ -2234,6 +2222,267 @@ mod topology_override_tests {
}
}
fn unique_temp_dir(name: &str) -> std::path::PathBuf {
let unique = format!(
"libreqos-uisp-full2-{name}-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap_or_default()
.as_nanos()
);
let path = std::env::temp_dir().join(unique);
std::fs::create_dir_all(&path).expect("temp dir should be created");
path
}
fn raw_site(id: &str, name: &str, site_type: &str) -> Site {
serde_json::from_value(json!({
"id": id,
"identification": {
"id": id,
"name": name,
"type": site_type,
"suspended": false
}
}))
.expect("raw site fixture should deserialize")
}
fn raw_device(id: &str, name: &str, site_id: &str) -> Device {
serde_json::from_value(json!({
"identification": {
"id": id,
"hostname": name,
"site": {
"id": site_id
}
}
}))
.expect("raw device fixture should deserialize")
}
struct LinkEndpoint<'a> {
site_id: &'a str,
site_name: &'a str,
device_id: &'a str,
device_name: &'a str,
}
fn data_link_endpoint(endpoint: &LinkEndpoint<'_>) -> serde_json::Value {
json!({
"site": {
"identification": {
"id": endpoint.site_id,
"name": endpoint.site_name
}
},
"device": {
"identification": {
"id": endpoint.device_id,
"name": endpoint.device_name
}
}
})
}
fn device_data_link(id: &str, from: LinkEndpoint<'_>, to: LinkEndpoint<'_>) -> DataLink {
serde_json::from_value(json!({
"id": id,
"from": data_link_endpoint(&from),
"to": data_link_endpoint(&to),
"canDelete": true
}))
.expect("data link fixture should deserialize")
}
fn parsed_site(id: &str, name: &str) -> UispSite {
UispSite {
id: id.to_string(),
name: name.to_string(),
site_type: UispSiteType::Site,
max_down_mbps: 10_000,
max_up_mbps: 10_000,
..Default::default()
}
}
fn parsed_device(id: &str, name: &str, site_id: &str) -> UispDevice {
UispDevice {
id: id.to_string(),
name: name.to_string(),
mac: String::new(),
role: None,
wireless_mode: None,
site_id: site_id.to_string(),
raw_download: 10_000,
raw_upload: 10_000,
download: 10_000,
upload: 10_000,
ipv4: HashSet::new(),
ipv6: HashSet::new(),
probe_ipv4: HashSet::new(),
probe_ipv6: HashSet::new(),
negotiated_ethernet_mbps: None,
negotiated_ethernet_interface: None,
transport_cap_mbps: None,
transport_cap_reason: None,
attachment_rate_source: UispAttachmentRateSource::Static,
}
}
fn find_network_node_by_id<'a>(
value: &'a Value,
target_id: &str,
) -> Option<&'a Map<String, Value>> {
let map = value.as_object()?;
if map.get("id").and_then(Value::as_str) == Some(target_id) {
return Some(map);
}
map.values()
.find_map(|child| find_network_node_by_id(child, target_id))
}
#[tokio::test]
async fn full2_import_emits_queue_auto_for_device_backed_branches() {
let temp_dir = unique_temp_dir("uisp-full2-queue-auto");
let mut config = lqos_config::Config {
lqos_directory: temp_dir.to_string_lossy().to_string(),
state_directory: Some(temp_dir.join("state").to_string_lossy().to_string()),
..Default::default()
};
config.uisp_integration.enable_uisp = true;
config.uisp_integration.site = "Root Site".to_string();
config.queues.generated_pn_download_mbps = 10_000;
config.queues.generated_pn_upload_mbps = 10_000;
config.topology.queue_auto_virtualize_threshold_mbps = 5_000;
let config_for_topology = config.clone();
let root_site = raw_site("site-root", "Root Site", "site");
let child_site = raw_site("site-child", "Child Site", "site");
let root_device = raw_device("device-root-switch", "Root Switch", "site-root");
let child_device = raw_device("device-child-switch", "Child Switch", "site-child");
let root_parsed_device = parsed_device("device-root-switch", "Root Switch", "site-root");
let child_parsed_device =
parsed_device("device-child-switch", "Child Switch", "site-child");
let uisp_data = UispData::from_parts(
vec![root_site, child_site],
vec![root_device, child_device],
vec![device_data_link(
"link-root-child",
LinkEndpoint {
site_id: "site-root",
site_name: "Root Site",
device_id: "device-root-switch",
device_name: "Root Switch",
},
LinkEndpoint {
site_id: "site-child",
site_name: "Child Site",
device_id: "device-child-switch",
device_name: "Child Switch",
},
)],
vec![
parsed_site("site-root", "Root Site"),
parsed_site("site-child", "Child Site"),
],
vec![root_parsed_device, child_parsed_device],
);
let bundle = build_imported_full2_bundle_from_data(
Arc::new(config),
uisp_data,
true,
std::time::Instant::now(),
)
.await
.expect("in-memory UISP import should build");
let editor = bundle
.native_editor
.as_ref()
.expect("full2 import should include native editor state");
let root_switch = editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:device-root-switch")
.expect("root switch should be exported as a topology editor node");
assert_eq!(
root_switch.queue_visibility_policy,
TopologyQueueVisibilityPolicy::QueueAuto
);
let compiled = lqos_topology_compile::compile_topology(
bundle,
lqos_topology_compile::TopologyCompileMode::Full,
)
.expect("full topology should compile from full2 import");
let compiled_child_switch = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:device:device-child-switch")
.expect("compiled editor should include the child switch");
assert_eq!(
compiled_child_switch.queue_visibility_policy,
TopologyQueueVisibilityPolicy::QueueAuto
);
assert_eq!(
compiled_child_switch.current_parent_node_id.as_deref(),
Some("uisp:device:device-root-switch")
);
assert!(
compiled_child_switch
.allowed_parents
.iter()
.any(|parent| parent.parent_node_id == "uisp:device:device-root-switch")
);
let compiled_child_site = compiled
.editor
.nodes
.iter()
.find(|node| node.node_id == "uisp:site:site-child")
.expect("compiled editor should include the child site");
assert_eq!(
compiled_child_site.queue_visibility_policy,
TopologyQueueVisibilityPolicy::QueueAuto
);
assert_eq!(
compiled_child_site.current_parent_node_id.as_deref(),
Some("uisp:device:device-child-switch")
);
assert!(
compiled_child_site
.allowed_parents
.iter()
.any(|parent| parent.parent_node_id == "uisp:device:device-child-switch")
);
let artifacts = lqos_topology::build_effective_topology_artifacts_from_canonical(
&config_for_topology,
&compiled.canonical,
&lqos_overrides::TopologyOverridesFile::default(),
&lqos_config::TopologyAttachmentHealthStateFile::default(),
)
.expect("effective topology should compile from full2 canonical output");
let effective_network = artifacts
.effective_network
.as_ref()
.expect("effective topology should include a network tree");
let effective_root_switch =
find_network_node_by_id(effective_network, "uisp:device:device-root-switch")
.expect("root switch should remain visible in the effective tree");
assert_eq!(
effective_root_switch.get("virtual").and_then(Value::as_bool),
Some(true)
);
let _ = std::fs::remove_dir_all(temp_dir);
}
fn build_test_graph() -> (
GraphType,
petgraph::graph::NodeIndex,