This commit is contained in:
Robert Chacon committed 2026-03-29 15:22:52 -06:00
1 parent 01bd46012d
commit d23b8cce9a
35 files changed
+2102 -333

No files matched your search

@@ -127,6 +127,8 @@ If a network.json file exists, it will not be overwritten, unless you set ```alw
ShapedDevices.csv will be overwritten every time the UISP integration is run.
Cuando varios sitios cliente de UISP comparten el mismo nombre, LibreQoS ahora intenta diferenciar los nombres visibles generados para circuitos/sitios con un sufijo más humano, como el primer segmento de la dirección, recurriendo al nombre del servicio y luego a un ID corto solo cuando hace falta. La identidad estable del circuito sigue viniendo del ID de sitio/servicio de UISP, no del nombre visible.
To ensure the network.json is always overwritten with the newest version pulled in by the integration, please edit `/etc/lqos.conf` with the command `sudo nano /etc/lqos.conf`.
Edit the file to set the value of `always_overwrite_network_json` to `true`.
Then, run `sudo systemctl restart lqosd`.
+8 -1
View File
@@ -13,7 +13,7 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta
- El widget `Pipeline` de Bakery muestra las etapas actuales del control de colas, incluyendo el estado activo de aplicación, el estado de verificación y la temporización del intervalo de TC.
- `Runtime Operations` resume mutaciones en vivo de topología entre TreeGuard y Bakery, trabajo de limpieza diferida, fallos, subárboles marcados como `Dirty` y si los cambios incrementales están congelados hasta un full reload.
- `Recent Bakery Events` muestra por defecto operaciones agrupadas orientadas al operador, y deja el log crudo de eventos de Bakery en una vista separada de `Event Log` cuando hace falta troubleshooting detallado.
- `TreeGuard Activity` muestra por defecto operaciones agrupadas orientadas al operador, incluyendo lotes consolidados de cambios SQM, y deja el log crudo de eventos de TreeGuard en una vista separada de `Event Log` cuando hace falta troubleshooting detallado.
- `TreeGuard Activity` muestra por defecto operaciones agrupadas orientadas al operador, incluyendo lotes consolidados de cambios SQM, y deja el log crudo de eventos de TreeGuard en una vista separada de `Event Log` cuando hace falta troubleshooting detallado; ambas vistas usan paginación.
- `TreeGuard Control Loop` muestra el estado actual de observar/evaluar/actuar sin repetir acciones recientes que ya aparecen en TreeGuard Activity.
- `TreeGuard Decision Impact` se centra en el impacto actual y en las advertencias o errores actuales, en lugar de repetir acciones recientes.
- `TreeGuard State Mix` muestra nodos gestionados, virtualización en runtime, circuitos gestionados y la mezcla actual de circuitos `cake / mixed / fq_codel`.
@@ -28,6 +28,7 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta
- `Node Details` resume el tipo de nodo seleccionado, el tamaño de la rama, las velocidades configuradas y la velocidad efectiva actual.
- `Node Snapshot` ofrece un resumen visual rápido del throughput actual y el QoO del nodo seleccionado.
- Los circuitos adjuntos se muestran en una tabla dedicada para el nodo seleccionado.
- Los circuitos adjuntos limitados por Ethernet pueden mostrar insignias `10M`, `100M` o `1G` junto al valor de `Plan (Mbps)`; al pasar el cursor se explica el auto-cap y al hacer clic en la insignia se abre la página dedicada de revisión Ethernet.
- Los administradores pueden guardar o limpiar valores de `Operator Override` cuando el nodo admite overrides a nivel de nodo. Los usuarios de solo lectura y los nodos no compatibles siguen mostrando los valores actuales sin controles de edición.
### Site Map
@@ -61,6 +62,7 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta
### Página de circuito
- Las páginas de circuito combinan comportamiento de colas, throughput en vivo, RTT, retransmisiones y troubleshooting por flujo para un suscriptor/circuito individual.
- Cuando los metadatos de la integración informan la velocidad Ethernet negociada del CPE, la fila `Max` puede mostrar una insignia de advertencia como `100M`; al pasar el cursor sobre la insignia se explica cuándo LibreQoS redujo automáticamente el shaping por debajo del plan solicitado para respetar ese límite del puerto, y al hacer clic en la insignia se abre la página de revisión Ethernet.
- `Queue Dynamics` muestra el comportamiento del throughput y RTT del circuito a lo largo del tiempo, incluyendo un KPI de `Active Flows` basado en la misma ventana reciente usada por la tabla `Traffic Flows`.
- `Queue Stats` muestra los 3 minutos más recientes del historial en vivo de la cola del circuito como muestras scatter crudas de 1 segundo, incluyendo backlog, delay, longitud de cola, tráfico, marcas ECN y drops.
- Los gráficos de Queue Stats usan hover sincronizado para inspeccionar el mismo segundo en todos los gráficos de cola al mismo tiempo.
@@ -69,6 +71,11 @@ Esta página documenta las vistas clave de la WebUI (Node Manager) y su comporta
- `Traffic Flows` incluye paginación y un filtro `Hide Small Flows` para que los circuitos grandes y ocupados sigan siendo utilizables sin intentar renderizar cada fila.
- `Flow Sankey` enfatiza los flujos recientes más activos en lugar de todos los flujos retenidos más antiguos.
### Ethernet Caps
- La página de revisión Ethernet es una tabla ligera para operadores con los circuitos reducidos automáticamente porque la velocidad Ethernet detectada quedó por debajo del plan solicitado.
- Intencionalmente no aparece en la navegación principal; los operadores llegan a ella haciendo clic en las insignias de advertencia Ethernet de la página de circuito o de la tabla de circuitos adjuntos del árbol.
- La página soporta búsqueda, filtro por tier (`10M`, `100M`, `1G+`) y paginación sobre los circuitos actualmente auto-capped.
### Árbol/ponderación de CPU
- Muestra distribución de colas/circuitos por núcleo de CPU.
- Ayuda a validar comportamiento de binpacking y balance de carga.
+2
View File
@@ -173,6 +173,8 @@ If a `network.json` file exists, it is only overwritten when `always_overwrite_n
ShapedDevices.csv will be overwritten every time the UISP integration is run.
When UISP client sites share the same name, LibreQoS now tries to disambiguate the generated circuit/site display names with a human-friendly suffix such as the first street-address segment, falling back to service name and then a short ID only when needed. Stable circuit identity still comes from the UISP site/service ID, not the display name.
For integration-driven deployments, keep `always_overwrite_network_json = true` so topology stays aligned with UISP on each refresh cycle.
You have the option to run `uisp_integration` automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_uisp = true``` in `/etc/lqos.conf`. Once set, run `sudo systemctl restart lqos_scheduler`.
+8 -1
View File
@@ -13,7 +13,7 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
- The Bakery `Pipeline` widget shows the current queue-control stages, including active apply state, verification state, and TC interval timing.
- `Runtime Operations` summarizes live TreeGuard/Bakery topology mutations, deferred cleanup work, failures, dirty subtrees, and whether incremental changes are frozen pending a full reload.
- `Recent Bakery Events` defaults to merged operator-facing operations, with the raw Bakery event log available in a separate `Event Log` view when detailed troubleshooting is needed.
- `TreeGuard Activity` defaults to grouped operator-facing operations, including consolidated SQM change batches, with the raw TreeGuard event log available in a separate `Event Log` view when detailed troubleshooting is needed.
- `TreeGuard Activity` defaults to grouped operator-facing operations, including consolidated SQM change batches, with the raw TreeGuard event log available in a separate `Event Log` view when detailed troubleshooting is needed; both views are paginated.
- `TreeGuard Control Loop` shows the current observe/evaluate/act state without repeating recent actions already visible in TreeGuard Activity.
- `TreeGuard Decision Impact` focuses on current impact and current warnings or errors, rather than replaying recent actions.
- `TreeGuard State Mix` shows managed nodes, runtime virtualization, managed circuits, and the current `cake / mixed / fq_codel` circuit SQM split.
@@ -28,6 +28,7 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
- `Node Details` summarizes the selected node’s type, branch size, configured rates, and current effective rate.
- `Node Snapshot` provides a quick visual summary of current throughput and QoO for the selected node.
- Attached circuits are shown in a dedicated table for the selected node.
- Ethernet-limited attached circuits can show inline `10M`, `100M`, or `1G` warning badges beside the `Plan (Mbps)` value; hovering explains the auto-cap and clicking the badge opens the dedicated Ethernet review page.
- Administrators can save or clear `Operator Override` values where node-level overrides are supported. Read-only users and unsupported nodes continue to display current values without edit controls.
- Tree-page operator rate edits write operator-owned overrides to `lqos_overrides.json`; they do not rewrite legacy integration bandwidth CSV files.
- Tree-page operator rate edits require an administrator session, a stable node ID, and a non-generated node. Generated/integration-synthetic nodes remain read-only in this editor.
@@ -64,6 +65,7 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
### Circuit page
- Circuit pages combine queue behavior, live throughput, RTT, retransmits, and per-flow troubleshooting for an individual subscriber/circuit.
- When integration metadata reports a negotiated CPE Ethernet speed, the `Max` row can show a warning badge such as `100M`; hovering the badge explains when LibreQoS auto-capped shaping below the requested plan to stay within that port speed, and clicking the badge opens the Ethernet review page.
- `Queue Dynamics` shows circuit throughput and RTT behavior over time, including an `Active Flows` KPI based on the same recent flow window used by the `Traffic Flows` table.
- `Queue Stats` shows the most recent 3 minutes of live queue history for the circuit as raw 1-second scatter samples, including backlog, delay, queue length, traffic, ECN marks, and drops.
- Queue Stats charts use synchronized hover so operators can inspect the same second across all queue charts together.
@@ -72,6 +74,11 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
- `Traffic Flows` includes paging and a `Hide Small Flows` filter so large busy circuits remain usable without trying to render every row at once.
- `Flow Sankey` emphasizes the hottest recent flows rather than every older retained flow.
### Ethernet Caps
- The Ethernet review page is a lightweight operator table of circuits automatically down-rated because detected Ethernet speed was below the requested plan.
- It is intentionally not in the main navigation; operators reach it by clicking Ethernet warning badges on the Circuit page or Tree attached-circuits table.
- The page supports search, tier filtering (`10M`, `100M`, `1G+`), and paging across currently auto-capped circuits.
### CPU Tree / CPU Weights
- Shows queue/circuit distribution by CPU core.
- Helps evaluate binpacking and load distribution behavior.
+93 -46
View File
@@ -1246,26 +1246,18 @@ struct GroupedBakeryEventLimiter {
}
impl GroupedBakeryEventLimiter {
fn emit(
fn emit_with_site_name(
&mut self,
group_key: impl Into<String>,
event: &str,
status: &str,
site: Option<(i64, Option<String>)>,
summary: String,
suppression_summary: String,
) {
self.emit_with_site(group_key, event, status, None, summary, suppression_summary);
}
fn emit_with_site(
&mut self,
group_key: impl Into<String>,
event: &str,
status: &str,
site_hash: Option<i64>,
summary: String,
suppression_summary: String,
) {
let (site_hash, site_name) = site.map_or((None, None), |(site_hash, site_name)| {
(Some(site_hash), site_name)
});
let state =
self.groups
.entry(group_key.into())
@@ -1277,7 +1269,7 @@ impl GroupedBakeryEventLimiter {
suppression_summary,
});
if state.emitted < BAKERY_GROUPED_EVENT_DETAIL_LIMIT {
push_bakery_event_with_site(event, status, site_hash, summary);
push_bakery_event_with_site_name(event, status, site_hash, site_name, summary);
state.emitted += 1;
} else {
state.suppressed += 1;
@@ -2542,6 +2534,13 @@ fn runtime_site_label(site_hash: i64, site_name: Option<&str>) -> String {
}
}
fn runtime_site_display_name(site_hash: i64, site_name: Option<&str>) -> String {
match site_name {
Some(name) if !name.is_empty() => name.to_string(),
_ => site_hash.to_string(),
}
}
fn runtime_circuit_label(circuit_hash: i64, circuit_name: Option<&str>) -> String {
match circuit_name {
Some(name) if !name.is_empty() => format!("{name} ({circuit_hash})"),
@@ -8375,6 +8374,9 @@ fn flush_deferred_runtime_site_prunes(
);
for site_hash in pending_site_hashes {
if let Some(state) = virtualized_sites.get_mut(&site_hash) {
let site_name = state.site_name.clone();
let site_display =
runtime_site_display_name(site_hash, Some(site_name.as_str()));
let retry_at = now_unix.saturating_add(RUNTIME_SITE_PRUNE_RETRY_SECONDS);
state.next_prune_attempt_unix = retry_at;
if let Some(operation) = runtime_node_operations.get_mut(&site_hash) {
@@ -8389,13 +8391,14 @@ fn flush_deferred_runtime_site_prunes(
Some(summary.clone()),
None,
);
grouped_events.emit(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_dirty|down_snapshot|{summary}"),
"runtime_site_prune_dirty",
"error",
Some((site_hash, Some(site_name.clone()))),
format!(
"Deferred runtime site prune for site {} marked Dirty after snapshot failure: {}",
site_hash, summary
site_display, summary
),
format!(
"runtime site prune Dirty events due to downlink snapshot failure: {}",
@@ -8411,15 +8414,16 @@ fn flush_deferred_runtime_site_prunes(
Some(summary.clone()),
Some(retry_at),
);
grouped_events.emit(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_retry|down_snapshot|{summary}"),
"runtime_site_prune_retry",
"warning",
Some((site_hash, Some(site_name))),
format!(
"Deferred runtime site prune retry {}/{} for site {} postponed: {}",
operation.attempt_count,
RUNTIME_SITE_PRUNE_MAX_ATTEMPTS,
site_hash,
site_display,
summary
),
format!(
@@ -8443,6 +8447,9 @@ fn flush_deferred_runtime_site_prunes(
);
for site_hash in pending_site_hashes {
if let Some(state) = virtualized_sites.get_mut(&site_hash) {
let site_name = state.site_name.clone();
let site_display =
runtime_site_display_name(site_hash, Some(site_name.as_str()));
let retry_at = now_unix.saturating_add(RUNTIME_SITE_PRUNE_RETRY_SECONDS);
state.next_prune_attempt_unix = retry_at;
if let Some(operation) = runtime_node_operations.get_mut(&site_hash) {
@@ -8457,13 +8464,14 @@ fn flush_deferred_runtime_site_prunes(
Some(summary.clone()),
None,
);
grouped_events.emit(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_dirty|up_snapshot|{summary}"),
"runtime_site_prune_dirty",
"error",
Some((site_hash, Some(site_name.clone()))),
format!(
"Deferred runtime site prune for site {} marked Dirty after snapshot failure: {}",
site_hash, summary
site_display, summary
),
format!(
"runtime site prune Dirty events due to uplink snapshot failure: {}",
@@ -8479,15 +8487,16 @@ fn flush_deferred_runtime_site_prunes(
Some(summary.clone()),
Some(retry_at),
);
grouped_events.emit(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_retry|up_snapshot|{summary}"),
"runtime_site_prune_retry",
"warning",
Some((site_hash, Some(site_name))),
format!(
"Deferred runtime site prune retry {}/{} for site {} postponed: {}",
operation.attempt_count,
RUNTIME_SITE_PRUNE_MAX_ATTEMPTS,
site_hash,
site_display,
summary
),
format!(
@@ -8557,6 +8566,8 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
if state.lifecycle == RuntimeVirtualizedBranchLifecycle::CutoverPending
&& state.active_branch_hides_original_site()
{
let site_name = state.site_name.clone();
let site_display = runtime_site_display_name(site_hash, Some(site_name.as_str()));
debug!(
"Bakery: evaluating runtime cutover activation for site {}",
site_hash
@@ -8574,14 +8585,14 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
None,
);
}
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
"runtime_cutover_completed|completed",
"runtime_cutover_completed",
"info",
Some(site_hash),
Some((site_hash, Some(site_name.clone()))),
format!(
"Runtime cutover completed for site {}; shadow branch is active and original branch is standby.",
site_hash
site_display
),
"runtime cutover completion events".to_string(),
);
@@ -8604,14 +8615,14 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
Some(summary.clone()),
None,
);
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_cutover_dirty|activation_failed|{summary}"),
"runtime_cutover_dirty",
"error",
Some(site_hash),
Some((site_hash, Some(site_name.clone()))),
format!(
"Runtime cutover for site {} marked Dirty after {} attempts: {}",
site_hash, operation.attempt_count, summary
site_display, operation.attempt_count, summary
),
format!(
"runtime cutover Dirty events due to activation verification failure: {}",
@@ -8634,16 +8645,16 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
Some(summary.clone()),
Some(retry_at),
);
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_cutover_retry|activation_failed|{summary}"),
"runtime_cutover_retry",
"warning",
Some(site_hash),
Some((site_hash, Some(site_name.clone()))),
format!(
"Runtime cutover retry {}/{} for site {} waiting for active/standby convergence: {}",
operation.attempt_count,
RUNTIME_SITE_PRUNE_MAX_ATTEMPTS,
site_hash,
site_display,
summary
),
format!(
@@ -8681,14 +8692,14 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
None,
);
}
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
"runtime_site_prune_completed|completed",
"runtime_site_prune_completed",
"info",
Some(site_hash),
Some((site_hash, Some(state.site_name.clone()))),
format!(
"Deferred runtime site prune completed for site {}.",
site_hash
runtime_site_display_name(site_hash, Some(state.site_name.as_str()))
),
"runtime site prune completion events".to_string(),
);
@@ -8713,14 +8724,15 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
Some(now_unix),
);
}
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_progress|{summary}"),
"runtime_site_prune_progress",
"info",
Some(site_hash),
Some((site_hash, Some(state.site_name.clone()))),
format!(
"Deferred runtime site prune for site {} made progress: {}",
site_hash, summary
runtime_site_display_name(site_hash, Some(state.site_name.as_str())),
summary
),
format!("runtime site prune progress events: {}", summary),
);
@@ -8763,14 +8775,16 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
Some(summary.clone()),
None,
);
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_dirty|execute_failed|{summary}"),
"runtime_site_prune_dirty",
"error",
Some(site_hash),
Some((site_hash, Some(state.site_name.clone()))),
format!(
"Deferred runtime site prune for site {} marked Dirty after {} attempts: {}",
site_hash, operation.attempt_count, summary
runtime_site_display_name(site_hash, Some(state.site_name.as_str())),
operation.attempt_count,
summary
),
format!(
"runtime site prune Dirty events due to execution failure: {}",
@@ -8788,16 +8802,19 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
Some(summary.clone()),
Some(retry_at),
);
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_retry|execute_failed|{summary}"),
"runtime_site_prune_retry",
"warning",
Some(site_hash),
Some((site_hash, Some(state.site_name.clone()))),
format!(
"Deferred runtime site prune retry {}/{} for site {} failed: {}",
operation.attempt_count,
RUNTIME_SITE_PRUNE_MAX_ATTEMPTS,
site_hash,
runtime_site_display_name(
site_hash,
Some(state.site_name.as_str())
),
summary
),
format!(
@@ -8809,14 +8826,15 @@ fn flush_deferred_runtime_site_prunes_with_snapshots(
} else {
state.next_prune_attempt_unix =
now_unix.saturating_add(RUNTIME_SITE_PRUNE_RETRY_SECONDS);
grouped_events.emit_with_site(
grouped_events.emit_with_site_name(
format!("runtime_site_prune_retry|outside_tracking|{summary}"),
"runtime_site_prune_retry",
"warning",
Some(site_hash),
Some((site_hash, Some(state.site_name.clone()))),
format!(
"Deferred runtime site prune for site {} failed outside operation tracking: {}",
site_hash, summary
runtime_site_display_name(site_hash, Some(state.site_name.as_str())),
summary
),
format!(
"runtime site prune retry events outside operation tracking: {}",
@@ -11913,6 +11931,35 @@ mod tests {
);
}
#[test]
fn grouped_runtime_site_events_retain_human_site_name() {
let _guard = bakery_test_lock().lock().expect("lock");
reset_bakery_test_state();
let site_hash = 3571466403324592518i64;
let site_name = "JR_AP_TR_F".to_string();
let mut grouped = GroupedBakeryEventLimiter::default();
grouped.emit_with_site_name(
"runtime_cutover_completed|completed",
"runtime_cutover_completed",
"info",
Some((site_hash, Some(site_name.clone()))),
format!(
"Runtime cutover completed for site {}; shadow branch is active and original branch is standby.",
runtime_site_display_name(site_hash, Some(site_name.as_str()))
),
"runtime cutover completion events".to_string(),
);
grouped.flush();
let activity = bakery_activity_snapshot();
assert_eq!(activity.len(), 1);
assert_eq!(activity[0].site_hash, Some(site_hash));
assert_eq!(activity[0].site_name.as_deref(), Some(site_name.as_str()));
assert!(activity[0].summary.contains(site_name.as_str()));
assert!(!activity[0].summary.contains(&format!("site {site_hash}")));
}
#[test]
fn qdisc_budget_estimate_counts_fq_codel_leaf_qdiscs_separately() {
let config = Arc::new(Config::default());
@@ -0,0 +1,42 @@
use serde::{Deserialize, Serialize};
/// Runtime metadata filename for circuit Ethernet advisories emitted by integrations.
pub const CIRCUIT_ETHERNET_METADATA_FILENAME: &str = "circuit_ethernet_metadata.json";
/// Collection of circuit Ethernet advisories keyed by circuit identity.
#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)]
pub struct CircuitEthernetMetadataFile {
/// Per-circuit Ethernet advisory entries.
pub circuits: Vec<CircuitEthernetMetadata>,
}
/// Describes a detected negotiated Ethernet speed and any automatic shaping cap applied to a circuit.
#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)]
pub struct CircuitEthernetMetadata {
/// Circuit identifier as emitted to `ShapedDevices.csv`.
pub circuit_id: String,
/// Human-readable circuit name for UI display.
pub circuit_name: String,
/// Device IDs considered when determining the circuit Ethernet limit.
pub device_ids: Vec<String>,
/// Integration/source that produced the advisory.
pub source: String,
/// Negotiated Ethernet speed in Mbps for the limiting device/interface.
pub negotiated_ethernet_mbps: u64,
/// Requested download max before any Ethernet-based cap was applied.
pub requested_download_mbps: f32,
/// Requested upload max before any Ethernet-based cap was applied.
pub requested_upload_mbps: f32,
/// Applied download max after Ethernet-based capping.
pub applied_download_mbps: f32,
/// Applied upload max after Ethernet-based capping.
pub applied_upload_mbps: f32,
/// Whether the Ethernet advisory reduced at least one shaping direction.
pub auto_capped: bool,
/// Device ID of the limiting device when known.
pub limiting_device_id: Option<String>,
/// Device name of the limiting device when known.
pub limiting_device_name: Option<String>,
/// Interface name that reported the limiting Ethernet speed when known.
pub limiting_interface_name: Option<String>,
}
+4
View File
@@ -8,6 +8,7 @@
#![deny(clippy::unwrap_used)]
#![warn(missing_docs)]
pub mod authentication;
mod circuit_ethernet_metadata;
mod cpu_topology;
mod etc;
mod network_json;
@@ -17,6 +18,9 @@ mod qoo_profiles;
mod shaped_devices;
pub use authentication::{AuthenticatedUser, UserRole, WebUser, WebUsers};
pub use circuit_ethernet_metadata::{
CIRCUIT_ETHERNET_METADATA_FILENAME, CircuitEthernetMetadata, CircuitEthernetMetadataFile,
};
pub use cpu_topology::{
CpuListParseError, ShapingCpuDetection, ShapingCpuSource, detect_shaping_cpus,
};
@@ -8,6 +8,7 @@ help.js
unknown-ips.js
configuration.js
circuit.js
ethernet_caps.js
site_map.js
flow_map.js
all_tree_sankey.js
@@ -75,6 +75,72 @@ const RECENT_TRAFFIC_FLOW_WINDOW_NANOS = 30_000_000_000;
const TRAFFIC_FLOW_HIDE_THRESHOLD_BPS = 1024 * 1024;
const DEFAULT_TRAFFIC_PAGE_SIZE = 100;
function formatEthernetPortLabel(mbps) {
const value = toNumber(mbps, 0);
if (value >= 1000 && value % 1000 === 0) {
return `${value / 1000}G`;
}
if (value >= 1000) {
return `${(value / 1000).toFixed(1)}G`;
}
return `${Math.round(value)}M`;
}
function ethernetCapsPageHref(advisory) {
const portLabel = encodeURIComponent(formatEthernetPortLabel(advisory?.negotiated_ethernet_mbps));
return `/ethernet_caps.html?tier=${portLabel}`;
}
function ethernetTooltipHtml(advisory) {
return (
`Requested plan ${formatMbps(advisory.requested_download_mbps)} / ${formatMbps(advisory.requested_upload_mbps)} exceeded detected Ethernet speed. ` +
`Shaping auto-capped to ${formatMbps(advisory.applied_download_mbps)} / ${formatMbps(advisory.applied_upload_mbps)}.`
);
}
function formatPlanSpeedPair(downloadMbps, uploadMbps) {
const down = toNumber(downloadMbps, 0).toFixed(1);
const up = toNumber(uploadMbps, 0).toFixed(1);
return `${down} / ${up} Mbps`;
}
function renderEthernetAdvisory(advisory) {
const badge = document.getElementById("ethernetAdvisoryBadge");
const badgeText = document.getElementById("ethernetAdvisoryBadgeText");
if (!badge || !badgeText) {
return;
}
const disposeTooltip = () => {
if (typeof bootstrap === "undefined" || !bootstrap.Tooltip) {
return;
}
const existing = bootstrap.Tooltip.getInstance?.(badge);
existing?.dispose();
};
if (!advisory?.auto_capped) {
badge.classList.add("d-none");
badgeText.textContent = "";
badge.removeAttribute("title");
badge.removeAttribute("data-bs-original-title");
disposeTooltip();
return;
}
const portLabel = formatEthernetPortLabel(advisory.negotiated_ethernet_mbps);
const note = ethernetTooltipHtml(advisory);
badgeText.textContent = portLabel;
badge.classList.remove("d-none");
badge.href = ethernetCapsPageHref(advisory);
badge.setAttribute("title", note);
badge.setAttribute("data-bs-original-title", note);
badge.setAttribute("aria-label", `Review ${portLabel} Ethernet-limited circuits`);
disposeTooltip();
initTooltipsWithin(badge.parentElement || document);
}
function retransmitCountFromSample(sample) {
return toNumber(sample?.retransmits, 0);
}
@@ -1016,7 +1082,8 @@ function requestCircuitById(onSuccess, onError) {
if (onError) onError();
return;
}
onSuccess(msg.devices || []);
const payload = msg.data || {};
onSuccess(payload);
});
wsClient.send({ CircuitById: { id: circuit_id } });
}
@@ -2086,8 +2153,8 @@ function wireupAnalysis(circuits) {
//console.log(url);
// Restore the buttons
requestCircuitById((circuits) => {
wireupAnalysis(circuits);
requestCircuitById((payload) => {
wireupAnalysis(payload.devices || []);
});
}
return;
@@ -2119,13 +2186,16 @@ function loadInitial() {
initFlowFilters();
initQueuingActivityControls();
loadRttThresholds();
requestCircuitById((circuits) => {
requestCircuitById((payload) => {
const circuits = payload.devices || [];
const advisory = payload.ethernet_advisory || null;
let circuit = circuits[0];
$("#circuitName").text(circuit.circuit_name);
$("#circuitName").attr("title", circuit.circuit_name || "");
applyParentNodeLink(circuit.parent_node);
$("#bwMax").text(formatMbps(circuit.download_max_mbps) + " / " + formatMbps(circuit.upload_max_mbps));
$("#bwMin").text(formatMbps(circuit.download_min_mbps) + " / " + formatMbps(circuit.upload_min_mbps));
$("#bwMax").text(formatPlanSpeedPair(circuit.download_max_mbps, circuit.upload_max_mbps));
$("#bwMin").text(formatPlanSpeedPair(circuit.download_min_mbps, circuit.upload_min_mbps));
renderEthernetAdvisory(advisory);
plan = {
down: toNumber(circuit.download_max_mbps, 0),
up: toNumber(circuit.upload_max_mbps, 0),
@@ -6,6 +6,7 @@ import {renderOperationCards} from "./operation_cards";
const TREEGUARD_OPERATION_MERGE_WINDOW_SECONDS = 180;
const TREEGUARD_ACTIVITY_VIEW_STORAGE_KEY = "lqos_treeguard_activity_view";
const TREEGUARD_ACTIVITY_PAGE_SIZE = 8;
function formatUnixSecondsToLocalTime(unixSeconds) {
const n = typeof unixSeconds === "number" ? unixSeconds : parseInt(unixSeconds, 10);
@@ -490,6 +491,7 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
this.size = 12;
this.nodeIdByName = new Map();
this.lastEntries = [];
this.currentPage = 0;
this.viewMode = loadViewMode();
}
@@ -591,6 +593,46 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
tableWrap.appendChild(table);
this.eventsSection.appendChild(tableWrap);
wrap.appendChild(this.eventsSection);
const footer = document.createElement("div");
footer.classList.add("d-flex", "justify-content-between", "align-items-center", "gap-2", "pt-2", "flex-wrap");
this.pageSummary = document.createElement("div");
this.pageSummary.classList.add("small", "text-secondary");
footer.appendChild(this.pageSummary);
const pager = document.createElement("div");
pager.classList.add("btn-group", "btn-group-sm");
this.prevButton = document.createElement("button");
this.prevButton.type = "button";
this.prevButton.classList.add("btn", "btn-outline-secondary");
this.prevButton.innerHTML = "<i class='fa fa-chevron-left'></i> Newer";
this.prevButton.setAttribute("aria-label", "Show newer TreeGuard activity");
this.prevButton.addEventListener("click", () => {
if (this.currentPage > 0) {
this.currentPage -= 1;
this.renderEntries(this.lastEntries);
}
});
this.nextButton = document.createElement("button");
this.nextButton.type = "button";
this.nextButton.classList.add("btn", "btn-outline-secondary");
this.nextButton.innerHTML = "Older <i class='fa fa-chevron-right'></i>";
this.nextButton.setAttribute("aria-label", "Show older TreeGuard activity");
this.nextButton.addEventListener("click", () => {
if (this.currentPage + 1 < this.totalPages()) {
this.currentPage += 1;
this.renderEntries(this.lastEntries);
}
});
pager.appendChild(this.prevButton);
pager.appendChild(this.nextButton);
footer.appendChild(pager);
wrap.appendChild(footer);
base.appendChild(wrap);
this.renderViewMode();
return base;
@@ -598,10 +640,12 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
setViewMode(mode) {
this.viewMode = mode === "events" ? "events" : "operations";
this.currentPage = 0;
try {
window?.localStorage?.setItem(TREEGUARD_ACTIVITY_VIEW_STORAGE_KEY, this.viewMode);
} catch (_) {}
this.renderViewMode();
this.renderEntries(this.lastEntries);
}
renderViewMode() {
@@ -619,6 +663,16 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
}
}
totalPagesForCount(count) {
return Math.max(1, Math.ceil(count / TREEGUARD_ACTIVITY_PAGE_SIZE));
}
totalPages() {
const operationCount = buildTreeGuardOperationGroups(this.lastEntries).length;
const activeCount = this.viewMode === "events" ? this.lastEntries.length : operationCount;
return this.totalPagesForCount(activeCount);
}
onMessage(msg) {
if (msg.event !== "TreeGuardActivity") {
return;
@@ -636,7 +690,14 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
}
renderEntries(entries) {
const groups = buildTreeGuardOperationGroups(entries).map((group) => {
const operationGroups = buildTreeGuardOperationGroups(entries);
const totalRows = this.viewMode === "events" ? entries.length : operationGroups.length;
const totalPages = this.totalPagesForCount(totalRows);
this.currentPage = Math.min(this.currentPage, totalPages - 1);
const start = this.currentPage * TREEGUARD_ACTIVITY_PAGE_SIZE;
const end = Math.min(start + TREEGUARD_ACTIVITY_PAGE_SIZE, totalRows);
const pagedGroups = operationGroups.slice(start, end).map((group) => {
const fullReason = group.kind === "sqm_batch"
? summarizeSqmBatch(group)
: (group.reason.label || "");
@@ -658,7 +719,7 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
progressBarClass: treeguardProgressClass(group),
};
});
renderOperationCards(this.operationsList, groups, { emptyText: "No recent operations" });
renderOperationCards(this.operationsList, pagedGroups, { emptyText: "No recent operations" });
this.tbody.innerHTML = "";
@@ -670,11 +731,14 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
td.textContent = "No recent activity";
tr.appendChild(td);
this.tbody.appendChild(tr);
this.pageSummary.textContent = this.viewMode === "events" ? "0 events" : "0 operations";
this.prevButton.disabled = true;
this.nextButton.disabled = true;
this.renderViewMode();
return;
}
entries.slice(0, 50).forEach((entry) => {
entries.slice(start, end).forEach((entry) => {
const tr = document.createElement("tr");
tr.classList.add("small");
@@ -760,6 +824,11 @@ export class TreeGuardActivityDashlet extends BaseDashlet {
tr.appendChild(tdReason);
this.tbody.appendChild(tr);
});
const label = this.viewMode === "events" ? "events" : "operations";
this.pageSummary.textContent = `${start + 1}-${end} of ${totalRows} ${label}`;
this.prevButton.disabled = this.currentPage === 0;
this.nextButton.disabled = this.currentPage + 1 >= totalPages;
this.renderViewMode();
}
}
@@ -0,0 +1,220 @@
import {clearDiv, simpleRow, theading} from "./helpers/builders";
import {formatMbps} from "./helpers/scaling";
import {get_ws_client} from "./pubsub/ws";
const wsClient = get_ws_client();
const params = new Proxy(new URLSearchParams(window.location.search), {
get: (searchParams, prop) => searchParams.get(prop),
});
let page = 0;
let pageSize = 100;
let search = "";
let tier = "";
let totalRows = 0;
function sendWsRequest(responseEvent, request) {
return new Promise((resolve, reject) => {
let done = false;
const responseHandler = (msg) => {
if (done) return;
done = true;
wsClient.off(responseEvent, responseHandler);
wsClient.off("Error", errorHandler);
resolve(msg);
};
const errorHandler = (msg) => {
if (done) return;
done = true;
wsClient.off(responseEvent, responseHandler);
wsClient.off("Error", errorHandler);
reject(msg);
};
wsClient.on(responseEvent, responseHandler);
wsClient.on("Error", errorHandler);
wsClient.send(request);
});
}
function formatPlanPair(down, up) {
return `${formatMbps(down)} / ${formatMbps(up)} Mbps`;
}
function tierBadgeClass(tierLabel) {
if (tierLabel === "10M") return "text-bg-danger";
if (tierLabel === "100M") return "text-bg-warning";
return "text-bg-info";
}
function buildTierBadge(row) {
const badge = document.createElement("span");
badge.className = `badge rounded-pill ${tierBadgeClass(row?.badge?.tier_label || "")}`;
badge.textContent = row?.badge?.tier_label || "?";
return badge;
}
function tableRow(row) {
const tr = document.createElement("tr");
tr.classList.add("small");
const circuitCell = document.createElement("td");
const link = document.createElement("a");
link.href = `/circuit.html?id=${encodeURIComponent(row.circuit_id)}`;
link.classList.add("redactable");
link.textContent = row.circuit_name || row.circuit_id;
circuitCell.appendChild(link);
tr.appendChild(circuitCell);
tr.appendChild(simpleRow(row.parent_node || "-", true));
const ethernetCell = document.createElement("td");
ethernetCell.appendChild(buildTierBadge(row));
tr.appendChild(ethernetCell);
tr.appendChild(simpleRow(formatPlanPair(row.badge.requested_download_mbps, row.badge.requested_upload_mbps)));
tr.appendChild(simpleRow(formatPlanPair(row.badge.applied_download_mbps, row.badge.applied_upload_mbps)));
tr.appendChild(simpleRow(row.limiting_device_name || "-"));
tr.appendChild(simpleRow(row.limiting_interface_name || "-"));
return tr;
}
function renderTable(rows) {
const target = document.getElementById("ethernetCapsTableWrap");
if (!target) return;
clearDiv(target);
if (!Array.isArray(rows) || rows.length === 0) {
const empty = document.createElement("div");
empty.className = "text-muted small";
empty.textContent = "No Ethernet-limited circuits matched this filter.";
target.appendChild(empty);
return;
}
const wrap = document.createElement("div");
wrap.classList.add("lqos-table-wrap");
const table = document.createElement("table");
table.classList.add("lqos-table", "lqos-table-tight");
const thead = document.createElement("thead");
thead.appendChild(theading("Circuit"));
thead.appendChild(theading("Parent"));
thead.appendChild(theading("Ethernet"));
thead.appendChild(theading("Requested"));
thead.appendChild(theading("Applied"));
thead.appendChild(theading("Limiting Device"));
thead.appendChild(theading("Interface"));
table.appendChild(thead);
const tbody = document.createElement("tbody");
rows.forEach((row) => tbody.appendChild(tableRow(row)));
table.appendChild(tbody);
wrap.appendChild(table);
target.appendChild(wrap);
}
function updateSummary(query, rows) {
const summary = document.getElementById("ethernetCapsSummary");
const pager = document.getElementById("ethernetCapsPager");
const prev = document.getElementById("ethernetCapsPrev");
const next = document.getElementById("ethernetCapsNext");
if (!summary || !pager || !prev || !next) return;
const totalPages = Math.max(1, Math.ceil(totalRows / pageSize));
const currentPage = (query?.page ?? 0) + 1;
const start = totalRows === 0 ? 0 : (currentPage - 1) * pageSize + 1;
const end = totalRows === 0 ? 0 : start + Math.max(0, rows.length - 1);
summary.textContent = totalRows === 0
? "No active Ethernet caps"
: `${totalRows} Ethernet-limited circuits`;
pager.textContent = totalRows === 0
? "Page 1 / 1"
: `Showing ${start}–${end} of ${totalRows} • Page ${currentPage} / ${totalPages}`;
prev.disabled = currentPage <= 1;
next.disabled = currentPage >= totalPages;
}
async function requestPage() {
try {
const query = {
page,
page_size: pageSize,
};
if (search.trim()) {
query.search = search.trim();
}
if (tier) {
query.tier = tier === "10M" ? "TenM" : tier === "100M" ? "HundredM" : "GigPlus";
}
const msg = await sendWsRequest("EthernetCapsPage", {
EthernetCapsPage: { query },
});
const data = msg?.data || { rows: [], total_rows: 0, query };
totalRows = Number.isFinite(Number(data.total_rows)) ? Number(data.total_rows) : 0;
renderTable(data.rows || []);
updateSummary(data.query, data.rows || []);
} catch (_error) {
totalRows = 0;
renderTable([]);
updateSummary({ page: 0 }, []);
}
}
function bindControls() {
const searchInput = document.getElementById("ethernetCapsSearch");
const tierSelect = document.getElementById("ethernetCapsTier");
const pageSizeSelect = document.getElementById("ethernetCapsPageSize");
const prev = document.getElementById("ethernetCapsPrev");
const next = document.getElementById("ethernetCapsNext");
if (searchInput) {
searchInput.value = search;
searchInput.addEventListener("input", () => {
search = searchInput.value || "";
page = 0;
requestPage();
});
}
if (tierSelect) {
tierSelect.value = tier;
tierSelect.addEventListener("change", () => {
tier = tierSelect.value || "";
page = 0;
requestPage();
});
}
if (pageSizeSelect) {
pageSizeSelect.value = String(pageSize);
pageSizeSelect.addEventListener("change", () => {
const nextSize = parseInt(pageSizeSelect.value, 10);
pageSize = Number.isFinite(nextSize) && nextSize > 0 ? nextSize : 100;
page = 0;
requestPage();
});
}
prev?.addEventListener("click", () => {
page = Math.max(0, page - 1);
requestPage();
});
next?.addEventListener("click", () => {
const totalPages = Math.max(1, Math.ceil(totalRows / pageSize));
page = Math.min(totalPages - 1, page + 1);
requestPage();
});
}
function initFromQueryString() {
const rawTier = typeof params.tier === "string" ? params.tier.trim().toUpperCase() : "";
tier = rawTier === "10M" || rawTier === "100M" || rawTier === "1G" ? rawTier : "";
search = typeof params.search === "string" ? params.search : "";
}
wsClient.on("join", () => {
requestPage();
});
initFromQueryString();
bindControls();
requestPage();
@@ -180,6 +180,36 @@ function summarizeIpListForTable(ipList) {
return displayList.join(", ");
}
function formatEthernetPortLabel(mbps) {
const value = toNumber(mbps, 0);
if (value >= 1000 && value % 1000 === 0) {
return `${value / 1000}G`;
}
if (value >= 1000) {
return `${(value / 1000).toFixed(1)}G`;
}
return `${Math.round(value)}M`;
}
function ethernetCapsPageHref(badge) {
const tier = encodeURIComponent(formatEthernetPortLabel(badge?.negotiated_ethernet_mbps));
return `/ethernet_caps.html?tier=${tier}`;
}
function formatEthernetTooltip(badge) {
if (!badge) {
return "";
}
return `Requested plan ${toNumber(badge.requested_download_mbps, 0)} / ${toNumber(badge.requested_upload_mbps, 0)} Mbps exceeded detected Ethernet speed. ` +
`Shaping auto-capped to ${toNumber(badge.applied_download_mbps, 0)} / ${toNumber(badge.applied_upload_mbps, 0)} Mbps.`;
}
function ethernetBadgeClass(tierLabel) {
if (tierLabel === "10M") return "text-bg-danger";
if (tierLabel === "100M") return "text-bg-warning";
return "text-bg-info";
}
function configuredMax(node) {
return node.configured_max_throughput || node.max_throughput || [0, 0];
}
@@ -1627,7 +1657,21 @@ function renderAttachedCircuitsRows(rows) {
}
tr.appendChild(deviceCell);
tr.appendChild(simpleRow(`${toNumber(circuit.plan_mbps?.down, 0)} / ${toNumber(circuit.plan_mbps?.up, 0)}`));
const planCell = simpleRow(
`${toNumber(circuit.plan_mbps?.down, 0)} / ${toNumber(circuit.plan_mbps?.up, 0)}`
);
if (circuit.ethernet_cap_badge) {
const badge = document.createElement("a");
badge.className = `badge rounded-pill ms-2 text-decoration-none ${ethernetBadgeClass(circuit.ethernet_cap_badge.tier_label)}`;
badge.href = ethernetCapsPageHref(circuit.ethernet_cap_badge);
badge.setAttribute("aria-label", `Review ${circuit.ethernet_cap_badge.tier_label} Ethernet-limited circuits`);
badge.setAttribute("data-bs-toggle", "tooltip");
badge.setAttribute("data-bs-placement", "top");
badge.setAttribute("title", formatEthernetTooltip(circuit.ethernet_cap_badge));
badge.textContent = circuit.ethernet_cap_badge.tier_label;
planCell.appendChild(badge);
}
tr.appendChild(planCell);
tr.appendChild(simpleRow(circuit.parent_node, true));
const ipList = Array.isArray(circuit.ip_addrs) ? circuit.ip_addrs : [];
@@ -1666,6 +1710,7 @@ function renderAttachedCircuitsRows(rows) {
tableWrap.appendChild(table);
target.appendChild(sectionLabel);
target.appendChild(tableWrap);
enableTooltipsWithin(target);
}
function attachedCircuitsUpdate(msg) {
@@ -6,6 +6,7 @@ pub(crate) mod cpu_affinity;
pub(crate) mod dashboard_themes;
pub(crate) mod device_counts;
pub(crate) mod directories;
pub(crate) mod ethernet_caps;
pub(crate) mod executive;
pub(crate) mod executive_cache;
pub(crate) mod flow_explorer;
@@ -1,7 +1,29 @@
use crate::node_manager::local_api::ethernet_caps::ethernet_advisory_for_circuit;
use crate::shaped_devices_tracker::SHAPED_DEVICES;
use lqos_config::ShapedDevice;
use lqos_config::{CircuitEthernetMetadata, ShapedDevice};
use serde::{Deserialize, Serialize};
pub fn circuit_by_id_data(id: &str) -> Option<Vec<ShapedDevice>> {
/// Circuit-page payload containing shaped devices plus optional Ethernet advisory metadata.
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct CircuitByIdData {
/// Shaped-device rows for the requested circuit.
pub devices: Vec<ShapedDevice>,
/// Optional negotiated-Ethernet advisory derived from integration metadata.
pub ethernet_advisory: Option<CircuitEthernetMetadata>,
}
fn load_ethernet_advisory(
circuit_id: &str,
devices: &[ShapedDevice],
) -> Option<CircuitEthernetMetadata> {
let device_ids: std::collections::HashSet<&str> = devices
.iter()
.map(|device| device.device_id.as_str())
.collect();
ethernet_advisory_for_circuit(circuit_id, &device_ids)
}
pub fn circuit_by_id_data(id: &str) -> Option<CircuitByIdData> {
let safe_id = id.to_lowercase().trim().to_string();
let reader = SHAPED_DEVICES.load();
let devices: Vec<ShapedDevice> = reader
@@ -14,6 +36,10 @@ pub fn circuit_by_id_data(id: &str) -> Option<Vec<ShapedDevice>> {
if devices.is_empty() {
None
} else {
Some(devices)
let ethernet_advisory = load_ethernet_advisory(&safe_id, &devices);
Some(CircuitByIdData {
devices,
ethernet_advisory,
})
}
}
@@ -734,24 +734,15 @@ fn resolve_snapshot_cpu_sets(
};
if detection.has_hybrid_split {
let shaping = detection
.shaping
.into_iter()
.collect::<BTreeSet<_>>();
let excluded = possible
.difference(&shaping)
.copied()
.collect::<Vec<_>>();
let shaping = detection.shaping.into_iter().collect::<BTreeSet<_>>();
let excluded = possible.difference(&shaping).copied().collect::<Vec<_>>();
return (shaping.into_iter().collect(), excluded, true);
}
if detection.exclude_efficiency_cores {
let inferred = inferred_included_cpus(planned_core_metrics, placements);
if !inferred.is_empty() && inferred.len() < possible.len() {
let excluded = possible
.difference(&inferred)
.copied()
.collect::<Vec<_>>();
let excluded = possible.difference(&inferred).copied().collect::<Vec<_>>();
return (inferred.into_iter().collect(), excluded, false);
}
}
@@ -1340,8 +1331,8 @@ pub fn cpu_affinity_preview_weights_data(q: CircuitsQuery) -> Vec<PreviewWeightI
#[cfg(test)]
mod tests {
use super::{
PlannedCoreMetrics, RuntimeNodePlacement, derive_runtime_node_placements,
is_cpu_root_node, resolve_snapshot_cpu_sets,
PlannedCoreMetrics, RuntimeNodePlacement, derive_runtime_node_placements, is_cpu_root_node,
resolve_snapshot_cpu_sets,
};
use crate::node_manager::local_api::network_tree_lite::NetworkTreeLiteNode;
use lqos_config::{ShapingCpuDetection, ShapingCpuSource};
@@ -0,0 +1,294 @@
use crate::shaped_devices_tracker::SHAPED_DEVICES;
use lqos_config::{
CIRCUIT_ETHERNET_METADATA_FILENAME, CircuitEthernetMetadata, CircuitEthernetMetadataFile,
load_config,
};
use serde::{Deserialize, Deserializer, Serialize};
use std::collections::HashMap;
use std::path::Path;
const DEFAULT_ETHERNET_CAPS_PAGE_SIZE: usize = 100;
const MAX_ETHERNET_CAPS_PAGE_SIZE: usize = 250;
fn deserialize_optional_string<'de, D>(deserializer: D) -> Result<Option<String>, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum MaybeString {
String(String),
Null,
}
Ok(match Option::<MaybeString>::deserialize(deserializer)? {
Some(MaybeString::String(value)) => Some(value),
Some(MaybeString::Null) | None => None,
})
}
/// Normalized Ethernet-cap tier used by UI badges and filters.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
pub enum EthernetCapTier {
TenM,
HundredM,
GigPlus,
}
impl EthernetCapTier {
/// Returns the short operator-facing label for this Ethernet tier.
pub const fn label(&self) -> &'static str {
match self {
Self::TenM => "10M",
Self::HundredM => "100M",
Self::GigPlus => "1G",
}
}
}
/// Compact Ethernet-cap badge payload for circuit tables and detail views.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct EthernetCapBadge {
/// Normalized tier used for styling and filtering.
pub tier: EthernetCapTier,
/// Short display label such as `10M`, `100M`, or `1G`.
pub tier_label: String,
/// Negotiated Ethernet speed in Mbps reported by the limiting interface.
pub negotiated_ethernet_mbps: u64,
/// Requested download max before the cap was applied.
pub requested_download_mbps: f32,
/// Requested upload max before the cap was applied.
pub requested_upload_mbps: f32,
/// Applied download max after the cap.
pub applied_download_mbps: f32,
/// Applied upload max after the cap.
pub applied_upload_mbps: f32,
}
/// Server-side paging/filter query for the Ethernet caps review page.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct EthernetCapsPageQuery {
pub page: Option<usize>,
pub page_size: Option<usize>,
#[serde(default, deserialize_with = "deserialize_optional_string")]
pub search: Option<String>,
pub tier: Option<EthernetCapTier>,
}
/// One Ethernet-limited circuit row for the review page.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct EthernetCapsPageRow {
/// Stable circuit identifier used for deep-linking to the circuit page.
pub circuit_id: String,
/// Human-facing circuit name.
pub circuit_name: String,
/// Parent node from shaped devices when available.
pub parent_node: String,
/// Compact warning badge metadata.
pub badge: EthernetCapBadge,
/// Device name that supplied the limiting Ethernet speed when known.
pub limiting_device_name: Option<String>,
/// Interface name that supplied the limiting Ethernet speed when known.
pub limiting_interface_name: Option<String>,
}
/// A server-paged slice of Ethernet-capped circuits.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct EthernetCapsPage {
/// The normalized query that produced this page.
pub query: EthernetCapsPageQuery,
/// Total rows matching the query before paging.
pub total_rows: usize,
/// Page rows after filtering and paging.
pub rows: Vec<EthernetCapsPageRow>,
}
fn normalized_page_size(query: &EthernetCapsPageQuery) -> usize {
query
.page_size
.unwrap_or(DEFAULT_ETHERNET_CAPS_PAGE_SIZE)
.clamp(1, MAX_ETHERNET_CAPS_PAGE_SIZE)
}
fn tier_for_speed(negotiated_ethernet_mbps: u64) -> EthernetCapTier {
match negotiated_ethernet_mbps {
0..=10 => EthernetCapTier::TenM,
11..=100 => EthernetCapTier::HundredM,
_ => EthernetCapTier::GigPlus,
}
}
fn advisory_to_badge(advisory: &CircuitEthernetMetadata) -> Option<EthernetCapBadge> {
if !advisory.auto_capped {
return None;
}
let tier = tier_for_speed(advisory.negotiated_ethernet_mbps);
Some(EthernetCapBadge {
tier: tier.clone(),
tier_label: tier.label().to_string(),
negotiated_ethernet_mbps: advisory.negotiated_ethernet_mbps,
requested_download_mbps: advisory.requested_download_mbps,
requested_upload_mbps: advisory.requested_upload_mbps,
applied_download_mbps: advisory.applied_download_mbps,
applied_upload_mbps: advisory.applied_upload_mbps,
})
}
fn load_advisory_file() -> Option<CircuitEthernetMetadataFile> {
let cfg = load_config().ok()?;
let path = Path::new(&cfg.lqos_directory).join(CIRCUIT_ETHERNET_METADATA_FILENAME);
let payload = std::fs::read(path).ok()?;
serde_json::from_slice(&payload).ok()
}
fn parent_node_by_circuit_id() -> HashMap<String, String> {
let devices = SHAPED_DEVICES.load();
let mut parent_nodes = HashMap::new();
for device in &devices.devices {
let circuit_id = device.circuit_id.trim();
if circuit_id.is_empty() {
continue;
}
parent_nodes
.entry(circuit_id.to_string())
.or_insert_with(|| device.parent_node.trim().to_string());
}
parent_nodes
}
fn sort_rank(tier: &EthernetCapTier) -> u8 {
match tier {
EthernetCapTier::TenM => 0,
EthernetCapTier::HundredM => 1,
EthernetCapTier::GigPlus => 2,
}
}
/// Returns a compact badge map keyed by circuit ID for Ethernet auto-capped circuits.
pub(crate) fn ethernet_cap_badge_map() -> HashMap<String, EthernetCapBadge> {
let mut badges = HashMap::new();
let Some(file) = load_advisory_file() else {
return badges;
};
for advisory in file.circuits {
let Some(badge) = advisory_to_badge(&advisory) else {
continue;
};
badges.insert(advisory.circuit_id.to_ascii_lowercase(), badge);
}
badges
}
/// Finds one Ethernet advisory for the requested circuit and matching shaped-device IDs.
pub(crate) fn ethernet_advisory_for_circuit(
circuit_id: &str,
device_ids: &std::collections::HashSet<&str>,
) -> Option<CircuitEthernetMetadata> {
let file = load_advisory_file()?;
file.circuits.into_iter().find(|entry| {
entry.auto_capped
&& entry.circuit_id.eq_ignore_ascii_case(circuit_id)
&& entry
.device_ids
.iter()
.any(|device_id| device_ids.contains(device_id.as_str()))
})
}
/// Returns one filtered, sorted page of Ethernet-limited circuits for operator review.
pub fn ethernet_caps_page(query: EthernetCapsPageQuery) -> EthernetCapsPage {
let page = query.page.unwrap_or(0);
let page_size = normalized_page_size(&query);
let search = query.search.as_deref().unwrap_or("").trim().to_lowercase();
let parent_nodes = parent_node_by_circuit_id();
let mut rows = Vec::new();
if let Some(file) = load_advisory_file() {
for advisory in file.circuits {
let Some(badge) = advisory_to_badge(&advisory) else {
continue;
};
if let Some(filter_tier) = query.tier.as_ref()
&& &badge.tier != filter_tier
{
continue;
}
let parent_node = parent_nodes
.get(&advisory.circuit_id)
.cloned()
.unwrap_or_default();
let row = EthernetCapsPageRow {
circuit_id: advisory.circuit_id,
circuit_name: advisory.circuit_name,
parent_node,
badge,
limiting_device_name: advisory.limiting_device_name,
limiting_interface_name: advisory.limiting_interface_name,
};
if !search.is_empty() {
let limiting_device = row.limiting_device_name.as_deref().unwrap_or("");
let limiting_interface = row.limiting_interface_name.as_deref().unwrap_or("");
if !row.circuit_id.to_lowercase().contains(&search)
&& !row.circuit_name.to_lowercase().contains(&search)
&& !row.parent_node.to_lowercase().contains(&search)
&& !row.badge.tier_label.to_lowercase().contains(&search)
&& !limiting_device.to_lowercase().contains(&search)
&& !limiting_interface.to_lowercase().contains(&search)
{
continue;
}
}
rows.push(row);
}
}
rows.sort_by(|left, right| {
sort_rank(&left.badge.tier)
.cmp(&sort_rank(&right.badge.tier))
.then_with(|| left.circuit_name.cmp(&right.circuit_name))
.then_with(|| left.circuit_id.cmp(&right.circuit_id))
});
let total_rows = rows.len();
let start = page.saturating_mul(page_size);
let rows = if start >= total_rows {
Vec::new()
} else {
let end = (start + page_size).min(total_rows);
rows[start..end].to_vec()
};
EthernetCapsPage {
query: EthernetCapsPageQuery {
page: Some(page),
page_size: Some(page_size),
search: if search.is_empty() {
None
} else {
query.search
},
tier: query.tier,
},
total_rows,
rows,
}
}
#[cfg(test)]
mod tests {
use super::{EthernetCapTier, sort_rank, tier_for_speed};
#[test]
fn ethernet_cap_tier_classifies_expected_speeds() {
assert_eq!(tier_for_speed(10), EthernetCapTier::TenM);
assert_eq!(tier_for_speed(100), EthernetCapTier::HundredM);
assert_eq!(tier_for_speed(1000), EthernetCapTier::GigPlus);
}
#[test]
fn ethernet_cap_sort_rank_prioritizes_low_speed_tiers() {
assert!(sort_rank(&EthernetCapTier::TenM) < sort_rank(&EthernetCapTier::HundredM));
assert!(sort_rank(&EthernetCapTier::HundredM) < sort_rank(&EthernetCapTier::GigPlus));
}
}
@@ -1,3 +1,4 @@
use crate::node_manager::local_api::ethernet_caps::{EthernetCapBadge, ethernet_cap_badge_map};
use crate::shaped_devices_tracker::NETWORK_JSON;
use crate::shaped_devices_tracker::circuit_live::fresh_circuit_live_snapshot;
use lqos_utils::units::{DownUpOrder, TcpRetransmitSample};
@@ -51,6 +52,8 @@ pub struct TreeAttachedCircuitsQuery {
pub struct TreeAttachedCircuitRow {
pub circuit_id: String,
pub circuit_name: String,
/// Optional Ethernet speed-cap advisory badge when this circuit was auto-capped.
pub ethernet_cap_badge: Option<EthernetCapBadge>,
pub parent_node: String,
pub device_names: Vec<String>,
pub ip_addrs: Vec<String>,
@@ -133,6 +136,7 @@ pub fn tree_attached_circuits(query: TreeAttachedCircuitsQuery) -> TreeAttachedC
let node_name = resolve_node_name(&query);
let snapshot = fresh_circuit_live_snapshot();
let ethernet_badges = ethernet_cap_badge_map();
let mut rows: Vec<TreeAttachedCircuitRow> = match node_name.as_deref() {
Some(node_name) => snapshot
.circuit_ids_by_parent_node
@@ -143,6 +147,9 @@ pub fn tree_attached_circuits(query: TreeAttachedCircuitsQuery) -> TreeAttachedC
.map(|row| TreeAttachedCircuitRow {
circuit_id: row.circuit_id.clone(),
circuit_name: row.circuit_name.clone(),
ethernet_cap_badge: ethernet_badges
.get(&row.circuit_id.to_ascii_lowercase())
.cloned(),
parent_node: row.parent_node.clone(),
device_names: row.device_names.clone(),
ip_addrs: row.ip_addrs.clone(),
@@ -3,21 +3,34 @@
<div class="lqos-circuit-panel">
<h5><i class="fa fa-user"></i> <span id="circuitName" class="redactable lqos-ellipsis-inline"></span></h5>
<div class="lqos-table-wrap">
<table class="lqos-table lqos-table-compact">
<table class="lqos-table lqos-table-compact lqos-circuit-header-table">
<tr>
<td class="table-label-cell">Parent Node</td>
<td class="table-value-cell"><a id="parentNode" class="redactable" href="#"></a></td>
<td class="table-label-cell">Parent</td>
<td class="table-value-cell"><a id="parentNode" class="redactable lqos-circuit-parent-link lqos-ellipsis-inline" href="#"></a></td>
</tr>
<tr>
<td class="table-label-cell">Maximum Speed</td>
<td class="table-value-cell"><span id="bwMax"></span></td>
<td class="table-label-cell">Max</td>
<td class="table-value-cell">
<span id="bwMax"></span>
<a
id="ethernetAdvisoryBadge"
class="badge rounded-pill text-bg-warning ms-2 d-none"
href="#"
tabindex="0"
aria-label="Ethernet speed warning"
data-bs-toggle="tooltip"
data-bs-placement="top"
>
<i class="fa fa-ethernet me-1"></i><span id="ethernetAdvisoryBadgeText"></span>
</a>
</td>
</tr>
<tr>
<td class="table-label-cell">Minimum Speed</td>
<td class="table-label-cell">Min</td>
<td class="table-value-cell"><span id="bwMin"></span></td>
</tr>
<tr>
<td class="table-label-cell">RTT Metrics</td>
<td class="table-label-cell">RTT</td>
<td class="table-value-cell">
<div class="form-check form-switch mb-0">
<input class="form-check-input" type="checkbox" id="excludeRttToggle">
@@ -0,0 +1,58 @@
<div class="row g-4">
<div class="col-12">
<div class="card shadow-sm">
<div class="card-body">
<div class="d-flex flex-wrap align-items-center justify-content-between gap-3">
<div>
<h5 class="card-title mb-1"><i class="fa fa-ethernet me-2"></i>Ethernet Caps</h5>
<p class="text-muted mb-0">Circuits automatically down-rated because detected Ethernet speed was below the requested plan.</p>
</div>
<div id="ethernetCapsSummary" class="small text-body-secondary"></div>
</div>
</div>
</div>
</div>
<div class="col-12">
<div class="card shadow-sm">
<div class="card-body">
<div class="row g-2 align-items-end mb-3">
<div class="col-12 col-lg-5">
<label class="form-label small text-body-secondary" for="ethernetCapsSearch">Search</label>
<input id="ethernetCapsSearch" class="form-control" type="text" placeholder="Search circuits, parents, device names…">
</div>
<div class="col-6 col-lg-2">
<label class="form-label small text-body-secondary" for="ethernetCapsTier">Ethernet</label>
<select id="ethernetCapsTier" class="form-select">
<option value="">All</option>
<option value="10M">10M</option>
<option value="100M">100M</option>
<option value="1G">1G+</option>
</select>
</div>
<div class="col-6 col-lg-2">
<label class="form-label small text-body-secondary" for="ethernetCapsPageSize">Per page</label>
<select id="ethernetCapsPageSize" class="form-select">
<option value="50">50</option>
<option value="100" selected>100</option>
<option value="250">250</option>
</select>
</div>
<div class="col-12 col-lg-3 d-flex justify-content-lg-end gap-2">
<button id="ethernetCapsPrev" type="button" class="btn btn-outline-secondary">
<i class="fa fa-arrow-left"></i>
</button>
<button id="ethernetCapsNext" type="button" class="btn btn-outline-secondary">
<i class="fa fa-arrow-right"></i>
</button>
</div>
</div>
<div id="ethernetCapsTableWrap">
<div class="text-muted small"><i class="fa fa-spinner fa-spin me-1"></i>Loading Ethernet cap data…</div>
</div>
<div id="ethernetCapsPager" class="small text-body-secondary mt-3"></div>
</div>
</div>
</div>
</div>
<script src="ethernet_caps.js%CACHEBUSTERS%"></script>
@@ -405,6 +405,22 @@ html[data-bs-theme="light"] .sidebar .navbar-toggler-icon {
.lqos-circuit-panel .lqos-table {
height: 100%;
}
.lqos-circuit-header-table.lqos-table-compact tbody td {
padding: 0.5rem 0.55rem;
}
.lqos-table.lqos-circuit-header-table .table-label-cell {
width: 6.75rem;
white-space: nowrap;
padding-right: 0.25rem;
}
.lqos-table.lqos-circuit-header-table .table-value-cell {
width: auto;
padding-left: 0.25rem;
}
.lqos-circuit-parent-link {
display: block;
max-width: 100%;
}
.lqos-circuit-gauge-panel {
display: flex;
align-items: center;
@@ -36,6 +36,7 @@ pub(super) fn static_routes() -> Result<Router> {
"unknown_ips.html",
"configuration.html",
"circuit.html",
"ethernet_caps.html",
"site_map.html",
"flow_map.html",
"all_tree_sankey.html",
+15 -7
View File
@@ -11,9 +11,9 @@ use std::sync::Arc;
use crate::node_manager::auth::{LoginResult, login_from_token};
use crate::node_manager::local_api::{
circuit, circuit_count, config, cpu_affinity, dashboard_themes, device_counts, directories,
executive, flow_explorer, flow_map, lts, network_tree, network_tree_lite, node_rate_overrides,
packet_analysis, reload_libreqos, scheduler, search, shaped_device_api, shaped_devices_page,
unknown_ips, urgent, warnings,
ethernet_caps, executive, flow_explorer, flow_map, lts, network_tree, network_tree_lite,
node_rate_overrides, packet_analysis, reload_libreqos, scheduler, search, shaped_device_api,
shaped_devices_page, unknown_ips, urgent, warnings,
};
use crate::node_manager::shaper_queries_actor::ShaperQueryCommand;
use crate::node_manager::ws::messages::{
@@ -380,6 +380,14 @@ async fn receive_channel_message(
return true;
}
}
WsRequest::EthernetCapsPage { query } => {
let response = WsResponse::EthernetCapsPage {
data: ethernet_caps::ethernet_caps_page(query),
};
if send_ws_response(&tx, response).await {
return true;
}
}
WsRequest::ExecutiveHeatmapPage { query } => {
let response = WsResponse::ExecutiveHeatmapPage {
data: executive::executive_heatmap_page(query),
@@ -421,11 +429,11 @@ async fn receive_channel_message(
}
}
WsRequest::CircuitById { id } => {
let (ok, devices) = match circuit::circuit_by_id_data(&id) {
Some(devices) => (true, devices),
None => (false, Vec::new()),
let (ok, data) = match circuit::circuit_by_id_data(&id) {
Some(data) => (true, Some(data)),
None => (false, None),
};
let response = WsResponse::CircuitByIdResult { id, devices, ok };
let response = WsResponse::CircuitByIdResult { id, data, ok };
if send_ws_response(&tx, response).await {
return true;
}
@@ -1,10 +1,12 @@
use crate::lts2_sys::control_channel::{SupportTicket, SupportTicketSummary};
use crate::node_manager::WarningLevel;
use crate::node_manager::local_api::circuit::CircuitByIdData;
use crate::node_manager::local_api::dashboard_themes::{DashletIdentity, ThemeEntry};
use crate::node_manager::local_api::device_counts::DeviceCount;
use crate::node_manager::local_api::directories::{
CircuitDirectoryPage, CircuitDirectoryQuery, NodeDirectoryEntry, TreeGuardMetadataSummary,
};
use crate::node_manager::local_api::ethernet_caps::{EthernetCapsPage, EthernetCapsPageQuery};
use crate::node_manager::local_api::executive::{
ExecutiveDashboardSummary, ExecutiveHeatmapPage, ExecutiveHeatmapPageQuery,
ExecutiveLeaderboardPage, ExecutiveLeaderboardPageQuery,
@@ -202,6 +204,9 @@ pub enum WsRequest {
CircuitDirectoryPage {
query: CircuitDirectoryQuery,
},
EthernetCapsPage {
query: EthernetCapsPageQuery,
},
NodeDirectory,
TreeGuardMetadataSummary,
GetUsers,
@@ -636,6 +641,9 @@ pub enum WsResponse {
CircuitDirectoryPage {
data: CircuitDirectoryPage,
},
EthernetCapsPage {
data: EthernetCapsPage,
},
NodeDirectory {
data: Vec<NodeDirectoryEntry>,
},
@@ -746,7 +754,7 @@ pub enum WsResponse {
},
CircuitByIdResult {
id: String,
devices: Vec<ShapedDevice>,
data: Option<CircuitByIdData>,
ok: bool,
},
RequestAnalysisResult {
@@ -9,7 +9,7 @@ use crate::shaped_devices_tracker::NETWORK_JSON;
use lqos_bakery::{
BakeryActivityEntry as BakeryActivitySnapshot, BakeryApplyType, BakeryMode,
BakeryPreflightSnapshot, BakeryRuntimeNodeOperationAction, BakeryRuntimeNodeOperationStatus,
BakeryStatusSnapshot,
BakeryStatusSnapshot, bakery_runtime_node_branch_snapshot,
};
use lqos_utils::hash_to_i64;
use std::sync::Arc;
@@ -86,6 +86,10 @@ fn resolve_site_name(site_hash: i64) -> Option<String> {
.find_map(|node| (hash_to_i64(&node.name) == site_hash).then(|| node.name.clone()))
}
fn resolve_runtime_site_name(site_hash: i64) -> Option<String> {
bakery_runtime_node_branch_snapshot(site_hash).map(|snapshot| snapshot.site_name)
}
fn extract_site_hash_from_summary(summary: &str) -> Option<i64> {
let lower = summary.to_ascii_lowercase();
let site_index = lower.find("site ")?;
@@ -134,6 +138,7 @@ fn map_status(snapshot: BakeryStatusSnapshot) -> BakeryStatusData {
site_hash: entry.site_hash,
site_name: entry
.site_name
.or_else(|| resolve_runtime_site_name(entry.site_hash))
.or_else(|| resolve_site_name(entry.site_hash)),
action: runtime_action_to_string(entry.action),
status: runtime_status_to_string(entry.status),
@@ -180,6 +185,7 @@ fn map_activity(entry: BakeryActivitySnapshot) -> BakeryActivityEntry {
.or_else(|| extract_site_hash_from_summary(&entry.summary));
let site_name = entry
.site_name
.or_else(|| site_hash.and_then(resolve_runtime_site_name))
.or_else(|| site_hash.and_then(resolve_site_name));
BakeryActivityEntry {
ts: entry.ts,
+14 -14
View File
@@ -1850,12 +1850,12 @@ fn run_tick(
cpu_max_pct,
dry_run: tg.dry_run,
circuit_id,
circuit_entity_id: &entry.circuit_entity_id,
circuit_label: &entry.circuit_label,
devices: &entry.devices,
sqm_batch_id: &sqm_batch_id,
allowlisted: tg.circuits.all_circuits
|| allowlisted_circuits.contains(circuit_id),
circuit_entity_id: &entry.circuit_entity_id,
circuit_label: &entry.circuit_label,
devices: &entry.devices,
sqm_batch_id: &sqm_batch_id,
allowlisted: tg.circuits.all_circuits
|| allowlisted_circuits.contains(circuit_id),
cap_down: entry.cap_down,
cap_up: entry.cap_up,
bps: live_rollup
@@ -3164,14 +3164,14 @@ where
activity,
now_unix,
dry_run,
persist_sqm_overrides: circuits_cfg.persist_sqm_overrides,
circuit_id,
circuit_entity_id,
circuit_label,
devices,
base_sqm,
batch_id: sqm_batch_id,
},
persist_sqm_overrides: circuits_cfg.persist_sqm_overrides,
circuit_id,
circuit_entity_id,
circuit_label,
devices,
base_sqm,
batch_id: sqm_batch_id,
},
state,
transition,
persist_override,
+2
View File
@@ -185,6 +185,7 @@ pub struct DeviceInterface {
#[allow(non_snake_case)]
#[derive(Serialize, Deserialize, Debug, Allocative)]
pub struct InterfaceIdentification {
pub r#type: Option<String>,
pub name: Option<String>,
pub mac: Option<String>,
}
@@ -200,6 +201,7 @@ pub struct DeviceAddress {
pub struct InterfaceStatus {
pub status: Option<String>,
pub speed: Option<String>,
pub currentSpeed: Option<String>,
}
#[allow(non_snake_case)]
@@ -0,0 +1,405 @@
use crate::errors::UispIntegrationError;
use crate::uisp_types::UispDevice;
use lqos_config::{CIRCUIT_ETHERNET_METADATA_FILENAME, CircuitEthernetMetadata, Config};
use std::path::Path;
use tracing::error;
const ETH_PORT_HEADROOM_FACTOR_SUB_GIG: f32 = 0.95;
/// Requested and applied circuit rates after negotiated-Ethernet evaluation.
#[derive(Clone, Debug, PartialEq)]
pub struct EthernetRateDecision {
/// Minimum download after any cap was applied.
pub download_min: f32,
/// Minimum upload after any cap was applied.
pub upload_min: f32,
/// Maximum download after any cap was applied.
pub download_max: f32,
/// Maximum upload after any cap was applied.
pub upload_max: f32,
/// Optional UI advisory for the affected circuit.
pub advisory: Option<CircuitEthernetMetadata>,
}
/// Applies a negotiated-Ethernet cap to a circuit and returns the adjusted rates plus advisory.
pub fn apply_ethernet_rate_cap<'a>(
circuit_id: &str,
circuit_name: &str,
devices: impl IntoIterator<Item = &'a UispDevice>,
download_min: f32,
upload_min: f32,
download_max: f32,
upload_max: f32,
) -> EthernetRateDecision {
let devices: Vec<&UispDevice> = devices.into_iter().collect();
let related_device_ids = devices.iter().map(|device| device.id.clone()).collect();
let candidate_devices: Vec<&UispDevice> = {
let station_devices: Vec<&UispDevice> = devices
.iter()
.copied()
.filter(|device| device.is_wireless_station_cpe())
.collect();
if !station_devices.is_empty() {
station_devices
} else {
devices
.iter()
.copied()
.filter(|device| !device.is_router_like())
.collect()
}
};
let mut limiting_device: Option<&UispDevice> = None;
for device in &candidate_devices {
let Some(speed_mbps) = device.negotiated_ethernet_mbps else {
continue;
};
if limiting_device
.as_ref()
.is_none_or(|current| speed_mbps < current.negotiated_ethernet_mbps.unwrap_or(u64::MAX))
{
limiting_device = Some(device);
}
}
let Some(limiting_device) = limiting_device else {
return EthernetRateDecision {
download_min,
upload_min,
download_max,
upload_max,
advisory: None,
};
};
let usable_cap = ethernet_usable_cap_mbps(
limiting_device
.negotiated_ethernet_mbps
.expect("limiting device must have ethernet speed"),
);
let applied_download_max = download_max.min(usable_cap);
let applied_upload_max = upload_max.min(usable_cap);
let applied_download_min = download_min.min(applied_download_max);
let applied_upload_min = upload_min.min(applied_upload_max);
let auto_capped = applied_download_max < download_max || applied_upload_max < upload_max;
EthernetRateDecision {
download_min: applied_download_min,
upload_min: applied_upload_min,
download_max: applied_download_max,
upload_max: applied_upload_max,
advisory: Some(CircuitEthernetMetadata {
circuit_id: circuit_id.to_string(),
circuit_name: circuit_name.to_string(),
device_ids: related_device_ids,
source: "uisp".to_string(),
negotiated_ethernet_mbps: limiting_device.negotiated_ethernet_mbps.unwrap_or_default(),
requested_download_mbps: download_max,
requested_upload_mbps: upload_max,
applied_download_mbps: applied_download_max,
applied_upload_mbps: applied_upload_max,
auto_capped,
limiting_device_id: Some(limiting_device.id.clone()),
limiting_device_name: Some(limiting_device.name.clone()),
limiting_interface_name: limiting_device.negotiated_ethernet_interface.clone(),
}),
}
}
/// Persists circuit Ethernet advisories alongside other generated runtime files.
pub fn write_ethernet_advisories(
config: &Config,
advisories: &[CircuitEthernetMetadata],
) -> Result<(), UispIntegrationError> {
let path = Path::new(&config.lqos_directory).join(CIRCUIT_ETHERNET_METADATA_FILENAME);
let payload = serde_json::to_vec_pretty(&lqos_config::CircuitEthernetMetadataFile {
circuits: advisories
.iter()
.filter(|advisory| advisory.auto_capped)
.cloned()
.collect(),
})
.map_err(|e| {
error!("Unable to serialize circuit Ethernet metadata: {e:?}");
UispIntegrationError::CsvError
})?;
std::fs::write(path, payload).map_err(|e| {
error!("Unable to write circuit Ethernet metadata: {e:?}");
UispIntegrationError::CsvError
})?;
Ok(())
}
fn ethernet_usable_cap_mbps(negotiated_mbps: u64) -> f32 {
match negotiated_mbps {
0..=999 => negotiated_mbps as f32 * ETH_PORT_HEADROOM_FACTOR_SUB_GIG,
1000..=2499 => negotiated_mbps.saturating_sub(5) as f32,
2500..=9999 => negotiated_mbps.saturating_sub(10) as f32,
_ => negotiated_mbps.saturating_sub(50) as f32,
}
}
#[cfg(test)]
mod tests {
use super::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::uisp_types::UispDevice;
use lqos_config::{CIRCUIT_ETHERNET_METADATA_FILENAME, CircuitEthernetMetadataFile, Config};
use std::collections::HashSet;
use std::fs;
use std::time::{SystemTime, UNIX_EPOCH};
fn sample_device(
id: &str,
name: &str,
speed: Option<u64>,
iface: Option<&str>,
role: Option<&str>,
wireless_mode: Option<&str>,
) -> UispDevice {
UispDevice {
id: id.to_string(),
name: name.to_string(),
mac: String::new(),
role: role.map(str::to_string),
wireless_mode: wireless_mode.map(str::to_string),
site_id: "site-1".to_string(),
download: 1000,
upload: 1000,
ipv4: HashSet::new(),
ipv6: HashSet::new(),
negotiated_ethernet_mbps: speed,
negotiated_ethernet_interface: iface.map(str::to_string),
}
}
#[test]
fn ethernet_cap_uses_lowest_detected_port_speed() {
let fast = sample_device(
"dev-fast",
"Fast",
Some(1000),
Some("eth0"),
Some("station"),
Some("sta-ptmp"),
);
let slow = sample_device(
"dev-slow",
"Slow",
Some(100),
Some("eth1"),
Some("station"),
Some("sta-ptmp"),
);
let decision = apply_ethernet_rate_cap(
"circuit-1",
"Circuit 1",
[&fast, &slow],
25.0,
25.0,
300.0,
300.0,
);
assert_eq!(decision.download_max, 95.0);
assert_eq!(decision.upload_max, 95.0);
assert_eq!(decision.download_min, 25.0);
assert_eq!(decision.upload_min, 25.0);
let advisory = decision.advisory.expect("advisory should exist");
assert!(advisory.auto_capped);
assert_eq!(advisory.negotiated_ethernet_mbps, 100);
assert_eq!(advisory.limiting_device_id.as_deref(), Some("dev-slow"));
}
#[test]
fn ethernet_cap_preserves_lower_upload_plan() {
let slow = sample_device(
"dev-slow",
"Slow",
Some(100),
Some("eth1"),
Some("station"),
Some("sta-ptmp"),
);
let decision =
apply_ethernet_rate_cap("circuit-1", "Circuit 1", [&slow], 10.0, 10.0, 300.0, 50.0);
assert_eq!(decision.download_max, 95.0);
assert_eq!(decision.upload_max, 50.0);
let advisory = decision.advisory.expect("advisory should exist");
assert!(advisory.auto_capped);
assert_eq!(advisory.applied_download_mbps, 95.0);
assert_eq!(advisory.applied_upload_mbps, 50.0);
}
#[test]
fn gigabit_ports_use_small_fixed_margin() {
let gig = sample_device(
"dev-gig",
"Gig",
Some(1000),
Some("eth0"),
Some("station"),
Some("sta-ptmp"),
);
let decision =
apply_ethernet_rate_cap("circuit-1", "Circuit 1", [&gig], 100.0, 100.0, 995.0, 995.0);
assert_eq!(decision.download_max, 995.0);
assert_eq!(decision.upload_max, 995.0);
let advisory = decision.advisory.expect("advisory should exist");
assert!(!advisory.auto_capped);
assert_eq!(advisory.applied_download_mbps, 995.0);
assert_eq!(advisory.applied_upload_mbps, 995.0);
}
#[test]
fn gigabit_ports_cap_above_line_rate_to_995() {
let gig = sample_device(
"dev-gig",
"Gig",
Some(1000),
Some("eth0"),
Some("station"),
Some("sta-ptmp"),
);
let decision = apply_ethernet_rate_cap(
"circuit-1",
"Circuit 1",
[&gig],
100.0,
100.0,
5000.0,
5000.0,
);
assert_eq!(decision.download_max, 995.0);
assert_eq!(decision.upload_max, 995.0);
let advisory = decision.advisory.expect("advisory should exist");
assert!(advisory.auto_capped);
assert_eq!(advisory.applied_download_mbps, 995.0);
assert_eq!(advisory.applied_upload_mbps, 995.0);
}
#[test]
fn station_devices_win_over_router_devices() {
let station = sample_device(
"dev-station",
"Station",
Some(1000),
Some("data"),
Some("station"),
Some("sta-ptmp"),
);
let router = sample_device(
"dev-router",
"Router",
Some(10),
Some("eth3"),
Some("homeWiFi"),
Some("ap"),
);
let decision = apply_ethernet_rate_cap(
"circuit-1",
"Circuit 1",
[&station, &router],
10.0,
10.0,
115.0,
22.0,
);
assert_eq!(decision.download_max, 115.0);
assert_eq!(decision.upload_max, 22.0);
let advisory = decision.advisory.expect("advisory should exist");
assert!(!advisory.auto_capped);
assert_eq!(advisory.negotiated_ethernet_mbps, 1000);
assert_eq!(advisory.limiting_device_id.as_deref(), Some("dev-station"));
}
#[test]
fn router_only_devices_do_not_drive_ethernet_cap() {
let router = sample_device(
"dev-router",
"Router",
Some(100),
Some("eth0"),
Some("router"),
None,
);
let decision =
apply_ethernet_rate_cap("circuit-1", "Circuit 1", [&router], 10.0, 10.0, 115.0, 22.0);
assert_eq!(decision.download_max, 115.0);
assert_eq!(decision.upload_max, 22.0);
assert!(decision.advisory.is_none());
}
#[test]
fn writer_persists_only_auto_capped_advisories() {
let auto_capped = sample_device(
"dev-100m",
"Hundred",
Some(100),
Some("eth0"),
Some("station"),
Some("sta-ptmp"),
);
let observed_only = sample_device(
"dev-1g",
"Gig",
Some(1000),
Some("eth0"),
Some("station"),
Some("sta-ptmp"),
);
let capped = apply_ethernet_rate_cap(
"circuit-capped",
"Capped Circuit",
[&auto_capped],
10.0,
10.0,
115.0,
22.0,
)
.advisory
.expect("auto-capped advisory should exist");
let observed = apply_ethernet_rate_cap(
"circuit-observed",
"Observed Circuit",
[&observed_only],
100.0,
100.0,
995.0,
995.0,
)
.advisory
.expect("observed advisory should exist");
let mut config = Config::default();
let unique = SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("time should move forward")
.as_nanos();
let temp_dir = std::env::temp_dir().join(format!("lqos-ethernet-writer-{unique}"));
fs::create_dir_all(&temp_dir).expect("temp dir should be created");
config.lqos_directory = temp_dir.to_string_lossy().to_string();
write_ethernet_advisories(&config, &[capped.clone(), observed])
.expect("writer should succeed");
let payload = fs::read(temp_dir.join(CIRCUIT_ETHERNET_METADATA_FILENAME))
.expect("metadata file should exist");
let parsed: CircuitEthernetMetadataFile =
serde_json::from_slice(&payload).expect("metadata file should parse");
assert_eq!(parsed.circuits, vec![capped]);
let _ = fs::remove_dir_all(temp_dir);
}
}
+1
View File
@@ -5,6 +5,7 @@
mod blackboard;
#[warn(missing_docs)]
mod errors;
mod ethernet_advisory;
pub mod ip_ranges;
mod strategies;
pub mod uisp_types;
@@ -1,8 +1,9 @@
use crate::errors::UispIntegrationError;
use crate::ethernet_advisory::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::ip_ranges::IpRanges;
use crate::strategies::common::UispData;
use crate::strategies::full::shaped_devices_writer::ShapedDevice;
use lqos_config::Config;
use lqos_config::{CircuitEthernetMetadata, Config};
use std::collections::HashSet;
use std::fs::write;
use std::path::Path;
@@ -68,6 +69,7 @@ pub async fn build_ap_only_network(
// Write ShapedDevices.csv
let file_path = Path::new(&config.lqos_directory).join("ShapedDevices.csv");
let mut shaped_devices = Vec::new();
let mut ethernet_advisories: Vec<CircuitEthernetMetadata> = Vec::new();
let mut seen_pairs = HashSet::new();
for (parent, client_ids) in mappings.iter() {
for client_id in client_ids {
@@ -77,50 +79,53 @@ pub async fn build_ap_only_network(
.iter()
.filter(|d| d.site_id == *client_id)
.collect::<Vec<_>>();
let requested = if let Some((dl_min, dl_max, ul_min, ul_max)) =
site.burst_rates(&config)
{
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended && config.uisp_integration.suspended_strategy == "slow" {
(0.1, 0.1, 0.1, 0.1)
} else {
(
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_up_mbps as f32 * config.uisp_integration.bandwidth_overhead_factor,
),
)
};
let ethernet_decision = apply_ethernet_rate_cap(
&site.id,
&site.name,
devices.iter().copied(),
requested.0,
requested.2,
requested.1,
requested.3,
);
if let Some(advisory) = ethernet_decision.advisory.clone() {
ethernet_advisories.push(advisory);
}
for device in devices.iter().filter(|d| d.has_address()) {
// Compute subscriber rates: prefer UISP QoS + burst
let (download_min, mut download_max, upload_min, mut upload_max) =
if let Some((dl_min, dl_max, ul_min, ul_max)) = site.burst_rates(&config) {
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended && config.uisp_integration.suspended_strategy == "slow"
{
(0.1, 0.1, 0.1, 0.1)
} else {
(
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor,
),
)
};
if download_max < download_min {
download_max = download_min;
}
if upload_max < upload_min {
upload_max = upload_min;
}
let key = (site.id.clone(), device.id.clone());
if !seen_pairs.insert(key) {
continue;
@@ -135,10 +140,10 @@ pub async fn build_ap_only_network(
mac: device.mac.clone(),
ipv4: device.ipv4_list(),
ipv6: device.ipv6_list(),
download_min,
upload_min,
download_max,
upload_max,
download_min: ethernet_decision.download_min,
upload_min: ethernet_decision.upload_min,
download_max: ethernet_decision.download_max,
upload_max: ethernet_decision.upload_max,
comment: "".to_string(),
};
shaped_devices.push(sd);
@@ -158,6 +163,7 @@ pub async fn build_ap_only_network(
error!("{e:?}");
UispIntegrationError::CsvError
})?;
write_ethernet_advisories(&config, &ethernet_advisories)?;
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
@@ -1,9 +1,10 @@
use crate::blackboard_blob;
use crate::errors::UispIntegrationError;
use crate::ethernet_advisory::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::ip_ranges::IpRanges;
use crate::strategies::common::UispData;
use crate::strategies::full::shaped_devices_writer::ShapedDevice;
use lqos_config::Config;
use lqos_config::{CircuitEthernetMetadata, Config};
use std::collections::{HashMap, HashSet};
use std::fs::write;
use std::path::Path;
@@ -52,7 +53,14 @@ pub async fn build_ap_site_network(
//println!("{:#?}", root);
let mut shaped_devices = Vec::new();
let net_json = root.walk_children(None, &uisp_data, &mut shaped_devices, &config);
let mut ethernet_advisories = Vec::new();
let net_json = root.walk_children(
None,
&uisp_data,
&mut shaped_devices,
&mut ethernet_advisories,
&config,
);
let network_path = Path::new(&config.lqos_directory).join("network.json");
if network_path.exists() && !config.integration_common.always_overwrite_network_json {
@@ -69,7 +77,7 @@ pub async fn build_ap_site_network(
info!("Written network.json");
}
let _ = write_shaped_devices(&config, &mut shaped_devices);
let _ = write_shaped_devices(&config, &mut shaped_devices, &ethernet_advisories);
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
@@ -78,6 +86,7 @@ pub async fn build_ap_site_network(
pub(crate) fn write_shaped_devices(
config: &Arc<Config>,
shaped_devices: &mut Vec<ShapedDevice>,
ethernet_advisories: &[CircuitEthernetMetadata],
) -> Result<(), UispIntegrationError> {
let file_path = Path::new(&config.lqos_directory).join("ShapedDevices.csv");
let mut seen_pairs = HashSet::new();
@@ -95,6 +104,7 @@ pub(crate) fn write_shaped_devices(
error!("{e:?}");
UispIntegrationError::CsvError
})?;
write_ethernet_advisories(config, ethernet_advisories)?;
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
}
@@ -166,6 +176,7 @@ impl Layer {
parent: Option<&str>,
uisp_data: &UispData,
shaped_devices: &mut Vec<ShapedDevice>,
ethernet_advisories: &mut Vec<CircuitEthernetMetadata>,
config: &Config,
) -> serde_json::Map<String, serde_json::Value> {
let mut children = serde_json::Map::new();
@@ -181,13 +192,24 @@ impl Layer {
children.insert(
name.clone(),
child
.walk_children(Some(&parent_name), uisp_data, shaped_devices, config)
.walk_children(
Some(&parent_name),
uisp_data,
shaped_devices,
ethernet_advisories,
config,
)
.into(),
);
}
GraphMapping::ClientById(_client_id) => {
let _ =
child.walk_children(Some(&parent_name), uisp_data, shaped_devices, config);
let _ = child.walk_children(
Some(&parent_name),
uisp_data,
shaped_devices,
ethernet_advisories,
config,
);
}
_ => {}
}
@@ -244,67 +266,61 @@ impl Layer {
.iter()
.filter(|d| d.site_id == *client_id)
.collect::<Vec<_>>();
let requested = if let Some(site) =
uisp_data.sites.iter().find(|s| s.id == *client_id)
{
if let Some((dl_min, dl_max, ul_min, ul_max)) = site.burst_rates(config)
{
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended
&& config.uisp_integration.suspended_strategy == "slow"
{
(0.1, 0.1, 0.1, 0.1)
} else {
(
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_down_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config.uisp_integration.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor,
),
)
}
} else {
(0.1, 0.1, 0.1, 0.1)
};
let ethernet_decision = apply_ethernet_rate_cap(
&site.id,
&site.name,
devices.iter().copied(),
requested.0,
requested.2,
requested.1,
requested.3,
);
if let Some(advisory) = ethernet_decision.advisory.clone() {
ethernet_advisories.push(advisory);
}
for device in devices.iter().filter(|d| d.has_address()) {
// Compute subscriber rates: prefer UISP QoS + burst
let (download_min, mut download_max, upload_min, mut upload_max) =
if let Some(site) =
uisp_data.sites.iter().find(|s| s.id == *client_id)
{
if let Some((dl_min, dl_max, ul_min, ul_max)) =
site.burst_rates(config)
{
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended
&& config.uisp_integration.suspended_strategy == "slow"
{
(0.1, 0.1, 0.1, 0.1)
} else {
(
f32::max(
0.1,
site.max_down_mbps as f32
* config
.uisp_integration
.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_down_mbps as f32
* config
.uisp_integration
.bandwidth_overhead_factor,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config
.uisp_integration
.commit_bandwidth_multiplier,
),
f32::max(
0.1,
site.max_up_mbps as f32
* config
.uisp_integration
.bandwidth_overhead_factor,
),
)
}
} else {
(0.1, 0.1, 0.1, 0.1)
};
if download_max < download_min {
download_max = download_min;
}
if upload_max < upload_min {
upload_max = upload_min;
}
let sd = ShapedDevice {
circuit_id: site.id.clone(),
circuit_name: site.name.clone(),
@@ -314,10 +330,10 @@ impl Layer {
mac: device.mac.clone(),
ipv4: device.ipv4_list(),
ipv6: device.ipv6_list(),
download_min,
upload_min,
download_max,
upload_max,
download_min: ethernet_decision.download_min,
upload_min: ethernet_decision.upload_min,
download_max: ethernet_decision.download_max,
upload_max: ethernet_decision.upload_max,
comment: "".to_string(),
};
shaped_devices.push(sd);
@@ -19,7 +19,54 @@ pub(crate) struct UispData {
//pub data_links: Vec<UispDataLink>,
}
/// Ensure site names are unique by appending the site ID to duplicates.
fn short_address_segment(site: &Site) -> Option<String> {
let address = site.address()?;
let first_segment = address.split(',').next()?.trim();
if first_segment.is_empty() {
None
} else {
Some(first_segment.to_string())
}
}
fn service_name(site: &Site) -> Option<String> {
let name = site.ucrm.as_ref()?.service.as_ref()?.name.trim();
if name.is_empty() {
None
} else {
Some(name.to_string())
}
}
fn short_site_id(site: &Site) -> String {
site.id.chars().take(8).collect()
}
fn disambiguation_candidates(site: &Site, base_name: &str) -> Vec<String> {
let mut candidates = Vec::new();
if let Some(address_segment) = short_address_segment(site)
&& address_segment != base_name
{
candidates.push(format!("{base_name} ({address_segment})"));
}
if let Some(service_name) = service_name(site)
&& service_name != base_name
{
candidates.push(format!("{base_name} ({service_name})"));
}
let short_id = short_site_id(site);
if !short_id.is_empty() {
candidates.push(format!("{base_name} ({short_id})"));
}
candidates.push(format!("{base_name} ({})", site.id));
candidates
}
/// Ensure site names are unique by appending a human-friendly disambiguator to duplicates.
pub(crate) fn dedup_site_names(sites: &mut [Site]) {
let mut name_counts: HashMap<String, usize> = HashMap::new();
for site in sites.iter() {
@@ -30,6 +77,18 @@ pub(crate) fn dedup_site_names(sites: &mut [Site]) {
*name_counts.entry(base_name).or_insert(0) += 1;
}
let mut used_names: HashSet<String> = sites
.iter()
.filter_map(|site| {
let base_name = site.name_or_blank();
if base_name.is_empty() || name_counts.get(&base_name).copied().unwrap_or(0) > 1 {
None
} else {
Some(base_name)
}
})
.collect();
for site in sites.iter_mut() {
let base_name = site.name_or_blank();
if base_name.is_empty() {
@@ -37,7 +96,11 @@ pub(crate) fn dedup_site_names(sites: &mut [Site]) {
}
if name_counts.get(&base_name).copied().unwrap_or(0) > 1 {
let candidate = format!("{base_name} {}", site.id);
let candidate = disambiguation_candidates(site, &base_name)
.into_iter()
.find(|candidate| !used_names.contains(candidate))
.unwrap_or_else(|| format!("{base_name} ({})", site.id));
used_names.insert(candidate.clone());
if let Some(ident) = site.identification.as_mut() {
ident.name = Some(candidate.clone());
}
@@ -45,7 +108,7 @@ pub(crate) fn dedup_site_names(sites: &mut [Site]) {
site_id = %site.id,
original = %base_name,
renamed = %candidate,
"Duplicate UISP site name detected; renaming with site ID for LibreQoS uniqueness"
"Duplicate UISP site name detected; renaming with human-friendly disambiguator for LibreQoS uniqueness"
);
}
}
@@ -225,3 +288,94 @@ impl UispData {
.collect()
}
}
#[cfg(test)]
mod tests {
use super::dedup_site_names;
use serde_json::json;
use uisp::Site;
fn mk_site(id: &str, name: &str, address: Option<&str>, service_name: Option<&str>) -> Site {
let mut value = json!({
"id": id,
"identification": {
"id": id,
"name": name,
"type": "endpoint",
"suspended": false
}
});
if let Some(address) = address {
value["description"] = json!({ "address": address });
}
if let Some(service_name) = service_name {
value["ucrm"] = json!({
"service": {
"id": format!("svc-{id}"),
"name": service_name,
"status": 1,
"tariffId": "1",
"trafficShapingOverrideEnabled": false
}
});
}
serde_json::from_value(value).expect("site JSON must deserialize")
}
#[test]
fn duplicate_site_names_prefer_address_segment() {
let mut sites = vec![
mk_site(
"213f4b53-bddf-41dd-af65-2dbaa5bf0927",
"Pathway Communications",
Some("7100 Binational Way, Santa Teresa, 88008, New Mexico, United States"),
None,
),
mk_site(
"96bb13cd-10cc-43a1-ae56-a389161178d7",
"Pathway Communications",
Some("175 Lindburgh Dr, El Paso, 79932, Texas, United States"),
None,
),
];
dedup_site_names(&mut sites);
assert_eq!(
sites[0].name_or_blank(),
"Pathway Communications (7100 Binational Way)"
);
assert_eq!(
sites[1].name_or_blank(),
"Pathway Communications (175 Lindburgh Dr)"
);
}
#[test]
fn duplicate_site_names_fall_back_to_service_name() {
let mut sites = vec![
mk_site("site-1", "Acme", None, Some("1000/1000 Mbps - Primary")),
mk_site("site-2", "Acme", None, Some("1000/1000 Mbps - Backup")),
];
dedup_site_names(&mut sites);
assert_eq!(sites[0].name_or_blank(), "Acme (1000/1000 Mbps - Primary)");
assert_eq!(sites[1].name_or_blank(), "Acme (1000/1000 Mbps - Backup)");
}
#[test]
fn duplicate_site_names_fall_back_to_short_id() {
let mut sites = vec![
mk_site("abcd1234-0000-0000-0000-000000000000", "Acme", None, None),
mk_site("efgh5678-0000-0000-0000-000000000000", "Acme", None, None),
];
dedup_site_names(&mut sites);
assert_eq!(sites[0].name_or_blank(), "Acme (abcd1234)");
assert_eq!(sites[1].name_or_blank(), "Acme (efgh5678)");
}
}
@@ -1,9 +1,10 @@
use crate::blackboard_blob;
use crate::errors::UispIntegrationError;
use crate::ethernet_advisory::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::ip_ranges::IpRanges;
use crate::strategies::common::dedup_site_names;
use crate::uisp_types::UispDevice;
use lqos_config::Config;
use lqos_config::{CircuitEthernetMetadata, Config};
use serde::Serialize;
use std::collections::HashSet;
use std::fs;
@@ -82,6 +83,7 @@ pub async fn build_flat_network(
// Simple Shaped Devices File
let mut shaped_devices = Vec::new();
let mut ethernet_advisories: Vec<CircuitEthernetMetadata> = Vec::new();
let mut seen_pairs = HashSet::new();
let ipv4_to_v6 = Vec::new();
for site in sites.iter() {
@@ -96,7 +98,7 @@ pub async fn build_flat_network(
.map(|id| id.suspended)
.unwrap_or(false)
&& config.uisp_integration.suspended_strategy == "slow";
let (download_min, upload_min, mut download_max, mut upload_max) = if suspended_slow {
let requested = if suspended_slow {
(0.1, 0.1, 0.1, 0.1)
} else if let Some(qos) = &site.qos {
let base_down = qos
@@ -172,16 +174,25 @@ pub async fn build_flat_network(
f32::max(0.1, ul_max_f),
)
};
// Ensure max >= min
if download_max < download_min {
download_max = download_min;
}
if upload_max < upload_min {
upload_max = upload_min;
let site_devices: Vec<UispDevice> = devices
.iter()
.map(|device| UispDevice::from_uisp(device, &config, &ip_ranges, &ipv4_to_v6))
.filter(|device| device.site_id == site.id && device.has_address())
.collect();
let ethernet_decision = apply_ethernet_rate_cap(
&site.id,
&site.name_or_blank(),
site_devices.iter(),
requested.0,
requested.1,
requested.2,
requested.3,
);
if let Some(advisory) = ethernet_decision.advisory.clone() {
ethernet_advisories.push(advisory);
}
for device in devices.iter() {
let dev = UispDevice::from_uisp(device, &config, &ip_ranges, &ipv4_to_v6);
if dev.site_id == site.id {
if let Some(dev) = site_devices.iter().find(|dev| dev.id == device.get_id()) {
// We're an endpoint in the right sight. We're getting there
let key = (site.id.clone(), device.get_id());
if !seen_pairs.insert(key) {
@@ -197,10 +208,10 @@ pub async fn build_flat_network(
mac: device.identification.mac.clone().unwrap_or("".to_string()),
ipv4: dev.ipv4_list(),
ipv6: dev.ipv6_list(),
download_min,
download_max,
upload_min,
upload_max,
download_min: ethernet_decision.download_min,
download_max: ethernet_decision.download_max,
upload_min: ethernet_decision.upload_min,
upload_max: ethernet_decision.upload_max,
comment: "".to_string(),
};
shaped_devices.push(sd);
@@ -224,6 +235,7 @@ pub async fn build_flat_network(
error!("{e:?}");
UispIntegrationError::CsvError
})?;
write_ethernet_advisories(&config, &ethernet_advisories)?;
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
@@ -1,5 +1,7 @@
use crate::errors::UispIntegrationError;
use crate::ethernet_advisory::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::uisp_types::{UispDevice, UispSite, UispSiteType};
use lqos_config::CircuitEthernetMetadata;
use lqos_config::Config;
use serde::Serialize;
use std::path::Path;
@@ -24,6 +26,11 @@ pub struct ShapedDevice {
pub comment: String,
}
struct ShapedDeviceOutputs<'a> {
shaped_devices: &'a mut Vec<ShapedDevice>,
ethernet_advisories: &'a mut Vec<CircuitEthernetMetadata>,
}
/// Writes the ShapedDevices.csv file for UISP
///
/// # Arguments
@@ -39,17 +46,14 @@ pub fn write_shaped_devices(
) -> Result<(), UispIntegrationError> {
let file_path = Path::new(&config.lqos_directory).join("ShapedDevices.csv");
let mut shaped_devices = Vec::new();
let mut ethernet_advisories: Vec<CircuitEthernetMetadata> = Vec::new();
// Traverse
traverse(
sites,
root_idx,
0,
devices,
&mut shaped_devices,
config,
root_idx,
);
let mut outputs = ShapedDeviceOutputs {
shaped_devices: &mut shaped_devices,
ethernet_advisories: &mut ethernet_advisories,
};
traverse(sites, root_idx, 0, devices, &mut outputs, config, root_idx);
// Write the CSV
let mut writer = csv::WriterBuilder::new()
@@ -65,6 +69,7 @@ pub fn write_shaped_devices(
error!("{e:?}");
UispIntegrationError::CsvError
})?;
write_ethernet_advisories(config, &ethernet_advisories)?;
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
@@ -75,52 +80,60 @@ fn traverse(
idx: usize,
depth: u32,
devices: &[UispDevice],
shaped_devices: &mut Vec<ShapedDevice>,
outputs: &mut ShapedDeviceOutputs<'_>,
config: &Config,
root_idx: usize,
) {
if !sites[idx].device_indices.is_empty() {
// We have devices!
if sites[idx].site_type == UispSiteType::Client {
let site_devices: Vec<&UispDevice> = sites[idx]
.device_indices
.iter()
.filter_map(|device_idx| devices.get(*device_idx))
.collect();
let requested =
if let Some((dl_min, dl_max, ul_min, ul_max)) = sites[idx].burst_rates(config) {
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else {
let download_max_f32 = sites[idx].max_down_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor;
let upload_max_f32 = sites[idx].max_up_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor;
let download_min_f32 =
download_max_f32 * config.uisp_integration.commit_bandwidth_multiplier;
let upload_min_f32 =
upload_max_f32 * config.uisp_integration.commit_bandwidth_multiplier;
(
f32::max(0.1, download_min_f32),
f32::max(0.1, download_max_f32),
f32::max(0.1, upload_min_f32),
f32::max(0.1, upload_max_f32),
)
};
let ethernet_decision = apply_ethernet_rate_cap(
&sites[idx].id,
&sites[idx].name,
site_devices.iter().copied(),
requested.0,
requested.2,
requested.1,
requested.3,
);
if let Some(advisory) = ethernet_decision.advisory.clone() {
outputs.ethernet_advisories.push(advisory);
}
// Add as normal clients
for device in sites[idx].device_indices.iter() {
let device = &devices[*device];
if device.has_address() {
// Prefer UISP QoS + burst if available; else fallback to capacity-based
let (download_min, mut download_max, upload_min, mut upload_max) =
if let Some((dl_min, dl_max, ul_min, ul_max)) =
sites[idx].burst_rates(config)
{
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else {
let download_max_f32 = sites[idx].max_down_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor;
let upload_max_f32 = sites[idx].max_up_mbps as f32
* config.uisp_integration.bandwidth_overhead_factor;
let download_min_f32 = download_max_f32
* config.uisp_integration.commit_bandwidth_multiplier;
let upload_min_f32 = upload_max_f32
* config.uisp_integration.commit_bandwidth_multiplier;
(
f32::max(0.1, download_min_f32),
f32::max(0.1, download_max_f32),
f32::max(0.1, upload_min_f32),
f32::max(0.1, upload_max_f32),
)
};
// Ensure max >= min per LibreQoS requirements
if download_max < download_min {
download_max = download_min;
}
if upload_max < upload_min {
upload_max = upload_min;
}
let sd = ShapedDevice {
circuit_id: sites[idx].id.clone(),
circuit_name: sites[idx].name.clone(),
@@ -130,13 +143,13 @@ fn traverse(
mac: device.mac.clone(),
ipv4: device.ipv4_list(),
ipv6: device.ipv6_list(),
download_min,
download_max,
upload_min,
upload_max,
download_min: ethernet_decision.download_min,
download_max: ethernet_decision.download_max,
upload_min: ethernet_decision.upload_min,
upload_max: ethernet_decision.upload_max,
comment: "".to_string(),
};
shaped_devices.push(sd);
outputs.shaped_devices.push(sd);
}
}
} else {
@@ -178,7 +191,7 @@ fn traverse(
upload_max,
comment: "Infrastructure Entry".to_string(),
};
shaped_devices.push(sd);
outputs.shaped_devices.push(sd);
}
}
}
@@ -194,7 +207,7 @@ fn traverse(
child_idx,
depth + 1,
devices,
shaped_devices,
outputs,
config,
root_idx,
);
@@ -6,6 +6,7 @@ mod net_json_parent;
use crate::blackboard_blob;
use crate::errors::UispIntegrationError;
use crate::ethernet_advisory::{apply_ethernet_rate_cap, write_ethernet_advisories};
use crate::ip_ranges::IpRanges;
use crate::strategies::common::UispData;
use crate::strategies::full::routes_override::RouteOverride;
@@ -18,7 +19,7 @@ use crate::strategies::full2::graph_mapping::GraphMapping;
use crate::strategies::full2::link_mapping::LinkMapping;
use crate::strategies::full2::net_json_parent::{NetJsonParent, walk_parents};
use crate::uisp_types::UispDevice;
use lqos_config::Config;
use lqos_config::{CircuitEthernetMetadata, Config};
use petgraph::Directed;
use petgraph::graph::NodeIndex;
use petgraph::visit::{EdgeRef, NodeRef};
@@ -375,6 +376,7 @@ pub async fn build_full_network_v2(
let mut seen_pairs = HashSet::new();
let mut processed_site_pairs = 0usize;
let mut shaped_device_count = 0usize;
let mut ethernet_advisories: Vec<CircuitEthernetMetadata> = Vec::new();
for (ap_id, client_sites) in client_mappings.iter() {
for site_id in client_sites.iter() {
@@ -389,47 +391,55 @@ pub async fn build_full_network_v2(
"Processing site: {} (ID: {}) with AP: {} (ID: {})",
site.name, site.id, ap_device.name, ap_device.id
);
let site_devices: Vec<&UispDevice> = uisp_data
.devices
.iter()
.filter(|d| d.site_id == *site_id && d.has_address())
.collect();
let requested =
if let Some((dl_min, dl_max, ul_min, ul_max)) = site.burst_rates(&config) {
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended && config.uisp_integration.suspended_strategy == "slow" {
(0.1, 0.1, 0.1, 0.1)
} else {
let download_f32 = (site.max_down_mbps as f32)
* config.uisp_integration.bandwidth_overhead_factor;
let upload_f32 = (site.max_up_mbps as f32)
* config.uisp_integration.bandwidth_overhead_factor;
let download_min_f32 =
download_f32 * config.uisp_integration.commit_bandwidth_multiplier;
let upload_min_f32 =
upload_f32 * config.uisp_integration.commit_bandwidth_multiplier;
(
f32::max(0.1, download_min_f32),
f32::max(0.1, download_f32),
f32::max(0.1, upload_min_f32),
f32::max(0.1, upload_f32),
)
};
let ethernet_decision = apply_ethernet_rate_cap(
&site.id,
&site.name,
site_devices.iter().copied(),
requested.0,
requested.2,
requested.1,
requested.3,
);
if let Some(advisory) = ethernet_decision.advisory.clone() {
ethernet_advisories.push(advisory);
}
for device in uisp_data.devices.iter().filter(|d| d.site_id == *site_id) {
if !device.has_address() {
continue;
}
// Compute subscriber rates: prefer UISP QoS + burst; fallback to capacity-based
let (download_min, mut download_max, upload_min, mut upload_max) =
if let Some((dl_min, dl_max, ul_min, ul_max)) = site.burst_rates(&config) {
(
f32::max(0.1, dl_min),
f32::max(0.1, dl_max),
f32::max(0.1, ul_min),
f32::max(0.1, ul_max),
)
} else if site.suspended && config.uisp_integration.suspended_strategy == "slow"
{
(0.1, 0.1, 0.1, 0.1)
} else {
let download_f32 = (site.max_down_mbps as f32)
* config.uisp_integration.bandwidth_overhead_factor;
let upload_f32 = (site.max_up_mbps as f32)
* config.uisp_integration.bandwidth_overhead_factor;
let download_min_f32 =
download_f32 * config.uisp_integration.commit_bandwidth_multiplier;
let upload_min_f32 =
upload_f32 * config.uisp_integration.commit_bandwidth_multiplier;
(
f32::max(0.1, download_min_f32),
f32::max(0.1, download_f32),
f32::max(0.1, upload_min_f32),
f32::max(0.1, upload_f32),
)
};
// Ensure max >= min
if download_max < download_min {
download_max = download_min;
}
if upload_max < upload_min {
upload_max = upload_min;
}
let parent_node = {
if parents.contains_key(&ap_device.name) {
ap_device.name.clone()
@@ -456,10 +466,10 @@ pub async fn build_full_network_v2(
mac: device.mac.to_owned(),
ipv4: device.ipv4_list(),
ipv6: device.ipv6_list(),
download_min,
upload_min,
download_max,
upload_max,
download_min: ethernet_decision.download_min,
upload_min: ethernet_decision.upload_min,
download_max: ethernet_decision.download_max,
upload_max: ethernet_decision.upload_max,
comment: "".to_string(),
};
debug!(
@@ -489,6 +499,7 @@ pub async fn build_full_network_v2(
error!("{e:?}");
UispIntegrationError::CsvError
})?;
write_ethernet_advisories(&config, &ethernet_advisories)?;
info!("Wrote {} lines to ShapedDevices.csv", shaped_devices.len());
Ok(())
@@ -15,14 +15,137 @@ pub struct UispDevice {
pub id: String,
pub name: String,
pub mac: String,
pub role: Option<String>,
pub wireless_mode: Option<String>,
pub site_id: String,
pub download: u64,
pub upload: u64,
pub ipv4: HashSet<String>,
pub ipv6: HashSet<String>,
pub negotiated_ethernet_mbps: Option<u64>,
pub negotiated_ethernet_interface: Option<String>,
}
impl UispDevice {
fn parse_speed_mbps(raw: &str) -> Option<u64> {
let lower = raw.trim().to_lowercase();
if lower.is_empty() {
return None;
}
let digits: String = lower
.chars()
.take_while(|ch| ch.is_ascii_digit() || *ch == '.')
.collect();
if digits.is_empty() {
return None;
}
let value = digits.parse::<f64>().ok()?;
if !(value.is_finite() && value > 0.0) {
return None;
}
let multiplier = if lower.contains("gbps") || lower.ends_with('g') {
1000.0
} else {
1.0
};
Some((value * multiplier).round() as u64)
}
fn negotiated_ethernet_from_device(device: &Device) -> (Option<u64>, Option<String>) {
let mut best: Option<(u64, Option<String>)> = None;
let Some(interfaces) = &device.interfaces else {
return (None, None);
};
for interface in interfaces {
let interface_type = interface
.identification
.as_ref()
.and_then(|id| id.r#type.clone())
.unwrap_or_default()
.to_lowercase();
if matches!(interface_type.as_str(), "wlan" | "wifi" | "wireless") {
continue;
}
let Some(status) = &interface.status else {
continue;
};
let speed_raw = if let Some(current_speed) = status.currentSpeed.as_deref() {
Some(current_speed)
} else if Self::allows_plain_speed(status.status.as_deref()) {
status.speed.as_deref()
} else {
None
};
let Some(speed_raw) = speed_raw else {
continue;
};
let Some(speed_mbps) = Self::parse_speed_mbps(speed_raw) else {
continue;
};
let iface_name = interface
.identification
.as_ref()
.and_then(|id| id.name.clone());
if best
.as_ref()
.is_none_or(|(current_speed, _)| speed_mbps < *current_speed)
{
best = Some((speed_mbps, iface_name));
}
}
best.map_or((None, None), |(speed, iface)| (Some(speed), iface))
}
fn allows_plain_speed(status: Option<&str>) -> bool {
let Some(status) = status else {
return false;
};
matches!(
status.trim().to_ascii_lowercase().as_str(),
"connected" | "up" | "active"
)
}
pub(crate) fn is_wireless_station_cpe(&self) -> bool {
let role = self
.role
.as_deref()
.map(str::trim)
.map(str::to_ascii_lowercase);
if matches!(role.as_deref(), Some("station" | "sta" | "cpe")) {
return true;
}
let wireless_mode = self
.wireless_mode
.as_deref()
.map(str::trim)
.map(str::to_ascii_lowercase);
matches!(
wireless_mode.as_deref(),
Some(mode) if mode.starts_with("sta") || mode.starts_with("station")
)
}
pub(crate) fn is_router_like(&self) -> bool {
let role = self
.role
.as_deref()
.map(str::trim)
.map(str::to_ascii_lowercase);
matches!(
role.as_deref(),
Some("router" | "homewifi" | "home-wifi" | "home wifi")
)
}
/// Check if an IP/CIDR represents a network address (all host bits are zero)
/// Returns true if it's a network address, false if it's a host address
#[cfg_attr(test, allow(dead_code))]
@@ -209,16 +332,25 @@ impl UispDevice {
site_id = exception.parent.clone();
}
}
let (negotiated_ethernet_mbps, negotiated_ethernet_interface) =
Self::negotiated_ethernet_from_device(device);
Self {
id: device.get_id(),
name: device.get_name().unwrap_or_default(),
mac,
role: device.identification.role.clone(),
wireless_mode: device
.overview
.as_ref()
.and_then(|o| o.wirelessMode.clone()),
site_id,
upload,
download,
ipv4,
ipv6,
negotiated_ethernet_mbps,
negotiated_ethernet_interface,
}
}
@@ -269,6 +401,8 @@ impl UispDevice {
#[cfg(test)]
mod tests {
use super::UispDevice;
use serde_json::json;
use uisp::Device;
#[test]
fn test_is_network_address() {
@@ -302,4 +436,74 @@ mod tests {
assert!(!UispDevice::is_network_address("192.168.1.0/invalid")); // Invalid prefix
assert!(!UispDevice::is_network_address("192.168.1.0/33")); // Invalid IPv4 prefix length
}
#[test]
fn test_parse_speed_mbps() {
assert_eq!(UispDevice::parse_speed_mbps("100Mbps-Full"), Some(100));
assert_eq!(UispDevice::parse_speed_mbps("1000"), Some(1000));
assert_eq!(UispDevice::parse_speed_mbps("1Gbps"), Some(1000));
assert_eq!(UispDevice::parse_speed_mbps("2.5Gbps"), Some(2500));
assert_eq!(UispDevice::parse_speed_mbps(""), None);
assert_eq!(UispDevice::parse_speed_mbps("unknown"), None);
}
fn mk_device_with_interfaces(interfaces: serde_json::Value) -> Device {
serde_json::from_value(json!({
"identification": {
"id": "dev-1",
"hostname": "dev-1",
"role": null,
},
"overview": {
"wirelessMode": null
},
"interfaces": interfaces,
}))
.expect("device JSON must deserialize")
}
#[test]
fn disconnected_plain_speed_is_ignored_for_negotiated_ethernet() {
let device = mk_device_with_interfaces(json!([
{
"identification": { "name": "eth0@1", "type": "eth" },
"status": { "status": "disconnected", "speed": "10-half", "currentSpeed": null },
"wireless": {}
}
]));
let (speed, iface) = UispDevice::negotiated_ethernet_from_device(&device);
assert_eq!(speed, None);
assert_eq!(iface, None);
}
#[test]
fn disconnected_currentspeed_is_still_accepted() {
let device = mk_device_with_interfaces(json!([
{
"identification": { "name": "data", "type": "eth" },
"status": { "status": "disconnected", "speed": "auto", "currentSpeed": "100-full" },
"wireless": {}
}
]));
let (speed, iface) = UispDevice::negotiated_ethernet_from_device(&device);
assert_eq!(speed, Some(100));
assert_eq!(iface.as_deref(), Some("data"));
}
#[test]
fn connected_plain_speed_is_accepted() {
let device = mk_device_with_interfaces(json!([
{
"identification": { "name": "eth0", "type": "eth" },
"status": { "status": "connected", "speed": "1000-full", "currentSpeed": null },
"wireless": {}
}
]));
let (speed, iface) = UispDevice::negotiated_ethernet_from_device(&device);
assert_eq!(speed, Some(1000));
assert_eq!(iface.as_deref(), Some("eth0"));
}
}