mirror of
https://github.com/LibreQoE/LibreQoS.git
synced 2026-10-25 06:17:39 -05:00
changes
This commit is contained in:
rchac committed
1 parent
9a5553f073
commit
d1a63b21bf
130 files changed
+10108
-1177
No files matched your search
@@ -20,7 +20,7 @@ Situation:
|
||||
|
||||
Approach:
|
||||
- Adopted built-in UISP integration as durable source of truth.
|
||||
- Standardized on integration-owned `ShapedDevices.csv` with explicit overwrite policy.
|
||||
- Standardized on integration-driven shaping data with a clear refresh policy.
|
||||
- Started with moderate hierarchy depth before considering deeper topology.
|
||||
|
||||
Outcome:
|
||||
|
||||
@@ -27,7 +27,7 @@ For most operators:
|
||||
|
||||
Most day-to-day LibreQoS configuration is handled in WebUI (`http://your_shaper_ip:9123/config_general.html`).
|
||||
|
||||
Current builds use shared configuration panels on the General, RTT, Queues, TreeGuard, Network Mode, Integration Defaults, Network Layout, Insight, provider-specific integration pages, IP Ranges, Flow Tracking, and Shaped Devices pages so node settings and operational tuning follow the same surface styling as the rest of Node Manager. The TreeGuard page starts with a live behavior summary rather than a permanent warning banner. Integration Defaults also carries the shared Ethernet port headroom policy used by integrations that can detect negotiated subscriber-facing port speeds.
|
||||
Current builds use a consistent configuration layout across the General, RTT, Queues, TreeGuard, Network Mode, Integration Defaults, Network Layout, Insight, provider integration, IP Ranges, Flow Tracking, and Shaped Devices pages. Integration Defaults also includes the shared Ethernet port headroom policy used by integrations that can detect negotiated subscriber-facing port speeds.
|
||||
|
||||
### Where in WebUI
|
||||
|
||||
@@ -37,11 +37,11 @@ Current builds use shared configuration panels on the General, RTT, Queues, Tree
|
||||
- Shaped devices editor: `Configuration -> Shaped Devices`
|
||||
- Runtime operational validation: `WebUI (Node Manager)` pages (dashboard/tree/flow/scheduler)
|
||||
|
||||
When an integration is enabled and acting as the source of truth, the `Network Layout` and `Shaped Devices` editors remain viewable but become read-only in WebUI.
|
||||
When an integration is managing your shaping data, the `Network Layout` and `Shaped Devices` editors remain visible but become read-only in WebUI.
|
||||
|
||||
Topology note:
|
||||
- `network.json` node names must be globally unique across the whole tree. Duplicate node names now fail validation and are not accepted by the WebUI save path or `LibreQoS.py`.
|
||||
- When a node exposes a stable `id`, current builds prefer that ID for operator-owned site bandwidth overrides while keeping legacy name-only matching as a fallback.
|
||||
- When a node exposes a stable `id`, LibreQoS prefers that ID for saved site bandwidth overrides while keeping legacy name-only matching as a fallback.
|
||||
|
||||
Queue-mode note:
|
||||
- Current builds use `queue_mode` with `shape` and `observe` values. Older `monitor_only` wording is a compatibility alias rather than the primary operator-facing setting.
|
||||
|
||||
@@ -42,8 +42,8 @@ Example symptoms:
|
||||
|
||||
Representative issues:
|
||||
|
||||
- [#860 - always_overwrite_network_json default behavior confusion](https://github.com/LibreQoE/LibreQoS/issues/860)
|
||||
- [#899 - UISP: always_overwrite_network_json=false and missing ShapedDevices.csv](https://github.com/LibreQoE/LibreQoS/issues/899)
|
||||
- [#860 - legacy network.json overwrite-default confusion](https://github.com/LibreQoE/LibreQoS/issues/860)
|
||||
- [#899 - UISP: legacy overwrite-disabled path and missing ShapedDevices.csv](https://github.com/LibreQoE/LibreQoS/issues/899)
|
||||
- [#845 - UISP: multi-services with same site name](https://github.com/LibreQoE/LibreQoS/issues/845)
|
||||
- [#699 - UISP: trailing spaces break matching](https://github.com/LibreQoE/LibreQoS/issues/699)
|
||||
|
||||
|
||||
@@ -23,10 +23,9 @@ Run a manual import with:
|
||||
python3 integrationNetzur.py
|
||||
```
|
||||
|
||||
The integration regenerates `ShapedDevices.csv` and, unless `always_overwrite_network_json` is disabled, updates `network.json`. Adjust the Integration → Common settings if you need to preserve an existing network map between Netzur syncs.
|
||||
The integration regenerates `ShapedDevices.csv` for its legacy DIY-style path, but built-in integrations do not write `network.json`. Keep `network.json` for DIY/manual deployments.
|
||||
|
||||
Overwrite policy:
|
||||
- Recommended: keep `always_overwrite_network_json = true` for integration-driven deployments so topology stays aligned with Netzur syncs.
|
||||
For integration-driven workflows, validate the import in WebUI and through the current topology/shaping files rather than editing `network.json`.
|
||||
|
||||
|
||||
## Related Pages
|
||||
|
||||
@@ -25,16 +25,16 @@ If you use your own scripts as the source of truth for `network.json` and `Shape
|
||||
|
||||
## What these integrations do
|
||||
|
||||
These integrations work by synchronizing with your CRM/NMS system to produce the two files LibreQoS requires for shaping - [network.json](configuration.md#network-hierarchy) and [ShapedDevices.csv](configuration.md#circuits).
|
||||
These integrations synchronize with your CRM/NMS system to produce integration-owned topology and shaping artifacts for LibreQoS.
|
||||
|
||||
Important behavior when integrations are enabled:
|
||||
- `ShapedDevices.csv` is typically regenerated by integration sync jobs.
|
||||
- `network.json` overwrite behavior depends on your integration settings (for example `always_overwrite_network_json`).
|
||||
- Built-in integrations publish integration-owned topology and shaping artifacts.
|
||||
- `network.json` remains a DIY/manual ingress file and is not written by built-in integrations.
|
||||
- Manual edits may be overwritten on the next refresh cycle.
|
||||
|
||||
## Splynx Integration
|
||||
|
||||
> **⚠️ Breaking Change Notice**: Prior to v1.5-RC-2, the default Splynx strategy was `full`. Starting with v1.5-RC-2, the default strategy is `ap_only` for improved CPU performance. If you require the previous behavior, explicitly set `strategy = "full"` in your Splynx configuration section.
|
||||
> **⚠️ Breaking Change Notice**: Prior to v1.5-RC-2, the default Splynx strategy was `full`. The default shared topology mode is now `ap_site` as the best balance of structure and operator clarity. If you require the previous behavior, explicitly set `strategy = "full"` in your Splynx configuration section.
|
||||
|
||||
First, set the relevant parameters for Splynx (splynx_api_key, splynx_api_secret, etc.) in `/etc/lqos.conf`.
|
||||
|
||||
@@ -45,8 +45,8 @@ LibreQoS supports multiple topology strategies for Splynx integration to balance
|
||||
| Strategy | Description | CPU Impact | Use Case |
|
||||
|----------|-------------|------------|----------|
|
||||
| `flat` | Only shapes subscribers, no hierarchy | Lowest | Maximum performance, simple subscriber-only shaping |
|
||||
| `ap_only` | Single layer: AP → Clients | Low | **Default**. Best balance of performance and structure |
|
||||
| `ap_site` | Two layers: Site → AP → Clients | Medium | Site-level aggregation with moderate complexity |
|
||||
| `ap_only` | Single layer: AP → Clients | Low | Lowest overhead when site-level grouping is not needed |
|
||||
| `ap_site` | Two layers: Site → AP → Clients | Medium | **Default**. Best balance of structure and operator clarity |
|
||||
| `full` | Complete topology mapping | Highest | Full network hierarchy representation |
|
||||
|
||||
Configure the strategy in `/etc/lqos.conf` under the `[splynx_integration]` section:
|
||||
@@ -54,12 +54,12 @@ Configure the strategy in `/etc/lqos.conf` under the `[splynx_integration]` sect
|
||||
```ini
|
||||
[splynx_integration]
|
||||
# ... other splynx settings ...
|
||||
strategy = "ap_only"
|
||||
strategy = "ap_site"
|
||||
```
|
||||
|
||||
**Performance Considerations:**
|
||||
- `flat` and `ap_only` strategies significantly reduce CPU load by limiting network depth
|
||||
- Choose `ap_only` for most deployments unless you need site-level traffic aggregation
|
||||
- Choose `ap_site` by default, and move to `ap_only` only when you want lower overhead and can give up site-level grouping
|
||||
- Only use `full` if you require complete network topology representation and have adequate CPU resources
|
||||
|
||||
### Promote to Root Nodes (Performance Optimization)
|
||||
@@ -122,12 +122,8 @@ To test the Splynx Integration, use
|
||||
python3 integrationSplynx.py
|
||||
```
|
||||
|
||||
On the first successful run, it will create a ShapedDevices.csv file and network.json.
|
||||
ShapedDevices.csv will be overwritten every time the Splynx integration is run.
|
||||
|
||||
To ensure the network.json is always overwritten with the newest version pulled in by the integration, please edit `/etc/lqos.conf` with the command `sudo nano /etc/lqos.conf`.
|
||||
Edit the file to set the value of `always_overwrite_network_json` to `true`.
|
||||
Then, run `sudo systemctl restart lqosd`.
|
||||
On the first successful run, it will create integration-owned shaping data.
|
||||
Built-in integrations do not overwrite `network.json`; keep DIY `network.json` operator-owned.
|
||||
|
||||
You have the option to run integrationSplynx.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_splynx = true``` under the ```[splynx_integration]``` section in `/etc/lqos.conf`.
|
||||
Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
@@ -167,7 +163,7 @@ Run a manual import with:
|
||||
python3 integrationNetzur.py
|
||||
```
|
||||
|
||||
The integration regenerates `ShapedDevices.csv` and, unless `always_overwrite_network_json` is disabled, updates `network.json`. Adjust the Integration → Common settings if you need to preserve an existing network map between Netzur syncs.
|
||||
The integration regenerates `ShapedDevices.csv` for its legacy DIY-style path, but built-in integrations no longer write `network.json`. Keep DIY `network.json` operator-owned.
|
||||
|
||||
## VISP Integration
|
||||
|
||||
@@ -190,7 +186,7 @@ timeout_secs = 20
|
||||
Notes:
|
||||
- VISP import is GraphQL-based and currently defaults to a flat topology strategy.
|
||||
- The integration writes `ShapedDevices.csv` every run.
|
||||
- `network.json` is only overwritten when `always_overwrite_network_json = true` (under `[integration_common]`).
|
||||
- `network.json` remains a DIY/manual ingress file; built-in integrations do not overwrite it.
|
||||
- VISP auth tokens are cached in `<lqos_directory>/.visp_token_cache_*.json`.
|
||||
|
||||
Run a manual import with:
|
||||
@@ -223,7 +219,7 @@ LibreQoS supports multiple topology strategies for UISP integration to balance C
|
||||
| `full` | Shapes entire network including backhauls, Sites, APs, and clients | Highest | Best after topology/overrides are validated and stable |
|
||||
|
||||
**Choosing the Right Strategy:**
|
||||
- Start with `ap_only` for new deployments and initial validation
|
||||
- Start with `ap_site` for new deployments and initial validation
|
||||
- Move to `ap_site` when you need site-level control but not backhaul shaping
|
||||
- Move to `full` after topology quality and overrides are validated, and CPU headroom is confirmed
|
||||
- Use `flat` only when maximum performance is critical and you don't need any hierarchy
|
||||
@@ -278,7 +274,7 @@ url = "https://uisp.your_domain.com"
|
||||
site = "Root_Site_Name" # Root site for topology perspective
|
||||
|
||||
# Topology Strategy (see table above)
|
||||
strategy = "ap_only" # Recommended starting point for new UISP deployments
|
||||
strategy = "ap_site" # Recommended starting point for new UISP deployments
|
||||
|
||||
# Suspension Handling (see table above)
|
||||
suspended_strategy = "none"
|
||||
@@ -300,7 +296,6 @@ commit_bandwidth_multiplier = 0.98 # Set minimum to 98% of maximum (CIR)
|
||||
|
||||
# Advanced Options
|
||||
ipv6_with_mikrotik = false # Enable if using DHCPv6 with MikroTik
|
||||
always_overwrite_network_json = true # Recommended for integration-driven deployments
|
||||
exception_cpes = [] # CPE exceptions in ["cpe:parent"] format
|
||||
squash_sites = [] # Optional: sites to squash
|
||||
do_not_squash_sites = [] # Optional: keep these site names unsquashed in the runtime/export tree
|
||||
@@ -341,15 +336,13 @@ sudo /opt/libreqos/src/bin/uisp_integration
|
||||
```
|
||||
|
||||
On the first successful run, it will create a network.json and ShapedDevices.csv file.
|
||||
If a network.json file exists, it will not be overwritten, unless you set ```always_overwrite_network_json = true```.
|
||||
Built-in integrations do not overwrite an existing `network.json`; keep DIY `network.json` operator-owned.
|
||||
|
||||
ShapedDevices.csv will be overwritten every time the UISP integration is run.
|
||||
|
||||
If UISP exposes a site and an AP with the same visible name in the same topology, current builds keep the site name stable in `network.json` and disambiguate the AP name during export so the site branch is not dropped from the tree.
|
||||
|
||||
To ensure the network.json is always overwritten with the newest version pulled in by the integration, please edit `/etc/lqos.conf` with the command `sudo nano /etc/lqos.conf`.
|
||||
Edit the file to set the value of `always_overwrite_network_json` to `true`.
|
||||
Then, run `sudo systemctl restart lqosd`.
|
||||
Built-in integrations do not overwrite `network.json`; keep DIY `network.json` operator-owned.
|
||||
|
||||
You have the option to run `uisp_integration` automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_uisp = true``` in `/etc/lqos.conf`. Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
|
||||
@@ -394,9 +387,7 @@ python3 integrationWISPGate.py
|
||||
On the first successful run, it will create a ShapedDevices.csv file and network.json.
|
||||
ShapedDevices.csv will be overwritten every time the WISPGate integration is run.
|
||||
|
||||
To ensure the network.json is always overwritten with the newest version pulled in by the integration, please edit `/etc/lqos.conf` with the command `sudo nano /etc/lqos.conf`.
|
||||
Edit the file to set the value of `always_overwrite_network_json` to `true`.
|
||||
Then, run `sudo systemctl restart lqosd`.
|
||||
Built-in integrations do not overwrite `network.json`; keep DIY `network.json` operator-owned.
|
||||
|
||||
You have the option to run integrationWISPGate.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_wispgate = true``` in `/etc/lqos.conf`.
|
||||
Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
@@ -427,7 +418,7 @@ python3 integrationSonar.py
|
||||
```
|
||||
|
||||
On the first successful run, it will create a ShapedDevices.csv file.
|
||||
If a network.json file exists, it will not be overwritten, unless you set ```always_overwrite_network_json = true```.
|
||||
Built-in integrations do not overwrite an existing `network.json`; keep DIY `network.json` operator-owned.
|
||||
You can modify the network.json file to more accurately reflect bandwidth limits.
|
||||
ShapedDevices.csv will be overwritten every time the Sonar integration is run.
|
||||
You have the option to run integrationSonar.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_sonar = true``` in `/etc/lqos.conf`.
|
||||
|
||||
@@ -18,11 +18,10 @@ To test the Sonar Integration, use
|
||||
python3 integrationSonar.py
|
||||
```
|
||||
|
||||
On the first successful run, it will create a ShapedDevices.csv file.
|
||||
If a network.json file exists, it will not be overwritten, unless you set ```always_overwrite_network_json = true```.
|
||||
You can modify the network.json file to more accurately reflect bandwidth limits.
|
||||
ShapedDevices.csv will be overwritten every time the Sonar integration is run.
|
||||
Recommended: keep `always_overwrite_network_json = true` for integration-driven deployments so topology stays aligned with Sonar syncs.
|
||||
On the first successful run, it creates the Sonar import and shaping data LibreQoS needs for scheduled refreshes.
|
||||
Built-in Sonar integrations do not write `network.json`; keep that file for DIY/manual deployments.
|
||||
You can maintain a separate DIY `network.json` when Sonar is not the system you want to use for topology.
|
||||
The Sonar integration refreshes its imported topology and shaping data every run.
|
||||
You have the option to run integrationSonar.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_sonar = true``` in `/etc/lqos.conf`.
|
||||
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Splynx Integration
|
||||
|
||||
> **⚠️ Breaking Change Notice**: Prior to v1.5-RC-2, the default Splynx strategy was `full`. Starting with v1.5-RC-2, the default strategy is `ap_only` for improved CPU performance. If you require the previous behavior, explicitly set `strategy = "full"` in your Splynx configuration section.
|
||||
> **⚠️ Breaking Change Notice**: Prior to v1.5-RC-2, the default Splynx strategy was `full`. The recommended shared topology mode is now `ap_site` as the best default balance of structure and operator clarity. If you require the previous behavior, explicitly set `topology.compile_mode = "full"`.
|
||||
|
||||
First, set the relevant parameters for Splynx (splynx_api_key, splynx_api_secret, etc.) in `/etc/lqos.conf`.
|
||||
|
||||
@@ -8,15 +8,11 @@ First, set the relevant parameters for Splynx (splynx_api_key, splynx_api_secret
|
||||
|
||||
Start with this baseline unless you have a known reason to deviate:
|
||||
|
||||
- `strategy = "ap_only"` (default, lowest confusion)
|
||||
- `topology.compile_mode = "ap_site"` (default)
|
||||
- `enable_splynx = true`
|
||||
- `always_overwrite_network_json = true` for integration-driven deployments
|
||||
|
||||
Then run one manual sync and validate outputs before enabling frequent scheduler refresh cycles.
|
||||
|
||||
Overwrite policy:
|
||||
- Recommended: keep `always_overwrite_network_json = true` so topology stays aligned with Splynx on each refresh cycle.
|
||||
|
||||
### Topology Strategies
|
||||
|
||||
LibreQoS supports multiple topology strategies for Splynx integration to balance CPU performance with network hierarchy needs:
|
||||
@@ -24,21 +20,22 @@ LibreQoS supports multiple topology strategies for Splynx integration to balance
|
||||
| Strategy | Description | CPU Impact | Use Case |
|
||||
|----------|-------------|------------|----------|
|
||||
| `flat` | Only shapes subscribers, no hierarchy | Lowest | Maximum performance, simple subscriber-only shaping |
|
||||
| `ap_only` | Single layer: AP → Clients | Low | **Default**. Best balance of performance and structure |
|
||||
| `ap_site` | Two layers: Site → AP → Clients | Medium | Site-level aggregation with moderate complexity |
|
||||
| `full` | Complete topology mapping | Highest | Full network hierarchy representation |
|
||||
| `ap_only` | Single layer: AP → Clients | Low | Lowest overhead when site-level grouping is not needed |
|
||||
| `ap_site` | Two layers: Site → AP → Clients | Medium | **Default**. Best balance of structure and operator clarity |
|
||||
| `full` | Complete monitoring-parent topology mapping | Highest | Preserves the richer Splynx monitoring hierarchy instead of flattening to Network Sites |
|
||||
|
||||
Configure the strategy in `/etc/lqos.conf` under the `[splynx_integration]` section:
|
||||
When Splynx `full` cannot infer a circuit parent from `access_device` or router metadata, LibreQoS now groups those circuits under a single generated `LibreQoS Unattached [Site]` node instead of leaving them fully unresolved at runtime.
|
||||
|
||||
Configure the shared topology mode in `/etc/lqos.conf` under the `[topology]` section:
|
||||
|
||||
```ini
|
||||
[splynx_integration]
|
||||
# ... other splynx settings ...
|
||||
strategy = "ap_only"
|
||||
[topology]
|
||||
compile_mode = "ap_site"
|
||||
```
|
||||
|
||||
**Performance Considerations:**
|
||||
- `flat` and `ap_only` strategies significantly reduce CPU load by limiting network depth
|
||||
- Choose `ap_only` for most deployments unless you need site-level traffic aggregation
|
||||
- Choose `ap_site` by default, and move to `ap_only` only when you want lower overhead and can give up site-level grouping
|
||||
- Only use `full` if you require complete network topology representation and have adequate CPU resources
|
||||
|
||||
### Promote to Root Nodes (Performance Optimization)
|
||||
@@ -101,10 +98,12 @@ To test the Splynx Integration, use
|
||||
python3 integrationSplynx.py
|
||||
```
|
||||
|
||||
On the first successful run, it creates `ShapedDevices.csv` and `network.json`.
|
||||
ShapedDevices.csv will be overwritten every time the Splynx integration is run.
|
||||
On the first successful run, it creates the Splynx import and shaping files LibreQoS needs for scheduled refreshes.
|
||||
Legacy `integrationSplynxBandwidths.csv` values are migrated into operator `AdjustSiteSpeed` overrides in `lqos_overrides.json`, and the original CSV is renamed to a `.backup` file.
|
||||
|
||||
Current builds also expose a shared Ethernet port headroom policy under `Configuration -> Integrations -> Integration Defaults`. Integrations that can supply negotiated subscriber-facing port speed use a conservative default multiplier of `0.94` unless the operator overrides it.
|
||||
Current builds also expose the shared topology compile mode under `Configuration -> Integrations -> Integration Defaults`. Existing `splynx_integration.strategy` values are retained only as a compatibility mirror during upgrade.
|
||||
Topology Manager now receives a native infrastructure topology export for Splynx imports, so real sites and APs use their resolved parent chain instead of being reconstructed from the compatibility tree. Customer-derived generated sites stay compatibility-only: they are excluded from canonical/editor topology and circuits shape under the nearest real infrastructure parent instead. Current Python-backed imports are intentionally conservative: infrastructure nodes import as fixed roots or fixed-parent branches unless the importer can provide a bounded, trustworthy set of alternative parents. When a bounded move set is available, it is limited to the node's current parent plus nearby sibling-parent alternatives under the same upstream branch, or peer root parents of the same type when the current parent is itself a root.
|
||||
|
||||
You have the option to run integrationSplynx.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_splynx = true``` under the ```[splynx_integration]``` section in `/etc/lqos.conf`.
|
||||
Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
@@ -117,8 +116,9 @@ python3 integrationSplynx.py
|
||||
```
|
||||
2. Confirm files exist and were updated:
|
||||
```shell
|
||||
ls -lh /opt/libreqos/src/network.json /opt/libreqos/src/ShapedDevices.csv
|
||||
ls -lh /opt/libreqos/src/topology_import.json /opt/libreqos/src/shaping_inputs.json
|
||||
```
|
||||
Built-in Splynx imports refresh LibreQoS's imported topology and shaping data automatically. They do not use `network.json` or `ShapedDevices.csv` as the normal working files for this integration.
|
||||
3. Confirm services are healthy:
|
||||
```shell
|
||||
sudo systemctl status lqosd lqos_scheduler
|
||||
@@ -132,7 +132,7 @@ If sync looks successful but data is incomplete, recheck API permissions and str
|
||||
|
||||
You can also modify the the file `integrationSplynxBandwidths.csv` to override the default bandwidths for each Node (Site, AP).
|
||||
|
||||
Tree-page `Operator Override` edits are separate from this legacy Splynx file. Current builds write those operator-owned node rate changes to `lqos_overrides.json` and do not rewrite `integrationSplynxBandwidths.csv`.
|
||||
Rate overrides saved from the tree page are separate from this legacy Splynx CSV workflow. Pick one long-term override method for a given node so your changes stay predictable.
|
||||
|
||||
A template is available in the `/opt/libreqos/src` folder. To utilize the template, copy the file `integrationSplynxBandwidths.template.csv` (removing the `.template` part of the filename) and set the appropriate information inside each file. For example, if you want to change the set bandwidth for a site, you would do:
|
||||
```
|
||||
@@ -140,7 +140,7 @@ sudo cp /opt/libreqos/src/integrationSplynxBandwidths.template.csv /opt/libreqos
|
||||
```
|
||||
And edit the CSV using LibreOffice or your preferred CSV editor.
|
||||
|
||||
To avoid conflicting sources of truth, prefer one durable override path per node: either the legacy Splynx CSV workflow or the operator override layer.
|
||||
To avoid conflicting edits, prefer one long-term override path per node: either the legacy Splynx CSV workflow or WebUI overrides.
|
||||
|
||||
|
||||
## Related Pages
|
||||
|
||||
@@ -4,9 +4,9 @@ First, set the relevant parameters for UISP in `/etc/lqos.conf`.
|
||||
|
||||
## New Operator Quick Chooser
|
||||
|
||||
Use this section first to avoid common strategy confusion.
|
||||
Use this section first to avoid common topology-mode confusion.
|
||||
|
||||
1. If you are new to UISP integration, start with `strategy = "ap_only"`.
|
||||
1. If you are new to UISP integration, start with `topology.compile_mode = "ap_site"` under `Integration - Common`.
|
||||
2. Move to `ap_site` when you need explicit site-level aggregation.
|
||||
3. Use `full` when you need full hierarchy/backhaul representation and have CPU headroom.
|
||||
4. Use `flat` only when hierarchy is not needed and maximum performance is the priority.
|
||||
@@ -39,8 +39,9 @@ sudo /opt/libreqos/src/bin/uisp_integration
|
||||
```
|
||||
2. Confirm outputs were generated/refreshed as expected:
|
||||
```shell
|
||||
ls -lh /opt/libreqos/src/network.json /opt/libreqos/src/ShapedDevices.csv
|
||||
ls -lh /opt/libreqos/src/topology_import.json /opt/libreqos/src/shaping_inputs.json
|
||||
```
|
||||
Built-in UISP imports refresh LibreQoS's imported topology and shaping data automatically. They do not use `network.json`, `ShapedDevices.csv`, or a standalone `circuit_anchors.json` as the normal working files for this integration.
|
||||
3. Confirm scheduler and shaper health:
|
||||
```shell
|
||||
sudo systemctl status lqosd lqos_scheduler
|
||||
@@ -48,9 +49,9 @@ journalctl -u lqos_scheduler --since "30 minutes ago"
|
||||
```
|
||||
4. Validate result in WebUI:
|
||||
- Scheduler Status is healthy
|
||||
- Tree/Flow views reflect expected hierarchy depth for selected strategy
|
||||
- Tree/Flow views reflect expected hierarchy depth for the selected compile mode
|
||||
|
||||
If hierarchy depth or parent mapping is not what you expect, revisit `strategy`, `use_ptmp_as_parent`, `exclude_sites`, and `exception_cpes` before changing other settings.
|
||||
If hierarchy depth or parent mapping is not what you expect, revisit `topology.compile_mode`, `use_ptmp_as_parent`, `exclude_sites`, and `exception_cpes` before changing other settings.
|
||||
|
||||
### Promote to Root Nodes (Performance Optimization)
|
||||
|
||||
@@ -113,6 +114,10 @@ Configure how LibreQoS handles suspended customer accounts:
|
||||
### Configuration Example
|
||||
|
||||
```ini
|
||||
[topology]
|
||||
# Shared topology compile mode (see table above)
|
||||
compile_mode = "ap_site" # Recommended starting point for new UISP deployments
|
||||
|
||||
[uisp_integration]
|
||||
# Core Settings
|
||||
enable_uisp = true
|
||||
@@ -120,17 +125,14 @@ token = "your-api-token-here"
|
||||
url = "https://uisp.your_domain.com"
|
||||
site = "Root_Site_Name" # Root site for topology perspective
|
||||
|
||||
# Topology Strategy (see table above)
|
||||
strategy = "ap_only" # Recommended starting point for new UISP deployments
|
||||
|
||||
# Suspension Handling (see table above)
|
||||
suspended_strategy = "none"
|
||||
|
||||
# Capacity Adjustments
|
||||
# UISP's reported AP capacities can be optimistic
|
||||
airmax_capacity = 0.8 # Use 80% of reported AirMax capacity on new installs
|
||||
# Current defaults normalize both AP capacity multipliers to 1.0
|
||||
airmax_capacity = 1.0 # Use 100% of reported AirMax capacity
|
||||
airmax_flexible_frame_download_ratio = 0.8 # Fallback split for AirMax flexible framing when UISP does not expose dlRatio
|
||||
ltu_capacity = 1.0 # Use 100% of reported LTU capacity on new installs
|
||||
ltu_capacity = 1.0 # Use 100% of reported LTU capacity
|
||||
infrastructure_transport_caps_enabled = true # Automatically cap radio capacity to active/model Ethernet transport ceilings
|
||||
|
||||
# Site Management
|
||||
@@ -143,7 +145,7 @@ commit_bandwidth_multiplier = 0.98 # Set minimum to 98% of maximum (CIR)
|
||||
|
||||
# Advanced Options
|
||||
ipv6_with_mikrotik = false # Enable if using DHCPv6 with MikroTik
|
||||
always_overwrite_network_json = true # Recommended when using UISP integration in production
|
||||
# `network.json` is for DIY/manual deployments; built-in integrations do not write it
|
||||
exception_cpes = [] # CPE exceptions in ["cpe:parent"] format
|
||||
squash_sites = [] # Optional: sites to squash
|
||||
do_not_squash_sites = [] # Optional: keep these site names unsquashed in the runtime/export tree
|
||||
@@ -184,32 +186,28 @@ Recommended use:
|
||||
|
||||
Legacy note:
|
||||
- Existing `enable_squashing` values in `/etc/lqos.conf` are ignored for backward compatibility.
|
||||
- Existing `uisp_integration.strategy` values are retained only as a compatibility mirror. Current builds read the shared `topology.compile_mode` first.
|
||||
|
||||
On the first successful run, the integration creates `network.json` and `ShapedDevices.csv`.
|
||||
If a `network.json` file exists, it is only overwritten when `always_overwrite_network_json = true`.
|
||||
On the first successful run, the integration creates the UISP import and shaping files LibreQoS needs for scheduled refreshes.
|
||||
|
||||
ShapedDevices.csv will be overwritten every time the UISP integration is run.
|
||||
|
||||
If UISP exposes a site and an AP with the same visible name in the same topology, current builds keep the site name stable in `network.json` and disambiguate the AP name during export so the site branch is not dropped from the tree.
|
||||
If UISP exposes a site and an AP with the same visible name in the same topology, current builds keep them distinct so the site branch is not dropped from runtime views.
|
||||
|
||||
When UISP client sites share the same name, LibreQoS now tries to disambiguate the generated circuit/site display names with a human-friendly suffix such as the first street-address segment, falling back to service name and then a short ID only when needed. Stable circuit identity still comes from the UISP site/service ID, not the display name.
|
||||
|
||||
For integration-driven deployments, keep `always_overwrite_network_json = true` so topology stays aligned with UISP on each refresh cycle.
|
||||
|
||||
You have the option to run `uisp_integration` automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_uisp = true``` in `/etc/lqos.conf`. Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
|
||||
### UISP Overrides
|
||||
|
||||
You can also use the following override inputs to more accurately reflect your network:
|
||||
- Tree-page `Rate Override` edits, stored as operator `AdjustSiteSpeed` entries in `lqos_overrides.json`
|
||||
- Tree-page `Topology Override` edits for supported UISP `full` nodes, stored in `lqos_overrides.json`
|
||||
- Tree-page `Rate Override` edits, stored as `AdjustSiteSpeed` entries in `lqos_overrides.json`
|
||||
- Tree-page `Topology Override` edits for supported UISP `full` nodes, also stored in `lqos_overrides.json`
|
||||
- integrationUISPbandwidths.csv as a legacy compatibility input only
|
||||
|
||||
Current UISP builds auto-migrate a legacy `integrationUISPbandwidths.csv` into operator `AdjustSiteSpeed` overrides on the next integration run when no operator rate overrides exist yet. If operator rate overrides already exist, the CSV is ignored and a warning is logged so there is only one active source of truth.
|
||||
Deprecated legacy `uisp.bandwidth_overrides` JSON entries are ignored. The authoritative bandwidth override path is operator `AdjustSiteSpeed` in `lqos_overrides.json`.
|
||||
Current UISP builds auto-migrate a legacy `integrationUISPbandwidths.csv` into `AdjustSiteSpeed` overrides on the next integration run when no newer rate overrides exist yet. If newer rate overrides already exist, the CSV is ignored and a warning is logged so only one override method stays active.
|
||||
Deprecated legacy `uisp.bandwidth_overrides` JSON entries are ignored. The supported long-term bandwidth override path is `AdjustSiteSpeed` in `lqos_overrides.json`.
|
||||
Current UISP builds ignore legacy `uisp.route_overrides` entries in `lqos_overrides.json` and legacy `integrationUISProutes.csv` files. If either is present, LibreQoS logs a warning and uses detected topology plus Topology Manager overrides instead.
|
||||
|
||||
UISP `full` strategy builds also expose tree-page `Topology Override` editing for supported nodes. These overrides are stored in `lqos_overrides.json` and resolve before final `network.json` / `ShapedDevices.csv` emission. Current WebUI support is `Pinned Parent` only.
|
||||
UISP `full` strategy builds also expose tree-page `Topology Override` editing for supported nodes. Current WebUI support is `Pinned Parent` only.
|
||||
|
||||
Each of the files above have templates available in the `/opt/libreqos/src` folder. If you don't find them there, you can navigate [here](https://github.com/LibreQoE/LibreQoS/tree/develop/src).
|
||||
|
||||
|
||||
@@ -18,9 +18,8 @@ timeout_secs = 20
|
||||
|
||||
Notes:
|
||||
- VISP import is GraphQL-based and currently defaults to a flat topology strategy.
|
||||
- The integration writes `ShapedDevices.csv` every run.
|
||||
- `network.json` is only overwritten when `always_overwrite_network_json = true` (under `[integration_common]`).
|
||||
- Recommended: keep `always_overwrite_network_json = true` for integration-driven deployments so topology stays aligned with VISP syncs.
|
||||
- The integration refreshes its shaping data every run.
|
||||
- `network.json` is for DIY/manual deployments; built-in integrations do not write it.
|
||||
- VISP auth tokens are cached in `<lqos_directory>/.visp_token_cache_*.json`.
|
||||
|
||||
Run a manual import with:
|
||||
|
||||
@@ -19,9 +19,9 @@ To test the WISPGate Integration, use
|
||||
python3 integrationWISPGate.py
|
||||
```
|
||||
|
||||
On the first successful run, it will create a ShapedDevices.csv file and network.json.
|
||||
ShapedDevices.csv will be overwritten every time the WISPGate integration is run.
|
||||
Recommended: keep `always_overwrite_network_json = true` for integration-driven deployments so topology stays aligned with WISPGate syncs.
|
||||
On the first successful run, it creates the WISPGate import and shaping data LibreQoS needs for scheduled refreshes.
|
||||
Those files are refreshed every time the WISPGate integration runs.
|
||||
Built-in integrations do not write `network.json`; keep that file for DIY/manual deployments.
|
||||
|
||||
You have the option to run integrationWISPGate.py automatically on boot and every X minutes (set by the parameter `queue_refresh_interval_mins`), which is highly recommended. This can be enabled by setting ```enable_wispgate = true``` in `/etc/lqos.conf`.
|
||||
Once set, run `sudo systemctl restart lqos_scheduler`.
|
||||
|
||||
+24
-13
@@ -2,9 +2,9 @@
|
||||
|
||||
## Page Purpose
|
||||
|
||||
Use this page to choose and configure supported CRM/NMS integrations. Use per-integration pages for specific setup details.
|
||||
Use this page to choose and configure a supported CRM/NMS integration. Use the per-integration pages for setup details.
|
||||
|
||||
Need definitions for overwrite/persistence terms? See the [Glossary](glossary.md).
|
||||
Need definitions for common terms? See the [Glossary](glossary.md).
|
||||
|
||||
Most operators use built-in integrations.
|
||||
If you have not selected a deployment path yet, start with [Quickstart](quickstart.md).
|
||||
@@ -12,17 +12,15 @@ If you have not selected a deployment path yet, start with [Quickstart](quicksta
|
||||
```{mermaid}
|
||||
flowchart LR
|
||||
A[CRM/NMS] --> B[LibreQoS Integration Job]
|
||||
B --> C[ShapedDevices.csv]
|
||||
B --> D[network.json]
|
||||
C --> E[lqos_scheduler]
|
||||
D --> E
|
||||
B --> H[topology_import.json]
|
||||
H --> E[lqos_scheduler]
|
||||
E --> F[Shaping Updates]
|
||||
E --> G[WebUI Status / Urgent Issues]
|
||||
```
|
||||
|
||||
## Choose Your Integration Path
|
||||
|
||||
| Path | Best fit | Primary source of truth |
|
||||
| Path | Best fit | Where permanent shaping changes belong |
|
||||
|---|---|---|
|
||||
| Built-in integration | Most operators using supported systems | CRM/NMS via LibreQoS integration jobs |
|
||||
| Custom source of truth | Operators with in-house CRM/NMS sync logic | External generated files (`network.json`, `ShapedDevices.csv`) |
|
||||
@@ -44,20 +42,33 @@ flowchart LR
|
||||
- [Powercode Integration](integrations-powercode.md)
|
||||
- [Sonar Integration](integrations-sonar.md)
|
||||
|
||||
## Important Overwrite Behavior
|
||||
## Important Refresh Behavior
|
||||
|
||||
When integrations are enabled:
|
||||
- `ShapedDevices.csv` is typically regenerated by integration sync jobs.
|
||||
- `network.json` may also be overwritten depending on integration settings (for example `always_overwrite_network_json`).
|
||||
- Integration sync refreshes the imported topology and shaping data LibreQoS uses.
|
||||
- Built-in integrations do not use `network.json` or `ShapedDevices.csv` as their normal output files.
|
||||
- Built-in integrations provide native infrastructure topology to Topology Manager; legacy compatibility trees remain derived/export-only data.
|
||||
- Python-backed integrations currently import infrastructure conservatively as fixed roots or fixed-parent branches. When bounded alternative-parent candidates are available, they are limited to the node's current local parent neighborhood, plus peer root parents of the same type when the current parent is itself a root, rather than the whole imported graph.
|
||||
- If you run a DIY or manual deployment, `network.json` and `ShapedDevices.csv` remain the files you maintain.
|
||||
- Direct edits to integration-managed data may be replaced on the next refresh cycle.
|
||||
|
||||
Mode capabilities are now explicit by integration:
|
||||
- UISP: `flat`, `ap_only`, `ap_site`, `full`
|
||||
- Splynx: `flat`, `ap_only`, `ap_site`, `full`
|
||||
- Sonar: `flat`, `full`
|
||||
- Manual edits may be overwritten on the next refresh cycle.
|
||||
|
||||
Legacy `integrationUISPbandwidths.csv` and `integrationSplynxBandwidths.csv` files are auto-migrated into operator `AdjustSiteSpeed` entries in `lqos_overrides.json` and renamed to `.backup` files.
|
||||
|
||||
DIY/manual deployments continue to use `network.json` and `ShapedDevices.csv` as their primary input files.
|
||||
|
||||
## Topology Node ID Support
|
||||
|
||||
LibreQoS supports an optional generic `"id"` field on `network.json` nodes. This field is intended to carry stable node identifiers from the integration source where possible.
|
||||
|
||||
Current behavior:
|
||||
- node IDs are the preferred match key for operator-owned site bandwidth overrides and tree-page node override editing when an ID is present
|
||||
- legacy name-only matching is still supported as a fallback for older data
|
||||
- when a node ID is available, LibreQoS uses it to match saved site bandwidth overrides more reliably
|
||||
- older name-only matching still works as a fallback for older data
|
||||
- topology names still need to remain globally unique in `network.json`
|
||||
|
||||
| Integration | `network.json` node ID support | Notes |
|
||||
@@ -65,7 +76,7 @@ Current behavior:
|
||||
| UISP | Yes | Real UISP sites/devices export generic `id` plus existing `uisp_site` / `uisp_device` metadata. Synthetic LibreQoS nodes use stable generated IDs. |
|
||||
| Splynx | Yes | Network sites and AP/site topology nodes export generic `id`. |
|
||||
| Sonar | Yes | Site and AP topology nodes export generic `id`. |
|
||||
| Netzur | Partial | Exported only when the upstream zone payload includes a stable zone ID. |
|
||||
| Netzur | Partial | Exported only when the upstream zone data includes a stable zone ID. |
|
||||
| VISP | No | Current importer shapes services/devices but does not build topology nodes in `network.json`. |
|
||||
| Powercode | No | Current importer does not build topology nodes in `network.json`. |
|
||||
| WISPGate | No | Current importer does not build topology nodes from stable upstream topology identifiers. |
|
||||
|
||||
@@ -28,16 +28,18 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- `Node Details` summarizes the selected node’s configured rates, override state, and effective rate.
|
||||
- `Node Snapshot` provides a quick visual summary of throughput and QoO for the selected node.
|
||||
- Attached circuits are shown in a dedicated table for the selected node.
|
||||
- The attached-circuits table is attachment-aware, not traffic-aware: idle circuits remain visible even when they have no current throughput, with live columns left empty or zeroed until traffic appears.
|
||||
- The attached-circuits IP column keeps rows compact by showing one address inline and collapsing additional addresses as `+X`, with the full list still available on hover.
|
||||
- Ethernet-limited attached circuits can show inline `10M`, `100M`, or `1G` warning badges beside the `Plan (Mbps)` value; hovering explains the auto-cap and clicking the badge opens the dedicated Ethernet review page.
|
||||
- Administrators can save or clear `Rate Override` values where node-level overrides are supported. Read-only users and unsupported nodes continue to display current values without edit controls.
|
||||
- Tree-page operator rate edits write operator-owned `AdjustSiteSpeed` overrides to `lqos_overrides.json`.
|
||||
- Tree-page rate edits save persistent `AdjustSiteSpeed` overrides in `lqos_overrides.json`.
|
||||
- On UISP `full` builds, legacy `integrationUISPbandwidths.csv` files are auto-migrated into those operator overrides on the next integration run when no operator rate overrides exist yet; otherwise the CSV is ignored.
|
||||
- Tree-page operator rate edits require an administrator session, a stable node ID, and a non-generated node. Generated/integration-synthetic nodes remain read-only in this editor.
|
||||
- The tree page keeps `Node Details` as a compact summary card and places the override editors as compact inline rows directly beneath the details table.
|
||||
- The tree page now shows topology override state as read-only summary only. Parent and attachment edits have moved to `Topology`, and the tree details panel links directly into Topology Manager for the selected node.
|
||||
- Runtime-selected attachment hops are collapsed out of the main tree hierarchy. When a site is currently using a specific backhaul or radio path, `Node Details` shows that as `Active Attachment` metadata on the site instead of exposing the attachment as its own branch node.
|
||||
- Inactive alternate PtP/wired backhaul attachment stubs are also pruned from the runtime tree. `tree.html` shows the effective path only; dormant alternates remain visible in Topology Manager rather than as standalone runtime nodes.
|
||||
- For integration-backed topology, `Network Tree Overview`, Tree Overview sankey views, and HTB queue planning all follow the squashed runtime-effective tree from `network.effective.json`. Topology Manager intentionally keeps the fuller canonical/editor topology so transport hops such as backhauls can still be managed there.
|
||||
- On the synthetic `Root` node, topology override still shows as not applicable rather than as a generic missing-node-ID warning.
|
||||
|
||||
### Topology Manager
|
||||
@@ -57,12 +59,11 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- Parent changes are gated behind an explicit `Start Move` step in the Details panel. In move mode, legal parent targets are highlighted in green and can be chosen from the target cards or by dragging the selected branch onto a highlighted target.
|
||||
- For UISP-backed sites, direct inter-site radio links can also surface sibling or alternate upstream sites as legal parent targets when the linked remote radio reaches the root without traversing the local site first.
|
||||
- Exported shaping/runtime trees anchor those cross-site moves under the parent-side peer attachment so the moved site stays visible after regeneration.
|
||||
- Canonical integration snapshots stay integration-derived; saved Topology Manager re-parents are applied only in the runtime-effective topology layer, which now refuses to publish an effective tree that drops or duplicates a site.
|
||||
- Saving a Topology Manager move, probe-policy change, manual attachment group, or attachment-rate override now triggers an immediate runtime-effective recompute and publish; routine topology edits do not need a fresh UISP import to take effect.
|
||||
- When a move or attachment-preference save is in flight, the primary `Save` button switches to a short spinner/disabled state so operators can see that the runtime-effective publish is still underway.
|
||||
- Effective publish validation is ID-based and fail-closed. Before validation, the runtime-effective compiler normalizes duplicate canonical attachment/device candidates by stable `node_id` so UISP duplicate radio rows do not block otherwise valid site moves. If a saved change would still select an invalid parent or attachment, create a cycle, duplicate a site, or drop a site from the exported tree, LibreQoS rejects the publish and keeps the last good effective topology in place.
|
||||
- Canonical and effective topology snapshots are written with atomic file replacement so normal refreshes do not rely on partially written JSON artifacts.
|
||||
- The Hierarchy Summary now shows three distinct states: `Canonical` for the latest integration snapshot, `Live` for the runtime-effective topology currently used for shaping, and `Saved`/`Proposed` for operator intent. When a saved move is already active in the runtime-effective topology, the status chip reads `Live Override`; otherwise it remains `Saved Override`.
|
||||
- Saved Topology Manager moves and attachment preferences are applied on top of the latest imported topology, so you do not need to wait for another UISP sync before seeing the result.
|
||||
- Saving a move, probe policy, manual attachment group, or attachment-rate override triggers an immediate refresh of the live topology used for shaping.
|
||||
- While a save is being applied, the main `Save` button briefly shows a spinner and stays disabled.
|
||||
- LibreQoS validates a move before making it active. If the change would create an invalid path, create a loop, duplicate a site, or remove a site from the tree, the save is rejected and the previous working topology stays in place.
|
||||
- The Hierarchy Summary helps compare the latest imported topology, the currently active topology, and any saved override for the selected branch.
|
||||
- Main operator-facing hierarchy chips now prefer display names only. If a label cannot be resolved, the page shows a neutral placeholder instead of a raw internal node ID.
|
||||
- Attachment or radio preference edits for the current logical parent do not require move mode; when multiple explicit attachments are available, inspect mode exposes a lighter `Edit Attachment Preference` path in Details.
|
||||
- Attachment rows now surface runtime health directly in the page, including `Healthy`, `Suppressed`, `Probe Unavailable`, or `Probe Disabled` state, plus reason text, suppression hold-down timestamps, and per-pair probe enable/disable actions.
|
||||
@@ -78,7 +79,7 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- When UISP exposes multiple parallel links between the same two sites, Topology Manager groups them under one logical parent target and exposes the concrete radios/devices as explicit attachment preferences.
|
||||
- `tree.html` only squashes runtime-selected UISP attachments when the effective attachment role is a true backhaul-style path (`PtP Backhaul` or `Wired Uplink`). PtMP access/uplink APs remain visible in the runtime tree.
|
||||
- When a chosen parent exposes only one valid explicit attachment option, the UI auto-saves that override immediately.
|
||||
- Saved topology overrides are stored as operator-owned intent in `topology_overrides.json`, separate from integration-generated runtime editor state.
|
||||
- Saved topology overrides are stored in `topology_overrides.json`.
|
||||
|
||||
### Topology Probes
|
||||
- Read-only debug page for system-wide topology probe troubleshooting, linked from Topology Manager rather than promoted as a primary sidebar destination.
|
||||
@@ -95,12 +96,12 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- APs without explicit coordinates are represented through their parent site and can be expanded temporarily around the selected site for inspection.
|
||||
- When a site or AP is selected and the right-hand details panel opens, the selected node name links directly to that node's `tree.html` page when a stable node ID is available.
|
||||
- Visible unclustered sites show labels as the operator zooms in, and the selected site keeps its label visible while it is being inspected.
|
||||
- When browser redaction mode is enabled, Site Map replaces displayed site names with `[redacted]` while leaving the underlying topology data unchanged.
|
||||
- When redaction mode is enabled, Site Map replaces displayed site names with `[redacted]`.
|
||||
- Initial map framing prefers site coordinates for the first view, falling back to AP coordinates when no sites are mapped yet.
|
||||
- Site Map uses an Insight-hosted OpenStreetMap raster tile cache.
|
||||
- In dark mode, the raster underlay is muted and tinted client-side toward the same cooler blue/cyan palette used by Flow Globe, so roads and geography stay visible without the bright light-theme basemap glare.
|
||||
- In dark mode, the map is muted toward the same cooler blue/cyan palette used by Flow Globe, so roads and geography stay visible without the bright light-theme basemap glare.
|
||||
- Site Map depends on outbound access to `https://insight.libreqos.com` for initial bbox/bootstrap and raster tile fetches.
|
||||
- When tiles are missing from the remote cache, the browser retries automatically for a short period instead of failing immediately, so initial map paint can lag briefly on cold tiles.
|
||||
- When tiles are missing from the remote cache, the page retries automatically for a short period instead of failing immediately, so initial map paint can lag briefly on cold tiles.
|
||||
|
||||
### Flow Globe
|
||||
- Geographic flow visualization based on endpoint geolocation.
|
||||
@@ -124,6 +125,7 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- `Devices` shows per-device detail tables and live charts for throughput, retransmits, and latency.
|
||||
- `Queue Stats` shows recent live queue history for the circuit, including backlog, delay, queue length, traffic, ECN marks, and drops.
|
||||
- Queue Stats charts use synchronized hover so operators can inspect the same second across all queue charts together.
|
||||
- `Queue Stats` should continue to show the circuit's live queue telemetry in `flat` mode as well as other topology modes.
|
||||
- `Queue Tree` shows the circuit's live upstream queue path, including a path summary and per-node throughput, retransmit, and latency context.
|
||||
- `Traffic Flows` is a recent-flow operational table rather than a long-term history view.
|
||||
- `Traffic Flows` includes paging and a `Hide Small Flows` filter so large busy circuits remain usable without trying to render every row at once.
|
||||
@@ -146,8 +148,13 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
- Supports paging and filtering.
|
||||
- Add, edit, and delete actions save immediately in the dedicated editor.
|
||||
|
||||
### Configuration Pages
|
||||
- Saved passwords, API tokens, and keys stay hidden when you open an integration page.
|
||||
- If a credential is already configured, the field stays blank until you type a replacement. Use `Clear` only when you want to remove the saved value.
|
||||
- Leave the field blank to keep the current value. Enter a new value to replace it.
|
||||
|
||||
### Urgent Issues
|
||||
- WebUI can display urgent operational issues generated by backend services.
|
||||
- WebUI can display urgent operational issues reported by LibreQoS services.
|
||||
- Examples include mapping/license-limit warnings and other high-priority health events.
|
||||
- Operators can acknowledge/clear issues in the UI after review.
|
||||
- Common codes include `MAPPED_CIRCUIT_LIMIT` and `TC_U16_OVERFLOW` (see [Troubleshooting](troubleshooting.md#urgent-issue-codes-and-first-actions)).
|
||||
@@ -164,10 +171,10 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
|
||||
## Privacy / Redaction Mode
|
||||
|
||||
- Toggle with the mask icon in the top navigation.
|
||||
- Redaction is client-side and stored in browser local storage.
|
||||
- Redaction is a display-only setting saved in your browser.
|
||||
- Redaction helps hide PII in screenshots/demos.
|
||||
- Site Map replaces displayed site names with `[redacted]` while redaction mode is enabled.
|
||||
- Redaction does not modify `ShapedDevices.csv`, `network.json`, or any backend data.
|
||||
- Redaction does not change your saved data or configuration files.
|
||||
|
||||
## Common Empty-State Behavior
|
||||
|
||||
|
||||
@@ -2,46 +2,56 @@
|
||||
|
||||
## Page Purpose
|
||||
|
||||
Use this page to choose and enforce your source of truth policy (built-in integrations vs custom source of truth) before production changes.
|
||||
Use this page to decide where permanent shaping changes should live before you go to production.
|
||||
|
||||
Need definitions for terms on this page? See the [Glossary](glossary.md).
|
||||
|
||||
```{mermaid}
|
||||
flowchart TD
|
||||
A[Choose Source of Truth] --> B{Using built-in CRM/NMS integration?}
|
||||
A[Choose Where Changes Live] --> B{Using built-in CRM/NMS integration?}
|
||||
B -->|Yes| C[Built-in Integrations Mode]
|
||||
B -->|No| D{Generating files from your own scripts/systems?}
|
||||
D -->|Yes| E[Custom Source of Truth Mode]
|
||||
D -->|No| F[Manual Files Mode]
|
||||
C --> G[Integration sync jobs own file refresh]
|
||||
E --> H[External script pipeline owns persistence]
|
||||
F --> I[Direct file maintenance; best for small networks]
|
||||
C --> G[Integration sync keeps shaping data current]
|
||||
E --> H[Your scripts refresh the shaping files]
|
||||
F --> I[You maintain the files directly]
|
||||
```
|
||||
|
||||
LibreQoS supports two first-class operating modes.
|
||||
LibreQoS supports three common operating modes.
|
||||
|
||||
## Built-in Integrations (Recommended for Most Operators)
|
||||
|
||||
In this mode, your CRM/NMS is the source of truth and LibreQoS integration jobs produce shaping inputs.
|
||||
In this mode, your CRM/NMS is where permanent subscriber and topology changes belong, and LibreQoS keeps itself in sync from that system.
|
||||
|
||||
Key behavior:
|
||||
- `ShapedDevices.csv` is regenerated by integration sync cycles.
|
||||
- `network.json` overwrite behavior depends on integration settings (for example `always_overwrite_network_json`).
|
||||
- Direct manual edits may be overwritten on the next scheduler refresh.
|
||||
- Integration sync refreshes the topology and shaping data LibreQoS uses.
|
||||
- Built-in integrations do not use `network.json` as their normal working file.
|
||||
- Direct file edits may be overwritten on the next scheduler refresh.
|
||||
- `flat` mode intentionally simplifies the tree when you want lower overhead.
|
||||
|
||||
## Custom Source of Truth
|
||||
|
||||
In this mode, your own scripts/systems generate `network.json` and `ShapedDevices.csv`.
|
||||
In this mode, your own scripts or systems generate `network.json` and `ShapedDevices.csv`.
|
||||
|
||||
Key behavior:
|
||||
- Your external workflow owns persistence.
|
||||
- WebUI edits are valid for quick operational changes.
|
||||
- Permanent changes should be made in your external source of truth workflow.
|
||||
- Your external workflow is where permanent changes belong.
|
||||
- WebUI edits are fine for quick operational changes.
|
||||
- Keep your long-term changes in your own scripts or automation.
|
||||
|
||||
## Manual Files Mode
|
||||
|
||||
In this mode, you maintain `network.json` and `ShapedDevices.csv` yourself.
|
||||
|
||||
Key behavior:
|
||||
- Best fit for small networks, short pilots, or temporary workarounds.
|
||||
- WebUI helps you validate what LibreQoS is using.
|
||||
- Manual discipline matters because there is no upstream system keeping those files in sync for you.
|
||||
|
||||
## Mode Declaration Checklist (Before Go-Live)
|
||||
|
||||
1. Pick one primary source of truth.
|
||||
2. Confirm which system can write production shaping inputs.
|
||||
1. Pick one primary place for permanent shaping changes.
|
||||
2. Confirm which system writes production shaping data.
|
||||
3. Confirm scheduler refresh behavior and overwrite cadence.
|
||||
4. Document your hotfix workflow (WebUI, external editor, or both).
|
||||
5. Do not maintain competing edits in multiple systems for the same objects.
|
||||
@@ -49,8 +59,8 @@ Key behavior:
|
||||
## Topology and Mode Expectations
|
||||
|
||||
- Single-interface (on-a-stick) and VLAN-heavy designs are valid, but require explicit queue/interface planning and careful validation after changes.
|
||||
- Integration mode is best when you want CRM/NMS-driven topology and subscriber lifecycle data to own shaping inputs.
|
||||
- If you need strict custom topology behavior not represented by integration output, use custom source of truth mode and keep ownership explicit.
|
||||
- Integration mode is best when you want CRM/NMS-driven topology and subscriber lifecycle data to control shaping.
|
||||
- If you need topology behavior that your integration cannot represent, use custom source of truth mode and keep the responsibility clear.
|
||||
|
||||
See:
|
||||
- [Advanced Configuration Reference](configuration-advanced.md)
|
||||
|
||||
+14
-18
@@ -8,8 +8,8 @@ Use this page in order:
|
||||
1. Complete install and bridge setup.
|
||||
2. Complete first-run WebUI access.
|
||||
3. Pass the 10-minute health check.
|
||||
4. Pick one source of truth owner.
|
||||
5. Follow the recommended path for that owner.
|
||||
4. Choose one system to manage your shaping data.
|
||||
5. Follow the recommended path for that system.
|
||||
|
||||
Need definitions for key terms? See the [Glossary](glossary.md).
|
||||
|
||||
@@ -89,23 +89,23 @@ Confirm:
|
||||
|
||||
If this fails, go to [Troubleshooting](troubleshooting.md) before proceeding.
|
||||
|
||||
## 3) Pick One Source of Truth Owner
|
||||
## 3) Choose One System To Manage Your Shaping Data
|
||||
|
||||
This is the most important early decision.
|
||||
|
||||
Source of truth means: the one place that should own persistent shaping data.
|
||||
On this page, "source of truth" just means the place where permanent shaping changes should be made.
|
||||
|
||||
If you make persistent changes somewhere else, they may be overwritten later.
|
||||
|
||||
Choose one:
|
||||
|
||||
| If this describes you | Mode | Owner of durable shaping data |
|
||||
| If this describes you | Mode | Where permanent shaping changes belong |
|
||||
|---|---|---|
|
||||
| You use a supported LibreQoS CRM/NMS integration | Built-In Integration Mode | Integration jobs |
|
||||
| You generate `network.json` and `ShapedDevices.csv` with your own automation | Custom Source of Truth Mode | Your scripts |
|
||||
| You intentionally maintain files by hand | Manual Files Mode | Manual edits |
|
||||
|
||||
Rule: keep one owner for persistent shaping inputs.
|
||||
Rule: pick one place for permanent shaping changes.
|
||||
|
||||
Most operators should stop here and choose **Built-In Integration Mode**.
|
||||
|
||||
@@ -121,18 +121,14 @@ When to choose:
|
||||
Do this now:
|
||||
1. Configure integration settings in WebUI.
|
||||
2. Run the initial sync.
|
||||
3. Confirm outputs were created or refreshed:
|
||||
- `network.json`
|
||||
- `ShapedDevices.csv`
|
||||
4. Confirm Scheduler Status is healthy.
|
||||
5. Confirm Network Tree reflects the expected hierarchy depth for your chosen integration strategy.
|
||||
6. Confirm there are no urgent/fatal issues that block shaping.
|
||||
7. Only then move to limited pilot traffic or inline use.
|
||||
3. Confirm Scheduler Status is healthy.
|
||||
4. Confirm Network Tree reflects the expected hierarchy depth for your chosen integration strategy.
|
||||
5. Confirm there are no urgent or fatal issues that block shaping.
|
||||
6. Only then move to limited pilot traffic or inline use.
|
||||
|
||||
Do not do this:
|
||||
- Do not hand-edit integration-owned files as your normal workflow.
|
||||
- Do not split ownership between integration syncs and manual file edits unless you intentionally want that complexity.
|
||||
- Do not change overwrite behavior unless you understand how your integration should own `network.json`.
|
||||
- Do not hand-edit files that your integration refreshes as part of your normal workflow.
|
||||
- Do not mix manual file edits with scheduled integration syncs unless you intentionally want that extra complexity.
|
||||
|
||||
Next:
|
||||
- [CRM/NMS Integrations](integrations.md)
|
||||
@@ -147,7 +143,7 @@ When to choose:
|
||||
|
||||
Do this now:
|
||||
1. Implement script/process to generate and refresh shaping files.
|
||||
2. Declare script outputs as your source of truth.
|
||||
2. Treat your script outputs as the place for permanent shaping changes.
|
||||
3. Place LibreQoS inline for pilot traffic.
|
||||
4. Use WebUI for operational checks and short-term adjustments.
|
||||
5. Keep permanent changes in your external script workflow.
|
||||
@@ -182,7 +178,7 @@ Next:
|
||||
|
||||
## 6) Common First-Run Mistakes
|
||||
|
||||
- Unclear source of truth ownership between integration and manual edits.
|
||||
- Not deciding where permanent shaping changes should be made.
|
||||
- Hand-editing files even though your integration is supposed to own them.
|
||||
- Changing topology depth before passing the health gate.
|
||||
- Skipping post-change service/log validation before pilot traffic.
|
||||
|
||||
@@ -16,7 +16,7 @@ Use this pattern when your operator workflow is already centered on a supported
|
||||
## Implementation
|
||||
|
||||
1. Configure integration credentials/settings in WebUI (`Configuration -> Integrations`).
|
||||
2. For integration-driven deployments, keep `always_overwrite_network_json = true`.
|
||||
2. For integration-driven deployments, treat `network.json` as DIY/manual-only and let the integration publish `topology_import.json` instead.
|
||||
3. Choose the lightest topology strategy that meets requirements.
|
||||
|
||||
| Requirement | Suggested strategy |
|
||||
|
||||
@@ -102,7 +102,7 @@ When enabled and not dry-run, TreeGuard may persist circuit SQM decisions to:
|
||||
|
||||
- `lqos_overrides.treeguard.json`
|
||||
|
||||
TreeGuard is designed to avoid fighting operator-owned overrides. If operator overrides exist for enrolled entities, TreeGuard skips those entities and reports warnings.
|
||||
TreeGuard is designed to avoid fighting manually saved overrides. If those overrides exist for enrolled entities, TreeGuard skips those entities and reports warnings.
|
||||
|
||||
TreeGuard virtual-node decisions are runtime-only Bakery operations. They are not materialized back
|
||||
into the base `network.json` by the scheduler, and they are not persisted as TreeGuard-owned
|
||||
|
||||
@@ -37,6 +37,12 @@ journalctl -u lqos_scheduler --since "30 minutes ago"
|
||||
|
||||
If integration-related, also include:
|
||||
|
||||
```bash
|
||||
ls -lh /opt/libreqos/src/topology_import.json /opt/libreqos/src/shaping_inputs.json
|
||||
```
|
||||
|
||||
If you run a manual or custom-file deployment instead of a built-in integration, include:
|
||||
|
||||
```bash
|
||||
ls -lh /opt/libreqos/src/network.json /opt/libreqos/src/ShapedDevices.csv
|
||||
```
|
||||
@@ -150,9 +156,7 @@ Routine package upgrades now invoke `lqos_setup --skip-if-ready` during `postins
|
||||
|
||||
If startup shaping fails because `shaping_inputs.json` is missing or stale, current builds leave the scheduler running in a degraded state and wait for the next scheduled full refresh to recover. The high-frequency topology refresh tick stays disabled until one shaping pass completes successfully, so repeated 3-second refresh attempts should not continue hammering a fresh install that has not produced runtime topology inputs yet.
|
||||
|
||||
Current builds no longer gate startup shaping on file mtimes. Instead, `lqos_topology` publishes `/opt/libreqos/src/topology_runtime_status.json` with a `source_generation` derived from the current shaping inputs, and `lqos_scheduler` waits for `ready: true` for that exact generation before calling `refreshShapers()`.
|
||||
|
||||
If scheduler startup is degraded with a message about topology runtime still building, that does not necessarily mean the scheduler service died. It means the current generation of `network.json`, `ShapedDevices.csv`, `circuit_anchors.json`, and active topology state has not finished producing committed runtime outputs yet. Once topology finishes and the status file flips to `ready: true` for the same generation, scheduled shaping refreshes recover automatically without restarting the service.
|
||||
If scheduler startup is degraded with a message about topology runtime still building, that usually means LibreQoS is still finishing an import or refreshing shaping data. Give the current cycle a little time to complete, then recheck Scheduler Status before changing unrelated settings.
|
||||
|
||||
If scheduler startup is degraded with a message that topology runtime failed for the current generation, inspect:
|
||||
|
||||
@@ -162,15 +166,15 @@ ls -lh /opt/libreqos/src/topology_effective_state.json /opt/libreqos/src/network
|
||||
journalctl -u lqos_scheduler --since "30 minutes ago"
|
||||
```
|
||||
|
||||
If Insight shows sites buried under AP/backhaul chains or only a narrow subset of sites loads cleanly, check whether the host is on a build before the separate Insight logical-topology export. Current builds submit an Insight-only logical-parent tree derived from canonical topology state, while `compatibility_network_json` remains reserved for local compatibility and `network.effective.json` generation.
|
||||
If Topology Manager changes or imports seem stuck on older data, check whether LibreQoS set older snapshots aside under `/opt/libreqos/src/.topology_stale/`, then review recent scheduler logs before retrying.
|
||||
|
||||
To inspect the exact topology JSON that `lqosd` is preparing for Insight, review:
|
||||
If Insight topology looks wrong, review the current troubleshooting snapshot that `lqosd` is preparing for Insight:
|
||||
|
||||
```bash
|
||||
cat /opt/libreqos/src/network.insight.debug.json
|
||||
```
|
||||
|
||||
This debug snapshot is separate from `network.insight.json`. The latter is an installed-runtime input path used by shaping components when Insight topology mode is enabled, while `network.insight.debug.json` is only a diagnostic snapshot of the payload source tree.
|
||||
Treat `network.insight.debug.json` as a troubleshooting snapshot only; do not edit it.
|
||||
|
||||
If specific APs or switches appear multiple times with suffixed names such as `... [AP deadbeef]`, check whether UISP is returning duplicate rows for the same device ID. Current builds defensively deduplicate raw UISP devices by `identification.id` before topology graph construction, and skip any residual duplicate device IDs during graph assembly.
|
||||
|
||||
@@ -180,6 +184,7 @@ If an integration subprocess fails, current builds keep the scheduler alive, pub
|
||||
|
||||
Recent builds expose scheduler readiness/state in the WebUI (Node Manager).
|
||||
If the scheduler is still starting, the sidebar now reports the current startup phase and a coarse progress ring rather than only a spinner.
|
||||
Current builds also treat scheduler progress, output, and error bus messages as proof that the scheduler is alive, so the sidebar should not stay stuck on `Scheduler available: false` while the scheduler is actively reporting work.
|
||||
|
||||
If scheduler status appears down/stale:
|
||||
1. Verify both services:
|
||||
@@ -249,7 +254,7 @@ Site Map has one extra dependency beyond normal WebUI data feeds: current builds
|
||||
If Site Map alone is blank or slow:
|
||||
1. Confirm `lqosd` is healthy.
|
||||
2. Confirm the box can reach `insight.libreqos.com` from its management network.
|
||||
3. Wait briefly and reload the page; the browser retries tile requests automatically while cold tiles are being populated upstream.
|
||||
3. Wait briefly and reload the page; the map page retries tile requests automatically while cold tiles are being populated upstream.
|
||||
4. Check recent `lqosd` logs:
|
||||
```
|
||||
journalctl -u lqosd --since "10 minutes ago"
|
||||
|
||||
Reference in new issue
Block a user