changes

docs/v2.0/components.md

@@ -33,6 +33,7 @@ flowchart LR
 - lqos_scheduler performs continuous refreshes of LibreQoS' shapers, including pulling from any enabled CRM Integrations (UISP, Splynx, Netzur).
 - Actions:
   - On start: Run a full setup of queues
+    - Current builds wait briefly for `lqosd` to finish binding the local bus before the first scheduler run.
   - Every X minutes: Update queues, pulling new configuration from CRM integration, if enabled.
     - The default minute interval is 30, so the refresh occurs every 30 minutes by default.
     - The minute interval is adjustable with the setting `queue_refresh_interval_mins` in `/etc/lqos.conf`.
@@ -84,6 +85,8 @@ To confirm that lqos_scheduler (scheduler.py) is able to work correctly, run:
 sudo python3 scheduler.py
 ```
 
+If an integration fails during a scheduler run, current builds keep the scheduler alive, surface a shortened output preview in scheduler status/error reporting, and save the full captured output to a timestamped `/tmp/lqos_scheduler_<integration>_*.log` file.
+
 Once you have any errors eliminated, restart lqos_scheduler with
 
 ```shell

docs/v2.0/configuration-advanced.md

@@ -125,13 +125,16 @@ How overrides apply:
 - `lqos_scheduler` applies overrides during refresh cycles.
 - persistent devices are merged into `ShapedDevices.csv`.
 - circuit/device/network adjustments are applied on top of imported/manual data.
+- operator-owned site bandwidth overrides prefer `node_id` when present and fall back to legacy name-only matching.
+- tree-page `Operator Override` writes to the operator override layer in `lqos_overrides.json`, not to legacy integration bandwidth CSV files.
+- automated runtime layers such as StormGuard and TreeGuard remain separate from the operator layer and are not written back into operator-authored source files.
 
 ### Network Hierarchy
 #### Network.json
 
 Network.json allows ISP operators to define a Hierarchical Network Topology, or Flat Network Topology.
 
-Each topology node may optionally include an `"id"` field. This is intended to carry a stable node identifier from the source CRM/NMS when one exists. LibreQoS currently still matches hierarchy and overrides by node name, not by this ID, so `"id"` is metadata for now.
+Each topology node may optionally include an `"id"` field. This is intended to carry a stable node identifier from the source CRM/NMS when one exists. Current builds prefer this ID when matching operator-owned site bandwidth overrides, while still supporting legacy name-only matching as a fallback.
 
 Recommended format:
 
@@ -151,6 +154,15 @@ Notes:
 - Generated LibreQoS-only nodes may use stable generated IDs such as `libreqos:generated:uisp:site:orphans`.
 - Existing integration-specific metadata fields such as `uisp_site` and `uisp_device` may also appear alongside the generic `id` field.
 
+#### Queue mode (`shape` / `observe`)
+
+LibreQoS currently uses `queue_mode` in the `[queues]` section to control whether the shaping tree is active:
+
+- `queue_mode = "shape"`: normal shaping mode
+- `queue_mode = "observe"`: remove the subscriber shaping tree for a true baseline while keeping the root MQ in place
+
+The older `monitor_only` setting is retained as a compatibility alias in some configs and serialized output, but `queue_mode` is the current operator-facing setting and documentation term.
+
 If you plan to use the built-in UISP, Splynx, or Netzur integrations, you do not need to create a network.json file quite yet.
 If you plan to use the built-in UISP integration, it will create this automatically on its first run (assuming network.json is not already present).
 

docs/v2.0/configuration.md

@@ -16,6 +16,8 @@ Notes:
 
 After install, sign in to WebUI at `http://your_shaper_ip:9123`.
 
+If no WebUI users exist yet, current builds redirect to first-run setup automatically.
+
 For most operators:
 1. Choose your operating mode: [Operating Modes and Source of Truth](operating-modes.md)
 2. Configure integration settings in WebUI: [CRM/NMS Integrations](integrations.md)
@@ -37,6 +39,10 @@ When an integration is enabled and acting as the source of truth, the `Network L
 
 Topology note:
 - `network.json` node names must be globally unique across the whole tree. Duplicate node names now fail validation and are not accepted by the WebUI save path or `LibreQoS.py`.
+- When a node exposes a stable `id`, current builds prefer that ID for operator-owned site bandwidth overrides while keeping legacy name-only matching as a fallback.
+
+Queue-mode note:
+- Current builds use `queue_mode` with `shape` and `observe` values. Older `monitor_only` wording is a compatibility alias rather than the primary operator-facing setting.
 
 ## Operating Modes and Source of Truth
 

docs/v2.0/integrations-sonar.md

@@ -2,6 +2,12 @@
 
 First, set the relevant parameters for Sonar (sonar_api_key, sonar_api_url, etc.) in `/etc/lqos.conf`.
 
+Current behavior notes:
+- `sonar_api_url` may be either the Sonar base URL or the full GraphQL endpoint. LibreQoS normalizes it to `/api/graphql` automatically.
+- Current builds page through Sonar GraphQL results instead of relying on a small first-page sample.
+- Emitted Sonar identities are namespaced (for example `sonar:account:<id>` and `sonar:device:<id>`) so they remain stable across overrides and downstream tooling.
+- If Sonar returns non-JSON content or GraphQL errors, the integration now raises a more specific error message showing the endpoint and a short response preview.
+
 To test the Sonar Integration, use
 
 ```shell

docs/v2.0/integrations-splynx.md

@@ -130,12 +130,16 @@ If sync looks successful but data is incomplete, recheck API permissions and str
 
 You can also modify the the file `integrationSplynxBandwidths.csv` to override the default bandwidths for each Node (Site, AP).
 
+Tree-page `Operator Override` edits are separate from this legacy Splynx file. Current builds write those operator-owned node rate changes to `lqos_overrides.json` and do not rewrite `integrationSplynxBandwidths.csv`.
+
 A template is available in the `/opt/libreqos/src` folder. To utilize the template, copy the file `integrationSplynxBandwidths.template.csv` (removing the `.template` part of the filename) and set the appropriate information inside each file. For example, if you want to change the set bandwidth for a site, you would do:
 ```
 sudo cp /opt/libreqos/src/integrationSplynxBandwidths.template.csv /opt/libreqos/src/integrationSplynxBandwidths.csv
 ```
 And edit the CSV using LibreOffice or your preferred CSV editor.
 
+To avoid conflicting sources of truth, prefer one durable override path per node: either the legacy Splynx CSV workflow or the operator override layer.
+
 
 ## Related Pages
 

docs/v2.0/integrations-uisp.md

@@ -183,6 +183,8 @@ You can also modify the following files to more accurately reflect your network:
 - integrationUISPbandwidths.csv
 - integrationUISProutes.csv
 
+Tree-page `Operator Override` edits are separate from these legacy UISP files. Current builds write those operator-owned node rate changes to `lqos_overrides.json` and do not rewrite `integrationUISPbandwidths.csv`.
+
 Each of the files above have templates available in the `/opt/libreqos/src` folder. If you don't find them there, you can navigate [here](https://github.com/LibreQoE/LibreQoS/tree/develop/src). To utilize the template, copy the file (removing the `.template` part of the filename) and set the appropriate information inside each file.
 For example, if you want to change the set bandwidth for a site, you would do:
 ```
@@ -190,6 +192,8 @@ sudo cp /opt/libreqos/src/integrationUISPbandwidths.template.csv /opt/libreqos/s
 ```
 And edit the CSV using LibreOffice or your preferred CSV editor.
 
+To avoid conflicting sources of truth, prefer one durable override path per node: either the legacy UISP CSV workflow or the operator override layer.
+
 #### UISP Route Overrides
 
 The default cost between nodes is typically 10. The integration creates a dot graph file `/opt/libreqos/src/graph.dot` which can be rendered using [Graphviz](https://dreampuf.github.io/GraphvizOnline/). This renders a map with the associated costs of all links.

docs/v2.0/integrations.md

@@ -53,7 +53,12 @@ When integrations are enabled:
 
 ## Topology Node ID Support
 
-LibreQoS supports an optional generic `"id"` field on `network.json` nodes. This field is intended to carry stable node identifiers from the integration source where possible. In the current release, the field is informational and is not yet the authoritative match key for shaping or overrides.
+LibreQoS supports an optional generic `"id"` field on `network.json` nodes. This field is intended to carry stable node identifiers from the integration source where possible.
+
+Current behavior:
+- node IDs are the preferred match key for operator-owned site bandwidth overrides and tree-page node override editing when an ID is present
+- legacy name-only matching is still supported as a fallback for older data
+- topology names still need to remain globally unique in `network.json`
 
 | Integration | `network.json` node ID support | Notes |
 |---|---|---|

docs/v2.0/libreqos-backend-architecture.md

@@ -36,6 +36,7 @@ In production terms:
 - `XDP`/`eBPF` and lookup maps decide packet identity and CPU path.
 - Linux traffic control (`tc`) enforces queueing policy.
 - Bakery manages update deltas and reload boundaries.
+- Queue mode (`shape` vs `observe`) controls whether the subscriber shaping tree is active or intentionally removed for baseline measurement.
 
 ## 2) Runtime Invariants
 
@@ -48,8 +49,23 @@ These invariants are useful for reasoning about whether backend behavior is heal
 | Data-plane mapping is stable between XDP and `tc` | Mis-mapped packets cause wrong queue assignment | Unexpected class counters, mis-accounted traffic | Compare expected class IDs vs observed `tc -s` class counters |
 | Control-plane changes stay within incremental-safe bounds | Reduces disruption from full-tree rebuilds | Frequent packet-impacting reload windows | Review change patterns: structural vs speed/mapping updates |
 | Queue count stays within CPU/RAM budget | Leaf qdisc scale directly impacts resources | Memory growth, reload slowdowns, jitter under churn | Track queue count, RAM headroom, and update cadence |
+| Queue mode and dataplane mappings are sequenced consistently | Packets must not be steered toward removed queue state | Brief outages, stale class targets, mis-accounted traffic | Check queue mode, IP mapping lifecycle, and live `tc` state together |
 
-## 3) End-to-End Packet and Control Path
+## 3) Runtime Authority and Configuration Model
+
+LibreQoS has both on-disk config files and a long-running runtime daemon, but they are not the same authority at every moment.
+
+1. `lqosd` is the runtime control-plane authority while it is running.
+2. UI/config API updates go through `lqosd`, which updates in-memory state and then drives apply/reload behavior.
+3. Manual edits to `/etc/lqos.conf` are operator-managed source inputs, but they do not automatically become active runtime state until the daemon reload path consumes them.
+
+Operational takeaway:
+
+- treat the UI/config API path as the authoritative live-update workflow
+- treat direct file edits as a separate operator action that still needs a runtime reload/apply boundary
+- do not assume “file changed” and “runtime desired state changed” are equivalent at the same instant
+
+## 4) End-to-End Packet and Control Path
 
 ```{mermaid}
 flowchart LR
@@ -76,9 +92,9 @@ flowchart LR
     N -. updates queue state .-> H
 ```
 
-## 4) Data Plane Design
+## 5) Data Plane Design
 
-### 4.1 XDP and classification pipeline
+### 5.1 XDP and classification pipeline
 
 LibreQoS performs early packet work in XDP where possible:
 
@@ -93,11 +109,11 @@ A key optimization direction has been reducing repeated lookups:
 - avoid duplicate work between XDP and `tc` when metadata can be passed forward
 
 
-### 4.2 Why `cpumap` is central
+### 5.2 Why `cpumap` is central
 
 `cpumap` is used to spread work across cores so shaping does not bottleneck on a single queue path. This is a major part of scaling from "works" to "works at ISP traffic levels".
 
-### 4.3 Cache, generation, and lock-pressure reduction
+### 5.3 Cache, generation, and lock-pressure reduction
 
 The high-level progression described in development notes/devblogs:
 
@@ -107,8 +123,19 @@ The high-level progression described in development notes/devblogs:
 
 Operationally, this helps stabilize latency and CPU under frequent updates.
 
+### 5.4 Mapping state is part of the data-path contract
 
-## 5) Queue Hierarchy: `mq` -> `HTB` -> leaf qdisc
+Queue classes and IP mappings are related but not identical backend state.
+
+1. The shaping tree defines where packets can land in `tc`.
+2. IP mappings define which circuit/class packets are currently steered toward.
+3. During ordinary `shape -> shape` updates, LibreQoS tries to preserve stable mappings and stable queue handles where safe.
+4. During disruptive transitions, especially `observe <-> shape`, LibreQoS can intentionally clear and later republish mappings so packets are not pointed at queue state that no longer exists.
+
+This sequencing matters as much as the queue commands themselves. A healthy backend design must reason about queue-tree changes and mapping changes together.
+
+
+## 6) Queue Hierarchy: `mq` -> `HTB` -> leaf qdisc
 
 LibreQoS queueing is intentionally layered:
 
@@ -134,7 +161,7 @@ flowchart TD
     I --> J
 ```
 
-### 5.1 HTB internals that matter in production
+### 6.1 HTB internals that matter in production
 
 Important mechanics:
 
@@ -150,16 +177,16 @@ Why operators care:
 - HTB is the rate envelope; leaf qdiscs are not a drop-in replacement for HTB policy
 
 
-## 6) AQM in LibreQoS: `fq_codel` and `CAKE`
+## 7) AQM in LibreQoS: `fq_codel` and `CAKE`
 
-### 6.1 Responsibility split
+### 7.1 Responsibility split
 
 Practical split:
 
 1. HTB: hierarchical bandwidth policy and limits
 2. `fq_codel`/`CAKE`: queue fairness and delay control within that policy
 
-### 6.2 Why AQM still helps even below line-rate saturation
+### 7.2 Why AQM still helps even below line-rate saturation
 
 Even when a link is not saturated, fq_codel and CAKE still affect packet behavior. The drop logic (CoDel/BLUE/COBALT) usually stays idle, but the fair-queueing scheduler continues to interleave flows, preventing bursts from one flow from monopolizing the serialization point of the link. This keeps latency-sensitive traffic responsive.
 
@@ -189,7 +216,7 @@ Even without sustained congestion:
 
 So even when the AQM drop logic is mostly idle, the fair-queueing part of SQM is still doing useful work by controlling how packets reach the wire.
 
-### 6.3 CAKE vs fq_codel in LibreQoS terms
+### 7.3 CAKE vs fq_codel in LibreQoS terms
 
 General pattern:
 
@@ -202,7 +229,7 @@ Resource reality:
 - both are flow-aware and keep state
 - CAKE can have higher memory/CPU footprint in large queue populations
 
-### 6.4 When below-line-rate gains may be limited
+### 7.4 When below-line-rate gains may be limited
 
 Lower latency under mixed load is common, but not guaranteed in every scenario.
 
@@ -218,7 +245,7 @@ Operator takeaway:
 - Treat AQM gains as a result of queue dynamics and contention control, then validate empirically on your own traffic mix.
 
 
-## 7) Bakery and Reload Behavior
+## 8) Bakery and Reload Behavior
 
 Bakery exists to avoid unnecessary queue rebuilds and reduce reload penalties.
 
@@ -230,7 +257,7 @@ High-level flow:
 4. Trigger full reload when a change is outside live-mutation support, or when runtime verification/drift detection marks incremental topology mutation unsafe.
 
 
-### 7.1 Lazy queueing and expiration
+### 8.1 Lazy queueing and expiration
 
 Key controls:
 
@@ -242,7 +269,7 @@ Practical effect:
 - reduced memory overhead for dormant endpoints
 - lower churn for large but partially active subscriber populations
 
-### 7.2 Incremental vs reload boundary
+### 8.2 Incremental vs reload boundary
 
 | Change type | Usually incremental-safe | Often requires full reload | Why |
 |---|---|---|---|
@@ -272,14 +299,49 @@ flowchart TD
     F --> K[Re-establish single authoritative queue model]
 ```
 
-### 7.3 Reload boundary quick rules
+### 8.3 Queue modes and transition semantics
+
+LibreQoS currently has two explicit queue modes:
+
+1. `shape`
+   - normal shaping mode
+   - root `mq` present
+   - HTB hierarchy present
+   - leaf qdiscs present
+   - per-circuit IP mappings present
+2. `observe`
+   - true-baseline mode
+   - root `mq` retained
+   - subscriber shaping tree removed
+   - per-circuit IP mappings cleared before teardown
+   - mappings republished after returning to `shape`
+
+Important operator boundary:
+
+- `observe` is intentionally honest, not hitless
+- switching `observe <-> shape` can briefly interrupt traffic because the shaping tree really is removed and later rebuilt
+- this is different from ordinary `shape -> shape` updates, where LibreQoS tries to preserve stable handles and queue placement where possible
+
+### 8.4 Retained-root full reload behavior
+
+Current Bakery full reloads try to avoid unnecessary root churn.
+
+1. Verify live kernel `tc` state, not just planned state.
+2. If the root `mq` is healthy and matches the expected layout, retain it.
+3. Prune child qdiscs beneath the retained root and verify the subtree is clean.
+4. Rebuild the shaping tree beneath that retained root.
+5. Fall back to root recovery only when retained-root reuse is unsafe or verification fails.
+
+This retained-root strategy reduces avoidable root-level churn, while still preferring explicit recovery when live state is ambiguous.
+
+### 8.5 Reload boundary quick rules
 
 1. Prefer frequent small mapping/speed deltas over large structural churn.
 2. Batch topology surgery into planned windows.
 3. Expect higher risk when many circuits and many structure-affecting changes happen together.
 4. Build operations cadence around incremental-safe updates by default.
 
-### 7.4 Full-reload safety guards
+### 8.6 Full-reload safety guards
 
 Current Bakery full reloads apply two conservative safety checks before and during large queue rebuilds:
 
@@ -288,7 +350,7 @@ Current Bakery full reloads apply two conservative safety checks before and duri
 3. During chunked full reload apply, Bakery re-checks host memory at chunk boundaries and aborts the remaining apply if available memory drops below its safety floor.
 4. These guards are intentionally biased toward false positives on large reloads so the system fails early with diagnostics instead of spiraling into an OOM event.
 
-### 7.5 Runtime safety model
+### 8.7 Runtime safety model
 
 Bakery's newer runtime-safety direction is intentionally narrow:
 
@@ -305,7 +367,7 @@ This is intentionally **not** a broad self-healing reconciler. LibreQoS is biase
 
 That design keeps failure handling easier to reason about than trying to incrementally repair arbitrary split-brain queue state.
 
-### 7.6 Explicit qdisc-handle management
+### 8.8 Explicit qdisc-handle management
 
 Bakery now treats leaf qdisc handles as persistent runtime state rather than disposable auto-allocation details.
 
@@ -316,7 +378,7 @@ Bakery now treats leaf qdisc handles as persistent runtime state rather than dis
 
 This handle model is one of the mechanisms that makes common live circuit migration safer than earlier Bakery generations.
 
-### 7.7 Runtime virtualization limits and operator expectations
+### 8.9 Runtime virtualization limits and operator expectations
 
 Current runtime virtualization support is intentionally constrained.
 
@@ -330,9 +392,9 @@ Operator takeaway:
 - treat runtime virtualization as a narrow live-mutation feature with verification gates
 - treat `reload required` as the authoritative signal that Bakery no longer trusts incremental topology mutation
 
-## 8) Design Boundaries for Operators
+## 9) Design Boundaries for Operators
 
-### 8.1 Observability boundaries
+### 9.1 Observability boundaries
 
 | Signal | Strong for | Weak for |
 |---|---|---|
@@ -340,8 +402,23 @@ Operator takeaway:
 | CAKE/fq_codel drops/marks | Detecting persistent queue pressure and policy effects | Full end-to-end application blame assignment |
 | CPU/RAM and command timing | Capacity and reload risk planning | Isolating every microburst source |
 
+### 9.2 Metric sample semantics and clock domains
 
-### 8.2 Capacity risk factors and mitigations
+Backend metrics do not all come from the same sampling source or clock edge.
+
+1. Some values are sampled from queue/kernel state.
+2. Some values are sampled from flow telemetry.
+3. Some rollups are built from canonical raw samples and only later rendered as percentages.
+
+Practical implication:
+
+- do not casually combine unrelated numerator/denominator sources and assume they describe the same exact second
+- prefer transporting canonical samples/counts and deriving percentages at presentation time
+- treat “same field name” and “same clock domain” as separate questions
+
+This matters especially for retransmit, packet, and rate-derived health metrics.
+
+### 9.3 Capacity risk factors and mitigations
 
 | Risk factor | Typical symptom | Mitigation |
 |---|---|---|
@@ -350,7 +427,7 @@ Operator takeaway:
 | Incomplete parent mapping in hierarchy | Subscribers unexpectedly unshaped | Validate parent relationships in `network.json` and input data |
 | Single-queue/weak NIC virtualization behavior | Poor spread and unstable shaping | Ensure multi-queue NIC path and verify queue mapping assumptions |
 
-## 9) Symptom-to-Cause Troubleshooting Matrix
+## 10) Symptom-to-Cause Troubleshooting Matrix
 
 | Symptom | Common backend cause | First checks | Typical corrective direction |
 |---|---|---|---|
@@ -361,11 +438,11 @@ Operator takeaway:
 | RAM growth during scale-up | Too many active leaf qdiscs or aggressive CAKE footprint | Measure queue count and memory trends over update windows | Use lazy queue creation/expiry and consider selective fq_codel use |
 | Dashboard traffic appears higher than expected user throughput | Counter scope differs from post-drop forwarded traffic | Compare dashboard metrics with `tc` drop/mark context | Align runbooks to metric semantics before escalating |
 
-## 10) Change Validation Workflow
+## 11) Change Validation Workflow
 
 Use this lightweight workflow for backend-impacting changes.
 
-### 10.1 Pre-change
+### 11.1 Pre-change
 
 1. Classify change type: mapping/speed/structure.
 2. Estimate impact scope: number of affected circuits/classes.
@@ -375,7 +452,7 @@ Use this lightweight workflow for backend-impacting changes.
    - CPU and RAM headroom
    - `tc` class/qdisc snapshot
 
-### 10.2 During change
+### 11.2 During change
 
 1. Watch control-plane behavior:
    - incremental apply vs full reload occurrence
@@ -385,14 +462,19 @@ Use this lightweight workflow for backend-impacting changes.
    - directional latency drift
    - per-class counter discontinuities
 
-### 10.3 Post-change
+### 11.3 Post-change
 
 1. Re-check the same baseline signals.
 2. Confirm hierarchy/class presence for changed circuits.
 3. Verify subscriber-facing latency and throughput expectations.
 4. If degraded, rollback or reduce change scope and re-apply in smaller batches.
 
-### 10.4 Minimal command checklist
+For `observe <-> shape` transitions, add two specific checks:
+
+1. confirm the queue mode actually matches the intended state
+2. confirm per-circuit mappings were cleared or republished at the right phase of the transition
+
+### 11.4 Minimal command checklist
 
 Adjust device names to your environment.
 
@@ -402,7 +484,7 @@ tc -s class show dev <ifname>
 journalctl -u lqosd --since "15 min ago"
 ```
 
-## 11) Practical Tuning Sequence
+## 12) Practical Tuning Sequence
 
 Recommended order:
 
@@ -413,7 +495,7 @@ Recommended order:
 5. Tune update cadence to favor incremental-safe changes.
 6. Re-test after major speed-plan, topology, or integration-cadence changes.
 
-## 12) Glossary
+## 13) Glossary
 
 - `XDP`: earliest high-performance packet hook in Linux.
 - `eBPF`: in-kernel programmable packet processing.

docs/v2.0/node-manager-ui.md

@@ -27,6 +27,8 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
 - `Node Snapshot` provides a quick visual summary of current throughput and QoO for the selected node.
 - Attached circuits are shown in a dedicated table for the selected node.
 - Administrators can save or clear `Operator Override` values where node-level overrides are supported. Read-only users and unsupported nodes continue to display current values without edit controls.
+- Tree-page operator rate edits write operator-owned overrides to `lqos_overrides.json`; they do not rewrite legacy integration bandwidth CSV files.
+- Tree-page operator rate edits require an administrator session, a stable node ID, and a non-generated node. Generated/integration-synthetic nodes remain read-only in this editor.
 
 ### Site Map
 - Flat operational map of Sites and APs using imported node geodata.
@@ -34,8 +36,9 @@ This page documents key WebUI (Node Manager) views and operational behavior in t
 - Uses a 30-second client-side average from `NetworkTree` data rather than adding backend rollup work.
 - APs can inherit parent site coordinates for display when explicit AP coordinates are missing.
 - Initial map framing now prefers site coordinates for a closer first view, falling back to AP coordinates when no sites are mapped yet.
-- Uses a local LibreQoS-themed basemap with country/state borders, coastlines, major lakes, major rivers, marine areas, subtle physical-region overlays, and higher-zoom major-highway context for geographic orientation.
-- Site Map uses a local Natural Earth-derived roads layer for orientation while keeping the rest of the basemap quiet and operational.
+- Current builds use an Insight-hosted OpenStreetMap raster tile cache rather than a fully local basemap bundle.
+- Site Map depends on outbound access to `https://insight.libreqos.com` for initial bbox/bootstrap and raster tile fetches.
+- When tiles are missing from the remote cache, the browser retries automatically for a short period instead of failing immediately, so initial map paint can lag briefly on cold tiles.
 
 ### Flow Globe
 - Geographic flow visualization based on endpoint geolocation.
@@ -109,6 +112,8 @@ When this happens:
 journalctl -u lqosd --since "10 minutes ago"
 ```
 
+If only Site Map is blank or slow while other pages are healthy, also check management-network reachability to `insight.libreqos.com` and allow a short delay for cold tile-cache retries.
+
 ## Useful Related Pages
 
 - [Components](components.md)

docs/v2.0/quickstart.md

@@ -1,12 +1,21 @@
 # Quickstart: ISP Deployment Path
 
-Use this page to move from install to a safe pilot with minimal ambiguity.
+Use this page to move from install to a safe pilot.
+
+Most ISPs, should use a LibreQoS built-in integration.
+
+Use this page in order:
+1. Complete install and bridge setup.
+2. Complete first-run WebUI access.
+3. Pass the 10-minute health check.
+4. Pick one source of truth owner.
+5. Follow the recommended path for that owner.
 
 Need definitions for key terms? See the [Glossary](glossary.md).
 
 ## 1) Common Install Foundation
 
-Complete this once:
+Complete this once before trying to shape live traffic:
 
 1. Review architecture and sizing:
 - [Deployment Scenarios](design.md)
@@ -16,9 +25,11 @@ Complete this once:
 - [Server Setup - Prerequisites](prereq.md)
 - [Install Ubuntu Server 24.04](ubuntu-server.md)
 
-3. Configure bridge mode:
+3. Configure how traffic will pass through the LibreQoS box:
 - [Configure Shaping Bridge](bridge.md)
 
+This step is required before production use. If bridge/interface setup is wrong, later WebUI and shaping checks will be misleading.
+
 4. Install LibreQoS (`.deb` recommended):
 
 ```bash
@@ -54,6 +65,13 @@ sudo apt install ./{deb_url_v1_5}
 
 5. Open WebUI at `http://your_shaper_ip:9123`.
 
+If no WebUI users exist yet, current builds redirect you to the first-run setup page automatically instead of presenting a normal login flow.
+
+6. Complete first-run access:
+- Create the initial WebUI user if prompted.
+- Confirm you can sign in.
+- Confirm the Dashboard opens before doing any integration or topology work.
+
 ## 2) 10-Minute Health Check
 
 Run:
@@ -67,54 +85,65 @@ Confirm:
 - WebUI Dashboard loads.
 - Scheduler Status is healthy.
 - No urgent/fatal startup errors in logs.
+- You can sign in successfully after first-run setup.
 
 If this fails, go to [Troubleshooting](troubleshooting.md) before proceeding.
 
-## 3) Decision A: Deployment Stage
+## 3) Pick One Source of Truth Owner
+
+This is the most important early decision.
+
+Source of truth means: the one place that should own persistent shaping data.
+
+If you make persistent changes somewhere else, they may be overwritten later.
 
 Choose one:
 
-- **Lab first**: validate behavior in a controlled environment before inline traffic.
-- **Inline pilot now**: proceed directly with limited production traffic.
-
-If lab-first:
-1. Build a lab topology.
-2. Generate test traffic.
-3. Validate Dashboard, Tree, Flow, Scheduler Status, and Urgent Issues.
-4. Continue to Decision B.
-
-## 4) Decision B: Source of Truth (Pick One Owner)
-
 | If this describes you | Mode | Owner of durable shaping data |
 |---|---|---|
-| You use a supported CRM/NMS integration | Built-in integration mode | Integration jobs |
-| You generate `network.json` and `ShapedDevices.csv` with your own scripts | Custom source of truth mode | Your scripts |
-| You maintain files manually for a small/simple network | Manual files mode | Manual edits |
+| You use a supported LibreQoS CRM/NMS integration | Built-In Integration Mode | Integration jobs |
+| You generate `network.json` and `ShapedDevices.csv` with your own automation | Custom Source of Truth Mode | Your scripts |
+| You intentionally maintain files by hand | Manual Files Mode | Manual edits |
 
 Rule: keep one owner for persistent shaping inputs.
 
-## 5) Path Cards
+Most operators should stop here and choose **Built-In Integration Mode**.
 
-### Built-In Integration Mode
+Only choose the other two modes if you already know why.
+
+## 4) Recommended Path: Built-In Integration Mode
+
+This is the default path for most ISPs, including most small ISPs.
 
 When to choose:
 - Your CRM/NMS is supported by built-in integrations.
 
 Do this now:
 1. Configure integration settings in WebUI.
-2. Run initial sync and validate imported shaping/topology data.
-3. Place LibreQoS inline for pilot traffic.
-4. Validate Scheduler Status, Urgent Issues, and topology/flow views.
-5. Expand pilot scope after stable operation.
+2. Run the initial sync.
+3. Confirm outputs were created or refreshed:
+   - `network.json`
+   - `ShapedDevices.csv`
+4. Confirm Scheduler Status is healthy.
+5. Confirm Network Tree reflects the expected hierarchy depth for your chosen integration strategy.
+6. Confirm there are no urgent/fatal issues that block shaping.
+7. Only then move to limited pilot traffic or inline use.
+
+Do not do this:
+- Do not hand-edit integration-owned files as your normal workflow.
+- Do not split ownership between integration syncs and manual file edits unless you intentionally want that complexity.
+- Do not change overwrite behavior unless you understand how your integration should own `network.json`.
 
 Next:
 - [CRM/NMS Integrations](integrations.md)
 - [Troubleshooting](troubleshooting.md)
 
-### Custom Source of Truth (Your Scripts)
+## 5) Other Paths
+
+### Custom Source of Truth Mode
 
 When to choose:
-- Your CRM/NMS is unsupported and you generate `network.json` + `ShapedDevices.csv` with your own pipeline.
+- Your CRM/NMS is unsupported and you already generate `network.json` + `ShapedDevices.csv` with your own pipeline.
 
 Do this now:
 1. Implement script/process to generate and refresh shaping files.
@@ -130,17 +159,22 @@ Next:
 - [Operating Modes and Source of Truth](operating-modes.md)
 - [Troubleshooting](troubleshooting.md)
 
-### Manual Files Mode (<100 Subscribers)
+### Manual Files Mode
 
 When to choose:
 - You intentionally maintain `network.json` + `ShapedDevices.csv` without CRM/NMS synchronization.
+- Your network is small/simple enough that manual discipline is realistic.
+- You are doing a temporary pilot or working around an unsupported system.
 
 Do this now:
-1. Build and maintain shaping files directly.
-2. Place LibreQoS inline for pilot traffic.
-3. Validate shaping and scheduler status in WebUI.
-4. Maintain strict manual change discipline.
-5. Plan migration to supported integration or scripts if scale/change volume grows.
+1. Build `network.json`.
+2. Build `ShapedDevices.csv`.
+3. Restart or validate services.
+4. Confirm the expected topology and devices appear in WebUI.
+5. Validate shaping and scheduler status before adding more complexity.
+6. Maintain strict manual change discipline.
+
+This is not the normal recommendation for operators who already use a supported LibreQoS integration.
 
 Next:
 - [Advanced Configuration Reference](configuration-advanced.md)
@@ -149,10 +183,22 @@ Next:
 ## 6) Common First-Run Mistakes
 
 - Unclear source of truth ownership between integration and manual edits.
+- Hand-editing files even though your integration is supposed to own them.
 - Changing topology depth before passing the health gate.
 - Skipping post-change service/log validation before pilot traffic.
+- Treating low-data or cold-start pages as proof that shaping is broken.
+- Changing too many things before you have one known-good baseline.
 
-## 7) Related Pages
+## 7) Done For Day 1 When
+
+- You can sign in to WebUI successfully.
+- Dashboard loads.
+- Scheduler Status is healthy.
+- No urgent/fatal startup errors are present.
+- The expected topology or devices appear.
+- One test subscriber/device behaves as expected.
+
+## 8) Related Pages
 
 - [Operating Modes and Source of Truth](operating-modes.md)
 - [CRM/NMS Integrations](integrations.md)

docs/v2.0/troubleshooting.md

@@ -134,6 +134,8 @@ This exports a log file to lqos_sched_log.txt. You can review this file to see w
 
 If the scheduler fails immediately after a restart with a message like `Socket (typically /run/lqos/bus) not found`, that indicates `lqosd` had not finished binding the local bus yet. Current builds wait briefly for bus readiness at scheduler startup instead of crashing immediately, so repeated startup panics after restart should no longer be expected.
 
+If an integration subprocess fails, current builds keep the scheduler alive, publish a shortened output preview to the scheduler status/error surfaces, and save the full captured output to a timestamped file under `/tmp` such as `lqos_scheduler_uisp_integration_YYYYMMDD_HHMMSS.log`.
+
 ### Scheduler status in WebUI looks unhealthy
 
 Recent builds expose scheduler readiness/state in the WebUI (Node Manager).
@@ -199,6 +201,21 @@ journalctl -u lqosd --since "10 minutes ago"
 
 If still blank under normal traffic, collect recent logs and open an issue.
 
+### Site Map appears blank or slow
+
+Site Map has one extra dependency beyond normal WebUI data feeds: current builds fetch bbox/bootstrap data and raster tiles from `https://insight.libreqos.com`.
+
+If Site Map alone is blank or slow:
+1. Confirm `lqosd` is healthy.
+2. Confirm the box can reach `insight.libreqos.com` from its management network.
+3. Wait briefly and reload the page; the browser retries tile requests automatically while cold tiles are being populated upstream.
+4. Check recent `lqosd` logs:
+```
+journalctl -u lqosd --since "10 minutes ago"
+```
+
+If the rest of WebUI is healthy but Site Map continues to fail, treat it as a map/tile dependency issue rather than a general scheduler or shaping failure.
+
 ### Virtual node promotion collision (network.json)
 
 If LibreQoS.py fails with an error like `Virtual node promotion collision: 'AP_A' already exists at this level.`, you have a `"virtual": true` node whose children get promoted into a parent level where a node with the same name already exists.
@@ -239,7 +256,7 @@ WebUI urgent issues include machine-readable codes. Use them to triage quickly.
 |---|---|---|---|
 | `MAPPED_CIRCUIT_LIMIT` | Bakery is enforcing a mapped-circuit limit. | Insight license status, `journalctl -u lqosd` for requested/allowed/dropped counts. | Reduce mapped circuits immediately or update license/limits. |
 | `TC_U16_OVERFLOW` | Queue/class minor IDs exceeded the Linux tc u16 range on a CPU queue. | `journalctl -u lqos_scheduler -u lqosd`, topology depth/queue distribution. | Increase queue count and/or simplify/rebalance hierarchy (for example with integration strategy or root promotion changes). |
-| `TC_QDISC_CAPACITY` | Planned auto-allocated qdiscs exceed the per-interface safe budget or Bakery's conservative memory-safety preflight before apply. | Estimated per-interface qdisc counts, qdisc-kind breakdown, and memory fields in the urgent issue context, `journalctl -u lqos_scheduler -u lqosd`, `on_a_stick` and `monitor_only` config. | Reduce the planned qdisc load for this run (for example fewer circuits/devices in the test shape) before retrying; do not trust partial apply. |
+| `TC_QDISC_CAPACITY` | Planned auto-allocated qdiscs exceed the per-interface safe budget or Bakery's conservative memory-safety preflight before apply. | Estimated per-interface qdisc counts, qdisc-kind breakdown, and memory fields in the urgent issue context, `journalctl -u lqos_scheduler -u lqosd`, `on_a_stick` and `queue_mode` config. | Reduce the planned qdisc load for this run (for example fewer circuits/devices in the test shape) before retrying; do not trust partial apply. |
 | `BAKERY_MEMORY_GUARD` | A chunked Bakery full reload was stopped mid-apply because available host memory fell below the safety floor. | `journalctl -u lqosd`, available/total memory in the urgent issue context, and recent Bakery apply progress. | Treat the run as failed, reduce memory pressure or queue footprint, and retry only after the host is stable. |
 | `XDP_IP_MAPPING_CAPACITY` | Required IP mappings exceed the current XDP kernel map capacity. | `ShapedDevices.csv` row shape, IPv4/IPv6 mix, one-device-vs-many-device assumptions, `journalctl -u lqos_scheduler -u lqosd`. | Reduce required mappings immediately (for example fewer devices or IPv4-only test shape), or raise kernel map capacity in a coordinated change. |
 | `XDP_IP_MAPPING_APPLY_FAILED` | One or more IP mapping inserts failed during apply. | `journalctl -u lqos_scheduler -u lqosd` for summarized failure examples and counts. | Fix the underlying mapping failure, then rerun; do not trust partial shaping. |