mirror of
https://github.com/LibreQoE/LibreQoS.git
synced 2026-10-25 14:20:05 -05:00
Merge branch 'develop' into LuisDanielEA-patch-2
This commit is contained in:
43 files changed
+2783
-259
No files matched your search
@@ -141,3 +141,4 @@ src/bin/lqos_api
|
||||
/src/linux_tc_rust.txt
|
||||
/src/linux_tc_rust_live.txt
|
||||
/src/linux_tc_rust_prune.txt
|
||||
/src/planner_state.json
|
||||
@@ -68,3 +68,6 @@ Once you have any errors eliminated, restart lqos_scheduler with
|
||||
```shell
|
||||
sudo systemctl start lqos_scheduler
|
||||
```
|
||||
|
||||
### Troubleshooting Service Components
|
||||
Please see [Troubleshooting](troubleshooting.md).
|
||||
+81
-12
@@ -1,7 +1,72 @@
|
||||
# Configure LibreQoS
|
||||
|
||||
## Main Configuration File
|
||||
### /etc/lqos.conf
|
||||
## Initial Configuration Via Setup Tool (From the .deb installer)
|
||||
<img width="1605" height="1030" alt="setup_tool" src="https://github.com/user-attachments/assets/5a645da8-c411-4635-9777-a881966981df" />
|
||||
|
||||
NOTES:
|
||||
- The LibreQoS Setup Tool can only be controlled with a keyboard. Use the arrows to move around the tool, and click ```Enter``` to Select.
|
||||
- Clicking the ```Q``` key will close the setup tool without saving.
|
||||
- If you are in the process of using the setup tool and it closes beforehand, in order to launch again the setup tool you will have to run the following commands:
|
||||
```
|
||||
sudo apt remove libreqos
|
||||
sudo apt install ./{deb_url_v1_5}
|
||||
```
|
||||
|
||||
The first step is to give a name to your Shaper Box (Node), by default it is named LibreQoS.
|
||||
Then, you can use the arrow keys on your keyboard to go over the different setup sections, as follows:
|
||||
|
||||
### Bridge Mode
|
||||
<img width="1668" height="397" alt="bridge_mode" src="https://github.com/user-attachments/assets/22bc05cc-f1e5-451a-b4f8-5e75d7b8d64f" />
|
||||
|
||||
By default, the Linux Bridge is selected. If you chose the XDP Bridge in previous step, please make the adjustment.
|
||||
|
||||
NOTE: Single Interface mode, as it name implies, its for users that can only use 1 interface, and require special support. For more information regarding Single Interface mode, please vistit our [Zulip Chat].
|
||||
|
||||
### Interfaces
|
||||
<img width="885" height="352" alt="interfaces" src="https://github.com/user-attachments/assets/4afedfe6-65b8-450c-a675-bea25ef4553c" />
|
||||
|
||||
Following the recommended setup diagram mentioned above, the "To Internet" interface should be the one facing the Edge Router (and therefore, the Internet) and the "To Network" interface should be the one facing the Core Router.
|
||||
|
||||
### Bandwidth
|
||||
<img width="1089" height="350" alt="bandwidth" src="https://github.com/user-attachments/assets/f68185c3-82dc-4fb5-b78a-d812665533fb" />
|
||||
|
||||
In the context of bandwidth hired from your upstream provider, ```To Internet``` means the Upload Bandwidth, and ```To Network``` means the download bandwidth.
|
||||
|
||||
### IP Range
|
||||
<img width="1331" height="481" alt="ip_ranges" src="https://github.com/user-attachments/assets/b846baa7-288e-460c-ab77-ad400384057c" />
|
||||
|
||||
In this section, you should specify all the ip ranges utilized by your customers' routers, including ranges for static IPs. By default, we include 4 common IP Ranges, as seen in the image above.
|
||||
|
||||
*Tip: In this section, in order to remove an IP range, you have to highlight the IP range with the keyboard arrows, and then click the ```Tab``` key until ```<Remove Selected>``` is highlighted, then click ```Enter```.
|
||||
|
||||
### Web Users
|
||||
<img width="1664" height="528" alt="web_users" src="https://github.com/user-attachments/assets/8db17e0e-cc3d-4d67-9c59-4751bc4d9b0f" />
|
||||
|
||||
In this section, you can create users to access the LibreQoS Web Dashboard, with their respective roles. There's 2 different types of roles: ```Admin``` and ```Read Only```.
|
||||
***
|
||||
After clicking on save file, you may see the following message on the terminal
|
||||
```
|
||||
No VM guests are running outdated hypervisor (qemu) binaries on this host.
|
||||
N: Download is performed unsandboxed as root as file '/home/libreqos/libreqos_1.5-RC2.202510052233-1_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
|
||||
```
|
||||
This error is benign and does not reflect any issues. Please disregard it.
|
||||
|
||||
### Next Steps
|
||||
|
||||
If the installation is successful, you will be able to access the LibreQoS WebUI at ```http://your_shaper_ip:9123```. Upon first visit, you can specify your preferred username and password.
|
||||
|
||||
Next, you will want to configure your [CRM or NMS integration](integrations.md) using the Web Interface Configuration page. If you do not use a supported CRM/NMS, you will need to create a script or process to produce the needed files for LibreQoS to shape traffic - specifically network.json and ShapedDevices.csv. The format of these files are explained in further detail in later sections of this page.
|
||||
|
||||
## Configuration Via Web Interface
|
||||
|
||||
Most LibreQoS shaper settings can be modified via the Configuration page on the WebUI (http://your_shaper_ip:9123/config_general.html).
|
||||
|
||||
## Configuration via Command Line
|
||||
|
||||
You can also modify settings using the command line.
|
||||
|
||||
### Main Configuration File
|
||||
#### /etc/lqos.conf
|
||||
|
||||
The configuration for each LibreQoS shaper box is stored in the file `/etc/lqos.conf`.
|
||||
|
||||
@@ -25,7 +90,7 @@ Note: If you find that traffic is not being shaped when it should, please make s
|
||||
|
||||
After changing any part of `/etc/lqos.conf` it is highly recommended to always restart lqosd, using `sudo systemctl restart lqosd`. This re-parses any new values in lqos.conf, making those new values accessible to both the Rust and Python sides of the code.
|
||||
|
||||
### Netflow (optional)
|
||||
#### Netflow (optional)
|
||||
To enable netflow, add the following `[flows]` section to the `/etc/lqos.conf` configuration file, setting the appropriate `netflow_ip`:
|
||||
```
|
||||
[flows]
|
||||
@@ -37,12 +102,12 @@ netflow_version = 5
|
||||
do_not_track_subnets = ["192.168.0.0/16"]
|
||||
```
|
||||
|
||||
### CRM/NMS Integrations
|
||||
#### CRM/NMS Integrations
|
||||
|
||||
Learn more about [configuring integrations here](integrations.md).
|
||||
|
||||
## Network Hierarchy
|
||||
### Network.json
|
||||
### Network Hierarchy
|
||||
#### Network.json
|
||||
|
||||
Network.json allows ISP operators to define a Hierarchical Network Topology, or Flat Network Topology.
|
||||
|
||||
@@ -58,7 +123,11 @@ For networks with no Parent Nodes (no strictly defined Access Points or Sites) e
|
||||
echo "{}" > network.json
|
||||
```
|
||||
|
||||
#### CSV to JSON conversion helper
|
||||
#### CPU Considerations
|
||||
|
||||
<img width="3276" height="1944" alt="cpu" src="https://github.com/user-attachments/assets/e4eeed5e-eeeb-4251-9258-d301c3814237" />
|
||||
|
||||
##### CSV to JSON conversion helper
|
||||
|
||||
You can use
|
||||
|
||||
@@ -71,7 +140,7 @@ manualNetwork.csv can be copied from the template file, manualNetwork.template.c
|
||||
|
||||
Note: The parent node name must match that used for clients in ShapedDevices.csv
|
||||
|
||||
## Circuits
|
||||
### Circuits
|
||||
|
||||
LibreQoS shapes individual devices by their IP addresses, which are grouped into "circuits".
|
||||
|
||||
@@ -79,7 +148,7 @@ A circuit represents an ISP subscriber's internet service, which may have just o
|
||||
|
||||
LibreQoS knows how to shape these devices, and what Node (AP, Site, etc) they are contained by, with the ShapedDevices.csv file.
|
||||
|
||||
### ShapedDevices.csv
|
||||
#### ShapedDevices.csv
|
||||
|
||||
The ShapedDevices.csv file correlates device IP addresses to Circuits (each internet subscriber's unique service).
|
||||
|
||||
@@ -100,17 +169,17 @@ Here is an example of an entry in the ShapedDevices.csv file:
|
||||
|
||||
If you are using one of our CRM integrations, this file will be automatically generated. If you are not using an integration, you can manually edit the file using either the WebUI or by directly editing the ShapedDevices.csv file through the CLI.
|
||||
|
||||
#### Multiple IPs per Circuit
|
||||
##### Multiple IPs per Circuit
|
||||
If you need to list multiple IPv4s in the IPv4 field, or multiple IPv6s in the IPv6 field, add a comma between them. If you are editing with a CSV editor (LibreOffice Calc, Excel), the CSV editor will automatically wrap these comma-seperated items with quotes for you. If you are editing the file manually with a utility like notepad or nano, please add quotes surrounding the comma-seperated entries.
|
||||
|
||||
```
|
||||
2794,"6 Littleton Drive, Ringgold, GA 30736",5,Device 5,AP_11,,100.64.0.5,"fdd7:b724:0:500::/56,fdd7:b724:0:600::/56",1,1,105,18,""
|
||||
```
|
||||
|
||||
#### Manual Editing by WebUI
|
||||
##### Manual Editing by WebUI
|
||||
Navigate to the LibreQoS WebUI (http://a.b.c.d:9123) and select Configuration > Shaped Devices.
|
||||
|
||||
#### Manual Editing by CLI
|
||||
##### Manual Editing by CLI
|
||||
|
||||
- Modify the ShapedDevices.csv file using your preferred spreadsheet editor (LibreOffice Calc, Excel, etc), following the template file - ShapedDevices.example.csv
|
||||
- Circuit ID is required. The Circuit ID can be a number or string. This field must NOT include any number symbols (#). Every circuit requires a unique CircuitID - they cannot be reused. Here, circuit essentially refers to a customer's service. If a customer has multiple locations on different parts of your network, use a unique CircuitID for each of those locations.
|
||||
|
||||
+17
-31
@@ -1,54 +1,40 @@
|
||||
# Network Design Assumptions
|
||||
# Deployment Scenarios
|
||||
|
||||
## Officially supported configuration
|
||||
## Standard Inline Deployment
|
||||
|
||||
- LibreQoS is placed inline at the edge of your network, usually between the network's border router (NAT, firewall) and the core distribution router / switch.
|
||||
LibreQoS is placed inline at the edge of your network, usually between the network's border router (NAT, firewall) and the core distribution router / switch.
|
||||
|
||||
|
||||
|
||||
### Primary and Backup Paths
|
||||
|
||||
We recommend using dynamic routing protocols such as OSPF to create a high cost and low cost path between the edge router and core distribution router/switch. The low cost path should pass "through" the LibreQoS shaper bridge interfaces to allow LibreQoS to observe and shape traffic. For example, a low cost OSPF path may be set to a value of 1. The high-cost (backup) link would completely bypass LibreQoS, being set to a higher cost (perhaps 100 for OSPF) to ensure that traffic only takes that path when the LibreQoS shaper bridge is not operational.
|
||||
|
||||
### NAT/CG-NAT
|
||||
#### NAT/CG-NAT
|
||||
If you use NAT/CG-NAT, please place LibreQoS inline prior to where NAT is applied, as LibreQoS needs to shape pre-NAT addresses (100.64.0.0/12) not public post-NAT IPs.
|
||||
|
||||
### MPLS/VPLS
|
||||
#### MPLS/VPLS
|
||||
LibreQoS can parse MPLS traffic, however the traffic must follow the standard pattern:
|
||||
```
|
||||
(mpls tags)(optional vlan tags)(ip header)
|
||||
```
|
||||
If you use MPLS with a different tag pattern, you would ideally want to terminate MPLS traffic at the core distribution router / switch, before it reaches LibreQoS.
|
||||
|
||||
## Testbed configuration
|
||||
When you are first testing out LibreQoS, we recommend deploying a small-scale testbed to see it in action.
|
||||
|
||||
#### Dynamic Routing: Bypass path
|
||||
- Primary path (low cost) *through* the server running LibreQoS
|
||||
- Backup path (high cost) *bypassing* the server running LibreQoS.
|
||||
|
||||
### Network Interface Card
|
||||
#### Diagram
|
||||
|
||||
|
||||
```{note}
|
||||
You must have one of these:
|
||||
- single NIC with two interfaces,
|
||||
- two NICs with single interface,
|
||||
- 2x VLANs interface (using one or two NICs).
|
||||
```
|
||||
### Option 1: Using Dynamic Routing (Strongly Recommended)
|
||||
|
||||
LibreQoS requires NICs to have 2 or more RX/TX queues and XDP support. While many cards theoretically meet these requirements, less commonly used cards tend to have unreported driver bugs which impede XDP functionality and make them unusable for our purposes. At this time we recommend the Intel x520, Intel x710, and Nvidia (ConnectX-5 or newer) NICs. We cannot guarantee compatibility with other cards.
|
||||
We recommend using dynamic routing protocols such as OSPF to create a high cost and low cost path between the edge router and core distribution router/switch. The low cost path should pass "through" the LibreQoS shaper bridge interfaces to allow LibreQoS to observe and shape traffic. For example, a low cost OSPF path may be set to a value of 1. The high-cost (backup) link would completely bypass LibreQoS, being set to a higher cost (perhaps 100 for OSPF) to ensure that traffic only takes that path when the LibreQoS shaper bridge is not operational.
|
||||
|
||||
## Alternate configuration
|
||||
### Option 2: Using Spanning Tree Protocol (Not Recommended)
|
||||
|
||||
This alternate configuration uses Spanning Tree Protocol (STP) to modify the data path in the event the LibreQoS device is offline for maintenance or another problem.
|
||||
You can also use the Spanning Tree Protocol with path costs if OSPF or another dynamic routing protocol is not an option.
|
||||
|
||||
```{note}
|
||||
Most of the same considerations apply to the alternate configuration as they do to the officially supported configuation
|
||||
```
|
||||
|
||||
- LibreQoS is placed inline at the edge of your network, usually between the network's border router (NAT, firewall) and the core distribution router / switch.
|
||||
- If you use NAT/CG-NAT, place LibreQoS inline prior to where NAT is applied, as LibreQoS needs to shape pre-NAT addresses (100.64.0.0/12) not public post-NAT IPs.
|
||||
- For networks using MPLS: LibreQoS can parse MPLS traffic, but the traffic must follow the standard pattern (mpls tags)(optional vlan tags)(ip header). If you use MPLS with a different pattern, you would ideally want to terminate MPLS traffic at the core distribution router / switch, before it reaches LibreQoS.
|
||||
- Spanning Tree primary link (low cost) through the server running LibreQoS
|
||||
- Spanning Tree backup link (high cost, maybe 80 for example)
|
||||
|
||||
Keep in mind that if you use different bandwidth links, for example, 10 Gbps through LibreQoS, and 1 Gbps between core switch and edge router, you may need to be more intentional with your STP costs.
|
||||
|
||||
|
||||
## Testbed Deployment (Optional)
|
||||
When you are first testing out LibreQoS, you can deploy a small-scale testbed to see it in action on a test network.
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
# LibreQoS Insight (Insight)
|
||||
|
||||
## About Insight
|
||||
Learn more about Insight on our website, [here](https://libreqos.io/insight/).
|
||||
|
||||
## Insight UI
|
||||
|
||||
### Taskbar
|
||||
|
||||
<img width="355" height="871" alt="taskbar" src="https://github.com/user-attachments/assets/796cba7b-49d0-4a49-96a5-cd12823a6bd8" />
|
||||
|
||||
If you have more than one LibreQoS shaper box, you can select the shaper box you want to view using the Shaper Nodes dropdown box (located just below the search box).
|
||||
|
||||
### Time Selector
|
||||
|
||||
<img width="529" height="83" alt="image" src="https://github.com/user-attachments/assets/b706a230-883a-4064-9436-2e82749eb8b7" />
|
||||
|
||||
By clicking the time selector (Last 24 Hours by default) you can set the time range for data display anywhere from 15 minutes to 28 days. You can also specify custom time periods (Now minus X minutes) or highly-specific detailed time periods.
|
||||
|
||||
### Dashboard
|
||||
|
||||
You can see which shaper box "perspective" is in use by the listed shpaer box name at the top-left corner next to "Select a destination".
|
||||
|
||||
The dashboard for LibreQoS Insight is widget-based, and has multiple tabs - each of which can be edited.
|
||||
|
||||
#### Traffic Overview Tab
|
||||
|
||||
<img width="3828" height="2160" alt="01 dashboard" src="https://github.com/user-attachments/assets/29ee98e3-55c7-4466-a444-de9542cc0940" />
|
||||
|
||||
The default tab is the Traffic Overview tab - which displays both Live traffic levels as well as top-traffic endpoints - both by ASN and by Node.
|
||||
|
||||
#### Shaper Tab
|
||||
|
||||
<img width="3840" height="2160" alt="02 shaper tab" src="https://github.com/user-attachments/assets/721fd195-35ad-421b-8d0a-e2aa6e5cf7e9" />
|
||||
|
||||
The Shaper tab shows high-level stats for a particular LibreQoS shaper box.
|
||||
|
||||
- Active Circuit Count: The number of active customer circuits observed - based on the subscriber's traffic movement.
|
||||
- Throughput: Shaper box throughput
|
||||
- Shaper Packets: Packet-per-second rate of the Shaper box over time.
|
||||
- Shaper TCP Retransmits Percentage: The percentage of TCP packets observed that were retransmitted. Consider this a proxy for packet loss. This value should typically remain below 1 % across the network.
|
||||
- Shaper CAKE Activity: How active the CAKE shapers are on the network over time.
|
||||
- Shaper Round Trip Time: The Round Trip Time average of traffic moving across the network.
|
||||
- Shaper Round Trip Time Histogram: The Round Trip Time average of traffic moving across the network in histogram format.
|
||||
- Shaper CPU Utilization: The peak and average CPU utilization of the shaper box over time.
|
||||
- Shaper Memory Utilization: The RAM utilization of the shaper box over time.
|
||||
|
||||
#### Children Tab
|
||||
|
||||
<img width="3840" height="2160" alt="03 child view" src="https://github.com/user-attachments/assets/1f0a0e3c-672b-4982-b334-60a681248b99" />
|
||||
|
||||
- Shaper Child Throughput: The throughput of each top-level child node on the shaper box.
|
||||
- Shaper Child TCP Retransmits: This can be used to spot outliers top-level nodes with regard to Retransmits (packet loss).
|
||||
- Shaper Child Round Trip Time: This can be used to spot outliers top-level nodes with regard to Round Trip Time.
|
||||
|
||||
#### Heatmaps
|
||||
|
||||
<img width="3840" height="2160" alt="04 heatmap" src="https://github.com/user-attachments/assets/f3911ca3-8157-43dc-9f57-1402a6cd0204" />
|
||||
|
||||
This view shows the Round Trip Time, Retransmit, and Capacity heatmaps for the top-level nodes on this shaper box.
|
||||
|
||||
#### Site Rankings
|
||||
|
||||
<img width="3840" height="2160" alt="05 health" src="https://github.com/user-attachments/assets/37e69c41-646c-458f-912d-8556acace102" />
|
||||
|
||||
This tab shows the health of Sites / Access Points / etc across the network in terms of their Round Trip Time, TCP Retransmits, and Capacity in each direction.
|
||||
|
||||
### Node Map
|
||||
|
||||
<img width="3840" height="2160" alt="06 map" src="https://github.com/user-attachments/assets/219d75b3-739a-4f41-86e6-5bc270f22afd" />
|
||||
|
||||
This view allows you to identify the general topology of the network from LibreQoS Insight's perspective. If you hover over the bands between nodes, it will display the current throughput and TCP retransmits for that leg of the network.
|
||||
|
||||
### Libby (AI Assistant)
|
||||
<img width="1784" height="882" alt="07 libby" src="https://github.com/user-attachments/assets/591a3fd1-3946-44ed-a4fd-e1b1d84b9ef6" />
|
||||
|
||||
Libby is LibreQoS Insight's AI Chat Assistant. Libby can help with questions you may have about using LibreQoS or LibreQoS Insight. Libby leverages the official LibreQoS documentation to help you with any questions you may have. Libby can access both live and long-term data - both from synced LibreQoS shaper boxes and from Insight itself. Libby can reply to queries in most major languages with automatic translation.
|
||||
|
||||
You may find new ways to use Libby that we had not originally considered. Please feel welcome to share with us ways you have found Libby to help your workflow.
|
||||
|
||||
### Site Heatmap
|
||||
<img width="3830" height="2160" alt="08 heatmap" src="https://github.com/user-attachments/assets/dfaea245-3221-4cea-874b-fd795ac8da33" />
|
||||
|
||||
This view provides RTT, Retransmit, and Capacity heatmaps for every Access Point, OLT, and Site across your network in one view. This allows you to quickly spot trouble points on your network.
|
||||
|
||||
### Tree History
|
||||
|
||||
<img width="3840" height="2160" alt="09 tree history" src="https://github.com/user-attachments/assets/ea0d0417-c937-41ee-ac4d-7d84c162c6dd" />
|
||||
|
||||
The Tree History is based on the Tree Overview display from the LibreQoS WebUI, and displays the sankey over time to help quickly identify network bottlenecks impacting performance.
|
||||
|
||||
### Reports
|
||||
<img width="3828" height="2160" alt="10 report" src="https://github.com/user-attachments/assets/37f6bd91-8937-4755-a095-6bc38822f544" />
|
||||
|
||||
LibreQoS Insight enables you to trigger AI-generated reports on specific subscriber circuits. These reports draw in from the customer's last 7 days of network activity - including per-ASN performance characteristics, network topology context, and geographic context. The report identifies the User Profile, Key Findings, Critical Issues, Performance Trends, an Upgrade Recommendation, and suggestions for items to manually review.
|
||||
|
||||
### Alerts
|
||||
<img width="3831" height="2160" alt="11 alerts" src="https://github.com/user-attachments/assets/66f0a465-eb00-4bfb-9cf8-c8302af78ead" />
|
||||
|
||||
The Alerts section provides you with automated warnings of out-of-norm performance for nodes across the network (Access Points, OLTs, Sites).
|
||||
@@ -12,20 +12,31 @@ Welcome to the LibreQoS documentation
|
||||
|
||||
docs/v2.0/design
|
||||
docs/v2.0/requirements
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Deploying LibreQoS (v1.5 to v2.0):
|
||||
|
||||
docs/v2.0/prereq
|
||||
docs/v2.0/ubuntu-server
|
||||
docs/v2.0/bridge
|
||||
docs/v2.0/quickstart
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Managing LibreQoS (v1.5 to v2.0):
|
||||
|
||||
docs/v2.0/configuration
|
||||
docs/v2.0/components
|
||||
docs/v2.0/integrations
|
||||
docs/v2.0/components
|
||||
docs/v2.0/update
|
||||
docs/v2.0/troubleshooting
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Insight
|
||||
|
||||
docs/v2.0/insight
|
||||
docs/v2.0/api
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:caption: Misc:
|
||||
|
||||
docs/v2.0/cake
|
||||
docs/v2.0/extras
|
||||
+288
-46
@@ -18,7 +18,7 @@ import psutil
|
||||
import argparse
|
||||
import logging
|
||||
import shutil
|
||||
import binpacking
|
||||
import time
|
||||
from deepdiff import DeepDiff
|
||||
|
||||
from liblqos_python import is_lqosd_alive, clear_ip_mappings, delete_ip_mapping, validate_shaped_devices, \
|
||||
@@ -30,6 +30,13 @@ from liblqos_python import is_lqosd_alive, clear_ip_mappings, delete_ip_mapping,
|
||||
fast_queues_fq_codel, \
|
||||
Bakery
|
||||
|
||||
# Optional: urgent issue submission (available in newer liblqos_python)
|
||||
try:
|
||||
from liblqos_python import submit_urgent_issue # type: ignore
|
||||
except Exception:
|
||||
def submit_urgent_issue(*_args, **_kwargs):
|
||||
return False
|
||||
|
||||
R2Q = 10
|
||||
#MAX_R2Q = 200_000
|
||||
MAX_R2Q = 60_000 # See https://lartc.vger.kernel.narkive.com/NKaH1ZNG/htb-quantum-of-class-100001-is-small-consider-r2q-change
|
||||
@@ -665,29 +672,122 @@ def refreshShapers():
|
||||
}
|
||||
generatedPNs.append(genPNname)
|
||||
if use_bin_packing_to_balance_cpu():
|
||||
print("Using binpacking module to sort circuits by CPU core")
|
||||
weights = get_weights()
|
||||
if weights is not None and dictForCircuitsWithoutParentNodes is not None:
|
||||
for circuitId in dictForCircuitsWithoutParentNodes:
|
||||
for circuit in subscriberCircuits:
|
||||
if 'idForCircuitsWithoutParentNodes' in circuit and circuit['idForCircuitsWithoutParentNodes'] == circuitId:
|
||||
for w in weights:
|
||||
if w.circuit_id == circuit['circuitID']:
|
||||
dictForCircuitsWithoutParentNodes[circuitId] = w.weight
|
||||
bins = binpacking.to_constant_bin_number(dictForCircuitsWithoutParentNodes, numberOfGeneratedPNs)
|
||||
genPNcounter = 0
|
||||
for binItem in bins:
|
||||
sumItem = 0
|
||||
logging.info(generatedPNs[genPNcounter] + " will contain " + str(len(binItem)) + " circuits")
|
||||
for key in binItem.keys():
|
||||
for circuit in subscriberCircuits:
|
||||
if circuit['ParentNode'] == 'none':
|
||||
if circuit['idForCircuitsWithoutParentNodes'] == key:
|
||||
circuit['ParentNode'] = generatedPNs[genPNcounter]
|
||||
genPNcounter += 1
|
||||
if genPNcounter >= queuesAvailable:
|
||||
genPNcounter = 0
|
||||
print("Finished binpacking")
|
||||
print("Using internal planner to sort circuits by CPU core")
|
||||
# Build item list with weights for circuits lacking a ParentNode
|
||||
items = []
|
||||
try:
|
||||
weights = get_weights()
|
||||
except Exception as e:
|
||||
warnings.warn("get_weights() failed; defaulting to equal weights (" + str(e) + ")", stacklevel=2)
|
||||
weights = None
|
||||
weight_by_circuit_id = {}
|
||||
if weights is not None:
|
||||
try:
|
||||
for w in weights:
|
||||
weight_by_circuit_id[str(w.circuit_id)] = float(w.weight)
|
||||
except Exception:
|
||||
pass
|
||||
for circuit in subscriberCircuits:
|
||||
if circuit.get('ParentNode') == 'none' and 'idForCircuitsWithoutParentNodes' in circuit:
|
||||
item_id = circuit['idForCircuitsWithoutParentNodes']
|
||||
# Prefer provided weights; default to 1.0
|
||||
w = dictForCircuitsWithoutParentNodes.get(item_id, 1.0)
|
||||
# If a specific circuit weight exists, prefer it
|
||||
if 'circuitID' in circuit and str(circuit['circuitID']) in weight_by_circuit_id:
|
||||
w = weight_by_circuit_id[str(circuit['circuitID'])]
|
||||
# Ignore placeholder default rates for weight purposes
|
||||
try:
|
||||
default_rate = float(generated_pn_download_mbps())
|
||||
max_dl = float(circuit.get('maxDownload', 0))
|
||||
if abs(max_dl - default_rate) < 1e-6:
|
||||
w = 0.0
|
||||
except Exception:
|
||||
pass
|
||||
items.append({"id": item_id, "weight": float(w)})
|
||||
|
||||
# Prepare bins and capacities
|
||||
bins_list = [{"id": pn} for pn in generatedPNs]
|
||||
capacities = {pn: 1.0 for pn in generatedPNs}
|
||||
|
||||
# Load planner state
|
||||
try:
|
||||
import bin_planner
|
||||
except ImportError:
|
||||
bin_planner = None
|
||||
# Store planner state directly in lqos_directory (no hidden subdirs)
|
||||
state_path = os.path.join(get_libreqos_directory(), "planner_state.json")
|
||||
state = {}
|
||||
if bin_planner is not None:
|
||||
state = bin_planner.load_state(state_path)
|
||||
now_ts = time.time()
|
||||
prev_assign = {}
|
||||
last_change_ts = {}
|
||||
if isinstance(state, dict):
|
||||
prev_assign = state.get("assignments", {}) or {}
|
||||
last_change_ts = state.get("last_change_ts", {}) or {}
|
||||
# Filter previous assignments to only items/bins in this context
|
||||
item_ids = {str(it["id"]) for it in items}
|
||||
valid_bins = set(capacities.keys())
|
||||
prev_assign = {iid: b for iid, b in prev_assign.items() if iid in item_ids and b in valid_bins}
|
||||
last_change_ts = {iid: last_change_ts.get(iid, 0.0) for iid in item_ids}
|
||||
|
||||
# Planner parameters
|
||||
params = {
|
||||
"candidate_set_size": 4,
|
||||
"headroom": 0.05,
|
||||
"alpha": 0.1,
|
||||
"hysteresis_threshold": 0.03,
|
||||
"cooldown_seconds": 3600,
|
||||
"move_budget_per_run": max(1, min(32, int(0.01 * max(1, len(items))))),
|
||||
"salt": state.get("salt", "default_salt") if isinstance(state, dict) else "default_salt",
|
||||
"last_change_ts_by_item": last_change_ts,
|
||||
}
|
||||
if monitor_mode_only() == True:
|
||||
params["move_budget_per_run"] = 0
|
||||
|
||||
if bin_planner is not None:
|
||||
assignments, changed = bin_planner.plan_assignments(
|
||||
items, bins_list, capacities, prev_assign, now_ts, params
|
||||
)
|
||||
else:
|
||||
# Fallback to simple greedy if planner unavailable
|
||||
bin_loads = {pn: 0.0 for pn in generatedPNs}
|
||||
pairs = [(str(it["id"]), float(it["weight"])) for it in items]
|
||||
pairs.sort(key=lambda iw: (-iw[1], str(iw[0])))
|
||||
assignments = {}
|
||||
for item_id, w in pairs:
|
||||
target_pn = min(bin_loads.items(), key=lambda kv: (kv[1], kv[0]))[0]
|
||||
assignments[item_id] = target_pn
|
||||
bin_loads[target_pn] += w
|
||||
changed = list(assignments.keys())
|
||||
|
||||
# Apply assignments to circuits
|
||||
for circuit in subscriberCircuits:
|
||||
if circuit.get('ParentNode') == 'none' and 'idForCircuitsWithoutParentNodes' in circuit:
|
||||
item_id = circuit['idForCircuitsWithoutParentNodes']
|
||||
if item_id in assignments:
|
||||
circuit['ParentNode'] = assignments[item_id]
|
||||
|
||||
# Update and save state
|
||||
if bin_planner is not None and isinstance(state, dict):
|
||||
if state.get("salt") is None:
|
||||
state["salt"] = "default_salt"
|
||||
if "assignments" not in state or not isinstance(state["assignments"], dict):
|
||||
state["assignments"] = {}
|
||||
if "last_change_ts" not in state or not isinstance(state["last_change_ts"], dict):
|
||||
state["last_change_ts"] = {}
|
||||
for iid, b in assignments.items():
|
||||
# record last change time if changed
|
||||
if iid in changed:
|
||||
state["last_change_ts"][iid] = now_ts
|
||||
state["assignments"][iid] = b
|
||||
try:
|
||||
print(f"Saving planner state to {state_path} (generated PNs)")
|
||||
bin_planner.save_state(state_path, state)
|
||||
except Exception as e:
|
||||
warnings.warn(f"Failed to save planner state at {state_path}: {e}", stacklevel=2)
|
||||
|
||||
print("Finished planning generated parent nodes")
|
||||
else:
|
||||
genPNcounter = 0
|
||||
for circuit in subscriberCircuits:
|
||||
@@ -816,6 +916,19 @@ def refreshShapers():
|
||||
return newDict
|
||||
network = flattenA(network, 1)
|
||||
|
||||
# Prepare planner weights (actual weighting may come from Insight or CSV defaults)
|
||||
weight_by_circuit_id = {}
|
||||
try:
|
||||
weights = get_weights()
|
||||
for w in weights:
|
||||
try:
|
||||
weight_by_circuit_id[str(w.circuit_id)] = float(w.weight)
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
# If we can't get weights, leave mapping empty; UI will fall back
|
||||
weight_by_circuit_id = {}
|
||||
|
||||
# Group circuits by parent node. Reduces runtime for section below this one.
|
||||
circuits_by_parent_node = {}
|
||||
circuit_min_down_combined_by_parent_node = {}
|
||||
@@ -916,7 +1029,13 @@ def refreshShapers():
|
||||
minorByCPU[queue] = minorByCPU[queue] + 1
|
||||
# Check for overflow - TC uses u16 for minor class ID (max 65535)
|
||||
if minorByCPU[queue] > 0xFFFF:
|
||||
logging.error(f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds TC's u16 limit (65535). Consider increasing queue count or restructuring network hierarchy.")
|
||||
msg = f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds TC's u16 limit (65535). Consider increasing queue count or restructuring network hierarchy."
|
||||
logging.error(msg)
|
||||
try:
|
||||
ctx = json.dumps({"cpu": queue, "minor": minorByCPU[queue]})
|
||||
submit_urgent_issue("LibreQoS", "Error", "TC_U16_OVERFLOW", msg, ctx, f"TC_U16_OVERFLOW_CPU_{queue}")
|
||||
except Exception:
|
||||
pass
|
||||
raise ValueError(f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds limit of 65535")
|
||||
# If a device from ShapedDevices.csv lists this node as its Parent Node, attach it as a leaf to this node HTB
|
||||
if node in circuits_by_parent_node:
|
||||
@@ -972,6 +1091,29 @@ def refreshShapers():
|
||||
"classMinor": hex(minorByCPU[queue]),
|
||||
"comment": circuit['comment']
|
||||
}
|
||||
# Attach the planner weight used by the planner/UI summary
|
||||
# Priority: explicit weight -> fallback to maxDownload
|
||||
try:
|
||||
cid = str(circuit.get('circuitID',''))
|
||||
w = None
|
||||
if cid in weight_by_circuit_id:
|
||||
w = float(weight_by_circuit_id[cid])
|
||||
if w is None:
|
||||
w = float(maxDownload)
|
||||
# Treat 1000 as a sentinel default from Insight; use maxDownload instead
|
||||
if abs(w - 1000.0) < 1e-6:
|
||||
w = float(maxDownload)
|
||||
# If the circuit's configured rate equals the generated PN default,
|
||||
# ignore it for weight purposes.
|
||||
try:
|
||||
default_rate = float(generated_pn_download_mbps())
|
||||
if abs(float(maxDownload) - default_rate) < 1e-6:
|
||||
w = 0.0
|
||||
except Exception:
|
||||
pass
|
||||
thisNewCircuitItemForNetwork['planner_weight'] = w
|
||||
except Exception:
|
||||
pass
|
||||
# Preserve optional per-circuit SQM override for downstream bakery call
|
||||
if 'sqm' in circuit and circuit['sqm']:
|
||||
thisNewCircuitItemForNetwork['sqm'] = circuit['sqm']
|
||||
@@ -994,7 +1136,13 @@ def refreshShapers():
|
||||
minorByCPU[queue] = minorByCPU[queue] + 1
|
||||
# Check for overflow - TC uses u16 for minor class ID (max 65535)
|
||||
if minorByCPU[queue] > 0xFFFF:
|
||||
logging.error(f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds TC's u16 limit (65535). Consider increasing queue count or restructuring network hierarchy.")
|
||||
msg = f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds TC's u16 limit (65535). Consider increasing queue count or restructuring network hierarchy."
|
||||
logging.error(msg)
|
||||
try:
|
||||
ctx = json.dumps({"cpu": queue, "minor": minorByCPU[queue]})
|
||||
submit_urgent_issue("LibreQoS", "Error", "TC_U16_OVERFLOW", msg, ctx, f"TC_U16_OVERFLOW_CPU_{queue}")
|
||||
except Exception:
|
||||
pass
|
||||
raise ValueError(f"Minor class ID overflow on CPU {queue}: {minorByCPU[queue]} exceeds limit of 65535")
|
||||
minorByCPU = traverseNetwork(sorted_children, depth+1, major, minorByCPU, queue, nodeClassID, upNodeClassID, data[node]['downloadBandwidthMbps'], data[node]['uploadBandwidthMbps'], data[node]['downloadBandwidthMbpsMin'], data[node]['uploadBandwidthMbpsMin'])
|
||||
# If top level node, increment to next queue / cpu core
|
||||
@@ -1008,21 +1156,87 @@ def refreshShapers():
|
||||
return minorByCPU
|
||||
|
||||
# If we're in binpacking mode, we need to sort the network structure a bit
|
||||
bins = []
|
||||
if use_bin_packing_to_balance_cpu() and not is_network_flat():
|
||||
print("BinPacking is enabled, so we're going to sort your network.")
|
||||
cpuBin = {}
|
||||
weights = get_tree_weights()
|
||||
# Stable tie-breaking: sort by (-weight, name) so equal weights have deterministic order
|
||||
print("Planner is enabled, so we're going to sort your network across CPU queues.")
|
||||
# Build items from top-level nodes with weights
|
||||
items = []
|
||||
try:
|
||||
weights = sorted(weights, key=lambda w: (-int(w.weight), str(w.name)))
|
||||
except Exception:
|
||||
pass
|
||||
for w in weights:
|
||||
cpuBin[w.name] = w.weight
|
||||
bins = binpacking.to_constant_bin_number(cpuBin, queuesAvailable)
|
||||
weights = get_tree_weights()
|
||||
except Exception as e:
|
||||
warnings.warn("get_tree_weights() failed; defaulting to equal weights (" + str(e) + ")", stacklevel=2)
|
||||
weights = None
|
||||
weight_by_name = {}
|
||||
if weights is not None:
|
||||
try:
|
||||
for w in weights:
|
||||
weight_by_name[str(w.name)] = float(w.weight)
|
||||
except Exception:
|
||||
pass
|
||||
for node in network:
|
||||
w = weight_by_name.get(str(node), 1.0)
|
||||
items.append({"id": str(node), "weight": float(w)})
|
||||
|
||||
# Prepare bins and capacities
|
||||
cpu_keys = ["CpueQueue" + str(cpu) for cpu in range(queuesAvailable)]
|
||||
bins_list = [{"id": key} for key in cpu_keys]
|
||||
capacities = {key: 1.0 for key in cpu_keys}
|
||||
|
||||
# Load planner state
|
||||
try:
|
||||
import bin_planner
|
||||
except ImportError:
|
||||
bin_planner = None
|
||||
# Store planner state directly in lqos_directory (no hidden subdirs)
|
||||
state_path = os.path.join(get_libreqos_directory(), "planner_state.json")
|
||||
state = {}
|
||||
if bin_planner is not None:
|
||||
state = bin_planner.load_state(state_path)
|
||||
now_ts = time.time()
|
||||
prev_assign = {}
|
||||
last_change_ts = {}
|
||||
if isinstance(state, dict):
|
||||
prev_assign = state.get("assignments", {}) or {}
|
||||
last_change_ts = state.get("last_change_ts", {}) or {}
|
||||
item_ids = {str(it["id"]) for it in items}
|
||||
valid_bins = set(capacities.keys())
|
||||
prev_assign = {iid: b for iid, b in prev_assign.items() if iid in item_ids and b in valid_bins}
|
||||
last_change_ts = {iid: last_change_ts.get(iid, 0.0) for iid in item_ids}
|
||||
|
||||
params = {
|
||||
"candidate_set_size": 4,
|
||||
"headroom": 0.05,
|
||||
"alpha": 0.1,
|
||||
"hysteresis_threshold": 0.03,
|
||||
"cooldown_seconds": 3600,
|
||||
"move_budget_per_run": max(1, min(32, int(0.01 * max(1, len(items))))),
|
||||
"salt": state.get("salt", "default_salt") if isinstance(state, dict) else "default_salt",
|
||||
"last_change_ts_by_item": last_change_ts,
|
||||
}
|
||||
if monitor_mode_only() == True:
|
||||
params["move_budget_per_run"] = 0
|
||||
|
||||
if bin_planner is not None:
|
||||
assignment, changed = bin_planner.plan_assignments(
|
||||
items, bins_list, capacities, prev_assign, now_ts, params
|
||||
)
|
||||
else:
|
||||
# Fallback to simple greedy if planner unavailable
|
||||
bin_loads = {key: 0.0 for key in cpu_keys}
|
||||
pairs = [(str(it["id"]), float(it["weight"])) for it in items]
|
||||
pairs.sort(key=lambda nw: (-nw[1], nw[0]))
|
||||
assignment = {}
|
||||
for name, w in pairs:
|
||||
target = min(bin_loads.items(), key=lambda kv: (kv[1], kv[0]))[0]
|
||||
assignment[name] = target
|
||||
bin_loads[target] += w
|
||||
changed = list(assignment.keys())
|
||||
|
||||
for x in range(queuesAvailable):
|
||||
print("Bin " + str(x) + " = ", bins[x])
|
||||
key = "CpueQueue" + str(x)
|
||||
assigned = [name for name, tgt in assignment.items() if tgt == key]
|
||||
print("Bin " + str(x) + " = ", assigned)
|
||||
|
||||
# Build the binned network structure
|
||||
binnedNetwork = {}
|
||||
for cpu in range(queuesAvailable):
|
||||
cpuKey = "CpueQueue" + str(cpu)
|
||||
@@ -1036,16 +1250,30 @@ def refreshShapers():
|
||||
'name': cpuKey
|
||||
}
|
||||
for node in network:
|
||||
found = False
|
||||
for bin in range(queuesAvailable):
|
||||
if node in bins[bin]:
|
||||
binnedNetwork["CpueQueue" + str(bin)]['children'][node] = network[node]
|
||||
found = True
|
||||
if found == False:
|
||||
newQueueId = queuesAvailable-1
|
||||
binnedNetwork["CpueQueue" + str(newQueueId)]['children'][node] = network[node]
|
||||
tgt = assignment.get(node)
|
||||
if tgt is None:
|
||||
tgt = "CpueQueue" + str(queuesAvailable - 1)
|
||||
binnedNetwork[tgt]['children'][node] = network[node]
|
||||
network = binnedNetwork
|
||||
|
||||
# Update and save state
|
||||
if bin_planner is not None and isinstance(state, dict):
|
||||
if state.get("salt") is None:
|
||||
state["salt"] = "default_salt"
|
||||
if "assignments" not in state or not isinstance(state["assignments"], dict):
|
||||
state["assignments"] = {}
|
||||
if "last_change_ts" not in state or not isinstance(state["last_change_ts"], dict):
|
||||
state["last_change_ts"] = {}
|
||||
for iid, b in assignment.items():
|
||||
if iid in changed:
|
||||
state["last_change_ts"][iid] = now_ts
|
||||
state["assignments"][iid] = b
|
||||
try:
|
||||
print(f"Saving planner state to {state_path} (top-level CPU binning)")
|
||||
bin_planner.save_state(state_path, state)
|
||||
except Exception as e:
|
||||
warnings.warn(f"Failed to save planner state at {state_path}: {e}", stacklevel=2)
|
||||
|
||||
# Here is the actual call to the recursive traverseNetwork() function. finalMinor is not used.
|
||||
minorByCPU = traverseNetwork(network, 0, major=1, minorByCPU=minorByCPUpreloaded, queue=1, parentClassID=None, upParentClassID=None, parentMaxDL=upstream_bandwidth_capacity_download_mbps(), parentMaxUL=upstream_bandwidth_capacity_upload_mbps(), parentMinDL=upstream_bandwidth_capacity_download_mbps(), parentMinUL=upstream_bandwidth_capacity_upload_mbps())
|
||||
|
||||
@@ -1535,9 +1763,23 @@ if __name__ == '__main__':
|
||||
help="Clear ip filters, qdiscs, and xdp setup if any",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
)
|
||||
parser.add_argument(
|
||||
'--planner-reset',
|
||||
help="Delete planner state file before running",
|
||||
action=argparse.BooleanOptionalAction,
|
||||
)
|
||||
args = parser.parse_args()
|
||||
logging.basicConfig(level=args.loglevel)
|
||||
|
||||
if getattr(args, 'planner_reset', False):
|
||||
try:
|
||||
state_path = os.path.join(get_libreqos_directory(), "planner_state.json")
|
||||
if os.path.exists(state_path):
|
||||
os.remove(state_path)
|
||||
print(f"Removed planner state: {state_path}")
|
||||
except Exception as e:
|
||||
print(f"Warning: could not remove planner state: {e}")
|
||||
|
||||
if args.validate:
|
||||
status = validateNetworkAndDevices()
|
||||
elif args.clearrules:
|
||||
|
||||
@@ -3,3 +3,7 @@
|
||||
|
||||
|
||||
See [wiki here](https://github.com/LibreQoE/LibreQoS/wiki/v1.4)
|
||||
|
||||
## Planner (binpacking replacement)
|
||||
|
||||
LibreQoS now uses an internal planner for assigning circuits and nodes to CPU queues and generated parent nodes. The planner favors stable, stateful assignments to minimize churn and reduce rebuilds, replacing the previous external binpacking dependency.
|
||||
@@ -0,0 +1,277 @@
|
||||
"""
|
||||
Internal planner for stable, damped item-to-bin assignments.
|
||||
|
||||
This module exposes a minimal interface initially, to be expanded in later
|
||||
steps. The baseline implementation uses a simple greedy balancing strategy
|
||||
as a placeholder to replace the external binpacking dependency.
|
||||
"""
|
||||
|
||||
from typing import Dict, List, Tuple, Any, Iterable
|
||||
import hashlib
|
||||
import math
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
|
||||
|
||||
def plan_assignments(
|
||||
items: List[Dict[str, Any]],
|
||||
bins: List[Dict[str, Any]],
|
||||
capacities: Dict[str, float],
|
||||
previous_assignments: Dict[str, str],
|
||||
now_ts: float,
|
||||
params: Dict[str, Any],
|
||||
) -> Tuple[Dict[str, str], List[str]]:
|
||||
"""
|
||||
Compute item -> bin assignments with a repair-oriented approach.
|
||||
|
||||
items: list of { "id": str, "weight": float }
|
||||
bins: list of { "id": str }
|
||||
capacities: { bin_id: capacity_float }
|
||||
previous_assignments: { item_id: bin_id }
|
||||
now_ts: current UNIX timestamp
|
||||
params: planner parameters (optional keys):
|
||||
- candidate_set_size: int
|
||||
- headroom: float in [0,1)
|
||||
- alpha: float >= 0
|
||||
- hysteresis_threshold: float >= 0
|
||||
- cooldown_seconds: int >= 0
|
||||
- move_budget_per_run: int >= 0
|
||||
- salt: str
|
||||
- last_change_ts_by_item: { item_id: unix_ts }
|
||||
- movement_penalty_k: float >= 0 (default 0)
|
||||
- penalty_tau_seconds: float > 0 (default 3600)
|
||||
|
||||
Returns (assignments, changed_items)
|
||||
"""
|
||||
# Defaults
|
||||
L = int(params.get("candidate_set_size", 4))
|
||||
headroom = float(params.get("headroom", 0.05))
|
||||
alpha = float(params.get("alpha", 0.1))
|
||||
hysteresis = float(params.get("hysteresis_threshold", 0.03))
|
||||
cooldown_s = int(params.get("cooldown_seconds", 3600))
|
||||
salt = str(params.get("salt", "default_salt"))
|
||||
last_change_ts = params.get("last_change_ts_by_item", {}) or {}
|
||||
penalty_k = float(params.get("movement_penalty_k", 0.0))
|
||||
penalty_tau = float(params.get("penalty_tau_seconds", 3600.0))
|
||||
|
||||
bin_ids: List[str] = [b["id"] for b in bins]
|
||||
cap = {bid: float(capacities.get(bid, 1.0)) for bid in bin_ids}
|
||||
# Avoid zero/negative capacity
|
||||
for bid, c in list(cap.items()):
|
||||
if c <= 0:
|
||||
cap[bid] = 1e-9
|
||||
|
||||
# Normalize items
|
||||
item_list: List[Tuple[str, float]] = []
|
||||
for it in items:
|
||||
it_id = str(it.get("id"))
|
||||
w = float(it.get("weight", 1.0))
|
||||
item_list.append((it_id, w))
|
||||
|
||||
# Current assignment: keep valid previous assignments
|
||||
assignment: Dict[str, str] = {}
|
||||
load: Dict[str, float] = {bid: 0.0 for bid in bin_ids}
|
||||
for it_id, w in item_list:
|
||||
prev = previous_assignments.get(it_id)
|
||||
if prev in cap:
|
||||
assignment[it_id] = prev
|
||||
load[prev] += w
|
||||
|
||||
# Cold start: assign items without previous placement via greedy using HRW candidates
|
||||
missing = [(it_id, w) for it_id, w in item_list if it_id not in assignment]
|
||||
weights_by_bin = {bid: cap[bid] for bid in bin_ids}
|
||||
missing.sort(key=lambda t: (-t[1], t[0]))
|
||||
initially_assigned: List[str] = []
|
||||
for it_id, w in missing:
|
||||
candidates = hrw_candidates(it_id, bin_ids, weights_by_bin, max(L, 1), salt)
|
||||
# Choose the least loaded among candidates; break ties by HRW rank, not lexicographic bin id
|
||||
if candidates:
|
||||
rank = {bid: idx for idx, bid in enumerate(candidates)}
|
||||
tgt = min(candidates, key=lambda bid: (load[bid] / cap[bid], rank[bid]))
|
||||
else:
|
||||
# As a fallback, rank all bins by HRW order to avoid lexicographic bias
|
||||
all_rank_list = hrw_candidates(it_id, bin_ids, weights_by_bin, len(bin_ids), salt)
|
||||
all_rank = {bid: idx for idx, bid in enumerate(all_rank_list)}
|
||||
tgt = min(bin_ids, key=lambda bid: (load[bid] / cap[bid], all_rank.get(bid, 1_000_000)))
|
||||
assignment[it_id] = tgt
|
||||
load[tgt] += w
|
||||
initially_assigned.append(it_id)
|
||||
|
||||
def score(loads: Dict[str, float]) -> float:
|
||||
s = 0.0
|
||||
for bid in bin_ids:
|
||||
denom = cap[bid] * max(1.0 - headroom, 1e-6)
|
||||
lj = loads[bid] / denom
|
||||
over = max(0.0, lj - 1.0)
|
||||
s += over * over
|
||||
# Add a small balancing term
|
||||
if alpha > 0.0:
|
||||
for bid in bin_ids:
|
||||
lj = loads[bid] / cap[bid]
|
||||
s += alpha * (lj * lj)
|
||||
return s
|
||||
|
||||
# Initial score
|
||||
base_score = score(load)
|
||||
|
||||
# Decide moves by considering HRW top-L candidates per item
|
||||
move_budget = params.get("move_budget_per_run")
|
||||
if move_budget is None:
|
||||
move_budget = max(1, min(32, int(0.01 * max(1, len(item_list)))))
|
||||
moves_left = int(move_budget)
|
||||
|
||||
# Worklist: heavier items first
|
||||
work = sorted(item_list, key=lambda t: (-t[1], t[0]))
|
||||
changed_items: List[str] = []
|
||||
|
||||
for it_id, w in work:
|
||||
if moves_left <= 0:
|
||||
break
|
||||
cur = assignment.get(it_id)
|
||||
candidates = hrw_candidates(it_id, bin_ids, weights_by_bin, max(L, 1), salt)
|
||||
# Ensure current bin is considered
|
||||
if cur and cur not in candidates:
|
||||
candidates.append(cur)
|
||||
best_gain = 0.0
|
||||
best_tgt = cur
|
||||
for tgt in candidates:
|
||||
if tgt == cur:
|
||||
continue
|
||||
# simulate move
|
||||
if cur:
|
||||
load[cur] -= w
|
||||
load[tgt] += w
|
||||
new_score = score(load)
|
||||
# revert
|
||||
load[tgt] -= w
|
||||
if cur:
|
||||
load[cur] += w
|
||||
gain = base_score - new_score
|
||||
if gain > best_gain:
|
||||
best_gain = gain
|
||||
best_tgt = tgt
|
||||
|
||||
if best_tgt and best_tgt != cur:
|
||||
# Cooldown check
|
||||
last_ts = float(last_change_ts.get(it_id, 0.0))
|
||||
age = max(0.0, float(now_ts) - last_ts)
|
||||
if age < float(cooldown_s):
|
||||
continue
|
||||
# Movement penalty
|
||||
penalty = 0.0
|
||||
if penalty_k > 0.0 and penalty_tau > 0.0:
|
||||
penalty = penalty_k * float(w) * math.exp(-age / penalty_tau)
|
||||
net_gain = best_gain - penalty
|
||||
# Hysteresis threshold: relative to base_score magnitude or absolute small value if base is tiny
|
||||
threshold = max(hysteresis * max(1.0, base_score), 1e-6)
|
||||
if net_gain >= threshold:
|
||||
# apply move
|
||||
if cur:
|
||||
load[cur] -= w
|
||||
load[best_tgt] += w
|
||||
assignment[it_id] = best_tgt
|
||||
changed_items.append(it_id)
|
||||
# update base score and budget
|
||||
base_score -= best_gain
|
||||
moves_left -= 1
|
||||
|
||||
# Mark initial placements as changes so callers can timestamp them
|
||||
for iid in initially_assigned:
|
||||
if iid not in changed_items:
|
||||
changed_items.append(iid)
|
||||
|
||||
return assignment, changed_items
|
||||
|
||||
|
||||
def load_state(path: str) -> Dict[str, Any]:
|
||||
"""
|
||||
Load planner state from JSON. If missing or invalid, return a fresh state.
|
||||
"""
|
||||
fresh = {
|
||||
"algo_version": "v1",
|
||||
"salt": None,
|
||||
"assignments": {},
|
||||
"last_change_ts": {},
|
||||
"updated_at": None,
|
||||
}
|
||||
try:
|
||||
if not os.path.exists(path):
|
||||
return fresh
|
||||
with open(path, 'r', encoding='utf-8') as f:
|
||||
data = json.load(f)
|
||||
# Minimal validation
|
||||
if not isinstance(data, dict):
|
||||
return fresh
|
||||
for k in ["algo_version", "salt", "assignments", "last_change_ts", "updated_at"]:
|
||||
if k not in data:
|
||||
data[k] = fresh.get(k)
|
||||
if data.get("salt") is None:
|
||||
data["salt"] = _generate_salt()
|
||||
# TTL: invalidate if older than 24 hours
|
||||
try:
|
||||
updated_at = float(data.get("updated_at") or 0.0)
|
||||
if updated_at <= 0.0:
|
||||
return data
|
||||
now = time.time()
|
||||
if (now - updated_at) > 24 * 3600:
|
||||
# Regenerate: return a fresh state with a new salt
|
||||
fresh["salt"] = _generate_salt()
|
||||
return fresh
|
||||
except Exception:
|
||||
# If parsing fails, return fresh
|
||||
fresh["salt"] = _generate_salt()
|
||||
return fresh
|
||||
return data
|
||||
except Exception:
|
||||
return fresh
|
||||
|
||||
|
||||
def save_state(path: str, state: Dict[str, Any]) -> None:
|
||||
"""
|
||||
Save planner state to JSON. Creates parent directory if necessary.
|
||||
"""
|
||||
try:
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
except Exception:
|
||||
pass
|
||||
state = dict(state)
|
||||
state["updated_at"] = time.time()
|
||||
if state.get("salt") is None:
|
||||
state["salt"] = _generate_salt()
|
||||
with open(path, 'w', encoding='utf-8') as f:
|
||||
json.dump(state, f, indent=2, sort_keys=True)
|
||||
|
||||
|
||||
def _generate_salt() -> str:
|
||||
return hashlib.sha256(str(time.time_ns()).encode('utf-8')).hexdigest()
|
||||
|
||||
|
||||
def hrw_candidates(
|
||||
item_id: str,
|
||||
bin_ids: List[str],
|
||||
weights_by_bin: Dict[str, float],
|
||||
L: int,
|
||||
salt: str,
|
||||
) -> List[str]:
|
||||
"""
|
||||
Weighted Rendezvous (HRW) hashing: return top-L candidate bins for an item.
|
||||
|
||||
score(bin) = weight(bin) / -ln(U), where U derives deterministically from
|
||||
SHA-256(salt || item_id || bin_id).
|
||||
"""
|
||||
scores: List[Tuple[float, str]] = []
|
||||
for bid in bin_ids:
|
||||
w = float(weights_by_bin.get(bid, 1.0))
|
||||
# Deterministic uniform in (0,1)
|
||||
h = hashlib.sha256((salt + '|' + str(item_id) + '|' + str(bid)).encode('utf-8')).digest()
|
||||
# Use first 8 bytes for a 64-bit integer
|
||||
r = int.from_bytes(h[:8], byteorder='big', signed=False)
|
||||
# Map to (0,1). Avoid 0 by adding a tiny epsilon.
|
||||
U = max((r / 2**64), 1e-12)
|
||||
score = w / (-math.log(U)) if U < 1.0 else float('inf')
|
||||
scores.append((score, bid))
|
||||
scores.sort(key=lambda t: (-t[0], t[1]))
|
||||
if L <= 0:
|
||||
return []
|
||||
return [bid for _, bid in scores[:min(L, len(scores))]]
|
||||
+4
-3
@@ -24,6 +24,7 @@ LQOS_FILES=(
|
||||
integrationRestHttp.py
|
||||
integrationSonar.py
|
||||
integrationSplynx.py
|
||||
bin_planner.py
|
||||
integrationUISP.py
|
||||
LibreQoS.py
|
||||
lqos.example
|
||||
@@ -113,9 +114,9 @@ pushd /opt/libreqos
|
||||
PIP_BREAK_SYSTEM_PACKAGES=1 python3 -m pip install -r src/requirements.txt
|
||||
sudo PIP_BREAK_SYSTEM_PACKAGES=1 python3 -m pip install -r src/requirements.txt
|
||||
# - Setup Python dependencies as a post-install task - handle issue with packages on Ubuntu Server 24.04
|
||||
sudo PIP_BREAK_SYSTEM_PACKAGES=1 pip uninstall binpacking apscheduler deepdiff --yes
|
||||
PIP_BREAK_SYSTEM_PACKAGES=1 pip uninstall binpacking apscheduler deepdiff --yes
|
||||
sudo PIP_BREAK_SYSTEM_PACKAGES=1 pip install binpacking apscheduler deepdiff
|
||||
sudo PIP_BREAK_SYSTEM_PACKAGES=1 pip uninstall apscheduler deepdiff --yes
|
||||
PIP_BREAK_SYSTEM_PACKAGES=1 pip uninstall apscheduler deepdiff --yes
|
||||
sudo PIP_BREAK_SYSTEM_PACKAGES=1 pip install apscheduler deepdiff
|
||||
|
||||
# Ensure folder permissions are correct post-install
|
||||
sudo chown -R $USER /opt/libreqos
|
||||
|
||||
@@ -98,6 +98,14 @@ popd > /dev/null || exit
|
||||
cp rust/target/$TARGET/liblqos_python.so ./liblqos_python.so.new
|
||||
mv liblqos_python.so.new liblqos_python.so
|
||||
|
||||
# Ensure the internal planner module is present for LibreQoS.py
|
||||
if [ ! -f "bin_planner.py" ]; then
|
||||
echo "Warning: bin_planner.py not found in $(pwd). The planner will be unavailable to LibreQoS.py."
|
||||
echo "If you are updating a running system, make sure bin_planner.py is deployed alongside LibreQoS.py."
|
||||
else
|
||||
echo "bin_planner.py present."
|
||||
fi
|
||||
|
||||
# Update the lqos_api binary
|
||||
echo "Updating lqos_api binary..."
|
||||
bash ./update_api.sh || echo "Warning: Failed to update lqos_api (continuing)."
|
||||
|
||||
@@ -11,8 +11,8 @@ mod unix_socket_server;
|
||||
pub use client::{LibreqosBusClient, bus_request};
|
||||
pub use queue_data::*;
|
||||
pub use reply::BusReply;
|
||||
pub use request::{BlackboardSystem, BusRequest, TopFlowType};
|
||||
pub use response::BusResponse;
|
||||
pub use request::{BlackboardSystem, BusRequest, TopFlowType, UrgentSeverity, UrgentSource};
|
||||
pub use response::{BakeryStatsSnapshot, BusResponse, UrgentIssue};
|
||||
pub use session::BusSession;
|
||||
use thiserror::Error;
|
||||
use tracing::error;
|
||||
|
||||
@@ -6,6 +6,28 @@ use allocative::Allocative;
|
||||
use lqos_config::Tunables;
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Source system for an urgent issue
|
||||
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq, Copy, Allocative)]
|
||||
pub enum UrgentSource {
|
||||
/// Raised by the scheduler process
|
||||
Scheduler,
|
||||
/// Raised by the LibreQoS Python orchestrator
|
||||
LibreQoS,
|
||||
/// Raised by the local API server
|
||||
API,
|
||||
/// Raised by lqosd or other components
|
||||
System,
|
||||
}
|
||||
|
||||
/// Severity of an urgent issue
|
||||
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, Eq, Copy, Allocative)]
|
||||
pub enum UrgentSeverity {
|
||||
/// Error requires attention
|
||||
Error,
|
||||
/// Warning is informative/high-visibility
|
||||
Warning,
|
||||
}
|
||||
|
||||
/// One or more `BusRequest` objects must be included in a `BusSession`
|
||||
/// request. Each `BusRequest` represents a single request for action
|
||||
/// or data.
|
||||
@@ -26,6 +48,14 @@ pub enum BusRequest {
|
||||
end: u32,
|
||||
},
|
||||
|
||||
/// Retrieve the top N uploads by bandwidth use.
|
||||
GetTopNUploaders {
|
||||
/// First row to retrieve (usually 0 unless you are paging)
|
||||
start: u32,
|
||||
/// Last row to retrieve (10 for top-10 starting at 0)
|
||||
end: u32,
|
||||
},
|
||||
|
||||
/// Retrieves the TopN hosts with the worst RTT, sorted by RTT descending.
|
||||
GetWorstRtt {
|
||||
/// First row to retrieve (usually 0 unless you are paging)
|
||||
@@ -308,6 +338,28 @@ pub enum BusRequest {
|
||||
|
||||
/// Check the scheduler status
|
||||
CheckSchedulerStatus,
|
||||
|
||||
/// Submit an urgent issue for high-priority operator visibility
|
||||
SubmitUrgentIssue {
|
||||
/// Source of the issue
|
||||
source: UrgentSource,
|
||||
/// Severity of the issue
|
||||
severity: UrgentSeverity,
|
||||
/// Machine-readable code for the issue (e.g. TC_U16_OVERFLOW)
|
||||
code: String,
|
||||
/// Human-readable message for display
|
||||
message: String,
|
||||
/// Optional JSON context payload
|
||||
context: Option<String>,
|
||||
/// Optional key to deduplicate repeated submissions
|
||||
dedupe_key: Option<String>,
|
||||
},
|
||||
|
||||
/// Retrieve current urgent issues
|
||||
GetUrgentIssues,
|
||||
|
||||
/// Clear a specific urgent issue by ID
|
||||
ClearUrgentIssue(u64),
|
||||
}
|
||||
|
||||
/// Defines the parts of the blackboard
|
||||
|
||||
@@ -11,6 +11,26 @@ use lqos_utils::units::DownUpOrder;
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::net::IpAddr;
|
||||
|
||||
/// An urgent issue to be displayed prominently in the UI
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Allocative)]
|
||||
pub struct UrgentIssue {
|
||||
/// Unique identifier
|
||||
pub id: u64,
|
||||
/// Unix timestamp (seconds)
|
||||
pub ts: u64,
|
||||
/// Source component
|
||||
pub source: crate::bus::request::UrgentSource,
|
||||
/// Severity level
|
||||
pub severity: crate::bus::request::UrgentSeverity,
|
||||
/// Machine-readable code (e.g., TC_U16_OVERFLOW)
|
||||
pub code: String,
|
||||
/// Human-readable message
|
||||
pub message: String,
|
||||
/// Optional JSON context
|
||||
pub context: Option<String>,
|
||||
/// Optional dedupe key
|
||||
pub dedupe_key: Option<String>,
|
||||
}
|
||||
/// Serializable snapshot of BakeryStats for bus transmission
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Allocative)]
|
||||
pub struct BakeryStatsSnapshot {
|
||||
@@ -61,6 +81,9 @@ pub enum BusResponse {
|
||||
/// Provides the Top N downloaders IP stats.
|
||||
TopDownloaders(Vec<IpStats>),
|
||||
|
||||
/// Provides the Top N uploaders IP stats.
|
||||
TopUploaders(Vec<IpStats>),
|
||||
|
||||
/// Provides the worst N RTT scores, sorted in descending order.
|
||||
WorstRtt(Vec<IpStats>),
|
||||
|
||||
@@ -187,4 +210,7 @@ pub enum BusResponse {
|
||||
/// Any error message from integrations
|
||||
error: Option<String>,
|
||||
},
|
||||
|
||||
/// List of urgent issues
|
||||
UrgentIssues(Vec<UrgentIssue>),
|
||||
}
|
||||
@@ -21,11 +21,11 @@ pub use ip_stats::{
|
||||
tos_parser,
|
||||
};
|
||||
mod tc_handle;
|
||||
pub use bus::response::BakeryStatsSnapshot;
|
||||
pub use bus::response::{BakeryStatsSnapshot, UrgentIssue};
|
||||
pub use bus::{
|
||||
BUS_SOCKET_PATH, BlackboardSystem, BusReply, BusRequest, BusResponse, BusSession,
|
||||
CakeDiffTinTransit, CakeDiffTransit, CakeTransit, LibreqosBusClient, QueueStoreTransit,
|
||||
TopFlowType, UnixSocketServer, bus_request,
|
||||
TopFlowType, UnixSocketServer, UrgentSeverity, UrgentSource, bus_request,
|
||||
};
|
||||
pub use tc_handle::TcHandle;
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
#![allow(non_local_definitions)] // Temporary: rewrite required for much of this, for newer PyO3.
|
||||
#![allow(unsafe_op_in_unsafe_fn)]
|
||||
use lqos_bus::{BlackboardSystem, BusRequest, BusResponse, TcHandle};
|
||||
use lqos_bus::{BlackboardSystem, BusRequest, BusResponse, TcHandle, UrgentSource, UrgentSeverity};
|
||||
use lqos_utils::hex_string::read_hex_string;
|
||||
use nix::libc::getpid;
|
||||
use pyo3::exceptions::PyOSError;
|
||||
@@ -117,6 +117,7 @@ fn liblqos_python(m: &Bound<'_, PyModule>) -> PyResult<()> {
|
||||
m.add_function(wrap_pyfunction!(calculate_hash, m)?)?;
|
||||
m.add_function(wrap_pyfunction!(scheduler_alive, m)?)?;
|
||||
m.add_function(wrap_pyfunction!(scheduler_error, m)?)?;
|
||||
m.add_function(wrap_pyfunction!(submit_urgent_issue, m)?)?;
|
||||
|
||||
m.add_class::<Bakery>()?;
|
||||
Ok(())
|
||||
@@ -1170,3 +1171,49 @@ fn scheduler_error(_py: Python, error: String) -> PyResult<bool> {
|
||||
}
|
||||
Ok(false)
|
||||
}
|
||||
|
||||
/// Submit an urgent issue for prominent display in the Node Manager UI.
|
||||
///
|
||||
/// Parameters:
|
||||
/// - source: one of "Scheduler", "LibreQoS", "API", "System"
|
||||
/// - severity: "Error" or "Warning"
|
||||
/// - code: short machine-readable code (e.g., "TC_U16_OVERFLOW")
|
||||
/// - message: human-readable description
|
||||
/// - context: optional JSON string with extra details
|
||||
/// - dedupe_key: optional key to deduplicate repeats (e.g., code+cpu)
|
||||
#[pyfunction]
|
||||
fn submit_urgent_issue(
|
||||
_py: Python,
|
||||
source: String,
|
||||
severity: String,
|
||||
code: String,
|
||||
message: String,
|
||||
context: Option<String>,
|
||||
dedupe_key: Option<String>,
|
||||
) -> PyResult<bool> {
|
||||
let src = match source.to_ascii_lowercase().as_str() {
|
||||
"scheduler" => UrgentSource::Scheduler,
|
||||
"libreqos" => UrgentSource::LibreQoS,
|
||||
"api" => UrgentSource::API,
|
||||
_ => UrgentSource::System,
|
||||
};
|
||||
let sev = match severity.to_ascii_lowercase().as_str() {
|
||||
"warning" => UrgentSeverity::Warning,
|
||||
_ => UrgentSeverity::Error,
|
||||
};
|
||||
if let Ok(reply) = run_query(vec![BusRequest::SubmitUrgentIssue {
|
||||
source: src,
|
||||
severity: sev,
|
||||
code,
|
||||
message,
|
||||
context,
|
||||
dedupe_key,
|
||||
}]) {
|
||||
for resp in reply.iter() {
|
||||
if let BusResponse::Ack = resp {
|
||||
return Ok(true);
|
||||
}
|
||||
}
|
||||
}
|
||||
Ok(false)
|
||||
}
|
||||
@@ -19,6 +19,7 @@ mod stats;
|
||||
mod system_stats;
|
||||
mod throughput_tracker;
|
||||
mod tool_status;
|
||||
mod urgent;
|
||||
mod tuning;
|
||||
mod validation;
|
||||
mod version_checks;
|
||||
@@ -355,6 +356,9 @@ fn handle_bus_requests(requests: &[BusRequest], responses: &mut Vec<BusResponse>
|
||||
BusRequest::GetTopNDownloaders { start, end } => {
|
||||
throughput_tracker::top_n(*start, *end)
|
||||
}
|
||||
BusRequest::GetTopNUploaders { start, end } => {
|
||||
throughput_tracker::top_n_up(*start, *end)
|
||||
}
|
||||
BusRequest::GetWorstRtt { start, end } => throughput_tracker::worst_n(*start, *end),
|
||||
BusRequest::GetWorstRetransmits { start, end } => {
|
||||
throughput_tracker::worst_n_retransmits(*start, *end)
|
||||
@@ -639,6 +643,18 @@ fn handle_bus_requests(requests: &[BusRequest], responses: &mut Vec<BusResponse>
|
||||
let error = tool_status::scheduler_error_message();
|
||||
BusResponse::SchedulerStatus { running, error }
|
||||
}
|
||||
BusRequest::SubmitUrgentIssue { source, severity, code, message, context, dedupe_key } => {
|
||||
urgent::submit(*source, *severity, code.clone(), message.clone(), context.clone(), dedupe_key.clone());
|
||||
BusResponse::Ack
|
||||
}
|
||||
BusRequest::GetUrgentIssues => {
|
||||
let list = urgent::list();
|
||||
BusResponse::UrgentIssues(list)
|
||||
}
|
||||
BusRequest::ClearUrgentIssue(id) => {
|
||||
urgent::clear(*id);
|
||||
BusResponse::Ack
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
scripts=( index.js template.js login.js first-run.js shaped-devices.js tree.js help.js unknown-ips.js configuration.js circuit.js flow_map.js all_tree_sankey.js asn_explorer.js lts_trial.js config_general.js config_tuning.js config_queues.js config_stormguard.js config_lts.js config_iprange.js config_flows.js config_integration.js config_spylnx.js config_netzur.js config_uisp.js config_powercode.js config_sonar.js config_interface.js config_network.js config_devices.js config_users.js config_wispgate.js chatbot.js)
|
||||
scripts=( index.js template.js login.js first-run.js shaped-devices.js tree.js help.js unknown-ips.js configuration.js circuit.js flow_map.js all_tree_sankey.js asn_explorer.js lts_trial.js config_general.js config_tuning.js config_queues.js config_stormguard.js config_lts.js config_iprange.js config_flows.js config_integration.js config_spylnx.js config_netzur.js config_uisp.js config_powercode.js config_sonar.js config_interface.js config_network.js config_devices.js config_users.js config_wispgate.js chatbot.js cpu_weights.js)
|
||||
for script in "${scripts[@]}"
|
||||
do
|
||||
echo "Building {$script}"
|
||||
|
||||
@@ -52,10 +52,22 @@ export function saveNetworkAndDevices(network_json, shaped_devices, onComplete)
|
||||
validationErrors.push(`Device ${index + 1}: Parent node '${device.parent_node}' does not exist`);
|
||||
}
|
||||
|
||||
// Bandwidth validation
|
||||
if (device.download_min_mbps < 1 || device.upload_min_mbps < 1 ||
|
||||
device.download_max_mbps < 1 || device.upload_max_mbps < 1) {
|
||||
validationErrors.push(`Device ${index + 1}: Bandwidth values must be greater than 0`);
|
||||
// Bandwidth validation (supports fractional Mbps)
|
||||
// Minimums must be >= 0.1 Mbps, maximums must be >= 0.2 Mbps
|
||||
const dmin = parseFloat(device.download_min_mbps);
|
||||
const umin = parseFloat(device.upload_min_mbps);
|
||||
const dmax = parseFloat(device.download_max_mbps);
|
||||
const umax = parseFloat(device.upload_max_mbps);
|
||||
|
||||
if (Number.isNaN(dmin) || Number.isNaN(umin) || Number.isNaN(dmax) || Number.isNaN(umax)) {
|
||||
validationErrors.push(`Device ${index + 1}: One or more bandwidth fields are not valid numbers`);
|
||||
} else {
|
||||
if (dmin < 0.1 || umin < 0.1) {
|
||||
validationErrors.push(`Device ${index + 1}: Min rates must be >= 0.1 Mbps`);
|
||||
}
|
||||
if (dmax < 0.2 || umax < 0.2) {
|
||||
validationErrors.push(`Device ${index + 1}: Max rates must be >= 0.2 Mbps`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
@@ -71,51 +83,26 @@ export function saveNetworkAndDevices(network_json, shaped_devices, onComplete)
|
||||
};
|
||||
console.log(submission);
|
||||
|
||||
// Send to server with enhanced error handling
|
||||
/*$.ajax({
|
||||
// Send to server and handle plain-text "Ok" response
|
||||
$.ajax({
|
||||
type: "POST",
|
||||
url: "/local-api/updateNetworkAndDevices",
|
||||
contentType: 'application/json',
|
||||
data: JSON.stringify(submission),
|
||||
dataType: 'json', // Expect JSON response
|
||||
success: (response) => {
|
||||
try {
|
||||
if (response && response.success) {
|
||||
if (onComplete) onComplete(true, "Saved successfully");
|
||||
} else {
|
||||
const msg = response?.message || "Unknown error occurred";
|
||||
if (onComplete) onComplete(false, msg);
|
||||
alert("Failed to save: " + msg);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error("Error parsing response:", e);
|
||||
if (onComplete) onComplete(false, "Invalid server response");
|
||||
alert("Invalid server response format");
|
||||
}
|
||||
success: (data) => {
|
||||
if (onComplete) onComplete(data === "Ok", data);
|
||||
},
|
||||
error: (xhr) => {
|
||||
let errorMsg = "Request failed";
|
||||
try {
|
||||
if (xhr.responseText) {
|
||||
const json = JSON.parse(xhr.responseText);
|
||||
errorMsg = json.message || xhr.responseText;
|
||||
} else if (xhr.statusText) {
|
||||
errorMsg = xhr.statusText;
|
||||
}
|
||||
console.error("AJAX Error:", {
|
||||
status: xhr.status,
|
||||
statusText: xhr.statusText,
|
||||
response: xhr.responseText
|
||||
});
|
||||
} catch (e) {
|
||||
console.error("Error parsing error response:", e);
|
||||
errorMsg = "Unknown error occurred";
|
||||
}
|
||||
|
||||
const errorMsg = xhr.responseText || xhr.statusText || "Request failed";
|
||||
console.error("AJAX Error:", {
|
||||
status: xhr.status,
|
||||
statusText: xhr.statusText,
|
||||
response: xhr.responseText
|
||||
});
|
||||
if (onComplete) onComplete(false, errorMsg);
|
||||
alert("Error saving configuration: " + errorMsg);
|
||||
}
|
||||
});*/
|
||||
});
|
||||
}
|
||||
|
||||
export function validNodeList(network_json) {
|
||||
|
||||
@@ -640,7 +640,7 @@ function makeSheetBox(rowId, boxId, value, small=false) {
|
||||
}
|
||||
|
||||
function makeSheetNumberBox(rowId, boxId, value) {
|
||||
let html = "<td style='padding: 0px'><input id='" + rowPrefix(rowId, boxId) + "' type=\"number\" value=\"" + value + "\" style='width: 100px; font-size: 8pt;'></input></td>"
|
||||
let html = "<td style='padding: 0px'><input id='" + rowPrefix(rowId, boxId) + "' type=\"number\" step=\"any\" value=\"" + value + "\" style='width: 100px; font-size: 8pt;'></input></td>"
|
||||
return html;
|
||||
}
|
||||
|
||||
@@ -1026,56 +1026,56 @@ function validateSd() {
|
||||
// Download Min
|
||||
controlId = "#" + rowPrefix(i, "download_min_mbps");
|
||||
let download_min = $(controlId).val();
|
||||
download_min = parseInt(download_min);
|
||||
download_min = parseFloat(download_min);
|
||||
if (isNaN(download_min)) {
|
||||
valid = false;
|
||||
errors.push("Download min is not a valid number");
|
||||
$(controlId).addClass("invalid");
|
||||
} else if (download_min < 1) {
|
||||
} else if (download_min < 0.1) {
|
||||
valid = false;
|
||||
errors.push("Download min must be 1 or more");
|
||||
errors.push("Download min must be 0.1 or more");
|
||||
$(controlId).addClass("invalid");
|
||||
}
|
||||
|
||||
// Upload Min
|
||||
controlId = "#" + rowPrefix(i, "upload_min_mbps");
|
||||
let upload_min = $(controlId).val();
|
||||
upload_min = parseInt(upload_min);
|
||||
upload_min = parseFloat(upload_min);
|
||||
if (isNaN(upload_min)) {
|
||||
valid = false;
|
||||
errors.push("Upload min is not a valid number");
|
||||
$(controlId).addClass("invalid");
|
||||
} else if (upload_min < 1) {
|
||||
} else if (upload_min < 0.1) {
|
||||
valid = false;
|
||||
errors.push("Upload min must be 1 or more");
|
||||
errors.push("Upload min must be 0.1 or more");
|
||||
$(controlId).addClass("invalid");
|
||||
}
|
||||
|
||||
// Download Max
|
||||
controlId = "#" + rowPrefix(i, "download_max_mbps");
|
||||
let download_max = $(controlId).val();
|
||||
upload_min = parseInt(download_max);
|
||||
download_max = parseFloat(download_max);
|
||||
if (isNaN(download_max)) {
|
||||
valid = false;
|
||||
errors.push("Download Max is not a valid number");
|
||||
$(controlId).addClass("invalid");
|
||||
} else if (download_max < 1) {
|
||||
} else if (download_max < 0.2) {
|
||||
valid = false;
|
||||
errors.push("Download Max must be 1 or more");
|
||||
errors.push("Download Max must be 0.2 or more");
|
||||
$(controlId).addClass("invalid");
|
||||
}
|
||||
|
||||
// Upload Max
|
||||
controlId = "#" + rowPrefix(i, "upload_max_mbps");
|
||||
let upload_max = $(controlId).val();
|
||||
upload_min = parseInt(upload_max);
|
||||
upload_max = parseFloat(upload_max);
|
||||
if (isNaN(upload_max)) {
|
||||
valid = false;
|
||||
errors.push("Upload Max is not a valid number");
|
||||
$(controlId).addClass("invalid");
|
||||
} else if (upload_max < 1) {
|
||||
} else if (upload_max < 0.2) {
|
||||
valid = false;
|
||||
errors.push("Upload Max must be 1 or more");
|
||||
errors.push("Upload Max must be 0.2 or more");
|
||||
$(controlId).addClass("invalid");
|
||||
}
|
||||
}
|
||||
@@ -1123,10 +1123,10 @@ function saveNetAndDevices() {
|
||||
row.mac = $("#" + rowPrefix(i, "mac")).val();
|
||||
row.ipv4 = ipAddressesToTuple($("#" + rowPrefix(i, "ipv4")).val());
|
||||
row.ipv6 = ipAddressesToTuple($("#" + rowPrefix(i, "ipv6")).val());
|
||||
row.download_min_mbps = parseInt($("#" + rowPrefix(i, "download_min_mbps")).val());
|
||||
row.upload_min_mbps = parseInt($("#" + rowPrefix(i, "upload_min_mbps")).val());
|
||||
row.download_max_mbps = parseInt($("#" + rowPrefix(i, "download_max_mbps")).val());
|
||||
row.upload_max_mbps = parseInt($("#" + rowPrefix(i, "upload_max_mbps")).val());
|
||||
row.download_min_mbps = parseFloat($("#" + rowPrefix(i, "download_min_mbps")).val());
|
||||
row.upload_min_mbps = parseFloat($("#" + rowPrefix(i, "upload_min_mbps")).val());
|
||||
row.download_max_mbps = parseFloat($("#" + rowPrefix(i, "download_max_mbps")).val());
|
||||
row.upload_max_mbps = parseFloat($("#" + rowPrefix(i, "upload_max_mbps")).val());
|
||||
row.comment = $("#" + rowPrefix(i, "comment")).val();
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,307 @@
|
||||
import { clearDiv, theading, simpleRow, simpleRowHtml, enableTooltips } from "./helpers/builders";
|
||||
import { DashboardGraph } from "./graphs/dashboard_graph";
|
||||
|
||||
let state = {
|
||||
selectedCpu: null,
|
||||
direction: "down",
|
||||
page: 1,
|
||||
pageSize: 50,
|
||||
plannerEnabled: null,
|
||||
};
|
||||
|
||||
let dlMaxPie = null; // dashboard-graph wrapped chart for DL Max distribution
|
||||
|
||||
function fmtMbps(x) {
|
||||
if (x === null || x === undefined) return "-";
|
||||
// keep up to 2 decimals, trim trailing zeros
|
||||
let s = Number(x).toFixed(2);
|
||||
s = s.replace(/\.00$/, "");
|
||||
return s;
|
||||
}
|
||||
|
||||
function renderSummary(data) {
|
||||
const target = document.getElementById("cpuSummary");
|
||||
clearDiv(target);
|
||||
if (!data || !Array.isArray(data) || data.length === 0) {
|
||||
target.innerHTML = '<p class="text-muted">No queue structure found. Run LibreQoS to generate queuingStructure.json.</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
// Update binpacking badge in header bar
|
||||
const badge = document.getElementById('binpackBadge');
|
||||
const enabled = !!state.plannerEnabled;
|
||||
if (badge) {
|
||||
badge.className = `badge ${enabled ? 'bg-success' : 'bg-secondary'}`;
|
||||
badge.textContent = enabled ? "Binpacking: Enabled" : "Binpacking: Disabled";
|
||||
}
|
||||
// Optional Insight promo below summary
|
||||
if (!window.hasInsight) {
|
||||
const promo = document.createElement("div");
|
||||
promo.className = "text-muted small mb-2";
|
||||
promo.innerHTML = `Enable <a href="lts_trial.html">Insight</a> to make binpacking smarter with historical and recent data.`;
|
||||
target.appendChild(promo);
|
||||
}
|
||||
|
||||
// Build a row with table (left) and chart (right)
|
||||
const row = document.createElement('div');
|
||||
row.className = 'row';
|
||||
const left = document.createElement('div');
|
||||
left.className = 'col-12 col-lg-8';
|
||||
const right = document.createElement('div');
|
||||
right.className = 'col-12 col-lg-4';
|
||||
|
||||
const table = document.createElement("table");
|
||||
table.classList.add("table", "table-striped", "table-sm");
|
||||
const thead = document.createElement("thead");
|
||||
thead.appendChild(theading("CPU"));
|
||||
thead.appendChild(theading("DL Circuits"));
|
||||
thead.appendChild(theading("Weight"));
|
||||
thead.appendChild(theading("DL Max (Mbps)"));
|
||||
thead.appendChild(theading("Actions"));
|
||||
table.appendChild(thead);
|
||||
const tbody = document.createElement("tbody");
|
||||
|
||||
// helper to build a centered cell
|
||||
const centerCell = (text) => {
|
||||
const td = simpleRow(String(text));
|
||||
td.classList.add('text-center');
|
||||
return td;
|
||||
};
|
||||
|
||||
// simple weight formatter
|
||||
const fmtWeight = (x) => {
|
||||
const n = Number(x || 0);
|
||||
return n > 0 ? n.toLocaleString() : '-';
|
||||
};
|
||||
|
||||
data.forEach((row) => {
|
||||
const tr = document.createElement("tr");
|
||||
tr.appendChild(centerCell(row.cpu));
|
||||
tr.appendChild(centerCell(row.download.circuits));
|
||||
tr.appendChild(centerCell(fmtWeight(row.download && row.download.weight_sum)));
|
||||
tr.appendChild(centerCell(fmtMbps(row.download.max_sum_mbps)));
|
||||
|
||||
const actions = document.createElement("td");
|
||||
actions.classList.add('text-center');
|
||||
const downBtn = document.createElement("button");
|
||||
downBtn.className = "btn btn-sm btn-outline-primary me-1";
|
||||
downBtn.innerHTML = "<i class='fa fa-eye'></i> View";
|
||||
downBtn.onclick = () => {
|
||||
state.selectedCpu = row.cpu;
|
||||
state.direction = "down";
|
||||
state.page = 1;
|
||||
renderDetailsHeader();
|
||||
fetchCircuits();
|
||||
};
|
||||
|
||||
actions.appendChild(downBtn);
|
||||
tr.appendChild(actions);
|
||||
tbody.appendChild(tr);
|
||||
});
|
||||
|
||||
table.appendChild(tbody);
|
||||
left.appendChild(table);
|
||||
|
||||
// Right column pie chart (DL Max per CPU)
|
||||
const chartDiv = document.createElement('div');
|
||||
chartDiv.id = 'cpuDlMaxPie';
|
||||
chartDiv.style.width = '100%';
|
||||
chartDiv.style.height = '260px';
|
||||
right.appendChild(chartDiv);
|
||||
|
||||
row.appendChild(left);
|
||||
row.appendChild(right);
|
||||
target.appendChild(row);
|
||||
|
||||
// Build donut (ring) chart with theme support
|
||||
try {
|
||||
// Recreate the graph each render to bind to the fresh DOM
|
||||
if (dlMaxPie && dlMaxPie.chart) {
|
||||
try { dlMaxPie.chart.dispose(); } catch (_) {}
|
||||
dlMaxPie = null;
|
||||
}
|
||||
dlMaxPie = new DashboardGraph('cpuDlMaxPie');
|
||||
const seriesData = data.map((row) => {
|
||||
const w = Number((row.download && row.download.weight_sum) ? row.download.weight_sum : 0);
|
||||
const v = Number(row.download.max_sum_mbps || 0);
|
||||
return {
|
||||
name: `CPU ${row.cpu}`,
|
||||
value: (w > 0 ? w : v)
|
||||
};
|
||||
});
|
||||
dlMaxPie.option = {
|
||||
tooltip: { trigger: 'item' },
|
||||
legend: { show: false },
|
||||
series: [
|
||||
{
|
||||
name: 'DL Max by CPU',
|
||||
type: 'pie',
|
||||
radius: ['55%','80%'],
|
||||
center: ['50%', '50%'],
|
||||
avoidLabelOverlap: true,
|
||||
data: seriesData,
|
||||
label: {
|
||||
show: true,
|
||||
formatter: '{b}: {d}%'
|
||||
}
|
||||
}
|
||||
]
|
||||
};
|
||||
dlMaxPie.chart.setOption(dlMaxPie.option);
|
||||
dlMaxPie.chart.hideLoading();
|
||||
} catch (e) {
|
||||
// If echarts is unavailable, leave chart blank
|
||||
}
|
||||
}
|
||||
|
||||
function renderDetailsHeader() {
|
||||
const label = document.getElementById("currentCpuLabel");
|
||||
if (state.selectedCpu === null) {
|
||||
label.textContent = "";
|
||||
} else {
|
||||
label.textContent = ` – CPU ${state.selectedCpu}`;
|
||||
}
|
||||
}
|
||||
|
||||
function renderCircuits(page) {
|
||||
const target = document.getElementById("circuitsTable");
|
||||
clearDiv(target);
|
||||
|
||||
if (!page || !Array.isArray(page.items)) {
|
||||
target.innerHTML = '<p class="text-secondary">No circuits found for this CPU.</p>';
|
||||
return;
|
||||
}
|
||||
|
||||
const smallNote = document.createElement("div");
|
||||
smallNote.className = "text-muted small mb-1";
|
||||
smallNote.innerText = `Total: ${page.total.toLocaleString()} circuits`;
|
||||
target.appendChild(smallNote);
|
||||
|
||||
const table = document.createElement("table");
|
||||
table.classList.add("table", "table-striped", "table-sm");
|
||||
const thead = document.createElement("thead");
|
||||
thead.appendChild(theading("Circuit ID"));
|
||||
thead.appendChild(theading("Circuit Name"));
|
||||
thead.appendChild(theading("Parent"));
|
||||
thead.appendChild(theading("ClassID"));
|
||||
thead.appendChild(theading("Weight"));
|
||||
thead.appendChild(theading("Max (Mbps)"));
|
||||
thead.appendChild(theading("IPs"));
|
||||
table.appendChild(thead);
|
||||
const tbody = document.createElement("tbody");
|
||||
|
||||
page.items.forEach((c) => {
|
||||
const tr = document.createElement("tr");
|
||||
if (c.ignored || (c.weight !== undefined && c.weight <= 0)) {
|
||||
tr.classList.add('text-muted');
|
||||
}
|
||||
const idCell = document.createElement("td");
|
||||
if (c.circuit_id) {
|
||||
const a = document.createElement("a");
|
||||
a.href = `circuit.html?id=${encodeURIComponent(c.circuit_id)}`;
|
||||
a.textContent = c.circuit_id;
|
||||
idCell.appendChild(a);
|
||||
} else {
|
||||
idCell.textContent = "";
|
||||
}
|
||||
tr.appendChild(idCell);
|
||||
tr.appendChild(simpleRow(c.circuit_name || ""));
|
||||
tr.appendChild(simpleRow(c.parent_node || ""));
|
||||
tr.appendChild(simpleRow(c.classid || ""));
|
||||
const weightCell = document.createElement('td');
|
||||
if (c.ignored || (c.weight !== undefined && c.weight <= 0)) {
|
||||
weightCell.innerHTML = "<span class='badge bg-secondary'>ignored</span>";
|
||||
} else {
|
||||
weightCell.innerText = (c.weight && c.weight > 0) ? c.weight.toLocaleString() : '-';
|
||||
}
|
||||
tr.appendChild(weightCell);
|
||||
tr.appendChild(simpleRow(fmtMbps(c.max_mbps)));
|
||||
tr.appendChild(simpleRow(String(c.ip_count || 0)));
|
||||
tbody.appendChild(tr);
|
||||
});
|
||||
|
||||
table.appendChild(tbody);
|
||||
target.appendChild(table);
|
||||
}
|
||||
|
||||
async function fetchSummary() {
|
||||
try {
|
||||
const r = await fetch("/local-api/cpuAffinity/summary");
|
||||
if (!r.ok) throw new Error(`${r.status}`);
|
||||
const data = await r.json();
|
||||
// data is { entries: [...] } or just array depending on server; we returned a tuple-like wrapper
|
||||
const entries = Array.isArray(data) ? data : (data["0"] ? data : (data["entries"] || data));
|
||||
// Get planner enabled flag from config
|
||||
try {
|
||||
const c = await fetch("/local-api/getConfig");
|
||||
if (c.ok) {
|
||||
const cfg = await c.json();
|
||||
state.plannerEnabled = !!(cfg.queues && cfg.queues.use_binpacking);
|
||||
}
|
||||
} catch (_) {}
|
||||
renderSummary(entries); // In our implementation: array
|
||||
} catch (e) {
|
||||
document.getElementById("cpuSummary").innerHTML = `<div class="text-danger">Failed to load summary: ${e}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchCircuits() {
|
||||
const target = document.getElementById("circuitsTable");
|
||||
target.innerHTML = '<i class="fa fa-spinner fa-spin"></i> Loading circuits...';
|
||||
const dir = state.direction;
|
||||
const cpu = state.selectedCpu;
|
||||
if (cpu === null || cpu === undefined) {
|
||||
target.innerHTML = '<i class="fa fa-info-circle"></i> <span class="text-secondary">Select a CPU from the table above.</span>';
|
||||
return;
|
||||
}
|
||||
const params = new URLSearchParams();
|
||||
params.set("direction", dir);
|
||||
params.set("page", String(state.page));
|
||||
params.set("page_size", String(state.pageSize));
|
||||
// no search param
|
||||
try {
|
||||
const r = await fetch(`/local-api/cpuAffinity/circuits/${cpu}?` + params.toString());
|
||||
if (!r.ok) throw new Error(`${r.status}`);
|
||||
const data = await r.json();
|
||||
renderCircuits(data);
|
||||
} catch (e) {
|
||||
target.innerHTML = `<div class="text-danger">Failed to load circuits: ${e}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
// Wire up controls
|
||||
function initControls() {
|
||||
const pageSizeInput = document.getElementById("pageSize");
|
||||
const btnPrev = document.getElementById("btnPrev");
|
||||
const btnNext = document.getElementById("btnNext");
|
||||
const btnRefresh = document.getElementById("btnRefresh");
|
||||
pageSizeInput.onchange = () => {
|
||||
let v = parseInt(pageSizeInput.value || "50", 10);
|
||||
if (!Number.isFinite(v) || v < 10) v = 10;
|
||||
if (v > 1000) v = 1000;
|
||||
state.pageSize = v;
|
||||
state.page = 1;
|
||||
fetchCircuits();
|
||||
};
|
||||
btnPrev.onclick = () => {
|
||||
if (state.page > 1) {
|
||||
state.page -= 1;
|
||||
fetchCircuits();
|
||||
}
|
||||
};
|
||||
btnNext.onclick = () => {
|
||||
state.page += 1;
|
||||
fetchCircuits();
|
||||
};
|
||||
btnRefresh.onclick = () => {
|
||||
fetchSummary();
|
||||
if (state.selectedCpu !== null) fetchCircuits();
|
||||
};
|
||||
}
|
||||
|
||||
// Initialize
|
||||
initControls();
|
||||
renderDetailsHeader();
|
||||
fetchSummary().then(() => enableTooltips());
|
||||
|
||||
// No preview section (binpacking) in this view
|
||||
@@ -15,19 +15,26 @@ class ChildrenSankeyGraph extends DashboardGraph {
|
||||
this.chart.setOption(this.option);
|
||||
}
|
||||
update(items) {
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
items.forEach(it => {
|
||||
const color = colorByRetransmitPct(it.rxmit ?? 0);
|
||||
nodes.push({ name: it.name, itemStyle: { color } });
|
||||
if (this.direction === 'down') {
|
||||
links.push({ source: it.name, target: 'Shaper', value: it.value });
|
||||
} else {
|
||||
links.push({ source: 'Shaper', target: it.name, value: it.value });
|
||||
}
|
||||
});
|
||||
this.option.series[0].data = nodes;
|
||||
this.option.series[0].links = links;
|
||||
// Backward compatibility: if items is an array of {name,value,rxmit}, draw 1-level
|
||||
if (Array.isArray(items)) {
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
items.forEach(it => {
|
||||
const color = colorByRetransmitPct(it.rxmit ?? 0);
|
||||
nodes.push({ name: it.name, itemStyle: { color } });
|
||||
if (this.direction === 'down') {
|
||||
links.push({ source: it.name, target: 'Shaper', value: it.value });
|
||||
} else {
|
||||
links.push({ source: 'Shaper', target: it.name, value: it.value });
|
||||
}
|
||||
});
|
||||
this.option.series[0].data = nodes;
|
||||
this.option.series[0].links = links;
|
||||
} else if (items && items.nodes && items.links) {
|
||||
// New 2-level path: items = { nodes, links }
|
||||
this.option.series[0].data = items.nodes;
|
||||
this.option.series[0].links = items.links;
|
||||
}
|
||||
this.chart.hideLoading();
|
||||
this.chart.setOption(this.option);
|
||||
}
|
||||
@@ -39,11 +46,13 @@ export class ShaperChildrenDown extends BaseDashlet {
|
||||
this.last=null; this._emptyId = this.id + "_empty";
|
||||
this._smooth = new Map();
|
||||
this._ALPHA = 0.3; this._ALPHA_RX = 0.2; this._DECAY = 0.15; this._LINGER = 3;
|
||||
this._lastSummary = null; // raw TreeSummary
|
||||
this._lastL2 = null; // TreeSummaryL2
|
||||
}
|
||||
canBeSlowedDown(){ return true; }
|
||||
title(){ return "Shaper Children (Download)"; }
|
||||
tooltip(){ return "<h5>Child Throughput</h5><p>Top child nodes by download throughput."; }
|
||||
subscribeTo(){ return ["TreeSummary"]; }
|
||||
subscribeTo(){ return ["TreeSummary", "TreeSummaryL2"]; }
|
||||
buildContainer(){ let b = super.buildContainer(); b.appendChild(this.graphDiv()); return b; }
|
||||
setup(){ this.graph = new ChildrenSankeyGraph(this.graphDivId(), 'down'); if (this.last) this.graph.update(this.last); }
|
||||
_showEmpty(show, msg = "No recent data"){
|
||||
@@ -87,9 +96,66 @@ export class ShaperChildrenDown extends BaseDashlet {
|
||||
out.sort((a,b)=> b.value - a.value);
|
||||
return out.slice(0,9);
|
||||
}
|
||||
_renderTwoLevel(){
|
||||
if (!this._lastSummary || !this._lastL2) return false;
|
||||
// Build parent map
|
||||
const parentMap = new Map();
|
||||
(this._lastSummary || []).slice(1).forEach(r => parentMap.set(r[0], r[1]));
|
||||
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
let hasData = false;
|
||||
|
||||
for (const [parentId, children] of this._lastL2) {
|
||||
const p = parentMap.get(parentId);
|
||||
if (!p) continue;
|
||||
const pName = p.name || String(parentId);
|
||||
// Parent color by rxmit (download)
|
||||
let pRx = 0;
|
||||
if ((p.current_tcp_packets?.[0]||0) > 0) {
|
||||
pRx = ((p.current_retransmits?.[0]||0) / Math.max(1, p.current_tcp_packets?.[0]||0)) * 100.0;
|
||||
}
|
||||
nodes.push({ name: pName, itemStyle: { color: colorByRetransmitPct(pRx) } });
|
||||
|
||||
let parentSum = 0;
|
||||
for (const [, c] of children) {
|
||||
const v = Number((c.current_throughput?.[0]||0));
|
||||
if (v <= 0) continue;
|
||||
parentSum += v;
|
||||
hasData = hasData || v > 0.5;
|
||||
let cRx = 0;
|
||||
if ((c.current_tcp_packets?.[0]||0) > 0) {
|
||||
cRx = ((c.current_retransmits?.[0]||0) / Math.max(1, c.current_tcp_packets?.[0]||0)) * 100.0;
|
||||
}
|
||||
nodes.push({ name: c.name, itemStyle: { color: colorByRetransmitPct(cRx) } });
|
||||
links.push({ source: c.name, target: pName, value: v });
|
||||
}
|
||||
if (parentSum > 0) {
|
||||
links.push({ source: pName, target: 'Shaper', value: parentSum });
|
||||
}
|
||||
}
|
||||
|
||||
this._showEmpty(!hasData);
|
||||
if (hasData) {
|
||||
this.graph.update({ nodes, links });
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
onMessage(msg){
|
||||
if (msg.event === "TreeSummary") {
|
||||
this._lastSummary = msg.data;
|
||||
} else if (msg.event === "TreeSummaryL2") {
|
||||
this._lastL2 = msg.data;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
|
||||
// Prefer 2-level; if unavailable, fallback to 1-level smoothing
|
||||
if (this._renderTwoLevel()) return;
|
||||
|
||||
if (msg.event !== "TreeSummary") return;
|
||||
// msg.data: [ [id, metrics], [id, metrics], ... ] - first is root
|
||||
let rows = (msg.data || []).slice(1).map(r => {
|
||||
const m = r[1] || {};
|
||||
const name = m.name || String(r[0]);
|
||||
|
||||
@@ -12,15 +12,20 @@ class ChildrenSankeyGraphUp extends DashboardGraph {
|
||||
this.chart.setOption(this.option);
|
||||
}
|
||||
update(items){
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
items.forEach(it => {
|
||||
const color = colorByRetransmitPct(it.rxmit ?? 0);
|
||||
nodes.push({ name: it.name, itemStyle: { color } });
|
||||
links.push({ source: 'Shaper', target: it.name, value: it.value });
|
||||
});
|
||||
this.option.series[0].data = nodes;
|
||||
this.option.series[0].links = links;
|
||||
if (Array.isArray(items)) {
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
items.forEach(it => {
|
||||
const color = colorByRetransmitPct(it.rxmit ?? 0);
|
||||
nodes.push({ name: it.name, itemStyle: { color } });
|
||||
links.push({ source: 'Shaper', target: it.name, value: it.value });
|
||||
});
|
||||
this.option.series[0].data = nodes;
|
||||
this.option.series[0].links = links;
|
||||
} else if (items && items.nodes && items.links) {
|
||||
this.option.series[0].data = items.nodes;
|
||||
this.option.series[0].links = items.links;
|
||||
}
|
||||
this.chart.hideLoading();
|
||||
this.chart.setOption(this.option);
|
||||
}
|
||||
@@ -32,11 +37,13 @@ export class ShaperChildrenUp extends BaseDashlet {
|
||||
this.last=null; this._emptyId = this.id + "_empty";
|
||||
this._smooth = new Map();
|
||||
this._ALPHA = 0.3; this._ALPHA_RX = 0.2; this._DECAY = 0.15; this._LINGER = 3;
|
||||
this._lastSummary = null;
|
||||
this._lastL2 = null;
|
||||
}
|
||||
canBeSlowedDown(){ return true; }
|
||||
title(){ return "Shaper Children (Upload)"; }
|
||||
tooltip(){ return "<h5>Child Throughput</h5><p>Top child nodes by upload throughput."; }
|
||||
subscribeTo(){ return ["TreeSummary"]; }
|
||||
subscribeTo(){ return ["TreeSummary", "TreeSummaryL2"]; }
|
||||
buildContainer(){ let b=super.buildContainer(); b.appendChild(this.graphDiv()); return b; }
|
||||
setup(){ this.graph = new ChildrenSankeyGraphUp(this.graphDivId()); if (this.last) this.graph.update(this.last); }
|
||||
_showEmpty(show, msg = "No recent data"){
|
||||
@@ -80,7 +87,61 @@ export class ShaperChildrenUp extends BaseDashlet {
|
||||
out.sort((a,b)=> b.value - a.value);
|
||||
return out.slice(0,9);
|
||||
}
|
||||
_renderTwoLevel(){
|
||||
if (!this._lastSummary || !this._lastL2) return false;
|
||||
const parentMap = new Map();
|
||||
(this._lastSummary || []).slice(1).forEach(r => parentMap.set(r[0], r[1]));
|
||||
|
||||
const nodes = [{ name: 'Shaper' }];
|
||||
const links = [];
|
||||
let hasData = false;
|
||||
|
||||
for (const [parentId, children] of this._lastL2) {
|
||||
const p = parentMap.get(parentId);
|
||||
if (!p) continue;
|
||||
const pName = p.name || String(parentId);
|
||||
// Parent color by rxmit (upload)
|
||||
let pRx = 0;
|
||||
if ((p.current_tcp_packets?.[1]||0) > 0) {
|
||||
pRx = ((p.current_retransmits?.[1]||0) / Math.max(1, p.current_tcp_packets?.[1]||0)) * 100.0;
|
||||
}
|
||||
nodes.push({ name: pName, itemStyle: { color: colorByRetransmitPct(pRx) } });
|
||||
|
||||
let parentSum = 0;
|
||||
for (const [, c] of children) {
|
||||
const v = Number((c.current_throughput?.[1]||0));
|
||||
if (v <= 0) continue;
|
||||
parentSum += v;
|
||||
hasData = hasData || v > 0.5;
|
||||
let cRx = 0;
|
||||
if ((c.current_tcp_packets?.[1]||0) > 0) {
|
||||
cRx = ((c.current_retransmits?.[1]||0) / Math.max(1, c.current_tcp_packets?.[1]||0)) * 100.0;
|
||||
}
|
||||
nodes.push({ name: c.name, itemStyle: { color: colorByRetransmitPct(cRx) } });
|
||||
links.push({ source: pName, target: c.name, value: v });
|
||||
}
|
||||
if (parentSum > 0) {
|
||||
links.push({ source: 'Shaper', target: pName, value: parentSum });
|
||||
}
|
||||
}
|
||||
|
||||
this._showEmpty(!hasData);
|
||||
if (hasData) {
|
||||
this.graph.update({ nodes, links });
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
onMessage(msg){
|
||||
if (msg.event === "TreeSummary") {
|
||||
this._lastSummary = msg.data;
|
||||
} else if (msg.event === "TreeSummaryL2") {
|
||||
this._lastL2 = msg.data;
|
||||
} else { return; }
|
||||
|
||||
if (this._renderTwoLevel()) return;
|
||||
|
||||
if (msg.event !== "TreeSummary") return;
|
||||
let rows = (msg.data || []).slice(1).map(r => {
|
||||
const m = r[1] || {};
|
||||
|
||||
@@ -30,6 +30,8 @@ import {LtsLast24Hours} from "./ltsLast24Hours";
|
||||
import {TcpRetransmitsDash} from "./total_retransmits";
|
||||
import {StormguardStatusDashlet} from "./stormguard_status";
|
||||
import {BakeryStatusDashlet} from "./bakery_status";
|
||||
import {Top10UploadersVisual} from "./top10_uploads_graphic";
|
||||
import {Top10Uploaders} from "./top10_uploaders";
|
||||
// New Traffic Overview dashlets
|
||||
import {ShaperTopAsnDownload} from "./top_asn_download";
|
||||
import {ShaperTopAsnUpload} from "./top_asn_upload";
|
||||
@@ -47,6 +49,8 @@ export const DashletMenu = [
|
||||
{ name: "Round-Trip Time Histogram", tag: "rttHistogram", size: 6, category: "RTT" },
|
||||
{ name: "Top 10 Downloaders", tag: "top10downloaders", size: 6, category: "Top 10" },
|
||||
{ name: "Top 10 Downloaders (Visual)", tag: "top10downloadersV", size: 6, category: "Top 10" },
|
||||
{ name: "Top 10 Uploaders", tag: "top10uploaders", size: 6, category: "Top 10" },
|
||||
{ name: "Top 10 Uploaders (Visual)", tag: "top10uploadersV", size: 6, category: "Top 10" },
|
||||
{ name: "Worst 10 Round-Trip Time", tag: "worst10downloaders", size: 6, category: "Top 10" },
|
||||
{ name: "Worst 10 Round-Trip Time (Visual)", tag: "worst10downloadersV", size: 6, category: "Top 10" },
|
||||
{ name: "Worst 10 Retransmits", tag: "worst10retransmits", size: 6, category: "Top 10" },
|
||||
@@ -92,6 +96,8 @@ export function widgetFactory(widgetName, count) {
|
||||
case "rttHistogram3D": widget = new RttHisto3dDash(count); break;
|
||||
case "top10downloaders":widget = new Top10Downloaders(count); break;
|
||||
case "top10downloadersV":widget = new Top10DownloadersVisual(count); break;
|
||||
case "top10uploaders":widget = new Top10Uploaders(count); break;
|
||||
case "top10uploadersV":widget = new Top10UploadersVisual(count); break;
|
||||
case "worst10downloaders":widget = new Worst10Downloaders(count); break;
|
||||
case "worst10downloadersV":widget = new Worst10DownloadersVisual(count); break;
|
||||
case "worst10retransmits":widget = new Worst10Retransmits(count); break;
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import {clearDashDiv, TopNTableFromMsgData} from "../helpers/builders";
|
||||
import {TimedCache} from "../lq_js_common/helpers/timed_cache";
|
||||
import {DashletBaseInsight} from "./insight_dashlet_base";
|
||||
|
||||
export class Top10Uploaders extends DashletBaseInsight {
|
||||
constructor(slot) {
|
||||
super(slot);
|
||||
this.timeCache = new TimedCache(10);
|
||||
}
|
||||
|
||||
title() {
|
||||
return "Top 10 Uploaders";
|
||||
}
|
||||
|
||||
tooltip() {
|
||||
return "<h5>Top 10 Uploaders</h5><p>Top 10 Uploaders by bits per second, including IP address, download and upload rates, round-trip time, TCP retransmits, and shaping plan.</p>";
|
||||
}
|
||||
|
||||
subscribeTo() {
|
||||
return [ "TopUploads" ];
|
||||
}
|
||||
|
||||
buildContainer() {
|
||||
let base = super.buildContainer();
|
||||
base.style.height = "250px";
|
||||
base.style.overflow = "auto";
|
||||
return base;
|
||||
}
|
||||
|
||||
canBeSlowedDown() {
|
||||
return true;
|
||||
}
|
||||
|
||||
setup() {
|
||||
super.setup();
|
||||
window.timeGraphs.push(this);
|
||||
}
|
||||
|
||||
onMessage(msg) {
|
||||
if (msg.event === "TopUploads" && window.timePeriods.activePeriod === "Live") {
|
||||
let target = document.getElementById(this.id);
|
||||
|
||||
msg.data.forEach((r) => {
|
||||
let key = r.circuit_id;
|
||||
this.timeCache.addOrUpdate(key, r, r.bits_per_second.up);
|
||||
});
|
||||
this.timeCache.tick();
|
||||
|
||||
let items = this.timeCache.get();
|
||||
let t = TopNTableFromMsgData(items);
|
||||
|
||||
// Display it
|
||||
clearDashDiv(this.id, target);
|
||||
target.appendChild(t);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,49 @@
|
||||
import {TopNSankey} from "../graphs/top_n_sankey";
|
||||
import {TimedCache} from "../lq_js_common/helpers/timed_cache";
|
||||
import {DashletBaseInsight} from "./insight_dashlet_base";
|
||||
|
||||
export class Top10UploadersVisual extends DashletBaseInsight {
|
||||
constructor(slot) {
|
||||
super(slot);
|
||||
this.timeCache = new TimedCache(10);
|
||||
}
|
||||
|
||||
title() {
|
||||
return "Top 10 Uploaders (Visual)";
|
||||
}
|
||||
|
||||
tooltip() {
|
||||
return "<h5>Top 10 Uploaders</h5><p>The top-10 users by bits-per-second. The ribbon goes red as they approach capacity. The name border indicates round-trip time, while the main label color indicates TCP retransmits. The legend is current upload speed, followed by number of retransmits.</p>";
|
||||
}
|
||||
|
||||
subscribeTo() {
|
||||
return [ "TopUploads" ];
|
||||
}
|
||||
|
||||
buildContainer() {
|
||||
let base = super.buildContainer();
|
||||
base.appendChild(this.graphDiv());
|
||||
return base;
|
||||
}
|
||||
|
||||
canBeSlowedDown() {
|
||||
return true;
|
||||
}
|
||||
|
||||
setup() {
|
||||
super.setup();
|
||||
this.graph = new TopNSankey(this.graphDivId(), true);
|
||||
}
|
||||
|
||||
onMessage(msg) {
|
||||
if (msg.event === "TopUploads") {
|
||||
msg.data.forEach((r) => {
|
||||
let key = r.circuit_id;
|
||||
this.timeCache.addOrUpdate(key, r, r.bits_per_second.up);
|
||||
});
|
||||
this.timeCache.tick();
|
||||
let items = this.timeCache.get();
|
||||
this.graph.processMessage({ data: items });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -69,6 +69,8 @@ class TopTreeSankeyGraph extends DashboardGraph {
|
||||
export class TopTreeSankey extends BaseDashlet {
|
||||
constructor(slot) {
|
||||
super(slot);
|
||||
this._lastTreeSummary = null;
|
||||
this._lastL2 = null;
|
||||
}
|
||||
|
||||
canBeSlowedDown() {
|
||||
@@ -84,7 +86,7 @@ export class TopTreeSankey extends BaseDashlet {
|
||||
}
|
||||
|
||||
subscribeTo() {
|
||||
return [ "TreeSummary" ];
|
||||
return [ "TreeSummary", "TreeSummaryL2" ];
|
||||
}
|
||||
|
||||
buildContainer() {
|
||||
@@ -100,63 +102,143 @@ export class TopTreeSankey extends BaseDashlet {
|
||||
|
||||
onMessage(msg) {
|
||||
if (msg.event === "TreeSummary") {
|
||||
//console.log(msg.data);
|
||||
this._lastTreeSummary = msg.data;
|
||||
} else if (msg.event === "TreeSummaryL2") {
|
||||
this._lastL2 = msg.data;
|
||||
} else {
|
||||
return;
|
||||
}
|
||||
|
||||
let redact = isRedacted();
|
||||
if (!this._lastTreeSummary) return;
|
||||
|
||||
let nodes = [];
|
||||
let links = [];
|
||||
let redact = isRedacted();
|
||||
|
||||
nodes.push({
|
||||
name: "Root",
|
||||
label: "Root",
|
||||
});
|
||||
// Build 2-level sankey if L2 is present, else fallback to 1-level
|
||||
const nodes = [];
|
||||
const links = [];
|
||||
nodes.push({ name: "Root", label: "Root" });
|
||||
|
||||
msg.data.slice(1).forEach((r) => {
|
||||
let label = {
|
||||
fontSize: 9,
|
||||
color: "#999"
|
||||
};
|
||||
const firstLevel = (this._lastTreeSummary || []).slice(1);
|
||||
|
||||
// Map of parent id -> parent transport for quick lookup
|
||||
const parentMap = new Map();
|
||||
firstLevel.forEach((r) => parentMap.set(r[0], r[1]));
|
||||
|
||||
// Helper to build node style from RTT
|
||||
const nodeStyleFromRtt = (name, rttsArr) => {
|
||||
let rtt = 0;
|
||||
if (rttsArr && rttsArr.length > 0) {
|
||||
rtt = rttsArr[0];
|
||||
} else if (lastRtt[name] !== undefined) {
|
||||
rtt = lastRtt[name];
|
||||
}
|
||||
lastRtt[name] = rtt;
|
||||
const rttPercent = Math.min(100, (rtt / 200) * 100);
|
||||
const color = isColorBlindMode()
|
||||
? lerpViridis(rttPercent / 100)
|
||||
: lerpGreenToRedViaOrange(200 - rtt, 200);
|
||||
return { itemStyle: { color } };
|
||||
};
|
||||
|
||||
if (this._lastL2 && Array.isArray(this._lastL2) && this._lastL2.length > 0) {
|
||||
// L2 data shape: [ [parent_id, [ [child_id, child_transport], ... ] ], ... ]
|
||||
for (const [parentId, children] of this._lastL2) {
|
||||
const p = parentMap.get(parentId);
|
||||
if (!p) continue;
|
||||
|
||||
// Parent node styling
|
||||
const pName = p.name;
|
||||
const label = { fontSize: 9, color: "#999" };
|
||||
if (redact) label.fontFamily = "Illegible";
|
||||
|
||||
let name = r[1].name;
|
||||
let bytes = r[1].current_throughput[0];
|
||||
let bytesAsMegabits = bytes / 1000000;
|
||||
let maxBytes = r[1].max_throughput[0] / 8;
|
||||
let percent = Math.min(100, (bytesAsMegabits / maxBytes) * 100);
|
||||
let capacityColor = isColorBlindMode()
|
||||
const pStyle = nodeStyleFromRtt(pName, p.rtts);
|
||||
nodes.push({ name: pName, label, ...pStyle });
|
||||
|
||||
// Compute Root->Parent value as sum of included child totals (down + up)
|
||||
let parentSum = 0;
|
||||
// Compute link color from parent's capacity percent (as before)
|
||||
const bytesAsMegabits = p.current_throughput[0] / 1000000;
|
||||
const maxBytes = p.max_throughput[0] / 8;
|
||||
const percent = Math.min(100, maxBytes > 0 ? (bytesAsMegabits / maxBytes) * 100 : 0);
|
||||
const capacityColor = isColorBlindMode()
|
||||
? lerpViridis(percent / 100)
|
||||
: lerpGreenToRedViaOrange(100 - percent, 100);
|
||||
|
||||
if (r[1].rtts.length > 0) {
|
||||
lastRtt[name] = r[1].rtts[0];
|
||||
} else {
|
||||
lastRtt[name] = 0;
|
||||
}
|
||||
let rttPercent = Math.min(100, (lastRtt[name] / 200) * 100);
|
||||
let color = isColorBlindMode()
|
||||
? lerpViridis(rttPercent / 100)
|
||||
: lerpGreenToRedViaOrange(200 - lastRtt[name], 200);
|
||||
for (const [, child] of children) {
|
||||
const cName = child.name;
|
||||
const cTotal = (child.current_throughput?.[0] || 0) + (child.current_throughput?.[1] || 0);
|
||||
if (cTotal <= 0) continue;
|
||||
parentSum += cTotal;
|
||||
|
||||
if (bytesAsMegabits > 0) {
|
||||
nodes.push({
|
||||
name: r[1].name,
|
||||
label: label,
|
||||
itemStyle: {
|
||||
color: color
|
||||
}
|
||||
// Child node and link parent->child
|
||||
const cLabel = { fontSize: 9, color: "#999" };
|
||||
if (redact) cLabel.fontFamily = "Illegible";
|
||||
const cStyle = nodeStyleFromRtt(cName, child.rtts);
|
||||
nodes.push({ name: cName, label: cLabel, ...cStyle });
|
||||
|
||||
// Link color for child can use child's capacity percent
|
||||
const cBytesAsMegabits = (child.current_throughput?.[0] || 0) / 1000000;
|
||||
const cMaxBytes = (child.max_throughput?.[0] || 0) / 8;
|
||||
const cPercent = Math.min(100, cMaxBytes > 0 ? (cBytesAsMegabits / cMaxBytes) * 100 : 0);
|
||||
const cCapacityColor = isColorBlindMode()
|
||||
? lerpViridis(cPercent / 100)
|
||||
: lerpGreenToRedViaOrange(100 - cPercent, 100);
|
||||
|
||||
links.push({
|
||||
source: pName,
|
||||
target: cName,
|
||||
value: cTotal,
|
||||
lineStyle: { color: cCapacityColor },
|
||||
});
|
||||
}
|
||||
|
||||
if (parentSum > 0) {
|
||||
links.push({
|
||||
source: "Root",
|
||||
target: r[1].name,
|
||||
value: r[1].current_throughput[0] + r[1].current_throughput[1],
|
||||
lineStyle: {
|
||||
color: capacityColor,
|
||||
}
|
||||
target: pName,
|
||||
value: parentSum,
|
||||
lineStyle: { color: capacityColor },
|
||||
});
|
||||
}
|
||||
});
|
||||
}
|
||||
this.graph.update(nodes, links);
|
||||
return;
|
||||
}
|
||||
|
||||
// Fallback to 1-level sankey using current logic
|
||||
firstLevel.forEach((r) => {
|
||||
let label = { fontSize: 9, color: "#999" };
|
||||
if (redact) label.fontFamily = "Illegible";
|
||||
|
||||
let name = r[1].name;
|
||||
let bytes = r[1].current_throughput[0];
|
||||
let bytesAsMegabits = bytes / 1000000;
|
||||
let maxBytes = r[1].max_throughput[0] / 8;
|
||||
let percent = Math.min(100, maxBytes > 0 ? (bytesAsMegabits / maxBytes) * 100 : 0);
|
||||
let capacityColor = isColorBlindMode()
|
||||
? lerpViridis(percent / 100)
|
||||
: lerpGreenToRedViaOrange(100 - percent, 100);
|
||||
|
||||
if (r[1].rtts.length > 0) {
|
||||
lastRtt[name] = r[1].rtts[0];
|
||||
} else {
|
||||
lastRtt[name] = 0;
|
||||
}
|
||||
let rttPercent = Math.min(100, (lastRtt[name] / 200) * 100);
|
||||
let color = isColorBlindMode()
|
||||
? lerpViridis(rttPercent / 100)
|
||||
: lerpGreenToRedViaOrange(200 - lastRtt[name], 200);
|
||||
|
||||
if (bytesAsMegabits > 0) {
|
||||
nodes.push({ name: r[1].name, label, itemStyle: { color } });
|
||||
links.push({
|
||||
source: "Root",
|
||||
target: r[1].name,
|
||||
value: r[1].current_throughput[0] + r[1].current_throughput[1],
|
||||
lineStyle: { color: capacityColor },
|
||||
});
|
||||
}
|
||||
});
|
||||
this.graph.update(nodes, links);
|
||||
}
|
||||
}
|
||||
@@ -24,12 +24,12 @@ function lerpViridis(t) {
|
||||
let b = Math.round(c0[2] + frac * (c1[2] - c0[2]));
|
||||
return "#" + ((1 << 24) + (r << 16) + (g << 8) + b).toString(16).slice(1);
|
||||
}
|
||||
import {scaleNumber} from "../lq_js_common/helpers/scaling";
|
||||
import {isRedacted} from "../helpers/redact";
|
||||
|
||||
export class TopNSankey extends DashboardGraph {
|
||||
constructor(id) {
|
||||
constructor(id, upload=false) {
|
||||
super(id);
|
||||
this.upload = upload;
|
||||
this.nodeMap = {};
|
||||
this.option = {
|
||||
series: [
|
||||
@@ -49,23 +49,6 @@ export class TopNSankey extends DashboardGraph {
|
||||
this.option.series[0].links = links;
|
||||
this.chart.hideLoading();
|
||||
this.chart.setOption(this.option);
|
||||
|
||||
/*this.chart.on('click', (params) => {
|
||||
let name = params.name;
|
||||
// Trim to before " ("
|
||||
name = name.substring(0, name.indexOf(" ("));
|
||||
if (name.indexOf(" > ") === -1) {
|
||||
if (this.nodeMap[name] !== undefined) {
|
||||
window.location.href = "/circuit.html?id=" + encodeURI(this.nodeMap[name]);
|
||||
}
|
||||
} else {
|
||||
let actualName = params.data.target;
|
||||
actualName = actualName.substring(0, actualName.indexOf(" ("));
|
||||
if (this.nodeMap[actualName] !== undefined) {
|
||||
window.location.href = "/circuit.html?id=" + encodeURI(this.nodeMap[actualName]);
|
||||
}
|
||||
}
|
||||
});*/
|
||||
}
|
||||
|
||||
processMessage(msg) {
|
||||
@@ -92,10 +75,14 @@ export class TopNSankey extends DashboardGraph {
|
||||
if (isRedacted()) label.fontFamily = "Illegible";
|
||||
|
||||
let name = r.ip_address;
|
||||
let bytes = r.bits_per_second.down / 8;
|
||||
let bytesAsMegabits = bytes / 1000000;
|
||||
let maxBytes = r.plan.down / 8;
|
||||
let percent = Math.min(100, (bytesAsMegabits / maxBytes) * 100);
|
||||
// Choose the correct direction for value and capacity coloring
|
||||
const bps = this.upload ? r.bits_per_second.up : r.bits_per_second.down;
|
||||
const planMbps = this.upload ? r.plan.up : r.plan.down;
|
||||
// Convert bits/s to MB/s (decimal) and Mbps plan to MB/s for a comparable ratio
|
||||
const bytes = bps / 8;
|
||||
const bytesAsMegabytes = bytes / 1000000;
|
||||
const maxBytes = planMbps / 8;
|
||||
const percent = Math.min(100, (maxBytes > 0 ? (bytesAsMegabytes / maxBytes) * 100 : 0));
|
||||
let capacityColor = isColorBlindMode()
|
||||
? lerpViridis(percent / 100)
|
||||
: lerpGreenToRedViaOrange(100 - percent, 100);
|
||||
@@ -119,11 +106,12 @@ export class TopNSankey extends DashboardGraph {
|
||||
borderColor: rttColor,
|
||||
}
|
||||
});
|
||||
|
||||
|
||||
let value = this.upload ? r.bits_per_second.up : r.bits_per_second.down;
|
||||
links.push({
|
||||
source: "Root",
|
||||
target: name,
|
||||
value: r.bits_per_second.down,
|
||||
value,
|
||||
lineStyle: {
|
||||
color: capacityColor
|
||||
}
|
||||
@@ -132,4 +120,4 @@ export class TopNSankey extends DashboardGraph {
|
||||
|
||||
this.update(nodes, links);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
// Setup any WS feeds for this page
|
||||
import { ws_proto } from '../../pubsub/ws.js';
|
||||
let ws = null;
|
||||
|
||||
export function subscribeWS(channels, handler) {
|
||||
@@ -9,7 +10,7 @@ export function subscribeWS(channels, handler) {
|
||||
ws.close();
|
||||
}
|
||||
|
||||
ws = new WebSocket('ws://' + window.location.host + '/websocket/ws');
|
||||
ws = new WebSocket(ws_proto() + window.location.host + '/websocket/ws');
|
||||
ws.onopen = () => {
|
||||
for (let i=0; i<channels.length; i++) {
|
||||
ws.send("{ \"channel\" : \"" + channels[i] + "\"}");
|
||||
@@ -29,4 +30,4 @@ export function subscribeWS(channels, handler) {
|
||||
|
||||
export function resetWS() {
|
||||
ws = null;
|
||||
}
|
||||
}
|
||||
@@ -8,9 +8,11 @@ mod flow_explorer;
|
||||
mod flow_map;
|
||||
pub mod lts;
|
||||
mod network_tree;
|
||||
mod cpu_affinity;
|
||||
mod packet_analysis;
|
||||
mod reload_libreqos;
|
||||
mod scheduler;
|
||||
mod urgent;
|
||||
mod search;
|
||||
mod shaped_device_api;
|
||||
mod support;
|
||||
@@ -66,6 +68,19 @@ pub fn local_api(shaper_query: tokio::sync::mpsc::Sender<ShaperQueryCommand>) ->
|
||||
)
|
||||
.route("/pcapDump/:id", get(packet_analysis::pcap_dump))
|
||||
.route("/flowMap", get(flow_map::flow_lat_lon))
|
||||
.route("/cpuAffinity/summary", get(cpu_affinity::cpu_affinity_summary))
|
||||
.route(
|
||||
"/cpuAffinity/circuits/:cpu",
|
||||
get(cpu_affinity::cpu_affinity_circuits),
|
||||
)
|
||||
.route(
|
||||
"/cpuAffinity/circuitsAll",
|
||||
get(cpu_affinity::cpu_affinity_circuits_all),
|
||||
)
|
||||
.route(
|
||||
"/cpuAffinity/previewWeights",
|
||||
get(cpu_affinity::cpu_affinity_preview_weights),
|
||||
)
|
||||
.route("/globalWarnings", get(warnings::get_global_warnings))
|
||||
.route("/asnList", get(flow_explorer::asn_list))
|
||||
.route("/countryList", get(flow_explorer::country_list))
|
||||
@@ -103,6 +118,10 @@ pub fn local_api(shaper_query: tokio::sync::mpsc::Sender<ShaperQueryCommand>) ->
|
||||
.route("/ltsRecentMedian", get(lts::recent_medians))
|
||||
.route("/scheduler/status", get(scheduler::scheduler_status))
|
||||
.route("/scheduler/details", get(scheduler::scheduler_details))
|
||||
.route("/urgent/status", get(urgent::urgent_status))
|
||||
.route("/urgent/list", get(urgent::urgent_list))
|
||||
.route("/urgent/clear/:id", post(urgent::urgent_clear))
|
||||
.route("/urgent/clear_all", post(urgent::urgent_clear_all))
|
||||
.route("/chatbot_sso_token", get(chatbot_sso_token))
|
||||
.layer(Extension(shaper_query))
|
||||
.layer(CorsLayer::very_permissive())
|
||||
|
||||
@@ -0,0 +1,486 @@
|
||||
use axum::{extract::Path, extract::Query, Json};
|
||||
use serde::Serialize;
|
||||
use serde_json::Value;
|
||||
use std::collections::HashMap;
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct CpuSideSummary {
|
||||
pub circuits: usize,
|
||||
pub min_sum_mbps: f64,
|
||||
pub max_sum_mbps: f64,
|
||||
pub weight_sum: f64,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct CpuAffinitySummaryEntry {
|
||||
pub cpu: u32,
|
||||
pub download: CpuSideSummary,
|
||||
pub upload: CpuSideSummary,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct CpuAffinitySummaryResponse(pub Vec<CpuAffinitySummaryEntry>);
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct CircuitBrief {
|
||||
pub circuit_id: Option<String>,
|
||||
pub circuit_name: Option<String>,
|
||||
pub parent_node: Option<String>,
|
||||
pub classid: Option<String>,
|
||||
pub max_mbps: f64,
|
||||
pub weight: f64,
|
||||
pub ip_count: usize,
|
||||
pub ignored: bool,
|
||||
}
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct CpuAffinityCircuitsPage {
|
||||
pub cpu: u32,
|
||||
pub direction: String,
|
||||
pub total: usize,
|
||||
pub page: usize,
|
||||
pub page_size: usize,
|
||||
pub items: Vec<CircuitBrief>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Default, Clone)]
|
||||
struct CircuitRecord {
|
||||
cpu_down: Option<u32>,
|
||||
cpu_up: Option<u32>,
|
||||
min_down: f64,
|
||||
max_down: f64,
|
||||
min_up: f64,
|
||||
max_up: f64,
|
||||
planner_weight: f64,
|
||||
has_planner_weight: bool,
|
||||
classid_down: Option<String>,
|
||||
classid_up: Option<String>,
|
||||
circuit_id: Option<String>,
|
||||
circuit_name: Option<String>,
|
||||
parent_node: Option<String>,
|
||||
ip_count: usize,
|
||||
}
|
||||
|
||||
// Query params for the circuits endpoint
|
||||
#[derive(Debug, Default, Clone, serde::Deserialize)]
|
||||
pub struct CircuitsQuery {
|
||||
pub direction: Option<String>, // "down" | "up" (default: down)
|
||||
pub page: Option<usize>, // default 1
|
||||
pub page_size: Option<usize>, // default 100
|
||||
pub search: Option<String>, // substring match on id/name
|
||||
}
|
||||
|
||||
fn queuing_structure_path() -> Option<PathBuf> {
|
||||
let cfg = lqos_config::load_config().ok()?;
|
||||
let mut p = PathBuf::from(cfg.lqos_directory.clone());
|
||||
p.push("queuingStructure.json");
|
||||
Some(p)
|
||||
}
|
||||
|
||||
fn parse_hex_u32(s: &str) -> Option<u32> {
|
||||
// Accept formats like "0x1" or decimal strings
|
||||
if let Some(stripped) = s.strip_prefix("0x").or_else(|| s.strip_prefix("0X")) {
|
||||
u32::from_str_radix(stripped, 16).ok()
|
||||
} else {
|
||||
s.parse::<u32>().ok()
|
||||
}
|
||||
}
|
||||
|
||||
fn val_as_f64(v: &Value) -> Option<f64> {
|
||||
if let Some(x) = v.as_f64() {
|
||||
Some(x)
|
||||
} else if let Some(x) = v.as_i64() {
|
||||
Some(x as f64)
|
||||
} else if let Some(x) = v.as_u64() {
|
||||
Some(x as f64)
|
||||
} else if let Some(s) = v.as_str() {
|
||||
s.parse::<f64>().ok()
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
fn collect_ip_count(circuit: &serde_json::Map<String, Value>) -> usize {
|
||||
let mut count = 0usize;
|
||||
if let Some(Value::Array(devs)) = circuit.get("devices") {
|
||||
for d in devs {
|
||||
if let Value::Object(dev) = d {
|
||||
if let Some(Value::Array(v4s)) = dev.get("ipv4s") {
|
||||
count += v4s.len();
|
||||
}
|
||||
if let Some(Value::Array(v6s)) = dev.get("ipv6s") {
|
||||
count += v6s.len();
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
count
|
||||
}
|
||||
|
||||
fn add_circuit_records(
|
||||
parent_name: Option<&str>,
|
||||
node: &serde_json::Map<String, Value>,
|
||||
out: &mut Vec<CircuitRecord>,
|
||||
) {
|
||||
// Helper: parse a JSON object that represents a circuit-level entry
|
||||
fn parse_circuit_map(
|
||||
parent_name: Option<&str>,
|
||||
cm: &serde_json::Map<String, Value>,
|
||||
) -> CircuitRecord {
|
||||
let mut rec = CircuitRecord::default();
|
||||
// cpu numbers
|
||||
if let Some(Value::String(s)) = cm.get("cpuNum") {
|
||||
rec.cpu_down = parse_hex_u32(s);
|
||||
} else if let Some(Value::String(s)) = cm.get("classMajor") {
|
||||
if let Some(v) = parse_hex_u32(s) { rec.cpu_down = Some(v.saturating_sub(1)); }
|
||||
} else if let Some(Value::String(s)) = cm.get("classid") {
|
||||
// Fallback: parse major from TC handle like "0x1:0x51"
|
||||
if let Some((maj, _)) = s.split_once(':') {
|
||||
if let Some(v) = parse_hex_u32(maj) { rec.cpu_down = Some(v.saturating_sub(1)); }
|
||||
}
|
||||
}
|
||||
if let Some(Value::String(s)) = cm.get("up_cpuNum") {
|
||||
rec.cpu_up = parse_hex_u32(s);
|
||||
} else if let Some(Value::String(s)) = cm.get("up_classMajor") {
|
||||
if let Some(v) = parse_hex_u32(s) { rec.cpu_up = Some(v.saturating_sub(1)); }
|
||||
} else if let Some(Value::String(s)) = cm.get("up_classid") {
|
||||
if let Some((maj, _)) = s.split_once(':') {
|
||||
if let Some(v) = parse_hex_u32(maj) { rec.cpu_up = Some(v.saturating_sub(1)); }
|
||||
}
|
||||
}
|
||||
// classids
|
||||
if let Some(Value::String(s)) = cm.get("classid") {
|
||||
rec.classid_down = Some(s.clone());
|
||||
}
|
||||
if let Some(Value::String(s)) = cm.get("up_classid") {
|
||||
rec.classid_up = Some(s.clone());
|
||||
}
|
||||
// bandwidths
|
||||
if let Some(v) = cm.get("minDownload").and_then(val_as_f64) {
|
||||
rec.min_down = v;
|
||||
}
|
||||
if let Some(v) = cm.get("maxDownload").and_then(val_as_f64) {
|
||||
rec.max_down = v;
|
||||
}
|
||||
if let Some(v) = cm.get("minUpload").and_then(val_as_f64) {
|
||||
rec.min_up = v;
|
||||
}
|
||||
if let Some(v) = cm.get("maxUpload").and_then(val_as_f64) {
|
||||
rec.max_up = v;
|
||||
}
|
||||
// planner weight if present
|
||||
if let Some(v) = cm.get("planner_weight").and_then(val_as_f64) {
|
||||
rec.planner_weight = v;
|
||||
rec.has_planner_weight = true;
|
||||
}
|
||||
// identity
|
||||
if let Some(Value::String(s)) = cm.get("circuitID") {
|
||||
if !s.is_empty() { rec.circuit_id = Some(s.clone()); }
|
||||
}
|
||||
if let Some(Value::String(s)) = cm.get("circuitName") {
|
||||
if !s.is_empty() { rec.circuit_name = Some(s.clone()); }
|
||||
}
|
||||
if rec.circuit_name.is_none() {
|
||||
// Fallback to the JSON key is not available here; leave empty
|
||||
}
|
||||
rec.parent_node = parent_name.map(|x| x.to_string());
|
||||
// IP count
|
||||
rec.ip_count = collect_ip_count(cm);
|
||||
rec
|
||||
}
|
||||
|
||||
// Process circuits array at this node
|
||||
if let Some(Value::Array(circuits)) = node.get("circuits") {
|
||||
for c in circuits {
|
||||
if let Value::Object(cm) = c {
|
||||
out.push(parse_circuit_map(parent_name, cm));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Also check children: some builds embed circuits directly as child entries (rare).
|
||||
if let Some(Value::Object(children)) = node.get("children") {
|
||||
for (child_name, child_value) in children.iter() {
|
||||
if let Value::Object(child_map) = child_value {
|
||||
// Treat as circuit only if it has a devices array (avoids counting structural nodes)
|
||||
let is_circuit_like = matches!(child_map.get("devices"), Some(Value::Array(_)))
|
||||
|| child_map.get("circuitID").is_some();
|
||||
if is_circuit_like {
|
||||
out.push(parse_circuit_map(Some(child_name.as_str()), child_map));
|
||||
}
|
||||
add_circuit_records(Some(child_name.as_str()), child_map, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn load_all_circuits() -> Vec<CircuitRecord> {
|
||||
let Some(path) = queuing_structure_path() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let raw = match std::fs::read_to_string(path) {
|
||||
Ok(s) => s,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let json: Value = match serde_json::from_str(&raw) {
|
||||
Ok(v) => v,
|
||||
Err(_) => return Vec::new(),
|
||||
};
|
||||
let mut circuits = Vec::new();
|
||||
if let Value::Object(map) = json {
|
||||
if let Some(Value::Object(net)) = map.get("Network") {
|
||||
for (_k, v) in net.iter() {
|
||||
if let Value::Object(node) = v {
|
||||
add_circuit_records(None, node, &mut circuits);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
circuits
|
||||
}
|
||||
|
||||
pub async fn cpu_affinity_summary() -> Json<CpuAffinitySummaryResponse> {
|
||||
let circuits = load_all_circuits();
|
||||
let mut down: HashMap<u32, (usize, f64, f64, f64)> = HashMap::new();
|
||||
let mut up: HashMap<u32, (usize, f64, f64, f64)> = HashMap::new();
|
||||
|
||||
for c in circuits.iter() {
|
||||
let ignored = c.has_planner_weight && c.planner_weight <= 0.0;
|
||||
if let Some(cpu) = c.cpu_down {
|
||||
if !ignored {
|
||||
let entry = down.entry(cpu).or_insert((0, 0.0, 0.0, 0.0));
|
||||
entry.0 += 1;
|
||||
entry.1 += c.min_down;
|
||||
entry.2 += c.max_down;
|
||||
entry.3 += c.planner_weight;
|
||||
}
|
||||
}
|
||||
if let Some(cpu) = c.cpu_up {
|
||||
if !ignored {
|
||||
let entry = up.entry(cpu).or_insert((0, 0.0, 0.0, 0.0));
|
||||
entry.0 += 1;
|
||||
entry.1 += c.min_up;
|
||||
entry.2 += c.max_up;
|
||||
entry.3 += c.planner_weight;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Union of CPUs seen in either direction
|
||||
let mut cpus: Vec<u32> = down.keys().chain(up.keys()).copied().collect();
|
||||
cpus.sort_unstable();
|
||||
cpus.dedup();
|
||||
|
||||
let mut entries = Vec::with_capacity(cpus.len());
|
||||
for cpu in cpus.into_iter() {
|
||||
let d = down.get(&cpu).cloned().unwrap_or((0, 0.0, 0.0, 0.0));
|
||||
let u = up.get(&cpu).cloned().unwrap_or((0, 0.0, 0.0, 0.0));
|
||||
entries.push(CpuAffinitySummaryEntry {
|
||||
cpu,
|
||||
download: CpuSideSummary {
|
||||
circuits: d.0,
|
||||
min_sum_mbps: d.1,
|
||||
max_sum_mbps: d.2,
|
||||
weight_sum: d.3,
|
||||
},
|
||||
upload: CpuSideSummary {
|
||||
circuits: u.0,
|
||||
min_sum_mbps: u.1,
|
||||
max_sum_mbps: u.2,
|
||||
weight_sum: u.3,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
Json(CpuAffinitySummaryResponse(entries))
|
||||
}
|
||||
|
||||
pub async fn cpu_affinity_circuits(
|
||||
Path(cpu): Path<u32>,
|
||||
Query(q): Query<CircuitsQuery>,
|
||||
) -> Json<CpuAffinityCircuitsPage> {
|
||||
let direction = q
|
||||
.direction
|
||||
.as_deref()
|
||||
.map(|s| if s.eq_ignore_ascii_case("up") { "up" } else { "down" })
|
||||
.unwrap_or("down");
|
||||
|
||||
let page = q.page.unwrap_or(1).max(1);
|
||||
let page_size = q.page_size.unwrap_or(100).clamp(1, 1000);
|
||||
let search = q.search.as_deref().map(|s| s.to_lowercase());
|
||||
|
||||
let circuits = load_all_circuits();
|
||||
let mut filtered: Vec<CircuitBrief> = circuits
|
||||
.into_iter()
|
||||
.filter(|c| match direction {
|
||||
"up" => c.cpu_up == Some(cpu),
|
||||
_ => c.cpu_down == Some(cpu),
|
||||
})
|
||||
.filter(|c| {
|
||||
if let Some(ref needle) = search {
|
||||
let id = c.circuit_id.as_deref().unwrap_or("").to_lowercase();
|
||||
let nm = c.circuit_name.as_deref().unwrap_or("").to_lowercase();
|
||||
id.contains(needle) || nm.contains(needle)
|
||||
} else {
|
||||
true
|
||||
}
|
||||
})
|
||||
.map(|c| CircuitBrief {
|
||||
circuit_id: c.circuit_id,
|
||||
circuit_name: c.circuit_name,
|
||||
parent_node: c.parent_node,
|
||||
classid: if direction == "up" {
|
||||
c.classid_up
|
||||
} else {
|
||||
c.classid_down
|
||||
},
|
||||
max_mbps: if direction == "up" { c.max_up } else { c.max_down },
|
||||
weight: c.planner_weight,
|
||||
ip_count: c.ip_count,
|
||||
ignored: c.has_planner_weight && c.planner_weight <= 0.0,
|
||||
})
|
||||
.collect();
|
||||
|
||||
// Sort by circuit name, then id
|
||||
filtered.sort_by(|a, b| {
|
||||
let an = a.circuit_name.as_deref().unwrap_or("");
|
||||
let bn = b.circuit_name.as_deref().unwrap_or("");
|
||||
let ord = an.cmp(bn);
|
||||
if ord == std::cmp::Ordering::Equal {
|
||||
a.circuit_id
|
||||
.as_deref()
|
||||
.unwrap_or("")
|
||||
.cmp(b.circuit_id.as_deref().unwrap_or(""))
|
||||
} else {
|
||||
ord
|
||||
}
|
||||
});
|
||||
|
||||
let total = filtered.len();
|
||||
let start = (page - 1) * page_size;
|
||||
let end = std::cmp::min(start + page_size, total);
|
||||
let page_items = if start < total {
|
||||
filtered[start..end].to_vec()
|
||||
} else {
|
||||
Vec::new()
|
||||
};
|
||||
|
||||
Json(CpuAffinityCircuitsPage {
|
||||
cpu,
|
||||
direction: direction.to_string(),
|
||||
total,
|
||||
page,
|
||||
page_size,
|
||||
items: page_items,
|
||||
})
|
||||
}
|
||||
|
||||
// Return all circuits (direction-filtered) without pagination to support client-side previews.
|
||||
pub async fn cpu_affinity_circuits_all(
|
||||
Query(q): Query<CircuitsQuery>,
|
||||
) -> Json<Vec<CircuitBrief>> {
|
||||
let direction = q
|
||||
.direction
|
||||
.as_deref()
|
||||
.map(|s| if s.eq_ignore_ascii_case("up") { "up" } else { "down" })
|
||||
.unwrap_or("down");
|
||||
let search = q.search.as_deref().map(|s| s.to_lowercase());
|
||||
|
||||
let circuits = load_all_circuits();
|
||||
let mut items: Vec<CircuitBrief> = circuits
|
||||
.into_iter()
|
||||
.filter(|c| {
|
||||
if let Some(ref needle) = search {
|
||||
let id = c.circuit_id.as_deref().unwrap_or("").to_lowercase();
|
||||
let nm = c.circuit_name.as_deref().unwrap_or("").to_lowercase();
|
||||
id.contains(needle) || nm.contains(needle)
|
||||
} else {
|
||||
true
|
||||
}
|
||||
})
|
||||
.map(|c| CircuitBrief {
|
||||
circuit_id: c.circuit_id,
|
||||
circuit_name: c.circuit_name,
|
||||
parent_node: c.parent_node,
|
||||
classid: if direction == "up" {
|
||||
c.classid_up
|
||||
} else {
|
||||
c.classid_down
|
||||
},
|
||||
max_mbps: if direction == "up" { c.max_up } else { c.max_down },
|
||||
weight: c.planner_weight,
|
||||
ip_count: c.ip_count,
|
||||
ignored: c.has_planner_weight && c.planner_weight <= 0.0,
|
||||
})
|
||||
.collect();
|
||||
|
||||
// Sort deterministically
|
||||
items.sort_by(|a, b| {
|
||||
let an = a.circuit_name.as_deref().unwrap_or("");
|
||||
let bn = b.circuit_name.as_deref().unwrap_or("");
|
||||
let ord = an.cmp(bn);
|
||||
if ord == std::cmp::Ordering::Equal {
|
||||
a.circuit_id
|
||||
.as_deref()
|
||||
.unwrap_or("")
|
||||
.cmp(b.circuit_id.as_deref().unwrap_or(""))
|
||||
} else {
|
||||
ord
|
||||
}
|
||||
});
|
||||
|
||||
Json(items)
|
||||
}
|
||||
|
||||
#[derive(Serialize, Clone, Debug)]
|
||||
pub struct PreviewWeightItem {
|
||||
pub key: String,
|
||||
pub weight: f64,
|
||||
}
|
||||
|
||||
// Minimal payload for preview: just (key, weight) for each circuit in a direction.
|
||||
pub async fn cpu_affinity_preview_weights(
|
||||
Query(q): Query<CircuitsQuery>,
|
||||
) -> Json<Vec<PreviewWeightItem>> {
|
||||
let direction = q
|
||||
.direction
|
||||
.as_deref()
|
||||
.map(|s| if s.eq_ignore_ascii_case("up") { "up" } else { "down" })
|
||||
.unwrap_or("down");
|
||||
let search = q.search.as_deref().map(|s| s.to_lowercase());
|
||||
|
||||
let circuits = load_all_circuits();
|
||||
let mut items: Vec<PreviewWeightItem> = circuits
|
||||
.into_iter()
|
||||
.filter(|c| {
|
||||
if let Some(ref needle) = search {
|
||||
let id = c.circuit_id.as_deref().unwrap_or("").to_lowercase();
|
||||
let nm = c.circuit_name.as_deref().unwrap_or("").to_lowercase();
|
||||
id.contains(needle) || nm.contains(needle)
|
||||
} else {
|
||||
true
|
||||
}
|
||||
})
|
||||
.map(|c| PreviewWeightItem {
|
||||
key: c
|
||||
.circuit_id
|
||||
.clone()
|
||||
.or(c.circuit_name.clone())
|
||||
.unwrap_or_default(),
|
||||
weight: if direction == "up" { c.max_up } else { c.max_down },
|
||||
})
|
||||
.collect();
|
||||
|
||||
// Deterministic ordering for ties
|
||||
items.sort_by(|a, b| {
|
||||
let ow = b.weight.partial_cmp(&a.weight).unwrap_or(std::cmp::Ordering::Equal);
|
||||
if ow == std::cmp::Ordering::Equal {
|
||||
a.key.cmp(&b.key)
|
||||
} else {
|
||||
ow
|
||||
}
|
||||
});
|
||||
|
||||
Json(items)
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
use axum::extract::Path;
|
||||
use axum::http::StatusCode;
|
||||
use axum::Json;
|
||||
use serde::Serialize;
|
||||
|
||||
use crate::urgent;
|
||||
|
||||
// Remove ANSI escape sequences (basic CSI/OSC handling) for browser display
|
||||
fn strip_ansi(input: &str) -> String {
|
||||
let bytes = input.as_bytes();
|
||||
let mut out = String::with_capacity(bytes.len());
|
||||
let mut i = 0;
|
||||
while i < bytes.len() {
|
||||
if bytes[i] == 0x1B {
|
||||
// ESC sequence
|
||||
i += 1;
|
||||
if i >= bytes.len() {
|
||||
break;
|
||||
}
|
||||
match bytes[i] as char {
|
||||
'[' => {
|
||||
// CSI: ESC [ ... final byte 0x40..=0x7E
|
||||
i += 1;
|
||||
while i < bytes.len() {
|
||||
let b = bytes[i];
|
||||
if (0x40..=0x7E).contains(&b) {
|
||||
i += 1;
|
||||
break;
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
']' => {
|
||||
// OSC: ESC ] ... BEL (0x07) or ESC \
|
||||
i += 1;
|
||||
while i < bytes.len() {
|
||||
if bytes[i] == 0x07 {
|
||||
i += 1;
|
||||
break;
|
||||
}
|
||||
if bytes[i] == 0x1B {
|
||||
// ESC
|
||||
if i + 1 < bytes.len() && bytes[i + 1] as char == '\\' {
|
||||
i += 2; // ESC \
|
||||
break;
|
||||
}
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
_ => {
|
||||
// Other ESC-seq: skip next char at least
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
out.push(bytes[i] as char);
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct UrgentStatus {
|
||||
pub has_urgent: bool,
|
||||
pub count: usize,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct UrgentItem {
|
||||
pub id: u64,
|
||||
pub ts: u64,
|
||||
pub source: String,
|
||||
pub severity: String,
|
||||
pub code: String,
|
||||
pub message: String,
|
||||
pub context: Option<String>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
pub struct UrgentList {
|
||||
pub items: Vec<UrgentItem>,
|
||||
}
|
||||
|
||||
pub async fn urgent_status() -> (StatusCode, Json<UrgentStatus>) {
|
||||
let items = urgent::list();
|
||||
(
|
||||
StatusCode::OK,
|
||||
Json(UrgentStatus { has_urgent: !items.is_empty(), count: items.len() }),
|
||||
)
|
||||
}
|
||||
|
||||
pub async fn urgent_list() -> (StatusCode, Json<UrgentList>) {
|
||||
let items = urgent::list();
|
||||
let items: Vec<UrgentItem> = items
|
||||
.into_iter()
|
||||
.map(|i| UrgentItem {
|
||||
id: i.id,
|
||||
ts: i.ts,
|
||||
source: format!("{:?}", i.source),
|
||||
severity: format!("{:?}", i.severity),
|
||||
code: i.code,
|
||||
message: strip_ansi(&i.message),
|
||||
context: i.context,
|
||||
})
|
||||
.collect();
|
||||
(StatusCode::OK, Json(UrgentList { items }))
|
||||
}
|
||||
|
||||
pub async fn urgent_clear(Path(id): Path<u64>) -> (StatusCode, &'static str) {
|
||||
let ok = urgent::clear(id);
|
||||
if ok {
|
||||
(StatusCode::OK, "OK")
|
||||
} else {
|
||||
(StatusCode::NOT_FOUND, "NOT_FOUND")
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn urgent_clear_all() -> (StatusCode, &'static str) {
|
||||
urgent::clear_all();
|
||||
(StatusCode::OK, "OK")
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
<div class="row">
|
||||
<div class="col-12">
|
||||
<div class="row align-items-end mb-3">
|
||||
<div class="col-12 col-lg-8">
|
||||
<h5 class="mb-1"><i class="fa fa-microchip"></i> CPU Affinity</h5>
|
||||
<p class="text-muted mb-0">View current circuit placement per CPU/queue from the active TC plan.</p>
|
||||
</div>
|
||||
<div class="col-12 col-lg-4">
|
||||
<div class="d-flex flex-wrap justify-content-lg-end align-items-end gap-2 mt-2 mt-lg-0">
|
||||
<div>
|
||||
<label for="pageSize" class="form-label mb-0">Page Size</label>
|
||||
<input type="number" id="pageSize" class="form-control form-control-sm" value="50" min="10" max="1000">
|
||||
</div>
|
||||
<div class="d-flex align-items-end">
|
||||
<span id="binpackBadge" class="badge bg-secondary">Binpacking: …</span>
|
||||
</div>
|
||||
<div class="ms-auto ms-lg-0">
|
||||
<button class="btn btn-sm btn-outline-secondary" id="btnRefresh">
|
||||
<i class="fa fa-rotate"></i> Refresh
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="cpuSummary" class="mb-4">
|
||||
<i class="fa fa-spinner fa-spin"></i> Loading CPU summary...
|
||||
</div>
|
||||
|
||||
<hr/>
|
||||
|
||||
<div id="cpuDetails">
|
||||
<div class="d-flex justify-content-between align-items-center mb-2">
|
||||
<div>
|
||||
<strong class="text-secondary">Circuits</strong>
|
||||
<span id="currentCpuLabel" class="text-muted"></span>
|
||||
</div>
|
||||
<div class="btn-group btn-group-sm" role="group" aria-label="Pagination">
|
||||
<button class="btn btn-outline-secondary" id="btnPrev">Prev</button>
|
||||
<button class="btn btn-outline-secondary" id="btnNext">Next</button>
|
||||
</div>
|
||||
</div>
|
||||
<div id="circuitsTable">
|
||||
<i class="fa fa-info-circle"></i> <span class="text-secondary">Select a CPU from the table above.</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script src="cpu_weights.js%CACHEBUSTERS%"></script>
|
||||
@@ -87,6 +87,12 @@
|
||||
<i class="fa fa-fw fa-centerline fa-globe nav-icon"></i> ASN Explorer
|
||||
</a>
|
||||
</li>
|
||||
<!-- CPU Affinity -->
|
||||
<li class="nav-item">
|
||||
<a class="nav-link" href="cpu_weights.html">
|
||||
<i class="fa fa-fw fa-centerline fa-microchip nav-icon"></i> CPU Affinity
|
||||
</a>
|
||||
</li>
|
||||
<!-- Statistics -->
|
||||
%%LTS_LINK%%
|
||||
%%API_LINK%%
|
||||
@@ -110,6 +116,7 @@
|
||||
<i class="fa fa-fw fa-centerline fa-recycle"></i> Reload LibreQoS
|
||||
</a>
|
||||
</li>
|
||||
%%URGENT_STATUS%%
|
||||
%%SCHEDULER_STATUS%%
|
||||
<hr />
|
||||
<!-- Dark mode switch -->
|
||||
@@ -204,6 +211,27 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Urgent Issues Modal -->
|
||||
<div class='modal fade' id='urgentModal' tabindex='-1' aria-labelledby='urgentModalLabel' aria-hidden='true'>
|
||||
<div class='modal-dialog modal-xl modal-dialog-scrollable'>
|
||||
<div class='modal-content'>
|
||||
<div class='modal-header'>
|
||||
<h1 class='modal-title fs-5 text-secondary' id='urgentModalLabel'>Urgent Issues</h1>
|
||||
<button type='button' class='btn btn-outline-secondary btn-sm me-2' id='urgentClearAll' title='Acknowledge All'><i class='fa fa-times'></i></button>
|
||||
<button type='button' class='btn-close' data-bs-dismiss='modal' aria-label='Close'></button>
|
||||
</div>
|
||||
<div class='modal-body'>
|
||||
<div id='urgentListContainer'>
|
||||
<div class="text-center text-muted"><i class='fa fa-spinner fa-spin'></i> Loading...</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class='modal-footer'>
|
||||
<button type='button' class='btn btn-secondary' data-bs-dismiss='modal'>Close</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<footer class="justify-content-center text-body-tertiary text-center mt-2">
|
||||
LibreQoS %%VERSION%%. © Copyright 2022-<span id="year"></span>, LibreQoE LLC. All Rights Reserved.
|
||||
</footer>
|
||||
@@ -228,6 +256,103 @@
|
||||
|
||||
document.getElementById('year').textContent = new Date().getFullYear();
|
||||
|
||||
|
||||
// Urgent Issues: poll status, update badge, and show modal with details
|
||||
(function(){
|
||||
const containerId = 'urgentStatus';
|
||||
const linkId = 'urgentStatusLink';
|
||||
const badgeId = 'urgentBadge';
|
||||
|
||||
function ensurePlaceholder(){
|
||||
// no-op: server injects a placeholder; if not present, skip
|
||||
return document.getElementById(containerId) !== null;
|
||||
}
|
||||
|
||||
function renderStatus(count){
|
||||
const cont = document.getElementById(containerId);
|
||||
if (!cont) return;
|
||||
const cls = count > 0 ? 'text-danger' : 'text-secondary';
|
||||
const icon = count > 0 ? 'fa-bell' : 'fa-bell-slash';
|
||||
cont.innerHTML = `
|
||||
<a class="nav-link ${cls}" href="#" id="${linkId}">
|
||||
<i class="fa fa-fw fa-centerline ${icon}"></i> Urgent Issues
|
||||
<span id="${badgeId}" class="badge bg-danger ${count>0?'':'d-none'}">${count}</span>
|
||||
</a>`;
|
||||
$("#"+containerId).off("click").on("click", `#${linkId}`, (e)=>{
|
||||
e.preventDefault();
|
||||
showModal();
|
||||
});
|
||||
}
|
||||
|
||||
function poll(){
|
||||
if (!ensurePlaceholder()) return;
|
||||
$.get('/local-api/urgent/status', (res)=>{
|
||||
renderStatus(res.count||0);
|
||||
});
|
||||
}
|
||||
|
||||
function showModal(){
|
||||
const modalEl = document.getElementById('urgentModal');
|
||||
if (!modalEl) return;
|
||||
new bootstrap.Modal(modalEl,{focus:true}).show();
|
||||
const holder = document.getElementById('urgentListContainer');
|
||||
holder.innerHTML = `<div class="text-center text-muted"><i class='fa fa-spinner fa-spin'></i> Loading...</div>`;
|
||||
$.get('/local-api/urgent/list', (res)=>{
|
||||
const items = res.items||[];
|
||||
if (items.length===0){ holder.innerHTML = '<div class="text-center text-success">No urgent issues.</div>'; return; }
|
||||
const table = document.createElement('table');
|
||||
table.className = 'table table-sm table-striped';
|
||||
const tbody = document.createElement('tbody');
|
||||
items.forEach(it=>{
|
||||
const tr = document.createElement('tr');
|
||||
const td = document.createElement('td');
|
||||
const when = new Date(it.ts*1000).toLocaleString();
|
||||
const sev = it.severity === 'Error' ? 'danger' : 'warning';
|
||||
td.innerHTML = `
|
||||
<div>
|
||||
<span class="badge bg-${sev}">${it.severity}</span>
|
||||
<strong class="ms-2">${it.code}</strong>
|
||||
<span class="text-muted ms-2">(${it.source})</span>
|
||||
<span class="text-muted float-end">${when}</span>
|
||||
<a href="#" class="text-secondary float-end ms-3 urgent-clear" data-id="${it.id}" title="Acknowledge"><i class="fa fa-times"></i></a>
|
||||
</div>
|
||||
<div class="mt-1" style="white-space: pre-wrap;">${it.message}</div>
|
||||
${it.context?`<pre class="mt-2">${it.context}</pre>`:''}
|
||||
`;
|
||||
tr.appendChild(td);
|
||||
tbody.appendChild(tr);
|
||||
});
|
||||
table.appendChild(tbody);
|
||||
// Clear existing content and insert table
|
||||
holder.innerHTML = '';
|
||||
holder.appendChild(table);
|
||||
// Bind clear buttons (acknowledge)
|
||||
$(holder).off('click').on('click','a.urgent-clear', function(e){
|
||||
e.preventDefault();
|
||||
const id = $(this).data('id');
|
||||
$.post(`/local-api/urgent/clear/${id}`, ()=>{ showModal(); poll(); });
|
||||
});
|
||||
}).fail(()=>{
|
||||
holder.innerHTML = '<div class="text-center text-danger">Failed to load urgent issues.</div>';
|
||||
});
|
||||
}
|
||||
|
||||
// Initial placeholder in case server didn’t inject content (defensive)
|
||||
if (!document.getElementById(containerId)){
|
||||
const ul = document.querySelector('.sidebar .navbar-nav');
|
||||
if (ul){
|
||||
const li = document.createElement('li'); li.className='nav-item'; li.id=containerId; ul.appendChild(li);
|
||||
}
|
||||
}
|
||||
// Clear all action
|
||||
$(document).off('click','#urgentClearAll').on('click','#urgentClearAll', function(){
|
||||
$.post('/local-api/urgent/clear_all', ()=>{ showModal(); poll(); });
|
||||
});
|
||||
|
||||
// Start polling
|
||||
poll();
|
||||
setInterval(poll, 30000);
|
||||
})();
|
||||
</script>
|
||||
|
||||
</body>
|
||||
|
||||
@@ -63,6 +63,7 @@ pub(super) fn static_routes() -> Result<Router> {
|
||||
"config_wispgate.html",
|
||||
"config_stormguard.html",
|
||||
"api.html",
|
||||
"cpu_weights.html",
|
||||
];
|
||||
|
||||
// Iterate through pages and construct the router
|
||||
|
||||
@@ -167,6 +167,17 @@ pub async fn apply_templates(
|
||||
"##;
|
||||
let byte_string = byte_string.replace("%%SCHEDULER_STATUS%%", scheduler_placeholder);
|
||||
|
||||
// Placeholder for urgent issues indicator
|
||||
let urgent_placeholder = r##"
|
||||
<li class="nav-item" id="urgentStatus">
|
||||
<a class="nav-link text-secondary" href="#" id="urgentStatusLink">
|
||||
<i class="fa fa-fw fa-centerline fa-bell-slash"></i> Urgent Issues
|
||||
<span id="urgentBadge" class="badge bg-danger d-none">0</span>
|
||||
</a>
|
||||
</li>
|
||||
"##;
|
||||
let byte_string = byte_string.replace("%%URGENT_STATUS%%", urgent_placeholder);
|
||||
|
||||
let byte_string = byte_string.replace("%CACHEBUSTERS%", &format!("?gh={}", GIT_HASH));
|
||||
if let Some(length) = res_parts.headers.get_mut("content-length") {
|
||||
*length = HeaderValue::from(byte_string.len());
|
||||
|
||||
@@ -10,6 +10,7 @@ pub enum PublishedChannels {
|
||||
RttHistogram,
|
||||
FlowCount,
|
||||
TopDownloads,
|
||||
TopUploads,
|
||||
WorstRTT,
|
||||
WorstRetransmits,
|
||||
TopFlowsBytes,
|
||||
@@ -21,6 +22,7 @@ pub enum PublishedChannels {
|
||||
Cpu,
|
||||
Ram,
|
||||
TreeSummary,
|
||||
TreeSummaryL2,
|
||||
QueueStatsTotal,
|
||||
NetworkTree,
|
||||
NetworkTreeClients,
|
||||
|
||||
@@ -24,6 +24,7 @@ mod top_flows;
|
||||
mod asn_top;
|
||||
mod tree_capacity;
|
||||
mod tree_summary;
|
||||
mod tree_summary_l2;
|
||||
|
||||
use crate::system_stats::SystemStats;
|
||||
pub use network_tree::all_circuits;
|
||||
@@ -54,6 +55,7 @@ async fn one_second_cadence(
|
||||
rtt_histogram::rtt_histo(channels.clone(), bus_tx.clone()),
|
||||
flow_counter::flow_count(channels.clone(), bus_tx.clone()),
|
||||
top_10::top_10_downloaders(channels.clone(), bus_tx.clone()),
|
||||
top_10::top_10_uploaders(channels.clone(), bus_tx.clone()),
|
||||
top_10::worst_10_downloaders(channels.clone(), bus_tx.clone()),
|
||||
top_10::worst_10_retransmit(channels.clone(), bus_tx.clone()),
|
||||
top_flows::top_flows_bytes(channels.clone(), bus_tx.clone()),
|
||||
@@ -65,6 +67,7 @@ async fn one_second_cadence(
|
||||
flow_endpoints::flow_duration(channels.clone(), bus_tx.clone()),
|
||||
endpoint_latlon::endpoint_latlon(channels.clone(), bus_tx.clone()),
|
||||
tree_summary::tree_summary(channels.clone(), bus_tx.clone()),
|
||||
tree_summary_l2::tree_summary_l2(channels.clone()),
|
||||
network_tree::all_subscribers(channels.clone(), bus_tx.clone()),
|
||||
queue_stats_total::queue_stats_totals(channels.clone()),
|
||||
network_tree::network_tree(channels.clone(), bus_tx.clone()),
|
||||
|
||||
@@ -48,6 +48,48 @@ pub async fn top_10_downloaders(
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn top_10_uploaders(
|
||||
channels: Arc<PubSub>,
|
||||
bus_tx: Sender<(tokio::sync::oneshot::Sender<BusReply>, BusRequest)>,
|
||||
) {
|
||||
if !channels
|
||||
.is_channel_alive(PublishedChannels::TopUploads)
|
||||
.await
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
let (tx, rx) = tokio::sync::oneshot::channel::<BusReply>();
|
||||
let request = BusRequest::GetTopNUploaders { start: 0, end: 10 };
|
||||
if let Err(e) = bus_tx.send((tx, request)).await {
|
||||
tracing::warn!("TopUploads: failed to send request to bus: {:?}", e);
|
||||
return;
|
||||
}
|
||||
let replies = match rx.await {
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
tracing::warn!("TopUploads: failed to receive throughput from bus: {:?}", e);
|
||||
return;
|
||||
}
|
||||
};
|
||||
for reply in replies.responses.into_iter() {
|
||||
if let BusResponse::TopUploaders(top) = reply {
|
||||
let result: Vec<IpStatsWithPlan> = top.iter().map(|stat| stat.into()).collect();
|
||||
|
||||
let message = json!(
|
||||
{
|
||||
"event": PublishedChannels::TopUploads.to_string(),
|
||||
"data": result
|
||||
}
|
||||
)
|
||||
.to_string();
|
||||
channels
|
||||
.send(PublishedChannels::TopUploads, message)
|
||||
.await;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub async fn worst_10_downloaders(
|
||||
channels: Arc<PubSub>,
|
||||
bus_tx: Sender<(tokio::sync::oneshot::Sender<BusReply>, BusRequest)>,
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
use crate::node_manager::ws::publish_subscribe::PubSub;
|
||||
use crate::node_manager::ws::published_channels::PublishedChannels;
|
||||
use crate::shaped_devices_tracker::NETWORK_JSON;
|
||||
use lqos_config::NetworkJsonTransport;
|
||||
use serde_json::json;
|
||||
use std::collections::BTreeMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Publishes a curated two-level summary of the network tree:
|
||||
/// root -> first-level parents -> top-N second-level children across all parents.
|
||||
/// N is currently fixed at 10. Only includes second-level items, grouped by parent id.
|
||||
/// Data shape:
|
||||
/// {
|
||||
/// "event": "TreeSummaryL2",
|
||||
/// "data": [ [ parent_id, [ [child_id, NetworkJsonTransport], ... ] ], ... ]
|
||||
/// }
|
||||
pub async fn tree_summary_l2(channels: Arc<PubSub>) {
|
||||
if !channels
|
||||
.is_channel_alive(PublishedChannels::TreeSummaryL2)
|
||||
.await
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// Build top-N second-level children across all first-level parents
|
||||
let grouped: Vec<(usize, Vec<(usize, NetworkJsonTransport)>)> = {
|
||||
let net_json = NETWORK_JSON.read();
|
||||
let nodes = net_json.get_nodes_when_ready();
|
||||
// Collect candidates as (parent_idx, child_idx, transport, total_bytes_per_sec)
|
||||
let mut candidates: Vec<(usize, usize, NetworkJsonTransport, u64)> = Vec::new();
|
||||
|
||||
// Identify first-level parents (immediate_parent == Some(0))
|
||||
for (p_idx, p_node) in nodes.iter().enumerate() {
|
||||
if p_node.immediate_parent == Some(0) {
|
||||
// For each child-of-child under this parent
|
||||
for (c_idx, c_node) in nodes.iter().enumerate() {
|
||||
if c_node.immediate_parent == Some(p_idx) {
|
||||
let t = c_node.clone_to_transit();
|
||||
let total = t.current_throughput.0 + t.current_throughput.1;
|
||||
candidates.push((p_idx, c_idx, t, total));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sort by total throughput descending and cap to N
|
||||
candidates.sort_by(|a, b| b.3.cmp(&a.3));
|
||||
let n: usize = 10;
|
||||
if candidates.len() > n {
|
||||
candidates.truncate(n);
|
||||
}
|
||||
|
||||
// Group by parent id
|
||||
let mut map: BTreeMap<usize, Vec<(usize, NetworkJsonTransport)>> = BTreeMap::new();
|
||||
for (p_idx, c_idx, t, _total) in candidates.into_iter() {
|
||||
map.entry(p_idx).or_default().push((c_idx, t));
|
||||
}
|
||||
map.into_iter().collect()
|
||||
};
|
||||
|
||||
let message = json!({
|
||||
"event": PublishedChannels::TreeSummaryL2.to_string(),
|
||||
"data": grouped,
|
||||
})
|
||||
.to_string();
|
||||
channels
|
||||
.send(PublishedChannels::TreeSummaryL2, message)
|
||||
.await;
|
||||
}
|
||||
|
||||
@@ -217,7 +217,8 @@ pub fn get_all_circuits() -> BusResponse {
|
||||
let mut device_id = None;
|
||||
let mut device_name = None;
|
||||
let mut parent_node = None;
|
||||
let mut plan = DownUpOrder { down: 0.0, up: 0.0 };
|
||||
// Plan is expressed in Mbps as f32
|
||||
let mut plan: DownUpOrder<f32> = DownUpOrder { down: 0.0, up: 0.0 };
|
||||
let lookup = match ip {
|
||||
IpAddr::V4(ip) => ip.to_ipv6_mapped(),
|
||||
IpAddr::V6(ip) => ip,
|
||||
@@ -228,8 +229,8 @@ pub fn get_all_circuits() -> BusResponse {
|
||||
device_id = Some(devices.devices[*c.1].device_id.clone());
|
||||
device_name = Some(devices.devices[*c.1].device_name.clone());
|
||||
parent_node = Some(devices.devices[*c.1].parent_node.clone());
|
||||
plan.down = devices.devices[*c.1].download_max_mbps;
|
||||
plan.up = devices.devices[*c.1].upload_max_mbps;
|
||||
plan.down = devices.devices[*c.1].download_max_mbps.round();
|
||||
plan.up = devices.devices[*c.1].upload_max_mbps.round();
|
||||
}
|
||||
|
||||
Circuit {
|
||||
|
||||
@@ -347,6 +347,50 @@ pub fn top_n(start: u32, end: u32) -> BusResponse {
|
||||
BusResponse::TopDownloaders(result)
|
||||
}
|
||||
|
||||
pub fn top_n_up(start: u32, end: u32) -> BusResponse {
|
||||
let mut full_list: Vec<TopList> = {
|
||||
let tp_cycle = THROUGHPUT_TRACKER
|
||||
.cycle
|
||||
.load(std::sync::atomic::Ordering::Relaxed);
|
||||
THROUGHPUT_TRACKER
|
||||
.raw_data
|
||||
.lock()
|
||||
.iter()
|
||||
.filter(|(k, _v)| !k.as_ip().is_loopback())
|
||||
.filter(|(_k, d)| retire_check(tp_cycle, d.most_recent_cycle))
|
||||
.map(|(k, te)| {
|
||||
(
|
||||
*k,
|
||||
te.bytes_per_second,
|
||||
te.packets_per_second,
|
||||
te.median_latency().unwrap_or(0.0),
|
||||
te.tc_handle,
|
||||
te.circuit_id.as_ref().unwrap_or(&String::new()).clone(),
|
||||
down_up_divide(te.tcp_retransmits, te.tcp_packets),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
full_list.sort_by(|a, b| b.1.up.cmp(&a.1.up));
|
||||
let result = full_list
|
||||
.iter()
|
||||
//.skip(start as usize)
|
||||
.take((end as usize) - (start as usize))
|
||||
.map(
|
||||
|(ip, bytes, packets, median_rtt, tc_handle, circuit_id, tcp_retransmits)| IpStats {
|
||||
ip_address: ip.as_ip().to_string(),
|
||||
circuit_id: circuit_id.clone(),
|
||||
bits_per_second: bytes.to_bits_from_bytes(),
|
||||
packets_per_second: *packets,
|
||||
median_tcp_rtt: *median_rtt,
|
||||
tc_handle: *tc_handle,
|
||||
tcp_retransmits: *tcp_retransmits,
|
||||
},
|
||||
)
|
||||
.collect();
|
||||
BusResponse::TopUploaders(result)
|
||||
}
|
||||
|
||||
pub fn worst_n(start: u32, end: u32) -> BusResponse {
|
||||
let mut full_list: Vec<TopList> = {
|
||||
let tp_cycle = THROUGHPUT_TRACKER
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
use std::collections::VecDeque;
|
||||
use std::sync::atomic::{AtomicU64, Ordering};
|
||||
|
||||
use lqos_bus::{UrgentIssue, UrgentSeverity, UrgentSource};
|
||||
use parking_lot::Mutex;
|
||||
|
||||
use lqos_utils::unix_time::unix_now;
|
||||
|
||||
static NEXT_ID: AtomicU64 = AtomicU64::new(0);
|
||||
static URGENT: Mutex<VecDeque<UrgentIssue>> = Mutex::new(VecDeque::new());
|
||||
|
||||
const MAX_ISSUES: usize = 100;
|
||||
const TTL_SECONDS: u64 = 24 * 60 * 60; // 24h
|
||||
const DEDUPE_WINDOW_SECONDS: u64 = 300; // 5 minutes
|
||||
|
||||
fn now_unix() -> u64 {
|
||||
if let Ok(n) = unix_now() {
|
||||
n
|
||||
} else {
|
||||
0
|
||||
}
|
||||
}
|
||||
|
||||
fn prune_expired(q: &mut VecDeque<UrgentIssue>) {
|
||||
let now = now_unix();
|
||||
while let Some(front) = q.front() {
|
||||
if front.ts + TTL_SECONDS < now {
|
||||
q.pop_front();
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
while q.len() > MAX_ISSUES {
|
||||
q.pop_front();
|
||||
}
|
||||
}
|
||||
|
||||
pub fn submit(
|
||||
source: UrgentSource,
|
||||
severity: UrgentSeverity,
|
||||
code: String,
|
||||
message: String,
|
||||
context: Option<String>,
|
||||
dedupe_key: Option<String>,
|
||||
) {
|
||||
let ts = now_unix();
|
||||
let id = NEXT_ID.fetch_add(1, Ordering::Relaxed) + 1;
|
||||
let mut guard = URGENT.lock();
|
||||
prune_expired(&mut guard);
|
||||
|
||||
// Dedupe: same (code, dedupe_key) within window → update timestamp/message
|
||||
let key = dedupe_key.clone().unwrap_or_else(|| code.clone());
|
||||
if let Some(existing) = guard.iter_mut().rev().find(|i| {
|
||||
(i.code == code)
|
||||
&& (i.dedupe_key.as_deref().unwrap_or("") == key)
|
||||
&& (ts.saturating_sub(i.ts) <= DEDUPE_WINDOW_SECONDS)
|
||||
}) {
|
||||
existing.ts = ts;
|
||||
existing.message = message;
|
||||
existing.context = context;
|
||||
return;
|
||||
}
|
||||
|
||||
let issue = UrgentIssue {
|
||||
id,
|
||||
ts,
|
||||
source,
|
||||
severity,
|
||||
code,
|
||||
message,
|
||||
context,
|
||||
dedupe_key: Some(key),
|
||||
};
|
||||
guard.push_back(issue);
|
||||
prune_expired(&mut guard);
|
||||
}
|
||||
|
||||
pub fn list() -> Vec<UrgentIssue> {
|
||||
let mut guard = URGENT.lock();
|
||||
prune_expired(&mut guard);
|
||||
let mut v: Vec<UrgentIssue> = guard.iter().cloned().collect();
|
||||
v.sort_by_key(|i| i.ts);
|
||||
v.reverse();
|
||||
v
|
||||
}
|
||||
|
||||
pub fn clear(id: u64) -> bool {
|
||||
let mut guard = URGENT.lock();
|
||||
if let Some(pos) = guard.iter().position(|i| i.id == id) {
|
||||
guard.remove(pos);
|
||||
true
|
||||
} else {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
pub fn clear_all() {
|
||||
let mut guard = URGENT.lock();
|
||||
guard.clear();
|
||||
}
|
||||
Reference in new issue
Block a user