Commit Graph

1527 Commits

Author SHA1 Message Date
Herbert "TheBracket
1ada4e5a5c
Build deb as part of Continuous Integration (#281)
Build the .deb release as part of the Github Actions run. The .deb file is available for download as an artifact of that build for 90 days. Please squash this commit.
2023-03-20 08:32:04 -07:00
Robert Chacón
44518e76a5
Fix race condition where partial reload runs at same time 2023-03-19 20:50:22 -06:00
Robert Chacón
b9adea70f4
Multiple fixes to partial reload 2023-03-15 20:25:56 -06:00
Robert Chacón
1d3ce0f725
Update README.md 2023-03-10 08:19:50 -07:00
Herbert "TheBracket
4f9bc7e867
Merge pull request #279 from LibreQoE/the-funnel
Merge the funnel branch
2023-03-09 09:12:36 -06:00
Dave Taht
38a6ccbbba Merge branch 'main' of github.com:LibreQoE/LibreQoS 2023-03-09 06:52:50 -08:00
Dave Taht
c2098880c3 InterfaceA vs InterfaceB
So far new users have got these two parameters inverted more
than 50% of the time, with roughly the same probability of
toast landing buttered side down.

Rewriting the python to read the toml, or accessing rust via
the bus, would make this mistake impossible.

I would really like to retire ispConfig.py entirely.
2023-03-09 06:51:07 -08:00
Herbert Wolverson
23736eaab7 Correct logic for detecting if the bandwidth test is already running. compare_exchange returns the CURRENT value, not the NEW value. 2023-03-09 14:25:43 +00:00
Robert Chacón
f102b6622e
Screenshot update 2023-03-08 15:34:31 -07:00
Herbert Wolverson
798e65b107 Properly redact funnel entry names on the circuit_queue page. 2023-03-08 22:23:18 +00:00
Herbert Wolverson
6575517040 Squish JSON into MsgPack for most common queries, resulting in a big reduction in bandwidth usage. 2023-03-08 21:08:38 +00:00
Herbert Wolverson
b1091a94aa Add the beginnings of internal stats keeping for lqosd and a not-advertised /api/stats page to view. 2023-03-08 19:25:30 +00:00
Herbert Wolverson
bbbe1e5b83 The watched queues structure no longer RwLocks - it's an interior mutable DashMap now. 2023-03-08 18:17:17 +00:00
Herbert Wolverson
c5dca04ade Update tree tracker RTT store to use a DashSet for lock-free access. 2023-03-08 18:01:10 +00:00
Herbert Wolverson
10cf95e5c9 Success: the ThroughputTracker system is lock-free, other than interior locking on the DashMap. 2023-03-08 17:48:36 +00:00
Herbert Wolverson
395b8e0857 Throughput tracker unlocking part 2, cycle is an atomic. 2023-03-08 17:29:15 +00:00
Herbert Wolverson
ae1d5efb4a Throughput tracker refactor part 1: raw_data is now a DashMap (no locking required). 2023-03-08 17:19:35 +00:00
Herbert Wolverson
497abd15f8 The last of the periodic lqos_node_manager bus requests is gone. Unknown devices and host counts are calculated when requested, leading to zero bus usage when you aren't viewing the GUI. 2023-03-08 17:06:25 +00:00
Herbert Wolverson
b47032d006 Overall RTT histogram stats are no longer stored globally in lqos_node_manager. Instead, they are only calculated when requested. Slightly higher utilization with lots of people using the node manager, and no bus traffic during the 99% of the time when its idle. 2023-03-08 16:45:25 +00:00
Herbert Wolverson
67c8f9dec0 Worst 10 stats are no longer stored globally in lqos_node_manager. Instead, they are only calculated when requested. Slightly higher utilization with lots of people using the node manager, and no bus traffic during the 99% of the time when its idle. 2023-03-08 16:38:19 +00:00
Herbert Wolverson
9f8bf22ff9 Top 10 stats are no longer stored globally in lqos_node_manager. Instead, they are only calculated when requested. Slightly higher utilization with lots of people using the node manager, and no bus traffic during the 99% of the time when its idle. 2023-03-08 16:32:08 +00:00
Herbert Wolverson
1053693785 Remove a mut I forgot about 2023-03-08 16:16:44 +00:00
Herbert Wolverson
b6e2091152 Further reduce write locking of NETWORK_JSON by utilizing interior mutability on the RTT tracker. 2023-03-08 16:16:21 +00:00
Herbert Wolverson
1376d7e812 Elide some write locks on NETWORK_JSON altogether by making
the throughput counts atomic with interior mutability.
2023-03-08 15:58:52 +00:00
Herbert Wolverson
ee47549728 Tighten up write locks for NETWORK_JSON 2023-03-08 15:31:29 +00:00
Herbert Wolverson
1196f7a7fb Remove unused code. 2023-03-07 21:44:31 +00:00
Herbert Wolverson
67cc8d8e99 Large batch of improvements:
* The JavaScript RingBuffer structure updated correctly.
* Replaced the funnel graph with text - easier to read.
* Discovered that the current "parking_lot" could become unstable
  under very heavy load, and only with "fat" LTO. Since it's
  no longer recommended (recent change), removed it.
* Replaced the "lazy_static" macro suite with the newly recommended
  "once_cell" system. Less code.
* Full source format.
* Update some dependency versions.
2023-03-07 21:37:23 +00:00
Herbert Wolverson
9fa1318350 Lots of polish. Funnel chart on the front page, rename menu item to tree, move login to the right section, breadcrumbs in the tree. 2023-03-07 15:41:15 +00:00
Herbert Wolverson
fc5b9ad3d4 Stop storing a node-manager wide ringbuffer of data utilization, instead request it when it's wanted. Use the same MultiRingBuffer class, with added render method, to handle front-page utilization. 2023-03-06 22:28:36 +00:00
Herbert Wolverson
39f605e195 Add a MultiRingBuffer type, and use it to cleanly emit stacked charts for the tree system. 2023-03-06 21:44:04 +00:00
Herbert Wolverson
664d29e783 Create RttHistogram class, and use it for both histogram displays. (DRY). 2023-03-06 21:12:07 +00:00
Herbert Wolverson
a2b4c85886 Colorize based on percentage of limit used 2023-03-06 19:41:47 +00:00
Herbert Wolverson
e536f47395 Funnel now displays network sites and clients, supports redaction, and graphing. 2023-03-06 19:23:34 +00:00
Herbert Wolverson
0042bb5f1b Don't include auto-generated 'children' nodes in the display tree. 2023-03-06 16:12:20 +00:00
Herbert Wolverson
78a45515e6 Update to latest crate dependencies. 2023-03-06 16:04:43 +00:00
Herbert Wolverson
9392b43e3c Proper fix for submitting IP/CPU/Queue mapping batches.
* Bring the per-client buffer size back down to a reasonable 2k.
* Divide submission batches into groups and submit those. It's
  still MASSIVELY faster, but it can't fall victim to guessing
  the number of batches incorrectly.
2023-03-06 15:58:57 +00:00
Herbert Wolverson
bfe9601be1 Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-03-05 14:29:21 +00:00
Dave Taht
875ab4b8dc Support 80k Queues in a batch
a 2048000 size read buffer will support 80k queues being installed
in a batch.

H/T Lake Linx and Herbert for this improvement. Perhaps it too
can be improved to have no limit, or the right limit, in the future.
2023-03-04 19:57:05 -08:00
Herbert Wolverson
f64862a8ff Network funnel - work in progress
Step 1 of the network funnel

* network.json reader now tags throughput entries with their tree
  location and parents to whom data should be applied.
* Data flows "up the tree", giving totals all the way up.
* Simple network map page for displaying the data while it's worked
  on.
2023-03-04 16:58:17 +00:00
Robert Chacón
2843061bbd
Merge pull request #278 from LibreQoE/toggleDisplayNameOrAddress
Integrations Improvement - Toggle using display name or address for Circuit Name
2023-03-03 20:40:28 -07:00
Robert Chacón
2e50a2a108
Update integrationSplynx.py 2023-03-03 20:38:32 -07:00
Robert Chacón
fae48c85a5
Update ispConfig.example.py 2023-03-03 20:35:15 -07:00
Robert Chacón
0f9a2b0e21
Update integrationSplynx.py 2023-03-03 20:34:18 -07:00
Robert Chacón
b100870e3c
Update integrationCommon.py 2023-03-03 20:28:22 -07:00
Robert Chacón
6c84100832
Update integrationUISP.py 2023-03-03 20:27:36 -07:00
Robert Chacón
aa3184c64b
Update integrationSplynx.py 2023-03-03 19:50:47 -07:00
Robert Chacón
38fba4a0a4
Add warnings, try to troubleshoot issue 273 2023-03-03 18:30:17 -07:00
Herbert Wolverson
7e5b432253 First refactor towards the "funnel" - shaped devices in lqosd
ShapedDevices.csv is now monitored in lqosd. This brings some
advantages:

* The Tracked Devices list now knows the circuit id association
  for every tracked IP.
* The associations auto-update after a ShapedDevices reload.
* The webserver is no longer doing Trie lookups to figure
  out what name to display.

Moving forwards, this will allow for stats gathering to group
IPs by circuit, and allow calculation of the "funnel".
2023-03-03 21:42:22 +00:00
Herbert Wolverson
42c2c63f55 In lqosd, replace lazy_static with once_cell, per Rust best practices
These days the Rust team are recommending "once_cell", which has
simpler syntax and does the same thing.
2023-03-03 20:26:14 +00:00
Herbert Wolverson
99dc9fc066 Add batching to LibreQoS.py python XDP mapping system
* Add a new Python-exported class ('BatchedCommands') to the
  Python-Rust library.
* Replace direct calls to xdp_iphash_cpu_cmdline with batched
  commands.
* Execute the single batch and obtain counts from the batch.
2023-03-03 16:52:34 +00:00