Commit Graph

1012 Commits

Author SHA1 Message Date
Dave Taht
5be5111954 Use the cake types that are defined in the cake qdisc
While json sends everything around as a u64, many of the cake
variables internally and in the protocol, are 16-32 bit. It
seemed a major memory savings to have the types more closely
match the underlying reality, and one small step towards
actually getting these values from the kernel more directly.

In terms of surprise several of those can actually roll over,
which is something we need to look out for in the future.
2023-01-15 14:38:47 +00:00
Herbert Wolverson
b620826476 Replace BpfMap and BpfPerCpuMap's dump_vec function with a
callback-based setup that elides the need to clone data on the
hot path.

Clean up the throughput data collection into clearly defined
stages, using Rayon where appropriate to speed up iterator-based
tasks.

This removes: 2 memcpy and 1 memmove per circuit, per second.
2023-01-14 17:47:36 +00:00
Herbert Wolverson
66356d66d2 Don't unwrap() - use a result for proper error handling and not terminating the queue tracking when monitor mode or something unexpected happens. 2023-01-13 19:43:35 +00:00
Herbert Wolverson
c2ae327c48 Rename "use_kernel_bridge" in "/etc/lqos" to "use_xdp_bridge"
The previous wording was confusing, making people think that the
configuration entry was referring to the Linux bridge - while it
meant "use the XDP bridge that happens to run in kernel space".

The new wording should be more obvious.
2023-01-13 18:18:46 +00:00
Herbert Wolverson
fecb27c6bc Only show unknown IPs from the last 5 minutes.
IMPORTANT: run "remove_pinned_maps.sh" before you run the new
version of lqosd. The eBPF map structure has changed.

1) Add a "last_seen" value to the map_traffic eBPF map.
2) Whenever traffic is seen, update "last_seen" to equal the
   result of a bpf_ktime_get_boot_ns() call.
3) When retrieving the unknown IP list, perform a syscall to
   obtain the time since boot in nanoseconds and subtract
   five minutes.
4) Filter out any unknown IPs that exceed the 5 minute window.

This should solve the problem with "unknown IPs" filling up
on start, and gradually accumulating.
2023-01-13 17:48:20 +00:00
Herbert Wolverson
25234cc28a Add suggestion to run remove_pinned_maps to the Rust builder. 2023-01-13 16:40:19 +00:00
Herbert Wolverson
f3a5739133 Add 'download unknown' link to the unknown IPs page. 2023-01-13 14:16:48 +00:00
Herbert Wolverson
d7034d1e42 Update testing file to indicate that the branch is no longer needed. 2023-01-13 13:59:50 +00:00
Herbert Wolverson
78e26d1b43 Merge branch 'v1.4-pre-alpha-rust-integration' 2023-01-13 13:57:47 +00:00
Robert Chacón
87e5ca90f9
Update TESTING-1.4.md 2023-01-12 19:45:00 -07:00
Herbert Wolverson
d9ae80e56f #184 - Experimental code for unpacking MPLS labels, finding the bottom-of-stack and extracting IPv4 or IPv6 payload. Does not supported VLANs nested inside VPLS. Needs considerably more testing. 2023-01-12 18:48:53 +00:00
Herbert Wolverson
f79f850079 Oops, turning off redaction didn't work properly. 2023-01-12 17:23:38 +00:00
Herbert Wolverson
f5cfd48dd1 Oops, turning off redaction didn't work properly. 2023-01-12 17:22:10 +00:00
Herbert Wolverson
187e0dd275 Add bounds checking to quantile calculation 2023-01-12 16:56:06 +00:00
Herbert Wolverson
115a3fed7b Add bounds checking to quantile calculation 2023-01-12 16:54:10 +00:00
Herbert Wolverson
2906c1b8f0 Ignore 0 band in quantile display. 2023-01-12 16:50:39 +00:00
Herbert Wolverson
858ed06286 Update redaction, it's now in Klingon. 2023-01-12 16:41:52 +00:00
Herbert Wolverson
44171d15ea Support redaction mode. 2023-01-12 16:13:10 +00:00
Herbert Wolverson
8fade0b1b1 Add a CSS file and move a lot of inline styles out of the body. 2023-01-12 14:57:23 +00:00
Robert Chacón
75fc2f22f7
Merge pull request #188 from cclauss/patch-1
GitHub Action to lint Python code
2023-01-12 03:51:38 -07:00
Herbert Wolverson
80b79684da #148 - Show quantile utilization on queue details. 2023-01-11 20:44:44 +00:00
Herbert Wolverson
7118ad948f #148 - Show quantile utilization on dashboard for total flow analysis. 2023-01-11 20:07:35 +00:00
Herbert Wolverson
0715f76022 Remove redundant reload link 2023-01-11 18:22:56 +00:00
Herbert Wolverson
7b0715756f Real-time reloading of lqosd tunables and queue check frequency
Extends the bus to include a "reload lqosd" command that reloads
the tunables and queue frequency in real-time, without requiring
a bridge restart.
2023-01-11 18:16:30 +00:00
Herbert Wolverson
7b18ca4196 Add libbpf-dev to the dependency list. 2023-01-11 16:00:24 +00:00
Herbert Wolverson
8d9eee8f47 TESTING: Add a CI build/test step to the Rust project for Github. This will probably fail and require more commits to get working. 2023-01-11 15:53:34 +00:00
Herbert Wolverson
2c829fc09c Add documentation to lqos_config 2023-01-11 15:42:10 +00:00
Herbert Wolverson
d51c1b8622 Add more unit tests to lqos_bus, checking that encode/decode of request and replies is valid. 2023-01-11 15:12:38 +00:00
Herbert Wolverson
5cf20a9afe Clean up and document the lqos_bus crate. 2023-01-11 15:04:26 +00:00
Herbert Wolverson
02c3a835a1 Cargo fmt run - format all code. 2023-01-11 14:12:40 +00:00
Herbert Wolverson
778de71402 Comment out queue diff types we aren't currently using. 2023-01-11 14:12:16 +00:00
Herbert Wolverson
393a489bca Allow logged-in users to update ispConfig.py via web UI
1) lqos_config understands writing to a new ispConfig.py file, copying the
   previous one and amending to avoid changing any not-covered data.
2) The web UI can adjust the config, upload a new one and it is saved.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 16:12:09 +00:00
Christian Clauss
cfb42895d0
Use requirements.txt 2023-01-10 16:49:27 +01:00
Robert Chacón
86c26bb9d4
Merge pull request #189 from cclauss/patch-2
python3 -m pip install -r requirements.txt
2023-01-10 08:46:20 -07:00
Herbert Wolverson
bebcbaf7c6 Merge branch 'v1.4-pre-alpha-rust-integration' of https://github.com/LibreQoE/LibreQoS into v1.4-pre-alpha-rust-integration 2023-01-10 14:16:34 +00:00
Herbert Wolverson
76077a3de0 Use checked subtraction to avoid overflow when counting hosts. 2023-01-10 14:12:51 +00:00
Herbert Wolverson
add187fa6f Fix warning 2023-01-10 14:12:51 +00:00
Herbert Wolverson
165dae030b Adds an authentication system.
* The new Rust utility "webusers" manages /opt/libreqos/webusers.toml.
* You can add/update/remove/list users from that tool.
* The "allow anonymous" option in webusers.toml permits access for
  unauthenticated users, but won't let them change anything. This is
  for payne demonstrations.
* All web APIs and pages should now be secured, requiring a login.
* The login requires cookies.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:51 +00:00
Herbert Wolverson
8060a50f0d Update test instructions to include how to update. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
a95f6d45de Update TESTING-1.4 instructions 2023-01-10 14:12:50 +00:00
Herbert Wolverson
939302af63 Throughput graph is responsive, doesn't fill to zero on unshaped, and has an X-axis label. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
66b5e822af #1 TCP Rtt graph is responsive, has a colour gradient, and an X axis label 2023-01-10 14:12:50 +00:00
Herbert Wolverson
702b8034df CPU graph is responsive, and scales colors based on load. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
bb3358ebcc #183 Flexible layout, no legend and more padding for RAM graph. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
5d92eb7a61 Update lqos.example to more sane defaults, and add a TESTING-1.4.md document to provide some temporary instructions for testing this release. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
0cddf107de Easier to read queue data, now showing graphs for throughput, delays, queue length, etc. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
56f421daed TC Queue Check Frequency and Display
* The frequency with which TC queues are polled for statistics is
  now controled by `queue_check_period_ms` in `/etc/lqos`.
* Example file updated.
* Configuration display updated.

This is designed to act as a compromise: for detailed analysis, it
can be updated to a low number. For regular running, it can be updated
to 1000 or more.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
c5e8da8159 Oops - forgot to multiply bytes into bits for consistency. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
0422384b21 Show a few more data items per queue 2023-01-10 14:12:50 +00:00
Herbert Wolverson
dbfee1f079 Less horrible graphing code, ringbuffer properly scrolling.
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00