Commit Graph

892 Commits

Author SHA1 Message Date
Herbert Wolverson
c6661520dd Support dashes in generated enums by replacing - with _ during
the match phase. Enum options have to replace minus with underscore.

E.g. `ack-filter` becomes `ack_filter`.

So you can do:

`string_table_enum!(DashingEnum, option_1, option2);`

And read `DashingEnum::from_str("option-1")`
2023-01-17 21:42:33 +00:00
Herbert Wolverson
d83c1dc852 Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-01-17 21:15:00 +00:00
Herbert Wolverson
e26a9982ab Add a string table macro to make it easier to pass between JSON
exported text and known-types in queue definitions.
2023-01-17 21:13:08 +00:00
Herbert Wolverson
f9351757ca Remove per-tick printing
Cap number of watched queues at (CPU cores * 2), per belyivulk
2023-01-17 20:06:33 +00:00
Herbert Wolverson
7ea5ab3cd8 Convert diffserv from a string to a nice enum.
Co-authored-by: Dave Taht <davet@taht.net>
2023-01-17 20:00:10 +00:00
Herbert Wolverson
87aaddd82b Cargo format run 2023-01-17 19:54:11 +00:00
Herbert Wolverson
6269f41b67 Use Rayon to parallelize the queue reader 2023-01-17 19:53:59 +00:00
Herbert Wolverson
6f98e0f088 Lazy queue tracking
Instead of polling every TC queue, maintain a "watch list" (with
auto expiration). Opening a queue in the GUI adds that queue to
the "watch list".

Only queues in the watch list are polled.
2023-01-17 19:44:53 +00:00
Dave Taht
7be6a3051c Note some possible string to byte conversions
I wish I understood how to write a simple lookup table in rust...
2023-01-17 09:23:44 -08:00
Herbert Wolverson
5855a3b3ed Move tc queue monitoring into a separate crate
* Remove the `lqosd` lib/main hack.
* Move benchmarks from lqosd into the new `lqos_queue_tracker` crate.
* Move queue tracking into a separate crate, to make it easier to
  work with.
2023-01-17 15:51:11 +00:00
Herbert Wolverson
424925d0d6 Improve the /etc/lqosd example 2023-01-17 14:09:14 +00:00
Herbert Wolverson
1a0b20aa11 Add Criterion-based benchmarks to lqosd
* Add a secondary build option (as library) to lqosd, by including
  a `lib.rs` file that mirrors `main.rs`.
* Add the boilerplate required for a Criterion-based benchmark
  setup.
* Add support for `cargo bench` launching a set of tests that
  benchmark deserializing TC queues, and benchmarks retrieving
  queue statistics from `tc`.
2023-01-17 14:05:28 +00:00
Herbert Wolverson
6403ad21e4 Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-01-16 16:04:05 +00:00
Herbert Wolverson
7548413165 Use Log and Env_log for output. Suppress kernel messages that confuse
people on startup.

Default to "warn" level of logging. You can override by setting
RUST_LOG to "info". (Environment variable)
2023-01-16 16:03:14 +00:00
Robert Chacón
ba44bd3ca5
Update lint_python.yml
Test skipping the part of lint_python.yml that keeps breaking
2023-01-16 08:38:05 -07:00
Herbert Wolverson
6f49b2498a Implement correct signal handling for SIGINT, SIGTERM and SIGHUP
Extend the signals handler to reload configuration on SIGHUP, and terminate
on SIGINT and SIGTERM.

FIXES #202
2023-01-16 15:07:20 +00:00
Herbert Wolverson
05bb1ee959 Fix failure to display Cake Tins in web UI
Cake Diff functions now use "checked_sub" to ensure that an overflow
doesn't mess up data.

Web UI includes checks that Cake data is available on each history
tick.
2023-01-16 14:30:48 +00:00
Robert Chacón
76513466fa
Merge pull request #201 from LibreQoE/change1244
Disable refreshLatencyGraphs in schedulerAdvanced.py
2023-01-16 01:18:26 -07:00
Robert Chacón
1238123143
Rename influxDBdashboardTemplate_v2.5.1.json to influxDBdashboardTemplate.json 2023-01-16 01:10:47 -07:00
Robert Chacón
56f842371d
Delete influxDBdashboardTemplate.json 2023-01-16 01:10:31 -07:00
Robert Chacón
6b4432e899
Update README.md 2023-01-16 01:08:03 -07:00
Robert Chacón
246056e055
Update schedulerAdvanced.py 2023-01-16 01:05:32 -07:00
Robert Chacón
6a3fe4957a
Merge pull request #200 from LibreQoE/v1.4-pre-alpha-rust-integration
reforked schedulerAdvanced, plus a few other bits
2023-01-16 01:02:11 -07:00
Dave Täht
a1bb729a78
Merge pull request #199 from khandieyea/v1.4-schadvanced
reforked schedulerAdvanced, plus a few other bits
2023-01-15 21:56:26 -08:00
Robert Chacón
5cb5e8e7aa
Compress json depth to 8 (HTB max) 2023-01-15 22:03:38 -07:00
Warren
e4ae7c8cf3 Default http api client back to false 2023-01-16 16:49:17 +13:00
Warren
b7b78962a3 reforked schedulerAdvanced, plus a few other bits 2023-01-16 16:31:18 +13:00
Robert Chacón
9afc07cec2
Update TESTING-1.4.md 2023-01-15 14:56:38 -07:00
Robert Chacón
d503150b2b
Merge pull request #197 from khandieyea/v1.4-manfix
fix typo in apt-get
2023-01-15 14:51:35 -07:00
Warren
4375211432 fix typo 2023-01-16 09:52:10 +13:00
Dave Taht
5be5111954 Use the cake types that are defined in the cake qdisc
While json sends everything around as a u64, many of the cake
variables internally and in the protocol, are 16-32 bit. It
seemed a major memory savings to have the types more closely
match the underlying reality, and one small step towards
actually getting these values from the kernel more directly.

In terms of surprise several of those can actually roll over,
which is something we need to look out for in the future.
2023-01-15 14:38:47 +00:00
Herbert Wolverson
b620826476 Replace BpfMap and BpfPerCpuMap's dump_vec function with a
callback-based setup that elides the need to clone data on the
hot path.

Clean up the throughput data collection into clearly defined
stages, using Rayon where appropriate to speed up iterator-based
tasks.

This removes: 2 memcpy and 1 memmove per circuit, per second.
2023-01-14 17:47:36 +00:00
Herbert Wolverson
66356d66d2 Don't unwrap() - use a result for proper error handling and not terminating the queue tracking when monitor mode or something unexpected happens. 2023-01-13 19:43:35 +00:00
Herbert Wolverson
c2ae327c48 Rename "use_kernel_bridge" in "/etc/lqos" to "use_xdp_bridge"
The previous wording was confusing, making people think that the
configuration entry was referring to the Linux bridge - while it
meant "use the XDP bridge that happens to run in kernel space".

The new wording should be more obvious.
2023-01-13 18:18:46 +00:00
Herbert Wolverson
fecb27c6bc Only show unknown IPs from the last 5 minutes.
IMPORTANT: run "remove_pinned_maps.sh" before you run the new
version of lqosd. The eBPF map structure has changed.

1) Add a "last_seen" value to the map_traffic eBPF map.
2) Whenever traffic is seen, update "last_seen" to equal the
   result of a bpf_ktime_get_boot_ns() call.
3) When retrieving the unknown IP list, perform a syscall to
   obtain the time since boot in nanoseconds and subtract
   five minutes.
4) Filter out any unknown IPs that exceed the 5 minute window.

This should solve the problem with "unknown IPs" filling up
on start, and gradually accumulating.
2023-01-13 17:48:20 +00:00
Herbert Wolverson
25234cc28a Add suggestion to run remove_pinned_maps to the Rust builder. 2023-01-13 16:40:19 +00:00
Herbert Wolverson
f3a5739133 Add 'download unknown' link to the unknown IPs page. 2023-01-13 14:16:48 +00:00
Herbert Wolverson
d7034d1e42 Update testing file to indicate that the branch is no longer needed. 2023-01-13 13:59:50 +00:00
Herbert Wolverson
78e26d1b43 Merge branch 'v1.4-pre-alpha-rust-integration' 2023-01-13 13:57:47 +00:00
Robert Chacón
87e5ca90f9
Update TESTING-1.4.md 2023-01-12 19:45:00 -07:00
Herbert Wolverson
d9ae80e56f #184 - Experimental code for unpacking MPLS labels, finding the bottom-of-stack and extracting IPv4 or IPv6 payload. Does not supported VLANs nested inside VPLS. Needs considerably more testing. 2023-01-12 18:48:53 +00:00
Herbert Wolverson
f79f850079 Oops, turning off redaction didn't work properly. 2023-01-12 17:23:38 +00:00
Herbert Wolverson
f5cfd48dd1 Oops, turning off redaction didn't work properly. 2023-01-12 17:22:10 +00:00
Herbert Wolverson
187e0dd275 Add bounds checking to quantile calculation 2023-01-12 16:56:06 +00:00
Herbert Wolverson
115a3fed7b Add bounds checking to quantile calculation 2023-01-12 16:54:10 +00:00
Herbert Wolverson
2906c1b8f0 Ignore 0 band in quantile display. 2023-01-12 16:50:39 +00:00
Herbert Wolverson
858ed06286 Update redaction, it's now in Klingon. 2023-01-12 16:41:52 +00:00
Herbert Wolverson
44171d15ea Support redaction mode. 2023-01-12 16:13:10 +00:00
Herbert Wolverson
8fade0b1b1 Add a CSS file and move a lot of inline styles out of the body. 2023-01-12 14:57:23 +00:00
Robert Chacón
75fc2f22f7
Merge pull request #188 from cclauss/patch-1
GitHub Action to lint Python code
2023-01-12 03:51:38 -07:00