Commit Graph

1337 Commits

Author SHA1 Message Date
Robert Chacón
d22ac8a73b
Improve documentation of useBinPackingToBalanceCPU 2023-02-04 03:28:34 -07:00
Robert Chacón
d1fda9290a
Fix cases where Nodes >= CPU Cores
Previously, generated parent nodes aka generatedPNs (which hold ShapedDevices without a defined Parent Node) would reside on the last few cores of the CPU. A problem became apparent here where if an operator had more Top Level Parent Nodes defined in network.json than CPU cores, there were no CPU cores left to use for generatedPNs. With this change, generatedPNs are created for each CPU core. Additionally, to reduce verbosity of the console output, the warning "uploadMax of Circuit ID exceeded that of its parent node." has been changed to an info log.
2023-02-04 03:15:58 -07:00
Herbert Wolverson
d688522c13 Hide UI elements for features that aren't ready yet. 2023-02-03 15:51:59 +00:00
Herbert Wolverson
89b2eafda7 Fix node manager not letting you run bandwidth tests from child displays. 2023-02-03 14:38:35 +00:00
Herbert Wolverson
b2be48da8a Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-02-03 14:32:13 +00:00
Herbert Wolverson
08cc2f6415 Fix node manager not letting you run bandwidth tests from child displays. 2023-02-03 14:31:58 +00:00
Robert Chacón
848986adcd
Merge pull request #242 from LibreQoE/fix2389r5
Fix TCP RTT Graphing For InfluxDB
2023-02-02 21:47:09 -07:00
Robert Chacón
b89f5254c6
Update graphInfluxDB.py 2023-02-02 21:46:24 -07:00
Robert Chacón
4acfb77ea8
Update scheduler.py 2023-02-02 21:45:45 -07:00
Dave Taht
f636cd9ff0 lqtop: make more room for large rtts
IPv6 is a max of 40 characters, so I cut the allowed space to 42.
That gave room to have multi-second long observable RTTs in the
RTT segment.

Compile tested only. A slightly better approach might be to
display time in ns/us/ms/s.
2023-02-02 20:40:45 +00:00
Herbert Wolverson
48b75a9b10 Fix spelling of 'Environment' 2023-02-02 18:03:31 +00:00
Herbert Wolverson
dc259fb5c8 Rework the file watcher to restart the watcher after firing.
Sometimes the watcher persists, sometimes it doesn't. This is
awful behavior. Work around by looping the watcher and returning
after it fires - restarting the watch process. This has the
added advantage of handling "file deleted" gracefully.

When ShapedDevices cannot be read, an empty set is loaded so
its obvious on the web UI that there is an issue.

In testing, I've been through a bunch of "break shaped devices",
"unbreak shaped devices" and seen the data come and go correctly
now.

ISSUE #239
2023-02-02 15:38:13 +00:00
Herbert Wolverson
7a18099587 Correct ordering of validation checks in LibreQoS.py
The Rust validator was running, and the very next line set the
result to success irregardless of the result. Move the validation
initialization to the top.

ISSUE #240
2023-02-02 14:29:21 +00:00
Herbert Wolverson
2926fa5c7a Add "trim(always)" to CSV decoder. Whitespace between comma
separated items will no longer cause validation issues - excess
whitespace is automatically trimmed at the beginning and end
of comma separated entries before parsing.

First part of ISSUE #240
(Fixing the validation issue rather than the actual cause)
2023-02-02 14:26:57 +00:00
Herbert Wolverson
27debaf0f1 Update to latest rustc syntax. Since we're now updating Rust,
take advantage of some of the newer Rust syntax formats
(mostly syntax sugar on format! macros).
2023-02-02 14:17:59 +00:00
Herbert Wolverson
715829cc9a Add 'rustup update' to the build chain. 2023-02-02 14:10:06 +00:00
Herbert Wolverson
30a6a2c5c5 Revert my truly awful decision to try and run LibreQoS.py during
lqosd startup (after checking that config files existed). It
made startup unstable.
2023-02-02 14:08:20 +00:00
Herbert Wolverson
911f107f3f Add a delay when starting the polling thread in lqos_node_manager
to let the server finish starting.

The delay is in the polling thread, not global - so it doesn't
cause a stall, or affect data access. The ringbuffer will be
slightly delayed in starting (and show zeroes until then).

Testing shows no more logged messages on reboot.

ISSUE #235
2023-02-01 22:29:55 +00:00
Herbert Wolverson
cb3fa889fb ISSUE #229 - Use checked add with 0 default when adding packet counters. 2023-02-01 20:00:32 +00:00
Herbert Wolverson
e2508a6e57 ISSUE #229 - More saturating_sub instead of just subtracting to avoid underflows. 2023-02-01 19:54:53 +00:00
Herbert Wolverson
309b399a56 ISSUE #229 - Add division by zero guard to the busy_quantile function. 2023-02-01 19:51:15 +00:00
Herbert Wolverson
5a1a3db5be ISSUE #229 - try to manually run cargo audit 2023-02-01 19:40:08 +00:00
Herbert Wolverson
d14c4239b9 ISSUE #229 - Remove github provided cargo audit, it doesn't support code in subdirectories! 2023-02-01 19:36:02 +00:00
Herbert Wolverson
ecc2fb5ed7 ISSUE #229 - YAML really wants to eat my brain tonight 2023-02-01 19:31:07 +00:00
Herbert Wolverson
568d55f317 ISSUE #229 - Add CVE database check to cargo lock scanning in audit phase. 2023-02-01 19:25:57 +00:00
Herbert Wolverson
5d709da7b0 Fix lqtop compilation with new crossterm. 2023-02-01 19:23:02 +00:00
Herbert Wolverson
38a2a78062 Finish Rust package update cycle per ISSUE #229 2023-02-01 19:13:23 +00:00
Herbert Wolverson
8864ebab70 Work around stupid type implementation in the latest sysinfo version. 2023-02-01 19:07:28 +00:00
Herbert Wolverson
317587fe00 Another try at the yaml. Ugh. 2023-02-01 18:58:27 +00:00
Herbert Wolverson
175b833d96 Fix build issue from new version of system, another try at CI. 2023-02-01 18:56:58 +00:00
Herbert Wolverson
a392112f7a Trying to get the outdated workflow system to work. YAML is not my friend. 2023-02-01 18:49:08 +00:00
Herbert Wolverson
3991aa404a Add "cargo outdated" checks to the GitHub CI workflow.
Part of ISSUE #229
2023-02-01 18:46:09 +00:00
Herbert Wolverson
d198c0feac Update Tokio and all dependency versions based on local run
of cargo audit and cargo outdated.

Part of ISSUE #229
2023-02-01 18:44:38 +00:00
Herbert Wolverson
7b1c285afa Adjust build_rust.sh to build to separate files and then move
into running files to avoid service interruption. The script
detects if you are using systemd (with the default names) and
will restart the services at the end of the process - for a
very brief interruption rather than several minutes.

Also suppresses pushd/popd output.

Related to ISSUE #208
2023-02-01 17:53:15 +00:00
Herbert Wolverson
9ad1de6ef5 Add safegaurd against running LibreQoS.py more than once at a time.
ISSUE #52

* Added file locking commands to the Python/Rust library.
* When LibreQoS.py starts, it checks that /var/run/libreqos.lock
  does not exist. If it does, it checks that it contains a PID
  and that PID is not still running with a python process.
* If the lock file exists and is valid, execution aborts.
* If the lock file exists and is invalid, it is cleaned.
* Cleans the lock on termination.
2023-02-01 17:09:42 +00:00
Herbert Wolverson
d71f41033c If ShapedDevices and ispConfig are in good shape, try to run LibreQoS.py when lqosd starts to avoid running with no queues. 2023-02-01 16:26:50 +00:00
Herbert Wolverson
3a9e72901b Correct issues with failing to collect data when started at
boot time. ISSUE #235 . Also relevant to ISSUE #209

* Discovered that the BOOT_TIME clock can fail if called
  immediately after boot.
* Refactored time fetching functions into `lqos_utils` with
  proper error wrapping.
* Adjusted unknown IP expiration to issue a bus response of
  "not ready yet" if the boot time clock is not available.
* Adjusted unknown IP expiration to handle 5-minutes in the
  past being a negative number.
* Adjusted queue collection to suggest that you run
  LibreQoS.py if queues don't exist - and fail gracefully,
  without causing a hitch.
2023-02-01 16:08:31 +00:00
Herbert Wolverson
d77fffe4f5 ISSUE #234
Remove the unused feature from notify. We actually moved notify
into a single crate (as opposed to all over the place) in
816ca7e651

As of this commit, build_rust runs without warnings.
2023-01-31 22:34:14 +00:00
Herbert Wolverson
dce8d76d9c Applied 'cargo fmt' to all files, forcing equal formatting for all. 2023-01-31 22:23:55 +00:00
Herbert Wolverson
51cf5d51fa ISSUE #209 General soundness check, assisted with clippy (linter).
* Corrected a lot of small issues like passing a string when a char
  will be (marginally) faster.
* Cleaned up single-arm match statements for the much more compiler
  friendly if let.
* Combined nested if statements.
* Cleaned all remaining unchecked unwrap() calls.
2023-01-31 22:22:05 +00:00
Herbert Wolverson
74101655d8 ISSUE #209 - Full error pass on lqos_queue_tracker module
* Replace every value unwrap with unwrap_or to not panic.
* Replace Anyhow errors with specific errors and log entries.
2023-01-31 21:31:03 +00:00
Herbert Wolverson
982e7314c1 Thorough error-handling pass on lqos_bus crate.
ISSUE #209

Replace "anyhow" with "thiserror". Add logging for all errors,
and only allow pass-through for errors that have already been
converted to a local error type and reported.
2023-01-31 19:33:22 +00:00
Herbert Wolverson
816ca7e651 Refactor the multiple "notify" systems into a single helper
structure.

* Creates FileWatcher, in lqos_utils.
* Removes "notify" dependency from other crates.

FileWatcher is designed to watch a file. If the file doesn't exist,
then an optional callback is called - and the watcher waits,
periodically checking to see if the file has appeared yet. When the
file appears, another optional callback is executed.

Once the file exists, a `notify` system is started (it uses Linux's
`inotify` system internally) for that file. When the file changes,
the process sleeps briefly and then executes an `on_change` callback.
Further messages are then suppressed for a short period to avoid
duplicates.

All uses of notify have been updated to use this system. Errors are
handled cleanly, per ISSUE #209.
2023-01-31 17:52:35 +00:00
Herbert Wolverson
3e4e7ebe64 Another chicken/egg issue:
* ShapedDevices.csv may not exist on first run.
* This previously caused lqos_node_manager to emit a hard error
  and not show any shaped devices at all.

To rectify this:

* ShapedDevices in-memory starts as an empty set.
* When the "watcher" spawns, if the file exists then ShapedDecices
  is loaded.
* If the "watcher" can't find ShapedDevices, it sleeps periodically
  looking for the file to be created. Once created, it loads it
  and starts the change monitor.
2023-01-31 16:26:54 +00:00
Herbert Wolverson
c911d8c190 Fix a chicken & egg problem with queueingStructure.json monitor
ISSUE #209

Using the inode watcher on a file that doesn't exist fails, and
was previously failing silently! This would result in queue
mappings not updating when LibreQoS.py was executed - even though
the queueingStructure.json file became available.

* Replace "anyhow" with specific errors.
* Track and log each step of the file monitor process for
  queueingStructure.json
* If the watcher cannot start because the file doesn't exist,
  the watcher loop sleeps for 30 seconds at a time (to keep
  load very low) and checks if the file exists yet. If it does,
  it loads it and then commences watching.
2023-01-31 15:49:29 +00:00
Herbert Wolverson
6f00386d61 Rename function to match what it does (copy/paste issue) 2023-01-31 15:22:23 +00:00
Herbert "TheBracket
cf33d62d85
Merge pull request #237 from interduo/patch-24
improvement: allow easily to turn on debugging mode
2023-01-31 07:16:39 -08:00
Interduo
5e02885c5b
improvement: allow easily to turn on debugging 2023-01-31 16:12:30 +01:00
Interduo
bd7f9a4a12
improvement: allow easily to turn on debugging mode 2023-01-31 16:11:32 +01:00
Herbert "TheBracket
bb9b911491
Merge pull request #232 from interduo/patch-22
bugfix: propper path and remove many sudo
2023-01-31 06:32:41 -08:00