Commit Graph

1009 Commits

Author SHA1 Message Date
Dave Taht
79d13718f4 Added Run Bandwidth Test to the Shaped.html page 2023-01-21 21:05:29 +00:00
Dave Taht
40758ffe6b lq_utils: Move packet scaling routines to library
This moves the packet scaling routines for g/m/k gps and
g/m/k bytes to a shared library.
2023-01-21 20:41:37 +00:00
Dave Taht
b2cf27ea9a lqtop: Reorder summary to use same order of result
The default total summar was the opposite of the table below,
and confusing.

Ideally the layout of the topmost bar should be:

LibreQos 'NOT CONNECTED' 'OTHER STATUS' 'down bps' 'up bps' 'pps' 'pps'

But my table-fu failed me.
2023-01-21 20:06:07 +00:00
Dave Taht
20727d44ca lqtop: Consistently right justify most output
By using the handy fmt macros this is feasible if the cell width
is known, via "{:>11}" for example.

However my intuitive thought that this was a "Constraint",
that you could apply to a Cell or Span, and it isn't. There are
multiple calls on the github for this. Until such a day, fmt
goes to 11.
2023-01-21 19:30:27 +00:00
Dave Taht
ee055cc322 H for histograms
Outlining features faster than they can be implemented...

... or considered if they are a good idea or not. More letters
to come!
2023-01-21 18:04:38 +00:00
Herbert Wolverson
37355189b9 Work in progress: transmute netlink cake stats to readable data. Currently uses bytemuck, should be using zerocopy. 2023-01-21 03:31:48 +00:00
Herbert Wolverson
25742c25f6 Remember to add the files... 2023-01-20 21:55:01 +00:00
Herbert Wolverson
14e49ce183 Super incomplete hacking away at netlink access for queues. 2023-01-20 21:52:56 +00:00
Herbert Wolverson
fc8b3ca9e4 Allow paging on TOP-N queries
Change BusRequest::GetTopNDownloaders and GetWorstRtt to accept
a start and end field, rather than just "n_rows".
2023-01-20 21:18:19 +00:00
Herbert Wolverson
f742acafda Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-01-20 19:29:04 +00:00
Herbert Wolverson
922ddd602f A bit of a hybrid commit, sorry. Provides various default pre-allocations that seem reasonable, and fixup on bus permissions. 2023-01-20 19:28:54 +00:00
Dave Taht
3c6f438e06 queue_tracker: use info_once on new fields
Modern linuxes add fields all the time. On a kernel upgrade we
shouldn't crash just because there's a new json field.

Also there are some further optimizations to represent the kernel
structures themselves. Some fields can overflow which would lead
to some surprizing behaviors in polling the json over time.

FIXME - always check for overflow elsewhere, in packets, overlimits
etc. Could that be a trait?
2023-01-20 18:37:23 +00:00
Herbert Wolverson
287ddb6e10 Global allocator and compilation flags update
On systems that support it, `jemallocator` will replace the default
allocator for `lqosd` and `lqos_node_manager`. The Jem allocator
is a LOT more friendly to Rust's preferred vector allocation patterns,
and actually cares about your cache.

Enable "fat" Link-Time Optimization. This slows down compilation a lot,
but results in smaller binaries and permits the LLVM optimizer to
cross crate boundaries when inlining, optimizing tail calls and a
few other optimizations.
2023-01-20 18:06:41 +00:00
Dave Taht
1742232eaf Made tc_fq_codel actually compile
Running cargo build in that dir did not detect the compile issue
cargo build --all from the root dir, did.
2023-01-20 17:54:03 +00:00
Dave Taht
a8cb399630 Shrink fq_codel structures to match the kernel API
FIXME: We need, in general, to check for wrapping in long term runs

* FIXME: Need a test to ensure fq_codel is still parsing
* Still want a size before and after test.

Lastly...

Newer versions of the kernel now have newer options for fq_codel
such as ce_threshold. The present implementation will spam the log
on encountering a newer kernel and tc.
2023-01-20 17:44:10 +00:00
Herbert Wolverson
ff5ad10240 LqTop can use full screen size for data
Added a note for where `n_rows` gains the current terminal size.
`tui` turns out to consume the `Event::Resize` event, so it's
never received - instead, we have to calculate it from the
creation of the UI "chunks".

The rendering length turned out to be an artefact of forgetting
to remove a height constraint from a UI chunk that was tested
and never used. (It briefly contained some graph data).
2023-01-20 17:21:55 +00:00
Herbert Wolverson
d83b8d23e8 Missing documentation tags 2023-01-20 17:17:10 +00:00
Herbert Wolverson
e2f09d14fa Persistent BusClient offers "is_connected"
* At any time, you can ask a BusClient if it is connected.
* If `lqtop` loses connectivity while running, it displays NOT
  CONNECTED in red on the title bar.
* If `lqtop` can't reach the daemon on start, it bails out with
  an error message.
2023-01-20 17:05:58 +00:00
Herbert Wolverson
2deb98a4c1 Fix permissions on creating lock file 2023-01-20 16:46:57 +00:00
Herbert Wolverson
ed07a4666b Add file locking to lqosd
The file locking is "smart": it checks to see if a lock is
valid before refusing to run (and updates the lock if
it can run anyway).

The locking mechanism will fail if you manually create
the lock file and dump random data into it that doesn't
readily convert to an i32.

Affects issue #54 and issue #52

* Add a new structure `FileLock` to `lqosd`.
* FileLock first checks /run/lqos/lqosd.lock. If it exists,
  it opens it and attempts to read a PID from it. If that PID
  is running and the associated name includes "lqosd", the
  FileLock returns an error.
* If no lock exists, then a file is created in
  /run/lqos/lqosd.lock containing the running PID.
* Includes Drop and signal termination support.
2023-01-20 16:05:55 +00:00
Herbert Wolverson
357bec9ad2 Forgot to remove a debug line 2023-01-20 16:04:50 +00:00
Herbert Wolverson
dbe0b3bdfd Provide explicit cleanup of the lqos_bus socket on signals.
Since guarantying the execution of Drop traits on process
termination from signals is now "unsound" in the Rust specs,
provide an explicit clean-up path for `lqosd`, called when
a termination signal is processed. This removes the Unix
stream socket on termination.
2023-01-20 15:31:21 +00:00
Herbert Wolverson
e64e41c96b Bus Client is more resilient to socket failure
* BusClient will survive creation without a successful connection.
* Attempting to use the BusClient without a connection will try to
  connect.
* Attempts to use the BusClient are gated by timeouts and error
  checking. Failures result in returning the client stream to
  "None" - triggering a reconnect attempt on the next use.
* Client programs have added the Tokio "time" feature, needed for
  timeout support.
* lqtop and lqos_node_manager have been tested to run without
  lqosd started, and with stop/start cycles triggering resumption.
2023-01-20 15:20:21 +00:00
Herbert Wolverson
adb8c1f471 Merge branch 'main' of https://github.com/LibreQoE/LibreQoS 2023-01-20 14:30:17 +00:00
Herbert Wolverson
fba591595b Move the socket from /tmp/lqos_bus to /run/lqos/bus and change permissions to match. 2023-01-20 14:27:37 +00:00
Dave Taht
fc07f502e0 Odd Events Operations Research and Logging
My largest desire in the data collection work so far is to be able
to compare "Saturday to Saturday, 8PM". There are always details
about given events that show up in the data, and the data gives
you details you might not have had at the time.

That "Replay" facility I started to describe is a means of doing that
from the historical data, although I kind of intended it to go up
against the bus. Also sampling rapidly some subset of devices all the
time, to compare against the longer period data, seems potentially useful.

Secondly, what is happening that we are *not* observing? When did it
start or end?

* Patch Tuesday
* DDOS attack
* Routing loop or broadcast storm
* Weather - a tower falls, how do you get in touch with the customer?
* New Counter Strike release (roughly the same as a DDOS)

So I added a filter for "Odd" - which is kind of fungible, and
"Log" to pull up a logbook from "somewhere" around that event.
An Odd filter could include "Silence" from week to week - why is this
device not online?
2023-01-20 14:12:58 +00:00
Dave Taht
2a3d71c5aa lqtop: Filter on my network or unshaped 2023-01-20 06:39:07 +00:00
Dave Taht
b89fd095e4 lqtop: more thinking 2023-01-20 05:33:32 +00:00
Dave Taht
e81d862e1e lqtop: still simulating 2023-01-20 05:14:03 +00:00
Dave Taht
381dcd1887 more ideas for lqtop 2023-01-20 03:55:06 +00:00
Dave Taht
45202ff3cf Rustfmt pass with the new defaults
Looks mildly better
2023-01-20 02:46:24 +00:00
Dave Taht
4f72d3ed69 Merge branch 'main' of LibreQoE/LibreQoS
I am not sure if we should default to rebasing, or this.
2023-01-20 01:02:57 +00:00
Dave Taht
01e6f10f22 DRAFT: lqtop keyboard input API
This is a draft attempt at working out what keystrokes and
functionality lqtop could gain for the beleaguered BOFH.

Presently functionality is stuck on:

* Capturing a screen resize event doesn't work
* Needing to be able to send a starting row to the bus
* More bus messages in general
* BOFH ignorance has to how to message the other thread
2023-01-20 00:58:39 +00:00
Herbert Wolverson
e857f20ea1 Roll back on the persistent bus client until reconnect is figured out 2023-01-19 22:42:23 +00:00
Herbert Wolverson
0e03d3bbba CSV reading of ShapedDevices correctly ignores lines starting with # 2023-01-19 20:27:47 +00:00
Herbert Wolverson
12e0f6e0bd Show correct plans on shaped.html 2023-01-19 20:17:25 +00:00
Herbert Wolverson
7a0f066c63 Display errors when loading ShapedDevices 2023-01-19 20:16:20 +00:00
Herbert Wolverson
3e12e95f66 Create lqos_utils as a repository for useful macros. Move the string table macro. Start working on some command wrappers for easier use. 2023-01-19 17:52:19 +00:00
Herbert Wolverson
c676b8a33f Add a standard format file, and use #[rustfmt::skip] to say 'stop formatting this block' 2023-01-19 16:40:36 +00:00
Herbert Wolverson
9c135e8b4f Preallocate ring buffer result size 2023-01-19 15:34:53 +00:00
Herbert Wolverson
d0dd559f1d config_control.rs: Eliminate vector reallocation from get_nic_list
Refactor the get_nic_list function to use iterators with a collect
at the end, ensuring that the result vector is allocated at exactly
the needed size.
2023-01-19 15:15:42 +00:00
Herbert Wolverson
7cb8f930d1 De-async two things that shouldn't have been async to begin with. 2023-01-19 14:59:12 +00:00
Herbert Wolverson
29a9985a5a Remove file that isn't needed anymore. 2023-01-19 14:51:01 +00:00
Dave Taht
d58115fd3d Create a interfaces to test against
Dummy interfaces are commonly used to test stuff like this.
Most Linuxes do support them. A more advanced approach

Linux interfaces have a 15 character limit.

This implementation presently does not remove the created interface
and is a sterling example of how "side effects" in the OS itself
often need to be accounted for.
2023-01-19 14:44:50 +00:00
Herbert Wolverson
4d569e698c Add doc tags and general cleanup 2023-01-19 14:36:09 +00:00
Herbert Wolverson
a5fc96abc9 The lqos_node_manager cache manager uses a persistent BusClient 2023-01-18 23:39:12 +00:00
Herbert Wolverson
2633974839 Missed file from previous commit 2023-01-18 23:29:58 +00:00
Herbert Wolverson
3cb2799beb Still a work in progress, needs work - but adds a persistent client option for the bus. It currently won't work well without a pause between request/replies - but it works well enough for lqtop. More improvements needed. 2023-01-18 23:15:09 +00:00
Herbert Wolverson
da6a8f08a2 Move the socketserver into the bus crate also, makes more sense there. 2023-01-18 22:28:15 +00:00
Herbert Wolverson
2b573708ac Remove auth cookie from bus. Benchmarks show a tiny improvement in overall performance. 2023-01-18 22:18:16 +00:00