Commit Graph

787 Commits

Author SHA1 Message Date
Herbert Wolverson
8060a50f0d Update test instructions to include how to update. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
a95f6d45de Update TESTING-1.4 instructions 2023-01-10 14:12:50 +00:00
Herbert Wolverson
939302af63 Throughput graph is responsive, doesn't fill to zero on unshaped, and has an X-axis label. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
66b5e822af #1 TCP Rtt graph is responsive, has a colour gradient, and an X axis label 2023-01-10 14:12:50 +00:00
Herbert Wolverson
702b8034df CPU graph is responsive, and scales colors based on load. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
bb3358ebcc #183 Flexible layout, no legend and more padding for RAM graph. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
5d92eb7a61 Update lqos.example to more sane defaults, and add a TESTING-1.4.md document to provide some temporary instructions for testing this release. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
0cddf107de Easier to read queue data, now showing graphs for throughput, delays, queue length, etc. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
56f421daed TC Queue Check Frequency and Display
* The frequency with which TC queues are polled for statistics is
  now controled by `queue_check_period_ms` in `/etc/lqos`.
* Example file updated.
* Configuration display updated.

This is designed to act as a compromise: for detailed analysis, it
can be updated to a low number. For regular running, it can be updated
to 1000 or more.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
c5e8da8159 Oops - forgot to multiply bytes into bits for consistency. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
0422384b21 Show a few more data items per queue 2023-01-10 14:12:50 +00:00
Herbert Wolverson
dbfee1f079 Less horrible graphing code, ringbuffer properly scrolling.
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
e3c7245838 Forgot to invert upload
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
d8bffa2137 Initial work on displaying queue data. Very early days.
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
fbd3c22566 Missing td tags 2023-01-10 14:12:50 +00:00
Herbert Wolverson
2d9937518a Read-only configuration of non-integration portions is working locally, sending upstream for testing. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
8afdc2e0d1 WIP - Basic interface and Bifrost config displayable. 2023-01-10 14:12:50 +00:00
Herbert Wolverson
64ae074fbe Oops - missed an active navlink tag
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
97ff2ad288 Work in progress: visual prototype for most of the configuration
system.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
42c77f3432 Add example systemctl .service files
Add example .service files for lqosd and lqos_node_manager.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
b01b9520a5 First iteration of a Python module for direct bus interaction
* Adds the `lqos_python` module to the Rust build system.
* `lqos_python` exposes an API for calling bus commands
  directly, instead of via a helper. Currently, "is alive"
  for testing the daemon, and IP mappings are exported.
* Update the build script to build the .so file and put
  it where Python can find it.
* Exclude the binary from Git.
* Update `LibreQoS.py` to use bus commands for clearing,
  deleting IP mappings. Adding is not yet covered, due
  to the way commands are batched - and that not yet being
  supported by the library.
* Add a check to `LibreQoS.py` that the `lqosd` daemon
  is running and accepting connections. Abort if it isn't,
  since nothing else will work anyway.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
bd5ace9fbf Couple more .gitignore entries to avoid including
Linux TC status and lastRun.txt in the tree.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
f0a9f045fa Update build_rust.sh to include node manager
Update the Rust build script to include the node manager,
and its required files.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
deded47cac Add build script and associated path changes
* Adds build_rust.sh:
   * Creates `src/bin` and `src/bin/static` if needed.
   * Compiles all of the executables in the `rust` tree.
   * Copies the executables into `src/bin`
   * Copies the static web data for `lqos_node_manager`
     into `bin/static`.
* Updates .gitignore to ignore the bin/ folder
* Updates LibreQoS.py to call bin/xdp_iphash_to_cpu_cmdline
  instead of ./xdp_iphash_to_cpu_cmdline (cleaner).
* Updates graphInfluxDB.py to call the temporary Rust shim
  that provides output like the previous one, hopefully
  (poorly tested) allowing the existing graph system to work
  with the new structure.

See #181

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
10a674abde Remove cpumap-pping submodule
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
e780b1411d Update .gitmodules for removing the cpumap submodule
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
32597813e8 Add Rust source tree to the main repo.
Replaces the previously separate repo, merging the Rust
system into the mainline ready for 1.4.

The Rust system currently provides:

* lqos_bus: type definitions for localhost control bus.
* lqos_config: handler for configuration files.
* lqos_node_manager: local web-based monitor and manager.
* lqos_sys: eBPF program that handles all of the tracking,
  CPU assignment, VLAN redirect, TC queue assignment,
  and RTT tracking. Wrapped in a Rust core, providing
  consistent in-Rust building and mostly-safe wrappers
  to C code.
* lqosd: a daemon designed to run continually while
  LibreQoS is operating. On start, it parses the configuration
  files and sets up interface mapping (the Python code is still
  required to actually build queues). It then assigns the various
  eBPF programs to appropriate interfaces. The eBPF systems are
  removed when the daemon terminates.

  lqosd also provides a "bus", listening to requests for changes
  or information on localhost, providing a control plane for
  the rest of the project.
* lqtop: An example program demonstrating how to use the bus,
  acts like "top", showing current network traffic and mappings.
* xdp_iphash_to_cpu_cmdline: a Rust wrapper providing the same
  services as the cpumap originated tool of the same name. This is
  a "shim" - it will go away once the native Python library is
  ready.
* xdp_pping: also a shim, providing equivalent services to the
  cpumap service of the same name.

A helper shell script "remove_pinned_maps.sh" can be used to
remove all pinned eBPF maps from the system, allowing for eBPF
program upgrades that change persistent map structures without
a reboot.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
b0ff63d278 Update .gitignore, fix syntax in LibreQoS.py
* Add src/rust/target to the .gitignore file,
  to avoid including Rust build artifacts in the
  Git tree.
* Fix the syntax when calling the IP mapping helper
  and deleting IP mappings.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:50 +00:00
Herbert Wolverson
485ce30b45 Patch LibreQoS.py and ispConfig.example.py for Rust Integration
* Modifies ispConfig.example.py to include work-in-progress
  "on a stick" configuration. You can specify 2 VLANs on a
  single interface and operate on a single interface.
* LibreQoS.py no longer calls cpumap commands, but instead
  calls the Rust shims that do the same thing with the new
  Rust environment (which is in the next commit).
* LibreQoS.py detects when running "on a stick", and divides
  the number of available queues by 2. The first half are
  used for download queues, the second for upload queues.
  This allows a single TC hierarchy to be installed on the
  single "stick" interface, and still divide traffic by core.

Part of #181

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-10 14:12:44 +00:00
Robert Chacón
b2bc5c12d6
Merge pull request #186 from cclauss/undefined-names
Fix undefined names in Python code
2023-01-09 17:41:36 -07:00
Dave Täht
cbac60cc26
Merge pull request #185 from cclauss/codespell
Fix typos discovered by codespell
2023-01-09 16:02:52 -08:00
Christian Clauss
c4ac2e45c9 Fix undefined names in Python code 2023-01-08 09:28:15 +01:00
Christian Clauss
3be73d3da2 Fix typos discovered by codespell 2023-01-08 09:09:37 +01:00
Dave Täht
2dc4e21e52
Merge pull request #182 from cclauss/patch-1
Fix typos - thank you!
2023-01-07 12:01:11 -08:00
Christian Clauss
b44afb7654
Fix typos 2023-01-07 08:50:28 +01:00
Robert Chacón
7165b1ff6b
Fix partial relaod bug 2022-12-17 09:29:58 -07:00
Robert Chacón
0e21123b05
Merge pull request #178 from LibreQoE/issue_177_uisp_duplicate_ips
Issue 177 uisp duplicate ips
2022-12-17 07:21:22 -07:00
Herbert Wolverson
85becbd5c6 Do not permit duplicate IP addresses for device
FIXES #177

Apply the change suggested in the issue body: UISP can report
devices with duplicate IP addresses across interfaces. Check
that an IP address is unique before applying it.

Changed for both Flat and Hierarchy networks.
2022-12-17 08:16:35 -06:00
Herbert Wolverson
b9b344464c Change path to network.example.json.
When the repo was rearranged, it broke a unit test that expected
to find network.json.example in the "1.3" folder. It now looks
in the src/ folder, and the test passes.
2022-12-17 08:09:50 -06:00
Robert Chacón
fc00a25c16
Update README.md 2022-12-11 10:58:07 -07:00
Dave Täht
fef70b7029 More test tools 2022-12-11 09:05:09 -08:00
Dave Täht
9c27aae7a6 TCP cong controls 2022-12-11 08:14:18 -08:00
Dave Täht
b0e3c5e289 Some more plans 2022-12-11 08:09:00 -08:00
Dave Täht
113a3f7c96 More gitignore fixes 2022-12-10 21:37:52 -08:00
Dave Täht
88d191604d Jitter plot 2022-12-10 21:37:23 -08:00
Dave Täht
b1b6605b37 More notes about RTT monitoring 2022-12-10 21:19:55 -08:00
Dave Täht
a6ab2a7ecf Added RTT example 2022-12-10 21:04:25 -08:00
Dave Täht
e4a86107e8 Simulation Notes
A rather too-large overview of what the world might look like in a
testbed.
2022-12-10 20:43:44 -08:00
Dave Täht
a1d779f2d4 Clean up .gitignore 2022-12-10 20:43:29 -08:00
Dave Täht
9e984b1216 Major move around of every thing
Clean up old versions
2022-12-07 11:04:54 -08:00