Commit Graph

762 Commits

Author SHA1 Message Date
Herbert Wolverson
3e7da99f0f Add example systemctl .service files
Add example .service files for lqosd and lqos_node_manager.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 20:22:20 +00:00
Herbert Wolverson
495b40f840 First iteration of a Python module for direct bus interaction
* Adds the `lqos_python` module to the Rust build system.
* `lqos_python` exposes an API for calling bus commands
  directly, instead of via a helper. Currently, "is alive"
  for testing the daemon, and IP mappings are exported.
* Update the build script to build the .so file and put
  it where Python can find it.
* Exclude the binary from Git.
* Update `LibreQoS.py` to use bus commands for clearing,
  deleting IP mappings. Adding is not yet covered, due
  to the way commands are batched - and that not yet being
  supported by the library.
* Add a check to `LibreQoS.py` that the `lqosd` daemon
  is running and accepting connections. Abort if it isn't,
  since nothing else will work anyway.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 19:39:29 +00:00
Herbert Wolverson
54fd5bcfce Couple more .gitignore entries to avoid including
Linux TC status and lastRun.txt in the tree.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 17:49:07 +00:00
Herbert Wolverson
2027957df2 Update build_rust.sh to include node manager
Update the Rust build script to include the node manager,
and its required files.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 17:22:32 +00:00
Herbert Wolverson
4404890731 Add build script and associated path changes
* Adds build_rust.sh:
   * Creates `src/bin` and `src/bin/static` if needed.
   * Compiles all of the executables in the `rust` tree.
   * Copies the executables into `src/bin`
   * Copies the static web data for `lqos_node_manager`
     into `bin/static`.
* Updates .gitignore to ignore the bin/ folder
* Updates LibreQoS.py to call bin/xdp_iphash_to_cpu_cmdline
  instead of ./xdp_iphash_to_cpu_cmdline (cleaner).
* Updates graphInfluxDB.py to call the temporary Rust shim
  that provides output like the previous one, hopefully
  (poorly tested) allowing the existing graph system to work
  with the new structure.

See #181

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 15:35:21 +00:00
Herbert Wolverson
2e5f0b9370 Remove cpumap-pping submodule
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 15:17:29 +00:00
Herbert Wolverson
f9c603352a Update .gitmodules for removing the cpumap submodule
Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 15:17:09 +00:00
Herbert Wolverson
c8721619bc Add Rust source tree to the main repo.
Replaces the previously separate repo, merging the Rust
system into the mainline ready for 1.4.

The Rust system currently provides:

* lqos_bus: type definitions for localhost control bus.
* lqos_config: handler for configuration files.
* lqos_node_manager: local web-based monitor and manager.
* lqos_sys: eBPF program that handles all of the tracking,
  CPU assignment, VLAN redirect, TC queue assignment,
  and RTT tracking. Wrapped in a Rust core, providing
  consistent in-Rust building and mostly-safe wrappers
  to C code.
* lqosd: a daemon designed to run continually while
  LibreQoS is operating. On start, it parses the configuration
  files and sets up interface mapping (the Python code is still
  required to actually build queues). It then assigns the various
  eBPF programs to appropriate interfaces. The eBPF systems are
  removed when the daemon terminates.

  lqosd also provides a "bus", listening to requests for changes
  or information on localhost, providing a control plane for
  the rest of the project.
* lqtop: An example program demonstrating how to use the bus,
  acts like "top", showing current network traffic and mappings.
* xdp_iphash_to_cpu_cmdline: a Rust wrapper providing the same
  services as the cpumap originated tool of the same name. This is
  a "shim" - it will go away once the native Python library is
  ready.
* xdp_pping: also a shim, providing equivalent services to the
  cpumap service of the same name.

A helper shell script "remove_pinned_maps.sh" can be used to
remove all pinned eBPF maps from the system, allowing for eBPF
program upgrades that change persistent map structures without
a reboot.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 15:09:06 +00:00
Herbert Wolverson
5ef602044a Update .gitignore, fix syntax in LibreQoS.py
* Add src/rust/target to the .gitignore file,
  to avoid including Rust build artifacts in the
  Git tree.
* Fix the syntax when calling the IP mapping helper
  and deleting IP mappings.

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 15:07:47 +00:00
Herbert Wolverson
24cb42a39d Patch LibreQoS.py and ispConfig.example.py for Rust Integration
* Modifies ispConfig.example.py to include work-in-progress
  "on a stick" configuration. You can specify 2 VLANs on a
  single interface and operate on a single interface.
* LibreQoS.py no longer calls cpumap commands, but instead
  calls the Rust shims that do the same thing with the new
  Rust environment (which is in the next commit).
* LibreQoS.py detects when running "on a stick", and divides
  the number of available queues by 2. The first half are
  used for download queues, the second for upload queues.
  This allows a single TC hierarchy to be installed on the
  single "stick" interface, and still divide traffic by core.

Part of #181

Signed-off-by: Herbert Wolverson <herberticus@gmail.com>
2023-01-04 14:57:42 +00:00
Robert Chacón
7165b1ff6b
Fix partial relaod bug 2022-12-17 09:29:58 -07:00
Robert Chacón
0e21123b05
Merge pull request #178 from LibreQoE/issue_177_uisp_duplicate_ips
Issue 177 uisp duplicate ips
2022-12-17 07:21:22 -07:00
Herbert Wolverson
85becbd5c6 Do not permit duplicate IP addresses for device
FIXES #177

Apply the change suggested in the issue body: UISP can report
devices with duplicate IP addresses across interfaces. Check
that an IP address is unique before applying it.

Changed for both Flat and Hierarchy networks.
2022-12-17 08:16:35 -06:00
Herbert Wolverson
b9b344464c Change path to network.example.json.
When the repo was rearranged, it broke a unit test that expected
to find network.json.example in the "1.3" folder. It now looks
in the src/ folder, and the test passes.
2022-12-17 08:09:50 -06:00
Robert Chacón
fc00a25c16
Update README.md 2022-12-11 10:58:07 -07:00
Dave Täht
fef70b7029 More test tools 2022-12-11 09:05:09 -08:00
Dave Täht
9c27aae7a6 TCP cong controls 2022-12-11 08:14:18 -08:00
Dave Täht
b0e3c5e289 Some more plans 2022-12-11 08:09:00 -08:00
Dave Täht
113a3f7c96 More gitignore fixes 2022-12-10 21:37:52 -08:00
Dave Täht
88d191604d Jitter plot 2022-12-10 21:37:23 -08:00
Dave Täht
b1b6605b37 More notes about RTT monitoring 2022-12-10 21:19:55 -08:00
Dave Täht
a6ab2a7ecf Added RTT example 2022-12-10 21:04:25 -08:00
Dave Täht
e4a86107e8 Simulation Notes
A rather too-large overview of what the world might look like in a
testbed.
2022-12-10 20:43:44 -08:00
Dave Täht
a1d779f2d4 Clean up .gitignore 2022-12-10 20:43:29 -08:00
Dave Täht
9e984b1216 Major move around of every thing
Clean up old versions
2022-12-07 11:04:54 -08:00
Dave Täht
a45b6aebaa killed v1.2 2022-12-07 10:06:26 -08:00
Dave Täht
ea276db542 Got rid of "beta" everywhere
The v1.3 release features fast latency tracking, partial queue reload,
IPv6 and IPv4 trie support, huge scalability improvements (30k subs
per core now), a Circuit ID identifier and other improvements across the
board.
2022-12-07 10:02:18 -08:00
Dave Täht
afa4bb0556 Fixed rchac backlinks 2022-12-07 10:01:41 -08:00
Dave Täht
e381d22bac Updated readme for some backward compatability notes 2022-12-07 10:00:01 -08:00
Robert Chacón
f3a629c427
Merge pull request #171 from LibreQoE/sqm
Replace fqOrCAKE with sqm as per https://github.com/LibreQoE/LibreQoS/issues/168
2022-12-01 21:27:29 -07:00
Robert Chacón
4fc9dc9350
Update graphInfluxDB.py 2022-12-01 21:22:17 -07:00
Robert Chacón
d02a9be270
Update LibreQoS.py 2022-12-01 21:20:26 -07:00
Robert Chacón
297d66cb56
Update ispConfig.example.py 2022-12-01 21:18:55 -07:00
Robert Chacón
1ae0a2d568
Swtich from using schedule module (unreliable) to simple scheduling . Make sure to add crontab described in wiki 2022-12-01 09:00:30 -07:00
Robert Chacón
a58757839c
Fix bug in partial reload - "argument of type 'int' is not iterable" 2022-12-01 08:51:01 -07:00
Robert Chacón
00c0222c5f
Version of InfluxDB template for InfluxDB v2.5.1. 2022-11-28 14:52:46 -07:00
Robert Chacón
53c945e25c
Merge pull request #166 from interduo/patch-6
bugfix: ShapedDevices.csv needs to have CircuitID in all rows - default file validation failed in v1.3
2022-11-23 06:27:12 -07:00
Interduo
55174dcfe2
bugfix: ShapedDevices.csv needs to have CircuitID in all rows 2022-11-23 11:04:55 +01:00
Robert Chacón
1bd4b46c8e
Revert to version compatible with InfluxDB 2.4 and prior 2022-11-20 20:02:25 -07:00
Robert Chacón
067ec1f1f2
Update logo 2022-11-20 09:28:39 -07:00
Robert Chacón
f9d773a876
Update screenshot 2022-11-20 09:19:46 -07:00
Robert Chacón
3859aad663
Add CPU graphing 2022-11-20 09:18:39 -07:00
Robert Chacón
445ad0fedb
Make cpumap-pping unchangeable again 2022-11-20 09:15:30 -07:00
Robert Chacón
8c7cb38dcb
Make cpumap-pping unchangable again 2022-11-20 09:14:50 -07:00
Robert Chacón
a3a839d803
Standardize timestamps. Add CPU graphing 2022-11-20 09:13:38 -07:00
Robert Chacón
5d2ff90c54
Fix partial reload bug. Also make cpumap-pping unchangable again
There was previously an issue where during the partial reload, LibreQoS.py would fail to parse the network structure correctly. Now resolved.
2022-11-20 09:12:10 -07:00
rchac
d206eda4d8 Pulled newest eversion of cpumap-pping 2022-11-20 01:28:34 -07:00
Dave Täht
d85d2e9996 Cleanup bullets 2022-11-17 14:11:41 -08:00
Dave Täht
a594a47ffb More spelling and grammar fixes 2022-11-17 14:09:16 -08:00
Dave Täht
9e3c4d6114 Some speling, grammar, and other fixes.
This is also a test to see if I can commit directly to the repo.
2022-11-17 13:50:07 -08:00