Commit Graph

1677 Commits

Author SHA1 Message Date
Herbert Wolverson
f4f2e81e04 Initial (primitive) login system functioning with a websocket messaging system. 2023-04-24 16:32:23 +00:00
Robert Chacón
b4feb920b0
Merge pull request #345 from LibreQoE/node_types_hotfix
Hotfix: handle node "type" entry
2023-04-24 09:49:47 -06:00
Herbert Wolverson
5795181d18 Update Rust package database to use newer versions of some crates, removing a non-threatening CVE. 2023-04-24 13:49:28 +00:00
Herbert Wolverson
36440bdd9c Handle cases where the field isn't present in network.json for backwards compatibility. 2023-04-24 13:48:19 +00:00
Herbert Wolverson
629ebb0293 Hotfix: handle node "type" entry
Adding "type" to `network.json` is awesome, but it broke a bit of
Rust. This patch adds support for the `type` field, and fixes
a rare "none" issue that mostly seems to affect my UISP setup.
2023-04-24 13:28:37 +00:00
Herbert Wolverson
7a1ea123db WSS is basically working, needs a lot of tuning. 2023-04-24 13:03:15 +00:00
Robert Chacón
125ef55e35
Update README.md 2023-04-23 21:57:19 -06:00
Robert Chacón
20d69aa6d5
Update ispConfig.example.py 2023-04-23 13:46:03 -06:00
Herbert Wolverson
274d1c93ac Starting to pull together an SPA structure: a router, page interface and the login page with template. 2023-04-21 19:40:07 +00:00
Herbert Wolverson
f03d305dc8 Starting to pull together an SPA structure: a router, page interface and the login page with template. 2023-04-21 19:39:48 +00:00
Herbert Wolverson
a84f590393 Very first baby-steps into the land of modern web development.
Use `esbuild` to build TypeScript, and bundle components together
in a minified form for the `lts_node` website.

This is very much a work in progress, committing as a checkpoint.
2023-04-21 18:52:53 +00:00
Herbert Wolverson
6e36ab7161 Cleanup warning 2023-04-21 15:16:26 +00:00
Herbert Wolverson
87839a83d7 Change licensing server host to 'license.libreqos.io' 2023-04-21 14:43:30 +00:00
Robert Chacón
f5d6257652
Print runtime of UISP Integration. Correct print statement before walkGraphOutwards() 2023-04-21 02:01:19 -06:00
Herbert Wolverson
2a9f80428c Create the "orchestrator"
The orchestrator accepts a "new free license request", checks
that there is an available stats host (it picks the least busy).
It then creates a license key, adds the organization and key
to the PostgreSQL database and creates a bucket for the new
organization in InfluxDB2.

The "licman" tool now accepts...
licman license freetrial "My Org Name"
... and returns a license key for inclusion in your lqos.conf
file.

This has been briefly tested, and basic stats appeared.
2023-04-20 21:46:46 +00:00
Herbert Wolverson
e75dbca434 To make it easier while I work on a bizarre split setup, store the influx url and the host IP separately. 2023-04-20 20:02:34 +00:00
Herbert Wolverson
07dc9c0745 Add a new "licman" command.
This is intended to manage the license management server, once it
is running. It starts with a command "licman add host <ip>" that
adds a data-collector to the available stats hosts pool.
2023-04-20 17:51:27 +00:00
Herbert Wolverson
dca44f294a Add sqlx database migrations support
To run the migration, you need to have a PostgreSQL installation
setup. It must accept the host you are on, have an empty database
ready to go named "libreqos", and a user with access to that
database.

Make sure you have the sqlx tool installed on the licensing server:
`cargo install sqlx-cli --no-default-features --features rustls,postgres`

You need to create a .env file in this pgdb directory, containing
DATABASE_URL=postgres://license:licensepassword@1.2.3.4/libreqos

Then run `sqlx migrate run` to populate the database with initial
data.
2023-04-20 17:01:50 +00:00
Herbert Wolverson
418f2d0ae8 Working on my testbed, ingesting total network stats from a node. Obviously lots more to do, but a great start. 2023-04-20 15:34:09 +00:00
Herbert Wolverson
ec53edd73d Merge Cargo.toml with develop 2023-04-20 13:32:59 +00:00
Robert Chacón
ee83b9f463
Update INTEGRATION_HOWTO.md 2023-04-19 19:27:56 -06:00
Robert Chacón
77f21b4118
Update .gitignore 2023-04-19 19:14:47 -06:00
Robert Chacón
71e7610c75
Update .gitignore 2023-04-19 19:09:56 -06:00
Robert Chacón
f68da7f7a2
Merge pull request #342 from LibreQoE/main
UISP Integration Improvements -> Develop
2023-04-19 19:02:25 -06:00
Robert Chacón
111272e3e1
Merge pull request #341 from LibreQoE/bpf_map_performance
Bpf map performance
2023-04-19 18:37:40 -06:00
Herbert Wolverson
54d59af347 Reduce default maximum tracked hosts to 64k from 128k 2023-04-19 21:29:15 +00:00
Herbert Wolverson
1727468e66 Remove now unused code. 2023-04-19 20:56:56 +00:00
Herbert Wolverson
775a903f2d Missing doctag 2023-04-19 20:55:16 +00:00
Herbert Wolverson
1e8ac64074 Update Heimdall to also use eBPF iterators
This eliminates another set of syscalls per cycle. The old per_cpu
map system is now removed, in favor of direct iteration of maps.
2023-04-19 20:54:10 +00:00
Herbert Wolverson
d5becf7190 Now that we're differentiating between single-CPU and per-CPU map types, we can eliminate 8 bytes of bus traffic by not reporting CPU counts (and padding) 2023-04-19 20:17:19 +00:00
Herbert Wolverson
239b271e0a Add a little padding to the output stream kernel side, to nicely align on 64-bit boundaries. Then use aligned casting for zero-copy userspace access to the data. Another smallish speed increase. 2023-04-19 18:31:16 +00:00
Herbert Wolverson
9261641f42 Avoid reallocation of multi-cpu vectors during iteration. 2023-04-19 16:38:39 +00:00
Herbert Wolverson
1c588a5a48 Start documenting the types created in this patch-chain. 2023-04-19 13:12:18 +00:00
Herbert Wolverson
122caaf10b Remove a no-longer-needed import. 2023-04-19 13:02:11 +00:00
Herbert Wolverson
04ff52a007 Remove 2 more syscalls by removing a useless mutex. 2023-04-18 21:51:50 +00:00
Herbert Wolverson
78e36d2e21 Handle per-CPU maps correctly in eBPF iterators
The eBPF iterator interface is now per-CPU aware, and correctly
extracts a slice of data for each per-CPU map (or a slice of 1
when using a non-CPU divided map).
2023-04-18 20:37:49 +00:00
Herbert Wolverson
f0975ccbcd Use eBPF iterators (requires kernel 5.8+) instead of next_entry
Replaces the eBPF interface "bpf_map_get_next_key" followed by
"bpf_map_lookup_elem" cycle for iterating maps with a call to
the kernel-provided eBPF iterator service.

Currently only implemented for throughput and RTT tracking.
Further commits will tighten the scope and eventually remove
the legacy setup completely.

For 10k hosts, this reduces the number of system calls from
20,002 to 4. The remaining items are a futex lock (required
for safety), and an iterator call.
2023-04-18 17:24:25 +00:00
Herbert Wolverson
cc88ed1c62 Add the new tool to the git ignore list. 2023-04-18 13:05:55 +00:00
Herbert Wolverson
80a1356720 Add a tool to track eBPF map performance
New tool: lqos_map_perf is included in the distribution. It
runs the two map iterators that pound the lqosd/eBPF system
with map iterations, counts the number of entries returned
and reports on overall performance (in µs).

This is intended to help with diagnosing map reading
performance issues and act as a bench-test to see if
using eBPF iterators actually helps.
2023-04-17 13:28:09 +00:00
Herbert Wolverson
6337bdb3d0 WIP - sumissions transition to a processing queue. Beginnings of some early Influx code. 2023-04-17 13:10:54 +00:00
Robert Chacón
c7d501a957
Add option to control frequency of queue refreshes in minutes - queueRefreshIntervalMins 2023-04-16 09:32:17 -06:00
Robert Chacón
d965f51a64
Update ispConfig.example.py 2023-04-16 09:28:53 -06:00
Robert Chacón
1ee7484d86
Add 'type' to network.json. For https://github.com/LibreQoE/LibreQoS/issues/308 2023-04-15 11:24:07 -06:00
Robert Chacón
4861b5f2fd
Rename integrationUISProutes.csv to integrationUISProutes.template.csv 2023-04-15 10:37:21 -06:00
Robert Chacón
ffda0a08f1
Render network map graph horizontally. Makes much more efficient use of space. 2023-04-15 10:29:49 -06:00
Robert Chacón
286c2a4931
Only print warning if actual site goes off PtMP AP, not client sites 2023-04-15 10:03:41 -06:00
Robert Chacón
2b16930586
https://github.com/LibreQoE/LibreQoS/issues/336 2023-04-15 09:56:45 -06:00
Robert Chacón
c62f6d3e3c
Fix https://github.com/LibreQoE/LibreQoS/issues/309 2023-04-15 09:44:50 -06:00
Robert Chacón
3d6d32c300
Update integrationUISP.py 2023-04-15 09:42:03 -06:00
Robert Chacón
eced3e2ac5
Update integrationUISP.py 2023-04-14 23:56:08 -06:00