Commit Graph

1904 Commits

Author SHA1 Message Date
Herbert Wolverson
1c22faa0f0 Packets per second chart now includes all shaper nodes rendered separately. 2023-04-26 15:31:57 +00:00
Herbert Wolverson
107efcf845 Remove debug code 2023-04-26 13:54:57 +00:00
Herbert Wolverson
b13bbf232b Allow node_name to appear in lqos.conf, and use it as the display name for nodes. 2023-04-26 13:46:25 +00:00
Herbert Wolverson
945da4c740 Comment out some debug code that was left enabled by mistake 2023-04-26 13:16:03 +00:00
Herbert Wolverson
a3c7b4cc99 Include node type in tree building. 2023-04-26 13:07:40 +00:00
Herbert Wolverson
566d96e280 Starting work on tree merging. 2023-04-25 21:23:29 +00:00
Herbert Wolverson
de35424359 Show RTT data on dashboard. Stacking is now accurate. 2023-04-25 19:41:44 +00:00
Herbert Wolverson
ca52f3f696 Remove unused imports 2023-04-25 17:58:42 +00:00
Herbert Wolverson
3b4f95fdb1 Basic display of total statistics. 2023-04-25 17:49:56 +00:00
Herbert Wolverson
bdfc9d4ab6 Separate components in TS 2023-04-25 13:29:31 +00:00
Herbert Wolverson
09a9c28e3a Finish merge, round dashboard servers. 2023-04-25 01:40:52 +00:00
Herbert Wolverson
df3be0b73e Speed up integrationUISP.py
* Adds one more "is not none" check for a line that failed on my
  installation.
* Add a level of indentation to the inner "walkGraph" function,
  so it doesn't painstakingly check every node when it already
  knows that it won't pick them (for a route cost).

My testing gave the same results, substantially faster.
2023-04-24 20:53:43 +00:00
Herbert Wolverson
c7ca4b171d Reduce default maximum tracked hosts to 64k from 128k 2023-04-24 20:53:43 +00:00
Herbert Wolverson
92db07b449 Remove now unused code. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
c510a9399a Missing doctag 2023-04-24 20:53:43 +00:00
Herbert Wolverson
f573001007 Update Heimdall to also use eBPF iterators
This eliminates another set of syscalls per cycle. The old per_cpu
map system is now removed, in favor of direct iteration of maps.
2023-04-24 20:53:43 +00:00
Herbert Wolverson
1e5f59c5ce Now that we're differentiating between single-CPU and per-CPU map types, we can eliminate 8 bytes of bus traffic by not reporting CPU counts (and padding) 2023-04-24 20:53:43 +00:00
Herbert Wolverson
6db2cb0245 Add a little padding to the output stream kernel side, to nicely align on 64-bit boundaries. Then use aligned casting for zero-copy userspace access to the data. Another smallish speed increase. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
0c62268ed6 Avoid reallocation of multi-cpu vectors during iteration. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
cf7ccb7954 Start documenting the types created in this patch-chain. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
b192478a43 Remove a no-longer-needed import. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
dcf2c5f7e2 Remove 2 more syscalls by removing a useless mutex. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
082a16e532 Handle per-CPU maps correctly in eBPF iterators
The eBPF iterator interface is now per-CPU aware, and correctly
extracts a slice of data for each per-CPU map (or a slice of 1
when using a non-CPU divided map).
2023-04-24 20:53:43 +00:00
Herbert Wolverson
f0ea3edcff Use eBPF iterators (requires kernel 5.8+) instead of next_entry
Replaces the eBPF interface "bpf_map_get_next_key" followed by
"bpf_map_lookup_elem" cycle for iterating maps with a call to
the kernel-provided eBPF iterator service.

Currently only implemented for throughput and RTT tracking.
Further commits will tighten the scope and eventually remove
the legacy setup completely.

For 10k hosts, this reduces the number of system calls from
20,002 to 4. The remaining items are a futex lock (required
for safety), and an iterator call.
2023-04-24 20:53:43 +00:00
Herbert Wolverson
cadd3aa800 Add the new tool to the git ignore list. 2023-04-24 20:53:43 +00:00
Herbert Wolverson
5cd5b44496 Add a tool to track eBPF map performance
New tool: lqos_map_perf is included in the distribution. It
runs the two map iterators that pound the lqosd/eBPF system
with map iterations, counts the number of entries returned
and reports on overall performance (in µs).

This is intended to help with diagnosing map reading
performance issues and act as a bench-test to see if
using eBPF iterators actually helps.
2023-04-24 20:53:43 +00:00
Herbert Wolverson
bbf74ce52d Basic login, resume from stored token and node shaper status display. 2023-04-24 20:06:22 +00:00
Herbert Wolverson
f4f2e81e04 Initial (primitive) login system functioning with a websocket messaging system. 2023-04-24 16:32:23 +00:00
Robert Chacón
b4feb920b0
Merge pull request #345 from LibreQoE/node_types_hotfix
Hotfix: handle node "type" entry
2023-04-24 09:49:47 -06:00
Herbert Wolverson
5795181d18 Update Rust package database to use newer versions of some crates, removing a non-threatening CVE. 2023-04-24 13:49:28 +00:00
Herbert Wolverson
36440bdd9c Handle cases where the field isn't present in network.json for backwards compatibility. 2023-04-24 13:48:19 +00:00
Herbert Wolverson
629ebb0293 Hotfix: handle node "type" entry
Adding "type" to `network.json` is awesome, but it broke a bit of
Rust. This patch adds support for the `type` field, and fixes
a rare "none" issue that mostly seems to affect my UISP setup.
2023-04-24 13:28:37 +00:00
Herbert Wolverson
7a1ea123db WSS is basically working, needs a lot of tuning. 2023-04-24 13:03:15 +00:00
Robert Chacón
125ef55e35
Update README.md 2023-04-23 21:57:19 -06:00
Robert Chacón
20d69aa6d5
Update ispConfig.example.py 2023-04-23 13:46:03 -06:00
Herbert Wolverson
274d1c93ac Starting to pull together an SPA structure: a router, page interface and the login page with template. 2023-04-21 19:40:07 +00:00
Herbert Wolverson
f03d305dc8 Starting to pull together an SPA structure: a router, page interface and the login page with template. 2023-04-21 19:39:48 +00:00
Herbert Wolverson
a84f590393 Very first baby-steps into the land of modern web development.
Use `esbuild` to build TypeScript, and bundle components together
in a minified form for the `lts_node` website.

This is very much a work in progress, committing as a checkpoint.
2023-04-21 18:52:53 +00:00
Herbert Wolverson
6e36ab7161 Cleanup warning 2023-04-21 15:16:26 +00:00
Herbert Wolverson
87839a83d7 Change licensing server host to 'license.libreqos.io' 2023-04-21 14:43:30 +00:00
Robert Chacón
f5d6257652
Print runtime of UISP Integration. Correct print statement before walkGraphOutwards() 2023-04-21 02:01:19 -06:00
Herbert Wolverson
2a9f80428c Create the "orchestrator"
The orchestrator accepts a "new free license request", checks
that there is an available stats host (it picks the least busy).
It then creates a license key, adds the organization and key
to the PostgreSQL database and creates a bucket for the new
organization in InfluxDB2.

The "licman" tool now accepts...
licman license freetrial "My Org Name"
... and returns a license key for inclusion in your lqos.conf
file.

This has been briefly tested, and basic stats appeared.
2023-04-20 21:46:46 +00:00
Herbert Wolverson
e75dbca434 To make it easier while I work on a bizarre split setup, store the influx url and the host IP separately. 2023-04-20 20:02:34 +00:00
Herbert Wolverson
07dc9c0745 Add a new "licman" command.
This is intended to manage the license management server, once it
is running. It starts with a command "licman add host <ip>" that
adds a data-collector to the available stats hosts pool.
2023-04-20 17:51:27 +00:00
Herbert Wolverson
dca44f294a Add sqlx database migrations support
To run the migration, you need to have a PostgreSQL installation
setup. It must accept the host you are on, have an empty database
ready to go named "libreqos", and a user with access to that
database.

Make sure you have the sqlx tool installed on the licensing server:
`cargo install sqlx-cli --no-default-features --features rustls,postgres`

You need to create a .env file in this pgdb directory, containing
DATABASE_URL=postgres://license:licensepassword@1.2.3.4/libreqos

Then run `sqlx migrate run` to populate the database with initial
data.
2023-04-20 17:01:50 +00:00
Herbert Wolverson
418f2d0ae8 Working on my testbed, ingesting total network stats from a node. Obviously lots more to do, but a great start. 2023-04-20 15:34:09 +00:00
Herbert Wolverson
ec53edd73d Merge Cargo.toml with develop 2023-04-20 13:32:59 +00:00
Robert Chacón
ee83b9f463
Update INTEGRATION_HOWTO.md 2023-04-19 19:27:56 -06:00
Robert Chacón
77f21b4118
Update .gitignore 2023-04-19 19:14:47 -06:00
Robert Chacón
71e7610c75
Update .gitignore 2023-04-19 19:09:56 -06:00