IntenseWebs/LibreQoS
3
0 Fork 0
mirror of https://github.com/LibreQoE/LibreQoS.git synced 2026-11-03 12:47:03 -06:00
Code Issues Packages Projects Releases Wiki Activity
1,993 Commits 278 Branches 8 Tags
a5bef2851a0232b67a82c553e7eb621cc00630d9
Commit Graph
1993 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Herbert Wolverson a5bef2851a Refactor to clean code up. 2024-03-05 11:02:47 -06:00
Herbert Wolverson f0ddbe62f8 Netflow V5 is largely working. Still a few kinks to work out, but the exporter sends them out - and they are received correctly by my test ehnt setup. 2024-03-05 09:31:49 -06:00
Herbert Wolverson 28e861aeee First attempt at adding NetFlow v5 support. 2024-03-05 08:44:57 -06:00
Herbert Wolverson 13f2fabe4c Merge branch 'develop' into per_flow 2024-03-02 09:16:08 -06:00
Herbert Wolverson d741a24ed2 Merge branch 'circuit_weights' into develop 2024-03-01 14:28:41 -06:00
Herbert Wolverson b9d2be426c Add a safety feature that if a site isn't found, auto-allocate its bin. 2024-03-01 14:28:36 -06:00
Herbert Wolverson cdadbb2c1a Undoes last commit. This is stable. 2024-03-01 14:28:36 -06:00
Herbert Wolverson a1314cccbf Add support for virtual nodes by accident, finishing up stage 2 that allows us to skip layers in the tree. 2024-03-01 14:28:36 -06:00
Herbert Wolverson 1ac7a44355 Stage 1 of the revamp. Instead of modifying traverseNetwork directly, builds a layer per CPU in the network and then uses the previous code. 2024-03-01 14:28:36 -06:00
Herbert Wolverson 7836dbc3bb Add a safety feature that if a site isn't found, auto-allocate its bin. 2024-03-01 14:17:59 -06:00
Herbert Wolverson e9530dff54 Undoes last commit. This is stable. 2024-03-01 13:59:34 -06:00
Herbert Wolverson 728b0a9153 Add support for virtual nodes by accident, finishing up stage 2 that allows us to skip layers in the tree. 2024-03-01 13:50:39 -06:00
Herbert Wolverson 008236796c Stage 1 of the revamp. Instead of modifying traverseNetwork directly, builds a layer per CPU in the network and then uses the previous code. 2024-03-01 13:36:44 -06:00
Robert Chacón 57c99cfa04 Merge pull request #462 from LibreQoE/main 
Update Develop
 2024-03-01 11:22:33 -07:00
Robert Chacón 5ff0737a76 Merge pull request #461 from LibreQoE/circuit_weights 
Circuit weights
 2024-03-01 11:21:57 -07:00
Herbert Wolverson fcf07db112 On flat networks, read calculated LTS (or not) weights and apply them to the binpacking circuit weights if found. This makes binpacking more dynamic for flat networks, taking into account usual usage patterns. 2024-03-01 12:12:32 -06:00
Herbert Wolverson 64fc075066 First try at binpacking the top-level shaper nodes. 2024-03-01 11:17:48 -06:00
Herbert Wolverson 0efa269816 Weight calculation for tree nodes. 2024-03-01 10:24:24 -06:00
Herbert Wolverson ff0b397395 Dedupe the RTT estimation code. 2024-02-29 12:26:31 -06:00
Herbert Wolverson 0173798981 WIP - Fix my botch at deduping the flow key code. 2024-02-29 12:16:45 -06:00
Herbert Wolverson 28ebecc44d Forgot to comment out a debug line. 2024-02-29 11:53:10 -06:00
Herbert Wolverson 2be8ce4aa9 Add wraparound logic to handle ACK sequence wraparound. 2024-02-29 11:35:25 -06:00
Herbert Wolverson 370b65fa67 Compromise on 10ms sampling time for rate estimation. Giving pretty accurate results on my tests, and not hitting the CPU too hard. 2024-02-29 11:19:35 -06:00
Herbert Wolverson 23f2cfb9d8 Reduce the flow rate estimation frequency from 1 second to 1 ms. 
This reduces round-error issues and is generally more accurate -
at the expense of slightly higher CPU usage and a "more bouncy"
rate - that is, for a widely variable flow such as Netflix it
jumps around a lot more. I remain unconvinced that 1ms is the
right number - will continue to test.

Thanks again to Simon Sundberg for this suggestion.
 2024-02-29 11:05:37 -06:00
Herbert Wolverson 7d6cbd417a * Remove most code duplication by tracking rate_index and its reciprocal. 
* Significantly nicer comments.
* I used a ternary operator. Please shoot me.

Thanks to Simon Sundberg for this recommendation.
 2024-02-29 10:17:28 -06:00
Herbert Wolverson 8f343b7c3b More flexible API for 'top N' calculations based on flow buffer. 2024-02-29 09:10:31 -06:00
Herbert Wolverson 0659cda225 Managed to reduce the amount of space required in the XDP metadata buffer to 32 bits. 2024-02-29 08:45:41 -06:00
Herbert Wolverson 9de53853e7 Improve comments on metadata 2024-02-29 08:28:33 -06:00
Herbert Wolverson f84798885b Add a stub for listing top flows by current bitrate. 2024-02-28 15:03:03 -06:00
Herbert Wolverson 7516715874 Add a real-time flow counter to the display and API. 2024-02-28 14:44:20 -06:00
Herbert Wolverson 393c3adc2a * Restore the tc_dissector, I didn't mean to remove that. 
* Add SKB metadata support for pass-through and eliding the
  second LPM check if we can. Checks at run-time to see if
  it is possible.
 2024-02-28 12:52:04 -06:00
Herbert Wolverson 95b7c9ac5c No more second stage dissector. 2024-02-28 10:32:02 -06:00
Herbert Wolverson 22d56a71bf Correct the ordering of src/dst ports in the flow map. 2024-02-28 09:56:59 -06:00
Herbert Wolverson ff4c070b7d Improve the JSON transfer data by listing protocol explicitly for flowbee. 2024-02-28 09:51:43 -06:00
Herbert Wolverson 29b0e07867 Massive improvement in RTT tracking accuracy. 2024-02-28 09:29:20 -06:00
Herbert Wolverson e98a1864ad Add a new api call - api/flows/dump_all - that lists all recent flows that have been collected. Intended for debugging. 2024-02-27 16:22:22 -06:00
Herbert Wolverson df2b9dfe32 Integrate the flow data into the moving average system for RTTs, giving a better spread of results. 2024-02-27 14:54:29 -06:00
Herbert Wolverson f33d22faa0 In progress. The RTT data from the flows system is integrated into the high-level per-IP RTT tracker. 2024-02-27 14:06:13 -06:00
Herbert Wolverson b7c02d251d Warning fix 2024-02-27 11:56:36 -06:00
Herbert Wolverson 8fa53782c6 Work in progress. Add endian conversions. Add Rust interface for iterating the flows list. Add a temporary interface that dumps flow data to the console to prove that it works. 2024-02-27 11:53:18 -06:00
Herbert Wolverson bfc9b8227c Skeleton for including UDP and ICMP flow data. 2024-02-27 10:38:17 -06:00
Herbert Wolverson c7df905e24 WIP - cleanup done, TCP is mostly working on the server-side. 2024-02-27 10:30:48 -06:00
Herbert Wolverson 0fd6b29e6c Considerably cleaned up flow tracking system. 2024-02-26 20:36:37 -06:00
Robert Chacón 4829861eb5 Merge pull request #460 from LibreQoE/rchac-patch-12 
Fix UISP Integration Bug - Disconnected Sites Should Still Be Shaped
 2024-02-16 06:59:48 -07:00
Robert Chacón 16efa540f3 Update integrationUISP.py 2024-02-16 06:55:52 -07:00
Herbert Wolverson 23487f3328 The flows now include their very own rate estimator, and only count RTT when rate exceeds a threshold - 5mbps right now, which is probably not a good choice. 2024-02-14 14:56:46 -06:00
Herbert Wolverson 1ca595b16c Hacking away - remove the fin/reset flags because they shouldn't be needed. 2024-02-14 14:40:58 -06:00
Herbert Wolverson 295caaad91 Very early days - kinda/sorta working per-flow work. 2024-02-14 13:51:55 -06:00
Herbert Wolverson 088eed555a Correct URL in device_weights.rs 2024-02-08 13:21:56 -06:00
Herbert Wolverson 05d2a398a3 Add a device_weights() call to the Python API 
Adds the ability to call device_weights() from liblqos_python.
This returns a set of objects containing circuit_id and weight.

1. The weights are loaded from ShapedDevices.csv using HALF the
   maximum download mbps.
2. If LTS is enabled, a REST call is made to an LTS API located at
   https:/stats.libreqos.io/api/device_weights and the results
   merged in (overwiting existing entries).
3. The merged weights are returned as a Python array of classes.

Example Python code:

```python
from liblqos_python import get_weights;
weights = get_weights();
for w in weights:
    print(w.circuit_id + " : " + str(w.weight));
```
 2024-02-08 12:01:12 -06:00