Herbert Wolverson
1fb151aa1b
Minimal protocol analysis beginnings.
2024-03-08 14:52:02 -06:00
Herbert Wolverson
5202f447dc
Begin adding a little UI
2024-03-08 14:03:29 -06:00
Herbert Wolverson
6b384b2a5f
Include ASN assessment in flows data.
2024-03-08 13:15:49 -06:00
Herbert Wolverson
b1cd8e5ed4
Setup initial stage of ASN analysis - downloading and loading the table, periodic upgrades.
2024-03-08 12:11:37 -06:00
Herbert Wolverson
9c7a9849ad
Replace mutex locked vector with a dashmap for flow data, to ease sharing and update rather than replace cycle.
2024-03-08 09:32:15 -06:00
Herbert Wolverson
b7d43567ff
Reworked the NetFlow code to batch packets into single submissions of up to 30 packets at a time.
2024-03-07 12:51:08 -06:00
Herbert Wolverson
04b0cd4246
Fix warning
2024-03-07 11:38:12 -06:00
Herbert Wolverson
33c1efdd2c
IPv6 encoding matches the same pattern, needs testing.
2024-03-07 11:27:33 -06:00
Herbert Wolverson
34a2ec7b88
Refactor netflow v9 into readable code, and the IPv4 version is at least somewhat working now.
2024-03-07 11:19:02 -06:00
Herbert Wolverson
10c56f9353
This time with a theoretically valid header...
2024-03-05 14:54:57 -06:00
Herbert Wolverson
b649f7004e
Netflow v9. Probably doesn't work yet, committing before I change PC.
2024-03-05 14:05:18 -06:00
Herbert Wolverson
a5bef2851a
Refactor to clean code up.
2024-03-05 11:02:47 -06:00
Herbert Wolverson
f0ddbe62f8
Netflow V5 is largely working. Still a few kinks to work out, but the exporter sends them out - and they are received correctly by my test ehnt setup.
2024-03-05 09:31:49 -06:00
Herbert Wolverson
28e861aeee
First attempt at adding NetFlow v5 support.
2024-03-05 08:44:57 -06:00
Herbert Wolverson
13f2fabe4c
Merge branch 'develop' into per_flow
2024-03-02 09:16:08 -06:00
Herbert Wolverson
d741a24ed2
Merge branch 'circuit_weights' into develop
2024-03-01 14:28:41 -06:00
Herbert Wolverson
b9d2be426c
Add a safety feature that if a site isn't found, auto-allocate its bin.
2024-03-01 14:28:36 -06:00
Herbert Wolverson
cdadbb2c1a
Undoes last commit. This is stable.
2024-03-01 14:28:36 -06:00
Herbert Wolverson
a1314cccbf
Add support for virtual nodes by accident, finishing up stage 2 that allows us to skip layers in the tree.
2024-03-01 14:28:36 -06:00
Herbert Wolverson
1ac7a44355
Stage 1 of the revamp. Instead of modifying traverseNetwork directly, builds a layer per CPU in the network and then uses the previous code.
2024-03-01 14:28:36 -06:00
Herbert Wolverson
7836dbc3bb
Add a safety feature that if a site isn't found, auto-allocate its bin.
2024-03-01 14:17:59 -06:00
Herbert Wolverson
e9530dff54
Undoes last commit. This is stable.
2024-03-01 13:59:34 -06:00
Herbert Wolverson
728b0a9153
Add support for virtual nodes by accident, finishing up stage 2 that allows us to skip layers in the tree.
2024-03-01 13:50:39 -06:00
Herbert Wolverson
008236796c
Stage 1 of the revamp. Instead of modifying traverseNetwork directly, builds a layer per CPU in the network and then uses the previous code.
2024-03-01 13:36:44 -06:00
Robert Chacón
57c99cfa04
Merge pull request #462 from LibreQoE/main
...
Update Develop
2024-03-01 11:22:33 -07:00
Robert Chacón
5ff0737a76
Merge pull request #461 from LibreQoE/circuit_weights
...
Circuit weights
2024-03-01 11:21:57 -07:00
Herbert Wolverson
fcf07db112
On flat networks, read calculated LTS (or not) weights and apply them to the binpacking circuit weights if found. This makes binpacking more dynamic for flat networks, taking into account usual usage patterns.
2024-03-01 12:12:32 -06:00
Herbert Wolverson
64fc075066
First try at binpacking the top-level shaper nodes.
2024-03-01 11:17:48 -06:00
Herbert Wolverson
0efa269816
Weight calculation for tree nodes.
2024-03-01 10:24:24 -06:00
Herbert Wolverson
ff0b397395
Dedupe the RTT estimation code.
2024-02-29 12:26:31 -06:00
Herbert Wolverson
0173798981
WIP - Fix my botch at deduping the flow key code.
2024-02-29 12:16:45 -06:00
Herbert Wolverson
28ebecc44d
Forgot to comment out a debug line.
2024-02-29 11:53:10 -06:00
Herbert Wolverson
2be8ce4aa9
Add wraparound logic to handle ACK sequence wraparound.
2024-02-29 11:35:25 -06:00
Herbert Wolverson
370b65fa67
Compromise on 10ms sampling time for rate estimation. Giving pretty accurate results on my tests, and not hitting the CPU too hard.
2024-02-29 11:19:35 -06:00
Herbert Wolverson
23f2cfb9d8
Reduce the flow rate estimation frequency from 1 second to 1 ms.
...
This reduces round-error issues and is generally more accurate -
at the expense of slightly higher CPU usage and a "more bouncy"
rate - that is, for a widely variable flow such as Netflix it
jumps around a lot more. I remain unconvinced that 1ms is the
right number - will continue to test.
Thanks again to Simon Sundberg for this suggestion.
2024-02-29 11:05:37 -06:00
Herbert Wolverson
7d6cbd417a
* Remove most code duplication by tracking rate_index and its reciprocal.
...
* Significantly nicer comments.
* I used a ternary operator. Please shoot me.
Thanks to Simon Sundberg for this recommendation.
2024-02-29 10:17:28 -06:00
Herbert Wolverson
8f343b7c3b
More flexible API for 'top N' calculations based on flow buffer.
2024-02-29 09:10:31 -06:00
Herbert Wolverson
0659cda225
Managed to reduce the amount of space required in the XDP metadata buffer to 32 bits.
2024-02-29 08:45:41 -06:00
Herbert Wolverson
9de53853e7
Improve comments on metadata
2024-02-29 08:28:33 -06:00
Herbert Wolverson
f84798885b
Add a stub for listing top flows by current bitrate.
2024-02-28 15:03:03 -06:00
Herbert Wolverson
7516715874
Add a real-time flow counter to the display and API.
2024-02-28 14:44:20 -06:00
Herbert Wolverson
393c3adc2a
* Restore the tc_dissector, I didn't mean to remove that.
...
* Add SKB metadata support for pass-through and eliding the
second LPM check if we can. Checks at run-time to see if
it is possible.
2024-02-28 12:52:04 -06:00
Herbert Wolverson
95b7c9ac5c
No more second stage dissector.
2024-02-28 10:32:02 -06:00
Herbert Wolverson
22d56a71bf
Correct the ordering of src/dst ports in the flow map.
2024-02-28 09:56:59 -06:00
Herbert Wolverson
ff4c070b7d
Improve the JSON transfer data by listing protocol explicitly for flowbee.
2024-02-28 09:51:43 -06:00
Herbert Wolverson
29b0e07867
Massive improvement in RTT tracking accuracy.
2024-02-28 09:29:20 -06:00
Herbert Wolverson
e98a1864ad
Add a new api call - api/flows/dump_all - that lists all recent flows that have been collected. Intended for debugging.
2024-02-27 16:22:22 -06:00
Herbert Wolverson
df2b9dfe32
Integrate the flow data into the moving average system for RTTs, giving a better spread of results.
2024-02-27 14:54:29 -06:00
Herbert Wolverson
f33d22faa0
In progress. The RTT data from the flows system is integrated into the high-level per-IP RTT tracker.
2024-02-27 14:06:13 -06:00
Herbert Wolverson
b7c02d251d
Warning fix
2024-02-27 11:56:36 -06:00
