Commit Graph

2044 Commits

Author SHA1 Message Date
Herbert Wolverson
b6eb08751c Fix order of magnitude on the display histogram. 2024-03-15 13:27:12 -05:00
Herbert Wolverson
d0e575f317 Add compatibility with the 5.x kernel version eBPF verifier. 2024-03-15 13:24:58 -05:00
Herbert Wolverson
6ce7e7a3f9 Strongly type RTT data in the throughput tracker to reduce confusion. 2024-03-15 13:03:49 -05:00
Herbert Wolverson
b8937bf9c2 Basic web UI update to show numbers in the right units 2024-03-15 12:55:23 -05:00
Herbert Wolverson
9d0e536089 Use a dual-structure to store RTT in both directions. 2024-03-15 12:29:47 -05:00
Herbert Wolverson
9938a94caf Comment out console log that was annoying me 2024-03-15 12:19:59 -05:00
Herbert Wolverson
beda42194a Include RTT events per second tracking in the /api/stats call 2024-03-15 12:18:19 -05:00
Herbert Wolverson
f44af37670 WIP - Compiles RTT data into the tracker, strong type RTTs to clarify the unit confusion. Web side is not done yet. 2024-03-15 12:15:11 -05:00
Herbert Wolverson
56b170f7e4 Truly minimal RTT to userspace eBPF ringbuffer implementation. This
breaks a lot of things - no RTTs are currently recorded or acted
upon. The goal is to measure the scale of the load hit for receiving
these events.
2024-03-15 09:45:49 -05:00
Herbert Wolverson
3ad023f69c Fix circuit flows display from last patch. 2024-03-15 09:02:07 -05:00
Herbert Wolverson
318e20dad7 These numbers line up on the iZones tracker, with wireshark dumps for comparison. This is a WIP - don't apply this. Unbreaking some of my previous code. 2024-03-15 08:32:06 -05:00
Herbert Wolverson
56dd4b6750 It's not quite done yet, but I spent the last 2 hours matching observed RTT, checking with Wireshark, and looking to see if the numbers line up. Implements a small buffer for RTT values per flow. Shrinks some stack entries. Will require a map rebuild. 2024-03-14 14:07:06 -05:00
Herbert Wolverson
69508d2753 Remove two variables and use pointers instead to save stack space. 2024-03-14 12:40:33 -05:00
Herbert Wolverson
a937820a87 Move rate estimate from 64-bits to 32-bits, saving some bytes. 2024-03-14 11:03:43 -05:00
Herbert Wolverson
8ec361b095 Fix a reporting message that was meant to be compile-conditional. 2024-03-14 10:42:22 -05:00
Herbert Wolverson
fbb3960b50 Warning fix 2024-03-14 08:52:36 -05:00
Herbert Wolverson
eab9df4874 Missing file from previous commit 2024-03-13 14:56:59 -05:00
Herbert Wolverson
fb91e8313a Very silly commit - adds a /showoff page to the node manager firing particles from all endpoints at my ISP in Missouri. Will turn into something useful in the future. 2024-03-13 14:45:53 -05:00
Herbert Wolverson
a63ff0a6f1 Rename all flowbee "retries" to "tcp_retransmits" in code, and
"retransmits" in visible HTML.
2024-03-13 08:20:36 -05:00
Herbert Wolverson
fae96280b9 Actually limit the top 10 endpoints to being 10 2024-03-12 16:41:57 -05:00
Herbert Wolverson
c16f06b0ab Sort countries by download bytes not upload 2024-03-12 15:47:29 -05:00
Herbert Wolverson
66a19c04a4 First example of an actual summary report - we can group flow endpoints in a 5-minute report. 2024-03-12 15:19:07 -05:00
Herbert Wolverson
1fb5838ebe Remove a test call that was spamming the logs. 2024-03-12 14:46:47 -05:00
Herbert Wolverson
e46aafe5ae Silly performance: making 12k mutex locks when I need one was really dumb. 2024-03-12 14:20:35 -05:00
Herbert Wolverson
5a3f90412d Working geocode system, albeit not as useful as I hoped. 2024-03-12 14:02:36 -05:00
Herbert Wolverson
55f24cf71b Revert previous bad idea 2024-03-12 11:38:19 -05:00
Herbert Wolverson
07239b3d24 That should avoid some flow duplication that was hitting too hard. 2024-03-12 11:30:13 -05:00
Herbert Wolverson
3ca7ca8a0d Lower threshold for capturing RTT by rate 2024-03-12 10:52:41 -05:00
Herbert Wolverson
e20d6d39b0 Another try at a flow system that cleans up. 2024-03-12 10:29:08 -05:00
Herbert Wolverson
3d9b52e627 More cleanup and logic improvements based on observation. In particular, flow cleanup is more accurate now, and we're a bit more aggressive in what RTT data we accept. 2024-03-12 08:57:29 -05:00
Herbert Wolverson
213a27498e Improve the flow cleanup logic. 2024-03-12 08:21:33 -05:00
Herbert Wolverson
79247e07f0 Fix a really silly mistake. I did NOT mean to always store the worst RTT we've ever seen. 2024-03-11 15:26:23 -05:00
Herbert Wolverson
d93726e538 Only report RTT for flows exceeding 4kbps - to eliminate noise from basically idle connections. 2024-03-11 15:03:47 -05:00
Herbert Wolverson
71fd1d558f Probable fix for building this on earlier kernel versions. 2024-03-11 14:04:41 -05:00
Herbert Wolverson
445cdcda81 Remove most of the Heimdall mode 1 path, cleaning up the execution path now that we have global flow tracking. 2024-03-11 13:27:42 -05:00
Herbert Wolverson
91a48bc275 Use the new flows system rather than Heimdall to display the circuits flows tab. Not complete yet - the backend is all there still. Also needs some tweaking on the hyperfocus for packet capture mode. 2024-03-11 12:24:18 -05:00
Herbert Wolverson
eb281b3edd Revert from a dashmap to a regular mutex for the flow container. Performance improved, and flow removal is less troublesome. 2024-03-11 11:20:27 -05:00
Herbert Wolverson
82ecd5eb17 Eliminate all time fetching calls except one, and store the result in the dissector. Minor speed improvement. 2024-03-10 21:11:55 -05:00
Herbert Wolverson
4a8be30c09 First update of the 'recently finished flows' mechanism for tracking the last 5 minutes of data. 2024-03-09 10:29:49 -06:00
Herbert Wolverson
79fa1d42a8 Revert per-ms timings for now. 2024-03-08 14:53:49 -06:00
Herbert Wolverson
1fb151aa1b Minimal protocol analysis beginnings. 2024-03-08 14:52:02 -06:00
Herbert Wolverson
5202f447dc Begin adding a little UI 2024-03-08 14:03:29 -06:00
Herbert Wolverson
6b384b2a5f Include ASN assessment in flows data. 2024-03-08 13:15:49 -06:00
Herbert Wolverson
b1cd8e5ed4 Setup initial stage of ASN analysis - downloading and loading the table, periodic upgrades. 2024-03-08 12:11:37 -06:00
Herbert Wolverson
9c7a9849ad Replace mutex locked vector with a dashmap for flow data, to ease sharing and update rather than replace cycle. 2024-03-08 09:32:15 -06:00
Herbert Wolverson
b7d43567ff Reworked the NetFlow code to batch packets into single submissions of up to 30 packets at a time. 2024-03-07 12:51:08 -06:00
Herbert Wolverson
04b0cd4246 Fix warning 2024-03-07 11:38:12 -06:00
Herbert Wolverson
33c1efdd2c IPv6 encoding matches the same pattern, needs testing. 2024-03-07 11:27:33 -06:00
Herbert Wolverson
34a2ec7b88 Refactor netflow v9 into readable code, and the IPv4 version is at least somewhat working now. 2024-03-07 11:19:02 -06:00
Herbert Wolverson
10c56f9353 This time with a theoretically valid header... 2024-03-05 14:54:57 -06:00