IntenseWebs/LibreQoS
3
0
Fork 0
mirror of https://github.com/LibreQoE/LibreQoS.git synced 2025-02-25 18:55:32 -06:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #405 from LibreQoE/main

Browse Source 
Get long_term_stats up to date
This commit is contained in:
Robert Chacón
2023-10-17 13:47:21 -06:00
committed by GitHub
parent 2098baa38b 5e2be51115
commit 65909429b5
7 changed files with 93 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files

64
.readthedocs.yaml Normal file
View File

@@ -0,0 +1,64 @@
# Read the Docs configuration file for Sphinx projects
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
# Required
version: 2
# Set the OS, Python version and other tools you might need
build:
os: ubuntu-22.04
tools:
python: "3.11"
# You can also specify other tool versions:
# nodejs: "20"
# rust: "1.70"
# golang: "1.20"
# Build documentation in the "docs/" directory with Sphinx
sphinx:
configuration: conf.py
# You can configure Sphinx to use a different builder, for instance use the dirhtml builder for simpler URLs
# builder: "dirhtml"
# Fail on all warnings to avoid broken references
# fail_on_warning: true
# Optionally build your docs in additional formats such as PDF and ePub
# formats:
# - pdf
# - epub
# Optional but recommended, declare the Python requirements required
# to build your documentation
# See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
python:
install:
- requirements: requirements.txt

93
README.md
View File

@@ -15,7 +15,7 @@ Learn more about [Equinix Metal here](https://deploy.equinix.com/metal/).
Please support the continued development of LibreQoS by sponsoring us via [GitHub Sponsors](https://github.com/sponsors/LibreQoE) or [Patreon](https://patreon.com/libreqos).
## Documentation / Get Started
## Documentation / Get Started / System Requirements
[ReadTheDocs](https://libreqos.readthedocs.io/en/latest/)
@@ -24,94 +24,3 @@ Please support the continued development of LibreQoS by sponsoring us via [GitHu
Our Matrix chat channel is available at [https://matrix.to/#/#libreqos:matrix.org](https://matrix.to/#/#libreqos:matrix.org).
<img alt="LibreQoS" src="https://user-images.githubusercontent.com/22501920/223866474-603e1112-e2e6-4c67-93e4-44c17b1b7c43.png"></a>
## Features
### Flexible Hierarchical Shaping / Back-Haul Congestion Mitigation
<img src="https://raw.githubusercontent.com/LibreQoE/LibreQoS/main/docs/nestedHTB2.png" width="350"></img>
Starting in version v1.1+, operators can map their network hierarchy in LibreQoS. This enables both simple network hierarchies (Site>AP>Client) as well as much more complex ones (Site>Site>Micro-PoP>AP>Site>AP>Client). This can be used to ensure that a given site’s peak bandwidth will not exceed the capacity of its back-haul links (back-haul congestion control). Operators can support more users on the same network equipment with LibreQoS than with competing QoE solutions which only shape by AP and Client.
### CAKE
CAKE is the product of nearly a decade of development efforts to improve on fq\_codel. With the diffserv\_4 parameter enabled – CAKE groups traffic in to Bulk, Best Effort, Video, and Voice. This means that without having to fine-tune traffic priorities as you would with DPI products – CAKE automatically ensures your clients’ OS update downloads will not disrupt their zoom calls. It allows for multiple video conferences to operate on the same connection which might otherwise “fight” for upload bandwidth causing call disruptions. With work-from-home, remote learning, and tele-medicine becoming increasingly common – minimizing video call disruptions can save jobs, keep students engaged, and help ensure equitable access to medical care.
### XDP
Fast, multi-CPU queueing leveraging xdp-cpumap-tc and cpumap-pping. Currently tested in the real world past 11 Gbps (so far) with just 30% CPU use on a 16 core Intel Xeon Gold 6254. It's likely capable of 30Gbps or more.
### Graphing
You can graph bandwidth and TCP RTT by client and node (Site, AP, etc), using InfluxDB.
### CRM Integrations
## Server Recommendations
It is most cost-effective to buy a used server with specifications matching your unique requirements, as laid out in the System Requirements section below.
For those who do not have the time to do that, here are some off-the-shelf options to consider:
* 1 Gbps | [Supermicro SuperServer E100-9W-L](https://www.thinkmate.com/system/superserver-e100-9w-l)
* 10 Gbps | [Supermicro SuperServer 510T-ML (Choose E-2388G)](https://www.thinkmate.com/system/superserver-510t-ml)
* 20 Gbps | [Dell R450 Config](https://www.dell.com/en-us/shop/servers-storage-and-networking/poweredge-r450-rack-server/spd/poweredge-r450/pe_r450_15127_vi_vp?configurationid=a7663c54-6e4a-4c96-9a21-bc5a69d637ba)
The [AsRock 1U4LW-B6502L2T](https://www.thinkmate.com/system/asrock-1u4lw-b6502l2t/635744) can be a great lower-cost option as well.
## System Requirements
### VM or physical server
* For VMs, NIC passthrough is required for optimal throughput and latency (XDP vs generic XDP). Using Virtio / bridging is much slower than NIC passthrough. Virtio / bridging should not be used for large amounts of traffic.
### CPU
* 2 or more CPU cores
* A CPU with solid [single-thread performance](https://www.cpubenchmark.net/singleThread.html#server-thread) within your budget. Queuing is very CPU-intensive, and requires high single-thread performance.
Single-thread CPU performance will determine the max throughput of a single HTB (cpu core), and in turn, what max speed plan you can offer customers.
| Customer Max Plan | Passmark Single-Thread |
| --------------------| ------------------------ |
| 100 Mbps | 1000 |
| 250 Mbps | 1250 |
| 500 Mbps | 1500 |
| 1 Gbps | 1750 |
| 2 Gbps | 2000 |
| 3 Gbps | 2500 |
| 4 Gbps | 3000 |
Below is a table of approximate aggregate throughput capacity, assuming a a CPU with a [single thread](https://www.cpubenchmark.net/singleThread.html#server-thread) performance of 2700 or greater:
| Aggregate Throughput | CPU Cores |
| ------------------------| ------------- |
| 500 Mbps | 2 |
| 1 Gbps | 4 |
| 5 Gbps | 6 |
| 10 Gbps | 8 |
| 20 Gbps | 16 |
| 50 Gbps* | 32 |
(* Estimated)
So for example, an ISP delivering 1Gbps service plans with 10Gbps aggregate throughput would choose a CPU with a 2500+ single-thread score and 8 cores, such as the Intel Xeon E-2388G @ 3.20GHz.
### Memory
* Minimum RAM = 2 + (0.002 x Subscriber Count) GB
* Recommended RAM:
| Subscribers | RAM |
| ------------- | ------------- |
| 100 | 4 GB |
| 1,000 | 8 GB |
| 5,000 | 16 GB |
| 10,000* | 18 GB |
| 50,000* | 24 GB |
(* Estimated)
### Network Interface Requirements
* One management network interface completely separate from the traffic shaping interfaces. Usually this would be the Ethernet interface built in to the motherboard.
* Dedicated Network Interface Card for Shaping Interfaces
* NIC must have 2 or more interfaces for traffic shaping.
* NIC must have multiple TX/RX transmit queues. [Here's how to check from the command line](https://serverfault.com/questions/772380/how-to-tell-if-nic-has-multiqueue-enabled).
* Known supported cards:
* [NVIDIA Mellanox MCX512A-ACAT](https://www.fs.com/products/119649.html)
* NVIDIA Mellanox MCX416A-CCAT
* [Intel X710](https://www.fs.com/products/75600.html) * Note - possible i40e driver issue with XDP Redirect for high throughput 10G+
* Intel X520

2
conf.py
View File

@@ -7,7 +7,7 @@
# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
project = "LibreQoE"
copyright = "2023, LibreQoE"
copyright = "2023, LibreQoE, LLC"
author = "Zach Biles"
# -- General configuration ---------------------------------------------------

19
docs/SystemRequirements/Compute.md
View File

@@ -11,10 +11,11 @@ Single-thread CPU performance will determine the max throughput of a single HTB
| Customer Max Plan | Passmark Single-Thread |
| --------------------| ------------------------ |
| 100 Mbps | 1000 |
| 250 Mbps | 1500 |
| 500 Mbps | 2000 |
| 1 Gbps | 2500 |
| 250 Mbps | 1250 |
| 500 Mbps | 1500 |
| 1 Gbps | 2000 |
| 2 Gbps | 3000 |
| 4 Gbps | 4000 |
Below is a table of approximate aggregate throughput capacity, assuming a a CPU with a [single thread](https://www.cpubenchmark.net/singleThread.html#server-thread) performance of 2700 or greater:
@@ -48,8 +49,12 @@ So for example, an ISP delivering 1Gbps service plans with 10Gbps aggregate thro
### Server Recommendations
It is most cost-effective to buy a used server with specifications matching your unique requirements, as laid out in the System Requirements section below.
For those who do not have the time to do that, here are some off-the-shelf options to consider:
* 1 Gbps | [Supermicro SuperServer E100-9W-L](https://www.thinkmate.com/system/superserver-e100-9w-l)
* 10 Gbps | [Supermicro SuperServer 510T-ML (Choose E-2388G)](https://www.thinkmate.com/system/superserver-510t-ml)
* 20 Gbps | [Dell R450 Config](https://www.dell.com/en-us/shop/servers-storage-and-networking/poweredge-r450-rack-server/spd/poweredge-r450/pe_r450_15127_vi_vp?configurationid=a7663c54-6e4a-4c96-9a21-bc5a69d637ba)
The [AsRock 1U4LW-B6502L2T](https://www.thinkmate.com/system/asrock-1u4lw-b6502l2t/635744) can be a great lower-cost option as well.
| Aggregate | 100Mbps Plans | 1Gbps Plans | 4Gbps Plans |
| ------------- | ------------- | ------------- | ------------- |
| 1 Gbps Total | A | | |
| 10 Gbps Total | | B | C |
* A | [Supermicro SuperServer E100-9W-L](https://www.thinkmate.com/system/superserver-e100-9w-l)
* B | [Supermicro SuperServer 510T-ML](https://www.thinkmate.com/system/superserver-510t-ml) (Select E-2388G)
* C | [Supermicro AS-1015A-MT](https://store.supermicro.com/us_en/as-1015a-mt.html) (Ryzen 9 7700X, 2x16GB DDR5 4800MHz ECC, 1xSupermicro 10-Gigabit XL710+ X557)

2
docs/SystemRequirements/Networking.md
View File

@@ -6,5 +6,5 @@
* Known supported cards:
* [NVIDIA Mellanox MCX512A-ACAT](https://www.fs.com/products/119649.html)
* NVIDIA Mellanox MCX416A-CCAT
* [Intel X710](https://www.fs.com/products/75600.html)
* [Intel X710](https://www.fs.com/products/75600.html) * Note - possible i40e driver issue with XDP Redirect for high throughput 10G+
* Intel X520

2
docs/TechnicalDocs/integrations.md
View File

@@ -20,6 +20,8 @@ You have the option to run integrationUISP.py automatically on boot and every 30
First, set the relevant parameters for Splynx (splynx_api_key, splynx_api_secret, etc.) in ispConfig.py.
The Splynx Integration uses Basic authentication. For using this type of authentication, please make sure you enable [Unsecure access](https://splynx.docs.apiary.io/#introduction/authentication) in your Splynx API key settings. Also the Splynx API key should be granted access to the necessary permissions.
To test the Splynx Integration, use
```shell

12
docs/TechnicalDocs/troubleshooting.md
View File

@@ -22,3 +22,15 @@ This tends to show up when the MQ qdisc cannot be added correctly to the NIC int
The scheduler (scheduler.py) runs the InfluxDB integration within a try/except statement. If it fails to update InfluxDB, it will report "Failed to update bandwidth graphs".
To find the exact cause of the failure, please run ```python3 graphInfluxDB.py``` which will provde more specific errors.
### All customer IPs are listed under Unknown IPs, rather than Shaped Devices in GUI
```
cd /opt/libreqos/src
sudo systemctl stop lqos_scheduler
sudo python3 LibreQoS.py
```
The console output from running LibreQoS.py directly provides more specific errors regarding issues with ShapedDevices.csv and network.json
Once you have identified the error and fixed ShapedDevices.csv and/or Network.json, please then run
```sudo systemctl start lqos_scheduler```