IntenseWebs/LibreQoS
3
0
Fork 0
mirror of https://github.com/LibreQoE/LibreQoS.git synced 2024-11-26 10:00:17 -06:00
Code Issues Packages Projects Releases Wiki Activity
d29588f6b3
LibreQoS/README.md
Robert Chacón d29588f6b3
Update README.md
2021-02-27 09:17:43 -07:00

9.9 KiB
Raw Blame History

LibreQoS

Banner LibreQoS is an application that allows you to apply fq_codel traffic shaping to hundreds of clients. Fq_codel is a Free and Open Source Active Queue Management algorithm that reduces bufferbloat, and can improve the quality of customer connections significantly. LibreQoS works with both IPv4 and IPv6. It apples hundreds of filter rules to direct customer traffic through individual fq_codel instances within an HTB (HTB+fq_codel). By utilizing hashing filters, thousands of rules can be applied with minimal impact on traffic throughput or CPU use. This is beta software - please do not deploy in production without testing to ensure compatability with your network architecture and design.

Who should use LibreQoS?

This software is intended for small Internet Service Providers. Large Internet Service Providers would likely benefit more from using commercially supported alternatives with better NMS/CRM integrations such as Preseem or Sensei.

How does fq_codel work?

Fq_codel distinguishes interactive flows of traffic (web browsing, audio streaming, VoIP, gaming) from bulk traffic (streaming video services, software updates). Interactive flows are prioritized to optimize their performance, while bulk traffic gets steady throughput and variable latency. The general reduction of connection latency offered by fq_codel is highly beneficial to end-users.

The impact of fq_codel on a 3000Mbps connection vs hard rate limiting — a 30x latency reduction.

“FQ_Codel provides great isolation... if you've got low-rate videoconferencing and low rate web traffic they never get dropped. A lot of issues with IW10 go away, because all the other traffic sees is the front of the queue. You don't know how big its window is, but you don't care because you are not affected by it. FQ_Codel increases utilization across your entire networking fabric, especially for bidirectional traffic... If we're sticking code into boxes to deploy codel, don't do that. Deploy fq_codel. It's just an across the board win.”

  • Van Jacobson | IETF 84 Talk

Features

  • Dual stack: client can be shaped by same qdisc for both IPv4 and IPv6
  • Up to 1000 clients (IPv4/IPv6)
  • HTB + fq_codel
  • Experimental support for CAKE (Common Applications Kept Enhanced)
  • TC filters split into groups through hashing filters to significantly increase throughput
  • Simple client management via csv file
  • Simple statistics - table shows top 20 subscribers by packet loss, with APs listed

Limitations

  • Tested up to 5Gbps/500Mbps asymmetrical throughput. Qdisc locking problem may require integrating xdp-cpumap-tc in future to increase bandwidth capacity.
  • Linux tc hash tables can only handle ~4000 rules each. This limits total possible clients to 1000 at this time. Eventually we will rework the code to allow for more clients by linking more hash tables.

Requirements

  • Edge and Core routers with MTU 1500 on links between them
    • If you use MPLS, you would terminate MPLS traffic at the core router. LibreQoS cannot decapsulate MPLS on its own.
  • OSPF primary link (low cost) through the server running LibreQoS
  • OSPF backup link Diagram

Server Requirements

  • VM or physical server
  • One management network interface, completely seperate from the traffic shaping interface NIC.
  • NIC supporting two virtual interfaces for traffic shaping (in/out), preferably SFP+ capable
  • Tested with Ubuntu Server 20.04.1+, which is recommended. Ubuntu Desktop not recommended as it uses NetworkManager instead of Netplan.
  • Python 3, PIP, and some modules
sudo apt update
sudo apt install python3-pip
python3 -m pip install ipaddress schedule prettytable
sudo python3 -m pip install ipaddress schedule prettytable

VM Performance

Memory use

On ProxMox VMs you can enable ballooning memory to allow freed up memory to be reclaimed by the hypervisor.

How to run LibreQoS

Enable Offloading in Guest VM

Performance can greatly benefit from enabling certrain hardware offloading inside the guest VM. If you're using a system that uses Netplan (e.g. Ubuntu) to configure the network then you can use a Netplan post-up script to configure offloading. You create a script in the following directory with a name prefixed by a number to indicate load order e.g. /usr/lib/networkd-dispatcher/routable.d/10-enable-offloading - which is executable and owned by root. e.g.

#!/bin/sh
ethtool --offload [interfaceA] rx on tx on gso on tso on lro on sg on gro on
ethtool --offload [interfaceB] rx on tx on gso on tso on lro on sg on gro on

Add a bridge between in/out interfaces

  • Add linux interface bridge br0 to the two dedicated interfaces
    • For example on Ubuntu Server 20.04 which uses NetPlan, you would add the following to the .yaml file in /etc/netplan/
bridges:
    br0:
      interfaces:
           - eth4
           - eth5

Then run

sudo netplan apply

Configure for your environment

Modify the ispConfig.py

  • Set pipeBandwidthCapacityMbps to match the bandwidth in Mbps of your network's (presumably symmetrical) WAN Internet connection
  • Set interfaceA to be the interface facing your edge router
  • Set interfaceB to be the interface facing your core router (or bridged internal network if your network is bridged)
  • Set
enableActualShellCommands = True

to allow the program to actually run the commands.

Configure for your clients

  • Modify the Shaper.csv file using your preferred spreadsheet editor (LibreOffice Calc, Excel, etc), following the template.
  • Either an IPv4 or IPv6 address is required for each entry, and Hostname is recommended.

One-Time Runs

One-time runs show the response from the terminal for each filter rule applied, and can be very helpful for debugging and to make sure it is correctly configured.

Cd to your preferred directory and download the latest release

git clone https://github.com/rchac/LibreQoS.git
  • Modify setting parameters in ispConfig.py to suit your environment
  • For one-time runs, use
sudo python3 ./LibreQoS.py

Running as a service

You can use the scheduled.py file to set the time of day you want the shapers to be refreshed at after the initial run. On linux distributions that use systemd, such as Ubuntu, add the following to /etc/systemd/system/LibreQoS.service, replacing "/home/$USER/LibreQoSDirectory" with wherever you downloaded LibreQoS to:

[Unit]
After=network.service

[Service]
WorkingDirectory=/home/$USER/LibreQoSDirectory
ExecStart=/usr/bin/python3 /home/$USER/LibreQoSDirectory/scheduled.py
Restart=always

[Install]
WantedBy=default.target

Then run

sudo systemctl daemon-reload
sudo systemctl start LibreQoS.service

You can check the status of the service using

sudo systemctl status LibreQoS.service

You can restart the service to refresh any changes you've made to the Shaper.csv file by doing

sudo systemctl restart LibreQoS.service

Statistics

python3 ./stats.py

Stats

Performance tips

Memory use

Generally memory use should be under 2GB. If for any reason memory exceeds that, please make sure memory ballooning is enabled on VM host, and try

sudo sh -c 'echo 1 >/proc/sys/vm/drop_caches'

Server CPU Recommendations

Donate

LibreQoS is based on fq_codel - an open source project led by Dave Taht, and contrinuted to by dozens of others. Without Dave's work, there would be no LibreQoS, Preseem, or Sensei. Please contribute to Dave's patreon here: https://www.patreon.com/dtaht

To support 1000 subscribers using proprietary wrappers for fq_codel would cost the average small ISP $6000 per year. If this application helps your network, please consider donating to Dave's patreon. Donating just $0.2/sub/month ($100/month for 500 subs) comes out to be 60% less than any proprietary solution, and you get direct access to our source code to tinker with LibreQoS and its HTB+fq_codel shaper to optimize your network's performance.

Special Thanks

Thank you to the hundreds of contributors to the fq_codel and cake projects, especially Dave Taht and Toke Høiland-Jørgensen. Thank you to Jesper Dangaard Brouer, Phil Sutter, Bert Hubert, Gregory Maxwell, Remco van Mook, Martijn van Oosterhout, Paul B Schroeder, and Jasper Spaans for contributing to the guides and documentation listed below.

References

License

Copyright (C) 2020-2021 Robert Chacón

LibreQoS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.

LibreQoS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with LibreQoS. If not, see http://www.gnu.org/licenses/.