Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.

Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

“AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

Permission is hereby granted, free of charge, to any person obtaining a copy of this software
and associated documentation files (the “Software”), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial
portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH
THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Error in "currentmodule" directive:
maximum 1 argument(s) allowed, 3 supplied.

Error in “currentmodule” directive:

No directive entry for "autoclass" in module "docutils.parsers.rst.languages.en".
Trying "autoclass" as canonical directive name.

No directive entry for “autoclass” in module “docutils.parsers.rst.languages.en”.
Trying “autoclass” as canonical directive name.

.. autoclass:: {{ objname }}

   {% block methods %}
   .. automethod:: __init__

   {% if methods %}
   .. rubric:: {{ _('Methods') }}

   .. autosummary::
   {% for item in methods %}
      ~{{ name }}.{{ item }}
   {%- endfor %}
   {% endif %}
   {% endblock %}

   {% block attributes %}
   {% if attributes %}
   .. rubric:: {{ _('Attributes') }}

   {% for item in attributes %}
   {% endif %}

No directive entry for "automodule" in module "docutils.parsers.rst.languages.en".
Trying "automodule" as canonical directive name.

No directive entry for “automodule” in module “docutils.parsers.rst.languages.en”.
Trying “automodule” as canonical directive name.

.. automodule:: {{ fullname }}

   .. rubric:: {{ _('Module Attributes') }}

      {{ item }}
   {% endif %}

   {% block functions %}
   {% if functions %}
   .. rubric:: {{ _('Functions') }}

   {% for item in functions %}
      {{ item }}
   {% endif %}

   {% block classes %}
   {% if classes %}
   .. rubric:: {{ _('Classes') }}

   {% for item in classes %}
      {{ item }}
   {% endif %}

   {% block exceptions %}
   {% if exceptions %}
   .. rubric:: {{ _('Exceptions') }}

   {% for item in exceptions %}
      {{ item }}
   {% endif %}

No directive entry for "autosummary" in module "docutils.parsers.rst.languages.en".
Trying "autosummary" as canonical directive name.

No directive entry for “autosummary” in module “docutils.parsers.rst.languages.en”.
Trying “autosummary” as canonical directive name.

LibreQoE documentation master file, created by
sphinx-quickstart on Fri May 19 10:37:27 2023.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.

Welcome to the LibreQoS documentation!
====================================

We as members, contributors, and leaders pledge to make participation in our
community a harassment-free experience for everyone, regardless of age, body
size, visible or invisible disability, ethnicity, sex characteristics, gender
identity and expression, level of experience, education, socio-economic status,
nationality, personal appearance, race, religion, or sexual identity
and orientation.

We pledge to act and interact in ways that contribute to an open, welcoming,
diverse, inclusive, and healthy community.

Examples of behavior that contributes to a positive environment for our
community include:

Being respectful of differing opinions, viewpoints, and experiences

Accepting responsibility and apologizing to those affected by our mistakes,
and learning from the experience

Focusing on what is best not just for us as individuals, but for the
overall community

The use of sexualized language or imagery, and sexual attention or
advances of any kind

Trolling, insulting or derogatory comments, and personal or political attacks

Publishing others' private information, such as a physical or email
address, without their explicit permission

Publishing others’ private information, such as a physical or email

Other conduct which could reasonably be considered inappropriate in a
professional setting

Community leaders are responsible for clarifying and enforcing our standards of
acceptable behavior and will take appropriate and fair corrective action in
response to any behavior that they deem inappropriate, threatening, offensive,
or harmful.

Community leaders have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are
not aligned to this Code of Conduct, and will communicate reasons for moderation
decisions when appropriate.

This Code of Conduct applies within all community spaces, and also applies when
an individual is officially representing the community in public spaces.
Examples of representing our community include using an official e-mail address,
posting via an official social media account, or acting as an appointed
representative at an online or offline event.

Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported to the community leaders responsible for enforcement at
robert@libreqos.io.
All complaints will be reviewed and investigated promptly and fairly.

All community leaders are obligated to respect the privacy and security of the
reporter of any incident.

Community leaders will follow these Community Impact Guidelines in determining
the consequences for any action they deem in violation of this Code of Conduct:

**Community Impact**: Use of inappropriate language or other behavior deemed
unprofessional or unwelcome in the community.

**Consequence**: A private, written warning from community leaders, providing
clarity around the nature of the violation and an explanation of why the
behavior was inappropriate. A public apology may be requested.

**Community Impact**: A violation through a single incident or series
of actions.

**Consequence**: A warning with consequences for continued behavior. No
interaction with the people involved, including unsolicited interaction with
those enforcing the Code of Conduct, for a specified period of time. This
includes avoiding interactions in community spaces as well as external channels
like social media. Violating these terms may lead to a temporary or
permanent ban.

**Community Impact**: A serious violation of community standards, including
sustained inappropriate behavior.

**Consequence**: A temporary ban from any sort of interaction or public
communication with the community for a specified period of time. No public or
private interaction with the people involved, including unsolicited interaction
with those enforcing the Code of Conduct, is allowed during this period.
Violating these terms may lead to a permanent ban.

**Community Impact**: Demonstrating a pattern of violation of community
standards, including sustained inappropriate behavior,  harassment of an
individual, or aggression toward or disparagement of classes of individuals.

**Consequence**: A permanent ban from any sort of public interaction within
the community.

This Code of Conduct is adapted from the [Contributor Covenant][homepage],
version 2.0, available at
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.

Community Impact Guidelines were inspired by [Mozilla's code of conduct
enforcement ladder](https://github.com/mozilla/diversity).

For answers to common questions about this code of conduct, see the FAQ at
https://www.contributor-covenant.org/faq. Translations are available at
https://www.contributor-covenant.org/translations.

modification, are permitted provided that the following conditions are met:

Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”

Plugin for creating block-level custom containers for [markdown-it](https://github.com/markdown-it/markdown-it) markdown parser.

.... and specify how they should be rendered. If no renderer defined, `<div>` with
container name class will be created:

…. and specify how they should be rendered. If no renderer defined, 

Markup is the same as for [fenced code blocks](http://spec.commonmark.org/0.18/#fenced-code-blocks).
Difference is, that marker use another character and content is rendered as markdown markup.

$ npm install markdown-it-container --save
$ bower install markdown-it-container --save

var md = require('markdown-it')()
            .use(require('markdown-it-container'), name [, options]);

__validate__ - optional, function to validate tail after opening marker, should
return `true` on success.

 - optional, function to validate tail after opening marker, should

__render__ - optional, renderer function for opening/closing tokens.

var md = require('markdown-it')();

md.use(require('markdown-it-container'), 'spoiler', {

  validate: function(params) {
    return params.trim().match(/^spoiler\s+(.*)$/);
  },

  render: function (tokens, idx) {
    var m = tokens[idx].info.trim().match(/^spoiler\s+(.*)$/);

    if (tokens[idx].nesting === 1) {
      // opening tag
      return '<details><summary>' + md.utils.escapeHtml(m[1]) + '</summary>\n';

    } else {
      // closing tag
      return '</details>\n';
    }
  }
});

console.log(md.render('::: spoiler click me\n*content*\n:::\n'));

// Output:
//
// <details><summary>click me</summary>
// <p><em>content</em></p>
// </details>

Definition list (`<dl>`) tag plugin for [markdown-it](https://github.com/markdown-it/markdown-it) markdown parser.

Syntax is based on [pandoc definition lists](http://johnmacfarlane.net/pandoc/README.html#definition-lists).

npm install markdown-it-deflist --save
bower install markdown-it-deflist --save

            .use(require('markdown-it-deflist'));

md.render(/*...*/);

_Differences in browser._ If you load script directly into the page, without
package system, module will add itself globally as `window.markdownitDeflist`.

Add TeX math equations to your Markdown documents rendered by [markdown-it](https://github.com/markdown-it/markdown-it) parser. [KaTeX](https://github.com/Khan/KaTeX) is used as a fast math renderer.

Simplify the process of authoring markdown documents containing math formulas.
This extension is a comfortable tool for scientists, engineers and students with markdown as their first choice document format.

View a [test table](https://goessner.github.io/markdown-it-texmath/index.html).

[try it out ...](https://goessner.github.io/markdown-it-texmath/markdown-it-texmath-demo.html)

Install the extension. Verify having `markdown-it` and `katex` already installed .

let kt = require('katex'),
    tm = require('markdown-it-texmath').use(kt),
    md = require('markdown-it')().use(tm,{delimiters:'dollars',macros:{"\\RR": "\\mathbb{R}"}});

md.render('Euler\'s identity \(e^{i\pi}+1=0\) is a beautiful formula in $\\RR 2$.')

<html>
<head>
  <meta charset='utf-8'>
  <link rel="stylesheet" href="katex.min.css">
  <link rel="stylesheet" href="texmath.css">
  <script src="markdown-it.min.js"></script>
  <script src="katex.min.js"></script>
  <script src="texmath.js"></script>
</head>
<body>
  <div id="out"></div>
  <script>
    let md;
    document.addEventListener("DOMContentLoaded", () => {
        const tm = texmath.use(katex);
        md = markdownit().use(tm,{delimiters:'dollars',macros:{"\\RR": "\\mathbb{R}"}});
        out.innerHTML = md.render('Euler\'s identity $e^{i\pi}+1=0$ is a beautiful formula in //RR 2.');
    })
  </script>
</body>
</html>

[`markdown-it`](https://github.com/markdown-it/markdown-it): Markdown parser done right. Fast and easy to extend.

[`katex`](https://github.com/Khan/KaTeX): This is where credits for fast rendering TeX math in HTML go to.

: This is where credits for fast rendering TeX math in HTML go to.

__`markdown-it-texmath` with React Native does not work, why ?__

`markdown-it-texmath` is using regular expressions with `y` [(sticky) property](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/sticky) and cannot avoid this. The use of the `y` flag in regular expressions means the plugin is not compatible with React Native (which as of now doesn't support it and throws an error `Invalid flags supplied to RegExp constructor`).

 flag in regular expressions means the plugin is not compatible with React Native (which as of now doesn’t support it and throws an error 

Add support for [Julia Markdown](https://docs.julialang.org/en/v1/stdlib/Markdown/) on [request](https://github.com/goessner/markdown-it-texmath/issues/15).

Remove [rendering bug with brackets delimiters](https://github.com/goessner/markdown-it-texmath/issues/9).

Remove pathological [bug within blockquotes](https://github.com/goessner/mdmath/issues/50).

Add support for Tex macros (https://katex.org/docs/supported.html#macros) .

Bug with [brackets delimiters](https://github.com/goessner/markdown-it-texmath/issues/9) .

Fatal blockquote bug investigated. Implemented workaround to vscode bug, which has finally gone with vscode 1.26.0 .

Modifying the `block` mode regular expression with `gitlab` delimiters, so removing the `newline` bug.

`markdown-it-texmath` is licensed under the [MIT License](./license.txt)

This is **draft 1** of the contributing guide. Comments/edits appreciated.

So your interested in contributing to LibreQoS! Awesome! Feel free to pitch in with whatever interests you, and we'll be happy to help.

So your interested in contributing to LibreQoS! Awesome! Feel free to pitch in with whatever interests you, and we’ll be happy to help.

Check out our [Code of Conduct](https://github.com/LibreQoE/LibreQoS/blob/main/.github/CODE_OF_CONDUCT.md), we'd like to keep this a happy, constructive place. Also, please join the chat on [Matrix](https://app.element.io/#/room/#libreqos:matrix.org)---the core developers hang out there, and will be happy to help you.

, we’d like to keep this a happy, constructive place. Also, please join the chat on 

—the core developers hang out there, and will be happy to help you.

We can check that an issue isn't already being worked on. There's nothing more frustrating than working hard on an issue only to discover that someone else already fixed it!

We can check that an issue isn’t already being worked on. There’s nothing more frustrating than working hard on an issue only to discover that someone else already fixed it!

**Battle-Testing LibreQos**: use the software, let us know what does/doesn't work!

**Let us Know Your Thoughts**: Hop on the chat or our discussions page and let us know what you're thinking about.

: Hop on the chat or our discussions page and let us know what you’re thinking about.

**Finding Bugs** - something doesn't work for you? We can't fix it if you don't tell us about it.

 - something doesn’t work for you? We can’t fix it if you don’t tell us about it.

**Fixing Bugs** and **Adding Features**: See the "Development Guide", below.

**Teaching Others**: if LibreQoS is working well for you, you can help others benefit by sharing your experience. See someone struggling? Feel free to pitch in and help them.

: if LibreQoS is working well for you, you can help others benefit by sharing your experience. See someone struggling? Feel free to pitch in and help them.

LibreQos strive to create an open, friendly environment. If you have an idea for how to help, let us know.

This section contains some advice to help get you started writing code for LibreQoS.

`lqosd` is probably the most important part of the Rust system. It loads the eBPF system that provides bridging and traffic shaping control, runs the "bus" for other systems to communicate, and gathers information directly from the eBPF and kernel systems. It has several sub-crates:

 is probably the most important part of the Rust system. It loads the eBPF system that provides bridging and traffic shaping control, runs the “bus” for other systems to communicate, and gathers information directly from the eBPF and kernel systems. It has several sub-crates:

`lqos_heimdall` handles all of the packet sniffing, flow tracking and `libcap` compatible packet data.

`lqos_queue_tracker` maintains statistics for Linux `tc` shaping queues, particularly Cake.

`lqos_node_manager` provides a per-node management web interface. It gathers most of its data from `lqosd` via the bus.

 provides a per-node management web interface. It gathers most of its data from 

`lqusers` provides a CLI interface for managing authentication to the node manager.

 provides a CLI interface for managing authentication to the node manager.

`lqtop` provides a quick and easy way to see what the shaper is doing from the local management console.

 provides a quick and easy way to see what the shaper is doing from the local management console.

`xdp_iphash_to_cpu_cmdline` (the name is inherited from previous projects) provides a command-line interface to map IP subnets to TC and CPU handles.

 (the name is inherited from previous projects) provides a command-line interface to map IP subnets to TC and CPU handles.

`xdp_pping` provides a CLI tool to give a quick summary of TCP RTT times, grouped by TC handle.

 provides a CLI tool to give a quick summary of TCP RTT times, grouped by TC handle.

`lqos_bus` provides a local-only (never leaving the shaper node) inter-process communication system. It's used by programs that need to ask `lqosd` to do something, or retrieve information from it.

 provides a local-only (never leaving the shaper node) inter-process communication system. It’s used by programs that need to ask 

The `lqos_bus` crate also acts as a repository for shared data structures when data is passed between portions of the program.

 crate also acts as a repository for shared data structures when data is passed between portions of the program.

`lqos_config` manages Rust integration with the `ispConfig.py` configuration file, and the `/etc/lqos.conf` file. It is designed as a helper for other systems to quickly access configuration parameters.

 file. It is designed as a helper for other systems to quickly access configuration parameters.

`lqos_python` compiles into a Python loadable library, providing a convenient interface to Rust code from the Python portions of the program.

 compiles into a Python loadable library, providing a convenient interface to Rust code from the Python portions of the program.

`lqos_setup` provides a text-based initial setup system for users who install LibreQoS via `apt-get`.

 provides a text-based initial setup system for users who install LibreQoS via 

`lqos_utils` provides a grab-bag of handy functions we've found useful elsewhere in the system.

 provides a grab-bag of handy functions we’ve found useful elsewhere in the system.

`LibreQoS.py` maps all circuits to TC handles and gets the shaper system running.

 maps all circuits to TC handles and gets the shaper system running.

`integrationX.py` provide integrations with UISP, Spylnx and other CRM tools.

LibreQoS is a free/open source fair queueing system. It's designed for ISPs, but can be useful elsewhere. Primary goals:

LibreQoS is a free/open source fair queueing system. It’s designed for ISPs, but can be useful elsewhere. Primary goals:

Provide fair-queueing to help maximize the use of the Internet resources you have.

Don't alter user traffic (for example by lowering the quality of their streaming video).

Don’t alter user traffic (for example by lowering the quality of their streaming video).

Provide excellent support tools to help keep your ISP running smoothly.

Visualize data to facilitate moving the state-of-the-art in fair queueing forwards.

When we use the word "agile", we don't mean the heavily formalized process with scrums, kanban boards and similar. We mean lightweight, fast moving, and adhering to useful guides like "Code Complete" and "The Pragmatic Programmer". Not getting bogged down in heavy process.

When we use the word “agile”, we don’t mean the heavily formalized process with scrums, kanban boards and similar. We mean lightweight, fast moving, and adhering to useful guides like “Code Complete” and “The Pragmatic Programmer”. Not getting bogged down in heavy process.

Other community members review and comment on your change in an informal manner.

Once consensus is reached, we'll merge to the `develop` branch (or a branch parented from `develop` if it's a big change) and test it on our server resources at Equinix.

 if it’s a big change) and test it on our server resources at Equinix.

If you need/want a complicated change, please get in touch with us on the Matrix chat first. We'd like to avoid duplicating effort and wasting anyone's time. We'll be happy to offer advice and guidance. Then the process is similar:

If you need/want a complicated change, please get in touch with us on the Matrix chat first. We’d like to avoid duplicating effort and wasting anyone’s time. We’ll be happy to offer advice and guidance. Then the process is similar:

Create a PR (targeting `develop`). If you'd like interim feedback, create a "draft PR" and we can help test your branch before the PR finalizes.

). If you’d like interim feedback, create a “draft PR” and we can help test your branch before the PR finalizes.

`develop` - Parented from `main` and rebased on release. Nothing gets directly comitted into `develop`, it's the parent tree for ongoing development work.

`my_feature` - if you're working on a feature, your feature branch goes here - with `develop` as the parent. PRs---once the feature is ready for inclusion---should be targeted at `develop`.

 - if you’re working on a feature, your feature branch goes here - with 

 as the parent. PRs—once the feature is ready for inclusion—should be targeted at 

`issue_xxx_name` - if you're working on a bugfix, work on it in a branch here. When the issue resolution is complete, PRs should be targeted at `develop`.

 - if you’re working on a bugfix, work on it in a branch here. When the issue resolution is complete, PRs should be targeted at 

`hotfix_xxx` - If an emergency occurs and you have to push a fix to `main` in a hurry, hotfix branches can be parented from `main`. The resulting PR should target `main`. Let the developers know that they need to rebase `develop`.

The goal is for `main` to *always* be safe to clone and run, without surprises.

Use `cargo fmt` to format your code. We've got a customized format setup in place.

 to format your code. We’ve got a customized format setup in place.

Check that you aren't including any dependencies that incompatible with our license---GPL v2.

Check that you aren’t including any dependencies that incompatible with our license—GPL v2.

Look through the `Cargo.toml` files (or run `cargo tree`) and try to prefer using a dependency we use elsewhere.

Don't use short, incomprehensible names in any API or function accessible from outside. You don't save any RAM by naming your variable `sz` instead of `size`---you just make it harder for anyone reading the code.

Don’t use short, incomprehensible names in any API or function accessible from outside. You don’t save any RAM by naming your variable 

It's fine to use `i` and similar for internal counting iterators. Try to use meaningful names for everything else.

 and similar for internal counting iterators. Try to use meaningful names for everything else.

Prefer functional/iterative code to imperative. Sometimes you need a `for` loop, but if it can be replaced with an `iter`, `map` and `fold` it will both compile into faster code and be less error prone.

Prefer functional/iterative code to imperative. Sometimes you need a 

It's better to have lots of small functions than one big one. Rust is really good at inlining, and it's much easier to understand short functions. It's also easier to test small functions.

It’s better to have lots of small functions than one big one. Rust is really good at inlining, and it’s much easier to understand short functions. It’s also easier to test small functions.

If you have to override a Clippy warning, add a comment explaining why you did so.

Functions accessible from other crates should use the RustDoc standard for documentation.

If you fix an issue, and it's testable: add a unit test to check that we don't regress and suffer from that bug again.

If you fix an issue, and it’s testable: add a unit test to check that we don’t regress and suffer from that bug again.

Use `thiserror` to emit readable error messages from your functions.

It's fine to use `?` and `anyhow` inside function chains; prefer `result.map_err` to transform your errors into your own errors whenever it's possible that an error can be returned from a function accessible beyond the immediate crate.

 to transform your errors into your own errors whenever it’s possible that an error can be returned from a function accessible beyond the immediate crate.

Issue a `log::error!` or `log::warn!` messaage when an error occurs. Don't trust the callee to do it for you. It's better to have duplicate error messages than none at all.

 messaage when an error occurs. Don’t trust the callee to do it for you. It’s better to have duplicate error messages than none at all.

LibreQoS is a Quality of Experience (QoE) Smart Queue Management (SQM) system designed for Internet Service Providers to optimize the flow of their network traffic and thus reduce bufferbloat, keep the network responsive, and improve the end-user experience.

Servers running LibreQoS can shape traffic for many thousands of customers.

Special thanks to Equinix for providing server resources to support the development of LibreQoS.
Learn more about [Equinix Metal here](https://deploy.equinix.com/metal/).

Please support the continued development of LibreQoS by sponsoring us via [GitHub Sponsors](https://github.com/sponsors/LibreQoE) or [Patreon](https://patreon.com/libreqos).

Please support the continued development of LibreQoS by sponsoring us via 

Our Matrix chat channel is available at [https://matrix.to/#/#libreqos:matrix.org](https://matrix.to/#/#libreqos:matrix.org).

Starting in version v1.1+, operators can map their network hierarchy in LibreQoS. This enables both simple network hierarchies (Site>AP>Client) as well as much more complex ones (Site>Site>Micro-PoP>AP>Site>AP>Client). This can be used to ensure that a given site’s peak bandwidth will not exceed the capacity of its back-haul links (back-haul congestion control). Operators can support more users on the same network equipment with LibreQoS than with competing QoE solutions which only shape by AP and Client.

CAKE is the product of nearly a decade of development efforts to improve on fq\_codel. With the diffserv\_4 parameter enabled – CAKE groups traffic in to Bulk, Best Effort, Video, and Voice. This means that without having to fine-tune traffic priorities as you would with DPI products – CAKE automatically ensures your clients’ OS update downloads will not disrupt their zoom calls. It allows for multiple video conferences to operate on the same connection which might otherwise “fight” for upload bandwidth causing call disruptions. With work-from-home, remote learning, and tele-medicine becoming increasingly common – minimizing video call disruptions can save jobs, keep students engaged, and help ensure equitable access to medical care.

CAKE is the product of nearly a decade of development efforts to improve on fq_codel. With the diffserv_4 parameter enabled – CAKE groups traffic in to Bulk, Best Effort, Video, and Voice. This means that without having to fine-tune traffic priorities as you would with DPI products – CAKE automatically ensures your clients’ OS update downloads will not disrupt their zoom calls. It allows for multiple video conferences to operate on the same connection which might otherwise “fight” for upload bandwidth causing call disruptions. With work-from-home, remote learning, and tele-medicine becoming increasingly common – minimizing video call disruptions can save jobs, keep students engaged, and help ensure equitable access to medical care.

Fast, multi-CPU queueing leveraging xdp-cpumap-tc and cpumap-pping. Currently tested in the real world past 11 Gbps (so far) with just 30% CPU use on a 16 core Intel Xeon Gold 6254. It's likely capable of 30Gbps or more.

Fast, multi-CPU queueing leveraging xdp-cpumap-tc and cpumap-pping. Currently tested in the real world past 11 Gbps (so far) with just 30% CPU use on a 16 core Intel Xeon Gold 6254. It’s likely capable of 30Gbps or more.

You can graph bandwidth and TCP RTT by client and node (Site, AP, etc), using InfluxDB.

Disable hyperthreading on the BIOS/UEFI of your host system. Hyperthreaading is also known as Simultaneous Multi Threading (SMT) on AMD systems. Disabling this is very important for optimal performance of the XDP cpumap filtering and, in turn, throughput and latency.

For AMD systems, you will have to navigate the settings to find the "SMT Control" setting. Usually it is under something like ```Advanced -> AMD CBS -> CPU Common Options -> Thread Enablement -> SMT Control``` Once you find it, switch to "Disabled" or "Off"

For AMD systems, you will have to navigate the settings to find the “SMT Control” setting. Usually it is under something like 

Advanced -> AMD CBS -> CPU Common Options -> Thread Enablement -> SMT Control

For Intel systems, you will also have to navigate the settings to find the "hyperthrading" toggle option. On HP servers it's under ```System Configuration > BIOS/Platform Configuration (RBSU) > Processor Options > Intel (R) Hyperthreading Options.```

For Intel systems, you will also have to navigate the settings to find the “hyperthrading” toggle option. On HP servers it’s under 

System Configuration > BIOS/Platform Configuration (RBSU) > Processor Options > Intel (R) Hyperthreading Options.

We recommend Ubuntu Server because its kernel version tends to track closely with the mainline Linux releases. Our current documentation assumes Ubuntu Server. To run LibreQoS v1.4, Linux kernel 5.11 or greater is required, as 5.11 includes some important XDP patches. Ubuntu Server 22.04 uses kernel 5.13, which meets that requirement.

You can download Ubuntu Server 22.04 from <a href="https://ubuntu.com/download/server">https://ubuntu.com/download/server</a>.

If you use a Mellanox network card, the Ubuntu Server installer will ask you whether to install the mellanox/intel NIC drivers. Check the box to confirm. This extra driver is important.

On the Networking settings step, it is recommended to assign a static IP address to the management NIC.

Ensure SSH server is enabled so you can more easily log into the server later.

You can use scp or sftp to access files from your LibreQoS server for easier file editing. Here's how to access via scp or sftp using an [Ubuntu](https://www.addictivetips.com/ubuntu-linux-tips/sftp-server-ubuntu/) or [Windows](https://winscp.net/eng/index.php) machine.

You can use scp or sftp to access files from your LibreQoS server for easier file editing. Here’s how to access via scp or sftp using an 

There are two options for the bridge to pass data through your two interfaces:

The Bifrost Bridge is faster and generally recommended, but may not work perfectly in a VM setup using virtualized NICs.
To use the Bifrost bridge, skip the regular Linux bridge section below, and be sure to enable Bifrost/XDP in lqos.conf a few sections below.

From the Ubuntu VM, create a linux interface bridge - br0 - with the two shaping interfaces.
Find your existing .yaml file in /etc/netplan/ with

Editing the .yaml file, we need to define the shaping interfaces (here, ens19 and ens20) and add the bridge with those two interfaces. Assuming your interfaces are ens18, ens19, and ens20, here is what your file might look like:

# This is the network config written by 'subiquity'
network:
  ethernets:
    ens18:
      addresses:
      - 10.0.0.12/24
      routes:
      - to: default
        via: 10.0.0.1
      nameservers:
        addresses:
        - 1.1.1.1
        - 8.8.8.8
        search: []
    ens19:
      dhcp4: no
    ens20:
      dhcp4: no
  version: 2
  bridges:
    br0:
      interfaces:
        - ens19
        - ens20

Make sure to replace 10.0.0.12/24 with your LibreQoS VM's address and subnet, and to replace the default gateway 10.0.0.1 with whatever your default gateway is.

Make sure to replace 10.0.0.12/24 with your LibreQoS VM’s address and subnet, and to replace the default gateway 10.0.0.1 with whatever your default gateway is.

InfluxDB allows you to track long-term stats beyond what lqos_node_manager can so far.

To install InfluxDB 2.x., follow the steps at [https://portal.influxdata.com/downloads/](https://portal.influxdata.com/downloads/).

For high throughput networks (5+ Gbps) you will likely want to install InfluxDB to a separate machine or VM from that of the LibreQoS server to avoid CPU load.

Check to ensure InfluxDB is running properly. This command should show "Active: active" with green dot.

Check to ensure InfluxDB is running properly. This command should show “Active: active” with green dot.

Call the bucket `libreqos` (all lowercase).<br>
Have it store as many days of data as you prefer. 7 days is standard.<>
Import Dashboard

Boards > Create Dashboard > Import Dashboard
Then upload the file [influxDBdashboardTemplate.json](https://github.com/rchac/LibreQoS/blob/main/src/influxDBdashboardTemplate.json) to InfluxDB.

[Generate an InfluxDB Token](https://docs.influxdata.com/influxdb/cloud/security/tokens/create-token/). It will be added to ispConfig.py in the following steps.

LibreQoS placed inline in network, usually between an edge router (NAT, firewall) and core router (distribution to sites across network).

If you use NAT/CG-NAT, place LibreQoS inline south of where NAT is applied, as LibreQoS needs to shape internal addresses (100.64.0.0/12) not public post-NAT IPs.

Edge and Core routers should have 1500 MTU on links between them

If you use MPLS, you would terminate MPLS traffic at the core router. LibreQoS cannot decapsulate MPLS on its own.

OSPF primary link (low cost) through the server running LibreQoS

Is it possible to use LibreQoS in-line without a core router, but that setup requires depending on STP instead of OSPF, which can cause issues. Such configurations are not officially supported.

You must have one of these:
    *single NIC with two interfaces,
    *two NICs with single interface,
    *2x VLANs interface (using one or two NICs).

LibreQoS requires NICs to have 2 or more RX/TX queues and XDP support. While many cards theoretically meet these requirements, less commonly used cards tend to have unreported driver bugs which impede XDP functionality and make them unusable for our purposes. At this time we recommend the Intel x520, Intel x710, and Nvidia (ConnectX-5 or newer) NICs. We cannot guarantee compatibility with other cards.

If you need to create an integration for your network, we've tried to give you the tools you need. We currently ship integrations for UISP and Spylnx. We'd love to include more.

If you need to create an integration for your network, we’ve tried to give you the tools you need. We currently ship integrations for UISP and Spylnx. We’d love to include more.

LibreQoS enforces customer bandwidth limits, and applies CAKE-based optimizations at several levels:

Per-user Cake flows are created. These require the maximum bandwidth permitted per customer.

Customers can have more than one device that share a pool of bandwidth. Customers are grouped into "circuits"

Customers can have more than one device that share a pool of bandwidth. Customers are grouped into “circuits”

*Optional* Access points can have a speed limit/queue, applied to all customers associated with the access point.

 Access points can have a speed limit/queue, applied to all customers associated with the access point.

*Optional* Sites can contain access points, and apply a speed limit/queue to all access points (and associated circuits).

 Sites can contain access points, and apply a speed limit/queue to all access points (and associated circuits).

*Optional* Sites can be nested beneath other sites and access point, providing for a queue hierarchy that represents physical limitations of backhaul connections.

 Sites can be nested beneath other sites and access point, providing for a queue hierarchy that represents physical limitations of backhaul connections.

Additionally, you might grow to have more than one shaper - and need to express your network topology from the perspective of different parts of your network. (For example, if *Site A* and *Site B* both have Internet connections - you want to generate an efficient topology for both sites. It's helpful if you can derive this from the same overall topology)

Additionally, you might grow to have more than one shaper - and need to express your network topology from the perspective of different parts of your network. (For example, if 

 both have Internet connections - you want to generate an efficient topology for both sites. It’s helpful if you can derive this from the same overall topology)

LibreQoS's network modeling accomplishes this by modeling your network as a *graph*: a series of interconnected nodes, each featuring a "parent". Any "node" (entry) in the graph can be turned into a "root" node, allowing you to generate the `network.json` and `ShapedDevices.csv` files required to manage your customers from the perspective of that root node.

LibreQoS’s network modeling accomplishes this by modeling your network as a 

: a series of interconnected nodes, each featuring a “parent”. Any “node” (entry) in the graph can be turned into a “root” node, allowing you to generate the 

 files required to manage your customers from the perspective of that root node.

The simplest form of integration produces a "flat" network. This is the highest performance model in terms of raw throughput, but lacks the ability to provide shaping at the access point or site level: every customer site is parented directly off the root.

The simplest form of integration produces a “flat” network. This is the highest performance model in terms of raw throughput, but lacks the ability to provide shaping at the access point or site level: every customer site is parented directly off the root.

For an integration, it's recommended that you fetch the customer/device data from your management system rather than type them all in Python.

For an integration, it’s recommended that you fetch the customer/device data from your management system rather than type them all in Python.

A flat integration is relatively simple. Start by importing the common API:

from integrationCommon import isIpv4Permitted, fixSubnet, NetworkGraph, NetworkNode, NodeType

Then create an empty network graph (it will grow to represent your network):

Once you have your `NetworkGraph` object, you start adding customers and devices. Customers may have any number of devices. You can add a single customer with one device as follows:

 object, you start adding customers and devices. Customers may have any number of devices. You can add a single customer with one device as follows:

# Add the customer
customer = NetworkNode(
    id="Unique Customer ID",
    displayName="The Doe Family",
    type=NodeType.client,
    download=100, # Download is in Mbit/second
    upload=20, # Upload is in Mbit/second
    address="1 My Road, My City, My State")
net.addRawNode(customer) # Insert the customer ID

# Give them a device
device = NetworkNode(
    id="Unique Device ID", 
    displayName="Doe Family CPE",
    parentId="Unique Customer ID", # must match the customer's ID
    type=NodeType.device, 
    ipv4=["100.64.1.5/32"], # As many as you need, express networks as the network ID - e.g. 192.168.100.0/24
    ipv6=["feed:beef::12/64"], # Same again. May be [] for none.
    mac="00:00:5e:00:53:af"
)
net.addRawNode(device)

If the customer has multiple devices, you can add as many as you want - with `ParentId` continuing to match the parent customer's `id`.

If the customer has multiple devices, you can add as many as you want - with 

Once you have entered all of your customers, you can finish the integration:

net.prepareTree() # This is required, and builds parent-child relationships.
net.createNetworkJson() # Create `network.json`
net.createShapedDevices() # Create the `ShapedDevices.csv` file.

Creating a full hierarchy (with as many levels as you want) uses a similar strategy to flat networks---we recommend that you start by reading the "flat shaping" section above.

Creating a full hierarchy (with as many levels as you want) uses a similar strategy to flat networks—we recommend that you start by reading the “flat shaping” section above.

Now you can start to insert sites and access points. Sites and access points are inserted like customer or device nodes: they have a unique ID, and a `ParentId`. Customers can then use a `ParentId` of the site or access point beneath which they should be located.

Now you can start to insert sites and access points. Sites and access points are inserted like customer or device nodes: they have a unique ID, and a 

 of the site or access point beneath which they should be located.

For example, let's create `Site_1` and `Site_2` - at the top of the tree:

net.addRawNode(NetworkNode(id="Site_1", displayName="Site_1", parentId="", type=NodeType.site, download=1000, upload=1000))
net.addRawNode(NetworkNode(id="Site_2", displayName="Site_2", parentId="", type=NodeType.site, download=500, upload=500))

net.addRawNode(NetworkNode(id="AP_A", displayName="AP_A", parentId="Site_1", type=NodeType.ap, download=500, upload=500))
net.addRawNode(NetworkNode(id="Site_3", displayName="Site_3", parentId="Site_1", type=NodeType.site, download=500, upload=500))
net.addRawNode(NetworkNode(id="PoP_5", displayName="PoP_5", parentId="Site_3", type=NodeType.site, download=200, upload=200))        
net.addRawNode(NetworkNode(id="AP_9", displayName="AP_9", parentId="PoP_5", type=NodeType.ap, download=120, upload=120))
net.addRawNode(NetworkNode(id="PoP_6", displayName="PoP_6", parentId="PoP_5", type=NodeType.site, download=60, upload=60))
net.addRawNode(NetworkNode(id="AP_11", displayName="AP_11", parentId="PoP_6", type=NodeType.ap, download=30, upload=30))
net.addRawNode(NetworkNode(id="PoP_1", displayName="PoP_1", parentId="Site_2", type=NodeType.site, download=200, upload=200))
net.addRawNode(NetworkNode(id="AP_7", displayName="AP_7", parentId="PoP_1", type=NodeType.ap, download=100, upload=100))
net.addRawNode(NetworkNode(id="AP_1", displayName="AP_1", parentId="Site_2", type=NodeType.ap, download=150, upload=150))

When you attach a customer, you can specify a tree entry (e.g. `PoP_5`) as a parent:

    parentId="PoP_5",

)

Once you have entered all of your network topology and customers, you can finish the integration:

You can also add a call to `net.plotNetworkGraph(False)` (use `True` to also include every customer; this can make for a HUGE file) to create a PDF file (currently named `network.pdf.pdf`) displaying your topology. The example shown here looks like this:

 to also include every customer; this can make for a HUGE file) to create a PDF file (currently named 

) displaying your topology. The example shown here looks like this:

[@thebracket](https://github.com/thebracket/) has created [cpumap-pping](https://github.com/thebracket/cpumap-pping) which merges the functionality of the [xdp-cpumap-tc](https://github.com/xdp-project/xdp-cpumap-tc) and [ePPing](https://github.com/xdp-project/bpf-examples/tree/master/pping) projects, while keeping CPU use within ~1% of xdp-cpumap-tc.

UISP integration overhaul by [@thebracket](https://github.com/thebracket/)

[LMS integation](https://github.com/interduo/LMSLibreQoS) for Polish ISPs by [@interduo](https://github.com/interduo)

In v1.2 and prior, the the entire queue structure had to be reloaded to make any changes. This led to a few milliseconds of packet loss for some clients each time that reload happened. The scheduled.py was set to reload all queues each morning at 4AM to avoid any potential disruptions that could theoretically cause.

Starting with v1.3 - LibreQoS tracks the state of the queues, and can do incremental changes without a full reload of all queues. Every 30 minutes - scheduler.py runs the CRM import, and runs a partial reload affecting just the queues that have changed. It still runs a full reload at 4AM.

HTB uses a hex handle for classes. It is two 16-bit hex values joined by a colon - major:minor (<u16>:<u16>). In LibreQoS, each CPU core uses a different major handle.

HTB uses a hex handle for classes. It is two 16-bit hex values joined by a colon - major:minor (

In v1.2 and prior, the minor handle was unique across all CPUs, meaning only 30k subscribers could be added total.

Starting with LibreQoS v1.3 - minor handles are counted independently by CPU core. With this change, the maximum possible subscriber qdiscs/classes goes from a hard limit of 30k to instead be 30k x CPU core count. So for a higher end system with a 64 core processor such as the AMD EPYC™ 7713P, that would mean ~1.9 million possible subscriber classes. Of course CPU use will be the bottleneck well before class handles are in that scenario. But at least we have that arbitrary 30k limit out of the way.

In order to improve queue reload time in v1.3, it was necessary to use a unique identifier for each circuit. We went with Circuit ID. It can be a number or string, it just needs to be unique between circuits, and the same for multiple devices in the same circuit. This allows us to avoid costly lookups when sorting through the queue structure.

If you have your own script creating ShapedDevices.csv - you could use your CRM's unique identifier for customer services / circuits to serve as this Circuit ID. The UISP and Splynx integrations already do this automatically.

If you have your own script creating ShapedDevices.csv - you could use your CRM’s unique identifier for customer services / circuits to serve as this Circuit ID. The UISP and Splynx integrations already do this automatically.

The most major changes are the renaming of the fqorCake variable to "sqm",
and the addition of the Circuit identifier field.

The most major changes are the renaming of the fqorCake variable to “sqm”,

Also after upgrading to LibreQos v1.3, a reboot is required to clear out the
old ebpf code.

You could theoretically throttle all unknown IPs until they are associated with a client. For example, you could limit every unknown to 1.5x0.5 with single entry in ShapedDevices.csv, until you associate them with an account. IPs need to be non-exact matches. So you can't have two 192.168.1.1 entries, but you can have a 192.168.1.0/24 subnet and a 192.168.1.2/32 - they aren't duplicates, and the LPM search is smart enough to pick the most exact match.

You could theoretically throttle all unknown IPs until they are associated with a client. For example, you could limit every unknown to 1.5x0.5 with single entry in ShapedDevices.csv, until you associate them with an account. IPs need to be non-exact matches. So you can’t have two 192.168.1.1 entries, but you can have a 192.168.1.0/24 subnet and a 192.168.1.2/32 - they aren’t duplicates, and the LPM search is smart enough to pick the most exact match.

Version 1.4 is still undergoing active development, but if you'd like to benefit from it right now (or help us test/develop it!), here's a guide.

Version 1.4 is still undergoing active development, but if you’d like to benefit from it right now (or help us test/develop it!), here’s a guide.

<strong> Please see the [v1.4 Wiki](https://github.com/LibreQoE/LibreQoS/wiki/v1.4) which replaces this document. </strong>

Run ```sudo crontab -e``` and remove any entries pertaining to LibreQoS from v1.3.

The recommended install location is `/opt/libreqos`.
Go to the install location, and clone the repo:

cd /opt/
git clone https://github.com/LibreQoE/LibreQoS.git libreqos
sudo chown YOUR_USER /opt/libreqos -R

By specifying `libreqos` at the end, git will ensure the folder name is lowercase.

Now that this is in `main`, you no longer need to switch git branch. If you were previously on the `v1.4-pre-alpha-rust-integration` branch, please switch to main with `git checkout main; git pull`.

, you no longer need to switch git branch. If you were previously on the 

apt-get install -y python3-pip clang gcc gcc-multilib llvm libelf-dev git nano graphviz curl screen llvm pkg-config linux-tools-common linux-tools-`uname -r` libbpf-dev

python3 -m pip install ipaddress schedule influxdb-client requests flask flask_restful flask_httpauth waitress psutil binpacking graphviz

Go to [RustUp](https://rustup.rs) and follow the instructions. Basically, run the following:

When Rust finishes installing, it will tell you to execute a command to place the Rust build tools into your path. You need to either execute this command or logout and back in again.

Once that's done, change directory to `/wherever_you_put_libreqos/src/`, and run:

This will take a while the first time, but it puts everything in the right place.
Remember to run this command everytime after `git pull`.

lqos_directory = '/opt/libreqos/src'
queue_check_period_ms = 1000

[tuning]
stop_irq_balance = true
netdev_budget_usecs = 8000
netdev_budget_packets = 300
rx_usecs = 8
tx_usecs = 8
disable_rxvlan = true
disable_txvlan = true
disable_offload = [ "gso", "tso", "lro", "sg", "gro" ]

interface_mapping = [
       { name = "enp1s0f1", redirect_to = "enp1s0f2", scan_vlans = false },
       { name = "enp1s0f2", redirect_to = "enp1s0f1", scan_vlans = false }
]
vlan_mapping = []

Change `enp1s0f1` and `enp1s0f2` to match your network interfaces. It doesn't matter which one is which.

 to match your network interfaces. It doesn’t matter which one is which.

Follow the regular instructions to set your interfaces in `ispConfig.py` and your `network.json` and `ShapedDevices.csv` files.

sudo cp /opt/libreqos/src/bin/lqos_node_manager.service.example /etc/systemd/system/lqos_node_manager.service
sudo cp /opt/libreqos/src/bin/lqosd.service.example /etc/systemd/system/lqosd.service
sudo cp /opt/libreqos/src/bin/lqos_scheduler.service.example /etc/systemd/system/lqos_scheduler.service
sudo systemctl daemon-reload
sudo systemctl enable lqosd lqos_node_manager lqos_scheduler

lqos_scheduler handles statistics and performs continuous refreshes of LibreQoS' shapers, including pulling from any enabled CRM Integrations (UISP, Splynx).

lqos_scheduler handles statistics and performs continuous refreshes of LibreQoS’ shapers, including pulling from any enabled CRM Integrations (UISP, Splynx).

Every 30 minutes: Update queues, pulling new configuration from CRM integration if enabled

You can setup `lqosd` and `lqos_node_manager` as daemons to keep running (there are example `systemd` files in the `src/bin` folder). Since v1.4 is under such heavy development, I recommend using `screen` to run detached instances - and make finding issues easier.

 folder). Since v1.4 is under such heavy development, I recommend using 

You can now point a web browser at `http://a.b.c.d:9123` (replace `a.b.c.d` with the management IP address of your shaping server) and enjoy a real-time view of your network.

 with the management IP address of your shaping server) and enjoy a real-time view of your network.

In the web browser, click `Reload LibreQoS` to setup your shaping rules.

Note: On January 22nd 2023 /etc/lqos was changed to /etc/lqos.conf to remedy Issue #205. If upgrading, be sure to move /etc/lqos to /etc/lqos.conf

Note: If you use the XDP bridge, traffic will stop passing through the bridge during the update (XDP bridge is only operating while lqosd runs).

Go to console 1 (`Ctl-A, 1`) and stop `lqos_node_manager` with `ctrl+c`.

Go to console 0 (`Ctrl-A, 0`) and run `sudo ./lqosd` to restart the bridge/manager.

Go to console 1 (`Ctrl-A, 1`) and run `./lqos_node_manager` to restart the web server.

Bugfix for slowly Ubuntu starting (~2 minutes penalty) in situation when one of the networking interface is down during startup

bugfix for slowly ubuntu starting (~2 minutes penalty) in situation when one of the networking interface is down during startup

Deciding what the sane things are, for each error return, is hard.

deciding what the sane things are, for each error return, is hard.

The example pinned.c file is a good example - What goes wrong if

the example pinned.c file is a good example - what goes wrong if

Very much a work in progress. Details will be filled out as it stabilizes.

This project contains a number of projects arranged in a workspace. The projects are:

`lqos_sys` - a library that builds, installs, removes and manages the LibreQoS XDP and TC programs.

 - a library that builds, installs, removes and manages the LibreQoS XDP and TC programs.

`lqos_bus` - definitions and helper functions for passing data across the local management bus.

 - definitions and helper functions for passing data across the local management bus.

`lqos_config` - a crate that handles pulling configuration from the Python manager.

 - a crate that handles pulling configuration from the Python manager.

`lqosd` - the management daemon that should eventually be run as a `systemd` service.

When started, the daemon sets up XDP/TC eBPF programs for the interfaces specified in the LibreQoS configuration.

`lqtop` - A CLI tool that outputs the top X downloaders and mostly verifies that the bus and daemons work.

 - A CLI tool that outputs the top X downloaders and mostly verifies that the bus and daemons work.

`xdp_iphash_to_cpu_cmdline` - An almost-compatible command that acts like the tool of the same name from the previous verion.

 - An almost-compatible command that acts like the tool of the same name from the previous verion.

`xdp_pping` - Port of the previous release's `xdp_pping` tool, for compatibility. Will eventually not be needed.

`linux-tools-5.15.0-56-generic` (the common version doesn't work?)

`remove_pinned_maps.sh` deletes all of the BPF shared maps. Useful during development.

Since all of the parts of the system need to know where to find LibreQoS, it first looks for a file named `/etc/lqos.conf` and uses that to locate the LibreQoS installation.

Since all of the parts of the system need to know where to find LibreQoS, it first looks for a file named 

`lqos_directory`: where LibreQoS is installed (e.g. `/opt/libreqos`)

ways_collisions, if you are seeing a lot of these, it's indicative of a ping flood of some kind.
  thresh          6Mbit
  target          5.0ms
  interval      100.0ms
these are the bandwidth and codel settiings. On a vpn interface, or in a place where the typical rtt is to sydney and back, you can set rtt lower in cake, otherwise, don't touch.
 pk_delay        440us
  av_delay         13us
  sp_delay          2us
All these are ewmas (I forget the interval). pk_delay would be a good statistic to graph, tho I suspect that the highest peak delays would be on the lowest end plans, so peak delay by plan would pull out folk that were above average here.
av_delay is pretty useless, like all averages, IMHO. Still I could be wrong.
sp_delay = sparse delay and is an indicator of overload. Customers with the largest sparse delay would be interesting. Ideally this number never cracks 2ms.
backlog - a persistent backlog is OK, but a sign that someone is using a torrent-like app, doing a big backup, or underprovisioned in some way. Seeing a persistent backlog in an AP on the other hand says the AP is overloaded.
pkts
bytes
do what they say and are a summary statistic since invokation of cake. 
way_inds
way_mis
way_cols are indicators of how well the FQ is working, but not ver good ones. 
drops
marks
ack_drop
in general I care about pkts, ack_drops, drops and marks, most, these are best plotted together on an invlog scale. When last I looked robert was combining drops and marks to treat as drops (which they are), but I am caring about the rise of ECN in general. As for the top 10 approach, the largest number of drops or marks relative to packets, would be useful.
  sp_flows            1
  bk_flows            1
  un_flows            0
bulk_flows indicate large transfers. They have nothing to do with diffserv marks, they are just bulk. 
max_len - if this is greater than your MTU, GRO and GSO are loose on your network. 
Quantum varies as a function of bandwidth, it may be 300 is more optimal than an MTU even at higher bandwidths than we first tested cake at.

ways_collisions, if you are seeing a lot of these, it’s indicative of a ping flood of some kind.

these are the bandwidth and codel settiings. On a vpn interface, or in a place where the typical rtt is to sydney and back, you can set rtt lower in cake, otherwise, don’t touch.

do what they say and are a summary statistic since invokation of cake.

way_cols are indicators of how well the FQ is working, but not ver good ones.

bulk_flows indicate large transfers. They have nothing to do with diffserv marks, they are just bulk.

max_len - if this is greater than your MTU, GRO and GSO are loose on your network.

This crate wraps the XDP component in externally callable Rust. This is
used by other systems to manage the XDP/TC eBPF system.

The `src/bpf` directory contains the C for the eBPF program, as well as
some wrapper helpers to bring it into Rust-space.

**The LibreQoS Daemon** is designed to run as a `systemd` service at all times. It provides:

Load/Unload the XDP/TC programs (they unload when the program exits)

Includes support for "on a stick" mode, using `OnAStick = True, StickVlanA = 1, StickVlanB = 2`.

Hosts a lightweight server offering "bus" queries for clients (such as `lqtop` and `xdp_iphash_to_cpu_cmdline`).

Hosts a lightweight server offering “bus” queries for clients (such as 

Periodically gathers statistics for distribution to other systems via the bus.

You *must* have a file present called `/etc/lqos.conf`. At a minimum, it must tell `lqosd` where to find the LibreQoS configuration. For example:

`lqosd` can set kernel tunables for you on start-up. These are specified in `/etc/lqos.conf` also, in the `[tuning]` section:

 can set kernel tunables for you on start-up. These are specified in 

[tuning]
netdev_budget_usecs = 20
netdev_budget_packets = 1
rx_usecs = 0
tx_usecs = 0

[bridge]
use_xdp_bridge = true
	{ name = "eth1", redirect_to = "eth2", scan_vlans = false },
	{ name = "eth2", redirect_to = "eth1", scan_vlans = false }
]

Each interface must be a *physical* interface, not a VLAN. If you set `scan_vlans` to `true`, you can specify mapping rules.

[bridge]
	{ name = "eth1", redirect_to = "eth1", scan_vlans = true },
]
vlan_mapping = [
	{ parent = "eth1", tag = 3, redirect_to = 4 },
	{ parent = "eth1", tag = 4, redirect_to = 3 },
]

Reciprocal mappings are created NOT automatically, you have to specify each mapping. When you are using "on a stick" mode, you need to redirect to the same interface.

Reciprocal mappings are created NOT automatically, you have to specify each mapping. When you are using “on a stick” mode, you need to redirect to the same interface.

Note that this is *almost* compatible with the previous version. The syntax has changed so that the command must go first, and doesn't require a `--`.

 compatible with the previous version. The syntax has changed so that the command must go first, and doesn’t require a 

Usage: xdp_iphash_to_cpu_cmdline [COMMAND]

Commands:
  add    Add an IP Address (v4 or v6) to the XDP/TC mapping system
  del    Remove an IP address (v4 or v6) from the XDP/TC mapping system
  clear  Clear all mapped IPs
  list   List all mapped IPs
  help   Print this message or the help of the given subcommand(s)

Options:
  -h, --help  Print help information

xdp_iphash_to_cpu_cmdline list
xdp_iphash_to_cpu_cmdline add --ip 192.168.100.1 --classid 5:3 --cpu 4
xdp_iphash_to_cpu_cmdline del 192.168.100.1
xdp_iphash_to_cpu_cmdline clear

For VMs, NIC passthrough is required for optimal throughput and latency (XDP vs generic XDP). Using Virtio / bridging is much slower than NIC passthrough. Virtio / bridging should not be used for large amounts of traffic.

A CPU with solid [single-thread performance](https://www.cpubenchmark.net/singleThread.html#server-thread) within your budget. Queuing is very CPU-intensive, and requires high single-thread performance.

 within your budget. Queuing is very CPU-intensive, and requires high single-thread performance.

Single-thread CPU performance will determine the max throughput of a single HTB (cpu core), and in turn, what max speed plan you can offer customers.

Below is a table of approximate aggregate throughput capacity, assuming a a CPU with a [single thread](https://www.cpubenchmark.net/singleThread.html#server-thread) performance of 2700 or greater:

Below is a table of approximate aggregate throughput capacity, assuming a a CPU with a 

So for example, an ISP delivering 1Gbps service plans with 10Gbps aggregate throughput would choose a CPU with a 2500+ single-thread score and 8 cores, such as the Intel Xeon E-2388G @ 3.20GHz.

It is most cost-effective to buy a used server with specifications matching your unique requirements, as laid out in the System Requirements section below.
For those who do not have the time to do that, here are some off-the-shelf options to consider:

1 Gbps | [Supermicro SuperServer E100-9W-L](https://www.thinkmate.com/system/superserver-e100-9w-l)

10 Gbps | [Supermicro SuperServer 510T-ML (Choose E-2388G)](https://www.thinkmate.com/system/superserver-510t-ml)

The [AsRock 1U4LW-B6502L2T](https://www.thinkmate.com/system/asrock-1u4lw-b6502l2t/635744) can be a great lower-cost option as well.

One management network interface completely separate from the traffic shaping interfaces. Usually this would be the Ethernet interface built in to the motherboard.

NIC must have multiple TX/RX transmit queues. [Here's how to check from the command line](https://serverfault.com/questions/772380/how-to-tell-if-nic-has-multiqueue-enabled).

[NVIDIA Mellanox MCX512A-ACAT](https://www.fs.com/products/119649.html)

- single NIC with two interfaces,
- two NICs with single interface,
- 2x VLANs interface (using one or two NICs).

This alternate configuration uses Spanning Tree Protocol (STP) to modify the data path in the event the LibreQoS device is offline for maintenance or another problem.

Most of the same considerations apply to the alternate configuration as they do to the officially supported configuation

Edge router and Core switch should have 1500 MTU on links between them

If you use MPLS, you would terminate MPLS traffic somewhere south of the core/distribution switch. LibreQoS cannot decapsulate MPLS on its own.

Spanning Tree primary link (low cost) through the server running LibreQoS

Keep in mind that if you use different bandwidth links, for example, 10 Gbps through LibreQoS, and 1 Gbps between core switch and edge router, you may need to be more intentional with your STP costs.

sudo systemctl disable offloadOff.service
sudo rm /usr/local/sbin/offloadOff.sh /etc/systemd/system/offloadOff.service

Use the deb package from the [latest v1.4 release](https://github.com/LibreQoE/LibreQoS/releases/).

The recommended install location is `/opt/libreqos`

cd /opt/
sudo chown -R YOUR_USER /opt/libreqos

sudo apt-get install -y python3-pip clang gcc gcc-multilib llvm libelf-dev git nano graphviz curl screen llvm pkg-config linux-tools-common linux-tools-`uname -r` libbpf-dev

cd /opt/libreqos
python3 -m pip install -r requirements.txt
sudo python3 -m pip install -r requirements.txt

Change `enp1s0f1` and `enp1s0f2` to match your network interfaces. It doesn't matter which one is which. Notice, it's paring the interfaces, so when you first enter enps0f<ins>**1**</ins> in the first line, the `redirect_to` parameter is enp1s0f<ins>**2**</ins> (replacing with your actual interface names).

 to match your network interfaces. It doesn’t matter which one is which. Notice, it’s paring the interfaces, so when you first enter enps0f

Then, if using Bifrost/XDP set `use_xdp_bridge = true` under that same `[bridge]` section.

cd /opt/libreqos/src/
cp ispConfig.example.py ispConfig.py
nano ispConfig.py

Set upstreamBandwidthCapacityDownloadMbps and upstreamBandwidthCapacityUploadMbps to match the bandwidth in Mbps of your network's upstream / WAN internet connection. The same can be done for generatedPNDownloadMbps and generatedPNUploadMbps.

Set upstreamBandwidthCapacityDownloadMbps and upstreamBandwidthCapacityUploadMbps to match the bandwidth in Mbps of your network’s upstream / WAN internet connection. The same can be done for generatedPNDownloadMbps and generatedPNUploadMbps.

Set interfaceA to the interface facing your core router (or bridged internal network if your network is bridged)

Set ```enableActualShellCommands = True``` to allow the program to actually run the commands.

First, set the relevant parameters for UISP (uispAuthToken, UISPbaseURL, etc.) in ispConfig.py.

On the first successful run, it will create a network.json and ShapedDevices.csv file.
If a network.json file exists, it will not be overwritten.
You can modify the network.json file to more accurately reflect bandwidth limits.
ShapedDevices.csv will be overwritten every time the UISP integration is run.
You have the option to run integrationUISP.py automatically on boot and every 30 minutes, which is recommended. This can be enabled by setting ```automaticImportUISP = True``` in ispConfig.py

You have the option to run integrationUISP.py automatically on boot and every 30 minutes, which is recommended. This can be enabled by setting 

First, set the relevant parameters for Splynx (splynx_api_key, splynx_api_secret, etc.) in ispConfig.py.

On the first successful run, it will create a ShapedDevices.csv file.
You can manually create your network.json file to more accurately reflect bandwidth limits.
ShapedDevices.csv will be overwritten every time the Splynx integration is run.
You have the option to run integrationSplynx.py automatically on boot and every 30 minutes, which is recommended. This can be enabled by setting ```automaticImportSplynx = True``` in ispConfig.py

You have the option to run integrationSplynx.py automatically on boot and every 30 minutes, which is recommended. This can be enabled by setting 

Network.json allows ISP operators to define a Hierarchical Network Topology, or Flat Network Topology.

For networks with no Parent Nodes (no strictly defined Access Points or Sites) edit the network.json to use a Flat Network Topology with
```nano network.json```
setting the following file content:

If you plan to use the built-in UISP or Splynx integrations, you do not need to create a network.json file quite yet.

If you plan to use the built-in UISP integration, it will create this automatically on its first run (assuming network.json is not already present). You can then modify the network.json to more accurately reflect your topology.

If you will not be using an integration, you can manually define the network.json following the template file - network.example.json

+-----------------------------------------------------------------------+
| Entire Network                                                        |
+-----------------------+-----------------------+-----------------------+
| Parent Node A         | Parent Node B         | Parent Node C         |
+-----------------------+-------+-------+-------+-----------------------+
| Parent Node D | Sub 3 | Sub 4 | Sub 5 | Sub 6 | Sub 7 | Parent Node F |
+-------+-------+-------+-------+-------+-------+-------+-------+-------+
| Sub 1 | Sub 2 |       |                       |       | Sub 8 | Sub 9 |
+-------+-------+-------+-----------------------+-------+-------+-------+

to convert manualNetwork.csv to a network.json file.
manualNetwork.csv can be copied from the template file, manualNetwork.template.csv

Note: The parent node name must match that used for clients in ShapedDevices.csv

If you are using an integration, this file will be automatically generated. If you are not using an integration, you can manually edit the file.

Modify the ShapedDevices.csv file using your preferred spreadsheet editor (LibreOffice Calc, Excel, etc), following the template file - ShapedDevices.example.csv

Circuit ID is required. Must be a string of some sort (int is fine, gets parsed as string). Must NOT include any number symbols (#).

The Access Point or Site name should be set in the Parent Node field. Parent Node can be left blank for flat networks.

The ShapedDevices.csv file allows you to set minimum guaranteed, and maximum allowed bandwidth per subscriber.

The minimum allowed plan rates for Circuits are 2Mbit. Bandwidth min and max should both be above that threshold.

Recommendation: set the min bandwidth to something like 25/10 and max to 1.15X advertised plan rate by using bandwidthOverheadFactor = 1.15

This way, when an AP hits its ceiling, users have any remaining AP capacity fairly distributed between them.

Ensure a reasonable minimum bandwidth minimum for every subscriber, allowing them to utilize up to the maximum provided when AP utilization is below 100%.

Note regarding SLAs: For customers with SLA contracts that guarantee them a minimum bandwidth, set their plan rate as the minimum bandwidth. That way when an AP approaches its ceiling, SLA customers will always get that amount.

You can setup `lqosd`, `lqos_node_manager`, and `lqos_scheduler` as systemd services.

In the background, lqos_scheduler runs scheduler.py, which in turn runs LibreQoS.py

One-time runs of these individual components can be very helpful for debugging and to make sure everything is correctly configured.

To confirm that lqos_scheduler (scheduler.py) is able to work correctly, run:

Once you have any errors eliminated, restart lqos_scheduler with

Import Dashboard `Boards > Create Dashboard > Import Dashboard`

You may want to install a reverse proxy in front of the web interfaces for influx and lqos. Setting these up is outside the scope of this document, but some examples are [Caddy](https://caddyserver.com/), and Nginx [Proxy Manager](https://nginxproxymanager.com/)

You may want to install a reverse proxy in front of the web interfaces for influx and lqos. Setting these up is outside the scope of this document, but some examples are 

sudo echo "deb http://stats.libreqos.io/ubuntu jammy main" > /etc/apt/sources.list.d/libreqos.list
wget -O - -q http://stats.libreqos.io/repo.asc | apt-key add -
apt-get update
apt-get install libreqos

You will be asked some questions about your configuration, and the management daemon and webserver will automatically start. Go to http://<your_ip>:9123/ to finish installation.

Use this install if you'd like to constantly deploy from the main branch on Github. For experienced users only!

Use this install if you’d like to constantly deploy from the main branch on Github. For experienced users only!

If you use the XDP bridge, traffic will stop passing through the bridge during the update (XDP bridge is only operating while lqosd runs).

sudo systemctl restart lqosd
sudo systemctl restart lqos_node_manager
sudo systemctl restart lqos_scheduler

All you should have to do in this case is run `sudo apt update && sudo apt upgrade` and LibreQoS should install the new package.

We ask that you please share an anonymized screenshot of your LibreQoS deployment before (monitor only mode) and after (queuing enabled) to our [Matrix Channel](https://matrix.to/#/#libreqos:matrix.org). This helps us gauge the impact of our software. It also makes us smile.

We ask that you please share an anonymized screenshot of your LibreQoS deployment before (monitor only mode) and after (queuing enabled) to our 

. This helps us gauge the impact of our software. It also makes us smile.

sudo systemctl stop lqos_scheduler

Please go to the Web UI and click Configuration. Toggle Redact Customer Information (screenshot mode) and then Apply Changes.

To generate a screenshot - please go to the Web UI and click Configuration. Toggle Redact Customer Information (screenshot mode), Apply Changes, and then return to the dashboard to take a screenshot.

In ispConfig.py, make sure the edge and core interfaces correspond to correctly to the edge and core. Try swapping the interfaces to see if shaping starts to work.

Make sure your services are running properly `lqos.service`, `lqos_node_manager`, `lqos_scheduler`. Node manager and scheduler are dependent on the `lqos.service` being in a healthy, running state.

This tends to show up when the MQ qdisc cannot be added correctly to the NIC interface. This would suggest the NIC has insufficient RX/TX queues. Please make sure you are using the [recommended NICs](#network-interface-card).

This tends to show up when the MQ qdisc cannot be added correctly to the NIC interface. This would suggest the NIC has insufficient RX/TX queues. Please make sure you are using the 

git clone https://github.com/brendangregg/FlameGraph.git
cd FlameGraph
sudo perf record -F 99 -a -g -- sleep 60
perf script > out.perf
./stackcollapse-perf.pl out.perf > out.folded
./flamegraph.pl --title LibreQoS --width 7200 out.folded > libreqos.svg

It is recommended to set the OSPF timers of both OSPF neighbors (core and edge router) to minimize downtime upon a reboot of the LibreQoS server.

LibreQoS v1.3.1 Installation & Usage Guide  - Physical Server and Ubuntu 22.04

libreqos v1.3.1 installation & usage guide - physical server and ubuntu 22.04

If you use a custom CRM integration, please ensure your integration uses a unique circuit identifier for the 'Circuit ID' field in ShapedDevices.csv. This is now required in v1.3 in order to make partial reloading possible. A good choice for this ID would be internet service plan unique ID, or the subscriber site ID your CRM provides for customer service locations. Multiple devices within the same circuit would use the same 'Circuit ID', but aside from that, all Circuit IDs should be distinct. The built-in Splynx and UISP integrations for v1.3 handle this automatically.

If you use a custom CRM integration, please ensure your integration uses a unique circuit identifier for the ‘Circuit ID’ field in ShapedDevices.csv. This is now required in v1.3 in order to make partial reloading possible. A good choice for this ID would be internet service plan unique ID, or the subscriber site ID your CRM provides for customer service locations. Multiple devices within the same circuit would use the same ‘Circuit ID’, but aside from that, all Circuit IDs should be distinct. The built-in Splynx and UISP integrations for v1.3 handle this automatically.

If you use MPLS, you would terminate MPLS traffic at the core router.
LibreQoS cannot decapsulate MPLS on its own.

LibreQoS requires a NIC with 2 or more RX/TX queues and XDP support. While many cards theoretically meet these requirements, less commonly used cards tend to have unreported driver bugs which impede XDP functionality and make them unusable for our purposes. At this time we can only recommend Intel x520, Intel x710, and Nvidia (ConnectX-5 or newer) NICs.

Download Ubuntu Server 22.04 from <a href="https://ubuntu.com/download/server">https://ubuntu.com/download/server</a>.

Sponsors can use the LibreQoS-Installer script. This script does the following:

Creates a bridge between two interfaces - applied by the above service at each boot

Once complete - skip to [this section](https://github.com/LibreQoE/LibreQoS/wiki/LibreQoS-v1.3-Installation-&-Usage-Guide-Physical-Server-and-Ubuntu-22.04#install-influxdb-for-graphing) of the guide.

sudo systemctl stop irqbalance
sudo systemctl disable irqbalance

We need to disable certain hardware offloading features, as they break XDP, used by XDP-CPUMAP-TC to send traffic to appropriate CPUs.
You can create a bash script to disabled these offload features upon boot.

#!/bin/sh
ethtool --offload eth1 gso off tso off lro off sg off gro off
ethtool --offload eth2 gso off tso off lro off sg off gro off

Replace eth1 and eth2 with your two shaper interfaces (order doesn't matter).
Then create

Replace eth1 and eth2 with your two shaper interfaces (order doesn’t matter).

[Unit]
After=network.service

[Service]
ExecStart=/usr/local/sbin/offloadOff.sh

[Install]
WantedBy=default.target

sudo chmod 664 /etc/systemd/system/offloadOff.service
sudo chmod 744 /usr/local/sbin/offloadOff.sh
sudo systemctl enable offloadOff.service
sudo reboot

cd home/$USER/
sudo apt update
sudo apt install python3-pip clang gcc gcc-multilib llvm libelf-dev git nano graphviz
git clone https://github.com/rchac/LibreQoS.git
git checkout v1.3.1

cd home/$USER/LibreQoS/src
git submodule update --init
cd cpumap-pping/
cd src/
make

Call the bucket "libreqos" (all lowercase, without quotes).<br>

cd /home/$USER/LibreQoS/src/

Integrations now share a common framework thanks to [this pull](https://github.com/rchac/LibreQoS/pull/145). This also allows for graphing the network topology with graphviz.

. This also allows for graphing the network topology with graphviz.

Modify the network.json file using your preferred JSON editor (Geany for example)

Parent node name must match that used for clients in ShapedDevices.csv

One-time runs show the response from the terminal for each filter rule applied, and can be very helpful for debugging and to make sure it is correctly configured.

Modify setting parameters in ispConfig.py to suit your environment

To run as a service, we create a systemd service to run scheduler.py.
scheduler.py does the following:

On Linux distributions that use systemd, such as Ubuntu, we create

Then paste the text below, replacing "/home/YOUR_USERNAME/LibreQoS" with wherever you downloaded LibreQoS to. Be sure to replace YOUR_USERNAME with your actual username, because otherwise when the root user executes it, it will look in the wrong directory.

Then paste the text below, replacing “/home/YOUR_USERNAME/LibreQoS” with wherever you downloaded LibreQoS to. Be sure to replace YOUR_USERNAME with your actual username, because otherwise when the root user executes it, it will look in the wrong directory.

[Unit]

[Service]
WorkingDirectory=/home/YOUR_USERNAME/LibreQoS/src
ExecStart=/usr/bin/python3 /home/YOUR_USERNAME/LibreQoS/src/scheduler.py
ExecStopPost=/bin/bash -c '/usr/bin/python3 /home/YOUR_USERNAME/LibreQoS/src/LibreQoS.py --clearrules'
ExecStop=/bin/bash -c '/usr/bin/python3 /home/YOUR_USERNAME/LibreQoS/src/LibreQoS.py --clearrules'
Restart=always

[Install]

sudo chmod 664 /etc/systemd/system/LibreQoS.service
sudo systemctl enable LibreQoS.service

You can restart the service to refresh any changes you've made to the ShapedDevices.csv file by doing

You can restart the service to refresh any changes you’ve made to the ShapedDevices.csv file by doing

You can also stop the service to remove all queues and IP rules by doing

At 4AM: Runs a full reload of all queues to make sure they perfectly match queueStructure.py and that any changes to network.json can be applied.

sudo crontab -l | grep -q 'LibreQoS'  && echo 'entry exists' || echo 'entry does not exist'

The above should output "entry does not exist". If so, proceed to add it with:

The above should output “entry does not exist”. If so, proceed to add it with:

(sudo crontab -l 2>/dev/null; echo "0 4 * * * /bin/systemctl try-restart LibreQoS") | sudo crontab -
sudo /etc/init.d/cron start

Version 1.4 is a huge milestone. A whole new back-end, new GUI, 30%+ performance improvements, support for single-interface mode.

Tested single-stream performance of just under 10 gbit/s on a 16-core Xeon Gold (single interface architecture, using 8 cores for each direction). The flow was shaped with Cake, and retained good (<10 ms RTT latency) performance.

Tested 25 gbit/s total throughput on the same system. CPU was not saturated---we didn't have a bigger network to test!

Tested 25 gbit/s total throughput on the same system. CPU was not saturated—we didn’t have a bigger network to test!

Running live at ISPs with 11 gbit/s of real customer performance and plenty of room to grow.

Provides a local "bus" for transporting data between components.

Provides a local “bus” for transporting data between components.

Sets "tunables", replacing the need for a separate offloading service.

Sets “tunables”, replacing the need for a separate offloading service.

Categorizes traffic to match your network hierarchy, letting you quickly find the bottlenecks.

Lists "unknown IP addresses" that are passing through the shaper but do not have a rule associated.

Lists “unknown IP addresses” that are passing through the shaper but do not have a rule associated.

`lqos_setup` - builds enough configuration files to get you started.

Batches bus-transactions to associate IP subnets with users for fast execution.

Optionally replace the Linux bridge system with an XDP-based bridge accelerator.

Viewing a circuit in the web UI displays a summary of IP traffic flows for that circuit.

A "capture" button will capture packet headers, and allow nanosecond-level analysis of traffic data.

A “capture” button will capture packet headers, and allow nanosecond-level analysis of traffic data.

You can download the packet captures in `libpcap` format, for analysis in Wireshark and similar tools.

Operate with a single network interface and VLANs for "in" and "out".

Operate with a single network interface and VLANs for “in” and “out”.

TCP "round trip time" histogram showing overall network latency performance.

TCP “round trip time” histogram showing overall network latency performance.

Per-network node RTT latency histogram, to let you zero-in on troublespots.

`build_rust.sh` builds the entire package from a Git update, with minimal (<1 second) downtime.

 builds the entire package from a Git update, with minimal (<1 second) downtime.

`build_dpkg.sh` assembles the entire system into an Ubuntu/Debian `.deb` installer.

Redact text into Klingon to allow screenshots without sharing customer data.

Preliminary support for reading IP data inside MPLS packets, as long as they are ordered "VLAN->MPLS->VPLS" and not the other way around.

Preliminary support for reading IP data inside MPLS packets, as long as they are ordered “VLAN->MPLS->VPLS” and not the other way around.

Improved rejection of TCP round-trip-time outliers (from long-polled connections).

This is in alpha testing. It has worked on some test setups, but needs production testing.

Installation via `apt-get` and LibreQoS's own repo. Add the `libreqos` repo, and you can use `apt-get` to install/update the traffic shaper. This doesn't get you the development toolchain.

 to install/update the traffic shaper. This doesn’t get you the development toolchain.

You will be asked some questions about your configuration, and the management daemon and webserver will automatically start. Go to `http://<your_ip>:9123/` to finish installation.

You will be asked some questions about your configuration, and the management daemon and webserver will automatically start. Go to 

![image](https://user-images.githubusercontent.com/14896751/227727398-ff9e0321-cae3-4daf-a9f7-59fafaf30061.png)
*The node manager displays current activity on your network*

![image](https://user-images.githubusercontent.com/14896751/227727646-c6eaaece-f2f0-4667-9b66-5cb08a7f4eb8.png)
*Find out exactly what's going on inside each circuit, monitoring all of the queue stats - you can even view the details of each category tin*

Find out exactly what's going on inside each circuit, monitoring all of the queue stats - you can even view the details of each category tin

Find out exactly what’s going on inside each circuit, monitoring all of the queue stats - you can even view the details of each category tin

![image](https://user-images.githubusercontent.com/14896751/227727691-e82bc021-4da4-436a-93af-0f7cd13824be.png)
*Analyze what's going on for a specific client, viewing real-time traffic flow data. No need to run `torch` or equivalent on their router. Ideal for finding connectivity problems.*

Analyze what's going on for a specific client, viewing real-time traffic flow data. No need to run torch or equivalent on their router. Ideal for finding connectivity problems.

Analyze what’s going on for a specific client, viewing real-time traffic flow data. No need to run 

 or equivalent on their router. Ideal for finding connectivity problems.

![image](https://user-images.githubusercontent.com/14896751/227727755-e8c9bce6-58b2-4c50-a466-2e68b29a4b18.png)
*Capture traffic and analyze per-packet in an intuitive, zoomable traffic analysis system. You can view down to nanosecond granularity to find performance problems, and see what's really going on with your traffic. Click "Download PCAP Dump" to analyze the same data in Wireshark.*

Capture traffic and analyze per-packet in an intuitive, zoomable traffic analysis system. You can view down to nanosecond granularity to find performance problems, and see what's really going on with your traffic. Click "Download PCAP Dump" to analyze the same data in Wireshark.

Capture traffic and analyze per-packet in an intuitive, zoomable traffic analysis system. You can view down to nanosecond granularity to find performance problems, and see what’s really going on with your traffic. Click “Download PCAP Dump” to analyze the same data in Wireshark.

Node manager and scheduler are dependent on the `lqos.service` being in a healthy, running state.

This tends to show up when the MQ qdisc cannot be added correctly to the NIC interface. This would suggest the NIC has insufficient RX/TX queues. Please make sure you are using the [recommended NICs](../SystemRequirements/Networking.md).

Once your configuration is complete. You're ready to run the application and start the [Deamons](./services-and-run.md)

Once your configuration is complete. You’re ready to run the application and start the 