mirror of
https://github.com/grafana/grafana.git
synced 2026-08-13 06:34:55 -05:00
Docs: Add Grafana basics section (#32693)
* moved files * single sourcing * Update _index.md * Update heatmap.md * Update _index.md * Update _index.md * Fixed 7 broken relrefs * Fixed one more broken relref. Co-authored-by: achatterjee-grafana <aparajita.chatterjee@grafana.com>
This commit is contained in:
Diana Payton
co-authored by
achatterjee-grafana
achatterjee-grafana
parent
54ad791c7e
commit
ec624aa1ef
@@ -0,0 +1,14 @@
|
||||
+++
|
||||
title = "Grafana basics"
|
||||
weight = 15
|
||||
+++
|
||||
|
||||
# Grafana basics
|
||||
|
||||
This section provides basic information about observability topics in general and Grafana in particular. These topics will help people who are just starting out with observability and monitoring.
|
||||
|
||||
{{< docs/shared "basics/what-is-grafana.md" >}}
|
||||
|
||||
{{< docs/shared "basics/grafana-cloud.md" >}}
|
||||
|
||||
{{< docs/shared "basics/grafana-enterprise.md" >}}
|
||||
@@ -0,0 +1,119 @@
|
||||
+++
|
||||
title = "Glossary"
|
||||
description = "Grafana glossary"
|
||||
keywords = ["grafana", "intro", "glossary", "dictionary"]
|
||||
aliases = ["/docs/grafana/latest/guides/glossary", "/docs/grafana/latest/getting-started/glossary"]
|
||||
weight = 800
|
||||
+++
|
||||
|
||||
# Glossary
|
||||
|
||||
This topic lists words and abbreviations that are commonly used in the Grafana documentation and community.
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Dashboard</td>
|
||||
<td>
|
||||
A set of one or more panels, organized and arranged into one or more rows, that provide an at-a-glance view of related information.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Data source</td>
|
||||
<td>
|
||||
A file, database, or service providing the data. Grafana supports several data sources by default, and can be extended to support additional data sources through plugins.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Exemplar</td>
|
||||
<td>
|
||||
An exemplar is any data that serves as a detailed example of one of the observations aggregated into a metric. An exemplar contains the observed value together with an optional timestamp and arbitrary labels, which are typically used to reference a trace.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Explore</td>
|
||||
<td>
|
||||
Explore allows a user to focus on building a query. Users can refine the query to return the expected metrics before building a dashboard. For more information, refer to the <a href="https://grafana.com/docs/grafana/latest/explore">Explore</a> topic.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Export/Import dashboard</td>
|
||||
<td>
|
||||
Grafana includes the ability to export your dashboards to a file containing JSON. Community members sometimes share their created dashboards on the <a href="https://grafana.com/grafana/dashboards">Grafana Dashboards page</a>. Dashboards previously exported or found on this site may be imported by other users.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Exporter</td>
|
||||
<td>
|
||||
An exporter translates data that comes out of a data source into a format that Prometheus can digest.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Integration (Grafana Cloud)</td>
|
||||
<td>
|
||||
Each Integration in Grafana Cloud uses the cloud agent to connect your data source to Grafana Cloud for visualizing. Note: Prometheus uses the word “integrations” to refer to software that exposes Prometheus metrics without needing an exporter, which is a different use of the same word we use here.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Graph</td>
|
||||
<td>
|
||||
A commonly-used visualization that displays data as points, lines, or bars.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Mixin</td>
|
||||
<td>
|
||||
A mixin is a set of Grafana dashboards and Prometheus rules and alerts, written in Jsonnet and packaged together in a bundle.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Panel</td>
|
||||
<td>
|
||||
Basic building block in Grafana, composed by a query and a visualization. Can be moved and resized within a dashboard.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Plugin</td>
|
||||
<td>
|
||||
An extension of Grafana that allows users to provide additional functionality to enhance their experience. The types of plugins currently supported are:
|
||||
<ul>
|
||||
<li>
|
||||
<b>App plugin:</b> Extends Grafana with a customized experience. It includes a set of panel and data source plugins, as well as custom pages.
|
||||
</li>
|
||||
<li>
|
||||
<b>Data source plugin:</b> Extends Grafana with support for additional data sources.
|
||||
</li>
|
||||
<li>
|
||||
<b>Panel plugin:</b> Extends Grafana with additional visualization options.
|
||||
</li>
|
||||
</ul>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Query</td>
|
||||
<td>
|
||||
Used to request data from a data source. The structure and format of the query depend on the specific data source.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Time series</td>
|
||||
<td>
|
||||
A series of measurements, ordered by time. Time series are stored in data sources and returned as the result of a query.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Trace</td>
|
||||
<td>
|
||||
An observed execution path of a request through a distributed system. For more information, refer to <a href="https://opentracing.io/docs/overview/what-is-tracing/">What is Distributed Tracing?</a>
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Transformation</td>
|
||||
<td>
|
||||
Transformations process the result set of a query before it’s passed on for visualization. For more information, refer to the <a href="https://grafana.com/docs/grafana/latest/panels/transformations">Transformations overview</a> topic.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td style="vertical-align: top">Visualization</td>
|
||||
<td>A graphical representation of query results.</td>
|
||||
</tr>
|
||||
</table>
|
||||
@@ -0,0 +1,55 @@
|
||||
+++
|
||||
title = "Histograms and heatmaps"
|
||||
description = "An introduction to histograms and heatmaps"
|
||||
keywords = ["grafana", "heatmap", "panel", "documentation", "histogram"]
|
||||
aliases = ["/docs/grafana/latest/getting-started/intro-histograms"]
|
||||
weight = 700
|
||||
+++
|
||||
|
||||
# Introduction to histograms and heatmaps
|
||||
|
||||
A histogram is a graphical representation of the distribution of numerical data. It groups values into buckets
|
||||
(sometimes also called bins) and then counts how many values fall into each bucket.
|
||||
|
||||
Instead of graphing the actual values, histograms graph the buckets. Each bar represents a bucket,
|
||||
and the bar height represents the frequency (such as count) of values that fell into that bucket's interval.
|
||||
|
||||
## Histogram example
|
||||
|
||||
This histogram shows the value distribution of a couple of time series. You can easily see that
|
||||
most values land between 240-300 with a peak between 260-280.
|
||||
|
||||
|
||||
|
||||
Histograms only look at _value distributions_ over a specific time range. The problem with histograms is you cannot see any trends or changes in the distribution over time.
|
||||
This is where heatmaps become useful.
|
||||
|
||||
## Heatmaps
|
||||
|
||||
A _heatmap_ is like a histogram, but over time where each time slice represents its own histogram. Instead of using bar height as a representation of frequency, it uses cells and colors the cell proportional to the number of values in the bucket.
|
||||
|
||||
In this example, you can clearly see what values are more common and how they trend over time.
|
||||
|
||||
|
||||
|
||||
## Pre-bucketed data
|
||||
|
||||
There are a number of data sources supporting histogram over time like Elasticsearch (by using a Histogram bucket
|
||||
aggregation) or Prometheus (with [histogram](https://prometheus.io/docs/concepts/metric_types/#histogram) metric type
|
||||
and *Format as* option set to Heatmap). But generally, any data source could be used if it meets the requirements:
|
||||
returns series with names representing bucket bound or returns series sorted by the bound in ascending order.
|
||||
|
||||
## Raw data vs aggregated
|
||||
|
||||
If you use the heatmap with regular time series data (not pre-bucketed), then it's important to keep in mind that your data
|
||||
is often already aggregated by your time series backend. Most time series queries do not return raw sample data
|
||||
but include a group by time interval or maxDataPoints limit coupled with an aggregation function (usually average).
|
||||
|
||||
This all depends on the time range of your query of course. But the important point is to know that the histogram bucketing
|
||||
that Grafana performs might be done on already aggregated and averaged data. To get more accurate heatmaps it is better
|
||||
to do the bucketing during metric collection or store the data in Elasticsearch, or in the other data source which
|
||||
supports doing histogram bucketing on the raw data.
|
||||
|
||||
If you remove or lower the group by time (or raise maxDataPoints) in your query to return more data points your heatmap will be
|
||||
more accurate but this can also be very CPU and memory taxing for your browser and could cause hangs and crashes if the number of
|
||||
data points becomes unreasonably large.
|
||||
@@ -0,0 +1,85 @@
|
||||
+++
|
||||
title = "Time series dimensions"
|
||||
description = "time series dimensions"
|
||||
keywords = ["grafana", "intro", "guide", "concepts", "timeseries", "labels"]
|
||||
aliases = ["/docs/grafana/latest/guides/timeseries-dimensions", "/docs/grafana/latest/getting-started/timeseries-dimensions"]
|
||||
weight = 600
|
||||
+++
|
||||
|
||||
# Time series dimensions
|
||||
|
||||
In [Introduction to time series]({{< relref "timeseries.md#time-series-databases" >}}), the concept of _labels_, also called _tags_, is introduced:
|
||||
|
||||
> Another feature of a TSDB is the ability to filter measurements using _tags_. Each data point is labeled with a tag that adds context information, such as where the measurement was taken.
|
||||
|
||||
With time series data, the data often contain more than a single series, and is a set of multiple time series. Many Grafana data sources support this type of data.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/example_graph_multi_dim.png" class="docs-image--no-shadow" max-width="850px" >}}
|
||||
|
||||
The common case is issuing a single query for a measurement with one or more additional properties as dimensions. For example, querying a temperature measurement along with a location property. In this case, multiple series are returned back from that single query and each series has unique location as a dimension.
|
||||
|
||||
To identify unique series within a set of time series, Grafana stores dimensions in _labels_.
|
||||
|
||||
## Labels
|
||||
|
||||
Each time series in Grafana optionally has labels. labels are set a of key/value pairs for identifying dimensions. Example labels could are `{location=us}` or `{country=us,state=ma,city=boston}`. Within a set of time series, the combination of its name and labels identifies each series. For example, `temperature {country=us,state=ma,city=boston}`.
|
||||
|
||||
Different sources of time series data have dimensions stored natively, or common storage patterns that allow the data to be extracted into dimensions.
|
||||
|
||||
Time series databases (TSDBs) usually natively support dimensionality. Prometheus also stores dimensions in _labels_. In TSDBs such as Graphite or OpenTSDB the term _tags_ is used instead.
|
||||
|
||||
In table databases such SQL, these dimensions are generally the `GROUP BY` parameters of a query.
|
||||
|
||||
## Multiple dimensions in table format
|
||||
|
||||
In SQL or SQL-like databases that return table responses, additional dimensions usually columns in the query response table.
|
||||
|
||||
### Single dimension
|
||||
|
||||
For example, consider a query like:
|
||||
|
||||
```sql
|
||||
SELECT BUCKET(StartTime, 1h), AVG(Temperature) AS Temp, Location FROM T
|
||||
GROUP BY BUCKET(StartTime, 1h), Location
|
||||
ORDER BY time asc
|
||||
```
|
||||
|
||||
Might return a table with three columns that each respectively have data types time, number, and string.
|
||||
|
||||
| StartTime | Temp | Location |
|
||||
| --------- | ---- | -------- |
|
||||
| 09:00 | 24 | LGA |
|
||||
| 09:00 | 20 | BOS |
|
||||
| 10:00 | 26 | LGA |
|
||||
| 10:00 | 22 | BOS |
|
||||
|
||||
The table format is _long_ formatted time series, also called _tall_. It has repeated time stamps, and repeated values in Location. In this case, we have two time series in the set that would be identified as `Temp {Location=LGA}` and `Temp {Location=BOS}`.
|
||||
|
||||
Individual time series from the set are extracted by using the time typed column `StartTime` as the time index of the time series, the numeric typed column `Temp` as the series name, and the name and values of the string typed `Location` column to build the labels, such as Location=LGA.
|
||||
|
||||
### Multiple dimensions
|
||||
|
||||
If the query is updated to select and group by more than just one string column, for example, `GROUP BY BUCKET(StartTime, 1h), Location, Sensor`, then an additional dimension is added:
|
||||
|
||||
| StartTime | Temp | Location | Sensor |
|
||||
| --------- | ---- | -------- | ------ |
|
||||
| 09:00 | 24 | LGA | A |
|
||||
| 09:00 | 24.1 | LGA | B |
|
||||
| 09:00 | 20 | BOS | A |
|
||||
| 09:00 | 20.2 | BOS | B |
|
||||
| 10:00 | 26 | LGA | A |
|
||||
| 10:00 | 26.1 | LGA | B |
|
||||
| 10:00 | 22 | BOS | A |
|
||||
| 10:00 | 22.2 | BOS | B |
|
||||
|
||||
In this case the labels that represent the dimensions will have two keys based on the two string typed columns `Location` and `Sensor`. This data results four series: `Temp {Location=LGA,Sensor=A}`, `Temp {Location=LGA,Sensor=B}`, `Temp {Location=BOS,Sensor=A}`, and `Temp {Location=BOS,Sensor=B}`.
|
||||
|
||||
> **Note:** More than one dimension is currently only supported in the Logs queries within the Azure Monitor service as of version 7.1.
|
||||
|
||||
> **Note:** Multiple dimensions are not supported in a way that maps to multiple alerts in Grafana, but rather they are treated as multiple conditions to a single alert. See the documentation on [creating alerts with multiple series]({{< relref "../alerting/create-alerts.md#multiple-series" >}}).
|
||||
|
||||
### Multiple values
|
||||
|
||||
In the case SQL-like data sources, more than one numeric column can be selected, with or without additional string columns to be used as dimensions. For example, `AVG(Temperature) AS AvgTemp, MAX(Temperature) AS MaxTemp`. This, if combined with multiple dimensions can result in a lot of series. Selecting multiple values is currently only designed to be used with visualization.
|
||||
|
||||
Additional technical information on tabular time series formats and how dimensions are extracted can be found in [the developer documentation on data frames as time series]({{< relref "../developers/plugins/data-frames.md#data-frames-as-time-series" >}}).
|
||||
@@ -0,0 +1,123 @@
|
||||
+++
|
||||
title = "Time series"
|
||||
description = "Introduction to time series"
|
||||
keywords = ["grafana", "intro", "guide", "concepts", "timeseries"]
|
||||
aliases = ["/docs/grafana/latest/guides/timeseries", "/docs/grafana/latest/getting-started/timeseries"]
|
||||
weight = 500
|
||||
+++
|
||||
|
||||
# Introduction to time series
|
||||
|
||||
Imagine you wanted to know how the temperature outside changes throughout the day. Once every hour, you'd check the thermometer and write down the time along with the current temperature. After a while, you'd have something like this:
|
||||
|
||||
| Time | Value |
|
||||
| ----- | ----- |
|
||||
| 09:00 | 24°C |
|
||||
| 10:00 | 26°C |
|
||||
| 11:00 | 27°C |
|
||||
|
||||
Temperature data like this is one example of what we call a *time series*—a sequence of measurements, ordered in time. Every row in the table represents one individual measurement at a specific time.
|
||||
|
||||
Tables are useful when you want to identify individual measurements but make it difficult to see the big picture. A more common visualization for time series is the _graph_, which instead places each measurement along a time axis. Visual representations like the graph make it easier to discover patterns and features of the data that otherwise would be difficult to see.
|
||||
|
||||
{{< docs-imagebox img="/img/docs/example_graph.png" class="docs-image--no-shadow" max-width="850px" >}}
|
||||
|
||||
Temperature data like the one in the example, is far from the only example of a time series. Other examples of time series are:
|
||||
|
||||
- CPU and memory usage
|
||||
- Sensor data
|
||||
- Stock market index
|
||||
|
||||
While each of these examples are sequences of chronologically ordered measurements, they also share other attributes:
|
||||
|
||||
- New data is appended at the end, at regular intervals—for example, hourly at 09:00, 10:00, 11:00, and so on.
|
||||
- Measurements are seldom updated after they were added—for example, yesterday's temperature doesn't change.
|
||||
|
||||
Time series are powerful. They help you understand the past by letting you analyze the state of the system at any point in time. Time series could tell you that the server crashed moments after the free disk space went down to zero.
|
||||
|
||||
Time series can also help you predict the future, by uncovering trends in your data. If the number of registered users has been increasing monthly by 4% for the past few months, you can predict how big your user base is going to be at the end of the year.
|
||||
|
||||
Some time series have patterns that repeat themselves over a known period. For example, the temperature is typically higher during the day, before it dips down at night. By identifying these periodic, or _seasonal_, time series, you can make confident predictions about the next period. If we know that the system load peaks every day around 18:00, we can add more machines right before.
|
||||
|
||||
## Aggregating time series
|
||||
|
||||
Depending on what you're measuring, the data can vary greatly. What if you wanted to compare periods longer than the interval between measurements? If you'd measure the temperature once every hour, you'd end up with 24 data points per day. To compare the temperature in August over the years, you'd have to combine the 31 times 24 data points into one.
|
||||
|
||||
Combining a collection of measurements is called _aggregation_. There are several ways to aggregate time series data. Here are some common ones:
|
||||
|
||||
- **Average** returns the sum of all values divided by the total number of values.
|
||||
- **Min** and **Max** return the smallest, and largest value in the collection.
|
||||
- **Sum** returns the sum of all values in the collection.
|
||||
- **Count** returns the number of values in the collection.
|
||||
|
||||
For example, by aggregating the data in a month, you can determine that August 2017 was, on average, warmer than the year before. Instead, to see which month had the highest temperature, you'd compare the maximum temperature for each month.
|
||||
|
||||
How you choose to aggregate your time series data is an important decision and depends on the story you want to tell with your data. It's common to use different aggregations to visualize the same time series data in different ways.
|
||||
|
||||
## Time series and monitoring
|
||||
|
||||
In the IT industry, time series data is often collected to monitor things like infrastructure, hardware, or application events. Machine-generated time series data is typically collected with short intervals, which allows you to react to any unexpected changes, moments after they occur. As a consequence, data accumulates at a rapid pace, making it vital to have a way to store and query data efficiently. As a result, databases optimized for time series data have seen a rise in popularity in recent years.
|
||||
|
||||
### Time series databases
|
||||
|
||||
A time series database (TSDB) is a database explicitly designed for time series data. While it's possible to use any regular database to store measurements, a TSDB comes with some useful optimizations.
|
||||
|
||||
Modern time series databases take advantage of the fact that measurements are only ever appended, and rarely updated or removed. For example, the timestamps for each measurement change very little over time, which results in redundant data being stored.
|
||||
|
||||
Look at this sequence of Unix timestamps:
|
||||
|
||||
```
|
||||
1572524345, 1572524375, 1572524404, 1572524434, 1572524464
|
||||
```
|
||||
|
||||
Looking at these timestamps, they all start with `1572524`, leading to poor use of disk space. Instead, we could store each subsequent timestamp as the difference, or _delta_, from the first one:
|
||||
|
||||
```
|
||||
1572524345, +30, +29, +30, +30
|
||||
```
|
||||
|
||||
We could even take it a step further, by calculating the deltas of these deltas:
|
||||
|
||||
```
|
||||
1572524345, +30, -1, +1, +0
|
||||
```
|
||||
|
||||
If measurements are taken at regular intervals, most of these delta-of-deltas will be 0. Because of optimizations like these, TSDBs uses drastically less space than other databases.
|
||||
|
||||
Another feature of a TSDB is the ability to filter measurements using _tags_. Each data point is labeled with a tag that adds context information, such as where the measurement was taken. Here's an example of the [InfluxDB data format](https://docs.influxdata.com/influxdb/v1.7/write_protocols/line_protocol_tutorial/#syntax) that demonstrates how each measurement is stored.
|
||||
|
||||
Here are some of the TSDBs supported by Grafana:
|
||||
|
||||
- [Graphite](https://graphiteapp.org/)
|
||||
- [InfluxDB](https://www.influxdata.com/products/influxdb-overview/)
|
||||
- [Prometheus](https://prometheus.io/)
|
||||
|
||||
```
|
||||
weather,location=us-midwest temperature=82 1465839830100400200
|
||||
| -------------------- -------------- |
|
||||
| | | |
|
||||
| | | |
|
||||
+-----------+--------+-+---------+-+---------+
|
||||
|measurement|,tag_set| |field_set| |timestamp|
|
||||
+-----------+--------+-+---------+-+---------+
|
||||
```
|
||||
|
||||
### Collecting time series data
|
||||
|
||||
Now that we have a place to store our time series, how do we actually gather the measurements? To collect time series data, you'd typically install a _collector_ on the device, machine, or instance you want to monitor. Some collectors are made with a specific database in mind, and some support different output destinations.
|
||||
|
||||
Here are some examples of collectors:
|
||||
|
||||
- [collectd](https://collectd.org/)
|
||||
- [statsd](https://github.com/statsd/statsd)
|
||||
- [Prometheus exporters](https://prometheus.io/docs/instrumenting/exporters/)
|
||||
- [Telegraf](https://github.com/influxdata/telegraf)
|
||||
|
||||
A collector either _pushes_ data to a database or lets the database _pull_ the data from it. Both methods come with their own set of pros and cons:
|
||||
|
||||
| | Pros | Cons |
|
||||
| ---- | ------------------------------------------------------------ | ------------------------------------------------------------ |
|
||||
| Push | Easier to replicate data to multiple destinations. | The TSDB has no control over how much data gets sent. |
|
||||
| Pull | Better control of how much data that gets ingested, and its authenticity. | Firewalls, VPNs or load balancers can make it hard to access the agents. |
|
||||
|
||||
Since it would be inefficient to write every measurement to the database, collectors pre-aggregate the data and write to the time series database at regular intervals.
|
||||
Reference in New Issue
Block a user