grafana/docs/sources/tutorials/stack_guide_graphite.md

---
page_title: Graphite + Grafana + StatsD - Stack Setup Guide
page_description: Installation and configuration guide and how to for Grafana, Graphite and StatsD
page_keywords: grafana, tutorials, graphite, statsd, setup, configuration, howto, installation
author: Torkel Ödegaard
---

# Stack Setup and Config Guide: Graphite + Grafana + StatsD

This lengthy article will guide you through installation, configuration and getting started with the amazing metric
stack that is composed of Graphite, Grafana and StatsD.

Graphite is still king when it comes to time series databases due to its simple data model, ingestion
with integrated aggregation and rollups, amazing query features and speed. No other time series
database has yet to match Graphite's query flexibility and analytics potential.

Graphite has a reputation for being tricky to install and scale. This guide aims to show
that is not really the case, or, at least, that it is a lot better than you expect.

> This guide does not only aim to be only be an install guide but to also teach you
> of the mechanics of metric collection, aggregation and querying. How Graphite
> stores and aggregates data is very important to understand in order to not
> get mislead by graphs.

## Installation - Ubuntu

To begin with we are going to install the 3 main components that define our metric stack. Later in the guide we will
install StatsD, but that is optional.

- Carbon is the graphite ingestion daemon responsible for
receiving metrics and storing them.
- Graphite-api is light weight version of graphite-web with only the HTTP API and is
responsible for executing metric queries.
- Grafana as the frontend to visualize metrics and the tool to help you build metric
queries that will make the most out of your collected metrics.

### Carbon

Graphite and Carbon are written in python, so we will start by installing python packages.

```
apt-get install \
    git \
    build-essential \
    libffi-dev libcairo2-dev \
    python-django \
    python-django-tagging \
    python-simplejson \
    python-memcache \
    python-ldap \
    python-cairo \
    python-twisted \
    python-pysqlite2 \
    python-support \
    python-dev \
    python-pip
```

Next we will clone carbon and whisper and install these components. Whisper is just a lib used
by carbon to write metrics to disk.

    cd /usr/local/src
    git clone https://github.com/graphite-project/carbon.git
    git clone https://github.com/graphite-project/whisper.git

    cd whisper && python setup.py install && cd ..
    cd carbon && python setup.py install && cd ..

### Configure carbon.conf

Copy example carbon config:
```
cp /opt/graphite/conf/carbon.conf.example /opt/graphite/conf/carbon.conf
```

Edit the config file `/opt/graphite/conf/carbon.conf`, find line `ENABLE_UDP_LISTENER` and
change this setting to `True`.

### Configure storage-schemas.conf

Create a new file at `/opt/graphite/conf/storage-schemas.conf` with the following content:

```
[carbon]
pattern = ^carbon\..*
retentions = 1m:30d,10m:1y,1h:5y

[default]
pattern = .*
retentions = 10s:1d,1m:7d,10m:1y
```

This config specifies the resolution of metrics and the retention periods. For example for all metrics beginning with the word `carbon` receive metrics every minute and store for 30 days, then
roll them up into 10 minute buckets and store those for 1 year, then roll those up into 1 hour buckets and store those for 5 years. For all other metrics
the default rule will be applied with other retention periods.

This configuration is very important, as the first retention period must match the rate at which you send metrics. The default rule has 10 seconds
as its first resolution so when configuring StatsD we should configure it to send metrics every 10 seconds.

> If you send values more frequently than the highest resolution, for example if you send data every second but
> the storage schema rules defines the highest resolution to be 10 seconds, then the values you send will just
> overwrite each other and the last value sent during every 10 second period will be saved. StatsD can work around this
> problem.

### Configure storage-aggregation.conf

Copy the default config and open it in an editor.
```
cp /opt/graphite/conf/storage-aggregation.conf.example /opt/graphite/conf/storage-aggregation.conf
```

Example config:
```
[min]
pattern = \.min$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = \.max$
xFilesFactor = 0.1
aggregationMethod = max

[sum]
pattern = \.count$
xFilesFactor = 0
aggregationMethod = sum

[default_average]
pattern = .*
xFilesFactor = 0.5
aggregationMethod = average
```

You do not really need to change the default config, but is very important to
understand what the config controls and what implications that it has. Graphite
does rollups as part of the metric ingestion according to the rules defined in
`storage-schemas.conf`. For example, given storage schema rule `10s:1d,1m:7d`,
when aggregating 6 values (each representing 10 seconds) into a 1min bucket graphite
will use an `aggregationMethod` like for example `average`. What method to use
will be determined by the rules specified in `storage-aggregation.conf`.

The default rules all look at the metric path ending. Does it end with `.count` then use `sum` when doing rollups, does it end with `max` then use `max` function, and if it does not
end with max, min or count then use average. This means that naming metrics is very important! But don't worry if you use StatsD it will send the correct names to graphite.

### Start carbon
Lets install supervisord and let it start carbon.

`apt-get install supervisor`

Create a new file in `/etc/supervisor/conf.d/carbon.conf` with the following:

```
[program:carbon-cache]
command = /opt/graphite/bin/carbon-cache.py --debug start
stdout_logfile = /var/log/supervisor/%(program_name)s.log
stderr_logfile = /var/log/supervisor/%(program_name)s.log
autorestart = true
stopsignal = QUIT
```

```
supervisorctl reload
```

### Graphite-api

Graphite api is a light weight version of graphite-web with only the API component (no web ui). It is dead simple
to install.

```
pip install gunicorn graphite-api
```

You should now have a graphite-api daemon running with an open HTTP API port of 8888.

### Configuring Graphite-api

Create a file `/etc/graphite-api.yaml` with an editor and set it's content to:

```
search_index: /opt/graphite/storage/index
finders:
  - graphite_api.finders.whisper.WhisperFinder
functions:
  - graphite_api.functions.SeriesFunctions
  - graphite_api.functions.PieFunctions
whisper:
  directories:
    - /opt/graphite/storage/whisper
time_zone: UTC
```

Lets create a supervisor file for graphite-api at `/etc/supervisor/graphite-api.conf`

```
[program:graphite-api]
command = gunicorn -b 0.0.0.0:8888 -w 2 --log-level info graphite_api.app:app
stdout_logfile = /var/log/supervisor/%(program_name)s.log
stderr_logfile = /var/log/supervisor/%(program_name)s.log
autorestart = true
stopsignal = QUIT
```

Reload supervisor

    supervisorctl reload

A carbon-cache daemon and graphite-api should now be running. Type `supervisorctl status` to verify that they are running. You can
also open `http://your_server_ip:8888/metrics/find?query?*` in your browser. You should see a json snippet.


### Install Grafana

    cd /tmp/
    wget https://grafanarel.s3.amazonaws.com/builds/grafana_2.1.1_amd64.deb
    sudo dpkg -i grafana_2.1.1_amd64.deb
    sudo service grafana-server start

Grafana should now be running with default config on port 3000.

## Grafana - first steps

### Add data source

Open http://your_server_ip:3000 in your browser and login with the default user and password (`admin/admin`).

- Click on `Data Sources` on the side menu.
- Click on `Add new` in the top menu
- Specify name `graphite` and check the `Default ` checkbox
- Specify URL `http://localhost:8888` and Access `proxy`
- Click `Add ` button

### Your first dashboard

- Click on `Dashboards`
- Click on `Home` button in the top menu, this should open the dashboard search dropdown
- Click on `New` button in the bottom of this dropdown

### Add a graph

- Click on the green icon to the left to open the row menu
- Select `Add Panel` > `Graph` from the row menu
- An empty graph panel should appear with title `no title (click here)`. Click on this title and then `Edit`
- This will open the graph in edit mode and take you to the metrics tab.
- There is one query already added (assigned letter A) but it is empty.
- Click on `select metric` to pick the first graphite metric node. A new `select metric` link will appear until you reached a leaf node.
- Try picking the metric paths for `carbon.agents.<server name>.cpuUsage`, you should now see a line appear in the graph!

## Writing metrics to Graphite
Graphite has the simplest metric write protocol imaginable. Something that has surely contributed to its wide adoption by metric
frameworks and numerous integrations.

    prod.server1.requests.count 10 1398969187

    <metric.name.and.path> <metric value> <unix_epoch_time_stamp_in_seconds>

There are hundreds of tools and instrumentation frameworks that can send metrics using this protocol.

### Installing StatsD
StatsD is a metrics aggregation daemon that makes it easy for apps on many machines to send measurements like timings and counters and have them aggregated or percentiles calculated.

### Sending metrics to StatsD