grafana/devenv
Chris Cowan f580c9149c
Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041)
* Elasticsearch: Add Top Metrics Aggregation

* Adding support for non-timeseries visualizations

* removing console.logs

* restoring loadOptions type

* Honor xpack setting

* Adding test for elastic_response

* adding test for query builder

* Adding support of alerting

* Fixing separator spelling

* Fixing linting issues

* attempting to reduce cyclomatic complexity

* Adding elastic77 Docker block

* Update public/app/plugins/datasource/elasticsearch/components/QueryEditor/MetricAggregationsEditor/MetricEditor.test.tsx

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* refactoring MetricsEditor tests

* Fixing typo

* Change getFields type & move TopMetrics to a separate component

* Fix SegmentAsync styles in TopMetrics Settings

* Fix field types for TopMetrics

* WIP

* Refactoring client side to support multiple top metrics

* Adding tests and finishing go implimentation

* removing fmt lib from debugging

* fixing tests

* reducing the cyclomatic complexity

* Update public/app/plugins/datasource/elasticsearch/elastic_response.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Update public/app/plugins/datasource/elasticsearch/hooks/useFields.ts

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>

* Checking for possible nil value

* Fixing types

* fix fake-data-gen param

* fix useFields hook

* Removing aggregateBy and size

* Fixing go tests

* Fixing TS tests

* fixing tests

* Fixes

* Remove date from top_metrics fields

* Restore previous formatting

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Update pkg/tsdb/elasticsearch/client/models.go

Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>

* Fix code review comments on processTopMetricValue

* Remove underscore from variable names

* Remove intermediate array definition

* Refactor test to use testify

Co-authored-by: Giordano Ricci <grdnricci@gmail.com>
Co-authored-by: Elfo404 <me@giordanoricci.com>
Co-authored-by: Dimitris Sotirakis <dimitrios.sotirakis@grafana.com>
2021-06-04 11:07:59 +01:00
..
benchmarks/ab moves benchmark script to devenv 2018-09-17 11:17:23 +02:00
bulk_alerting_dashboards Support using multiple datasources for testing bulk alerting. (#23258) 2020-04-01 20:55:37 +02:00
bulk-dashboards Fix bulk-dashboards path (#12978) 2018-08-20 19:21:31 +02:00
dev-dashboards Rename: Status grid to history (#34864) 2021-05-29 20:24:14 +02:00
dev-dashboards-without-uid Units: add inches (in) and pounds (lb) (#32196) 2021-03-22 08:28:39 -07:00
docker Elasticsearch: Add Top Metrics Aggregation and X-Pack support (#33041) 2021-06-04 11:07:59 +01:00
e2e-api-tests Prettier: Upgrade to 2 (#30387) 2021-01-20 07:59:48 +01:00
local-npm Add guide and tooling for local packages registry setup (#26110) 2020-07-07 14:07:00 +02:00
create_docker_compose.sh Build: allow dynamically change docker image (#18112) 2019-07-16 08:16:10 +02:00
dashboards.yaml Allow saving of provisioned dashboards (#19820) 2019-10-31 14:27:31 +01:00
datasources_docker.yaml Elasticsearch: Fix handling of inline scripts in different ES versions (#34070) 2021-05-14 11:50:15 +01:00
datasources.yaml Elasticsearch: Fix handling of inline scripts in different ES versions (#34070) 2021-05-14 11:50:15 +01:00
README.md DevEnv: Fix docker-driver plugin installation direction url 2021-06-02 13:31:04 +02:00
setup.sh devenv: Fix typo (#31589) 2021-03-02 17:09:27 +01:00

Set up your development environment

This folder contains useful scripts and configuration so you can:

  • Configure data sources in Grafana for development.
  • Configure dashboards for development and test scenarios.
  • Create docker-compose file with databases and fake data.

Install Docker

Grafana uses Docker to make the task of setting up databases a little easier. If you do not have it already, make sure you install Docker before proceeding to the next step.

Developer dashboards and data sources

./setup.sh

After restarting the Grafana server, there should be a number of data sources named gdev-<type> provisioned as well as a dashboard folder named gdev dashboards. This folder contains dashboard and panel features tests dashboards.

Please update these dashboards or make new ones as new panels and dashboards features are developed or new bugs are found. The dashboards are located in the devenv/dev-dashboards folder.

docker-compose with databases

This command creates a docker-compose file with specified databases configured and ready to run. Each database has a prepared image with some fake data ready to use. For available databases, see docker/blocks directory. Notice that for some databases there are multiple images with different versions. Some blocks such as slow_proxy_mac or apache_proxy_mac are specifically for Macs.

make devenv sources=influxdb,prometheus,elastic5

Some of the blocks support dynamic change of the image version used in the Docker file. The signature looks like this:

make devenv sources=postgres,openldap,grafana postgres_version=9.2 grafana_version=6.7.0-beta1

Notes per block

Grafana

The grafana block is pre-configured with the dev-datasources and dashboards.

Jaeger

Jaeger block runs both Jaeger and Loki container. Loki container sends traces to Jaeger and also logs its own logs into itself so it is possible to setup derived field for traceID from Loki to Jaeger. You need to install a docker plugin for the self logging to work, without it the container won't start. See https://grafana.com/docs/loki/latest/clients/docker-driver/#installing for installation instructions.

Graphite

version source name graphite-web port plaintext port pickle port
1.1 graphite 8180 2103 2103
1.0 graphite1 8280 2203 2203
0.9 graphite09 8380 2303 2303

Troubleshooting

Containers fail to start (Mac OS)

ERROR: for <service_name> Cannot start service <service_name>: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting ... merged/var/log/grafana: operation not permitted\\\"\"": unknown
ERROR: Encountered errors while bringing up the project.

If running Mac OSX the above error might be encountered when starting certain Docker containers that mount /var/log/. When first run this causes Docker to try to create the folder /var/log/grafana however by default Docker for Mac does not have permission to create folders at this location as it runs as the current user.

To solve this issue manually create the folder /var/log/grafana and give your user write permissions then try starting the containers again.