Torkel Ödegaard 140ecbcf79 QueryProcessing: Observable query interface and RxJS for query & stream processing (#18899)
* I needed to learn some rxjs and understand this more, so just playing around

* Updated

* Removed all the complete calls

* Refactoring

* StreamHandler -> observable start

* progress

* simple singal works

* Handle update time range

* added error handling

* wrap old function

* minor changes

* handle data format in the subscribe function

* Use replay subject to return last value to subscribers

* Set loading state after no response in 50ms

* added missing file

* updated comment

* Added cancelation of network requests

* runRequest: Added unit test scenario framework

* Progress on tests

* minor refactor of unit tests

* updated test

* removed some old code

* Shared queries work again, and also became so much simplier

* unified query and observe methods

* implict any fix

* Fixed closed subject issue

* removed comment

* Use last returned data for loading state

* WIP: Explore to runRequest makover step1

* Minor progress

* Minor progress on explore and runRequest

* minor progress

* Things are starting to work in explore

* Updated prometheus to use new observable query response, greatly simplified code

* Revert refId change

* Found better solution for key/refId/requestId problem

* use observable with loki

* tests compile

* fix loki query prep

* Explore: correct first response handling

* Refactorings

* Refactoring

* Explore: Fixes LoadingState and GraphResults between runs (#18986)

* Refactor: Adds state to DataQueryResponse

* Fix: Fixes so we do not empty results before new data arrives
Fixes: #17409

* Transformations work

* observable test data

* remove single() from loki promise

* Fixed comment

* Explore: Fixes failing Loki and Prometheus unit tests (#18995)

* Tests: Makes datasource tests work again

* Fix: Fixes loki datasource so highligthing works

* Chore: Runs Prettier

* Fixed query runner tests

* Delay loading state indication to 200ms

* Fixed test

* fixed unit tests

* Clear cached calcs

* Fixed bug getProcesedDataFrames

* Fix the correct test is a better idea

* Fix: Fixes so queries in Explore are only run if Graph/Table is shown (#19000)

* Fix: Fixes so queries in Explore are only run if Graph/Table is shown
Fixes: #18618

* Refactor: Removes unnecessary condition

* PanelData: provide legacy data only when needed  (#19018)

* no legacy

* invert logic... now compiles

* merge getQueryResponseData and getDataRaw

* update comment about query editor

* use single getData() function

* only send legacy when it is used in explore

* pre process rather than post process

* pre process rather than post process

* Minor refactoring

* Add missing tags to test datasource response

* MixedDatasource: Adds query observable pattern to MixedDatasource (#19037)

* start mixed datasource

* Refactor: Refactors into observable parttern

* Tests: Fixes tests

* Tests: Removes console.log

* Refactor: Adds unique requestId
2019-09-12 17:28:46 +02:00
2018-09-22 00:54:56 +03:00
2019-03-27 17:53:49 +01:00
2019-03-08 13:56:21 +01:00
2019-09-02 12:27:55 +02:00
2019-03-21 16:36:24 +01:00
2018-08-09 10:51:04 +02:00
2019-03-20 12:24:54 -07:00

Grafana Circle CI Go Report Card

Website | Twitter | Community & Forum

Grafana is an open source, feature rich metrics dashboard and graph editor for Graphite, Elasticsearch, OpenTSDB, Prometheus and InfluxDB.

Installation

Head to docs.grafana.org for documentation or download to get the latest release.

Documentation & Support

Be sure to read the getting started guide and the other feature guides.

Run from master

If you want to build a package yourself, or contribute - here is a guide for how to do that. You can always find the latest master builds here

Dependencies

  • Go (Latest Stable)
  • Node.js LTS
    • yarn [npm install -g yarn]

Get the project

The project located in the go-path will be your working directory.

go get github.com/grafana/grafana
cd $GOPATH/src/github.com/grafana/grafana

Run and rebuild on source change

Backend

To run the backend and rebuild on source change:

make run

Frontend

Install front-end dependencies first:

yarn install --pure-lockfile

Rebuild on file change, and serve them by Grafana's webserver (http://localhost:3000):

yarn start

Build the assets, rebuild on file change with Hot Module Replacement (HMR), and serve them by webpack-dev-server (http://localhost:3333):

yarn start:hot
# OR set a theme
env GRAFANA_THEME=light yarn start:hot

Note: HMR for Angular is not supported. If you edit files in the Angular part of the app, the whole page will reload.

Run tests and rebuild on source change:

yarn jest

Open grafana in your browser (default: e.g. http://localhost:3000) and login with admin user (default: user/pass = admin/admin).

Building

The backend

go run build.go setup
go run build.go build

Frontend assets

For this you need Node.js (LTS version).

yarn install --pure-lockfile

Building a Docker image

There are two different ways to build a Grafana docker image. If your machine is setup for Grafana development and you run linux/amd64 you can build just the image. Otherwise, there is the option to build Grafana completely within Docker.

Run the image you have built using: docker run --rm -p 3000:3000 grafana/grafana:dev

Building on linux/amd64 (fast)

  1. Build the frontend go run build.go build-frontend
  2. Build the docker image make build-docker-dev

The resulting image will be tagged as grafana/grafana:dev

Building anywhere (slower)

Choose this option to build on platforms other than linux/amd64 and/or not have to setup the Grafana development environment.

  1. make build-docker-full or docker build -t grafana/grafana:dev .

The resulting image will be tagged as grafana/grafana:dev

Notice: If you are using Docker for MacOS, be sure to set the memory limit to be larger than 2 GiB (at docker -> Preferences -> Advanced), otherwise grunt build may fail.

Development

Dev config

Create a custom.ini in the conf directory to override default configuration options. You only need to add the options you want to override. Config files are applied in the order of:

  1. grafana.ini
  2. custom.ini

In your custom.ini uncomment (remove the leading ;) sign. And set app_mode = development.

Running tests

Frontend

Execute all frontend tests

yarn test

Writing & watching frontend tests

  • Start watcher: yarn jest
  • Jest will run all test files that end with the name ".test.ts"

Backend

# Run Golang tests using sqlite3 as database (default)
go test ./pkg/...
Running the MySQL or Postgres backend tests:

Run these by setting GRAFANA_TEST_DB in your environment.

  • GRAFANA_TEST_DB=mysql to test MySQL
  • GRAFANA_TEST_DB=postgres to test Postgres

Follow the instructions in ./devenv to spin up test containers running the appropriate databases with docker-compose

  • Use docker/blocks/mysql_tests or docker/blocks/postgres_tests as appropriate
# MySQL
# Tests can only be ran in one Go package at a time due to clashing db queries. To run MySQL tests for the "pkg/services/sqlstore" package, run:
GRAFANA_TEST_DB=mysql go test ./pkg/services/sqlstore/...

# Or run all the packages using the circle CI scripts. This method will be slower as the scripts will run all the tests, including the integration tests.
./scripts/circle-test-mysql.sh
# Postgres
# Tests can only be ran in one Go package at a time due to clashing db queries. To run Postgres tests for the "pkg/services/sqlstore" package, run:
GRAFANA_TEST_DB=postgres go test ./pkg/services/sqlstore/...

# Or run all the packages using the circle CI scripts. This method will be slower as the scripts will run all the tests, including the integration tests.
./scripts/circle-test-postgres.sh

End-to-end

Execute all end-to-end tests

yarn e2e-tests

Execute all end-to-end tests using using a specific url

ENV BASE_URL=http://localhost:3333 yarn e2e-tests

Debugging all end-to-end tests (BROWSER=1 will start the browser and SLOWMO=1 will delay each puppeteer operation by 100ms)

ENV BROWSER=1 SLOWMO=1 yarn e2e-tests

Datasource and dashboard provisioning

Here you can find helpful scripts and docker-compose setup that will populate your dev environment for quicker testing and experimenting.

Contribute

If you have any ideas for improvement or have found a bug, do not hesitate to open an issue. And if you have time, clone this repo and submit a pull request to help me make Grafana the kickass metrics & devops dashboard we all dream about!

Read the contributing guide then check the beginner friendly label to find issues that are easy and that we would like help with.

Plugin development

Checkout the Plugin Development Guide and checkout the PLUGIN_DEV.md file for changes in Grafana that relate to plugin development.

License

Grafana is distributed under Apache 2.0 License.

Description
The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
Readme 1.9 GiB
Languages
TypeScript 56.6%
Go 40.2%
Rich Text Format 0.7%
HTML 0.6%
CUE 0.4%
Other 1.3%