E2E: Add plugin-e2e scenario verification tests (#79969)

* add playwright test and plugin-e2e

* run tests in ci

* add ds config tests

* add panel edit tests

* add annotation test

* add variable edit page tests

* add explore page tests

* add panel plugin tests

* add readme

* remove comments

* fix broken test

* remove user.json

* remove newline in starlark

* fix lint issue

* ignore failure of playwright tests

* update code owners

* add detailed error messages in every expect

* update message frame

* fix link

* upload report to gcp

* echo url

* add playwright developer guide

* bump plugin-e2e

* add custom provisioning dir

* update plugin-e2e

* remove not used imports

* fix typo

* minor fixes

* use latest version of plugin-e2e

* fix broken link

* use latest plugin-e2e

* add feature toggle scenario verification tests

* bump version

* use auth file from package

* fix type error

* add panel data assertions

* rename parent dir and bump version

* fix codeowners

* reset files

* remove not used file

* update plugin-e2e

* separate tests per role

* pass prov dir

* skip using provisioning fixture

* wip

* fix permission test

* move to e2e dir

* fix path to readme

* post comment with report url

* format starlark

* post comment with report url

* post comment with report url

* fix token

* make test fail

* fix exit code

* bump version

* bump to latest plugin-e2e

* revert reporting message

* remove comments

* readding report comment

* change exit code

* format starlark

* force test to fail

* add new step that posts comment

* fix link

* use latest playwright image

* fix failing test

* format starlark

* remove unused fixture

Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>

---------

Co-authored-by: Marcus Andersson <marcus.andersson@grafana.com>
This commit is contained in:
Erik Sundell
2024-02-23 12:39:30 +01:00
committed by GitHub
co-authored by Marcus Andersson
parent b25667223c
commit 3e456127cb
22 changed files with 2347 additions and 212 deletions
+32 -2
View File
@@ -171,9 +171,11 @@ make test-go-integration-postgres
### Run end-to-end tests
The end to end tests in Grafana use [Cypress](https://www.cypress.io/) to run automated scripts in a headless Chromium browser. Read more about our [e2e framework](/contribute/style-guides/e2e.md).
The end to end tests in Grafana use [Cypress](https://www.cypress.io/) and [Playwright](https://playwright.dev/) to run automated scripts in a browser. Read more about our Cypress [e2e framework](/contribute/style-guides/e2e.md).
To run the tests:
#### Running Cypress tests
To run all tests in a headless Chromium browser.
```
yarn e2e
@@ -197,6 +199,34 @@ To choose a single test to follow in the browser as it runs, use `yarn e2e:dev`
yarn e2e:dev
```
#### To run the Playwright tests:
**Note:** If you're using VS Code as your development editor, it's recommended to install the [Playwright test extension](https://marketplace.visualstudio.com/items?itemName=ms-playwright.playwright). It allows you to run, debug and generate Playwright tests from within the editor. For more information about the extension and how to install it, refer to the [Playwright documentation](https://playwright.dev/docs/getting-started-vscode).
Each version of Playwright needs specific versions of browser binaries to operate. You will need to use the Playwright CLI to install these browsers.
```
yarn playwright install chromium
```
To run all tests in a headless Chromium browser and display results in the terminal.
```
yarn e2e:playwright
```
For a better developer experience, open the Playwright UI where you can easily walk through each step of the test and visually see what was happening before, during and after each step.
```
yarn e2e:playwright:ui
```
To open the HTML reporter for the last test run session.
```
yarn e2e:playwright:report
```
## Configure Grafana for development
The default configuration, `defaults.ini`, is located in the `conf` directory.