diff --git a/contribute/developer-guide.md b/contribute/developer-guide.md index 650b9666263..6df07b690e0 100644 --- a/contribute/developer-guide.md +++ b/contribute/developer-guide.md @@ -172,7 +172,7 @@ make test-go-integration-postgres ### Run end-to-end tests -The end to end tests in Grafana use [Cypress](https://www.cypress.io/) and [Playwright](https://playwright.dev/) to run automated scripts in a browser. Read more about our Cypress [e2e framework](/contribute/style-guides/e2e.md). +Grafana uses [Cypress](https://www.cypress.io/) to end-to-end test core features. Core plugins use [Playwright](https://playwright.dev/) to run automated end-to-end tests. You can find more information on how to add end-to-end tests to your core plugin [here](./style-guides/e2e-plugins.md) #### Running Cypress tests diff --git a/contribute/style-guides/e2e-core.md b/contribute/style-guides/e2e-core.md index fb2c24e0bfc..321f02522c9 100644 --- a/contribute/style-guides/e2e-core.md +++ b/contribute/style-guides/e2e-core.md @@ -22,4 +22,4 @@ The above commands use some utils scripts under [_\/e2e_](../../e2e) ## Test suites -All the integration tests are located at _\/e2e/suite\/specs_. The page objects and reusable flows are in the [_\/packages/grafana-e2e_](../../packages/grafana-e2e) package. +All the integration tests are located at _\/e2e/suite\/specs_. diff --git a/contribute/style-guides/e2e-plugins.md b/contribute/style-guides/e2e-plugins.md index a40b9516fa7..f675db96784 100644 --- a/contribute/style-guides/e2e-plugins.md +++ b/contribute/style-guides/e2e-plugins.md @@ -1,28 +1,35 @@ -# End-to-End Tests for plugins +# end-to-end tests for plugins -Be sure that you've read the [generalized E2E document](e2e.md). +When end-to-end testing Grafana plugins, it's recommended to use the [`@grafana/plugin-e2e`](https://www.npmjs.com/package/@grafana/plugin-e2e?activeTab=readme) testing tool. `@grafana/plugin-e2e` extends [`@playwright/test`](https://playwright.dev/) capabilities with relevant fixtures, models, and expect matchers; enabling comprehensive end-to-end testing of Grafana plugins across multiple versions of Grafana. For information on how to get started with Plugin end-to-end testing and Playwright, checkout the [Get started](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/get-started) guide. + +## Adding end-to-end tests for a core plugin + +Playwright end-to-end tests for plugins should be added to the [`e2e/plugin-e2e`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e) directory. + +1. Add a new directory that has the name as your plugin [`here`](https://github.com/grafana/grafana/tree/main/e2e/plugin-e2e). This is where your plugin tests will be kept. + +2. Playwright uses [projects](https://playwright.dev/docs/test-projects) to logically group tests together. All tests in a project share the same configuration. + In the [Playwright config file](https://github.com/grafana/grafana/blob/main/playwright.config.ts), add a new project item. Make sure the `name` and the `testDir` sub directory matches the name of the directory that contains your plugin tests. + Adding `'authenticate'` to the list of dependencies and specifying `'playwright/.auth/admin.json'` as storage state will ensure all tests in your project will start already authenticated as an admin user. If you wish to use a different role for and perhaps test RBAC for some of your tests, please refer to the plugin-e2e [documentation](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/use-authentication). + + ```ts + { + name: 'mysql', + testDir: path.join(testDirRoot, '/mysql'), + use: { + ...devices['Desktop Chrome'], + storageState: 'playwright/.auth/admin.json', + }, + dependencies: ['authenticate'], + }, + ``` + +3. Update the [CODEOWNERS](https://github.com/grafana/grafana/blob/main/.github/CODEOWNERS/#L315) file so that your team is owner of the tests in the directory you added in step 1. ## Commands -- `yarn test:e2e` will run [Grafana's E2E utility](../../packages/grafana-e2e) against an already running Grafana server. -- `yarn test:e2e:update` will run `test:e2e` but instead of asserting that screenshots match their expected fixtures, they'll be replaced with new ones. +- `yarn e2e:playwright` will run all Playwright tests. Optionally, you can provide the `--project mysql` argument to run tests in a certain project. -Your running Grafana instance can be targeted by setting the `CYPRESS_BASE_URL`, `CYPRESS_USERNAME` and `CYPRESS_PASSWORD` environment variableS: +The script above assumes you have Grafana running on `localhost:3000`. You may change this by providing environment variables. -```shell -CYPRESS_BASE_URL=https://localhost:3000 CYPRESS_USERNAME=admin CYPRESS_PASSWORD=admin yarn test:e2e -``` - -## Test suites - -All tests are located at _\/cypress/integration_ by default. - -## Things to test - -- Add data source (if applicable) -- Add panel -- Edit panel -- Annotations (if applicable) -- Aliases (if applicable) -- Template variables -- "Explore" view +`HOST=127.0.0.1 PORT=3001 yarn e2e:playwright` diff --git a/contribute/style-guides/e2e.md b/contribute/style-guides/e2e.md index 76958b6d53b..285936f2c5b 100644 --- a/contribute/style-guides/e2e.md +++ b/contribute/style-guides/e2e.md @@ -1,13 +1,13 @@ # End-to-End tests -Grafana Labs uses a minimal [homegrown solution](../../packages/grafana-e2e) built on top of [Cypress](https://cypress.io) for its end-to-end (E2E) tests. +Grafana Labs uses a minimal [homegrown solution](../../e2e/utils/index.ts) built on top of [Cypress](https://cypress.io) for its end-to-end (E2E) tests. Important notes: - We generally store all element identifiers ([CSS selectors](https://mdn.io/docs/Web/CSS/CSS_Selectors)) within the framework for reuse and maintainability. - We generally do not use stubs or mocks as to fully simulate a real user. - Cypress' promises [do not behave as you'd expect](https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Mixing-Async-and-Sync-code). -- [Testing core Grafana](e2e-core.md) is slightly different than [testing plugins](e2e-plugins.md). +- [Testing core Grafana](e2e-core.md) is different than [testing plugins](e2e-plugins.md) - core Grafana uses Cypress whereas plugins use [Playwright test](https://playwright.dev/). ## Framework structure diff --git a/packages/grafana-e2e/README.md b/packages/grafana-e2e/README.md index 48e8a285956..a488db87725 100644 --- a/packages/grafana-e2e/README.md +++ b/packages/grafana-e2e/README.md @@ -1,5 +1,5 @@ # Grafana End-to-End Test library -> **@grafana/e2e is currently in BETA**. - -This package contains an API wrapper built on top of [Cypress](https://www.cypress.io) that simplifies creating end-to-end tests for Grafana. More information can be found [here](https://github.com/grafana/grafana/blob/main/contribute/style-guides/e2e.md). +> [!CAUTION] +> This package is deprecated. +> If you'd like to write end-to-end tests for a Grafana plugin (core or external), use the [`@grafana/plugin-e2e`](https://grafana.com/developers/plugin-tools/e2e-test-a-plugin/introduction) package.