grafana/contribute/style-guides/e2e.md

End-to-end tests

Grafana Labs uses a minimal homegrown solution built on top of Cypress for its end-to-end (E2E) tests.

Important notes:

• We generally store all element identifiers (CSS selectors) within the framework for reuse and maintainability.
• We generally do not use stubs or mocks as to fully simulate a real user.
• Cypress' promises don't behave as you might expect.
• Testing core Grafana is different than testing plugins—core Grafana uses Cypress whereas plugins use Playwright test.

Framework structure

Our framework structure is inspired by Martin Fowler's Page Object.

• Selector: A unique identifier that is used from the E2E framework to retrieve an element from the browser
• Page: An abstraction for an object that contains one or more Selector identifiers with the visit function to go to the page.
• Component: An abstraction for an object that contains one or more Selector identifiers but without the visit function
• Flow: An abstraction that contains a sequence of actions on one or more Page abstractions that can be reused and shared between tests

Basic example

Let's start with a simple JSX example containing a single input field that we want to populate during our E2E test:

<input className="gf-form-input login-form-input" type="text" />

It is possible to target the field with a CSS selector like .gf-form-input.login-form-input. However, doing so is a brittle solution because style changes occur frequently.

Furthermore, there is nothing that signals to future developers that this input is part of an E2E test. At Grafana, we use data-testid attributes as our preferred way of defining selectors. See Aria-Labels vs data-testid for more details.

<input data-testid="Username input field" className="gf-form-input login-form-input" type="text" />

The next step is to create a Page representation in our E2E framework. Doing so glues the test with the real implementation using the pageFactory function. For that function we can supply a url and selector like in the following example:

export const Login = {
  // Called via `Login.visit()`
  url: '/login',
  // Called via `Login.username()`
  username: 'data-testid Username input field',
};

In this example, the selector is prefixed with data-testid. The prefix is a signal to the framework to look for the selector in the data-testid attribute.

The next step is to add the Login page to the Pages export within <repo-root>/packages/grafana-e2e-selectors/src/selectors/pages.ts so that it appears when we type e2e.pages in your IDE.

export const Pages = {
  Login,
  …,
  …,
  …,
};

Now that we have a page called Login in our Pages const, use it to add a selector in our HTML as shown in the following example. This page really signals to future developers that it is part of an E2E test.

Example:

import { selectors } from '@grafana/e2e-selectors';

<input data-testid={selectors.pages.Login.username} className="gf-form-input login-form-input" type="text" />;

The last step in our example is to use our Login page as part of a test.

• Use the url property whenever you call the visit function. It is equivalent to the cy.visit() in Cypress.
• Access any defined selector from the Login page by invoking it. This is equivalent to the result of the Cypress function cy.get(…).

describe('Login test', () => {
  it('passes', () => {
    e2e.pages.Login.visit();
    // To prevent flaky tests, always do a `.should` on any selector that you expect to be in the DOM.
    // Read more here: https://docs.cypress.io/guides/core-concepts/retry-ability.html#Commands-vs-assertions
    e2e.pages.Login.username().should('be.visible').type('admin');
  });
});

Advanced example

Let's take a look at an example that uses the same selector for multiple items in a list for instance. In this example app, there's a list of data sources that we want to click on during an E2E test.

<ul>
  {dataSources.map(({ id, name }) => (
    <li className="card-item-wrapper" key={id}>
      <a className="card-item" href={`datasources/edit/${id}`}>
        <div className="card-item-name">{name}</div>
      </a>
    </li>
  ))}
</ul>

Like in the basic example, start by creating a page abstraction using the pageFactory function:

export const DataSources = {
  url: '/datasources',
  dataSources: (dataSourceName: string) => `data-testid Data source list item ${dataSourceName}`,
};

You might have noticed that instead of a simple string as the selector, there's a function that takes a string parameter as an argument and returns a formatted string using the argument.

Just as before, you need to add the DataSources page to the exported const Pages in packages/grafana-e2e-selectors/src/selectors/pages.ts.

The next step is to use the dataSources selector function as in the following example:

<ul>
  {dataSources.map(({ id, name }) => (
    <li className="card-item-wrapper" key={id}>
      <a className="card-item" href={`datasources/edit/${id}`}>
        <div className="card-item-name" data-testid={selectors.pages.DataSources.dataSources(name)}>
          {name}
        </div>
      </a>
    </li>
  ))}
</ul>

When this list is rendered with the data sources with names A, B and C ,the resulting HTML looks like this:

<div class="card-item-name" data-testid="data-testid Data source list item A">A</div>
<div class="card-item-name" data-testid="data-testid Data source list item B">B</div>
<div class="card-item-name" data-testid="data-testid Data source list item C">C</div>

Now we can write our test. The one thing that differs from the previous basic example is that we pass in which data source we want to click as an argument to the selector function:

describe('List test', () => {
  it('clicks on data source named B', () => {
    e2e.pages.DataSources.visit();
    // To prevent flaky tests, always do a .should on any selector that you expect to be in the DOM.
    // Read more here: https://docs.cypress.io/guides/core-concepts/retry-ability.html#Commands-vs-assertions
    e2e.pages.DataSources.dataSources('B').should('be.visible').click();
  });
});

aria-label versus data-testid

Our selectors are set up to work with both aria-label attributes and data-testid attributes. The aria-label attributes help assistive technologies such as screen readers identify interactive elements of a page for our users.

A good example of a time to use an aria-label might be if you have a button with an X to close:

<button aria-label="close">X<button>

It might be clear visually that the X closes the modal, but audibly it would not be clear, for example.

<button aria-label="close">Close<button>

The example might read aloud to a user as "Close, Close" or something similar.

However, adding an aria-label to an element that is already clearly labeled or not interactive can be confusing and redundant for users with assistive technologies.

In such cases, don't add an unnecessary aria-label to a component so as to make them selectable for testing. Instead, use a data attribute that will not be read aloud with an assistive technology. For example:

<button data-testid="modal-close-button">Close<button>

We have added support for data attributes in our selectors Prefix your selector string with data-testid:

export const Components = {
  Login: {
    openButton: 'open-button', // this looks for an aria-label
    closeButton: 'data-testid modal-close-button', // this  looks for a data-testid
  },
};

and in your component, import the selectors and add the data-testid:

<button data-testid={Selectors.Components.Login.closeButton}>