grafana/packages
Artur Wierzbicki 0276b029fc
Previews: crawler as a background service (#44891)
* add SQL migrations

* dashboard previews from sql: poc

* added todos

* refactor: use the same enums where possible

* use useEffect, always return json

* added todo

* refactor + delete files after use

* refactor + fix manual thumbnail upload

* refactor: move all interactions with sqlStore to thumbnail repo

* refactor: remove file operations in thumb crawler/service

* refactor: fix dashboard_thumbs sql store

* refactor: extracted thumbnail fetching/updating to a hook

* refactor: store thumbnails in redux store

* refactor: store thumbnails in redux store

* refactor: private'd repo methods

* removed redux storage, saving images as blobs

* allow for configurable rendering timeouts

* added 1) query for dashboards with stale thumbnails, 2) command for marking thumbnails as stale

* use sql-based queue in crawler

* ui for marking thumbnails as stale

* replaced `stale` boolean prop with `state` enum

* introduce rendering session

* compilation errors

* fix crawler stop button

* rename thumbnail state frozen to locked

* #44449: fix merge conflicts

* #44449: remove thumb methods from `Store` interface

* #44449: clean filepath, defer file closing

* #44449: fix rendering.Theme cyclic import

* #44449: linting

* #44449: linting

* #44449: mutex'd crawlerStatus access

* #44449: added integration tests for `sqlstore.dashboard_thumbs`

* #44449: added comments to explain the `ThumbnailState` enum

* #44449: use os.ReadFile rather then os.Open

* #44449: always enable dashboardPreviews feature during integration tests

* #44449: remove sleep time, adjust number of threads

* #44449: review fix: add `orgId` to `DashboardThumbnailMeta`

* #44449: review fix: automatic parsing of thumbnailState

* #44449: lint fixes

* #44449: crawler as a background service v0.1

* #44449: use ServerLockService

* #44449: use ServerLockService

* #44449: review fix: prefer `WithDbSession` over `WithTransactionalDbSession`

* #44449: review fix: add a comment explaining source of the filepath

* #44449: review fix: added filepath validation

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: fix FindDashboardsWithStaleThumbnails to include `theme` and `kind` in search params

* #44449: create function for crawler on demand

* #44449: improve crawler logging

* #44449: fix wire

* #44449: uncomment dummy thumb service, fix ticker interval

* #44449: prevent race condition

* #44449: improve logging

* #44449: fix theme

* #44449: review fixes https://github.com/grafana/grafana/pull/45063/files @fzambia

* #44449: add missing unlock

* #44449: merge

* #44449: review fix - logger @fzambia https://github.com/grafana/grafana/pull/45063/files

* #44449: formatting

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: merge conflict fix

* #44449: naming fix

* #44449: update authOpts

* #44449: change authOpts.role back to admin

* #44449: fix `walk` signature, move ctx to a first argument

* #44449: add `dashboardPreviewsScheduler` feature flag

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
Co-authored-by: Alexander Emelin <frvzmb@gmail.com>
2022-02-10 22:45:00 +04:00
..
grafana-data Previews: crawler as a background service (#44891) 2022-02-10 22:45:00 +04:00
grafana-e2e Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
grafana-e2e-selectors Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
grafana-runtime Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
grafana-schema Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
grafana-toolkit Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
grafana-ui Thresholds: fix crash when trying to render zero-height base thresholds (#45256) 2022-02-10 18:49:30 +01:00
jaeger-ui-components Chore: Set tsconfig target to ES6 (#44842) 2022-02-10 14:37:39 +01:00
README.md Add packages:publishTest command (#43696) 2022-01-05 10:53:46 +01:00

Grafana frontend packages

This document contains information about Grafana frontend package versioning and releases.

Versioning

We use Lerna for packages versioning and releases.

All packages are versioned according to the current Grafana version:

  • Grafana v6.3.0-alpha1 -> @grafana/* packages @ 6.3.0-alpha.1
  • Grafana v6.2.5 -> @grafana/* packages @ 6.2.5
  • Grafana - main branch version (based on package.json, i.e. 6.4.0-pre) -> @grafana/* packages @ 6.4.0-pre- (see details below about packages publishing channels)

Please note that @grafana/toolkit, @grafana/ui, @grafana/data, and @grafana/runtime packages are considered ALPHA even though they are not released as alpha versions.

Stable releases

Even though packages are released under a stable version, they are considered ALPHA until further notice!

Stable releases are published under the latest tag on npm. If there was alpha/beta version released previously, the next tag is updated to stable version.

Alpha and beta releases

Alpha and beta releases are published under the next tag on npm.

Automatic prereleases

Every commit to main that has changes within the packages directory is a subject of npm packages release. ALL packages must be released under version from lerna.json file with commit SHA added to it:

<lerna.json version>-<COMMIT_SHA>

Manual release

All of the steps below must be performed on a release branch, according to Grafana Release Guide.

Make sure you are logged in to npm in your terminal and that you are a part of Grafana org on npm.

  1. Run yarn packages:prepare script from the root directory. This performs tests on the packages and prompts for the version of the packages. The version should be the same as the one being released.

    • Make sure you use semver convention. So, place a dot between prerelease id and prerelease number, i.e. 6.3.0-alpha.1
    • Make sure you confirm the version bump when prompted!
  2. Commit changes (lerna.json and package.json files) - "Packages version update: <VERSION>"

  3. Run yarn packages:build script that prepares distribution packages in packages/grafana-*/dist. These directories are going to be published to npm.

  4. Depending whether or not it's a prerelease:

    • When releasing a prerelease run packages:publishNext to publish new versions.
    • When releasing a stable version run packages:publishLatest to publish new versions.
    • When releasing a test version run packages:publishTest to publish test versions.
  5. Push version commit to the release branch.

Building individual packages

To build individual packages, run:

grafana-toolkit package:build --scope=<ui|toolkit|runtime|data>

Setting up @grafana/* packages for local development

A known issue with @grafana/* packages is that a lot of times we discover problems on canary channel(see versioning overview) when the version was already pushed to npm.

We can easily avoid that by setting up a local packages registry and test the packages before actually publishing to npm.

In this guide you will set up Verdaccio registry locally to fake npm registry. This will enable testing @grafana/* packages without the need for pushing to main.

Setting up local npm registry

From your terminal:

  1. Modify /etc/hosts file and add the following entry: 127.0.0.1 grafana-npm.local
  2. Navigate to devenv/local-npm directory.
  3. Run docker-compose up. This will start your local npm registry, available at http://grafana-npm.local:4873/
  4. Run npm login --registry=http://grafana-npm.local:4873 --scope=@grafana . This will allow you to publish any @grafana/* package into the local registry.
  5. Run npm config set @grafana:registry http://grafana-npm.local:4873. This will config your npm to install @grafana scoped packages from your local registry.

Publishing packages to local npm registry

You need to follow manual packages release procedure. The only difference is you need to run yarn packages:publishDev task in order to publish to you local registry.

From your terminal:

  1. Run yarn packages:prepare.
  2. Commit changes in package.json and lerna.json files
  3. Build packages: yarn packages:build
  4. Run yarn packages:publishDev.
  5. Navigate to http://grafana-npm.local:4873 and verify that version was published

Locally published packages will be published under dev channel, so in your plugin package.json file you can use that channel. For example:

// plugin's package.json

{
  ...
  "@grafana/data": "dev"
}