History and Version Control for Dashboard Updates
A simple version control system for dashboards. Closes #1504.
Goals
1. To create a new dashboard version every time a dashboard is saved.
2. To allow users to view all versions of a given dashboard.
3. To allow users to rollback to a previous version of a dashboard.
4. To allow users to compare two versions of a dashboard.
Usage
Navigate to a dashboard, and click the settings cog. From there, click
the "Changelog" button to be brought to the Changelog view. In this
view, a table containing each version of a dashboard can be seen. Each
entry in the table represents a dashboard version. A selectable
checkbox, the version number, date created, name of the user who created
that version, and commit message is shown in the table, along with a
button that allows a user to restore to a previous version of that
dashboard. If a user wants to restore to a previous version of their
dashboard, they can do so by clicking the previously mentioned button.
If a user wants to compare two different versions of a dashboard, they
can do so by clicking the checkbox of two different dashboard versions,
then clicking the "Compare versions" button located below the dashboard.
From there, the user is brought to a view showing a summary of the
dashboard differences. Each summarized change contains a link that can
be clicked to take the user a JSON diff highlighting the changes line by
line.
Overview of Changes
Backend Changes
- A `dashboard_version` table was created to store each dashboard
version, along with a dashboard version model and structs to represent
the queries and commands necessary for the dashboard version API
methods.
- API endpoints were created to support working with dashboard
versions.
- Methods were added to create, update, read, and destroy dashboard
versions in the database.
- Logic was added to compute the diff between two versions, and
display it to the user.
- The dashboard migration logic was updated to save a "Version
1" of each existing dashboard in the database.
Frontend Changes
- New views
- Methods to pull JSON and HTML from endpoints
New API Endpoints
Each endpoint requires the authorization header to be sent in
the format,
```
Authorization: Bearer <jwt>
```
where `<jwt>` is a JSON web token obtained from the Grafana
admin panel.
`GET "/api/dashboards/db/:dashboardId/versions?orderBy=<string>&limit=<int>&start=<int>"`
Get all dashboard versions for the given dashboard ID. Accepts
three URL parameters:
- `orderBy` String to order the results by. Possible values
are `version`, `created`, `created_by`, `message`. Default
is `versions`. Ordering is always in descending order.
- `limit` Maximum number of results to return
- `start` Position in results to start from
`GET "/api/dashboards/db/:dashboardId/versions/:id"`
Get an individual dashboard version by ID, for the given
dashboard ID.
`POST "/api/dashboards/db/:dashboardId/restore"`
Restore to the given dashboard version. Post body is of
content-type `application/json`, and must contain.
```json
{
"dashboardId": <int>,
"version": <int>
}
```
`GET "/api/dashboards/db/:dashboardId/compare/:versionA...:versionB"`
Compare two dashboard versions by ID for the given
dashboard ID, returning a JSON delta formatted
representation of the diff. The URL format follows
what GitHub does. For example, visiting
[/api/dashboards/db/18/compare/22...33](http://ec2-54-80-139-44.compute-1.amazonaws.com:3000/api/dashboards/db/18/compare/22...33)
will return the diff between versions 22 and 33 for
the dashboard ID 18.
Dependencies Added
- The Go package [gojsondiff](https://github.com/yudai/gojsondiff)
was added and vendored.
2017-05-24 18:14:39 -05:00
|
|
|
package sqlstore
|
|
|
|
|
|
|
|
import (
|
2021-11-17 03:57:37 -06:00
|
|
|
"context"
|
2018-03-21 12:26:27 -05:00
|
|
|
"strings"
|
2017-11-14 04:34:27 -06:00
|
|
|
|
2020-02-29 06:35:15 -06:00
|
|
|
"github.com/grafana/grafana/pkg/models"
|
2017-11-14 04:34:27 -06:00
|
|
|
"github.com/grafana/grafana/pkg/setting"
|
History and Version Control for Dashboard Updates
A simple version control system for dashboards. Closes #1504.
Goals
1. To create a new dashboard version every time a dashboard is saved.
2. To allow users to view all versions of a given dashboard.
3. To allow users to rollback to a previous version of a dashboard.
4. To allow users to compare two versions of a dashboard.
Usage
Navigate to a dashboard, and click the settings cog. From there, click
the "Changelog" button to be brought to the Changelog view. In this
view, a table containing each version of a dashboard can be seen. Each
entry in the table represents a dashboard version. A selectable
checkbox, the version number, date created, name of the user who created
that version, and commit message is shown in the table, along with a
button that allows a user to restore to a previous version of that
dashboard. If a user wants to restore to a previous version of their
dashboard, they can do so by clicking the previously mentioned button.
If a user wants to compare two different versions of a dashboard, they
can do so by clicking the checkbox of two different dashboard versions,
then clicking the "Compare versions" button located below the dashboard.
From there, the user is brought to a view showing a summary of the
dashboard differences. Each summarized change contains a link that can
be clicked to take the user a JSON diff highlighting the changes line by
line.
Overview of Changes
Backend Changes
- A `dashboard_version` table was created to store each dashboard
version, along with a dashboard version model and structs to represent
the queries and commands necessary for the dashboard version API
methods.
- API endpoints were created to support working with dashboard
versions.
- Methods were added to create, update, read, and destroy dashboard
versions in the database.
- Logic was added to compute the diff between two versions, and
display it to the user.
- The dashboard migration logic was updated to save a "Version
1" of each existing dashboard in the database.
Frontend Changes
- New views
- Methods to pull JSON and HTML from endpoints
New API Endpoints
Each endpoint requires the authorization header to be sent in
the format,
```
Authorization: Bearer <jwt>
```
where `<jwt>` is a JSON web token obtained from the Grafana
admin panel.
`GET "/api/dashboards/db/:dashboardId/versions?orderBy=<string>&limit=<int>&start=<int>"`
Get all dashboard versions for the given dashboard ID. Accepts
three URL parameters:
- `orderBy` String to order the results by. Possible values
are `version`, `created`, `created_by`, `message`. Default
is `versions`. Ordering is always in descending order.
- `limit` Maximum number of results to return
- `start` Position in results to start from
`GET "/api/dashboards/db/:dashboardId/versions/:id"`
Get an individual dashboard version by ID, for the given
dashboard ID.
`POST "/api/dashboards/db/:dashboardId/restore"`
Restore to the given dashboard version. Post body is of
content-type `application/json`, and must contain.
```json
{
"dashboardId": <int>,
"version": <int>
}
```
`GET "/api/dashboards/db/:dashboardId/compare/:versionA...:versionB"`
Compare two dashboard versions by ID for the given
dashboard ID, returning a JSON delta formatted
representation of the diff. The URL format follows
what GitHub does. For example, visiting
[/api/dashboards/db/18/compare/22...33](http://ec2-54-80-139-44.compute-1.amazonaws.com:3000/api/dashboards/db/18/compare/22...33)
will return the diff between versions 22 and 33 for
the dashboard ID 18.
Dependencies Added
- The Go package [gojsondiff](https://github.com/yudai/gojsondiff)
was added and vendored.
2017-05-24 18:14:39 -05:00
|
|
|
)
|
|
|
|
|
|
|
|
// GetDashboardVersions gets all dashboard versions for the given dashboard ID.
|
2021-11-17 03:57:37 -06:00
|
|
|
func (ss *SQLStore) GetDashboardVersions(ctx context.Context, query *models.GetDashboardVersionsQuery) error {
|
|
|
|
return ss.WithDbSession(ctx, func(sess *DBSession) error {
|
|
|
|
if query.Limit == 0 {
|
|
|
|
query.Limit = 1000
|
|
|
|
}
|
2017-06-17 17:24:38 -05:00
|
|
|
|
2021-11-17 03:57:37 -06:00
|
|
|
err := sess.Table("dashboard_version").
|
|
|
|
Select(`dashboard_version.id,
|
2017-06-01 16:57:09 -05:00
|
|
|
dashboard_version.dashboard_id,
|
|
|
|
dashboard_version.parent_version,
|
|
|
|
dashboard_version.restored_from,
|
|
|
|
dashboard_version.version,
|
|
|
|
dashboard_version.created,
|
|
|
|
dashboard_version.created_by as created_by_id,
|
|
|
|
dashboard_version.message,
|
2017-06-06 08:40:10 -05:00
|
|
|
dashboard_version.data,`+
|
2021-11-17 03:57:37 -06:00
|
|
|
dialect.Quote("user")+`.login as created_by`).
|
|
|
|
Join("LEFT", dialect.Quote("user"), `dashboard_version.created_by = `+dialect.Quote("user")+`.id`).
|
|
|
|
Join("LEFT", "dashboard", `dashboard.id = dashboard_version.dashboard_id`).
|
|
|
|
Where("dashboard_version.dashboard_id=? AND dashboard.org_id=?", query.DashboardId, query.OrgId).
|
|
|
|
OrderBy("dashboard_version.version DESC").
|
|
|
|
Limit(query.Limit, query.Start).
|
|
|
|
Find(&query.Result)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
History and Version Control for Dashboard Updates
A simple version control system for dashboards. Closes #1504.
Goals
1. To create a new dashboard version every time a dashboard is saved.
2. To allow users to view all versions of a given dashboard.
3. To allow users to rollback to a previous version of a dashboard.
4. To allow users to compare two versions of a dashboard.
Usage
Navigate to a dashboard, and click the settings cog. From there, click
the "Changelog" button to be brought to the Changelog view. In this
view, a table containing each version of a dashboard can be seen. Each
entry in the table represents a dashboard version. A selectable
checkbox, the version number, date created, name of the user who created
that version, and commit message is shown in the table, along with a
button that allows a user to restore to a previous version of that
dashboard. If a user wants to restore to a previous version of their
dashboard, they can do so by clicking the previously mentioned button.
If a user wants to compare two different versions of a dashboard, they
can do so by clicking the checkbox of two different dashboard versions,
then clicking the "Compare versions" button located below the dashboard.
From there, the user is brought to a view showing a summary of the
dashboard differences. Each summarized change contains a link that can
be clicked to take the user a JSON diff highlighting the changes line by
line.
Overview of Changes
Backend Changes
- A `dashboard_version` table was created to store each dashboard
version, along with a dashboard version model and structs to represent
the queries and commands necessary for the dashboard version API
methods.
- API endpoints were created to support working with dashboard
versions.
- Methods were added to create, update, read, and destroy dashboard
versions in the database.
- Logic was added to compute the diff between two versions, and
display it to the user.
- The dashboard migration logic was updated to save a "Version
1" of each existing dashboard in the database.
Frontend Changes
- New views
- Methods to pull JSON and HTML from endpoints
New API Endpoints
Each endpoint requires the authorization header to be sent in
the format,
```
Authorization: Bearer <jwt>
```
where `<jwt>` is a JSON web token obtained from the Grafana
admin panel.
`GET "/api/dashboards/db/:dashboardId/versions?orderBy=<string>&limit=<int>&start=<int>"`
Get all dashboard versions for the given dashboard ID. Accepts
three URL parameters:
- `orderBy` String to order the results by. Possible values
are `version`, `created`, `created_by`, `message`. Default
is `versions`. Ordering is always in descending order.
- `limit` Maximum number of results to return
- `start` Position in results to start from
`GET "/api/dashboards/db/:dashboardId/versions/:id"`
Get an individual dashboard version by ID, for the given
dashboard ID.
`POST "/api/dashboards/db/:dashboardId/restore"`
Restore to the given dashboard version. Post body is of
content-type `application/json`, and must contain.
```json
{
"dashboardId": <int>,
"version": <int>
}
```
`GET "/api/dashboards/db/:dashboardId/compare/:versionA...:versionB"`
Compare two dashboard versions by ID for the given
dashboard ID, returning a JSON delta formatted
representation of the diff. The URL format follows
what GitHub does. For example, visiting
[/api/dashboards/db/18/compare/22...33](http://ec2-54-80-139-44.compute-1.amazonaws.com:3000/api/dashboards/db/18/compare/22...33)
will return the diff between versions 22 and 33 for
the dashboard ID 18.
Dependencies Added
- The Go package [gojsondiff](https://github.com/yudai/gojsondiff)
was added and vendored.
2017-05-24 18:14:39 -05:00
|
|
|
|
2021-11-17 03:57:37 -06:00
|
|
|
if len(query.Result) < 1 {
|
|
|
|
return models.ErrNoVersionsForDashboardId
|
|
|
|
}
|
|
|
|
return nil
|
|
|
|
})
|
History and Version Control for Dashboard Updates
A simple version control system for dashboards. Closes #1504.
Goals
1. To create a new dashboard version every time a dashboard is saved.
2. To allow users to view all versions of a given dashboard.
3. To allow users to rollback to a previous version of a dashboard.
4. To allow users to compare two versions of a dashboard.
Usage
Navigate to a dashboard, and click the settings cog. From there, click
the "Changelog" button to be brought to the Changelog view. In this
view, a table containing each version of a dashboard can be seen. Each
entry in the table represents a dashboard version. A selectable
checkbox, the version number, date created, name of the user who created
that version, and commit message is shown in the table, along with a
button that allows a user to restore to a previous version of that
dashboard. If a user wants to restore to a previous version of their
dashboard, they can do so by clicking the previously mentioned button.
If a user wants to compare two different versions of a dashboard, they
can do so by clicking the checkbox of two different dashboard versions,
then clicking the "Compare versions" button located below the dashboard.
From there, the user is brought to a view showing a summary of the
dashboard differences. Each summarized change contains a link that can
be clicked to take the user a JSON diff highlighting the changes line by
line.
Overview of Changes
Backend Changes
- A `dashboard_version` table was created to store each dashboard
version, along with a dashboard version model and structs to represent
the queries and commands necessary for the dashboard version API
methods.
- API endpoints were created to support working with dashboard
versions.
- Methods were added to create, update, read, and destroy dashboard
versions in the database.
- Logic was added to compute the diff between two versions, and
display it to the user.
- The dashboard migration logic was updated to save a "Version
1" of each existing dashboard in the database.
Frontend Changes
- New views
- Methods to pull JSON and HTML from endpoints
New API Endpoints
Each endpoint requires the authorization header to be sent in
the format,
```
Authorization: Bearer <jwt>
```
where `<jwt>` is a JSON web token obtained from the Grafana
admin panel.
`GET "/api/dashboards/db/:dashboardId/versions?orderBy=<string>&limit=<int>&start=<int>"`
Get all dashboard versions for the given dashboard ID. Accepts
three URL parameters:
- `orderBy` String to order the results by. Possible values
are `version`, `created`, `created_by`, `message`. Default
is `versions`. Ordering is always in descending order.
- `limit` Maximum number of results to return
- `start` Position in results to start from
`GET "/api/dashboards/db/:dashboardId/versions/:id"`
Get an individual dashboard version by ID, for the given
dashboard ID.
`POST "/api/dashboards/db/:dashboardId/restore"`
Restore to the given dashboard version. Post body is of
content-type `application/json`, and must contain.
```json
{
"dashboardId": <int>,
"version": <int>
}
```
`GET "/api/dashboards/db/:dashboardId/compare/:versionA...:versionB"`
Compare two dashboard versions by ID for the given
dashboard ID, returning a JSON delta formatted
representation of the diff. The URL format follows
what GitHub does. For example, visiting
[/api/dashboards/db/18/compare/22...33](http://ec2-54-80-139-44.compute-1.amazonaws.com:3000/api/dashboards/db/18/compare/22...33)
will return the diff between versions 22 and 33 for
the dashboard ID 18.
Dependencies Added
- The Go package [gojsondiff](https://github.com/yudai/gojsondiff)
was added and vendored.
2017-05-24 18:14:39 -05:00
|
|
|
}
|
2017-11-14 04:34:27 -06:00
|
|
|
|
2020-04-06 10:44:14 -05:00
|
|
|
const MAX_VERSIONS_TO_DELETE_PER_BATCH = 100
|
|
|
|
const MAX_VERSION_DELETION_BATCHES = 50
|
2018-03-22 07:18:05 -05:00
|
|
|
|
2021-11-17 03:57:37 -06:00
|
|
|
func (ss *SQLStore) DeleteExpiredVersions(ctx context.Context, cmd *models.DeleteExpiredVersionsCommand) error {
|
|
|
|
return ss.deleteExpiredVersions(ctx, cmd, MAX_VERSIONS_TO_DELETE_PER_BATCH, MAX_VERSION_DELETION_BATCHES)
|
2020-04-06 10:44:14 -05:00
|
|
|
}
|
2017-11-17 07:41:44 -06:00
|
|
|
|
2021-11-17 03:57:37 -06:00
|
|
|
func (ss *SQLStore) deleteExpiredVersions(ctx context.Context, cmd *models.DeleteExpiredVersionsCommand, perBatch int, maxBatches int) error {
|
2020-04-06 10:44:14 -05:00
|
|
|
versionsToKeep := setting.DashboardVersionsToKeep
|
|
|
|
if versionsToKeep < 1 {
|
|
|
|
versionsToKeep = 1
|
|
|
|
}
|
2018-03-21 12:26:27 -05:00
|
|
|
|
2020-04-06 10:44:14 -05:00
|
|
|
for batch := 0; batch < maxBatches; batch++ {
|
|
|
|
deleted := int64(0)
|
|
|
|
|
2021-11-17 03:57:37 -06:00
|
|
|
batchErr := ss.WithTransactionalDbSession(ctx, func(sess *DBSession) error {
|
2020-04-06 10:44:14 -05:00
|
|
|
// Idea of this query is finding version IDs to delete based on formula:
|
|
|
|
// min_version_to_keep = min_version + (versions_count - versions_to_keep)
|
|
|
|
// where version stats is processed for each dashboard. This guarantees that we keep at least versions_to_keep
|
|
|
|
// versions, but in some cases (when versions are sparse) this number may be more.
|
|
|
|
versionIdsToDeleteQuery := `SELECT id
|
|
|
|
FROM dashboard_version, (
|
|
|
|
SELECT dashboard_id, count(version) as count, min(version) as min
|
|
|
|
FROM dashboard_version
|
|
|
|
GROUP BY dashboard_id
|
|
|
|
) AS vtd
|
|
|
|
WHERE dashboard_version.dashboard_id=vtd.dashboard_id
|
|
|
|
AND version < vtd.min + vtd.count - ?
|
|
|
|
LIMIT ?`
|
|
|
|
|
|
|
|
var versionIdsToDelete []interface{}
|
|
|
|
err := sess.SQL(versionIdsToDeleteQuery, versionsToKeep, perBatch).Find(&versionIdsToDelete)
|
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
|
|
|
|
|
|
|
if len(versionIdsToDelete) < 1 {
|
|
|
|
return nil
|
|
|
|
}
|
2018-03-21 13:01:15 -05:00
|
|
|
|
2020-11-10 23:21:08 -06:00
|
|
|
deleteExpiredSQL := `DELETE FROM dashboard_version WHERE id IN (?` + strings.Repeat(",?", len(versionIdsToDelete)-1) + `)`
|
|
|
|
sqlOrArgs := append([]interface{}{deleteExpiredSQL}, versionIdsToDelete...)
|
2019-03-04 09:57:29 -06:00
|
|
|
expiredResponse, err := sess.Exec(sqlOrArgs...)
|
2018-03-21 13:01:15 -05:00
|
|
|
if err != nil {
|
|
|
|
return err
|
|
|
|
}
|
2020-04-06 10:44:14 -05:00
|
|
|
|
|
|
|
deleted, err = expiredResponse.RowsAffected()
|
|
|
|
return err
|
|
|
|
})
|
|
|
|
|
|
|
|
if batchErr != nil {
|
|
|
|
return batchErr
|
2017-11-17 07:41:44 -06:00
|
|
|
}
|
2017-11-14 04:34:27 -06:00
|
|
|
|
2020-04-06 10:44:14 -05:00
|
|
|
cmd.DeletedRows += deleted
|
|
|
|
|
|
|
|
if deleted < int64(perBatch) {
|
|
|
|
break
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return nil
|
2017-11-14 04:34:27 -06:00
|
|
|
}
|