Commit Graph

1882 Commits

Author SHA1 Message Date
Marcus Efraimsson
e4c1a7a141
Tracing: Standardize on otel tracing (#75528) 2023-10-03 14:54:20 +02:00
George Robinson
ed7d29f2b9
Alerting: Migrate old alerting templates to Go templates (#62911)
* Migrate old alerting templates to use $labels

* Fix imports

* Add test coverage and separate rewriting to Go templates

* Fix lint

* Check for additional closing braces

* Add logging of invalid message templates

* Fix tests

* Small fixes

* Update comments

* Panic on empty token

* Use logtest.Fake

* Fix lint

* Allow for spaces in variable names by not tokenizing spaces

* Add template function to deduplicate Labels in a Value map

* Fix behavior of mapLookupString

* Reference deduplicated labels in migrated message template

* Fix behavior of deduplicateLabelsFunc

* Don't create variable for parent logger

* Add more tests for deduplicateLabelsFunc

* Remove unused function

* Apply suggestions from code review

Co-authored by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>

* Give label val merge function better name

* Extract template migration and escape literal tokens

* Consolidate + simplify template migration

---------

Co-authored-by: William Wernert <william.wernert@grafana.com>
2023-10-02 11:25:33 -04:00
Ryan McKinley
0af7247612
Chore: cleanup old dashboardPreviews files (#75682) 2023-09-28 13:31:07 -07:00
Gabriel MABILLE
96cbe70b14
User: Support sort query param for user and org user, search endpoints (#75229)
* User: Add sort option to user search

* Switch to an approach that uses the dashboard search options

* Cable user sort on the org endpoint

* Alias user table with u in org store

* Add test and cover orgs/:orgID/users/search endpoint

* Add test to userimpl store

* Simplify the store_test with sortopts.ParseSortQueryParam

* Account for PR feedback

* Positive check

* Update docs

* Update docs

* Switch to ErrOrFallback

Co-authored-by: Karl Persson <kalle.persson@grafana.com>

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-09-28 10:16:18 +02:00
Jo
40a1f8434d
Anon: Scaffold anon service (#74744)
* remove API tagging method and authed tagging

* add anonstore

move debug to after cache

change test order

fix issue where mysql trims to second

* add old device cleanup

lint

utc-ize everything

trim whitespace

* remove dangling setting

* Add delete devices

* Move anonymous authnclient to anonimpl

* Add simple post login hook

* move registration of Background Service

cleanup

* add updated_at index

* do not untag device if login err

* add delete device integration test
2023-09-25 16:25:29 +02:00
George Robinson
15f6e8a500
SQLStore: Fix race condition in RecursiveQueriesAreSupported (#75274) 2023-09-22 11:30:14 +01:00
Piotr Jamróz
946da57b6a
Correlations: Allow creating correlations for provisioned data sources (#73737)
* Allow creating correlations for provisioned data sources

* Update docs

* Fix linting

* Add missing props

* Add missing props

* Fix linting

* Fix linting

* Clarify error name

* Removed error handling for a non-existing use case

* Create a list of deleted data datasources based on all configs

* Add org_id to correlations

* Add tests

* Allow org_id to be null in case org_id=0 is used

* Create organization to ensure stable id is generated

* Fix linting

* Ensure backwards compatibility

* Add deprecation information

* Update comments

* Override existing datasSource variable so the UID is retrieved correctly

* Migrate correlations indices

* Default org_id when migrating

* Remove redundant default

* Make PK non-nullable

* Post merge fixes

* Separate data sources / correlations provisioning

* Adjust comments

* Store new data sources in spy store so it can be used to test correlations as well

* Fix linting

* Update tests

* Ensure response is closed

* Avoid creating duplicates during provisioning

* Fix updating provisioned column and update tests

* Rename error message

* Fix linting errors

* Fix linting errors and rename variable

* Update test

* Update pkg/services/sqlstore/migrations/correlations_mig.go

Co-authored-by: Giordano Ricci <me@giordanoricci.com>

* Remove unused error

* Fix lining

---------

Co-authored-by: Giordano Ricci <me@giordanoricci.com>
2023-09-13 15:10:09 +02:00
Eric Leijonmarck
b00f3216c1
Auth: Refactor for revoking user tokens within last hours (#74616)
* fix: revoked tokens within last hours

adds check for unlimited sessions out of index
adds a function for specifing the hours to look back when revoking users tokens, otherwise we "assume" the clean up takes care of them adds a index for the `user_auth_token` - `revoked_at` for faster queries when using `revoked_at`

* fix: sqllite datetime conversion with unixtimestamps

* fix: postgres dialect

* fix: mysql dialect

* fix: mysql dialect missing closing )

* refactor: delete revoked tokens directly

* fix: tests for sqlite

* AuthToken: Simplify DeleteUserRevokedTokens and add test

* fix: linting newline

* Reset get time after test

* fix: test order by revoked

* fix: order by different db

* ascending

* test with seen at

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-09-13 10:24:37 +01:00
Marcus Efraimsson
edc34fc836
Instrumentation: Improve instrumentation for database migrations (#74659) 2023-09-11 14:10:21 +03:00
linoman
13f4382214
Auth: Implement requester interface in access control module (#74289)
* Implement requester interface in the access control module
2023-09-06 11:16:10 +02:00
Serge Zaitsev
7197ed9fbb
Fix: avoid panics on duplicate metrics registration (#74359) 2023-09-05 10:02:11 +02:00
Serge Zaitsev
58f6648505
Chore: capitalise messages for alerting (#74335) 2023-09-04 18:46:34 +02:00
Ryan McKinley
025b2f3011
Chore: use any rather than interface{} (#74066) 2023-08-30 18:46:47 +03:00
Kristin Laemmert
3e272d2bda
Bug(sqlstore): fix issue with postgres unable to find existing main organization (#73789)
* SQLStore: fix issue where postgres would not find the existing org

Grafana using a postgres databases would fail to find the created org when the CreatedAt and UpdatedAt times are (inaccurately) populated. This issue only occurs in postgres, and only shows up when getOrCreateOrg run to create the admin user AND the organization already exists. See https://github.com/grafana/grafana/issues/71781 for more information and a reproduction.

* add an integration test
2023-08-30 11:45:20 -04:00
Piotr Jamróz
b30e0aa5aa
Correlations: Add organization id (#72258)
* Add org_id to correlations

* Add tests

* Allow org_id to be null in case org_id=0 is used

* Create organization to ensure stable id is generated

* Fix linting

* Ensure backwards compatibility

* Add deprecation information

* Migrate correlations indices

* Default org_id when migrating

* Remove redundant default

* Make PK non-nullable
2023-08-24 09:39:30 +02:00
Marcus Efraimsson
040b7d2571
Chore: Add errutils helpers (#73577)
Add helpers for the errutil package in favor of errutil.NewBase.
2023-08-22 12:52:24 +02:00
Josh Hunt
7bc6d32eb9
Search API: Search by folder UID (#65040)
* Search: Attempt to support folderUID filter

* Search: Use folder UID instead of ID for searching folders

* Update swagger

* Fix JSON property casing

* Add integration test

* Remove redundant query condition

* Fix frontend test

* Fix listing dashboards in General/root

* Add support for fetching top level folders

using `folderUIDs=` (empty string) query parameter

* Add deprecation notice

* Send uid of general in sql.ts

* Use 'general' for query folderUIDs query param for fetching folder

* Add tests

* Fix FolderUIDFilter

---------

Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
2023-08-04 12:43:47 +03:00
Sofia Papagiannaki
8a24e891fe
Nested folders: Fix search query for empty self-contained permissions (#72727)
* Add tests

* Fix query for nested folders with zero self-contained permissions

* Fix query behind  permissionsFilterRemoveSubquery flag

* Apply suggestion from code review
2023-08-02 14:12:46 +03:00
Sofia Papagiannaki
2c26a02b82
Search v1: Remove unnecessary subqueries (#72388)
* Add feature flag

* Introduce interface and dummy implementation

* Add tests for the new filter

* accessControlDashboardPermissionFilterNoFolderSubquery implementation

* join only if it's necessary

* force ordering for tests

* Temporarily enable new query for benchmarks
2023-08-02 10:39:25 +03:00
Serge Zaitsev
7767ab6f43
Chore: Add folder data migration, fix unique index (#72602)
* add folder data migration, fix unique index

* fix unique index

* pass a fake store in tests

* pass store into other providers in tests

* and now with alerting!
2023-08-01 09:36:37 +02:00
Yuri Tseretyan
143683bd05
Alerting: Add more clear error to migration when rule cannot be parsed (#72374) 2023-07-27 09:47:04 -04:00
Matthew Jacobson
c7eb7fb58a
Alerting: Keep legacy alert rule maxDataPoints and intervalMs during migration (#71989)
Keep legacy alert rule max-data-points and interval during migration
2023-07-24 13:36:34 -04:00
Ieva
cfa1a2c55f
RBAC: Split non-empty scopes into kind, attribute and identifier fields for better search performance (#71933)
* add a feature toggle

* add the fields for attribute, kind and identifier to permission

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* set the new fields when new permissions are stored

* add migrations

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* remove comments

* Update pkg/services/accesscontrol/migrator/migrator.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* feedback: put column migrations behind the feature toggle, added an index, changed how wildcard scopes are split

* PR feedback: add a comment and revert an accidentally changed file

* PR feedback: handle the case with : in resource identifier

* switch from checking feature toggle through cfg to checking it through featuremgmt

* don't put the column migrations behind a feature toggle after all - this breaks permission queries from db

---------

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-07-21 15:23:01 +01:00
Ieva
25c4292a5f
RBAC: search v1 permission filter part 1 - cleanup & updating tests (#71913)
* update tests and remove some AC disabled checks

* remove test for old permission filter builder
2023-07-19 10:37:27 +01:00
Matthew Jacobson
8c6cdf51fc
Alerting: No longer silence paused alerts during legacy migration (#71596)
* Alerting: No longer silence paused alerts during legacy migration

Now that we migrate paused legacy alerts to paused UA alert rules, we no longer need to silence them.
2023-07-17 09:37:53 -04:00
Misi
5efc3386d3
AuthZ: Extend /api/search to work with self-contained permissions (#70749)
* Search sql filter draft, unfinished

* Search works for empty roles

* Add current AuthModule to SignedInUser

* clean up, changes to the search

* Use constant prefixes

* Change AuthModule to AuthenticatedBy

* Add tests for using the permissions from the SignedInUser

* Refactor and simplify code

* Fix sql generation for pg and mysql

* Fixes, clean up

* Add test for empty permission list

* Fix

* Fix any vs all in case of edit permission

* Update pkg/services/authn/authn.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Update pkg/services/sqlstore/permissions/dashboard_test.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

* Fixes, changes based on the review

---------

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
2023-07-12 12:31:36 +02:00
Sofia Papagiannaki
e21872bb15
SQLStore: Fix regression in PostgreSQL connection string (#71368)
* SQLStore: Fix connection string for PostgreSQL

* better testing
2023-07-11 16:47:35 +03:00
Guillaume Jacquet
b947222bf5
Platform: Add support for Postgresql pgpass file (#61517)
lib/pq has built-in support to use pgpass file for authentication when
no password has been provided. However this requires that the connection
does not contain the password parameter at all.

Removing password parameter when postgresql password is empty in
SQL store.
2023-07-07 11:17:35 +02:00
Jo
d6c468c1c2
Auth: Add empty role definition (#64694)
* Allow setting role as None

Co-authored-by: gamab <gabi.mabs@gmail.com>

Seeking for places where role.None would be used

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

Adding None role to the frontend

Co-authored-by: Jguer <joao.guerreiro@grafana.com>

unify org role declaration and remove from add permission

fix backend test

fix backend lint

* remove role none from frontend

* Simplify checks

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>

* nits

---------

Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
2023-07-06 15:40:06 +02:00
Matthew Jacobson
00d5f7fed7
Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration (#70781)
* Alerting: Convert 'Both' type Prometheus queries to 'Range' in migration
2023-06-28 14:02:57 -04:00
Matthew Jacobson
44972d0cd5
SQLStore: Fix Postgres dialect treating "false" migrator default as true (#69353)
* SQLStore: Fix Postgres dialect treating "false" migrator default as true

Previously, when creating a migration you could choose a default value for a new 
boolean column that looked correct but would be interpreted incorrectly by the 
Postgres dialect. For example, values such as "false" or "FALSE" would be treated 
as true by the Postgres dialect.

This refactors how migration dialects determine the Default column value for boolean
type columns. Each dialect now uses the same base code to parse the Default literal 
and panics if an unknown value is encountered.

So, now AddColumnMigration and AddTableMigration will ensure that across dialects:

- The exact same Default literals will be allowed.
- The literals are converted to equivalent defaults in their DDL.
- An error will be thrown if an invalid literal is provided.
2023-06-22 15:09:36 -04:00
Santiago
ff9eff49bd
Alerting: Bump grafana/alerting and refactor the ImageStore/Provider to provide image URL/bytes (#70182)
* implement alerting.images.Provider interface in our ImageStore

* add URLExists() method to fakeConfigStore

* make linter happy

* update integration tests
2023-06-21 20:53:30 -03:00
Dan Cech
a6279b2d62
Database: Make dialects independent of xorm Engine (#69955)
* make dialects independent of xorm Engine

* goimports
2023-06-14 16:13:36 -04:00
Artur Wierzbicki
15f06a93b2
Storage: increase size of the contents column (#69152)
* fix size of contents column

* temporarily re-enable integration tests

* lint

* revert: temporarily re-enable integration tests
2023-05-30 21:11:04 +04:00
Alexander Weaver
0ed5d3bdf2
Revert "Alerting: Refactor the ImageStore/Provider to provide image URL/bytes" (#69265)
Revert "Alerting: Refactor the ImageStore/Provider to provide image URL/bytes (#67693)"

This reverts commit 72a187b0be.
2023-05-30 11:33:33 -05:00
Santiago
72a187b0be
Alerting: Refactor the ImageStore/Provider to provide image URL/bytes (#67693)
* (WIP) Refactor the ImageStore interface to work with our latest alerting repository

* update alerting package

* refactor, new URLExists method in ImageProvider

* tests for the new methods

* fix linter warnings

* use alertingImages as an alias for grafana/alerting/images

* logs about image uris and not found images

* nerf image not found logs

* extract duplicated code to getImageFromURI() method

* refactor getImageFromURI()

* add index on url

* add comment about migration log

* sync generated files
2023-05-30 11:25:55 -03:00
Sofia Papagiannaki
74e87ccbbd
SQLStore: Align SQLite IsUniqueConstraintViolation() with other backend implementations (#69224)
* Add integration test for primary key and unique constrain violation

* Align SQLite IsUniqueConstraintViolation implementation with other backends
2023-05-29 14:45:08 +00:00
Gabriel MABILLE
edf1775d49
AuthN: Embed an OAuth2 server for external service authentication (#68086)
* Moving POC files from #64283 to a new branch

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Adding missing permission definition

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Force the service instantiation while client isn't merged

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Merge conf with main

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Leave go-sqlite3 version unchanged

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* tidy

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* User SearchUserPermissions instead of SearchUsersPermissions

* Replace DummyKeyService with signingkeys.Service

* Use user🆔<id> as subject

* Fix introspection endpoint issue

* Add X-Grafana-Org-Id to get_resources.bash script

* Regenerate toggles_gen.go
* Fix basic.go

* Add GetExternalService tests

* Add GetPublicKeyScopes tests

* Add GetScopesOnUser tests

* Add GetScopes tests

* Add ParsePublicKeyPem tests

* Add database test for GetByName

* re-add comments

* client tests added

* Add GetExternalServicePublicKey tests

* Add other test case to GetExternalServicePublicKey

* client_credentials grant test

* Add test to jwtbearer grant

* Test Comments

* Add handleKeyOptions tests

* Add RSA key generation test

* Add ECDSA by default to EmbeddedSigningKeysService

* Clean up org id scope and audiences

* Add audiences to the DB

* Fix check on Audience

* Fix double import

* Add AC Store mock and align oauthserver tests

* Fix test after rebase

* Adding missing store function to mock

* Fix double import

* Add CODEOWNER

* Fix some linting errors

* errors don't need type assertion

* Typo codeowners

* use mockery for oauthserver store

* Add feature toggle check

* Fix db tests to handle the feature flag

* Adding call to DeleteExternalServiceRole

* Fix flaky test

* Re-organize routes comments and plan futur work

* Add client_id check to Extended JWT client

* Clean up

* Fix

* Remove background service registry instantiation of the OAuth server

* Comment cleanup

* Remove unused client function

* Update go.mod to use the latest ory/fosite commit

* Remove oauth2_server related configs from defaults.ini

* Add audiences to DTO

* Fix flaky test

* Remove registration endpoint and demo scripts. Document code

* Rename packages

* Remove the OAuthService vs OAuthServer confusion

* fix incorrect import ext_jwt_test

* Comments and order

* Comment basic auth

* Remove unecessary todo

* Clean api

* Moving ParsePublicKeyPem to utils

* re ordering functions in service.go

* Fix comment

* comment on the redirect uri

* Add RBAC actions, not only scopes

* Fix tests

* re-import featuremgmt in migrations

* Fix wire

* Fix scopes in test

* Fix flaky test

* Remove todo, the intersection should always return the minimal set

* Remove unecessary check from intersection code

* Allow env overrides on settings

* remove the term app name

* Remove app keyword for client instead and use Name instead of ExternalServiceName

* LogID remove ExternalService ref

* Use Name instead of ExternalServiceName

* Imports order

* Inline

* Using ExternalService and ExternalServiceDTO

* Remove xorm tags

* comment

* Rename client files

* client -> external service

* comments

* Move test to correct package

* slimmer test

* cachedUser -> cachedExternalService

* Fix aggregate store test

* PluginAuthSession -> AuthSession

* Revert the nil cehcks

* Remove unecessary extra

* Removing custom session

* fix typo in test

* Use constants for tests

* Simplify HandleToken tests

* Refactor the HandleTokenRequest test

* test message

* Review test

* Prevent flacky test on client as well

* go imports

* Revert changes from 526e48ad45

* AuthN: Change the External Service registration form (#68649)

* AuthN: change the External Service registration form

* Gen default permissions

* Change demo script registration form

* Remove unecessary comment

* Nit.

* Reduce cyclomatic complexity

* Remove demo_scripts

* Handle case with no service account

* Comments

* Group key gen

* Nit.

* Check the SaveExternalService test

* Rename cachedUser to cachedClient in test

* One more test case to database test

* Comments

* Remove last org scope

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>

* Update pkg/services/oauthserver/utils/utils_test.go

* Update pkg/services/sqlstore/migrations/oauthserver/migrations.go

Remove comment

* Update pkg/setting/setting.go

Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>

---------

Co-authored-by: Mihály Gyöngyösi <mgyongyosi@users.noreply.github.com>
2023-05-25 15:38:30 +02:00
Yuri Tseretyan
3af95bebe1
Alerting: Migrate unknown NoData\Error settings to the default (#68403)
* use default execution if legacy is not known

* update docs

* Update docs/sources/alerting/migrating-alerts/migrating-legacy-alerts.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

* Update docs/sources/alerting/migrating-alerts/migrating-legacy-alerts.md

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>

---------

Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
2023-05-24 13:09:17 -04:00
owensmallwood
b43206e26b
MySQL: Use transaction_isolation instead of tx_isolation (#68575)
will use the correct system var name for transaction isolation
2023-05-23 09:22:05 -06:00
Ryan McKinley
26658d172f
Migrations: Support independent migration log for entity api (#68511) 2023-05-22 11:31:07 -07:00
Sofia Papagiannaki
633b7875e7
Chore: Ignore unique constrain failure when creating the main organization (#68644)
Chore: Ignore unique constrain failure when creatin the main organization
2023-05-18 11:26:29 +03:00
Dan Cech
7572988305
add uid to team table (#66920)
* add uid to team table

* concat

* add prefix to transition uids

---------

Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
2023-05-09 13:03:28 -04:00
Yuri Tseretyan
0ce7f7eaf4
Alerting: Migration to not fail if alert_configuration table is not empty (#67924) 2023-05-05 12:03:53 -04:00
Yuri Tseretyan
61484fa826
Alerting: Mention title of alert rule that caused migration to fail (#67451)
* add debug log for migration of alert rules
* add alert rule name and some information to conversion error
2023-05-01 10:32:16 -04:00
Carl Bergquist
692bb9ed1a
Instrumentation: Add support for instrumenting database queries (#66022)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
2023-04-28 15:19:06 +02:00
Yuri Tseretyan
a8b4a4bb45
Alerting: Update alerting module to 20230418161049-5f374e58cb32 + refactoring (#66622)
* update to alerting 20230418161049-5f374e58cb32
* rename renamed structs in https://github.com/grafana/alerting/pull/73
* update ValidateContactPoint to use BuildReceiverConfiguration
* update logger factory according to changes
* rewrite integration builder
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
2023-04-25 13:39:46 -04:00
Ieva
035bf29146
RBAC: Remove the option to disable RBAC and add automated permission migrations for instances that had RBAC disabled (#66652)
* RBAC: Stop reading enabeld from ini file and always set to true

* Migrations: Add a migration for rbac to reset data migrations if rbac
was disabled

* If rbac was disabled we reset the data and data migrations that rbac
  has to perform to get it to a correct state

* Migrator: Store migration logs on migrator and add function to clear it from the
in-memory stored logs

* update tests

---------

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2023-04-19 16:34:19 +01:00
Artur Wierzbicki
2136e680c4
Previews: remove dashboard previews backend (#66176)
* remove dashboard previews backend

* remove dashboard previews backend

* bring back the migration

* bring back the migration

* bring back the migration
2023-04-13 21:42:24 +04:00
Yuri Tseretyan
afd52d0866
Alerting: use alerting GrafanaReceiver and BuildReceiverConfiguration in Grafana (#65224)
* replace receiver errors with one from alerting
* add the converter to alerting models
* update buildReceiverIntegration to accept GrafanaReceiver
---------

Co-authored-by: George Robinson <george.robinson@grafana.com>
2023-04-13 12:25:32 -04:00