Commit Graph

1700 Commits

Author SHA1 Message Date
Ryan McKinley
3b1a8d45ed
Playlist: update service so it supports both read+write (#55959) 2022-10-04 11:11:18 -04:00
idafurjes
a25516fbe3
Chore: Copy sqlstore methods to user store (#56280)
* Chore: Copy sqlstore methods to user store

* Fix xorm tag

* Add tests

* Remove unused methods from sqlstore
2022-10-04 14:14:32 +02:00
idafurjes
a863a4d95d
Chore: Copy user methods over to user store (#56000)
* Chore: Copy user methods over to user store

* Fix some tests and bugs

* Add some more tests

* Move tests to user store

* Move back the tests

* Add some tests
2022-10-04 12:17:55 +02:00
Sofia Papagiannaki
8b77ee2734
SQLStore: Ensure that sessions are always closed (#55864)
* SQLStore: Ensure that sessions are always closed

Delete `NewSession()` in favour of `WithDbSession()`

* Add WithDbSessionForceNewSession to the interface

* Apply suggestions from code review
2022-09-29 15:55:47 +03:00
Eric Leijonmarck
1e8f8dff4b
Auth: Adds validation and ingestion of conflict file (#53014)
* add users-manager command

* add users-manager command

* rename files

* refactor: imports and renaming

* Command: add conflict merge user command

- MergeUser will
	- replace all user_ids from conflicting users to the chosen userId
	- delete users whose user_ids are not the chosen user
- SameIdentification will
	- update chosen user with chosen email,login details
	- delete users whose user_ids are not the chosen user

* refactor: clean up

* refactor: create structure for read, validate, ingest

* feat: ls and generate-file for conflicting users

* remove usagestats

* added back pkg/services/login/authinfoservice/database/stats.go

* Revert "added back pkg/services/login/authinfoservice/database/stats.go"

This reverts commit 2ba6e3c4d6.

* Revert "remove usagestats"

This reverts commit 1e3fa97810.

* cherry pick

* Revert "cherry pick"

This reverts commit 461626c306.

* validation of picked merge user

* fix test

* make lint

* make test run

* tests for ingest working

* clean up and refactored to align with downstream refactoring

* formatting

* refactor: name list instead of ls

* fix: static lint error use trimprefix

* WIP: permissions for validation

* fix: remove unused functions in sqlstore

* fix: remove unused function

* handling of multiple users and resolve discarded users

* fix tests

* fix: bug that did not exclude the blocks

* ioutil is blacklisted

* WIP: validation

* tests for merging a user working

* add latest changes to output print

* refactor: removed conflictEmail and conflictLogin that was not used

* refactor: code clean up, showChanges working

* test and linting fixes

* test and linting fixes

* refactor: removed logging of config and added more info for vlidation command

* refactor: fix order of code

* fix time now

* refactor: no longer need for check casesensitive login/email

* removed unnessecary loop

* refactor: move functions around

* test: working

* docs: add docuemntationf for file

* Add failing test for generating the conflict login block

* Fix regex

* Fix some stuff/tests

Co-authored-by: eleijonmarck <eric.leijonmarck@gmail.com>

* add: docs for conflict file

* add: conflict_email, conflict_login fields

* add: conflict_email, conflict_login fields

* WIP

* fix: tests working as intended

* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go

Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>

* review comments

* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>

* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go

Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>

* missspelling

* trailing new line

* update to use userimpl store

* remove newline

* remove newline

* refactor: initializing of resolver for conflicts

* fix: test sqlStore

* refactor: removed lines

* refactor: remove TODOs

Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
2022-09-29 14:26:24 +02:00
idafurjes
122e7c141d
Chore: Delete UpdateUser, ChangeUserPassword, UpdateLastSeenAt from sqlstore (#55928) 2022-09-28 16:17:09 +02:00
idafurjes
a45ef61d25
Chore: Remove GetUserByEmail and GetUserByLogin from sqlstore (#55903)
* Chore: Remove GetUserByEmail and GetUserByLogin from sqlstore
 Rename GetUserProfile to GetProfile

* Fix lint

* Skip test for mysql

* Add missing method to sqlstore mock
2022-09-28 13:18:19 +02:00
idafurjes
56cfe02ca2
Chore: Remove methods from sqlstore interface (#55802)
* Remove SearchOrgUsers from sqlstore interface

* Remove RemoveOrgUser method from sqlstore interface

* Delete RemoveOrgUser from sqlstore

* Fix lint
2022-09-27 15:33:38 +02:00
Kristin Laemmert
701f6d5436
UserService: use the UserService instead of calling sqlstore directly (#55745)
* UserService: update callers to use the UserService instead of calling sqlstore directly

There is one major change hiding in this PR. UserService.Delete originally called a number of services to delete user-related records. I moved everything except the actual call to the user table, and moved those into the API. This was done to avoid dependencies cycles; many of our services depend on the user service, so the user service itself should have as few dependencies as possible.
2022-09-27 07:58:49 -04:00
idafurjes
d8cd30d22a
Chore: Remove DeleteOrg and UpdateOrgAddress from sqlstore (#55795)
* Chore: Remove DeleteOrg and UpdateOrgAddress from sqlstore

* Remove commented out methods
2022-09-27 12:30:10 +02:00
Piotr Jamróz
becdf10f0e
Glue: Add DB migration & support provisioning for user-defined correlations config (#55560)
* Allow provisioning correlation config

* Simplify code

* Fix reading correlations test

* Fix linting errors

* Fix linting errors

* remove simpleJson

* Clean up

* Fix tests

* Update swagger docs

* Fix linting

* Fix linting

* Clean up swagger definitions

Co-authored-by: Elfo404 <me@giordanoricci.com>
2022-09-27 11:08:02 +02:00
idafurjes
2bfd26249b
Chore: Remove methods from store interface (#55765)
* Chore: Remove methods from store interface

* Fix api tests

* Remove sqlstore methods, add org store tests, add GetOrgUsers to org store interface

* Fix lint

* Remove debug logs

* Remove commented out methods
2022-09-27 10:34:31 +02:00
Jean-Philippe Quéméner
f3a307778a
Alerting: cache general folder in migration based on org id (#55620) 2022-09-23 18:22:45 +02:00
Sofia Papagiannaki
d0e7765c6a
Annotation: Optionally allow storing longer annotation tags (#54754)
* Annotation: Optionally allow longer annotation tags

* Do not accept configuration lower than today's default (500)

* Apply suggestion from code review
2022-09-23 06:04:41 -04:00
Serge Zaitsev
4c19e83ff0
Chore: Move team store implementation to a separate package (#55514)
* Chore: move team store implementation to a separate package

* trying to fix more tests

* fix tests in service accounts and access control

* fix common tests

* restore commented out test

* add todos
2022-09-22 19:16:21 +02:00
Sofia Papagiannaki
2f14575dd3
Chore: Move annotations cleanup to the annotations service (#55618) 2022-09-22 15:27:48 +03:00
Emil Tullstedt
647997cc4c
Alerting: Fix flaky test (#55551)
The length of the identifier from the underlying library is 9 or more characters depending on the rate at which the identifiers are generated. See https://pkg.go.dev/github.com/teris-io/shortid

The test previously made the assumption that the length will always be 10, which would intermittently fail.
2022-09-21 14:17:25 +02:00
ying-jeanne
7b4cea8151
Chore: StoreSplit tag service (#55453)
* move tag service outside

* fix dashboard

* fix test

* lint

* fix linter

* remove spew
2022-09-21 08:04:01 -04:00
Ieva
6d5bdf12e8
resolve merge conflicts (#55503) 2022-09-20 13:31:08 -04:00
Serge Zaitsev
305d494902
Chore: Switch over to team.Service instead of sqlstore (#55497)
* switch to using team service

* trying to fix tests

* more tests to fix

* add missing teamtest package
2022-09-20 18:58:04 +02:00
Marcus Efraimsson
862a6a2fa6
Logging: Introduce API for contextual logging (#55198)
Introduces a FromContext method on the log.Logger interface that 
allows contextual key/value pairs to be attached, e.g. per request, 
so that any logger using this API will automatically get the per request 
context attached. The proposal makes the traceID available for 
contextual logger , if available, and would allow logs originating from 
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
2022-09-20 18:32:06 +02:00
idafurjes
eff4daacaf
Bug: Rollback change in search (#55443) 2022-09-20 12:18:39 +02:00
Serge Zaitsev
8f1e2ed658
Chore: Split temporary user (invite) service (#55397)
* Chore: Split temporary users

* change references to tempuser service

* fix api tests

* restore tests
2022-09-20 11:29:17 +02:00
idafurjes
7ce7c9b64c
Chore: Move SearchOrgs to org service (#55416)
* Chore: Move SearchOrgs to org service

* Fix lint

* Fix lint 2
2022-09-20 09:55:40 +02:00
Serge Zaitsev
96b032e103
Chore: Split dashboard thumbnail service (#55344)
* Chore: split dashboard thumbnail service

* fix typo

* move tests

* make linter happy
2022-09-19 11:29:22 +02:00
Sofia Papagiannaki
754eea20b3
Chore: SQL store split for annotations (#55089)
* Chore: SQL store split for annotations

* Apply suggestion from code review
2022-09-19 10:54:37 +03:00
ying-jeanne
e4741ce8d6
remove datasource from sqlstore (#55288) 2022-09-16 06:20:26 -04:00
Piotr Jamróz
7352c181c2
Migrate user_id to BIGINT (#55084) 2022-09-15 13:29:46 +02:00
Alexander Weaver
9f45e2e706
Alerting: Fix legacy migration crash when rule name is too long (#55053)
* Extract standardized UID field length to constant

* Extract default length to constant

* Truncate rule names that are too long

* Add tests for name normalization

* Fix whitespace lint error

* Another linter fix

* Empty commit to kick build
2022-09-13 13:53:09 -05:00
Emil Tullstedt
b287047052
Chore: Upgrade Go to 1.19.1 (#54902)
* WIP

* Set public_suffix to a pre Ruby 2.6 version

* we don't need to install python

* Stretch->Buster

* Bump versions in lib.star

* Manually update linter

Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.

Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.

* undo reformatting

* Various lint improvements

* More from the linter

* goimports -w ./pkg/

* Disable gocritic

* Add/modify linter exceptions

* lint + flatten nested list

Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
2022-09-12 12:03:49 +02:00
Misi
c5be9605a2
Chore: Use TimeNow instead of time.Now in user sql store to make it testable (#54983) 2022-09-09 18:43:14 +02:00
Joe Blubaugh
22c937340e
Revert "Alerting: Write and Delete multiple alert instances. (#54072)" (#54885)
This reverts commit 5e4fd94413.
2022-09-09 17:44:06 +02:00
Carl Bergquist
b4e3c89f60
instrumentation: dont instrument ErrSkip at all (#54880)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
2022-09-08 11:14:55 +02:00
mhuangwm
39102c6656
Admin: Add support to configure default admin email (#54363) 2022-09-07 14:38:40 +02:00
Carl Bergquist
78978048c3
Instrumentation: log the total number of db queries per request (#54647)
Signed-off-by: bergquist <carl.bergquist@gmail.com>
Co-authored-by: Emil Tullstedt <emil.tullstedt@grafana.com>
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
2022-09-05 07:39:22 +02:00
linoman
d2bb72fb3c
Login: Remove single admin team restriction (#54534)
* Remove single member team restriction

* Add label when permissions list is empty

* Fix unit tests

* Add co-author.

Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
2022-09-02 18:16:39 +02:00
Joe Blubaugh
5e4fd94413
Alerting: Write and Delete multiple alert instances. (#54072)
Prior to this change, all alert instance writes and deletes happened
individually, in their own database transaction. This change batches up
writes or deletes for a given rule's evaluation loop into a single
transaction before applying it.

Before:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8           398           2991381 ns/op         1133537 B/op      27703 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
    util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created
    util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created
    util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created
PASS
ok      github.com/grafana/grafana/pkg/services/ngalert/store   1.619s
```

After:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8          1440            816484 ns/op          352297 B/op       6529 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
    util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created
    util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created
    util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created
PASS
ok      github.com/grafana/grafana/pkg/services/ngalert/store   1.383s
```

So we cut time by about 75% and memory allocations by about 60% when
storing and deleting 100 instances.

This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1
2022-09-02 11:17:20 +08:00
Serge Zaitsev
927ddf9376
Chore: Move login attempt methods to separate service (#54479)
* Chore: Move login attempt methods to separate service

* attempt to fix tests

* fix syntax

* better time mocking

* initialise now func
2022-09-01 18:08:42 +02:00
Dave Henderson
713075c494
Metrics: Fixed grafana_database_conn_* metrics, and added new go_sql_stats_* metrics as eventual replacement (#54405)
* metrics: Fix broken DBStats metrics

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

* Register the sqlstats metrics by default

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>

Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
2022-08-31 08:54:32 -04:00
ying-jeanne
6227528ea0
Chore: SQL Store Split of datasource (#54262)
* refectory datasource

* fix linter
2022-08-26 11:03:38 -04:00
ying-jeanne
fd01161bcc
Chore: replace xorm by sqlx in dashboardversion service (#53869) 2022-08-25 16:04:16 -05:00
Valério Valério
b5142832fa
Alerting: Fix saving of screenshots uploaded with a signed url (#53933)
The URL of screenshots uploaded to external image storages can be optionally signed, resulting in a long string (800+ chars).
2022-08-24 12:40:50 +01:00
ying-jeanne
4dbe0b4f02
Chore: Move updateorg out of sqlstore (#54111)
* Chore: move updateorg out of sqlstore

* fix api test
2022-08-23 12:26:21 -05:00
Jo
4a9137ac40
API Keys: Add revocation for SATs (#53896)
* add apikey is_revoked field

* add token store tests

* Apply suggestions from code review

* remove unused fields
2022-08-18 16:54:39 +02:00
ying-jeanne
82b63688d2
Chore: remove xorm from preference (#53803)
* Chore: remove xorm from preference

* separte feature toggle

* fix comments

* fix comments

* remove the dublicated namedexec
2022-08-17 22:07:20 -04:00
ying-jeanne
c7212643c2
add sqlx_store into star service (#53430) 2022-08-17 11:17:23 -05:00
ying-jeanne
25de383540
Chore: Replace xorm with sqlx (#52575)
* Change of sqlstore to use sqlx

* Use sqlx in the playlist store

* Refectory of the interface

* update playlist service

* go mod tidy

* some refectory on interface

* fix kyle
2022-08-16 13:17:14 -05:00
idafurjes
fa2e74cd6e
Chore: Remove GetSignedInUserWithCacheCtx from store interface (#53734)
* Remove delete suer from store interface

* Remove get signed in user with cache ctx from store interface

* Support options when setting up access control tests

* Fix broken tests

* Fix lint

* Add user fake to middleware

* Fix middleware tests, remove usertest being initialised twice

Co-authored-by: Karl Persson <kalle.persson@grafana.com>
2022-08-16 16:08:59 +02:00
idafurjes
1f442b419b
Chore: Remove disable user, disable batch users and searchusers methods from store interface (#53717)
* Chore: Remove disable user and searchusers methods from store interface

* Remove disable batch user from sqlstore interface

* Remove sqlstore from search store

* Fix lint
2022-08-16 14:24:57 +02:00
idafurjes
dfc75b1114
Remove delete suer from store interface (#53726) 2022-08-15 13:56:16 +02:00