* reload SSO settings for HA setups
* remove check for grafana HA
* add unit tests
* fetch all sso settings with one sql query
* register background service
* Plugin: Remove external service on plugin removal
* Early exit no service account
* Add log
* WIP
* Cable OAuth2Server client removal
* Move function lower
* Add function to test removal
* Add test to RemoveExternalService
* Test RemoveExtSvcAccount
* remove apostrophy in comment
* Add cfg to plugin installer to check features
* Add feature flag check in the service registration service
* Comments
* Move metrics Inc
* Initialize map
* Reorder
* Initialize mutex as well
* Add HasExternalService as suggested
* WIP: CleanUpOrphanedExternalServices
* Commit suggestion
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Nit on test.
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* oauthserver return names
* Name is not Slug
* Use plugin ID not slug
* Add background job
* remove negation on feature check
* Add test to the CleanUp function
* Test GetExternalServiceNames
* rename test
* Add test for ExtSvcAccountsService_GetExternalServiceNames
* Add a todo
* Add todo
* Option based on mix
* Rewrite a bit the comment
* Opinionated choice use slugs instead of names everywhere
* Nit.
* Comments and re-ordering
* Comment
* Add log
* Add context
---------
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* inital changes, db migration
* changes
* Implement basic GetAll, Delete
* Add first batch of tests
* Add more tests
* Add service tests for GetForProvider, List
* Update http_server.go + wire.go
* Lint + update fixed role
* Update CODEOWNERS
* Change API init
* Change roles, rename
* Review with @kalleep
* Revert a mistakenly changed part
* Updates based on @dmihai 's feedback
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Alerting: Move migration from background service run to ngalert init
sqlite database write contention between the migration's single transaction and
dashboard provisioning's frequent commits was causing the migration to
fail with SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT on all retries.
This is not a new issue for sqlite+grafana, but the discrepancy between the
length of the transactions was causing it to be very consistent. In addition,
since a failed migration has implications on the assumed correctness of the
alertmanager and alert rule definition state, we cause a server shutdown on
error. This can make e2e tests as well as some high-load provisioned
sqlite installations flaky on startup.
The correct fix for this is better transaction management across various
services and is out of scope for this change as we're primarily interested in
mitigating the current bout of server failures in e2e tests when using sqlite.
This PR replaces the vendored models in the migration with their equivalent ngalert models. It also replaces the raw SQL selects and inserts with service calls.
It also fills in some gaps in the testing suite around:
- Migration of alert rules: verifying that the actual data model (queries, conditions) are correct 9a7cfa9
- Secure settings migration: verifying that secure fields remain encrypted for all available notifiers and certain fields migrate from plain text to encrypted secure settings correctly e7d3993
Replacing the checks for custom dashboard ACLs will be replaced in a separate targeted PR as it will be complex enough alone.
* AuthN: Move identity struct to its own file
* IDForwarding: Add IDToken property to usr and identity structs and add GetIDToken to requester interface
* Inject IDService into background services
* IDForwarding: Register post auth hook when feature toggle is enabled
* remove API tagging method and authed tagging
* add anonstore
move debug to after cache
change test order
fix issue where mysql trims to second
* add old device cleanup
lint
utc-ize everything
trim whitespace
* remove dangling setting
* Add delete devices
* Move anonymous authnclient to anonimpl
* Add simple post login hook
* move registration of Background Service
cleanup
* add updated_at index
* do not untag device if login err
* add delete device integration test
* extract kindsys
* reinstate kindsys report
This may end up living somewhere else (or not! who knows!), but the important part is that I don't get rid of it right now :)
I hate the package layout (kindsysreport/codegen) for the main function and will take pretty much any alternative suggestion, but we can change also change it later.
Note that the generated report.json is in a different location - anything using this (ops something) needs to be updated.
* kindsysreport in codeowners
* Kindsys: Replace DeclForGen with kindsys.Kind
DeclForGen was always unnecessary - it just wasn't obvious on initial
implementation, when we were focused on generating unique types for each
core kind. This removes it, considerably simplifying interactions with
kindsys - virtually everything now just relies on kindsys.Kind and its
derived interfaces.
* Removed unused jenny
* Rename params in jennies
* Adding service accounts schema
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Change metadata field
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Merging the two DTOs into one schema
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Eric dedup things
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Add optional to teams/created/updated
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Lower case the OrgId
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* Use preferred package header for generated code
> To convey to humans and machine tools that code is generated, generated source should have a line that matches the following regular expression (in Go syntax):
> `^// Code generated .* DO NOT EDIT\.$`
https://pkg.go.dev/cmd/go#hdr-Generate_Go_files_by_processing_source
Co-authored-by: sam boyer <sdboyer@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
* Regenerate files with updated header
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
Signed-off-by: Jack Baldry <jack.baldry@grafana.com>
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Remove Raw references
* Remove more raws
* Re-generate files
* Remove raw folder from veneer
* Fix import
* Fix lint
* Bring back raw folder in grafana-schema
* Another lint
* Remove use of "Structured" word in kinds
* Delete unused function and remove some structured words
* Bunch more removals of structured name
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Add Team kind
* Minor changes
* Use ToMedatata annotation
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Use ToMedatata annotation
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Use ToMedatata annotation
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Use ToMedatata annotation
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Update comments
* Add created and updated fields
* Make orgId required
* Use int64 for datetime
Co-authored-by: sam boyer <sdboyer@grafana.com>
* Update thema to latest
* Deal with s/Library/*Runtime/
* Commit new, working results of codegen
* We like pointers now
* Always take runtime arg for NewBase()
* Sketchy handwavy pass at entity meta framework
* Little nibbles
* Update pkg/framework/coremodel/entityframework.cue
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* Move file into new framework location
* Introduce loaders, Go code
* Complete rename to kind
* Flesh out framework, add svg/dashboard examples
* Cruft removal
* Remove generated kind go files from gitignore
* Refine maturity concept, add SlotKind
* Update embed and go deps
* Export PrefixWithGrafanaCUE
* Make the loader actually work, holy crap
* Many small tweaks to type.cue
* Add Apache 2 licensing exceptions for kinds
* Add new kinds dir, start of generator
* Roll back to earlier oapi-codegen
* Introduce new grafana-specific CUE loaders
* Introduce new tidy code generators framework
* Catch up kind framework with tinkering
* Add slices for the generators
* Add write/verify step to main generator
* Many renames
* Split up kind framework cue files
* Use kind.Decl within generated kinds
* Create kind.SomeDecl wrapper type to cache lineages
* Better names again
* Get one generated implemented, hopefully
* Copy dashboard schema into new kind.cue
* Small fixes to make the initial gen work
* Put svg kind in its new home
* Add generated Go dashboard type
* More renames and cleanups
* Add base kind registry and generator
* Stop blacklisting *_gen.go files
This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.
* Change codegen output directories
pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind
* Rename pkg/framework/kind to pkg/kindsys
* Add core structured kind generator
* Add plural and machine names to kind spec
* Copy playlist over to kind system
* Consolidate kindsys files
* Add raw kind generator
* Update CODEOWNERS for kind framework
* Touch up comments a bit
* More docs tweaks
* Remove generated types to reduce noise for review
* Split each generator into its own file
* Rename Slot kind to Composable kind
* Add handwavy types for customkind loading
* Guard against init calls to framework loader
* First pass at doc on extending the kind system
* Improve attribute example in docs
* Fix wire imports
* Add basic TS types generator
* Fix composable kind category def
* No need for a separate file with generate directive
* Catch dashboard schema up
* Rename generator types to something saner and generic
* Make version configurable in ts/go generators
* Add CommonMeta to ease property access
* Add kindsys prop indicating whether lineage is group
* Put all kind categories back in a single file
* Finish with kindsys group props
* Refactor maturity progression per discussion
- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now
* Convert ts veneer index generator to modular system
* Move over to new jennywrites framework
* Strip down old coremodel generator
* Use public version of jennywrites
* Pull latest thema
* Commit generated Go types
* Add header injection postprocessor
* Move sdboyer/jennywrites to grafana/codejen
* Tweak header output
* Remove dashboard and playlist coremodels
* Fix up backend dashboards devenv test
* Fix TS import patterns to new gen filename
* Update internal imports, remove coremodel registry
* Fix compilation errors, wire generation
* Export and replace the prefix dropper
* More Go struct and field name changes
* Last name fixes, hopefully
* Fix lint errors
* Last lint error
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
Fixes#30144
Co-authored-by: dsotirakis <sotirakis.dim@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Ida Furjesova <ida.furjesova@grafana.com>
Co-authored-by: Jack Westbrook <jack.westbrook@gmail.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
Co-authored-by: Leon Sorokin <leeoniya@gmail.com>
Co-authored-by: Andrej Ocenas <mr.ocenas@gmail.com>
Co-authored-by: spinillos <selenepinillos@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Leonard Gram <leo@xlson.com>
* middleware: Move context handler to own service
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
Co-authored-by: Emil Tullsted <sakjur@users.noreply.github.com>
Co-authored-by: Will Browne <wbrowne@users.noreply.github.com>
* Alerting NG: prototype v2 (WIP)
* Separate eval package
* Modify eval alert definition endpoint
* Disable migration if ngalert is not enabled
* Remove premature test
* Fix lint issues
* Delete obsolete struct
* Apply suggestions from code review
* Update pkg/services/ngalert/ngalert.go
Co-authored-by: Kyle Brandt <kyle@grafana.com>
* Add API endpoint for listing alert definitions
* Introduce index for alert_definition table
* make ds object for expression to avoid panic
* wrap error
* Update pkg/services/ngalert/eval/eval.go
* Swith to backend.DataQuery
* Export TransformWrapper callback
* Fix lint issues
* Update pkg/services/ngalert/ngalert.go
Co-authored-by: Kyle Brandt <kyle@grafana.com>
* Validate alert definitions before storing them
* Introduce AlertQuery
* Add test
* Add QueryType in AlertQuery
* Accept only float64 (seconds) durations
* Apply suggestions from code review
* Get rid of bus
* Do not export symbols
* Fix failing test
* Fix failure due to service initialization order
Introduce MediumHigh service priority and assign it to backendplugin
service
* Fix test
* Apply suggestions from code review
* Fix renamed reference
Co-authored-by: Kyle Brandt <kyle@grafana.com>
In OSS provisioning service init after plugin registration, but in
Enterprise it's the opposite order and installed app plugin check
fails. This adjusts service registry init priority to make sure plugins
are registered before provisioning inits.
Which issue(s) this PR fixes:
Fixes#26336
* enable ee build on pr/master
* step1: of including group sync
* disable commit pinning for now
* fixes broken build
* enable team to ldap group sync
* avoid returning error for missing external handler
* services: allow routes to be added before http server start
* services: allows services to add their own migrations
* moves db migrations to ee code base
* build using master branch in ee
* disable enterprise build in .bra.toml
[skip ci]
* removes team sync extensions
* removes commented line
* refactor: tracing service refactoring
* refactor: sqlstore to instance service
* refactor: sqlstore & registory priority
* refactor: sqlstore refactor wip
* sqlstore: progress on getting tests to work again
* sqlstore: progress on refactoring and getting tests working
* sqlstore: connection string fix
* fix: not sure why this test is not working and required changing expires
* fix: updated grafana-cli
* extensions: import and build
* bus: use predefined error
* enterprise: build script for enterprise packages
* poc: auto registering services and dependency injection
(cherry picked from commit b5b1ef875f905473af41e49f8071cb9028edc845)
* poc: backend services registry progress
(cherry picked from commit 97be69725881241bfbf1e7adf0e66801d6b0af3d)
* poc: minor update
(cherry picked from commit 03d7a6888b81403f458b94305792e075568f0794)
* ioc: introduce manuel ioc
* enterprise: adds setting for enterprise
* build: test and build specific ee commit
* cleanup: test testing code
* removes example hello service
