* feat: add ability to launch targeted dskit modules in the grafana server CLI command
This commit adds a ModuleServer and ModuleRunner suitable for launching dskit services and updates the server cli command to use this instead of the full Server. The default behavior is unchanged and will launch the full Grafana server. Individual services are targeted by setting target=comma,seperated,list in the config file.
* require dev mode to target dskit modules
* remove unused type
* replace setting.CommandLineArgs w/setting.Cfg; the caller can deal with calling setting.NewCfg
* Update pkg/server/module_server.go
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
---------
Co-authored-by: Serge Zaitsev <serge.zaitsev@grafana.com>
This is preparative work for extending the grafana server cli command to target individual dskit modules that seemed worth breaking into this smaller PR. This moves the CLI flags and various reusable chunks of code into variables and methods.
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Avoid unnecessary byte/string conversion
We can use alternative functions/methods to avoid unnecessary
byte/string conversion calls.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* chore: remove export service POC from main
This is a POC and we'll see it, or something like it, again!
* remove frontend changes
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* add bundle registry service to avoid dependency cycles
* move user support bundle collector to user service
* move usage stat bundle implementation to usage stats
* add info for background service
* fix remaining imports
* whitespace
* Chore: Move team models to models pkg
* Fix ACL tests
* More ACL tests
* Change Id to ID in conflict user command test
* Remove team from models
* Fix ac test lint
* feat(grafana-cli): add a flag to control the admin user's ID for reset-admin-password
Since this is now more permissive, I've also added validation that the requested user is an admin.
* slight refactor to support testing
* refactor: renaming of files from database to store
* refactor: make service account store private
- moves store interface to manager package
- adds an interface to the ProvideAPI constructor
- refactors tests to use the store when necessary
- adds mocks for the new interface implementations in the tests package
* wip
* refactor: make fakestore in service
* wip
* wip
* wip
* working tests
* trailing whitespaces
* Update pkg/services/serviceaccounts/api/api.go
* Update pkg/services/serviceaccounts/tests/common.go
* Update pkg/services/serviceaccounts/tests/common.go
* refactor: doc string for retriever
* fix import unused
* remove: serviceaccount from featuretoggle
* added: back legacy serviceaccounts feature toggle
* added: docs
* refactor: make query for the SearchQuery
* add: validation of service input fields
* add validation
Adding support for backend plugin client middlewares. This allows headers in outgoing
backend plugin and HTTP requests to be modified using client middlewares.
The following client middlewares added:
Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client
and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies).
Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP
requests if the datasource has enabled Forward OAuth Identity (oauthPassThru).
Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming
HTTP request and used when authenticating to Grafana.
The current suggested way to register client middlewares is to have a separate package,
pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with
middlewares and/or longer term bootstrap/instantiate plugin management.
Fixes#54135
Related to #47734
Related to #57870
Related to #41623
Related to #57065
* Remove DeleteUser from sqlstore
* Use Delete instead of DeleteUser
* Remove unused method
* Remove DeleteUserSession from sqlstore
* Add fake for DeleteUseraccessControl
* Use user service, add fakes to tests
* Add comments to sqlstore
* Correct typo
* Add quota service initialisation
* Add config to acimpl initialisation
* Add routing to initialisation
* Making user provideStore private
* Use InTransaction instead of session
* Add cfg to userimpl
* Fix lint
* Make ProvideStore public again - enterprise tests
* Fix back ProvideStore to public
* Wrap merge user in transaction
* Delete DeleteUserAccessControl use DeleteUSerPermissions instead
* Add feature mgmt into acimpl
* DeleteUserAccessControl from ac database
* Remove case insensitive clause
* avoid the need for a second bulky binary for grafana-cli
* look for grafana-server in $PATH as well as same directory
* implement unified "grafana" command
* update dockerfiles, fix grafana-cli -v
* update packaging to work with single binary
- add wrapper scripts for grafana and grafana-server
- update and sync package files
- implement --sign flag of build package command
- stop packaging scripts folder, they are not useful for end users
- add support for --configOverrides in server command
- remove unused nfpm.yaml config file
* windows support
* LoginAttemps: Remove from sqlstore mock
* LoginAttemps: Move from models package to service package
* LoginAttemps: Implement functionallity from brute force login in service
* LoginAttemps: Call service
* LoginAttempts: Update name and remove internal functions
* LoginAttempts: Add tests
* LoginAttempt: Add service fake
* LoginAttempt: Register service as a background_services and remove job
from cleanup service
* LoginAttemps: Remove result from command struct
* LoginAttempt: No longer pass pointers
* Auth: move interface to its own file
* Auth: move to test package
* Auth: move quota consts to auth file
* Auth: move service to impl package
* Auth: move interfaces and related models to auth package
* Auth: Create sub package and type alias to avoid circular dependency
* docs: update terraform example and doc for file generation
* docs: updated the documnettaion to include the help command with fields included
* Update pkg/cmd/grafana-cli/commands/commands.go
* docs: add help command
* Update thema to latest
* Deal with s/Library/*Runtime/
* Commit new, working results of codegen
* We like pointers now
* Always take runtime arg for NewBase()
* Sketchy handwavy pass at entity meta framework
* Little nibbles
* Update pkg/framework/coremodel/entityframework.cue
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* Move file into new framework location
* Introduce loaders, Go code
* Complete rename to kind
* Flesh out framework, add svg/dashboard examples
* Cruft removal
* Remove generated kind go files from gitignore
* Refine maturity concept, add SlotKind
* Update embed and go deps
* Export PrefixWithGrafanaCUE
* Make the loader actually work, holy crap
* Many small tweaks to type.cue
* Add Apache 2 licensing exceptions for kinds
* Add new kinds dir, start of generator
* Roll back to earlier oapi-codegen
* Introduce new grafana-specific CUE loaders
* Introduce new tidy code generators framework
* Catch up kind framework with tinkering
* Add slices for the generators
* Add write/verify step to main generator
* Many renames
* Split up kind framework cue files
* Use kind.Decl within generated kinds
* Create kind.SomeDecl wrapper type to cache lineages
* Better names again
* Get one generated implemented, hopefully
* Copy dashboard schema into new kind.cue
* Small fixes to make the initial gen work
* Put svg kind in its new home
* Add generated Go dashboard type
* More renames and cleanups
* Add base kind registry and generator
* Stop blacklisting *_gen.go files
This is not the Go best practice, anyway. All we actually want to ignore
for enterprise is generated wire files.
* Change codegen output directories
pkg/kind -> pkg/kinds
pkg/registry/kindreg -> pkg/registry/corekind
* Rename pkg/framework/kind to pkg/kindsys
* Add core structured kind generator
* Add plural and machine names to kind spec
* Copy playlist over to kind system
* Consolidate kindsys files
* Add raw kind generator
* Update CODEOWNERS for kind framework
* Touch up comments a bit
* More docs tweaks
* Remove generated types to reduce noise for review
* Split each generator into its own file
* Rename Slot kind to Composable kind
* Add handwavy types for customkind loading
* Guard against init calls to framework loader
* First pass at doc on extending the kind system
* Improve attribute example in docs
* Fix wire imports
* Add basic TS types generator
* Fix composable kind category def
* No need for a separate file with generate directive
* Catch dashboard schema up
* Rename generator types to something saner and generic
* Make version configurable in ts/go generators
* Add CommonMeta to ease property access
* Add kindsys prop indicating whether lineage is group
* Put all kind categories back in a single file
* Finish with kindsys group props
* Refactor maturity progression per discussion
- Replace "committed" with "merged"
- All kindcats can use all maturity levels, at least for now
* Convert ts veneer index generator to modular system
* Move over to new jennywrites framework
* Strip down old coremodel generator
* Use public version of jennywrites
* Pull latest thema
* Commit generated Go types
* Add header injection postprocessor
* Move sdboyer/jennywrites to grafana/codejen
* Tweak header output
* Remove dashboard and playlist coremodels
* Fix up backend dashboards devenv test
* Fix TS import patterns to new gen filename
* Update internal imports, remove coremodel registry
* Fix compilation errors, wire generation
* Export and replace the prefix dropper
* More Go struct and field name changes
* Last name fixes, hopefully
* Fix lint errors
* Last lint error
Co-authored-by: Artur Wierzbicki <wierzbicki.artur.94@gmail.com>
* WIP
* add: better validation of conflict file
* add: better description of validation and ingest command
* add: check for at least one user to delete
* add: example in terraform to setup for conflicts
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Add: print of conflict block for error
- adds conflict block to error output for validation of the file to
easier diagnose in the file
* fix: formatting of errors
* fix: info strings improvements
* add: default 0 to blocks to check for users
* fixed: tests
* test integration
* fix strings fmt
* set store in resolver
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Add phlare datasource
* Rename
* Add parca
* Add self field to parca
* Make sure phlare works with add to dashboard flow
* Add profiling category and hide behind feature flag
* Update description and logos
* Update phlare icon
* Cleanup logging
* Clean up logging
* Fix for shift+enter
* onRunQuery to set label
* Update type naming
* Fix lint
* Fix test and quality issues
Co-authored-by: Joey Tawadrous <joey.tawadrous@grafana.com>
* Remove methods from sqlstore
* Remove commented out code
* Remove GetUserById from tests
* Adjust fake for get user profile
* Adjust test
* Adjust go mod files
* Try fix test
* Test adjustment
* Adjust test 2
* Remove commented out code
* chore: add alias for InitTestDB and Session
Adds an alias for the sqlstore InitTestDB and Session, and updates tests using these to reduce dependencies on the sqlstore.Store.
* next pass of removing sqlstore imports
* last little bit
* remove mockstore where possible
* Chore: move folder service interface into a separate package
* copy implementation into a standalone package
* move implementation and tests to the new folder package
* remove leftovers from wire
* add test doubles for folder service
* fix tests in library panels/elements
* fix provideservice in ngalert
* SQLStore: Ensure that sessions are always closed
Delete `NewSession()` in favour of `WithDbSession()`
* Add WithDbSessionForceNewSession to the interface
* Apply suggestions from code review
* add users-manager command
* add users-manager command
* rename files
* refactor: imports and renaming
* Command: add conflict merge user command
- MergeUser will
- replace all user_ids from conflicting users to the chosen userId
- delete users whose user_ids are not the chosen user
- SameIdentification will
- update chosen user with chosen email,login details
- delete users whose user_ids are not the chosen user
* refactor: clean up
* refactor: create structure for read, validate, ingest
* feat: ls and generate-file for conflicting users
* remove usagestats
* added back pkg/services/login/authinfoservice/database/stats.go
* Revert "added back pkg/services/login/authinfoservice/database/stats.go"
This reverts commit 2ba6e3c4d6.
* Revert "remove usagestats"
This reverts commit 1e3fa97810.
* cherry pick
* Revert "cherry pick"
This reverts commit 461626c306.
* validation of picked merge user
* fix test
* make lint
* make test run
* tests for ingest working
* clean up and refactored to align with downstream refactoring
* formatting
* refactor: name list instead of ls
* fix: static lint error use trimprefix
* WIP: permissions for validation
* fix: remove unused functions in sqlstore
* fix: remove unused function
* handling of multiple users and resolve discarded users
* fix tests
* fix: bug that did not exclude the blocks
* ioutil is blacklisted
* WIP: validation
* tests for merging a user working
* add latest changes to output print
* refactor: removed conflictEmail and conflictLogin that was not used
* refactor: code clean up, showChanges working
* test and linting fixes
* test and linting fixes
* refactor: removed logging of config and added more info for vlidation command
* refactor: fix order of code
* fix time now
* refactor: no longer need for check casesensitive login/email
* removed unnessecary loop
* refactor: move functions around
* test: working
* docs: add docuemntationf for file
* Add failing test for generating the conflict login block
* Fix regex
* Fix some stuff/tests
Co-authored-by: eleijonmarck <eric.leijonmarck@gmail.com>
* add: docs for conflict file
* add: conflict_email, conflict_login fields
* add: conflict_email, conflict_login fields
* WIP
* fix: tests working as intended
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* review comments
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
* Update pkg/cmd/grafana-cli/commands/conflict_user_command.go
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
* missspelling
* trailing new line
* update to use userimpl store
* remove newline
* remove newline
* refactor: initializing of resolver for conflicts
* fix: test sqlStore
* refactor: removed lines
* refactor: remove TODOs
Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Chore: move team store implementation to a separate package
* trying to fix more tests
* fix tests in service accounts and access control
* fix common tests
* restore commented out test
* add todos
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* RBAC: Allow app plugins restriction
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* Moving declaration to HttpServer
Co-Authored-By: marefr <marcus.efraimsson@gmail.com>
* Picking changes from the other branch
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Rename plugins.settings to plugins
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* Account for PluginAdminExternalManageEnabled
Co-authored-by: Will Browne <will.browne@grafana.com>
* Set metadata on instantiation
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: marefr <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
* split out plugin manager
* remove whitespace
* fix tests
* split up tests
* updating naming conventions
* simplify manager
* tidy
* add more fakes
* testing time
* add query verif to int test
* renaming
* add process tests
* tidy up manager tests
* add extra case to int test
* add more coverage to store and process tests
* remove comment
* fix capatilization
* init on provide
* remove addfromsource from API
* Refactor migrations and tests for secrets kvstore
* Use fake secrets store as a shortcut on tests
* Update wire
* Use global migration logger
* Fix ds proxy tests
* Fix linting issues
* Rename data source test setup function
* initial cut at migration from plugin
* create new migration from plugin
* only migrate to or from, not both
* remove cfg check from plugin migration itself
* update comments, clean up secret after migration
* add better error handling
* hook up REST API with migrations
* Minor fixes
* fix wire injection issue
* modify migrator to access plugin calls directly. create unit tests
* change pre-migration checks in admin api
* stop plugin after migrating from it
* fix compile issues after merge
* add comment about migration
* fix linting issue
* bleh, fix unit test
* fix another unit test
* update plugin error fatal flag after a migration from the plugin
* add extra logging to migration
* make linter happy
Co-authored-by: Leandro Deveikis <leandro.deveikis@gmail.com>
* RBAC: Rename interface to Store
* RBAC: Move ranme scopeInjector
* RBAC: Rename files to service
* RBAC: Rename to service
* RBAC: Split up accesscontrol into two components
* RBAC: Add DeclareFixedRoles to AccessControl interface
* Wire: Fix wire bindings
* RBAC: Move resolvers to root
* RBAC: Remove invalid test
* RBAC: Inject access control service
* RBAC: Implement the RoleRegistry interface in fake
* installer -> repo
* add semver format checking
* add plugin callbacks in test
* remove newline
* post install only scans new directories
* remove unused stuff
* everything in own package
* add missing cli params
* make grafana version part of the API
* resolve conflicts
* tidy up logger
* fix cli and tidy log statements
* rename log package
* update struct name
* fix linter issue
* fs -> filestore
* reorder imports
* alias import
* fix test
* fix test
* inline var
* revert jsonc file
* make repo dep of manager
* actually inject the thing
* accept all args for compatability checks
* accept compat from store
* pass os + arch vals
* don't inject fs
* tidy up
* tidy up
* merge with main and tidy fs storage
* fix test
* fix packages
* fix comment + field name
* update fs naming
* fixed wire
* remove unused func
* fix mocks
* fix storage test
* renaming
* fix log line
* fix test
* re-order field
* tidying
* add test for update with same version
* fix wire for CLI
* remove use of ioutil
* don't pass field
* small tidy
* ignore code scanning warn
* fix testdata link
* update lgtm code
* Rename file to store
* Move resource permission specific database functions to
resourcepermissions package
* Wire: Remove interface bind
* RBAC: Remove injection of resourcepermission Store
* RBAC: Export store constructor
* Tests: Use resource permission package to initiate store used in tests
* RBAC: Remove internal types package and move to resourcepermissions
package
* RBAC: Run database tests as itegration tests
* add users-manager command
* add users-manager command
* rename files
* refactor: imports and renaming
* Command: add conflict merge user command
- MergeUser will
- replace all user_ids from conflicting users to the chosen userId
- delete users whose user_ids are not the chosen user
- SameIdentification will
- update chosen user with chosen email,login details
- delete users whose user_ids are not the chosen user
* refactor: clean up
* refactor: create structure for read, validate, ingest
* feat: ls and generate-file for conflicting users
* remove usagestats
* added back pkg/services/login/authinfoservice/database/stats.go
* Revert "added back pkg/services/login/authinfoservice/database/stats.go"
This reverts commit 2ba6e3c4d6.
* Revert "remove usagestats"
This reverts commit 1e3fa97810.
* cherry pick
* Revert "cherry pick"
This reverts commit 461626c306.
* fix test
* make lint
* make test run
* clean up and refactored to align with downstream refactoring
* formatting
* refactor: name list instead of ls
* fix: static lint error use trimprefix
* fix: remove unused functions in sqlstore
* fix: remove unused function
* handling of multiple users and resolve discarded users
* fix tests
* fix: bug that did not exclude the blocks
* ioutil is blacklisted
* should not run tests for mysql
* fix tests
