* Unified Storage: First iteration Dual Write Syncer
Signed-off-by: Maicon Costa <maiconscosta@gmail.com>
Co-authored-by: Leonor Oliveira <9090754+leonorfmartins@users.noreply.github.com>
Co-authored-by: Dan Cech <dcech@grafana.com>
* model fixed roles for dashboards and folders
* Correctly translate fixed role assignments
* minor refactor
* assign fixed roles to teams
* fix linter errors
* Migrate general folder permissions for fixed roles
* fix dashboards:create permission
* Access control: Use composite cache key for team permissions
* use composite key for teams
* use cache for hotpath (getCachedUserPermissions)
* fix linter
* fix sorting
---------
Co-authored-by: Jeff Levin <jeff@levinology.com>
* add uid to template and populate it
* update delete method to support both uid and name
* update UpdateTemplate to support search by UID and fallback to name + support renaming of the template
* update upsert to exit if template not found and uid is specified
* update Get method to address by name or uid
---------
Co-authored-by: Matthew Jacobson <matthew.jacobson@grafana.com>
* Replace global authz abstraction with one compatible with uid scope
* Replace GettableApiReceiver with models.Receiver in receiver_svc
* GrafanaIntegrationConfig -> models.Integration
* Implement Create/Update methods
* Add optimistic concurrency to receiver API
* Add scope to ReceiversRead & ReceiversReadSecrets
migrates existing permissions to include implicit global scope
* Add receiver create, update, delete actions
* Check if receiver is used by rules before delete
* On receiver name change update in routes and notification settings
* Improve errors
* Linting
* Include read permissions are requirements for create/update/delete
* Alias ngalert/models to ngmodels to differentiate from v0alpha1 model
* Ensure integration UIDs are valid, unique, and generated if empty
* Validate integration settings on create/update
* Leverage UidToName to GetReceiver instead of GetReceivers
* Remove some unnecessary uses of simplejson
* alerting.notifications.receiver -> alerting.notifications.receivers
* validator -> provenanceValidator
* Only validate the modified receiver
stops existing invalid receivers from preventing modification of a valid
receiver.
* Improve error in Integration.Encrypt
* Remove scope from alert.notifications.receivers:create
* Add todos for receiver renaming
* Use receiverAC precondition checks in k8s api
* Linting
* Optional optimistic concurrency for delete
* make update-workspace
* More specific auth checks in k8s authorize.go
* Add debug log when delete optimistic concurrency is skipped
* Improve error message on authorizer.DecisionDeny
* Keep error for non-forbidden errutil errors
* WIP
* Validate new field, and add value in provisioning if not defined in correct spot
* Simplify logic, use correct value
* fix tests
* Fix linter errors
* fix swagger and tests
* 😬
* Auto-generation isnt doing this..
* Fix linter
* test if nullable is the issue…
* Change structure on the frontend fields
* Try with backtick
* try programatic quoting
* Try only quote non-ints
* quoting, no backticks
* Remove debugging
* feat(nameHeaders): add feature flag
* add safe parsing of headers
* use headers in loki datasource
* Loki: add option to pass headers to Loki
* Loki: add datasource tests for dashboard names
* cleanup
* DataSourceWithBackend: add test
* rename to `sanitizeHeader`
* Loki: add condition when to add headers
* Loki: add e2e tests
* Loki: change test name
* feat: Add new read filtering to datasources guardian
* Apply suggestion to use datasources read guardian check for frontend settings
---------
Co-authored-by: Eric Leijonmarck <eric.leijonmarck@gmail.com>
* chore: add tracing to quote API and service methods with contexts
I also fixed a typo (overriden -> overridden) and removed a method that looked like it wasn't useful anymore. (It seemed to exist to return an error, but never returned an error, and so just added many lines of unnecessary error checking).
* chore: Bump Go to 1.23.0
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* update swagger files
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* chore: update .bingo/README.md formatting to satisfy prettier
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* chore(lint): Fix new lint errors found by golangci-lint 1.60.1 and Go 1.23
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* keep golden file
* update openapi
* add name to expected output
* chore(lint): rearrange imports to a sensible order
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
---------
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* chore: add spans to publicdashboards service methods
* add tracing to test service
* test fixture whackamole
* move tracer to a package var
* Update pkg/services/publicdashboards/service/service.go
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
---------
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
* update oauthtoken service to use remote cache and server lock
* remove token cache
* retry is lock is held by an in-flight refresh
* refactor token renewal to avoid race condition
* re-add refresh token expiry cache, but in SyncOauthTokenHook
* Add delta to the cache ttl
* Fix merge
* Change lockTimeConfig
* Always set the token from within the server lock
* Improvements
* early return when user is not authed by OAuth or refresh is disabled
* Allow more time for token refresh, tracing
* Retry on Mysql Deadlock error 1213
* Update pkg/services/authn/authnimpl/sync/oauth_token_sync.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Update pkg/services/authn/authnimpl/sync/oauth_token_sync.go
Co-authored-by: Dan Cech <dcech@grafana.com>
* Add settings for configuring min wait time between retries
* Add docs for the new setting
* Clean up
* Update docs/sources/setup-grafana/configure-grafana/_index.md
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
---------
Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* add RenameTimeIntervalInNotificationSettings to storage
* update dependencies when the time interval is renamed
---------
Co-authored-by: William Wernert <william.wernert@grafana.com>
* Alerting: Fix duplicated silences in remote primary mode bug
* test that a new silence id returned by calling CreateSilence() on the internal Alertmanager is ignored
* Cloud migrations: GetSnapshotList only returns snapshots that belong to a session that exists
* Cloud migrations: test GetSnapshotList
* add one more test case for GetSnapshotList
* fix test
* store encryption key for testing
* Alerting: Add rule_group label to grafana_alerting_rule_group_rules metric (#62361)
* Alerting: Delete rule group metrics when the rule group is deleted
This commit addresses the issue where the GroupRules metric (a GaugeVec)
keeps its value and is not deleted when an alert rule is removed from the rule registry.
Previously, when an alert rule with orgID=1 was active, the metric was:
grafana_alerting_rule_group_rules{org="1",state="active"} 1
However, after deleting this rule, subsequent calls to updateRulesMetrics
did not update the gauge value, causing the metric to incorrectly remain at 1.
The fix ensures that when updateRulesMetrics is called it
also deletes the group rule metrics with the corresponding label values if needed.
* PluginDetailsRight panel is added. All the details were moved from the top to the right panel
* Add feature toggle pluginsDetailsRightPanel,Fix build, fix review comments
* Fix the typo
Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com>
* hasAccessToExplore
* changes after review, add translations
* fix betterer
* fix betterer
* fix css error
* fix betterer
* fix translation labels, fix position of the right panel
* fix the build
* add condition to show updatedAt for plugin details
* add test to check 2 new fields at plugin details right panel;
* change the gap and remove report abuse button from core plugins
* add more tests
---------
Co-authored-by: Giuseppe Guerra <giuseppe.guerra@grafana.com>
* Refactor identity struct to store type in separate field
* Update ResolveIdentity to take string representation of typedID
* Add IsIdentityType to requester interface
* Use IsIdentityType from interface
* Remove usage of TypedID
* Remote typedID struct
* fix GetInternalID
* Remove kubernetesPlaylists feature_toggle
* Remove unified_storage_mode
* Remove double import
* Regenerate feature-toggles
* Read from config instead from feature_toggle
* cover scenario for when unified storage is not defined
* Handle namespace and group query string params in Ruler API
* Use the new namespace and group query params when slashes in names
* Add validation, add group handling in GMA Api
* Move constants
* Use checkForPathSeparator function
* Fix linter issue
* support optimistic concurrency in template service
* update request handler to get version from query parameter
* return not found if a new template is set with version
* update PUT api to set version
* update documentation + for mute timings
---------
Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
* Feature (quota service): Use ReplDB for quota service Gets
This adds the replDB to the quota service, as well as some more test helper functions to simplify updating tests. My intent is that the helper functions can be removed when this is fully rolled out (or not) and we're consistently using the ReplDB interface (or not!)
* test updates
* Add migration to enable TraceQL streaming for Tempo datasources
* lint
* Always run migration but exit early if feature flag is disabled
* Require feature toggle OR datasource config to enable streaming
* minor performance improvement
* apply a warning to any non-core plugins that successfully migrate
* commit frontend wip while I refactor some stuff
* update api
* repurpose error dialog to be a generic details dialog
* whitespace
* add unit test
* fixes from testing
* fix migration summary
* add comment
* fix localization stuff
* fix backend test
* reduce number of queries to the db
* some PR feedback
* whitespace
This PR adds instrumentation for loading frontend SPA along with select methods in the dashboard service, and cleans up span handling in sqlstore.
---------
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
* Check for RBAC permissions when hitting query history endpoints; extract checking logic into a middleware
* Fix lint errors
* Fix test
* Use permissions for patch path; rename callback handler
* refactor replCfg to look more like plugins/plugin config
* validateReplicaConfigs must handle inconsistencies in type names due to the WithHooks suffix
* refactor `selectorString` and remove Selector struct
* move code from selector string to BuildLogQuery
* batch requests by folder UID
* update historian annotation store to handle multiple queries
* sort folder uids to make consistent queries
* add logs to loki http
* log batch size but not content. content is logged by the client
* send dashboard commands instead of dashboards
* move status updates before goroutine to ensure frontend polls
* fix syncing issues between snapshot state and resources
* make sessionUid a requirement for modifying snapshots
* move the function I meant to move earlier
* remove accidental commit
* another accidental commit
* verify UpdateSnapshot is called with sessionUid
* revert
* pass in session uid everywhere
* forgot to save
* fix unit test
* fix typo
* tiny tweak
* RBAC sync: Fix removal of roles which need to be added
* Optimize code
* cleanup: appease the linter
---------
Co-authored-by: Victor Cinaglia <victor@grafana.com>
* handle metadata map nil
* remove double context
* clean up logging in scheduler
* do not reuse loggers from previous ticks
* log the dropped tick
* log tick instead of ticknum
* replace with processing tick logs
* log sending notifications
* update logging in persister to fetch context
* logs to historian
moved them upstream to be able to log when store is overridden
* E2C: Add stat rollup to MigrationSummary
* fix report event url
* open form in new page
* sort folders by heirarchy
* undo accidental commit
* remove another commit
* make folder sorting dynamic
---------
Co-authored-by: joshhunt <josh@trtr.co>
* rename to getMuteTimingByName
* add UID to api model of MuteTiming
* update GetMuteTiming to search by UID
* update UpdateMuteTiming to support search by UID
* update DeleteMuteTiming to support uid
* make sure UID is populated
* update usages
* use base64 url-safe, no padding encoding for UID
* make the resource store the default unified storage backend
* add integration tests
* fix test non passing
* Update pkg/storage/unified/sql/test/integration_test.go
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* lint
* fix tests
* fix no rows
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* store encryption key in unified secrets table
* fix local dev mode
* make metadata more realistic
* fix tests
* fix sql queries against postgres
* fix stats endpoint
* Cfg: Move rbac settings to own struct
* Cfg: Add setting to control if resource should generate managed permissions when created
* Dashboards: Check if we should generate default permissions when dashboard is created
* Folders: Check if we should generate default permissions when folder is created
* Datasource: Check if we should generate default permissions when datasource is created
* ServiceAccount: Check if we should generate default permissions when service account is created
* Cfg: Add option to specify resources for wich we should default seed
* ManagedPermissions: Move providers to their own files
* Dashboards: Default seed all possible managed permissions if configured
* Folders: Default seed all possible managed permissions if configured
* Cfg: Remove service account from list
* RBAC: Move utility function
* remove managed permission settings from the config file examples, change the setting names
* remove ini file changes from the PR
* fix setting reading
* fix linting errors
* fix tests
* fix wildcard role seeding
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: jguer <me@jguer.space>
* Itroduce watcher and mode4
* Logging
* Mode4 should be initialized from the dual writer for observability
* Comment watch while it's not implemented
* Lint
* Use mode log when dual writer is initiated
* Use error from logger
* check that a user doesn't have higher plugin access on the destination folder than they have on the source folder when moving folders
* Update pkg/services/folder/folderimpl/folder_test.go
---------
Co-authored-by: Jo <joao.guerreiro@grafana.com>
* cleanup dependencies and improve list method
* Improve Resource Server API, remove unnecessary dependencies
* Reduce the API footprint of ResourceDBInterface and its implementation
* Improve LifecycleHooks to use context
* Improve testing
* reduce API size and improve code
* sqltemplate: add DialectForDriver func and improve naming
* improve lifecycle API
* many small fixes after adding more tests
* Navigation: Show list of pinned ites on the navigation
* Rename section to 'Bookmarks'
* Internationalization
* Rename everything to bookmarks
* Update public/app/core/reducers/navBarTree.ts
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* Ignore empty message as well
* Dont update navigation if there is an error patching
---------
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* add gms client function
* add timeout config for endpoint
* report events to gms
* fix lint error
* clean up report calls and make sure reports all have local ids
* extra validation
* improve error logging and fix url
* Add permissions check for viewer without viewers_can_edit
* Add test
* fix lint
* Add role checks on other handlers
* Linter and fix Go issue
* Fix conflict
* Remove invalid way of testing for error
* initial commit
* Action sets stored
remove the dependancy for actionsets
got the actionsets registered
storing the permissions
* fix golanglinting
* remove unused struct field
* wip
* actionset registry for a plugin from the actionsetservice
* update to make declareactionset the primary way of plugin registration and modification
* declare actually extends actionsets
* tests fixed
* tests skipped
* skip tests
* skip tests
* skip tests
* skip tests
* change to warning instead
* remove step from pipeline to see if it fails due to plugin not registering
* reintroduce step but remove features dependancy
* add back the tests that were failing
* remove comments and another skip test
* fix a comment and remove unneeded changes
* fix and clean up, put the behaviour behind a feature toggle
* clean up
* fixing tests
* hard-code allowed action sets for plugins
* Apply suggestions from code review
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* small cleanup
---------
Co-authored-by: IevaVasiljeva <ieva.vasiljeva@grafana.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* change the rule-group to be hashed when exporting to HCL
Signed-off-by: Aviv Guiser <avivguiser@gmail.com>
---------
Signed-off-by: Aviv Guiser <avivguiser@gmail.com>
* Add success case and tests for writer using metrics
* Use testable version of clock
* Assert a specific series was written
* Fix linter
* Fix manually constructed writer
* improve error handling a retries during async operations
* fix use of contexts
* updates to how we call the folder api
* fix urls for gms
* more progress on the folder issue
* fix folders
* refactor for readability
* add support of metadata to condition and adding it to request headers
* support for additional metadata when condition is built
* add additionall context to conditions: source and folder title
* add version
* use percent-encoding for header values
* Check if a time interval is used in alert rules before deleting it
* Add time interval to parameters of ListAlertRulesQuery and ListNotificationSettings of DbStore
== Refacorings ==
* refactor isMuteTimeInUse to accept a single route
* update getMuteTiming to not return err
* update delete to get the mute timing from config first
* Re-add feature flag with deprecation note
* Hide the field in frontend if ff disabled
* Block scope overriding if ff is disabled in backend
- Update promlib to forward logger to extendOptions
- Add warning
- Update tests
* Default toggle to true for now
* Update description
* Update prom tests
* Fix lint
* Add search index table
* Stab a test
* Add more tests
* Add basic index
* Switch to UID and add a test for the index
* Improve tests coverage
* Remove redundant whitespaces
* Load all data source APIs when query history is loaded
* Fix column type
* Fix migration
* Clean-up the index
* Fix linting
* Fix migrations
* Fix migrations
* Fix migrations
* Rename index to details
* implement querying gms for snapshot status
* add some documentation
* provide snapshot resources after snapshot is created
* add rate limiting to backend
* fix compilation error
* fix typo
* add unit tests
* finish merge
* lint
* swagger gen
* more testing
* remove duplicate test
* address a couple PR comments
* update switch statement to a map
* add timeouts to gms client through the http client
* remove extra whitespace
* put method back where it was so the PR is less confusing
* fix tests
* add todo
* fix final unit test
* Create some integration testing infra for RRs
* whoops
* Require no error in responding
* fix linter
* Panic, no need to pass testing around
* Extend status test
* Cloud migration: upload snapshot files using presigned url
* log error if index file cannot be closed
* log error if file cannot be closed in uploadUsingPresignedURL
* Implement EventDetails for expanded rows and pagination on the events list
* Add test for getPanelDataForRule function
* prettier
* refactor EventState component
* create interfaces for props
* Add missing translations
* Update some comments
* Add plus button in alertrulename , to add it into the filter
* Add plus button to add filters from the list labels and alert name
* Add clear labels filter button
* run prettier
* fix RBAC checks
* Update AlertLabels onLabelClick functionality
* add limit=0 in useCombinedRule call
* Add filter by state
* remove plus button in labels
* Fix state filter
* Add filter by previous state
* fix some errors after solving conflicts
* Add comments and remove some type assertions
* Update the number of transitions calculation to be for each instance
* Add tests for state filters
* remove type assertion
* Address review comments
* Update returnTo prop in alert list view url
* Update translations
* address review comments
* prettier
* update cursor to pointer
* Address Deyan review comments
* address review pr comments from Deyan
* fix label styles
* Visualize expanded row as a state graph and address some pr review comments
* Add warning when limit of events is reached and rename onClickLabel
* Update texts
* Fix translations
* Update some Labels in the expanded states visualization
* move getPanelDataForRule to a separate file
* Add header to the list of events
* Move HistoryErrorMessage to a separate file
* remove getPanelDataForRule function and test
* add comment
* fitler by instance label results shown inthe state chart
* remove defaults.ini changes
* fix having single event on time state chart
---------
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
* order session list descending
* add snapshot status method to store
* query stats while retrieving snapshot
* return stats in dto
* swagger
* fix tests
* commit results of bingo get
* fix swagger
* minor improvement
* fix typo
* forgot a file
* Plugins: Enhanced plugin instrumentation
* use backend.CallResourceResponseSenderFunc
* sdk v0.237.0
* support admission control
* cover all handlers in log and metrics middlewares
* fix after review
* soft delete
* Fix bench test
Co-authored-by: Bruno Abrantes <bruno@brunoabrantes.com>
* Add integration test for soft deletion
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* fix kind of TimeInterval
* register custom fields for selectors
* support field selectors in legacy storage
* support selectors in storage
===== Misc
* refactor conversions to build in one place
* hide implementation of provenance status behind accessors to use the key in selectors
* fix provenance error
* Unify values
* Fix with latest changes on main
* Fix up NaN test
* Keep refIDs with -1 as value
* Test that refIDs are preserved on Normal to Error transition
* Alerting to err test too
* Add a blurb to docs about this behavior
The contact point deletion API was returning 500 when it should have been
returning a 4xx error, when the contact point is in use:
- When in use by a notificiation policy, we were missing
the `.Errorf("")` to convert `errutil.Base` into `errutil.Error`.
- When in use by an alert rule, an regular error was returned.
* filter the k6 folder out in the SQL queries rather than during post processing to ensure that the correct number of results is always returned
* linting
* Add org_mapping and org_attribute_path to the UI
* Add validators, allow setting org mapping to only Grafana Admins
* comment
* Address feedback, improve validation, fix FE test, lint
* Cloud migrations: create snapshot and store it on disk
* fix merge conflicts
* implement StartSnapshot for gms client
* pass snapshot directory as argument to snapshot builder
* ensure snapshot folder is set
* make swagger-gen
* remove Test_ExecuteAsyncWorkflow
* pass signed in user to buildSnapshot method / use github.com/grafana/grafana-cloud-migration-snapshot to create snapshot files
* fix FakeServiceImpl.CreateSnapshot
* remove new line
Adds more spans for timing in accesscontrol and remove permission deduplicating code after benchmarking
---------
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Zanzana: Listen http to handle fga cli requests.
* make configurable
* start http server during service run
* wait for GRPC server is ready
* remove unnecessary logs
* fix linter errors
* run only in devenv
* make address configurable
* Alerting: Add setting for maximum allowed rule evaluation results
Added a new configuration setting `quota.alerting_rule_evaluation_results` to set the maximum number of alert rule evaluation results per rule. If the limit is exceeded, the evaluation will result in an error.
This PR reduces the number of allocations made while caching permissions from the database, fixes the hierarchy of spans and adds new spans for tracing.
---------
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Co-authored-by: Dave Henderson <dave.henderson@grafana.com>
* add method CanReadAllRules to rule authorization service
* add alias type Namespace for Folder in ngalert's models package. It implements the Namespacer interface that is used by authz logic
* update state history's backends to authorize access to rules.
* update Loki to add folders UIDs to query.
* Update BuildLogQuery to drop filter by folders if it's too long and fall back to in-memory filtering.
