* introduce feature toggle
* create base service structure
* fix sample metric
* register metrics
* add to codeowners
* separate api dtos from service models
* remove leading newline
* add clearer comment for function def
* update test to reflect change in range for 1w step
* clarify docs
* add more clarity
* add explanation to query options min interval and link to min step
* Update docs/sources/panels-visualizations/query-transform-data/_index.md
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
---------
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
* Add AuthNSvc reload handling
* Working, need to add test
* Remove commented out code
* Add Reload implementation to connectors
* Align and add tests, refactor
* Add more tests, linting
* Add extra checks + tests to oauth client
* Clean up based on reviews
* Move config instantiation into newSocialBase
* Use specific error
These don't get marshalled and unmarshalled in the same way as they are represented in Go
This PR changes the OpenAPI spec to reflect what the API accepts and sends back
* Simple, per-base-interval jitter
* Add log just for test purposes
* Add strategy approach, allow choosing between group or rule
* Add flag to jitter rules
* Add second toggle for jittering within a group
* Wire up toggles to strategy
* Slightly improve comment ordering
* Add tests for offset generation
* Rename JitterStrategyFrom
* Improve debug log message
* Use grafana SDK labels rather than prometheus labels
Sets that status code on backend data responses in prometheus to match the status code returned by prometheus. If the failure is below HTTP/Application Layer, Bad Gateway is returned (502).
---------
Co-authored-by: ismail simsek <ismailsimsek09@gmail.com>
* add deployment registry API cloud only
* update versions
* add feature flag endpoints
* use helpers
* merge main
* update AllowSelfServie and re-run code gen
* fix package name
* add allowselfserve flag to payload
* remove config
* update list api to return the full registry including states
* change enabled check
* fix compile error
* add feature toggle and split path in frontend
* changes
* with status
* add more status/state
* add back config thing
* add back config thing
* merge main
* merge main
* now on the /current api endpoint
* now on the /current api endpoint
* drop frontend changes
* change group name to featuretoggle (singular)
* use the same settings
* now with patch
* more common refs
* more common refs
* WIP actually do the webhook
* fix comment
* fewer imports
* registe standalone
* one less file
* fix singular name
---------
Co-authored-by: Michael Mandrus <michael.mandrus@grafana.com>
* ngalert openapi: Use same `basePath` as rest of Grafana
Currently, there are two issues that prevent easily merging `ngalert` and grafana openapi specs:
- The basePath is different. `grafana` has `/api` and `ngalert` has `/api/v1`. I changed `ngalert` to use `/api`
- The `ngalert` endpoints have their basePath in the each operation path. The basePath should actually be omitted
---------
Co-authored-by: Yuriy Tseretyan <yuriy.tseretyan@grafana.com>
* first touches
* Merge missing SSO settings to support Advanced Auth pages
* fix
* Update secrets correctly
* Add test for upsert with redactedsecret
* Verify decryption in the List tests
* AuthnSync: Rename files and structures
* AuthnSync: register rbac cloud role sync if feature toggle is enabled
* RBAC: Add new sync function to service interface
* RBAC: add common prefix and role names for cloud fixed roles
* AuthnSync+RBAC: implement rbac cloud role sync
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Change ruler API to expect the folder UID as namespace
* Update example requests
* Fix tests
* Update swagger
* Modify FIle field in /api/prometheus/grafana/api/v1/rules
* Fix ruler export
* Modify folder in responses to be formatted as <parent UID>/<title>
* Add alerting test with nested folders
* Apply suggestion from code review
* Alerting: use folder UID instead of title in rule API (#77166)
Co-authored-by: Sonia Aguilar <soniaaguilarpeiron@gmail.com>
* Drop a few more latent uses of namespace_id
* move getNamespaceKey to models package
* switch GetAlertRulesForScheduling to use folder table
* update GetAlertRulesForScheduling to return folder titles in format `parent_uid/title`.
* fi tests
* add tests for GetAlertRulesForScheduling when parent uid
* fix integration tests after merge
* fix test after merge
* change format of the namespace to JSON array
this is needed for forward compatibility, when we migrate to full paths
* update EF code to decode nested folder
---------
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
Co-authored-by: Virginia Cepeda <virginia.cepeda@grafana.com>
Co-authored-by: Sonia Aguilar <soniaaguilarpeiron@gmail.com>
Co-authored-by: Alex Weaver <weaver.alex.d@gmail.com>
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
* Alerting: Add metric to check for default AM configurations
* Use a gauge for the config hash
* don't go out of bounds when converting uint64 to float64
* expose metric for config hash
* update metrics after applying config
* remove latest.json and replace with api call to grafana.com
* remove latest.json
* Revert "remove latest.json"
This reverts commit bcff43d898.
* Revert "remove latest.json and replace with api call to grafana.com"
This reverts commit 02b867d84e.
* add deprecation message to latest.json
* first commit
* add: pagination to anondevices
* fmt
* swagger and tests
* swagger
* testing out test
* fixing tests
* made it possible to query for from and to time
* refactor: change to query for ip adress instead
* fix: tests
* separate nestedFolderPickerOverride toggle to force enable it without nestedFolders
* let's call it newFolderPicker
* update unit tests and keyboard handling
* reduce spacing when no folder open chevron
---------
Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com>
* Split subquery when cleaning annotations
* update comment
* Raise batch size, now that we pay attention to it
* Iterate in batches
* Separate cancellable batch implementation to allow for multi-statement callbacks, add overload for single-statement use
* Use split-out utility in outer batching loop so it respects context cancellation
* guard against empty queries
* Use SQL parameters
* Use same approach for tags
* drop unused function
* Work around parameter limit on sqlite for large batches
* Bulk insert test data in DB
* Refactor test to customise test data creation
* Add test for catching SQLITE_MAX_VARIABLE_NUMBER limit
* Turn annotation cleanup test to integration tests
* lint
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* add get mute timing by name to MuteTimingService
* update get mute timing request handler to use the service method
* replace validation, uniqueness and used errors with errutils
* update mute timing methods return errutil responses
* use the term "time interval" in errors bevause mute timings are deprecated in Alertmanager and will be replaced by time intervals in the future.
* update create and update methods to return struct instead of pointer
* Remove FolderID from service tests
* Add models
* Add folderID pack to publicdashboard tests
* Remove folderID from dashboard tests
* Remove folderID from folders
* Remove folderID from ngalert tests
* Remove nolint comment
* Add back some tests after rebase
* implement Reload() func for azuread provider
* add unit test for failure
* use mutex when updating the info field
* implement the Reload() func for the other providers
* use mutex when reading info
* retrieve info using GetOAuthInfo() in common file
* move Reload() to SocialBase
* Alerting: Increase size of kvstore value type for MySQL to LONGTEXT
alertmanager uses the kvstore to persist its notification log and the current
column limit for MySQL (16.7mb) puts the maximum entries at a level that is
potentially achievable for heavy alerting users (~40-80k entries).
In comparison, the current type for PSQL (TEXT) is effectively unlimited and
I believe SQLIte defaults to 2gb which is also plenty of leeway.
* Move scope type vars to testutil package
* Expose parts of state historian for use in annotation backend
* Implement Loki ASH Annotation store
This store will only implement the `Get` method of a RepositoryImpl since alert state history
writes to Loki elsewhere.
* Use interface for Loki HTTP Client
* Add tests for Loki ASH Annotation store
* Add missing test
* Fix lint
* Organize tests
* Add filter tests
* Improve tests
* Move filter logic into outer function
* Fix lint
* Add comment
* Fix tests
* Fix lint
* Rename historian store + refactor
* Cleanup historian store
* Fix tests
* Minor cleanup
* Use new `ShouldRecordAnnotation` filter
* Fix logic and add tests for this check
* Fix typos, remove unused variables, `< 1` -> `== 0`
* More closely mimic RBAC filter from xorm to ensure correct logic
* Move off weaveworks client
* Address PR comments
* Alerting: Create feature flag for alert query optimization
Adds a feature flag alertingQueryOptimization for an already existing
functionality: alert query optimization. This feature flag will now be disabled
by default.
* reload SSO settings for HA setups
* remove check for grafana HA
* add unit tests
* fetch all sso settings with one sql query
* register background service
* Add enablePluginsTracingByDefault feature flag
* Enable tracing for all plugins if enablePluginsTracingByDefault is set
* fix docstrings for IsEnabled and IsEnabledGlobally
* fix tests
* do not use separate feature manager
* add test case
* Revert "fix tests"
This reverts commit 46a2420ed1.
* cleanup
* fix plugin tracing disabled if wrong plugin setting is present
* add test case for enabled on plugin with wrong plugin setting but with enablePluginsTracingByDefault feature flag
* Add RequiresRestart = true to enablePluginsTracingByDefault
* re-generate feature flags
* pr review feedback
* Alerting: Add metrics to the remote Alertmanager struct
* rephrase http_requests_failed description
* make linter happy
* remove unnecessary metrics
* extract timed client to separate package
* use histogram collector from dskit
* remove weaveworks dependency
* capture metrics for all requests to the remote Alertmanager (both clients)
* use the timed client in the MimirAuthRoundTripper
* HTTPRequestsDuration -> HTTPRequestDuration, clean up mimir client factory function
* refactor
* less git diff
* gauge for last readiness check in seconds
* initialize LastReadinesCheck to 0, tweak metric names and descriptions
* add counters for sync attempts/errors
* last config sync and last state sync timestamps (gauges)
* change latency metric name
* metric for remote Alertmanager mode
* code review comments
* move label constants to metrics package
* Alerting: Fix NoData & Error alerts not resolving when rule is reset
On rule reset, when creating the PostableAlerts StateToPostableAlert did not
attach the correct NoData/Error alertname and rulename labels to expire/resolve
the active alerts when the previous cached state was NoData/Error.
* Return data in camelCase from the OAuth fb strategy
* changes
* wip
* Add defaults for oauth fb strategy
* revert other changes
* basic includeDefaults query param implementation
* basic secret removal and etag implementation
* correct imports
* rebase
* move default settings filter to models
* only replace ClientSecret value if set
* first GetForProvider test & use FNV for ETag to avoid Blocklisted import error
* add tests
* add annotation for the openapi spec & generate spec
* remove TODO
* use IsSecret, improve tests, remove DefaultOAuthSettings
* add comment explaining generateFNVETag
* add error handling for generateFNVETag
* run go generate
* Update pkg/services/ssosettings/api/api.go
Co-authored-by: Mihai Doarna <mihai.doarna@grafana.com>
* move isSecret to service, create GetForProviderWithRedactedSecrets func
* add unit test for GetForProviderWithRedactedSecrets & remove duplicated code
* regen openapi/swagger
* revert dependency bumps
---------
Co-authored-by: Mihaly Gyongyosi <mgyongyosi@users.noreply.github.com>
Co-authored-by: Mihai Doarna <mihai.doarna@grafana.com>
This PR has two steps that together create a functional dry-run capability for the migration.
By enabling the feature flag alertingPreviewUpgrade when on legacy alerting it will:
a. Allow all Grafana Alerting background services except for the scheduler to start (multiorg alertmanager, state manager, routes, …).
b. Allow the UI to show Grafana Alerting pages alongside legacy ones (with appropriate in-app warnings that UA is not actually running).
c. Show a new “Alerting Upgrade” page and register associated /api/v1/upgrade endpoints that will allow the user to upgrade their organization live without restart and present a summary of the upgrade in a table.
* extract get and save operations to a alertmanagerConfigStore. this removes duplicated code in service (currently only mute timings) and improves testing
* replace generic errors with errutils one with better messages.
* update provisioning services to use new store
---------
Co-authored-by: Alexander Weaver <weaver.alex.d@gmail.com>
* Docs: Add table data in PDF
* fix lint issues
* Switch to public preview
* Apply suggestions from code review
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
---------
Co-authored-by: Isabel <76437239+imatwawana@users.noreply.github.com>
There were a few errors that prevented these endpoints (which are the most up-to-date ones) from being present in the openapi spec:
- The `enterprise` tag excluded the endpoints from being generated
- `okRespoonse` typo
- Invalid templating on the parameters
- Missing parameter structs
Some refactoring that will simplify next changes for dry-run PRs. This should be no-op as far as the created ngalert resources and database state, though it does change some logs.
The key change here is to modify migrateOrg to return pairs of legacy struct + ngalert struct instead of actually persisting the alerts and alertmanager config. This will allow us to capture error information during dry-run migration.
It also moves most persistence-related operations such as title deduplication and folder creation to the right before we persist. This will simplify eventual partial migrations (individual alerts, dashboards, channels, ...).
Additionally it changes channel code to deal with PostableGrafanaReceiver instead of PostableApiReceiver (integration instead of contact point).
* Separate overlapping legacy and UA alerting routes
api/alert-notifiers, alerting/list, and alerting/notifications existed in both
legacy and UA.
Rename legacy route paths and nav ids to be independent of UA ones.
Backend:
* Update the Grafana Alerting engine to provide feedback to HysteresisCommand. The feedback information is stored in state.Manager as a fingerprint of each state. The fingerprint is persisted to the database. Only fingerprints that belong to Pending and Alerting states are considered as "loaded" and provided back to the command.
- add ResultFingerprint to state.State. It's different from other fingerprints we store in the state because it is calculated from the result labels.
- add rule_fingerprint column to alert_instance
- update alerting evaluator to accept AlertingResultsReader via context, and update scheduler to provide it.
- add AlertingResultsFromRuleState that implements the new interface in eval package
- update getExprRequest to patch the hysteresis command.
* Only one "Recovery Threshold" query is allowed to be used in the alert rule and it must be the Condition.
Frontend:
* Add hysteresis option to Threshold in UI. It's called "Recovery Threshold"
* Add test for getUnloadEvaluatorTypeFromCondition
* Hide hysteresis in panel expressions
* Refactor isInvalid and add test for it
* Remove unnecesary React.memo
* Add tests for updateEvaluatorConditions
---------
Co-authored-by: Sonia Aguilar <soniaaguilarpeiron@gmail.com>
* chore: Bump google.golang.org/grpc from 1.59.0 to 1.60.1
* Bump google.golang.org/protobuf to v1.32.0
* Fix make protobuf failing with latest protoc and protoc-gen-go
* Re-generate protobuf files
* Re-generate protobuf files
* Bump grafana-plugin-sdk-go
* go mod tidy
* merge with system settings before storing them in the db
* add base for validating sso settings
* add unit tests for sso settings validation
* call Reload() from sso service upsert()
* remove actual validation because it was moved in a separate pr
* use constant to fix lint error
* check if provider is configurable in service Upsert() method
* add unit tests for update provider settings api method
* fix lint error
* Canvas: Add Zoom
* Scale selecto components based on zoom state
* Fix pan by reverting to 3.1.0 for zoom-pan
* Update to latest library that fixes pan regression
* Add mini map to canvas pan zoom
* Fix selecto and anchors on hover
* Update naming to be more clear
* Switch back to contentComponent
* Apply transformScale to drag and resize
* Update connection source and target scaling
* Add option to display mini map
* Update yarn lock
* Revert "Update yarn lock"
This reverts commit 3d1dd65d57.
* Set yarn lock to main
* Revert "Set yarn lock to main"
This reverts commit 64bc50557e.
* Update to Yarn 4
* Add react-zoom-pan-pinch
* Update react-zoom-pan checksum
* Revert changes to json files
* Remove last line of api merged
* Remove last lines of all impacted jsons
* Update home json
* Update coordinate calc function to include scale
* Fix types in coordinate calc function
* Fix util calculation for transform
* Fix arrow anchor shift behavior
* Fix scale offset when adding elements during zoom
* Fix drag of selected group during zoom
* Add feature flag for canvas pan zoom
* Revert "Add feature flag for canvas pan zoom"
This reverts commit b026e31d8d.
* Regenerate feature flag after merge
* Apply feature flag to enable pan zoom wrappers
* Add mini map toggle behind feature flag
* Simplify minimap behavior
* Update feature flag registry
* Set minimap to false by default
* fix gen-cue
* Set toggles gen to main
Add blank line to toggle gen csv
* Add canvas pan zoom to csv
* Remove old comment
* Change ref parameter to be more descriptive
* Rename visibleFun to be more descriptive
* Consolidate transformScale transformRef in util
* Remove non-null assertion on connection parentRect
* Consolidate parentRect null coalescing into object
* Remove minimap and change toggle
* Add controls inline help for pan and zoom
* Clean up mouse events
* Pull scale out of ref and isolate transform
* Remove transform ref from scene div
* Fix context menu visible behavior
* Fix connections and update util functions
* Move transform component instance to util
* fix backend test
* minor updates
* Clean up connections / fix minor bug where offset of arrow wasn't being calculated correctly
* missed connection code cleanup
* cleanup scene code a bit more
* actually fix backend test
* move eslint disable line closer to actual issue
---------
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
* Unified Storage: Add resource from/to entity tests
* fixup
* Remove GRN
* Update tests
* truncate timestamps to account for RFC3339, set Group and GroupVersion in k8s object
* Update tests
---------
Co-authored-by: Dan Cech <dcech@grafana.com>
* (WIP) Alerting: Use the forked Alertmanager for remote secondary mode
* fall back to using internal AM in case of error
* remove TODOs, clean up .ini file, add orgId as part of remote AM config struct
* log warnings and errors, fall back to remoteSecondary, fall back to internal AM only
* extract logic to decide remote Alertmanager mode to a separate function, switch on mode
* tests
* make linter happy
* remove func to decide remote Alertmanager mode
* refactor factory function and options
* add default case to switch statement
* remove ineffectual assignment
