* Export Notification Policy correctly (#78020)
The JSON version of an exported Notification Policy now
inline correctly the policy in the same way the Yaml version
does.
Co-authored-by: Yuri Tseretyan <yuriy.tseretyan@grafana.com>
* ngalert `make`: Support GNU install on Darwin
Currently, the Makefile assumes that Darwin is using the Mac version of `sed`
I have the GNU version, so it failed. With this PR, it checks which version is installed
I also called `make` and there are some changes that came out of it
* swagger-gen
* Return data in camelCase from the OAuth fb strategy
* changes
* wip
* Add defaults for oauth fb strategy
* revert other changes
* Add tests
* Add Defaults to cfg and use it in OAuthStrategy
* Return *OAuthInfo from OAuthStrategy
* lint
* Remove unnecessary Defaults
* Introduce const for fields, fix import order
* Align failing tests
* clean up
* Changes requested by @gamab
* Update pkg/services/ssosettings/strategies/oauth_strategy_test.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Load data on startup
* Rename + simplify
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Alerting: Only warm alert state cache if execute_alerts=true.
If the Grafana instance is not executing alerts, then Warm()-ing the state
manager is wasteful and could lead to misleading rule status queries, as the
status returned will be always based on the state loaded from the database at
startup, and not the most recent evaluation state.
* Move Warm() down to shared conditional.
* Alerting: Add clean_upgrade config and deprecate force_migration
Upgrading to UA and rolling back will no longer delete any data by default.
Instead, each set of tables will remain unchanged when switching between
legacy and UA. As such, the force_migration config has been deprecated
and no extra configuration is required to roll back to legacy anymore.
If clean_upgrade is set to true when upgrading from legacy alerting to Unified
Alerting, grafana will first delete all existing Unified Alerting resources,
thus re-upgrading all organizations from scratch. If false or unset,
organizations that have previously upgraded will not lose their existing Unified
Alerting data when switching between legacy and Unified Alerting.
Similar to force_migration, it should be kept false when not needed as it may
cause unintended data-loss if left enabled.
---------
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* Alerting: Keep track of individual org migration status
Save migration status per migrated org.
Change the meaning (and key/value) of the org_id=0 entry
to store the current (previous) config value used by alerting.
This is so we can know when to upgrade/downgrade by
comparing with the new config value in
UnifiedAlerting.IsEnabled.
* Alerting: Add a sync interval for ApplyConfig in remote secondary mode
* remove out of scope code
* remove parentheses after CleanUp for consistency in test comments
* Add comment to ApplyConfig
* Add anonymous stats and user table
- anonymous users users page
- add feature toggle `anonymousAccess`
- remove check for enterprise for `Device-Id` header in request
- add anonusers/device count to stats
* promise all, review comments
* make use of promise all settled
* refactoring: devices instead of users
* review comments, moved countdevices to httpserver
* fakeAnonService for tests and generate openapi spec
* do not commit openapi3 and api-merged
* add openapi
* Apply suggestions from code review
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
* formatin
* precise anon devices to avoid confusion
---------
Co-authored-by: Alex Khomenko <Clarity-89@users.noreply.github.com>
Co-authored-by: jguer <me@jguer.space>
* refactor SSOSettings to use types
* test struct
* refactor SSOSettings struct to use types
* fix database tests
* fix populateSSOSettings() to accept an SSOSettings param
* fix all tests from the database layer
* handle errors for converting to/from SSOSettings
* add json tag on OAuthInfo fields
* use continue instead of if/else
* add the source field to SSOSettingsDTO conversion
* remove omitempty from json tags in OAuthInfo struct
* Alerting: In migration improve deduplication of title and group
This change improves alert titles generated in the legacy migration
that occur when we need to deduplicate titles. Now when duplicate
titles are detected we will first attempt to append a sequential index,
falling back to a random uid if none are unique within 10 attempts.
This should cause shorter and more easily readable deduplicated
titles in most cases.
In addition, groups are no longer deduplicated. Instead we set them
to a combination of truncated dashboard name and humanized alert
frequency. This way, alerts from the same dashboard share a group
if they have the same evaluation interval. In the event that truncation
causes overlap, it won't be a big issue as all alerts will still be in a
group with the correct evaluation interval.
* Alerting: Introduce a Mimir client as part of the Remote Alertmanager
Mimir client that understands the new APIs developed for mimir. Very much a WIP still.
* more wip
* appease the linter
* more linting
* add more code
* get state from kvstore, encode, send
* send state to the remote Alertmanager, extract fullstate logic into its own function
* pass kvstore to remote.NewAlertmanager()
* refactor
* add fake kvstore to tests
* tests
* use FileStore to get state
* always log 'completed state upload'
* refactor compareRemoteConfig
* base64-encode the state in the file store
* export silences and nflog filenames, refactor
* log 'completed state/config upload...' regardless of outcome
* add values to the state store in tests
* address code review comments
* log error from filestore
---------
Co-authored-by: gotjosh <josue.abreu@gmail.com>
* ExtSvcAuth: Assign roles locally
* Fix test
* HandlePluginStateChanged in the OrgID
* Remove Global from command
* Use AssignmentOrgID instead of OrgID
* Remove unecessary test case
* AuthN: Check API Key is not trying to access another organization
* Revert local change
* Add test
* Discussed with Kalle we should set r.OrgID
* Syntax sugar
* Suggestion org-mismatch
* Alerting: Apply query optimization to eval endpoints
Previously, query optimization was applied to alert queries when scheduled but
not when ran through `api/v1/eval` or `/api/v1/rule/test/grafana`. This could
lead to discrepancies between preview and scheduled alert results.
* Alerting: Add GetFullState method to FileStore
* make tests compile, create stateStore in NewAlertmanager
* return errors instead of logging, accept an arbitrary number of strings
* make NewAlertmanager() accept a stateStore
* Alerting: In migration, fallback to '1s' for malformed min interval
During legacy migration, when we encounter an alert datasource query
with a min interval (interval field in the query model) that is not
parseable, instead of failing the migration we fallback to a min interval
of 1s and continue.
The reason for this is a bug in legacy alerting (existing for a few major
versions) which allows arbitrary dashboard variables to be used as the
min interval, even though those variables do not work and will cause
the legacy alert to fail with `interval calculation failed: time: invalid
duration`.
* regression analysis first dragt
* Swap to better regression libraries
* fix name
* Interpolate x points instead of using source x points
* clean up ui and add feature toggle
* fix merge error
* change to loop for finding min max, rename resolution
* Add docs
* add docs and tests
* change name to regression analysis
* update docs
* Fix editor labels
* add regression images
* fix docs
* Remote Alertmanager(refactor): Only parse the URL once
Exactly what it says in the tin.
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* use the existing tests
Signed-off-by: gotjosh <josue.abreu@gmail.com>
---------
Signed-off-by: gotjosh <josue.abreu@gmail.com>
* Alerting: Introduce a Mimir client as part of the Remote Alertmanager
This is our first attempt at making Grafana communicate use Mimir as a backend - it uses a new set of APIs that we've developed on the Mimir side to upload the grafana configuration and alertmanager state so that it can then be ported over.
Codewise, we've introduced a couple of things:
A client to isolate in its own package all the communication that happens with Mimir
A few changes to the remote/alertmanager to include uploading the configuration and state when it starts
A few refactors that align a bit better with the design approach that we're thinking
An integration tests again these newly developed APIs using a custom image
---------
Signed-off-by: gotjosh <josue.abreu@gmail.com>
Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>
* Embed flame graph
* Update test
* Update test
* Use toggle
* Update test
* Add tests
* Use const
* Cleanup
* Update profile tag
* Move flame graph out of tags, remove request and other cleanup + tests
* Update test
* Set flame graph by profile id and simplify logic
* Cleanup and redrawListView
* Create/use feature toggle
* remove all the things
* fix OldFolderPicker tests
* i18n
* remove more unused code
* remove mutation of error object since it's now frozen in the redux state
* fix error handling
* remove use of SignedInUserCopies
* add extra safety to not cross assign permissions
unwind circular dependency
dashboardacl->dashboardaccess
fix missing import
* correctly set teams for permissions
* fix missing inits
* nit: check err
* exit early for api keys
* move the name finding logic for new data sources from frontend to backend
* cleanup and fix test
* linting
* change the way the number after the ds type is incremented - keep incrementing it without adding more hyphens
* enterprise spec updates (unrelated to the PR)
* Lock when creating external service
* Add local lock back
* Improve function signature
* Define lockName separately to make it more explicit
* Update pkg/infra/serverlock/serverlock.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Update pkg/infra/serverlock/serverlock.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Move test to the db so we test the queries and not just testing the mock
* Remove unused function and dependencies
* Remove unused functions from the database
* Add some integration tests
* prepare backend for structured metadata
* add `lokiStructuredMetadata` feature toggle
* use `lokiStructuredMetadata` feature flag
* add field type check to `labelTypesField`
* remove fixme
* fix feature toggle
* add field in dataplane mode
* use `data.Labels` where possible
* adjust framing tests
* improve verbiage
* improve naming
* update tests to "attributes"
* change where folder checks are done for dash creation/updates
* add test for folder not being found
* test fixes
* more test fixes
* add nlint directive to where folder IDs are used
* fix bad merge
* fix test
* Plugin: Remove external service on plugin removal
* Early exit no service account
* Add log
* WIP
* Cable OAuth2Server client removal
* Move function lower
* Add function to test removal
* Add test to RemoveExternalService
* Test RemoveExtSvcAccount
* remove apostrophy in comment
* Add cfg to plugin installer to check features
* Add feature flag check in the service registration service
* Comments
* Move metrics Inc
* Initialize map
* Reorder
* Initialize mutex as well
* Add HasExternalService as suggested
* WIP: CleanUpOrphanedExternalServices
* Commit suggestion
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Nit on test.
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* oauthserver return names
* Name is not Slug
* Use plugin ID not slug
* Add background job
* remove negation on feature check
* Add test to the CleanUp function
* Test GetExternalServiceNames
* rename test
* Add test for ExtSvcAccountsService_GetExternalServiceNames
* Add a todo
* Add todo
* Option based on mix
* Rewrite a bit the comment
* Opinionated choice use slugs instead of names everywhere
* Nit.
* Comments and re-ordering
* Comment
* Add log
* Add context
---------
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* LogRow: detect text selection
* LogRow: refactor menu as component
* LogRow: add actions to menu
* LogRow: hack menu position
* Remove unsused imports
* LogRowMessage: remove popover code
* PopoverMenu: refactor
* LogRows: implement PopoverMenu at log rows level
* PopoverMenu: implement copy
* PopoverMenu: receive row model
* PopoverMenu: fix onClick capture issue
* Explore: add new filter methods and props for line filters
* PopoverMenu: use new filter props
* Explore: separate toggleable and non toggleable filters
* PopoverMenu: improve copy
* ModifyQuery: extend line filter with value argument
* PopoverMenu: close with escape
* Remove unused import
* Prettier
* PopoverMenu: remove label filter options
* LogRow: rename text selection handling prop
* Update test
* Remove unused import
* Popover menu: add unit test
* LogRows: update unit test
* Log row: hide the log row menu if the user is selecting text
* Log row: dont hide row menu if popover is not in scope
* Log rows: rename state variable
* Popover menu: allow menu to scroll
* Log rows: fix classname prop
* Log rows: close popover if mouse event comes from outside the log rows
* Declare new class using object style
* Fix style declaration
* Logs Popover Menu: add string filtering functions (#76757)
* Loki modifyQuery: add line does not contain query modification
* Elastic modifyQuery: implement line filters
* Modify query: change action name to not be loki specific
* Prettier
* Prettier
* Elastic: escape filter values
* Popover menu: create feature flag
* Log Rows: integrate logsRowsPopoverMenu flag
* Rename feature flag
* Popover menu: track interactions
* Prettier
* logRowsPopoverMenu: update stage
* Popover menu: add ds type to tracking data
* Log rows: move feature flag check
* Improve handle deselection
* Swagger: Fix listTokensResponse
It should return a list of Tokens, not a single one
Also regenerated the API spec from the latest changes + this branch
* Remove pointer
* [WIP] Lift RBAC from xorm store
* Cleanup RBAC, fix tests
* Use the scope type map as a map
* Remove dependency on dashboard service
* Make dashboards a map for constant time lookups (useful later)
---
* Lift RBAC tests into a new file to test at service level
* Add necessary access resource structs to xorm store tests
* Move authorization into separate service
* Pass features to searchstore.Builder
* Sort imports
* Code cleanup
* Remove useless scope type check
* Lift permission check into `Authorize()`
* Use clearer language when checking scope types
* Include dashboard permissions in test to ensure they're ignored
* Switch to errutil
* Cleanup sql.Cfg refs
* Add `isManaged` property to frontend model
* Remove enabled and token buttons for managed SA
* Replace trash icon for lock icon for managed SA
* Block the role picker for managed SA
* Filter SA list usiong the managed filter
* Rename external for managed
* Add only managed filter
* Toggle the enable buttons for managed sa
* Disable add token and delete token buttons
* Remove the edit name button
* Disable the Role picker for managed sa
* Hide the permissions section
* Add managed by row
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Alerting: Add an empty Forked Alertmanager
* Alerting: Add methods for silences to the forked Alertmanager
* check for errors in tests
* make linter happy
* Alerting: Add methods for alerts to the forked Alertmanager
* Alerting: Add methods for receivers to the forked Alertmanager
* Alerting: Add TestTemplate method to the forked Alertmanager
* make linter happy
* separate into both forked AMs
* fix tests
* Alerting: Add lifecycle methods to the forked Alertmanager
* Plugin: Remove external service on plugin removal
* Add feature flag check in the service registration service
* Initialize map
* Add HasExternalService as suggested
* Commit suggestion
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Nit on test.
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
---------
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
Check embedded errors in query data response for plugin metrics/logs status label.
Plugin Request Completed log messages are now logged with info level if status=ok,
otherwise error level.
Fixes#76769
* Pass OTEL sampling config to plugins
* fix capital letters
* Do not pass sampler env vars if sampling is not configured
* Add tests
* PR review feedback
* Simplify tracing env vars logic
* Update test to reflect pkg/infra/tracing behaviour
* Plugins:Allow disabling angular deprecation UI for specific plugins
* add backend test
* changed test names
* lint
* Removed angular properties from DataSourceDTO
* Update tests
* Move angularDetected and hideAngularDeprecation in angularMeta property
* Fix angular property name in AppPluginConfig
* Fix reference to angularMeta.detected
* Fix hide_angular_deprecation not working for core plugins
* lint
* Add `isExternal` property to frontend model
* Remove enabled and token buttons for external SA
* Replace trash icon for lock icon for external SA
* Block the role picker for external SA
* Filter SA list using the external filter
* Add only external filter at backend
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Alerting: Add an empty Forked Alertmanager
* Alerting: Add methods for silences to the forked Alertmanager
* check for errors in tests
* make linter happy
* Alerting: Add methods for alerts to the forked Alertmanager
* Alerting: Add methods for receivers to the forked Alertmanager
* Alerting: Add TestTemplate method to the forked Alertmanager
* make linter happy
* separate into both forked AMs
* fix tests
* Alerting: Add an empty Forked Alertmanager
* Alerting: Add methods for silences to the forked Alertmanager
* check for errors in tests
* make linter happy
* Alerting: Add methods for alerts to the forked Alertmanager
* Alerting: Add methods for receivers to the forked Alertmanager
* make linter happy
* separate into both forked AMs
* fix tests
* rename testErr -> expErr
* Folders: Show folders user has access to at the root level
* Refactor
* Refactor
* Hide parent folders user has no access to
* Skip expensive computation if possible
* Fix tests
* Fix potential nil access
* Fix duplicated folders
* Fix linter error
* Fix querying folders if no managed permissions set
* Update benchmark
* Add special shared with me folder and fetch available non-root folders on demand
* Fix parents query
* Improve db query for folders
* Reset benchmark changes
* Fix permissions for shared with me folder
* Simplify dedup
* Add option to include shared folder permission to user's permissions
* Fix nil UID
* Remove duplicated folders from shared list
* Only left the base part
* Apply suggestions from code review
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Add tests
* Fix linter errors
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Alerting: Add an empty Forked Alertmanager
* Alerting: Add methods for silences to the forked Alertmanager
* check for errors in tests
* make linter happy
* Alerting: Add methods for alerts to the forked Alertmanager
* make linter happy
* separate into both forked AMs
* rename testErr -> expErr
* Alerting: Add an empty Forked Alertmanager
* Alerting: Add methods for silences to the forked Alertmanager
* check for errors in tests
* make linter happy
* make linter happy
* Alerting: Add methods for silences to the forked Alertmanager
* inital changes, db migration
* changes
* Implement basic GetAll, Delete
* Add first batch of tests
* Add more tests
* Add service tests for GetForProvider, List
* Update http_server.go + wire.go
* Lint + update fixed role
* Update CODEOWNERS
* Change API init
* Change roles, rename
* Review with @kalleep
* Revert a mistakenly changed part
* Updates based on @dmihai 's feedback
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Currently, in the schema, it's a global definition called `ItemDTO`. It's hard to figure out what that is
Renaming it to `Annotation` should be more helpful
* cumulative sum
* refactor and create new mode
* refactor - use reduceOptions for new mode also
* revert naming
* Add window function, rename statistical to cumulative (#77066)
* Add window function, rename statistical to cumulative
* Fix merge errors
* fix more merge errors
* refactor + add window funcs
Co-authored-by: Oscar Kilhed <oscar.kilhed@grafana.com>
* add ff + tests + centered moving avg
Co-authored-by: Oscar Kilhed <oscar.kilhed@grafana.com>
* make sum and mean cumulative more efficient (#77173)
* make sum and mean cumulative more efficient
* remove cumulative variance, add window stddev
* refactor window to not use reducer for mean. wip variance stdDev
* fix tests after optimization
---------
Co-authored-by: Victor Marin <victor.marin@grafana.com>
* optimize window func (#77266)
* make sum and mean cumulative more efficient
* remove cumulative variance, add window stddev
* refactor window to not use reducer for mean. wip variance stdDev
* fix tests after optimization
* fix test lint
* optimize window
* tests are passing
* fix nulls
* fix all nulls
---------
Co-authored-by: Victor Marin <victor.marin@grafana.com>
* change window size to be percentage
* fix tests to use percentage
* fixed/percentage window size (#77369)
* Add docs for cumulative and window functions of the add field from calculation transform. (#77352)
add docs
* splling
* change WindowType -> WindowAlignment
* update betterer
* refactor getWindowCreator
* add docs to content.ts
* add feature toggle message
---------
Co-authored-by: Oscar Kilhed <oscar.kilhed@grafana.com>
* add/update sqlstore-related helper functions
* add documentation & tests for InsertQuery and UpdateQuery, make generated SQL deterministic by sorting columns
* remove old log line
* Chore: Replace grafana-authnz-team with identity-access-team as code owner
* Chore: Replace grafana-authnz-team with identity-access-team as code owner
* Fix the failing test
* Add interface verification compliance
* rework service account api to a provider
* wire the service accounts api
* rewire the implementation of sa srv for the proxy
---------
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
* Storage: Add maxFiles to list functions
* Add maxDataPoints argument to listFiles function
* Add maxFiles to ResourceDimensionEditor
* Update pkg/services/store/http.go
* rename First to Limit
---------
Co-authored-by: jennyfana <110450222+jennyfana@users.noreply.github.com>
Co-authored-by: nmarrs <nathanielmarrs@gmail.com>
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
* Dashboards: Add integration tests for creating a dashboard
* Fix creating dashboard under folder using deprecated API
* Update swagger response
* Fix comments
* Update Tempo devenv to include profiles
* Update devenv to scrape profiles from local services
* Cleanup devenv
* Fix issue with flame graph
* Add width prop to ProfileTypeCascader
* Add trace to profiles settings
* Add new spanSelector API
* Add spanSelector to query editor
* Update span link query
* Conditionally show span link
* Combine profile and spanProfile query types and run specific query type in backend based on spanSelector presence
* Update placeholder
* Create feature toggle
* Remove spanProfile query type
* Cleanup
* Use feeature toggle
* Update feature toggle
* Update devenv
* Update devenv
* Tests
* Tests
* Profiles for this span
* Styling
* Types
* Update type check
* Tidier funcs
* Add config links from dataframe
* Remove time shift
* Update tests
* Update range in test
* Simplify span link logic
* Update default keys
* Update pyro link
* Use const
When running in dev mode, error messages would contain an additional "error" property alongside "message". Since this causes confusion, that has been removed and now error messages are the same both modes (using "message").
* Disable plugin service account
* Revert extsvc injection
* handle plugin state changes
* Use isProxyEnabled
* Remove plugininteg changes
* Change update function to also work for mysql 😩
* Plugin: enable service account based on plugin settings on
initialization
* Remove misleading comment
* Fix tests
* test message
* Clean up tests
* Simplify tests
* Re-order imports
* Remove unecessary comment
* Enable datasource plugins by default
Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>
---------
Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>
* Disable plugin service account
* Revert extsvc injection
* handle plugin state changes
* Use isProxyEnabled
* Remove plugininteg changes
* Change update function to also work for mysql 😩
* Change test to also check no collateral update
* Update pkg/services/serviceaccounts/database/store_test.go
* Update pkg/services/serviceaccounts/database/store_test.go
* add validation of team header values w. regex
* apply valid headers
* refactor testcases to account for badly formatted json
* refactoring to move validation code close to the validation itself
* removed tes
* Update pkg/api/datasources_test.go
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Update pkg/api/datasources.go
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* review comments
* review during pairing
---------
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* set default basic role permissions for dashboards even if dash creator permissions can't be set
* temporarily increase the test threshold until we can tweak the page
* Use singleflight to prevent logging error if the token has already been refreshed
* Change order of error checks
* align tests, change error name
* Change sf key
* Update based on the review
* refactor
* add FlagExternalServiceAccounts to proxy service
* add FlagExternalServiceAccounts value to tests
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* expand serviceaccount service interface
* implemet FakeServiceAccountService
* Replace SA service interface from api
* merge sa proxy tests with new fake service
* implement DeleteServiceAccountToken
* add test for DeleteServiceAccountToken
* Alerting: Rename remote.ExternalAlertmanager to remote.Alertmanager
* Alerting: Send alerts to the remote Alertmanager
* add ticker to readiness check, add tests
* use options when creating a new sender.ExternaAlertmanager
* unexport defaultMaxQueueCapacity
* delete unused defaultConfig field
* add debug log line when sending alerts to the remote alertmanager
* move and refactor readiness check
* update tests to not include defaultConfig
* AuthN: Add metrics to external service accounts management
* Add a new metric to count stored external service accounts
* Update variable names
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* Add test to SearchOrgServiceAccounts
* Add feature flags checks before registering and using the metrics
---------
Co-authored-by: linoman <2051016+linoman@users.noreply.github.com>
* correctly check permissions to list dashboards on the root
* correctly display the access inherited from general folder for dashboards
* Update pkg/services/sqlstore/permissions/dashboard.go
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Update dashboard_filter_no_subquery.go
---------
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
expose apiserver metrics
Add a route to the apiserver metrics on a new endpoint, `/apiserver-metrics`. This requires a signed-in user but otherwise ignores the MetricsEndpoind-relating configuration. that will come in a following PR
* Add proxy service template
* Replace SA srv with proxy for external SA srv
* Move service account prefix to a constant
* Prevent deletion from external service account
* Make SA validation a resusable function
* Add protection for creating service accounts
* Add protection when updating service accounts
* Add IsExternal field for service account
* Protect ext service account token generation
* Add verbose errors for form name or sa name
* add tests
* Add logs
* Adjusts tests
---------
Co-authored-by: Misi <mgyongyosi@users.noreply.github.com>
Co-authored-by: Gabriel MABILLE <gamab@users.noreply.github.com>
* Alerting: Move `ExternalAlertmanager` to its own package
We'll avoid import cycles when using components from other packages. In addition to that, I've created an `Options` approach for the multiorg alertmanger to allow us to override how per tenant alertmanagers are created.
* switch things around
* address review comments
* fix references and warnings
* initial commit for PromQAIL
* add feature toggle and start button
* add drawer
* set up drawer and state
* fix styles and start the conditional text display
* add data info list going to ai
* add logos and style
* metric display table style, neeed to make responsive
* make feature toggle frontend only
* add logic for want prompt for list or not, add helpers, addquerysuggestion type
* make query suggestion component
* add buttons to add or refine prompt
* refactor logic to add interactions to have multiple AI or historical interactions
* refactor and enable multiple questions, all flow
* add colorful AI icon to drawer open button
* fix linting
* refactor for hooking up promQail app and only giving one suggestion
* design fixes
* fix next prompt button styling
* historical suggestions give us 5, fixed that and some design things
* hook up the api, provide defense filler if it's down, refactor lots
* use query, fix linting
* add metadata to explain for ai suggestions
* styling fixes
* give metadata for historical suggestions by parsing query on the fly
* no prompt field to query-suggestion endpoint if prompt is empty
* fix linting
* use suggest rte for historical list, fix long code style
* use suggest rte for historical list, fix long code style
* fix historical bug
* added prompt file
* updated llm logic in explainer helper
* bump @grafana/experimental from 1.7.0 to 1.7.2
* use llmservice and vectorservice
* cleanup prompts + streaming explainer
* promqail feature toggle: fix re-order
* PromQL non-llm failback recommendation logic (#75469)
* added template recommendation logic directly in helpers
* also added selected labels to recommendation
* PromQail: query gen: fix prompt formatting and fetch metric labels to be used (#75450)
* PromQail: query gen: fix prompt formatting and fetch metric labels to be used
* Code fixes as suggested
* Use newly decided collection name for promql templates
* Prometheus: Promqail tests and bug fixes (#75852)
* add tests for drawer
* refine one prompt at a time, fix css
* scroll into view on interaction change
* fix styles for light
* disable prompt input after getting sugestions for that interaction
* make buttons disappear after selecting refine prompt or show historical queries to prevent user from clicking many times
* fix border radius
* fix new eslint rule about css requiring objects and not template literals
* add scrollIntoView for test
* grafana_prometheus_promqail_explanation_feedback - add feedback rudderstack interaction for explanation
* add form link to feedback for query suggestions
* fix bugs
* for prettier
* PromQL Builder Explainer: Added promql documentation and updated prompt (#75593)
* added promql documentation and updated prompt
* refactor prompt generation into isolated function
* updated prompt to answer with a question
* removed commented code
* updated metadata logic
* updated documentation body logic
* Prometheus: PromQAIL UI fixes (#76654)
* align buttons at 16px
* only autoscroll when an interaction has been added or the suggestions have been updated
* add 12px below explain for suggested queries
* add . after suggestion number
* fix linting error
* Prometheus: PromQAIL feedback improvements (#76711)
* align buttons at 16px
* only autoscroll when an interaction has been added or the suggestions have been updated
* add 12px below explain for suggested queries
* add . after suggestion number
* add text indication for explanation feedback
* add form for suggestion feedback, add form for not helpful explanation feedback
* fix linting error
* make radio button feedback required
* required text, padding additions, thank you for your feedback
* PromQL Builder Suggestion: Added type level templates and removed explainer steps for fallback suggestion logic (#75764)
* adding more detailed templates to promql fallback suggest
* remove debug logs
* added missing explain logic
* Fix brendan's type issue
---------
Co-authored-by: Brendan O'Handley <brendan.ohandley@grafana.com>
Co-authored-by: bohandley <brendan.ohandley@gmail.com>
* make yarn.lock equal to current in main
* fix feature toggles
* fix prettier issues
---------
Co-authored-by: Edward Qian <edward.qian@grafana.com>
Co-authored-by: Yasir Ekinci <yas.ekinci@grafana.com>
Co-authored-by: Edward Qian <edward.c.qian@gmail.com>
Co-authored-by: Gerry Boland <gerboland@users.noreply.github.com>
* Alerting: Move migration from background service run to ngalert init
sqlite database write contention between the migration's single transaction and
dashboard provisioning's frequent commits was causing the migration to
fail with SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT on all retries.
This is not a new issue for sqlite+grafana, but the discrepancy between the
length of the transactions was causing it to be very consistent. In addition,
since a failed migration has implications on the assumed correctness of the
alertmanager and alert rule definition state, we cause a server shutdown on
error. This can make e2e tests as well as some high-load provisioned
sqlite installations flaky on startup.
The correct fix for this is better transaction management across various
services and is out of scope for this change as we're primarily interested in
mitigating the current bout of server failures in e2e tests when using sqlite.
* introduce data source admin role and fix frontend check
* introduce fixed roles for data source creator and team reader
* add documentation
* undo an unintended change
* Alerting: post alerts to the remote Alertmanager and fetch them
* fix broken tests
* Alerting: Add Mimir Backend image to devenv (blocks)
* add alerting as code owner for mimir_backend block
* Alerting: Use Mimir image to run integration tests for the remote Alertmanager
* skip integration test when running all tests
* skipping integration test when no Alertmanager URL is provided
* fix bad host for mimir_backend
* remove basic auth testing until we have an nginx image in our CI
* add integration tests for alerts
* fix tests
* change SendCtx -> Send, add context.Context to Send, fix CI
* add reover() for functions from the Prometheus Alertmanager HTTP client that could panic
* add TODO to implement PutAlerts in a way that mimicks what Prometheus does
* fix log format
* Move rotate logic into its own function
* Move oauth token sync to session client
* Add user to the local cache if refresh tokens are not enabled for the provider so we can skip the check in other
requests
* feat: add cost management to admin and put adaptive metrics and log volume under it
* test: fix applinks test
* chore: fix lint error
* remove "new" from feature toggle description
---------
Co-authored-by: Ashley Harrison <ashley.harrison@grafana.com>
* Update cue to have an AuthProvider entry
* Cable the new auth provider
* Add feature flag check to the accesscontrol service
* Fix test
* Change the structure of externalServiceRegistration (#76673)
* Add teamHeaders for datasource proxy requests
* adds validation for the teamHeaders
* added tests for applying teamHeaders
* remove previous implementation
* validation for header values being set to authproxy
* removed unnecessary checks
* newline
* Add middleware for injecting headers on the data source backend
* renamed feature toggle
* Get user teams from context
* Fix feature toggle name
* added test for validation of the auth headers and fixed evaluation to cover headers
* renaming of teamHeaders to teamHTTPHeaders
* use of header set for non-existing header and add for existing headers
* moves types into datasources
* fixed unchecked errors
* Refactor
* Add tests for data model
* Update pkg/api/datasources.go
Co-authored-by: Victor Cinaglia <victor@grafana.com>
* Update pkg/api/datasources.go
Co-authored-by: Victor Cinaglia <victor@grafana.com>
---------
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Victor Cinaglia <victor@grafana.com>
* Alerting: Use Mimir image to run integration tests for the remote Alertmanager
* skip integration test when running all tests
* skipping integration test when no Alertmanager URL is provided
* fix bad host for mimir_backend
* remove basic auth testing until we have an nginx image in our CI
* update with sdk
* do sql
* fix core plugins
* fix proxy settings
* bump SDK version
* tidy
* enable pdc for test
* add codeowners
* bump dep
* go mod tidy
* bump SDK
* Replace FixedRoleUID function with a common function to generate these prefixes
* Use common function to generate prefixed uid for external service accounts
Co-authored-by: Gabriel MABILLE <gabriel.mabille@grafana.com>
---------
Co-authored-by: Gabriel MABILLE <gabriel.mabille@grafana.com>
fetch fresh permissions for global in AuthorizeInOrgMiddleware
Update pkg/services/accesscontrol/authorize_in_org_test.go
do not load viewer permissions in global ID
* update data migration to update rows that have changes
* fix migration for sqlite
* remove id; fix postgres
* Fix for MySQL
* delete old items from folder table
* change integer to boolean
---------
Co-authored-by: Sofia Papagiannaki <1632407+papagian@users.noreply.github.com>
* Add images
* Basic button functionality; TODO placeholders for dispatching contentOutlineToggle and rendering content outline component
* Basic content outline container
* Content outline toggles
* Remove icon files from explore
* Scroll into view v1
* outline that reflect's explore's order of vizs
* Update icon name
* Add scrollId to PanelChrome; scrolling enabled for Table
* Add queries icon
* Improve scroll behavior in split view
* Add wrapper so the sticky navigation doesn't scroll when on the bottom of the window
* Fix the issue with logs gap; center icons
* Memoize register and unregister functions; adjust content height
* Make displayOrderId optional
* Use Node API for finding position of panels in content outline; add tooltip
* Dock content outline in expanded mode; at tooltip to toggle button
* Handle content outline visibility from Explore and not redux; pass outlineItems as a prop
* Fix ContentOutline test
* Add interaction tracking
* Add padding to fix test
* Replace string literals with objects for styles
* Update event reporting payloads
* Custom content outline button; content outline container improvements
* Add aria-expanded to content outline button in ExploreToolbar
* Fix vertical and horizontal scrolling
* Add aria-controls
* Remove unneccessary css since ExploreToolbar is sticky
* Update feature toggles; Fix typos
* Make content outline button more prominent in split mode; add padding to content outline items;
* Diego's UX updates
* WIP: some scroll fixes
* Fix test and type error
* Add id to ContentOutline to differentiate in split mode
* No default exports
---------
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* Fix migration of custom dashboard permissions
Dashboard alert permissions were determined by both its dashboard and
folder scoped permissions, while UA alert rules only have folder
scoped permissions.
This means, when migrating an alert, we'll need to decide if the parent folder
is a correct location for the newly created alert rule so that users, teams,
and org roles have the same access to it as they did in legacy.
To do this, we translate both the folder and dashboard resource
permissions to two sets of SetResourcePermissionCommands. Each of these
encapsulates a mapping of all:
OrgRoles -> Viewer/Editor/Admin
Teams -> Viewer/Editor/Admin
Users -> Viewer/Editor/Admin
When the dashboard permissions (including those inherited from the parent
folder) differ from the parent folder permissions alone, we need to create a
new folder to represent the access-level of the legacy dashboard.
Compromises:
When determining the SetResourcePermissionCommands we only take into account
managed and basic roles. Fixed and custom roles introduce significant complexity
and synchronicity hurdles. Instead, we log a warning they had the potential to
override the newly created folder permissions.
Also, we don't attempt to reconcile datasource permissions that were
not necessary in legacy alerting. Users without access to the necessary
datasources to edit an alert rule will need to obtain said access separate from
the migration.
* Manage service account secrets
* Wip
* WIP
* WIP
* Revert to keep a light interface
* Implement SaveExternalService
* Remove unecessary functions from the interface
* Remove unused field
* Better log
* Leave ext svc credentials out of the extsvcauth package for now
* Remove todo
* Add tests to SaveExternalService
* Test that secret has been removed from store
* Lint
* Nit.
* Rename commands and structs
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* Account for PR feedback
Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>
* Linting
* Add nosec comment G101 - this is not a hardcoded secret
* Lowercase kvStoreType
---------
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: Andres Martinez Gotor <andres.martinez@grafana.com>
* Update origin annotation names
k8s does not support annotation names with multiple slashes in them, so this PR updates the origin annotations to match the format for updated and created annotations.
* fix tests
This PR replaces the vendored models in the migration with their equivalent ngalert models. It also replaces the raw SQL selects and inserts with service calls.
It also fills in some gaps in the testing suite around:
- Migration of alert rules: verifying that the actual data model (queries, conditions) are correct 9a7cfa9
- Secure settings migration: verifying that secure fields remain encrypted for all available notifiers and certain fields migrate from plain text to encrypted secure settings correctly e7d3993
Replacing the checks for custom dashboard ACLs will be replaced in a separate targeted PR as it will be complex enough alone.
* Move errors to error file
* Move check for both empty username and email to user service
* Move check for empty email and username to user service Update
* Wrap inner error
* Set username in test
* extend threshold command with second evaluator called `unloadEvaluator`
* Introduce a new expression command Hysteresis and update Threshold unmarshaller to create the HysteresisCommand if the second eval
* add feature flag `recoveryThreshold`
* update unmarshal threshold command to not re-marshall because it breaks frame definition by shuffling the schema and data fields
* Add fixed role UID
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Use base64 url encoding
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Create a state for dockedMegaMenu and the function to manage it
* Add the dockedMenu icon and handle the status when clicking it
* Add Megamenu to section nav area when it is docked
* get logic working
* fix mobile
* refactor state + persist in localStorage
* adjust icon and don't use position absolute
* restore old rudderstack tracking
* use Flex instead
* adjust feature toggle to be experimental
* extract out localStorage handling into utils
* don't need separate file
* use store.set/get instead
---------
Co-authored-by: eledobleefe <laura.fernandez@grafana.com>
* IDForwarding: change audience to be prefixed by org and remove JTI
* IDForwarding: Construct new signer each time we want to sign a token.
* SigningKeys: Simplify storage layer and move logic to service
* SigningKeys: Add private key to local cache
* Extract code to manage service accounts
* Add test with client credentials grants
* Fix test with the changed interface
* Wire
* Fix HandleTokenRequest
* Add tests to extsvcaccounts
* Rename Retrieve function
* Document the interface
* Unfurl OrgRole in pkg/api to allow using identity.Requester interface
* Unfurl Email in pkg/api to allow using identity.Requester interface
* Update UserID in pkg/api to allow using identity.Requester interface
* fix authed test
* fix datasource tests
* guard login
* fix preferences anon testing
* fix anonymous index rendering
* do not error with user id 0
