Introduces a FromContext method on the log.Logger interface that
allows contextual key/value pairs to be attached, e.g. per request,
so that any logger using this API will automatically get the per request
context attached. The proposal makes the traceID available for
contextual logger , if available, and would allow logs originating from
a certain HTTP request to be correlated with traceID.
In addition, when tracing not enabled, skip adding
traceID=00000000000000000000000000000000
to logs.
* Create struct for Slack's receiver settings
* Remove one layer of indirection when building slack notifier
* Delete un-used struct
* Validate against settings struct instead of simplejson object
* Genericize settings marshalling
* Remove repetition between fields on notifier and fields on settings struct
* Rename unmarshal settings wrapper
* Handle comma separated strings at marshalling time rather than validation time
* Address misc review feedback
* Add function to remove metadata from queries
* Add test for RemoveMetadataFromQueryData function
* Remove only custom data and executed query string
* Add sanity check to SanitizeMetadataFromQueryData
* Hide Correlations Page behind a toggle
* Remove redundant defaults
* Ensure top nav is rendered when correlations are disabled
This is for the case when:
[feature_toggles]
topnav = true
This commit fixes a bug where we did not send resolved alerts to Alertmanager for resolved alert instances. This meant that resolved notifications did not have the annotations from the resolved state, and a result did not also have the resolved screenshot.
* fix: min metric to hourly metric
* Update pkg/services/login/model.go
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* plugin client returns error base
* fix api test
* add plugin client test
* add fallback err
* fix linting
* wip
* replace bad query
* template is an error
* failing test of templated error
* add one test passing
* fix failing test
* move test
* rename ErrBadQuery to ErrQueryValidationFailure
* tidy diff
* Change to one error per specific error kind
* last err + fix test
* fix imports
* more tests
* keep req vars together
Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>
* access control to log user name if it does not have permissions
* update ngalert Evaluator to accept user instead of creating a pseudo one
* update alerting eval (rule\query testing) API to provide the real user to the Evaluator
* update scheduler to create a pseudo user with proper permissions
* RBAC: Add cache for oss permissions
* RBAC: include service account actions
* RBAC: revert changes to fetch service account permissions
* Update comment for setting
* RBAC: Disable permission chache for tests
* Extract standardized UID field length to constant
* Extract default length to constant
* Truncate rule names that are too long
* Add tests for name normalization
* Fix whitespace lint error
* Another linter fix
* Empty commit to kick build
* WIP
* Set public_suffix to a pre Ruby 2.6 version
* we don't need to install python
* Stretch->Buster
* Bump versions in lib.star
* Manually update linter
Sort of messy, but the .mod-file need to contain all dependencies that
use 1.16+ features, otherwise they're assumed to be compiled with
-lang=go1.16 and cannot access generics et al.
Bingo doesn't seem to understand that, but it's possible to manually
update things to get Bingo happy.
* undo reformatting
* Various lint improvements
* More from the linter
* goimports -w ./pkg/
* Disable gocritic
* Add/modify linter exceptions
* lint + flatten nested list
Go 1.19 doesn't support nested lists, and there wasn't an obvious workaround.
https://go.dev/doc/comment#lists
* RBAC: Allow app plugins restriction
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* Moving declaration to HttpServer
Co-Authored-By: marefr <marcus.efraimsson@gmail.com>
* Picking changes from the other branch
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Rename plugins.settings to plugins
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
* Account for PluginAdminExternalManageEnabled
Co-authored-by: Will Browne <will.browne@grafana.com>
* Set metadata on instantiation
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
Co-authored-by: Kalle Persson <kalle.persson@grafana.com>
Co-authored-by: marefr <marcus.efraimsson@gmail.com>
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Will Browne <will.browne@grafana.com>
Co-authored-by: Jguer <joao.guerreiro@grafana.com>
* RBAC: Remove service dependency for Evaluator component
* RBAC: Add service and load permissions in target org if they are not
there
* RBAC: Use service if we need to load permissions for org
* API: remove service injection into evaluator
* API: set new user for each request in tests
* PublicDashboards: Use fake service to provide permissions
* RBAC: Set org id for dashboard provisioning user
* Improve error handling for secrets manager plugin
* Update sever lock log for secret migration to Error
* Move plugin started check to a separate function
* Fix nil pointer exception on HasPluginStarted
* Alerting: Sanitize invalid label/annotation names for external alertmanagers
Grafana's built-in Alertmanager supports both Unicode label keys and values; however, if using an external
Prometheus Alertmanager label keys must be compatible with their data model.
This means label keys must only contain ASCII letters, numbers, as well as underscores and match the regex
`[a-zA-Z_][a-zA-Z0-9_]*`.
Any invalid characters will now be removed or replaced by the Grafana alerting engine before being sent to
the external Alertmanager according to the following rules:
- `Whitespace` will be removed.
- `ASCII characters` will be replaced with `_`.
- `All other characters` will be replaced with their lower-case hex representation.
* Prefix hex replacements with `0x`
* Refactor for clarity
* Apply suggestions from code review
Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
Co-authored-by: brendamuir <100768211+brendamuir@users.noreply.github.com>
* feat: allow jwt role to be set
* chore: update documentation
* fix: cr suggestions
* fix: lint issues
* respect org auto assign and default org ID
* add server admin to devenv
Co-authored-by: jguer <joao.guerreiro@grafana.com>
* Remove single member team restriction
* Add label when permissions list is empty
* Fix unit tests
* Add co-author.
Co-authored-by: Ieva <ieva.vasiljeva@grafana.com>
* Refactor fallback to be isolated to plugin secret store
* Check for error value on replace fallback test helper
* Move ResetPlugin from test_helpers.go to plugin.go
* Add check to GetUnwrappedStoreFromCache
* Add fallback GetAll query to WithFallbackEnabled
* Add mutex lock to WithFallbackEnabled
* Add cache to fallback store
* Fix linter issues
* Fix linter issues
* Fix linter issues
Prior to this change, all alert instance writes and deletes happened
individually, in their own database transaction. This change batches up
writes or deletes for a given rule's evaluation loop into a single
transaction before applying it.
Before:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 398 2991381 ns/op 1133537 B/op 27703 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: FovKXiRVzm} with title: "an alert definition FTvFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: foDFXmRVkm} with title: "an alert definition fovFXmRVkz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: VQvFuigVkm} with title: "an alert definition VwDKXmR4kz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.619s
```
After:
```
goos: darwin
goarch: arm64
pkg: github.com/grafana/grafana/pkg/services/ngalert/store
BenchmarkAlertInstanceOperations-8 1440 816484 ns/op 352297 B/op 6529 allocs/op
--- BENCH: BenchmarkAlertInstanceOperations-8
util.go:127: alert definition: {orgID: 1, UID: 302r_igVzm} with title: "an alert definition q0h9lmR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: 71hrlmR4km} with title: "an alert definition nJ29_mR4zz" interval: 60 created
util.go:127: alert definition: {orgID: 1, UID: Cahr_mR4zm} with title: "an alert definition ja2rlmg4zz" interval: 60 created
PASS
ok github.com/grafana/grafana/pkg/services/ngalert/store 1.383s
```
So we cut time by about 75% and memory allocations by about 60% when
storing and deleting 100 instances.
This change also updates some of our tests so that they run successfully against postgreSQL - we were using random Int64s, but postgres integers, which our tables use, max out at 2^31-1
* Update GetAlertRulesForScheduling to query for folders (if needed)
* Update scheduler's alertRulesRegistry to cache folder titles along with rules
* Update rule eval loop to take folder title from the
* Extract interface RuleStore
* Pre-fetch the rule keys with the version to detect changes, and query the full table only if there are changes.
* Toggle on the mixed mode option
* Ensure switching to mixed gives existing query prev datasource
* WIP - Populate datasource when switching between mixed and not
* WIP - handle change from mixed
* Remove preimport filter, refine filter to work for queries
* WIP debugging datasource transition
* Ensure creating a new query gets target data source if switching with no matches between
* Add mixed datasource to rich history display
* Cleanup console logs, add relevant comments
* Add feature toggle for mixed datasource
* Fix Wrapper tests
* Fix tests!
* Fix test types and add feature tracking
* Remove unnecessary default, remove explore/mixed workarounds for D2E
* Move display text logic to mixed datasource file
* Add in the default query parameters to a generated empty query
* Condense some code
* Apply suggestions from code review
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* Add more logic around mixed datasource being off for explore
* Build out logic to handle different datasource scenarios
* Add tests
* Finalize last test
* Fix mixed URL with mixed ds off, and relevant test
* Fix datasource to explore workflow
* Add datasource change function, call import queries if needed
* add logic for changing single query ds
Co-authored-by: Giordano Ricci <me@giordanoricci.com>
* metrics: Fix broken DBStats metrics
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* Register the sqlstats metrics by default
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
Signed-off-by: Dave Henderson <dave.henderson@grafana.com>
* split out plugin manager
* remove whitespace
* fix tests
* split up tests
* updating naming conventions
* simplify manager
* tidy
* add more fakes
* testing time
* add query verif to int test
* renaming
* add process tests
* tidy up manager tests
* add extra case to int test
* add more coverage to store and process tests
* remove comment
* fix capatilization
* init on provide
* remove addfromsource from API
* refactor apis for consistent outputs
* add dashboardUid validation at API layer
* add check for empty dashboardUid on SavePublicDashboard
* remove public dashboard errors from models package.
Co-authored-by: Ezequiel Victorero <evictorero@gmail.com>
* RBAC: Move metadata to own file
* RBAC: Rename test files
* RBAC: Add wildcard structure and helper function to generate wildcards
from prefix
* RBAC: Refactor filter to use WildcardsFromPrefix
* RBAC: Refactor GetResourceMetadata to use WildcardsFromPrefix
Removes various custom headers logic sprinkled around in the backend.
It should automatically be applied to outgoing HTTP requests via the
CustomHeadersMiddleware.
This also removes decryption of SecureJSONData to populate custom
headers in ngalert which seemed to have caused a ton of CPU usage.
* Search: use SQL search as a fallback when bluge indexing is ongoing
* Search: lint
* Search: feedback fixes - return an empty frame with a special name
* Search: revert readiness check query type
* Search: remove println
* remove sleep, get coffee
* Refactor migrations and tests for secrets kvstore
* Use fake secrets store as a shortcut on tests
* Update wire
* Use global migration logger
* Fix ds proxy tests
* Fix linting issues
* Rename data source test setup function
