* WIP of creating new components to support the Search tab using TraceQL
* Search fields now require an ID. Added duration fields to new Search UI
* Distinguish static from dynamic fields. Added dynamic tags input
* Moved new search behind traceqlSearch feature flag. Added handling of different types of values to accurately wrap them in quotes when generating query.
* Hold search state in TempoQuery to leverage state in URL. Moved types to schema file
* Use a read only monaco editor to render a syntax highlighted generated query. Added tooltip to duration. Added query options section
* Support multiple values using the regex operator and multi input
* Delete dynamic filters
* Automatically select the regex op when multiple values are selected. Revert to previous operator when only one value is selected
* Added tests for SearchField component
* Added tests for the TraceQLSearch component
* Added tests for function that generates the query
* Fix merge conflicts
* Update test
* Replace Search tab when traceqlSearch feature flag is enabled. Limit operators for both name fields to =,!=,=~
* Disable clear button for values
* Changed delete and add buttons to AccessoryButton. Added descriptions to operators
* Remove duplicate test
* Added a prismjs grammar for traceql. Replaced read only query editor with syntax highlighted query. Removed spaces between tag operator and value when generating query.
* Fix support for custom values when isMulti is enabled in Select
* Use toOption function
* remove state and refactor interface to IsDisabled
* update docs and span
* Update pkg/services/apikey/apikey.go
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
---------
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* revert to using folder store from the resolvers
* fixing tests after revert
* api test fixes
---------
Co-authored-by: Kristin Laemmert <mildwonkey@users.noreply.github.com>
* fix: create temp user no longer sets ID to 0 for all users
The xorm tag added to the tempuser ID field caused xorm to create all temp users with ID 0. Removing that tag allows xorm to set the ID based on the database result instead. I also added a test which was failing before this.
Fixes#63995
* move analytics identifiers to backend
* implement hash function
* grab secret from env
* expose and retrieve intercom secret from config
* concat email with appUrl to ensure uniqueness
* revert to just using email
* Revert "revert to just using email"
This reverts commit 8f10f9b1bc.
* add docstring
* AuthN: add utility functions to handle response and redirect after
successful login
* API: Reuse utility functions for logins if authnService flag is enabled
* chore(services): replace dependencies on dashboard store with dashboard service
This continues the backend service/store split by replacing dashboard store dependencies with service dependencies. the folder service remains the single exception for now; otherwise we'd have a dependency cycle between the folder and dashboard services. I have some ideas for that, but I'll take care of all the easy parts first.
While doing this, I identified and removed a number of unused arguments from the following functions:
NewFolderNameScopeResolver
NewFolderIDScopeResolver
NewFolderUIDScopeResolver
NewDashboardIDScopeResolver
NewDashboardUIDScopeResolver
resolveDashboardScope
I have a small enterprise PR to support this commit.
* lingering fmt
* chore: remove export service POC from main
This is a POC and we'll see it, or something like it, again!
* remove frontend changes
---------
Co-authored-by: Ryan McKinley <ryantxu@gmail.com>
This commit fixes a serious bug in Grafana 9.4.1 where on upgrade
a migration would pause all existing alert rules and change the
default value of the column to true.
* add: hide apikeys tab on start
* make use of store method
* added hiding of apikeys tab for new org creation
* missing err check
* removed unused files
* implemennted fake to make tests run
* move check for globalHideApikeys from org to admin
* refactor to remove the fake
* removed unused method calls for interface
* Update pkg/services/serviceaccounts/manager/service.go
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* Update pkg/services/serviceaccounts/manager/service.go
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
* remove the checkglobal method
* removed duplicate global set const
* add count of apikeys for performance
* remove apikeys adding in UI
* added back deleted file
* added comment on component
* changed wording and copy for hiding and migrating service accounts
* refactor: remove migrationstatus in front/backend
This removes the migrationstatus state from the UI in favor of only
looking at the number of API keys to determine what to show to the user.
This simplifies the logic and makes less calls to the backend with each
page load. This was called both on the API keys page and the Service
accounts page.
- removes the state of migrationstatus from the UI
- removes the backend call
- removes the backend endpoint for migrationstatus
* Update pkg/services/apikey/apikeyimpl/xorm_store.go
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* changes the contet to also be primary
* change id of version for footer component
---------
Co-authored-by: Alexander Zobnin <alexanderzobnin@gmail.com>
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Enable serving Swagger UI by default
It used to be served behind the `swaggerUi` feature toggle.
* Remove `swaggerUi` feature toggle
* Add docs and guidelines for updating swagger
* Apply suggestions from code review
Co-authored-by: Josh Hunt <joshhunt@users.noreply.github.com>
* MVP of a new datasource picker
* Add datasource select history, naming DatasourceSelect -> DataSourceDrawer
* refactor cards
* Cleanup and fixing sort order for recents
* add feature flag
* fix feature flag name and use it
* Highlight selected
* Move new ds picker to core
* Restore original datasource picker
* Remove unused property
* update yarn.lock
* Rename folder, update codeowners
* add test for util functions
* Remove es-lint exception
* Change feature toggle description
* remove unnecessary if
Co-authored-by: Ivan Ortega Alba <ivanortegaalba@gmail.com>
* Make test a bit more clear
Co-authored-by: Ivan Ortega Alba <ivanortegaalba@gmail.com>
* Clean up api, filter once and before maps, minor code cleanup
* Fix prettier issue
---------
Co-authored-by: Ivan Ortega Alba <ivanortegaalba@gmail.com>
* Reworked header
* Remove toggle from merge
* Update test
* Update how span is retrived
* Tests
* Update tests
* Move new trace page header into its own component
* Remove tests already covered in TracePageHeader.test.tsx
* Update findHeaderTags
* Tooltip updates
* update ldap library and use go module path
* add TLS min version and accepted min TLS version
* set default min ver to library default
* set default min ver to library default
* add cipher list to toml
* Update pkg/services/ldap/settings.go
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
* Apply suggestions from code review
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* lint
---------
Co-authored-by: Karl Persson <kalle.persson@grafana.com>
Co-authored-by: Christopher Moyer <35463610+chri2547@users.noreply.github.com>
* Setting: Remove global DisableLoginForm and add it to cfg
* Setting: Remove unused BasicAuthEnabled global
* Setting: Remove global OAuthAutoLogin and use from cfg
* Setting: Remove global AnonymousEnabled
* Setting: Remove global values for AuthProxy settings
* fix org user always getting org id = 1 on auto assign false
* make tests explicit
* use correct cfg in service accounts
* fix api tests
* fix database test of ac
* fix InsertOrgUser returning affected rows as orgID
* Loki backend and client depend on a requester
* Instrument all requests to loki using weaveworks TimedClient
* Construct collector in metrics package
* bring in source from database
* bring in transformations from database
* add regex transformations to scopevar
* Consolidate types, add better example, cleanup
* Add var only if match
* Change ScopedVar to not require text, do not leak transformation-made variables between links
* Add mappings and start implementing logfmt
* Add mappings and start implementing logfmt
* Remove mappings, turn off global regex
* Add example yaml and omit transformations if empty
* Fix the yaml
* Add logfmt transformation
* Cleanup transformations and yaml
* add transformation field to FE types and use it, safeStringify logfmt values
* Add tests, only safe stringify if non-string, fix bug with safe stringify where it would return empty string with false value
* Add test for transformation field
* Do not add null transformations object
* Break out transformation logic, add tests to backend code
* Fix lint errors I understand 😅
* Fix the backend lint error
* Remove unnecessary code and mark new Transformations object as internal
* Add support for named capture groups
* Remove type assertion
* Remove variable name from transformation
* Add test for overriding regexes
* Add back variable name field, but change to mapValue
* fix go api test
* Change transformation types to enum, add better provisioning checks for bad type name and format
* Check for expression with regex transformations
* AuthN: Add flag to control org role syncs
* JWT: Only sync org roles if the skip flag for jwt is false
* LDAP: Only sync org role if skip flag for ldap is false
* OAuth: Skip org roles sync if no roles were provided by upstream service
* Grafana: Set SyncOrgRoles to true for authentication through proxy with grafana as backend
This commit fixes an incorrect comment in the Result struct in eval.go
that I had written some time ago. The comment now documents the
actual behaviour and content of this field.
* add anon sessions package
* add usage stat fn
* implement count for cache
* add anonservice to authn broker
* lint
* add tests for remote cache count
* move anon service to services
* wrap tagging in goroutine
* make func used
* Alerting: get alert rules on faults (#61248)
Two functions used to fetch alert rules from DB are updated:
- GetAlertRulesForScheduling
- ListAlertRules
Rows are scanned one by one so good ones are returned.
Common Error is logged with indication how many
rules failed on deserialization.
Resolved: #61248
* updates from review comments
Use latest github.com/grafana/grafana-plugin-sdk-go which includes changes to the TypeVersion property (always present in JSON).
Also included is squtil changes: SQL util - allow using the database scan type for converters
* Update oapi library and thema
* Use fork commit to fix elasticsearch and cloudwatch generators
* Update thema
* Fixes
* Update thema with last fixes
* Sync
* Fix test
* Update thema and schemas
* Update thema
* AuthN: Update comments for ClientParams
* AuthN: Update flag name from SyncTeamMembers to SyncTeams
* UserSync: rename function and fix order of parameters so it is correct
* UserSync: Fix so we skip check if no authModule or authID is passed
* UserSync: move quota check to create user function
* UserSync: Move FetchSyncedUserHook to UserSync
* UserSync: Move last seen user hook to user sync service
* ApiKey: Implement last seen hook as a client hook instead
This commit adds filterLabels, filterLabelsRe, removeLabels, and
removeLabelsRe functions to templates for custom labels and annotations.
It allows for use cases such as removing all private labels.
This commit changes the Data struct in template.go to use Labels
instead of map[string]string. It changes how labels are printed
when using {{ .Labels }} from map[foo:bar bar:baz] to
foo=bar, bar=baz.
This commit changes how labels are printed in templates for custom
annotations and labels from map[foo:bar bar:baz] to foo=bar, bar=baz.
Labels are comma separated, and sorted in increasing order.
This commit moves templating from the state package to a sub-package
called template. This sub-package will be the logical package for
future ease-of-use improvements to templating custom annotations
and labels.
* Use existing row struct instead of [2]string, add deserialization helper
* Replace Stream struct with stream struct which is exactly the same
* Drop unused status field
* Don't export queryRes and queryData
* Tests for custom marshalling
* Rename row fields to T and V for consistency with prometheus samples
* Rename row to sample
* structure dtos and private methods
* add basic LDAP service
* use LDAP service in ldap debug API
* lower non fatal error
* remove unused globals
* wip
* remove final globals
* fix tests to use cfg enabled
* restructure errors
* remove logger from globals
* use ldap service in authn
* use ldap service in context handler
* fix failed tests
* fix ldap middleware provides
* fix provides in auth_test.go
* Range splitting: range splitting function
* Range splitting: experiment with 1 hour splits
* Range splitting: reorganize code
* Range splitting: improve code readability and meaning
* Range splitting: add partition limit to prevent infinite loops
* Range splitting: add error handling
* Range splitting: disable for logs queries
* Range splitting: support any arbitrary time splitting + respect original from/to in the partition
* Chore: remove console logs
* Chore: delete unused import
* Range splitting: actually send requests in sequence
* Range splitting: do not split when > 1 query
* Range splitting: combine frames
* Chore: rename function
* split in reverse
* polished reversing
* keep reference to the right frame in the result
* Range splitting: change request state to Streaming
* Range splitting: fix moving only 1 unit of time instead of the provided one
* Chore: change default parameter to timeShift = 1
* Range splitting: do not split for range queqries
* Range splitting: add initial support for log queries
* Range splitting: do not use MutableDataFrame
It has bad performance and it's not required
* Chore: remove unused export
* Query Splitting: move to module
* loki: split: fix off-by-one error (#62966)
loki: split: fix off-by-one loop
* Range splitting: disable for logs volume queries
* Range splitting: combine any number of fields, not just hardcoded 2
* Range splitting: optimize frame-combining function
* Range splitting: further optimize
* Range splitting: combine frame length
* Range splitting: combine stats
* Range splitting: combine stats without assuming the same order
* Query splitting: catch and raise errors
* Range splitting: create feature flag
* Range splitting: implement feature flag
* Range splitting: add unit test for datasource query
* Range splitting: add basic test for runPartitionedQuery
* Range splitting: add unit test for resultLimitReached
* Range splitting: test frame merging
* Chore: fix unit test
---------
Co-authored-by: Sven Grossmann <svennergr@gmail.com>
Co-authored-by: Gábor Farkas <gabor.farkas@gmail.com>
The `rule_groups_rules` metric is currently defined and computed by `State`.
It makes more sense for this metric to be computed off of the configured rule
set, not based on the rule evaluation state. There could be an edge condition
where a rule does not have a state yet, and so is uncounted.
Additionally, we would like this metric (and others), to have a `rule_group`
label, and this is much easier to achieve if the metric is produced from the
`Scheduler` package.
* Remove Result field from AddDataSourceCommand
* Remove DatasourcesPermissionFilterQuery Result
* Remove GetDataSourceQuery Result
* Remove GetDataSourcesByTypeQuery Result
* Remove GetDataSourcesQuery Result
* Remove GetDefaultDataSourceQuery Result
* Remove UpdateDataSourceCommand Result
* Export `StatsCount` function
* Add new statKV each time a bundle is created
* Export bundle count
---------
Co-authored-by: Jo <joao.guerreiro@grafana.com>
* fix(dashboard version service): add DashboardUID to query and responses
The DashboardUID was not populated in the response from Get and ListDashboardVersions. This adds the DashboardUID to the Get query (it was already in List) and populated the DashboardUID in the returned DashboardVersionDTOs.
* change error to warning, apply correct format
* Update pkg/services/ngalert/store/alertmanager.go
Co-authored-by: George Robinson <george.robinson@grafana.com>
* different wording and data
---------
Co-authored-by: George Robinson <george.robinson@grafana.com>
* fetch inherited folder scopes for search v2
* fix error and don't fetch parents for the general folder
* remove feature toggle check and lower log level
* add bundle registry service to avoid dependency cycles
* move user support bundle collector to user service
* move usage stat bundle implementation to usage stats
* add info for background service
* fix remaining imports
* whitespace
* reorganize auth usage stats
* usage stat privilege elevators
* stat count of modified role
* cfg related info
* add authn anon client
* kv store
* ensure anon enabled is collected even if client is not registered
* fix usage stats test
* AuthN: Add HookClient interface
* AuthN: Check if client implement authn.HookClient and call the hook if
it does
* AuthN: Convert refresh token hook into a client hook
* *Create Caching Config interface and OSS impl
*Create front-end facing DS Cache config
*Populate Caching Config on Datasource DTO
*Update OSS wire deps
* fix unit test
* handle query caching TTL override on the frontend
* Make sure the override works with pubdash
* move caching config to the right place in the ds info
* move caching config logic to enterprise index hook
* move queryCachingTTL to pubdash query payload
* Remove from metadata (not needed)
* rename struct and add comment
* remove invalid wire dependency
* manual revert of 395c74b
* fix frontend test
* fix backend test
* fix tests for real this time
* truly fix frontend test
* fix back end unit test for real
* Alerting: implement loki query for alert state history
* extract selector building
* add unit tests for selector creation
* backup
* give selectors their own type
* build dataframe
* add some tests
* small changes after manual testing
* use struct client
* golint
* more golint
* Make RuleUID optional for Loki implementation
* Drop initial assumption that we only have one series
* Pare down to three columns, fix timestamp overflows, improve failure cases in loki responses
* Embed structred log lines in the dataframe as objects rather than json strings
* Include state history label filter
* Remove dead code
---------
Co-authored-by: Alex Weaver <weaver.alex.d@gmail.com>
* Alerting: Fix template validation in provisioning api
Fix issue where provisioning API accepts a malformed template having extra
text outside of definition block and template name matching definition name.
* Mark AM configuration as applied
* add missing checks, make linter happy
* fix deadlock, mark as valid on save and on load
* mark configurations only if needed
* check error after applyConfig()
* code review comments
* code review changes
* more code review changes
* clean HistoricConfigFromAlertConfig function
* Define endpoint and generate
* Wire up and register endpoint
* Cleanup, define authorization
* Forgot the leading slash
* Wire up query and SignedInUser
* Wire up timerange query params
* Add todo for label queries
* Drop comment
* Update path to rules subtree
* Extract label merge, add test file
* Extract error/NoData to first class fields, remove a layer from values
* Include dashUID and panelID as line-level fields
* Drop unnecessary object receiver
* Add tests for stream building
* Drop NoData field from log lines
* Set YAML as default value for exporting alert rules
* use YAML format for rule list export
Co-authored-by: Sonia Aguilar <33540275+soniaAguilarPeiron@users.noreply.github.com>
* lint
* Add new format query param to swagger+docs
* Fix broken test
---------
Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>
Co-authored-by: Matt Jacobson <matthew.jacobson@grafana.com>
* Nested folders: Do not skip integration tests
* SQLStore: Fix folder migration
It reduces the length of the title column to be equal with the respective
dashboard column.
This change exposes more metrics from the embedded Alertmanager, which are
valuable for troubleshooting Alertmanager operation particularly in HA setups.
```
grafana_alerting_notifications_total
grafana_alerting_notifications_failed_total
grafana_alerting_notification_requests_total
grafana_alerting_notification_requests_failed_total
grafana_alerting_notification_latency_seconds
grafana_alerting_nflog_gc_duration_seconds
grafana_alerting_nflog_snapshot_duration_seconds
grafana_alerting_nflog_snapshot_size_bytes
grafana_alerting_nflog_queries_total
grafana_alerting_nflog_query_errors_total
grafana_alerting_nflog_query_duration_seconds
grafana_alerting_nflog_gossip_messages_propagated_total
grafana_alerting_dispatcher_aggregation_groups
grafana_alerting_dispatcher_alert_processing_duration_seconds
```
Note that `alertmanager_dispatcher_aggregation_group_limit_reached_total` is
explicitly not exposed, as the group limit metrics are not enabled.
* remove result field from alert notification commands and queries
* fix a few more tests
* ok, linter
* remove alert result fields
* fix api calls
* et tu, linter
