The ordering of links in the navigation bar is currently based the order of the slice containing the navigation tree. Since Grafana supports adding more links to the navigation bar with `RunIndexDataHooks` which runs at the very end of the function this means that any link registered through a hook will be placed last in the slice and be displayed last in the menu. With this PR the ordering can be specified with a weight which allows for placing links created by extensions in a more intuitive place where applicable.
Stable sorting is used to ensure that the current FIFO ordering is preserved when either no weight is set or two items shares the same weight.
* CloudWatch: Datasource improvements
* Add statistic as template variale
* Add wildcard to list of values
* Template variable intercept dimension key
* Return row specific errors when transformation error occured
* Add meta feedback
* Make it possible to retrieve values without known metrics
* Add curated dashboard for EC2
* Fix broken tests
* Use correct dashboard name
* Display alert in case multi template var is being used for some certain props in the cloudwatch query
* Minor fixes after feedback
* Update dashboard json
* Update snapshot test
* Make sure region default is intercepted in cloudwatch link
* Update dashboards
* Include ec2 dashboard in ds
* Do not include ec2 dashboard in beta1
* Display actual region
* LDAP: last org admin (that's going to be removed) can login
Previously, if you tried to login with LDAP but were that last org admin
of an org that you would no longer be an admin of after sync (which
happens at login), you wouldn't be able to login due to an error.
Adds a new setting disable_admin_user and when true the default
admin user will not be created when Grafana starts for the first
time (or no users exists in the system).
Closes#19038
* AuthProxy: Can now login with auth proxy and get a login token
* added unit tests
* renamed setting and updated docs
* AuthProxy: minor tweak
* Fixed tests and namings
* spellfix
* fix
* remove unused setting, probably from merge conflict
* fix
The arching goal of this commit is to enable single user
synchronisation with LDAP. Also, it included minor fixes of style,
error messages and minor bug fixing.
The changes are:
- bug: The `multildap` package has its own errors when the user is
not found. We fixed the conditional branch on this error by asserting
on the `multildap` errors as opposed to the `ldap` one
- bug: The previous interface usage of `RevokeAllUserTokens` did not
work as expected. This replaces the manual injection of the service by
leveraging the service injected as part of the `server` struct.
- chore: Better error messages around not finding the user in LDAP.
- fix: Enable the single sync button and disable it when we receive an
error from LDAP. Please note, that you can enable it by dispatching
the error. This allows you to try again without having to reload the
page.
- fix: Move the sync info to the top, then move the sync button above
that information and clearfix to have more harmony with the UI.
* Renamed ttl config in code to be more consistent with behaviour
* Introduced new setting `sync_ttl` in .ini file
* Keeping the old setting `ldap_sync_ttl` in the .ini file as fallback and compatibility.
Adds support for Generic OAuth role mapping. A new
configuration setting for generic oauth is added named
role_attribute_path which accepts a JMESPath expression.
Only Grafana roles named Viewer, Editor or Admin are
accepted.
Closes#9766
* LDAP Debug: No longer shows incorrectly matching groups based on role
Org Role was used as a shortcut to figure out what groups were matching
and which weren't. That lead to too all groups matching a specific role
to show up for a user if that user got that role.
* LDAP Debug: Fixes ordering of matches
The order of groups in the ldap.toml file is important, only the first
match for an organisation will be used. This means we have to iterate
based on the config and stop matching when a match is found.
We might want to think about showing further matches as potential
matches that are shadowed by the first match. That would possibly make
it easier to understand why one match is used instead of another one.
* LDAP Debug: never display more than one match for the same LDAP group/mapping.
* LDAP Debug: show all matches, even if they aren't used
* Update public/app/features/admin/ldap/LdapUserGroups.tsx
Co-Authored-By: gotjosh <josue.abreu@gmail.com>
* Update public/app/features/admin/ldap/LdapUserGroups.tsx
Co-Authored-By: gotjosh <josue.abreu@gmail.com>
* Licensing: supplies a service to handle licensing information
* Licensing: uses the license service further
Uses the license service instead of settings.isEnterprise:
- external team members
- saml
- usage stats
* Licensing: fixes broken tests due to new Licensing service dependency
* Licensing: fixes linting errors
* Licensing: exposes license expiry information to the frontend
Fixes so that all notification channels configured for an alert should
try to send notification even if one notification channel fails to send
a notification.
Signed-off-by: Abhilash Gnan <abhilashgnan@gmail.com>
Fixes#19768
* pkg/cmd: Check errors
* pkg/cmd: Make sure server waits on services, even in case of error
* pkg/cmd: Inform of error to show help
* pkg/cmd: Only warn on failure to send systemd notification
* pkg/cmd: Don't log errors stemming from context cancelation
* pkg/cmd: Don't fail if unable to write to systemd
* Rename GrafanaServerImpl to Server
* Make flag dependencies explicit
* Extract method for building service graph
* Document and clarify methods
* Rename HttpServer to HTTPServer
fix pkg/services/search/service.go:52:3: s.bus undefined (type *SearchService has no field or method bus, but does have Bus)
fixes issue introduced by #19765
* Convert Azure Application Insights datasource to Go
Allows for alerting of Application Insights data source
Closes: #15153
* Fix timeGrainReset
* Default time interval for querys for alerts
* Fix a few rename related bugs
* Update readme to indicate App Insights alerting
* Fix typo and add tests to ensure migration is happening
* Address code review feedback (mostly typos and unintended changes)
* Chore: Upgrade to Go 1.13
Fixes: #18878
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* Chore: Upgrade lint tools in order to work with Go 1.13
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* Chore: Fix Go linting issues
Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
* Fix: Handle empty nested keys on YAML provisioning datasources
As we provision a datasource via a YAML file, we attempt to transform the
file into sensible Go types that the provisioning code can use.
While this happens, there is a chance some of the keys nested within
the YAML array are empty.
This fix allows the YAML parser to handle empty keys by null checking
the return of `reflect.TypeOf` which according to the documentation:
> TypeOf returns the reflection Type that represents the dynamic type of i. If i is a nil interface value, TypeOf returns nil.
Can return nil.
* Add tests
* API: Add `createdAt` and `updatedAt` to api/users/lookup
In the past, we have added both `updatedAt` (#19004) and `createdAt` (#19475) to /api/users/:id
Turns out, api/users/lookup uses the same DTO for both. This fixes the serialization of both `createdAt` and `updatedAt`for this endpoint.
Also, adds a test to ensure no further regressions.
* Updated API documentation
* use grafana-plugin-model dependency that uses go modules
* use grafana-plugin-model with updated hashicorp/go-plugin
* use grafana-plugin-model with re-compiled protos
* test using protoc-gen-go v1.2.0 tag
* use grafana-plugin-model with re-compiled protos
* chore: fix deprecation warning for lint
* use latest grafana-plugin-model
Fixes#19454
Only return certain mysql errors from backend.
The following errors is returned as is from backend:
error code 1064 (parse error)
error code 1054 (bad column/field selected)
error code 1146 (table not exists)
Any other errors is logged and returned as a generic
error.
Restrict use of certain functions:
Do not allow usage of the following in query:
system_user()
session_user()
current_user() or current_user
user()
show grants
Fixes#19360
* Multi-LDAP: Do not fail-fast on invalid credentials
When configuring LDAP authentication, it is very common to have multiple
servers configured. When using user bind (authenticating with LDAP using
the same credentials as the user authenticating to Grafana) we don't
expect all the users to be on all LDAP servers.
Because of this use-case, we should not fail-fast when authenticating on
multiple LDAP server configurations. Instead, we should continue to try
the credentials with the next LDAP server configured.
Fixes#19066
* Update README.md
Capitalized the G and S in "Getting Started," and moved "guide" to match the section title in the docs.
* Fixed sentence structure. Changed "download" to "grafana.com/get" and changed "get" to "download".
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* Docs: Replace "datasources" with "data sources" (#19111)
* LDAP: Show all LDAP groups
* Use the returned LDAP groups as the reference when debugging LDAP
We need to use the LDAP groups returned as the main reference for
assuming what we were able to match and what wasn't. Before, we were
using the configured groups in LDAP TOML configuration file.
* s/User name/Username
* Add a title to for the LDAP mapping results
* LDAP: UI Updates to debug view
* LDAP: Make it explicit when we weren't able to match teams
This reverts commit 2514209 from #18384. Reason is that it doesn't
work due to xorm 0.7.1 which doesn't support this new connectionstring
format.
Fixes#19189
Ref #18384
Ref #17665
* Add new query mode picker with different states for each query. Also really simple migration script
* Populate cross resource dropdowns
* Cleanup. Handle change events
* Add multi select picker for subscriptions
* Fix markup issue
* Prepare for new query mode
* More cleanup
* Handle multiple queries both in ds and backend
* Refactoring
* Improve migration
* Add support for multiselect display name
* Use multiselect also for locations and resources
* Add more typings
* Fix migrations
* Custom multiselect built for array of options instead of variables
* Add url builder test
* fix datasource tests
* UI fixes
* Improve query editor init
* Fix brokens tests
* Cleanup
* Fix tslint issue
* Change query mode display name
* Make sure alerting works for single queries
* Friendly error for multi resources
* Add temporary typings
Don't update total stats metrics if reporting is disabled.
New setting disable_total_stats for turning off update
of total stats (stat_totals_*) metrics.
Ref #19137
* Add items for navmodel and basic page
* add reducer and actions
* adding user mapping table component
* adding components for ldap tables
* add alert box on error
* close error alert box
* LDAP status page: connect APIs WIP
* LDAP debug: fetch connection status from API
* LDAP debug: fetch user info from API
* LDAP debug: improve connection error view
* LDAP debug: connection error tweaks
* LDAP debug: fix role mapping view
* LDAP debug: role mapping view tweaks
* LDAP debug: add bulk-sync button stub
* LDAP debug: minor refactor
* LDAP debug: show user teams
* LDAP debug: user info refactor
* LDAP debug: initial user page
* LDAP debug: minor refactor, remove unused angular wrapper
* LDAP debug: add sessions to user page
* LDAP debug: tweak user page
* LDAP debug: tweak view for disabled user
* LDAP debug: get sync info from API
* LDAP debug: user sync info
* LDAP debug: sync user button
* LDAP debug: clear error on page load
* LDAP debug: add user last sync info
* LDAP debug: actions refactor
* LDAP debug: roles and teams style tweaks
* Pass showAttributeMapping to LdapUserTeams
* LDAP debug: hide bulk sync button
* LDAP debug: refactor sessions component
* LDAP debug: fix loading user sessions
* LDAP debug: hide sync user button
* LDAP debug: fix fetching unavailable /ldap-sync-status endpoint
* LDAP debug: revert accidentally added fix
* LDAP debug: show error when LDAP is not enabled
* LDAP debug: refactor, move ldap components into ldap/ folder
* LDAP debug: styles refactoring
* LDAP debug: ldap reducer tests
* LDAP debug: ldap user reducer tests
* LDAP debug: fix connection error placement
* Text update
* LdapUser: Minor UI changes moving things around
* AlertBox: Removed icon-on-top as everywhere else it is centered, want to have it be consistent
* LDAP: Allow an user to be synchronised against LDAP
This PR introduces the /ldap/sync/:id endpoint. It allows a user to be synchronized against LDAP on demand.
A few things to note are:
LDAP needs to be enabled for the sync to work
It only works against users that originally authenticated against LDAP
If the user is the Grafana admin and it needs to be disabled - it will not sync the information
Includes a tiny refactor that favours the JSONEq assertion helper instead of manually parsing JSON strings.
* API: Add `updatedAt` to api/users/:id
This adds the timestamp of when a particular user was last updated to
the `api/users/:id` endpoint.
This helps our administrators understand when was the user information last
updated. Particularly when it comes from external systems e.g. LDAP
Adds the definition of `GetTeamsForLDAPGroupCommand` which handles the lookup of team information based on LDAP groupDNs.
This is an Enterprise only feature. To diferentiate,a response will contain the `team` key as `null` on OSS while on Enterprise the key will contain an empty array `[]` when no teams are found.
* LDAP: Add API endpoint to query the LDAP server(s) status|
This endpoint returns the current status(es) of the configured LDAP server(s).
The status of each server is verified by dialling and if no error is returned we assume the server is operational.
This is the last piece I'll produce as an API before moving into #18759 and see the view come to life.
* Move the ReloadLDAPCfg function to the debug file
Appears to be a better suite place for this.
* LDAP: Return the server information when we find a specific user
We allow you to specify multiple LDAP servers as part of LDAP authentication integration. As part of searching for specific users, we need to understand from which server they come from. Returning the server configuration as part of the search will help us do two things:
- Understand in which server we found the user
- Have access the groups specified as part of the server configuration
* LDAP: Adds the /api/admin/ldap/:username endpoint
This endpoint returns a user found within the configured LDAP server(s). Moreso, it provides the mapping information for the user to help administrators understand how the users would be created within Grafana based on the current configuration.
No changes are executed or saved to the database, this is all an in-memory representation of how the final result would look like.
* Emails: resurrect template notification
* Phantomjs (oh yeah, there is another dev dep phantom :-) was failing for
the generation of the html templates so I had to update the dependencies
in order to fix it. While doing that I update the scripts field and docs
for it as well. yarn.lock is included
* Move splitting of the emails to separate helper function, since more services
coming up that would need to use this functionality
* Add support for enterprise specific email letters. Probably could
be done in the better way, but it's not a priority right now
* Auth: change the error HTTP status codes
* Use 407 HTTP status code for incorrect credentials error
* Improve proxy auth logs
* Remove no longer needed TODO comment
Fixes#18439
It seems `ldap` module introduced new error type of which
multildap module didn't know about.
This broke the multildap login logic
Fixes#18491
Ref #18587
* SQLite migrations
* cleanup
* migrate end times
* switch to update with a query
* real migration
* anno migrations
* remove old docs
* set isRegion from time changes
* use <> for is not
* add comment and fix index decleration
* single validation place
* add test
* fix test
* add upgrading docs
* use AnnotationEvent
* fix import
* remove regionId from typescript
Existing /api/alert-notifications now requires at least editor access.
Existing /api/alert-notifiers now requires at least editor access.
New /api/alert-notifications/lookup returns less information than
/api/alert-notifications and can be access by any authenticated user.
Existing /api/org/users now requires org admin role.
New /api/org/users/lookup returns less information than
/api/org/users and can be access by users that are org admins,
admin in any folder or admin of any team.
UserPicker component now uses /api/org/users/lookup instead
of /api/org/users.
Fixes#17318
* Do not set SameSite login_error cookie attribute if cookie_samesite is none
* Do not set SameSite grafana_session cookie attribute if cookie_samesite is none
* Update middleware tests
* Fix CreateTeam api endpoint
No team member should be created for requests
authenticated by API tokens.
* Update middleware test
Assert that `isAnonymous` is set for `SignedInUser`
authenticated via API key.
* Add test for team creation
Assert that no team member is created if the signed in user
is anomymous.
* Revert "Fix CreateTeam api endpoint"
This reverts commit 9fcc4e67f5.
* Revert "Update middleware test"
This reverts commit 75f767e58d.
* Fix CreateTeam api endpoint
No team member should be created for requests
authenticated by API tokens.
* Update team test
* Change error to warning and update tests
This commit addresses half of #13749 by making sure GetMetricData
works for alerting. Math Expressions (compound metrics) will still not
work for alerting, this would require a bigger refactoring of Grafana's
alerting service. However, with this commit at least alerting for basic
metrics with non empty query Id will work.
Fixes half of #13749
* Auth: Do not search for the user twice
Previously `initContextWithBasicAuth` did not use `LoginUserQuery`, doing
`GetUserByLoginQuery` only i.e. looking user in DB only, things changed when
this function started to check LDAP provider via `LoginUserQuery` (#6940),
however, this request was placed after `GetUserByLoginQuery`, so we first
looking in DB then in the LDAP - if LDAP user hasn't logged in we will
not find it in DB, so `LoginUserQuery` will never be reached.
`LoginUserQuery` request already performs `GetUserByLoginQuery`
request in correct sequence. So we can just remove redundant request.
* Correct sequence execution during authentification &
introduce tests for it
* Move basic auth tests to separate test file, since main test file already
pretty large
* Introduce `testing.go` for the middleware module
* Remove redundant test helper function
* Make handler names more explicit
Ref 5777f65d05Fixes#18329
* Auth: address review comment
* added alert rule tags in webhook notifications
* fix: don't include whole list of Tag objects but only key/value pairs in Webhook JSON
* marked webhook alerts to support alert rule tags
* Add tests for errors basic auth cases and moves tests to separate test-case.
Also names test cases consistently
* Add additional test helper
Ref 82661b9f69
* LDAP: nitpicks
* Add more tests
* Correct and clarify comment for Login() method
* Rename methods (hail consistency!)
* Uppercases first letter of the logs everywhere
* Moves method definitions around to more appropriate places
Fixes#18295
* Auth: consistently return same basic auth errors
* Put repeated errors in consts and return only those consts as error strings
* Add tests for errors basic auth cases and moves tests to separate test-case.
Also names test cases consistently
* Add more error logs and makes their messages consistent
* A bit of code style
* Add additional test helper
* Auth: do not expose even incorrect password
* Auth: address review comments
Use `Debug` for the cases when it's an user error
The `oauth_state` cookie used to be created with the SameSite value set
according to the `cookie_samesite` configuration.
However, due to a Safari bug SameSite=None or SameSite=invalid are treated
as Strict which results in "missing saved state" OAuth login failures
because the cookie is not sent with the redirect requests to the OAuth
provider.
This commit always creates the `oauth_state` cookie with SameSite=Lax
to compensate for this.
* Auth Proxy: Include additional headers as part of the cache key
Auth proxy has support to send additional user attributes as part of the
authentication flow. These attributes (e.g. Groups) need to be monitored
as part of the process in case of change.
This commit changes the way we compute the cache key to include all of the
attributes sent as part of the authentication request. That way, if we
change any user attributes we'll upsert the user information.
Allow non admins to see plugins list but only with readme. Any config tabs are hidden from the plugin page. Also plugin panel does not show action buttons (like Enable) for non admins.
* LDAP: improve POSIX support
* Correctly abtain DN attributes result
* Allow more flexibility with comparison mapping between POSIX group & user
* Add devenv for POSIX LDAP server
* Correct the docs
Fixes#18140
* Reapply MetricNamespace support
* Fixing tests
* refactor: move metricnamespace param to backend
* refactor: remove unused function
* azuremonitor: migration for new metric namespace field
* azuremonitor: add template query for metric namespace with a sub
* docs: template queries for azure monitor
Adds new lines for the metricnamespace template function and fixes
some messed up lines
Adds pulse waveform. Is predictable in the sense that the start of the waveform is aligned to epoch time (instead of the start of the query time). This makes a useful signal for manual testing of alerting in the devenv.
* Metrics: remove unused metrics
Metric `M_Grafana_Version` is not used anywhere, nor the mentioned
`M_Grafana_Build_Version`. Seems to be an artefact?
* Metrics: make the naming consistent
* Metrics: add comments to exported vars
* Metrics: use proper naming
Fixes#18110
* Add support for `is_disabled` to `CreateUser()`
* Add support for `is_disabled` to `SearchUsers()`
Had to add it as a `string` type not as `bool`, since if that's property
is omitted, we would have add it to SQL request, which might be dangerous
* Restructure desctructive tests and add more
* API: Duplicate API Key Name Handle With Useful HTTP Code
* 17447: make changes requested during review
- use dialect.IsUniqueContraintViolation
- change if statement to match others
- return error properly
* Revert "17447: make changes requested during review"
This reverts commit a4a674ea83.
* API: useful http code on duplicate api key error w/ tests
* API: API Key Duplicate Handling
fixed small typo associated with error
* Minor fix for nil pointer when trying to log error
* Do not return error if a dashboard is created
Only log the failures
* Do not return error if the folder is created
Only log the failures
* Users: show badges for each auth provider
* Chore: don't use functions in angular bindings
* Users: minor style changes to labels
* Chore: convert auth labels on the backed side, deduplicate frontend code
* Users: use authLabels everywhere instead of authModule
* User: fix edit user page style
* Users: minor fixes after review
* Add tests for login view
* Fix OAuth auto login redirect loop
login_error cookie is only set when the OAuth login fails
for some reason. Therefore, the login view should return
immediately if a login_error cookie exists before trying
to login the user using OAuth again.
* Fix test
Use 'index-template' instead of 'index' for testing
* Add some comments
* LDAP:Docs: `active_sync_enabled` setting
Mention `active_sync_enabled` setting and enable it by default
* LDAP: move "disableExternalUser" method
Idea behind new design of the LDAP module is to minimise conflation
between other parts of the system, so it would decoupled as much as
possible from stuff like database, HTTP transport and etc.
Following "Do One Thing and Do It Well" Unix philosophy principal, other things
could be better fitted on the consumer side of things.
Which what this commit trying to archive
* LDAP: correct user/admin binding
The second binding was not happening, so if the admin login/password
in LDAP configuration was correct, anyone could had login as anyone using
incorrect password
* Add SAML configuration options
* Add crewjam/saml as a depdency
Needed as part of the enterprise SAML integration.
* Vendor github.com/stretchr/testify/require
The package require implements the same assertions as the `assert` package but stops test execution when a test fails.
* feat: AzureMonitor implements legend key on backend
To be able to remove the duplicated query logic on the
frontend, the backend code needs to implement alias
patterns for legend keys as well as allowing the default
list of allowed time grains to be overridden. Some metrics
do not support all the time grains and the auto timegrain
calculation can be incorrect if the list is not overridden.
* feat: AzureMonitor - removes duplicate query logic on frontend
* AzureMonitor small refactoring
Extracted method and tidied up the auto time grain
code.
* azuremonitor: support for auto time grains for alerting
Converts allowed timegrains into ms and saves in dashboard json.
This makes queries for alerting with an auto time grain work in
the same way as the frontend.
* chore: typings -> implicitAny count down to 3413
* azuremonitor: add more typings
* LDAP: Divide the requests
Active Directory does indeed have a limitation with 1000 results
per search (default of course).
However, that limitation can be workaround with the pagination search feature,
meaning `pagination` number is how many times LDAP compatible server will be
requested by the client with specified amount of users (like 1000). That feature
already embeded with LDAP compatible client (including our `go-ldap`).
But slapd server has by default stricter settings. First, limitation is not 1000
but 500, second, pagination workaround presumably (information about it a bit
scarce and I still not sure on some of the details from my own testing)
cannot be workaround with pagination feature.
See
https://www.openldap.org/doc/admin24/limits.htmlhttps://serverfault.com/questions/328671/paging-using-ldapsearchhashicorp/vault#4162 - not sure why they were hitting the limit in
the first place, since `go-ldap` doesn't have one by default.
But, given all that, for me `ldapsearch` command with same request
as with `go-ldap` still returns more then 500 results, it can even return
as much as 10500 items (probably more).
So either there is some differences with implementation of the LDAP search
between `go-ldap` module and `ldapsearch` or I am missing a step :/.
In the wild (see serverfault link), apparently, people still hitting that
limitation even with `ldapsearch`, so it still seems to be an issue.
But, nevertheless, I'm still confused by this incoherence.
To workaround it, I divide the request by no more then
500 items per search
* Teams: show proper label for each auth provider
Teams: don't sore AuthModule in team_member table, use JOIN to get it instead
* Teams: fix AddTeamMember after last changes
* Teams: add more auth provider labels
* Teams: show external sync badge if LDAP is not enabled
* Teams: tests for getting auth module
* Build: use golangci-lint as a make command
* Since gometalinter was deprecated in favor of golangci-lint so it was
replaced by it. Responsibilities held by the gometalinter was moved to
golangci-lint
* There was some changes in implementation (that was also mentioned in
the code comment) between the tools, which uncovered couple errors
in the code. Those issues were either solved or disabled by
the inline comments
* Introduce the golangci-lint config, to make their
configuration more manageable
* Build: replace backend-lint.sh script with make
* Docs: correct link to the Grafana build-container.
* Docs: documents usage on the gopath when upgrading deps.
* Dev docs: info for upgrading deps only in one location.
* Add LDAP config instead sed use
* Add container name
* Add SizeLimit option to client and to server.
Probably useless at this point, but it's better to have it then otherwise
* OAuth: github team sync POC
* OAuth: minor refactor of github module
* OAuth: able to use team shorthands for github team sync
* support passing a list of groups via auth-proxy header
* Modify backend to allow expiration of API Keys
* Add middleware test for expired api keys
* Modify frontend to enable expiration of API Keys
* Fix frontend tests
* Fix migration and add index for `expires` field
* Add api key tests for database access
* Substitude time.Now() by a mock for test usage
* Front-end modifications
* Change input label to `Time to live`
* Change input behavior to comply with the other similar
* Add tooltip
* Modify AddApiKey api call response
Expiration should be *time.Time instead of string
* Present expiration date in the selected timezone
* Use kbn for transforming intervals to seconds
* Use `assert` library for tests
* Frontend fixes
Add checks for empty/undefined/null values
* Change expires column from datetime to integer
* Restrict api key duration input
It should be interval not number
* AddApiKey must complain if SecondsToLive is negative
* Declare ErrInvalidApiKeyExpiration
* Move configuration to auth section
* Update docs
* Eliminate alias for models in modified files
* Omit expiration from api response if empty
* Eliminate Goconvey from test file
* Fix test
Do not sleep, use mocked timeNow() instead
* Remove index for expires from api_key table
The index should be anyway on both org_id and expires fields.
However this commit eliminates completely the index for now
since not many rows are expected to be in this table.
* Use getTimeZone function
* Minor change in api key listing
The frontend should display a message instead of empty string
if the key does not expire.
* batch disable users
* batch revoke users tokens
* split batch disable user and revoke token
* API: get users with auth info and isExternal flag
* fix tests for batch disable users
* Users: refactor /api/users/search endpoint
* Users: use alias for "user" table
* Chore: add BatchDisableUsers() to the bus
* Users: order user list by id explicitly
* Users: return AuthModule from /api/users/:id endpoint
* Users: do not return unused fields
* Users: fix SearchUsers method after last changes
* User: return auth module as array for future purposes
* User: tests for SearchUsers()
* User: return only latest auth module in SearchUsers()
* User: fix JOIN, get only most recent auth module
* tsdb: add support for setting debug flag of tsdb query
* alerting: adds debug flag in eval context
Debug flag is set when testing an alert rule and this debug
flag is used to return more debug information in test aler rule
response. This debug flag is also provided to tsdb queries so
datasources can optionally add support for returning additional
debug data
* alerting: improve test alert rule ui
Adds buttons for expand/collapse json and copy json to clipboard,
very similar to how the query inspector works.
* elasticsearch: implement support for tsdb query debug flag
* elasticsearch: embedding client response in struct
* alerting: return proper query model when testing rule
* Feature: Introduce a grafana-cli wrapper
When our users install the *nix packed version of grafana, tendency is to use the services and scripts installed as part of the package for grafana-server. These leverage the default configuration options by specifying the several default paths.
This introduces a similar approach for the grafana-cli binary. We exposed it through a wrapper to ensure a proper configuration is in place. To enable that, we add the .real suffix to the original binary (grafana-cli.real) and then use a bash script named grafana-cli as the wrapper.
* Make the config and homepath flags global
* Introduce `configOverrides` as a global flag
This flag allows us to pass configuration overrides as a string.
The string follows the convention of configuration arguments separated by a space e.g. "cfg:default.paths.data=/dev/nullX cfg:default.paths.logs=/dev/nullX"
Also, it is backwards compatible with similar the previous configuration method through tailing arguments. Tailing arguments take presedence over the configuration options string.
* Only log configuration information in debug mode
* Move the grafana-cli binary to $GRAFANA_HOME/bin
As part of the package install process, we copy all the release files and
directories into the grafana home directory. This includes the /bin folder
from where we copied the binaries into their respective destinations.
After that, the /bin folder gets deleted as we don't want to keep
duplicates of the binaries around.
As part of this commit, we moved the re-creation of /bin within
grafana-home and the copy of the original binary (again) after the
folder gets deleted.
`grafana-cli` uses the third-party library to define the flags and not
the standard library. Using `flag.Parse` conflicts with the defined
flags from our third-party library.
In the case where `flag.Parse` is used, the CLI assumes that definitions
provided are not needed and will not define them; producing errors of
the kind `flag provided but not defined --example-flag`.
Using the context to read any arguments (including flags) is the
recommended approach by the third-party library.
grafana-cli should allow configuration overrides to be received from the command line. e.g.
```
grafana-cli admin reset-password cfg:default.paths.logs=custom/log/directory/
```
Seems like we missed the inclusion of `flag.Parse` as we run the command, to be able to consume them.
Additionally, it'll be useful for the user to know whenever these are being overriden or not - hence the addition of logging the configuration to be used as we run the command.
* LDAP: use only one struct
* Use only models.ExternalUserInfo
* Add additional helper method :/
* Move all the helpers to one module
* LDAP: refactoring
* Rename some of the public methods and change their behaviour
* Remove outdated methods
* Simplify logic
* More tests
There is no and never were tests for settings.go, added tests for helper
methods (cover is now about 100% for them). Added tests for the main
LDAP logic, but there is some stuff to add. Dial() is not tested and not
decoupled. It might be a challenge to do it properly
* Restructure tests:
* they wouldn't depend on external modules
* more consistent naming
* logical division
* More guards for erroneous paths
* Login: make login service an explicit dependency
* LDAP: remove no longer needed test helper fns
* LDAP: remove useless import
* LDAP: Use new interface in multildap module
* LDAP: corrections for the groups of multiple users
* In case there is several users their groups weren't detected correctly
* Simplify helpers module
* Implementation of optimistic lock pattern
Try to insert the remote cache key and handle integrity error
* Remove transaction
Integrity error inside a transaction results in deadlock
* Remove check for existing remote cache key
Is no longer needed since integrity constrain violations are handled
* Add check for integrity constrain violation
Do not update the row if the insert statement fails
for other than an integrity constrain violation
* Handle failing inserts because of deadlocks
If the insert statement fails because of a deadlock
try to update the row
* Add utility function for returning SQL error code
Useful for debugging
* Add logging for failing expired cache key deletion
Do not shallow it completely
* Revert "Add utility function for returning SQL error code"
This reverts commit 8e0b82c79633e7d8bc350823cbbab2ac7a58c0a5.
* Better log for failing deletion of expired cache key
* Add some comments
* Remove check for existing cache key
Attempt to insert the key without checking if it's already there
and handle the error situations
* Do not propagate deadlocks created during update
Most probably somebody else is trying to insert/update
the key at the same time so it is safe enough to ignore it
* x_xss_protection
* strict_transport_security (HSTS)
* x_content_type_options
these are currently defaulted to false (off) until the next minor release.
fixes#17509
* Feature: Parse user agent string in user auth token api response (#16222)
* Adding UA Parser Go modules attempt (#16222)
* Bring user agent vals up per req
* fix tests
* doc update
* update to flatten, no maps
* update doc
* wip: fix remote cache for redis
connstr parsing and non-negative expires for #17377
TODO: finish parse, check zero case, find out why negative duration in the first place
* finish parse.
Still TODO, find out negative value, and decide if would be better to make database specific entries in the .ini file
* update ini files
* remove accidental uncomment in defaults.ini
* auth_proxy: expiration non-negative so expiration is not in the past
* fix test, revert neg in redis
* review: use errutil
xorm introduced some changes in
https://github.com/go-xorm/xorm/pull/824 and
https://github.com/go-xorm/xorm/pull/876 which by default will use
public as the postgres schema and this was a breaking change compared
to before. Grafana has implemented a custom postgres dialect so above
changes wasn't a problem here. However, Grafana's custom database
migration was using xorm dialect to check if the migration table exists
or not.
For those using a custom search_path (schema) in postgres configured on
server, database or user level the migration table check would not find
the migration table since it was looking in public schema due to xorm
changes above. This had the consequence that Grafana's database
migration failed the second time since migration had already run
migrations in another schema.
This change will make xorm use an empty default schema for postgres and
by that mimic the functionality of how it was functioning before
xorm's changes above.
Fixes#16720
Co-Authored-By: Carl Bergquist <carl@grafana.com>
* Refactor: Removes replaceUrl from actions
* Refactor: Moves saveState thunk to epic
* Refactor: Moves thunks to epics
* Wip: removes resulttype and queries once
* Refactor: LiveTailing uses observer in query
* Refactor: Creates epics folder for epics and move back actioncreators
* Tests: Adds tests for epics and reducer
* Fix: Checks for undefined as well
* Refactor: Cleans up previous live tailing implementation
* Chore: merge with master
* Fix: Fixes url issuses and prom graph in Panels
* Refactor: Removes supportsStreaming and adds sockets to DataSourcePluginMeta instead
* Refactor: Changes the way we create TimeSeries
* Refactor: Renames sockets to streaming
* Refactor: Changes the way Explore does incremental updates
* Refactor: Removes unused method
* Refactor: Adds back Loading indication
Adds a new [server] setting `serve_from_sub_path`. By enabling
this setting and using a subpath in `root_url` setting, e.g.
`root_url = http://localhost:3000/grafana`, Grafana will be accessible
on `http://localhost:3000/grafana`. By default it is set to `false`
for compatibility reasons.
Closes#16623
* Removes Add/Remove methods
* Publicise necessary fields and methods so we could extend it
* Publicise mock API
* More comments and additional simplifications
* Sync with master
Still having low coverage :/ - should be addressed in #17208
Adds an additional sqlite error code 5 (SQLITE_BUSY) to the
transaction retry handler to add retries when sqlite
returns database is locked error.
More info: https://www.sqlite.org/rescode.html#busy
Ref #17247#16638
* Users: add is_disabled column
* Users: disable users removed from LDAP
* Auth: return ErrInvalidCredentials for failed LDAP auth
* User: return isDisabled flag in user search api
* User: mark disabled users at the server admin page
* Chore: refactor according to review
* Auth: prevent disabled user from login
* Auth: re-enable user when it found in ldap
* User: add api endpoint for disabling user
* User: use separate endpoints to disable/enable user
* User: disallow disabling external users
* User: able do disable users from admin UI
* Chore: refactor based on review
* Chore: use more clear error check when disabling user
* Fix login tests
* Tests for disabling user during the LDAP login
* Tests for disable user API
* Tests for login with disabled user
* Remove disable user UI stub
* Sync with latest LDAP refactoring
* fix: azuremonitor adds multi-sub support to alerting
* fix: AzureMonitor missing parameter in metadata func
getMetricMetadata function when called in the query ctrl
was missing a parameter for Subscription Id.
Also, made some tweaks to what happens when a chained
dropdown is changed to not reset all the fields that
are dependent on it.
Adds logs scenario which is quite basic and not that smart
to begin with. This will hopefully ease development of
Explore and support for logs in Grafana.
This makes sure the scenarios returned from API are sorted in a consistent
way and by that makes the values in scenario drop down always presented
ordered instead of jumping around.
* incapsulates multipleldap logic under one module
* abstracts users upsert and get logic
* changes some of the text error messages and import sort sequence
* heavily refactors the LDAP module – LDAP module now only deals with LDAP related behaviour
* integrates affected auth_proxy module and their tests
* refactoring of the auth_proxy logic
* Chore: explore possibilities of using makefile
This is an exploratory commit - I wanted to see how
revive/gosec linters could be integrated with makefile and our build scripts.
Looks better then I expected :)
* Chore: make revive happy
Revive execution was not supplied with path, if you restore there is couple
errors that were popping up - so I fixed them
* Chore: make revive happy
* remotecache: avoid race condition in set
since set called the database twice without transactions another
operation could insert a value before the first operation completed.
which would raise an error on insert since the data have been inserted
by the other request.
closes#17079
This adds support for using templated/dynamic urls in routes.
* refactor interpolateString into utils and add interpolation support for app plugin routes.
* cleanup and add error check for url parse failure
* add docs for interpolated route urls
Closes#16835
This fixes a regression introduced in #16646 where using Elasticsearch
pre-v7.0 and alerting resulted in an error when trying to deserialize the
response of total number of hits.
Total number of hits is not in use by the backend so we're removing it
for now to make ES 6 and 7 being able to deserialize search responses
without errors.
Closes#15622
When allow_embedding is false (default) the Grafana backend
will set the http header `X-Frame-Options: deny` in all responses
to non-static content which will instruct browser to not allow
Grafana to be embedded in `<frame>`, `<iframe>`,
`<embed>` or `<object>`.
Closes#14189
Currently all API requests set Cache-control: no-cache to avoid browsers
caching sensitive data. This fixes so that all responses returned from
backend not are cached using http headers. The exception is the data proxy
where we don't add these http headers in case datasource backend needs
to control whether data can be cached or not.
Fixes#16845
A new parameter `queryMode` is added to the InfluxDB datasource to provide a way to use POST instead of GET when querying the database. This prevents to get any error when querying the database with a heavy request.
Default configuration is kept to GET for backward compatibility. Tests and documentation have been added for this new behaviour.
If Grafana rotates the user's auth token during a request to the data
source proxy it will set the Set-Cookie header with new auth token in
response before proxying the request to the datasource.
Before this fix the Set-Cookie response header was cleared after the
proxied request was finished to make sure that proxied datasources
cannot affect cookies in users browsers. This had the consequence
of accidentally also clearing the new auth token set in Set-Cookie
header.
With this fix the original Set-Cookie value in response header is now
restored after the proxied datasource request is finished. The existing
logic of clearing Set-Cookie response header from proxied request
have been left intact.
Fixes#16757
* Add file path to metadata and show it in dialogs
* Make path relative to config directory
* Fix tests
* Add test for the relative path
* Refactor to use path relative to provisioner path
* Change return types
* Rename attribute
* Small fixes from review
* Provisioning: Trying to fix test
* Use better sync strategy for test
* Lower the timeout
* Remove commented code
Co-Authored-By: aocenas <mr.ocenas@gmail.com>
* Feature: introduce LdapActiveSyncEnabled setting
We probably remove it after the active sync is done.
But at the moment we do not want to affect the current users
with not fully tested feature
* Chore: move settings in more logical order
* Feature: add cron setting for the ldap settings
* Move ldap configuration read to special function
* Introduce cron setting (no docs for it yet, pending approval)
* Chore: duplicate ldap module as a service
* Feature: implement active sync
This is very early preliminary implementation of active sync.
There is only one thing that's going right for this code - it works.
Aside from that, there is no tests, error handling, docs, transactions,
it's very much duplicative and etc.
But this is the overall direction with architecture I'm going for
* Chore: introduce login service
* Chore: gradually switch to ldap service
* Chore: use new approach for auth_proxy
* Chore: use new approach along with refactoring
* Chore: use new ldap interface for auth_proxy
* Chore: improve auth_proxy and subsequently ldap
* Chore: more of the refactoring bits
* Chore: address comments from code review
* Chore: more refactoring stuff
* Chore: make linter happy
* Chore: add cron dep for grafana enterprise
* Chore: initialize config package var
* Chore: disable gosec for now
* Chore: update dependencies
* Chore: remove unused module
* Chore: address review comments
* Chore: make linter happy
* Feature: add cron setting for the ldap settings
* Move ldap configuration read to special function
* Introduce cron setting (no docs for it yet, pending approval)
* Chore: address code review comments
Adds a new version option 7.0+ (70 internally).
Version 7.0+ doesn't include document types in index mappings
so support for handling this have been added.
Version 7.0+ returns number of hits in a different way so
support for handling this have been added.
Version 7.0+ doesn't support sending
max_concurrent_shard_requests in multisearch header so
support for sending this in query string have been added.
Update elastic6 docker block and dashboards (devenv) to use
6.7.1 images, filebeat index name is now filebeat-YYYY.MM.DD
and dashboard include correct tags and links.
Add elastic7 docker block and provisioning (devenv).
Updates documentation regarding new version.
Closes#15622
* Add api to reaload provisioning
* Refactor and simplify the polling code
* Add test for the provisioning service
* Fix provider initialization and move some code to file reader
* Simplify the code and move initialization
* Remove unused code
* Update comment
* Add comment
* Change error messages
* Add DashboardProvisionerFactory type
* Update imports
* Use new assert lib
* Use mutext for synchronizing the reloading
* Fix typo
Co-Authored-By: aocenas <mr.ocenas@gmail.com>
* Add docs about the new api
* Add value types with custom unmarshalling logic
* Add env support for notifications config
* Use env vars in json data tests for values
* Add some more complexities to value tests
* Update comment with example usage
* Set env directly in the tests, removing patching
* Update documentation
* Add env var to the file reader tests
* Add raw value
* Post merge fixes
* Add comment
* add folderUid to DashbaordsAsConfig structs and DashbardProviderConfigs struct, set these values in mapping func
look for new folderUid values in config_reader tests
set dashboard folder Uid explicitly in file_reader, which has no affect when not given
* formatting and docstrings
* add folderUid to DashbaordsAsConfig structs and DashbardProviderConfigs struct, set these values in mapping func
look for new folderUid values in config_reader tests
set dashboard folder Uid explicitly in file_reader, which has no affect when not given
* formatting and docstrings
* add folderUid option, as well as documentation for the rest of the fields
* add blank folderUid in devenv example.
* add folderUid to provisioning sample yaml
* instead of just warning, return error if unmarshalling dashboard provisioning file fails
* Removing the error handling and adding comment
* Add duplicity check for folder Uids
Co-authored-by: swtch1 <joshua.thornton@protonmail.com>
* Chore: remove session storage references
* Small refactoring of the settings module
* Update docs - remove references for the session storage
* Update config files (sample and default configs)
* Add tests for warning during the config load on defined storage cache
* Remove all references to session storage
* Remove macaron session dependency
* Remove leftovers
* Fix: address review comments
* Fix: remove old deps
* Fix: add skipStaticRootValidation = true to tests
* Fix: improve the docs and warning message
As per discussion in here - https://github.com/grafana/grafana/pull/16445/files#r273026255
* Chore: make linter happy
Fixes#16148
Ref #16114
The iteration order over maps is not specified and is not guaranteed
to be the same from one iteration to the next. If a map entry that
has not yet been reached is removed during iteration, the corresponding
iteration value will not be produced. If a map entry is created during
iteration, that entry may be produced during the iteration or may be skipped.
The choice may vary for each entry created and from one iteration to
the next. If the map is nil, the number of iterations is 0.
https://golang.org/ref/spec#For_range
* Search: Fixes search limits and adds a page parameter
This adds a page parameter to search api without adding
any major breaking change.
It does at an api validation error when trying to use
a limit beyond 5000. This is a breaking change. We could
remove this and have it only in the docs and describe that this
is a limit that grafana will apply silently.
Fixes#16049
* Fix: Corrected wrong array slice change
* Docs: minor docs fix
* Search: fixed folder tests
* Fixed: Moved limit to correct inner query
* Search: moving limit check and page check
* Search: limit in handler is no longer needed
* Chore: refactor auth proxy
Introduced the helper struct for auth_proxy middleware.
Added couple unit-tests, but it seems "integration" tests already cover
most of the code paths.
Although it might be good idea to test every bit of it, hm.
Haven't refactored the extraction of the header logic that much
Fixes#16147
* Fix: make linters happy
- guide shamelessly stolen from prometheus/prometheus
- updates local interface of oauth exchange
- updates local impl of hclogger
- bump jaeger client version
closes#16088
* fix label handling
* SEARCH() results could have multiple namespace, remove from legend
* divide GetMetricStatistics related code
* divide GetMetricData related code
* divide parseGetMetricDataResponse()
* divide parseGetMetricDataQuery()
* divide test code
* add test for GetMetricData
* add test for GetMetricData parse response
* fix bug of terminating gap
* fix gofmt
This is basically implementation of the https://github.com/grafana/grafana/issues/8900#issuecomment-435437167
points, except for the type conversion bit.
I tried to implement idea mentioned in cockroachdb ticket (see below).
And it is possible, but it complicates things as lot - not only we have to
have 4 SQL statements instead of one, but we would have to copy the column
structure as well - PK, FG, indexes and stuff, plus there will
be additional downtime with this approach.
So idea for this pull is to prepare our SQL as much as possible, so when
cockroachdb will add support for full type conversions, we could easilly add
support for it as well.
* Add `CASCADE` to `DROP INDEX` statement
* Make string conversions explicit
Thanks @Luit
Ref #8900
Ref cockroach/cockroach#9851
* Store passwords in secureJsonData
* Revert unnecessary refactors
* Fix for nil jsonSecureData value
* Remove copied encryption code from migration
* Fix wrong field reference
* Remove migration and provisioning changes
* Use password getters in datasource proxy
* Refactor password handling in datasource configs
* Add provisioning warnings
* Update documentation
* Remove migration command, moved to separate PR
* Remove unused code
* Set the upgrade version
* Remove unused code
* Remove double reference
* app pages
* app pages
* workign example
* started alpha support
* remove app stuff
* show warning on alpha/beta panels
* put app back on plugin file
* fix go
* add enum for PluginType and PluginIncludeType
* Refactoring and moving settings to plugins section
fixes#16529
* Unprovision dashboard in case of DisableDeletion = true
* Rename command struct
* Handle removed provision files
* Allow html in confirm-modal
* Do not show confirm button without onConfirm
* Show dialog on deleting provisioned dashboard
* Changed DeleteDashboard to DeleteProvisionedDashboard
* Remove unreachable return
* Add provisioned checks to API
* Remove filter func
* Fix and add tests for deleting dashboards
* Change delete confirm text
* Added and used pkg/errors for error wrapping
Fixes so it's possible to create new notification channel and providing uid.
Fixes better error/result handling when updating a notifcation channel.
Fixes#16372
Ref #16219#16012
Adds new alert settings for configuring timeouts and retries named
evaluation_timeout_seconds, notification_timeout_seconds
and max_attempts.
Closes#16240
See https://github.com/securego/gosec for more info.
Disabled a lot of rules. I guess we should go through them
and recheck if we really need to disable some
Fixes#16204
Since we do not like some of the default golint rules,
this commit proposes to use https://github.com/mgechev/revive.
And potential revive speed-up should't hurt :).
Right now, presented config (./conf/revive.toml) is permissive,
we might improve it over time however. Fixes for found revive
issues in the code are very limited so it wouldn't be large to review.
Also in this commit:
* Add annotations for makefile commands and declare phony targets
* Rename "gometalinter" script and CI command to "lint"
since we are doing there a bit more then using gometalinter package
* Add Makefile rules to .editorconfig
* Documentation which mentioned "golint" replaced with revive
Fixes#16109
Ref #16160
Now returns uid in response to get notification channel by id.
Adds GET/PUT/DELETE support for notification channel by uid,
/api/alert-notifications/uid/:uid.
Break apart alerting and alert notification http api docs in two
pages and update documentation to make it up to date
with current implementation.
Fixes#16012
